Sufficient Context: A New Lens on Retrieval Augmented Generation Systems
Authors: Hailey Joren
Jianyi Zhang
Chun-Sung Ferng
Da-Cheng Juan
Ankur Taly
Cyrus Rashtchian
Venue: ICLR 2025 Poster (2025)
Keywords: retrieved augmented generation
RAG
LLMs
selective generation
sufficient context
factuality
Primary area: foundation or frontier models, including LLMs
TL;DR: We consider RAG performance through the lens of sufficient context and explore methods for improving LLM factuality

Abstract: Augmenting LLMs with context leads to improved performance across many applications. Despite much research on Retrieval Augmented Generation (RAG) systems, an open question is whether errors arise because LLMs fail to utilize the context from retrieval or the context itself is insufficient to answer the query. To shed light on this, we develop a new notion of sufficient context, along with a method to classify instances that have enough information to answer the query. We then use sufficient context to analyze several models and datasets. By stratifying errors based on context sufficiency, we find that larger models with higher baseline performance (Gemini 1.5 Pro, GPT 4o, Claude 3.5) excel at answering queries when the context is sufficient, but often output incorrect answers instead of abstaining when the context is not. On the other hand, smaller models with lower baseline performance (Llama 3.1, Mistral 3, Gemma 2) hallucinate or abstain often, even with sufficient context. We further categorize cases when the context is useful, and improves accuracy, even though it does not fully answer the query and the model errs without the context. Building on our findings, we explore ways to reduce hallucinations in RAG systems, including a new selective generation method that leverages sufficient context information for guided abstention. Our method improves the fraction of correct answers among times where the model responds by 2--10% for Gemini, GPT, and Gemma. Code for our selective generation method and the prompts used in our autorater analysis are available on our [github](https://github.com/hljoren/sufficientcontext).

Published as a conference paper at ICLR 2025
SUFFICIENT CONTEXT: A NEW LENS ON RETRIEVAL AUGMENTED GENERATION SYSTEMS
Hailey Joren"
Jianyi Zhang
Chun-Sung Ferng
UC San Diego
Duke University
Google
hjoren@ucsd.cm edu
jianyi. zhangeduke. edu
csferng@google..o
Da-Cheng Juan Google dacheng@googlee.com com
Ankur Taly
Cyrus Rashtchian Google cyroid@google. com
Google
ataly8google.com
ABSTRACT
Augmenting LLMs with context leads to improved performance across many applications. Despite much research on Retrieval Augmented Generation (RAG) systems, an open question is whether errors arise because LLMs fail to utilize the context from retrieval or the context itself is insufficient to answer the query. To shed light on this, we develop a new notion of sufficient context, along with a method classify instances that have enough information answer the query. We then use sufficient context to analyze several models and datasets. By stratifying errors based on context sufficiency, we find that larger models with higher baseline performance (Gemini 1.5 Pro, GPT 4o, Claude 3.5) excel at answering queries when the context sufficient, but often output incorrect answers instead of abstaining when the context is not. On the other hand, smaller models with lower baseline performance (Llama 3.1. Mistral 3, Gemma 2) hallucinate or abstain often, even with sufficient context. We further categorize cases when the context is useful, and improves accuracy, even though it does not fully answer the query and the model errs without the context. Building on our findings, we explore ways reduce hallucinations in RAG systems, including new selective generation method that leverages sufficient context information for guided abstention. Our method improves the fraction of correct answers among times where the model responds by 2-10% for Gemini, GPT, and Gemma. Code for our selective generation method and the prompts used in our autorater analysis are available on our github.
INTRODUCTION
Providing Large Language Models (LLMs) with additional context, such in Retrieval Augmented Generation (RAG) systems, has led to major improvements in LLM factuality and verifiability when adapting to new domains (Lewis al., 2020). In the case of retrieval model provides context at inference time in the form of snippets or long-form text (Zhu et al., 2021). Then, the model synthesizes the query along with this added context to generate the answer. Unfortunately, current RAG-based LLMs exhibit many undesirable traits, such as confidently predicting the incorrect answer with retrieved evidence (Mishra et al., 2024; Niu et al., 2024; Ru et al., 2024), being distracted by unrelated information (Cuconasu et al., 2024; Yoran al., 2024), and failing to properly extract answers from long text snippets (Hsieh et al., 2024; Liu et al., 2024).
question answering,
The ideal outcome is for the LLM to output the correct answer if the provided context contains enough information to answer the question when combined with the model's parametric knowledge. Otherwise, the model should abstain from answering and/or ask for more information. One core challenge in achieving this ideal outcome is building models that can use the provided context only when it helps answer the question correctly. Several works have investigated this issue by evaluating
Work done during an internship at Google. Work done during intersship Gongle.

---

Published as a conference paper at ICLR 2025
models in the presence of irrelevant information in the context (discussed in Section 2). However, "relevant information" can range from directly containing the answer to simply being topically related to the question. Even "golden" oracle documents in datasets vary in how much information they provide about the query, and whether they directly inform the ground truth answer or not. In other words, while the goal seems to be understand how LLMs behave when they do do not have sufficient information to answer the query, prior work fails to address this head-on.
As our first contribution, we put forth a new notion of sufficient context. We divide instances into two categories based on whether the context provides enough information to construct an answer to the query. The sufficient context designation is a function of an input pair consisting of one question and the associated context. Crucially, it does not require a ground truth answer. Figure I shows examples and breakdown of model responses after splitting the data hased on sufficient vs. insufficient context. To divide the dataset, we use an LLM-based autorater to classify instances as sufficient or not. Here, an autorater is model that evaluates instances based on a property, e.g., a sufficient context autorater.
Using our sufficient context autorater, we uncover new insights into LLM behavior and into existing benchmark datasets. First, we find models generate incorrect answers on a non-trivial fraction of instances that have sufficient context to answer the query. In other words, open-book QA cannot be solved by improving retrieval alone. Second, when given instances without sufficient context, models tend to hallucinate more than they abstain, especially for multi-hop questions. This finding complements prior work, which shows that LLMs are not robust to noisy retrieval (Yoran et al., 2024; Wu et al., 2024). Third, models generate correct answers in many cases, even when the provided context is insufficient. Surprisingly, this remains true after we filter out questions that the model answers correctly in a closed book (w/o RAG) setting. Together, our analysis deepens our understanding of RAG systems by revealing nuances in how models generate responses with retrieval.
As a final contribution, we explore ways to use sufficient context labels to reduce model hallucinations. We implement new selective generation framework that improves accuracy. We use a smaller, intervention model to determine when the model generates or abstains, providing a controllable trade-off. Moreover, we can combine our method with any LLM, including proprietary models like Gemini and GPT. Our main result is that using sufficient context as an additional signal leads to much higher accuracy over the fraction of answered queries, for most coverage levels and across multiple models/datasets, We also find that fine-tuning open-source models with sufficient context information does not easily reduce the hallucination rate. Instead, for LLama 3.1 and Mistral 3, fine-tuning can lead to higher abstention rate at the cost of fewer correct answers. Code for our selective generation method and the prompts used in our autorater analysis are available our github.
To summarize, our main contributions are
We define the notion of sufficient context, unifying existing work on relevance for RAG systems. Then, we design a sufficient context autorater (achieving 93% accuracy), enabling us to label instances scalably and to analyze model responses with or without sufficient context.
Our analysis leads to several new findings about eetrieval-augmente takeaway is that SOTA LLMs output correct responses 35-62% of the time with insufficient context. Hence, intervention strategies to increase accuracy should not solely rely on sufficiency.
model performance. One
Building on our findings above, we develop an efficient and general method for selective generation, using both confidence and sufficient context signals. Our method improves the fraction of correct answers (among total model responses) by up to 2-10% for Gemini, GPT, and Gemma.
RELATED WORK
Many papers have shown that reaping the benefits of RAG (e.g., better factuality) will require deeper understanding of how LLMs respond to variations in the queries and provided context (Asai et al., 2024; Fan et al., 2024; Ram et al., 2023; Rau et al., 2024). We review two main areas. First, much work has evaluated RAG systems with poor retrieval, uncovering cases where LLMs are led astray by irrelevant context. Another line of study aims to reduce LLM hallucinations in RAG settings.
rervaelevant Context. Prior studies uncover lack of robustness to imperfect retrieval. However, these studies vary in terms of how they evaluate retrieval quality, without anchoring to a precise "relevance" definition. Shi et al. (2023a) adds sentences to math questions (based GSM8K) which
2

---

Published as a conference paper at ICLR 2025
Examples: sufficient context
Categorizing Model Responses (Musique Dataset)
Abstein
He lucinniete
Correct
Question: Who is Lys married to?
Lya L married Paul in 2020... They looked happy together recent event.
Context A
25. 8%
6%
39.7%
Gemini 1.5 Pro
Sufficient Context
03.4%
6%
49.5%
Context
Sufficient
Context
Spouse: Paul (m. 2020)
Lya Wikipedia
Born: October 1980
54.2%
28.4%
17
71.2%
19.8% 19.8%
..00%
83.4%
14.8%
1.25
GPT 4o
Contect
Lys L married Tom 2006..
Insufficient Context
They divorced in 2014... Lya went dates
2018...
23.8%
57.4%
Gemma 27B
Contect
62.2%
16.05
23.
66.1%
20.1%
Lya L is an astronaut, born in Ohio.... Lya has two children... Lye's parents lawyers..
Insufficient Context
Without RAG
① With RAG
With RAG Suft: Context
@itt RAG Insuff. Context 55.4%% of Dataset
44.6% of Dataset
Figure I: New insights into RAG systems by looking at whether instances have sufficient context. On the left, show examples of sufficient context; the right, breakdown of model responses on the Musique dataset. Adding RAG improves the percentage of correct answers. Unfortunately, with RAG, models hallucinate more than abstain, and the insufficiency of the context does not account for this major issue. Also, standard datasets have many instances with insufficient context (here, 55.4%). We include results for other datasets (FreshQA, HotPotQA) in Appendix B, showing similar trends.
should not impact the answer at all (and GSM8K is designed to have sufficient context by definition). Xie et (2024) looks having counter counter-memory memory context, by either replacing the entity name with an erroneous one or using an LLM to generate a synthetic context supporting the erroneous entity. Ret-Robust (Yoran al., 2024) trains model to be robust to irrelevant context, with an Nl.e-based entailment to determine relevance, and only uses the relevance scores to influence the training mixture of relevant vs. irrelevant documents. Wu et al. (2024) looks at questions where the LLM gets the answer correct without retrieval and is non-robust to changes in the retrieval. Multiple methods use a model to predict relevance scores (as part of a larger pipeline), without calibration to a formal definition (Wang et al., 2024a; Zhou et al., 2024), including for iterative retrieval (Jiang et al., 2024; Yan et al., 2024). In terms of analysis studies, Cuconasu et al. (2024) distinguishes golden and relevant documents, but simply uses "does not contain the answer" proxy for irrelevant context.
Reducing Hallucinations. There have also been efforts to improve RAG factuality on open-book QA tasks (Asai al., 2023; Mineiro, 2024; Simhi al., 2024; Wang al., 2024b: Zhang et al., 2024b). The main theme is to improve both the generation and retrieval quality, often by fine-tuning one or more components. Also, since RAG leads to very long contexts, another issue that arises is the "lost in the middle" problem (Hsieh et al., 2024; Liu et al., 2024; Yu et al., 2024). These works start with the premise that the provided query/context should be precisely answerable by the LLM, and hence, only analyze their findings in the sufficient context scenario. Independent of RAG, many papers have studied interventions and tools for calibrating LLM confidence in their responses (Chuang et al.. 2024; Kadavath et al., 2022; Yin et al., 2023; Zhang et al., 2024a) and performance across disaggregated subsets data (Paes et al., 2022: Joren al., 2023).
3 SUFFICIENT CONTEXT
At a high level, our aim is to classify input instances based on whether the context contains enough information to answer the query. We split possible contexts into two cases: (1) Sufficient Context. The context is sufficient to answer the query if it contains all the necessary information to provide definitive answer. (2) Insufficient Context. Otherwise, a context is insufficient. A context may also be insufficient if the query requires specialized knowledge that is not provided in the context or if the information in the context is incomplete, inconclusive, or contradictory. In this section, we more thoroughly discuss sufficient context. Then, show how to accurately and scalably label instances.
3

---

Published as a conference paper at ICLR 2025
3.1 DEFINITION OF SUFFICIENT CONTEXT
We first set some notation for a generic open-domain question-answering setting following Trivedi al. (2020). Consider dataset D with instances of the form q = (Q, C; A), where Q is the query and C is the context that consists of a set of facts. At inference time, we also consider instances = (Q, C)) without the ground truth answer, where the goal is to predict an answer A' from Q, C, and the model's parametric knowledge. To measure correctness, we compare A' and A, where there are many options such as exact match, F1 score, or an LLM-based assessment of answer sameness (we use an LLM). Using this notation, we can now define our notion of sufficient context.
Definition (Sufficient Context), An instance q' (Q, ())CC) has sufficient context if and only if there exists an answer A' such that A' is plausible answer to the question Q given the information in C.
To understand this, we can build on the Attributable to Identified Sources (AIS) framework (Rashkin et al., 2023). Entailment via AIS answers slightly different question. Namely, given an instance q = (Q, C; A). the entailment objective is to determine the truth value of the proposition: The answer to the question Q is A given the information in C. The key difference between entailment and sufficient context is that for sufficient context we do not presuppose that we have the answer A' in advance, only that such an answer exists. Finally, we only consider "plausible" answers, where we mean that A' could be an answer to the question Q. For example, the question asks about a person's 's birthplace, then A' should be location. We note that this allows for the possibility that the context contains an incorrect answer to the question. This is a key requirement, because (i) we would like be able to use signal from sufficient context at inference time, where we do not have ground truth answers (see Section 5.1) and (ii) we hope to elucidate findings that are robust to ground truth label noise.
Remark 1 (Multi-hop queries). In most benchmark dataset (e.g., Musique, HotPotQA), models are expected to be able to do multi-hop reasoning up to four hops, in which they must combining facts to form the answer. However, they should not infer connections that are not in the context. For example, "Bob's mother was born in New York" then this does not suffice to say Bob was born in New York. But, if the context also says "Bob's 's mother is Alice..." and all of Alice's children were born in New York" then this instance has sufficient context.
Remark 2 (Ambiguous queries). If the query is ambiguous, then the context is sufficient if and only if (i) the context can disambiguate the query and (ii) the context provides an answer to the disambiguated query. For example, the question could be "What sport does Mia play?" and the
context could contain both "Mia, from New York, plays basketball... and "... Mia, from California, plays volleyball." This is sufficient because if the query is referring to either Mia from New York or Bob from California, then the context can answer the question.
Remark 3 (Ambiguous contexts). Assume the context contains multiple plausible answers the query. Then it is sufficient if and only if it also provides enough information to distinguish between queries that would lead each answer. For example, if the question is "What country does Ali live in?" and the context is "Ali lives in Paris" then this instance does not have sufficient context because it is not clear if Ali lives in Paris, France or Paris, Texas, USA. If the context further contains "This weekend, Ali took the train from Paris to Marseille." Then this becomes sufficient because it is almost certain that Ali lives France as one cannot take train from Texas to France.
3.2 SUFFICIENT CONTEXT AUTORATER
Next, we consider automating the task of labeling whether instances have sufficient context or not. We investigate two questions: (1) Can today's 's models achieve high accuracy on a challenging. human-annotated dataset? (2) How does an entailment model compare to general-purpose L.MSs? To answer these questions, we evaluate methods on human-labeled data. Table shows that Gemini 1.5 Pro can serve an accurate autorater to label instances in terms of sufficient context. It achieves 93% accuracy, outperforms other methods, and operates without needing a ground truth answer.
Sufficient Context Labeled Dataset. Using the above definition, we construct gold labels for each (query, context) ) pair. We did not use ground truth answers or model responses. For the instances, sample total of 115 instances (queries, contexts, and answers) from standard benchmarks (PopQA, FreshQA, Natural Questions, EntityQustiios).. We design the dataset to be very challenging, including single- and multi-hop questions, well adding highly related
4

---

Published as a conference paper at ICLR 2025
Table Sufficient Context AutoRater. Evaluating model ability to classify sufficient context on gold-labeled dataset of 115 (query, context; answer) instances. Gemini 1.5 Pro (1-shot) performs the best, while FLAMe can be a cheaper alternative. TRUE-NLI and Contains GT need ground truth (GT) answers, while others only use (query, context). Best in column in bold.
Metrics:
Fl Score
Accuracy
Precision
Recall
No GT Answer
Methods
Gemini 1.5 Pro (I-shot)
0.935
0.930
0.935
0.935
Gemini 1.5 Pro (0-shot)
0.878
0.870
0.885
0.871
FLAMe (fine-tune PaLM 24B)
0.892
0.878
0.853
0.935
TRUE-NLI (fine-tune T5 I1B)
0.818
0.826
0.938
0.726
Contains GT
0.810
0.809
0.870
0.758
information in the context even if it is not sufficient (e.g., a named entity from the question often appears in the context). We evaluate methods' abilities to classify sufficient context (binary labels).
Methods: Operating on Query-Context pairs. We use Gemini 1.5 Pro with either instructions (0-shot) or both instructions and 1-shot example, held out from our dataset. FLAMe 24B is a general autorater model (Vu et al., 2024), but it has small context window. For FLAMe, we divide the contexts into 1600 token chunks and ask whether each chunk sufficient. If any chunk labeled sufficient, we consider the instance to have sufficient context; otherwise, it's labeled insufficient. We design prompts (in Appendix C) for both models based on the sufficient context definition above.
Methods: When Ground Truth (GT) Answer is Available. For two basclines (TRUE-NLI, Contains GT), we use answers as an additional input to classify sufficient context. TRUE-NLI is a fine-tuned entailment model (Honovich et al., 2022) that checks if the context entails one of the GT answers. Contains GT checks if GT answer appears in the context. Comparing to entailment is particularly interesting because if a given answer A is entailed by (Q, C), then the context is also sufficient. On the other hand, the reverse is not true, since the answer A is only one possible choice for A'. As one consequence, if the ground truth answer A is incorrect, then it may not be entailed by the context. This happens when named entity is ambiguous (c.g., two people with the same name), and the GT answer is based on one of the people while the context describes the other.
Results. Table shows that Gemini 1.5 Pro (1-shot) performs the best overall in terms of Fl score and accuracy. As expected, TRUE-NLI has higher precision and lower recall: measures entailment, which implies sufficient context. FLAMe outperforms TRUE-NLI in F1 and accuracy, but lags behind Gemini (I-shot), likely because it is smaller model. The Contains GT method works surprisingly well, indicating that the presence of a ground truth answer correlates with context sufficiency.
Discussion. weal-world scenarios, we cannot expect candidate answers when evaluating model performance. Hence, it is desirable to use method that works using only the query and context. Among these methods, Gemini 1.5 Pro (I-shot) has high accuracy and balanced precision and recall. Therefore, we use it in Section Later, in Section 5.1, we use FLAMe as a computationally efficient autorater to provide a signal for selective generation. Our comparison with TRUE-NLI and Contains GT confirms that classifying sufficient context a different (and more complex) task than determining entailment.
our main method for analyzing datasets and model responses.
4 A NEW LENS ON RAG PERFORMANCE
We set out to understand RAG performance by looking at sufficient context. We first analyze datasets (Section 4. 4), then we investigate model performance with/without sufficient context (Section 4.2). We qualitatively discuss cases where insufficient context leads correct model responses (Section 4.3).
4.1 Do BENCHMARK DATASETS HAVE HIGH SUFFICIENT CONTEXT?
We introduce the datasets that for analysis. Then, we investigate the percentage of instances in these datasets that have sufficient context (according to our autorater). For our study, we do not aim to optimize the retrieval methods (which could increase the sufficient context percentage).
5

---

Published as a conference paper at ICLR 2025
% of Dataset with Sufticient Context.
Measuring Sufficiency by Context Length
1.00
10000 Tikens
80
17.4
2000 Tokans
6000
63.7
60
45.4
44.6 44.6
33,4
FreshQA
HotpotQA
Musique
Figure We compare the of instances that our autorater labels as sufficient across datasets, either with the first 10k, 6k, or 2k tokens of the provided sources. FreshQA has hand-curated URLs that support the answers and exhibits high sufficient context. HotPotQA and Musique have lower sufficient context (and even lower with 2000 tokens). We use 6000 token contexts in the remainder.
This is not the focus of our work, as we wish to understand how models perform with or without sufficient context. Having mix of both is inevitable in generic RAG systems.
Datasets. We consider FreshQA. Musique-Ans, and HotpotQA as representative spread of open book QA datasets. FreshQA (Vu et al., 2023) evaluates time-sensitive information and has up-to-date URLs that should support an answer to the queries, which we use to construct the context (see Appendix A.3 for details on the retrieval). We use the "True Premise' setting (452 instances), skipping 'False Premise' questions that mislead by design. Musique-Ans (Trivedi et al., 2022) is a multi-hop QA benchmark, created by composing two to four single-hop interconnected questions. Here, 'Ans' the standard 'answerable' subset. Musique instances have 20 supporting text snippets as sources, which we use as the context. HotPotQA (Yang et al., 2018) is a Wikipedia- single- and multi-hop questions. The corpus is large set of snippets; we retrieve the top 5 the context (via REPLUG (Shi et al., 2023b) from AI (Jin et al., 2024)). We randomly sample 500 instances from the development sets of Musique-Ans and HotPorQA for evaluation.
QA dataset, with
% of Datasets. Figure shows the fraction of instances that our autorater classifies having sufficient context. We explore three context lengths, ranging from maximum of 2000 to maximum of 10000 tokens. The motivation behind this is to assess if there is a large change in sufficient context if we were to simply truncate the retrieval (e.g., for models that have small context windows). In general, we see a modest difference from 2000 to 6000 token limit, but effectively none from 6000 10000 tokens. FreshQA has the highest sufficient context percentage, which makes sense as the context comes from oracle supporting documents. The lower sufficient context in Musique is perhaps surprising, given that the retrieval is fixed as part of the dataset. From the results in Figure 2, we truncate at 6000 tokens for all three datasets in the remainder of the paper.
Sufficient Context
4.2 INITIAL FINDINGS BASED SUFFICIENT CONTEXT
In general, the ideal behavior for language generation model answer questions correctly when possible and to otherwise abstain. RAG seeks to move models towards this desired behavior, such that the provided context shifts hallucinations to correct answers, or to abstentions if needed. We analyze several cases to assess how far we are from this ideal trade-off.
Experimental Set-up and LLMEval. We employed basic chain of thought (CoT) prompting approach, with the prompt structure and further information detailed in Appendix C.4. We then processed the outputted answers to identify matches between the response and any of the ground truth answers. Responses where a clear correct match could not be determined were processed through the LLMEval pipeline using a zero-shot approach, with the prompt based on Krishna et al. (2024) (see Appendix C.3). Then, for each example, depending on the LLMEval output. We use an LLM for evaluation instead of checking for an exact match because it is more robust to syntactic variations. See Appendix B.3 for details and examples.
rate it as "correct" or "abstain" or "hallucinate"
Models Abstain Less with RAG. While overall performance improves with RAG, the introduction of additional context paradoxically reduces the model's ability abstain from answering when

---

Published as a conference paper at ICLR 2025
Categorizing RAG Responses: Sufficient vs Insufficient Context
FreshQA
Musique
HotpotOA
Gemini 1.5 Pro
GPT 4o
Claude 3.5 Sonnet
Gemma 27B
100%
Sufficient Context
00%
60%
40%
20%
Correct
Abstain
Halluc.
Correct
Abstain
Halluc
Correct
Abstain
Halluc.
Correct
Abstain
c.lluc.
Gemini 1.5 Pro
GPT 4o
Claude 3.5 Sonnet
Gemma 27B
Insufficient Context
100%
80%
60%
40%
20%
Correct
Abstain
Hallur,
Correct
Abstain
Halluc
Correct
Abstain
Hallur.
Correct
Abstain
Hal luc.
Figure 3: Model Performance on Datasets Stratified by Sufficient Context. Given sufficient context, models have a higher correct percentage on these challenging datasets, Performance drops, but the models are still able to answer large portion of questions correct without sufficient context. One prevailing issue is that all models hallucinate rather than abstain in many cases with insufficient context. The smallest model Gemma 27B struggles avoid hallucinations given insufficient context.
appropriate. Without RAG, Claude 3.5 Sonnet abstains on 84.1 % questions, while with RAG, the fraction of abstentions drops to 52%. Similarly, GPT 4o's abstention fraction moves from 34.4% to 31.2% and Gemini 1.5 Pro's drops from 100% to 18.6%. This phenomenon may arise from the model's increased confidence in the presence of any contextual information, leading to a higher propensity for hallucination rather than abstention.
Models Hallucinate with Both Sufficient and Insufficient Context. Considering Figure 3, models generally achieve higher accuracy with sufficient context (higher green bars, top row) than without sufficient context (lower green bars, bottom row). However, looking at each row separately, we discover several findings. First, in the sufficient context case (top row), more than they abstain (red bars are higher than blue bars, usually). The trend holds across all three datasets. Moving insufficient context (bottom row), find different distribution of model responses, with more abstentions and hallucinations. This tendency varies notably across different models. For instance, Claude abstains more (higher blue bars) with insufficient context, but answers fewer questions correctly (lower green bars) than Gemini and GPT. These differences underscore the potential for improvement both retrieval and reasoning capabilities. Overall, Gemma has much more hallucinations (higher red bars) than the other models, except for HotPotQA, where we attribute the higher accuracy to the smaller retrieved contexts.
that models hallueinate
4.3 QUALITATIVELY ANALYZING RESPONSES WITH INSUFFICIENT CONTEXT
One curious observation in analysis is the ability of models to sometimes provide correct answers even when presented with insufficient context. For example, from Figure 3, all three models are able to correctly answer upwards of 35% of instances with insufficient context HotpotQA. A natural assumption is that the models already know the answer from pre-training, and they can generate a correct response from parametric memory. However, this only explains part of the story.
Looking deeper, we provide a qualitative categorization in Table 2 of instance types where our autorater labels instance as insufficient context, while the LIM evaluator marks the model answer

---

Published as a conference paper at ICLR 2025
Instance type
Why model may be correct Example
Yes/No question
50% chance of correct
Q: Is there a total eclipse in the United States this year?
Limited choice
Some chance of correct
Q: Which band has more members, Chvrches or
Goodlye Mr. Mackenzie?
Q:
original
for character
whose series Mickey's Safari in Letterland is from?
Multi-hop: fragment
Use parametric inference
Context says Mickey's 's Safari is a video game and Walt Disney voices Mickey Mouse
Must infer the game in the Mickey Mouse series.
Q: Claudine's 's Return starred the actress who played which role on "Marriod... with Childreaa"?
Multi-hop: partial
Use parametric knowledge
Context lists actresses but not their rolex in "Married...vit Children Must know extna facts.
Q: How many cyclists have won all three of women's s cycling Grand Tours equivalents the same year?
Too many hops
Execute complex reasoning
Context requires cross-refrrnncing lists events
and lists of winnrrs while tracking winners by year.
Q: Who is the spouse of a cast member from King of the Mountain?
Ambiguous query
Guess right interpretation
Context has many members and quersicontent do not specify which spoure to answer about.
Rater error
Mislabel insuff. or correct
Closed-book correct
Known from pre-training
Table 2:
Qualitative Analysis of Correct Answer & Insufficient Context. Examining model responses across datasets, we identify common cases where the model generates a correct answer even though our autorater labels the instance insufficient. We categorize such instances into eight types, as well as provide examples. Given that models are also correct on many questions in the closed-book setting, believe this mostly explains the 35-629 correct rate with insufficient context.
correct. For example, type accounts for when the provided context is not sufficient to answer the query, but it bridges gaps in the model's s knowledge. Another type is when the retrieved information clarifies ambiguities inherent in the question (without answering the question). Finally, we also have the times where either the autorater or the evaluator model makes an error. We note that our analysis expands on prior work by Yoran et al. (2024), who also find large fraction of cases where the model is correct with RAG (but not without) even though the context does not contain the answer.
We additionally investigated cases where the autorater labels an instance as having sufficient context while the LLM evaluator marks the answer as incorrect. One source of these discrepancies occurs when the ground truth answer conflicts with the answer provided in the souree. This represents key difference from methods that measure entailment, where context is evaluated relative to a specific ground truth answer (see Table 1). Another source of errors arises when the autorater correctly identifies that the necessary information is present, but the model fails to properly compose the information (e.g., multihop questions or questions requiring arithmetic). In substantial number of cases, however, determining the source of the error proves challenging.
TECHNIQUES TO REDUCE HALLUCINATIONS WITH RAG
From our previous analysis, we have seen that models may hallucinate rather than abstain and that this happens more with RAG than in closed-book setting. A natural next question is whether can prompt or fine-tune a model to perform closer to the ideal case. Can we steer the model to either output the correct answer or abstain, while hallucinating an incorrect answer as little as possible?
5.1 SELECTIVE RAG USING SUFFICIENT CONTEXT SIGNAL
One simple solution to improving RAG performance would be to use the sufficient context autorater to abstain given insufficient context. However, this heave- approach can lower overall performance, since all models answer some questions correctly even with insufficient context, as described in Table 2 and demonstrated in Figure Instead, propose method for combining
8

---

Published as a conference paper at ICLR 2025
the sufficient context autorater outputs with model self-rated confidence scores to tune selective trade-off, where "coverage" denotes the portion of inputs on which the model does not abstain. Specifically, use these signals to train simple linear model to predict hallucinations, and then use it to set coverage-accuracy trade-off thresholds.
This mechanism differs from other strategies for improving abstention in two key ways. First, because operates independently from generation, it mitigates unintended downstream effects, whereas strategies like fine-tuning to improve abstention can inadvertently worsen performance on certain inputs (see Section 5.2). Second, it offers a controllable mechanism for tuning abstention, which allows for different operating settings in differing applications, such as strict accuracy compliance in medical domains or maximal coverage on creative generation tasks.
Abstention Signals We utilize two main signals for abstention: the self-rated probabilities as in Li et al. (2024); Kadavath et al. (2022) and the sufficient context autorater. For the self-rated probabilities, we use two strategies: P(True) and P(Correct). P(True) requires sampling answers from the model multiple times, and then prompting the model multiple times to label each model as correct or incorrect, resulting in final probability of correctness associated with each question as in Kadavath al. (2022). For proprietary models, where extensive querying is prohibitively expensive, we use P(Correct) instead. We adapt the probability-generall the model's s response and its estimated probability of correctness. For the sufficient context signal, we use the binary label from an autorater. Our hypothesis is that combining these signals should lead to more effective abstention, particularly in cases where the context is insufficient.
prompt from Li et al. (2024) obtain
Methods. We calculate P(True) by sampling 20 responses for each question and querying the model 5 times to evaluate whether the answer is correct or incorrect (without using the ground truth) as in Kadavath et al. (2022). For P(Correct), the prompt requests the most likely and second most likely answers along with their probabilities. We use string matching to extract the response and self-predicted probability, keeping the one with the highest probability. To determine sufficient context, we use FLAMe, small and efficient model for determining the sufficient context label. We divide the retrievals into chunks of 1600 tokens fit in the context window and label the context as sufficient if any of these chunks are sufficient.
We combine the binary sufficient context label with the self-rated answer probability (P(True) for open-source models or P(Correct) for proprietary models) in a simple logistic regression model to predict hallucinations with 100 iterations of random hyperparameter search. At inference time, we use the logistic regression model scores to threshold the outputs, abstaining when the score is below a chosen threshold as in Joren et al. (2024). We measure the added value for selective accuracy of the sufficient context signal (purple line in Figure 4) by comparing it with the model self-rated confidence alone (gray line).
Results. We find that our approach leads to better selective accuracy coverage trade-off compared to using model confidence alone. In particular, see gains of over 10% for Gemma 27B on HotpotQA in the highest accuracy regions, and gains of over 5% for Gemini 1.5 Pro the same dataset near the 70% coverage region. These gains are less pronounced on datasets with lower overall accuracy, such as when using Gemma 27B on Musique. In this scenario, the low overall performance (18.4%) likely means that most of the predictive gains are seen by using the self-rated confidence to predict errors for the majority of samples. As a result, there is no added benefit from the sufficient context signal.
Discussion. As expected, we see downward trend in which higher coverage leads to lower selective accuracy for both methods. We conclude that the selective generation mechanism with sufficient context has an added benefit for accuracy-covergs alone. As a prerequisite for our method, models should have a non-trivial accuracy on sufficient and insufficient context instances. Then, intuitively, we can prioritize answering questions the model is likely to get right before those the model struggles with. While ordering examples is impossible in real settings, we can estimate coverage level and use threshold to choose when to answer.
trade-offs compared to self-rated confidence
5.2 FINE-TUNING
We also consider fine-tuning models to increase their ability to abstain instead of outputting an incorrect answer. To do so, we train the models with some examples that contain "I don't 't know" instead of their original ground truth answer. The intuition here is that training explicitly on such
9

---

Published as a conference paper at ICLR 2025
Gemini 1.5 Pro- HotpotQA
GPT 4o HotpotQA
Gemma 278- HotpotQA
100
Selectvv Accuracy (%)
P[Correct)
P)Comeet
P(Tue)
P[Camret) Suff. Context
C)Cmeett Suff. Conteat
PfTue)) Suft. Centeat
90
90
90
00
80
80-
70-
70
70-
60
20
40
60
40
60
100
60
80
100
Caverage (56)
Coverage (%)
Caverage (56)
Gemini 1.5 Pro Musique
GPT 4o Musique
Gemma 27B Musique
100
100
100
Selective Accuracy (%6
P(Correct)
P[Correc) Context
P(Cemeect
P[Comect Suff. Context
PiTrue)
P/True) Suft. Context
90
90
80
80
60-
40-
70-
70
20-
40
%
6D 80
Coverage (56)
Coverage (96
Coverage (56
Figure 4: Selective Generation: Coverage vs. Selective Accuracy. For selective generation, we use linear combination of sufficient context and self-rated confidence (purple) or confidence alone (gray). ). The x-axis shows coverage (% of questions answered); the y-axis shows accuracy at each coverage (# correct answered). The combined approach matches or outperforms the baseline confidence-only method, especially on HotpotQA, where our method improves accuracy for most coverages. For Gemma 27B on Musique, the methods are identical (coeff. for stuff. context is 0).
inputs could encourage the model to abstain instead of hallucinating. We also consider multiple models (Llama 3.1 8B and Mixtral 3 7B) and multiple settings, such as only changing the answers when the example has insufficient context. We present full details in Appendix B.1. The main takeaways are that fine-tuned models (i) have a higher rate of correct answers many cases, but (ii) still hallucinate quite often and more than they abstain. Overall, it is likely possible to use fine-tuning to steer the model towards better abstention and correctness, but more work is needed to determine develop a reliable strategy that can balance these objectives.
CONCLUSION
Our work provided a new lens on LLM responses in RAG systems centered around our notion of sufficient context. We constructed sufficient context autorater, which enabled scalable insights into model performance on different types of instances. Our analysis revealed that even with sufficient context, L.MSs frequently hallucinate answers. We also found, surprisingly, many cases where a model will output correct answer with access to only insufficient context. Qualitatively, we categorized such instances. leading to a fuller picture of ways context can be useful. Finally, we demonstrated a general-purpott selective generation method, which applies to Gemini, GPT, and Gemma, and can reduce hallucinations by 2-10% on queries that the model answers.
Limitations. Our analysis focuses on QA datasets, but summarization tasks also utilize context, which may or may not be sufficient. For example, models may behave differently on the prompt "Summarize the reviews of 5-star hotels in Mallorca" depending on whether the context mentions the hotel reviews, whether they are for 5-star hotels, etc. Another shortcoming is an exploration of how often difffrrnnt retrieval methods lead to sufficient context. Also to achieve the best performance, could have used our autorater to iteratively judge whether to retrieve more or answer the question.
Future Work. One direction is a fine-grained sufficient context autorater, which outputs a score instead of a binary label. This could be useful for ranking contexts after the retrieval step. Another direction is to extend the definition of sufficient context to multi- multi- RAG settings, such for visual QA (images) or document QA (pdf files). Finally, our selective generation results suggest that there is room for improvement in reducing hallucinations by using auxiliary signals from the inputs.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We thank Hrishikesh Garud, Vikram Gopali, Xun Sun, and Bruce Wang for annotating data. We thank Ranjay Krishna and Jacob Eisenstein for helpful discussions. We also thank Alyshia Olsen for help with the figure design and color palette. We thank the anonymous reviewers for suggestions to improve the presentation.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint WWXiv:2303.08774 2023.
Anthropic. Claude 3.5 sonnet model card addendum, 2024.
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannanch Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511 2023.
Akari Asai, Zexuan Zhong, Danqi Chen, Pang Wei Koh, Luke Zettlemoyer, Hannaneh Hajishirzi, and Wen-tau Yih. Reliable, adaptable, and attributable language models with retrieval. arXiv preprint arXiv:2403.03187 2024.
Yung-Sung Chuang, Linlu Qiu, Cheng- Yu Hsieh, Ranjay Krishna, Yoon Kim, and James Glass. Lookback lens: Detecting and mitigating contextual ballucinations in large language models using only attention maps. arXiv preprint arXiv:2407. 07071, 2024.
Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fahrizio Silvestri. The power noise: Redefining retrieval for rag systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 719-729. 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.22 21783, 2024.
Wenqi Fan, Yujuan Ding, Liangbo Ning. Shijie Wang, Hengyun Li. Dawei Yin. Tat-Seng Chua, and Qing Li. survey on rag meeting llms: Towards satrivallauggenn large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 6491-6501, 2024.
Gemini Team, Rohan Sebastian Borgeaud, Yonghui Wu, Jean-Bapten Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. 1805, 2023.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma Improving open language models at practical size. arXiv preprini 2000::2408.001 18, 2024.
Or Honovich, Roce Aharoni, Jonathan Herzig, Hagai Taitelbaum. Doron Kukliansy, Vered Cohen, Thomas Scialom. Idan Szpektor, Avinatan Hassidim, and Yossi Matias. True: Re-evaluating factual consistency evaluation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3905-3920, 2022.
Cheng- Yu Hsich, lung-Sung Chuang, Chun-Liang Li, Zifeng Wang. Long Lc, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Yu Lee, Ranjay Krishna, and Tomas Pfister. Found in the middle: Calibrating positional attention bias improves long context utilization. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics ACL 2024, pp. 14982-14995. Bangkok. Thailand and virtual meeting, August 2024. Association for Computational Linguistics. doi: 10. 1..18853/11/224..0 http:////222...iinnnssseee 890.
URL https:: http///aaaaaattooaoooooo org/2025 2224.22242220 findings--an acl.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022.
AQ Jiang, A Sablayrolles, A Mensch, C Bamford, DS Chaplot, D de las Casas, F Bressand, G Lengyel, G Lample, Saulnier, al. Mistral 7b (2023). arXiv preprint arXiv:2310.06625, 2023.
11

---

Published as a conference paper at ICLR 2025
Zhouyu Jiang. Mengshu Sun, Lei Liang, and Zhiqiang Zhang. Retrieve, summarize, plan: Advancing multi-hop question answering with an iterative approach. arXiv preprint arXiv:2407. 2024.
Jiajie Jin, Yutao 7hu, Xinyu Yang, Chenghao Zhang, and 7hicheng Dou. Flashrag: A modular toolkit for efficient eetrrieall-uggmente generation research. CoRR, abs/2405. 13576, 2024. http::/: /tttps://ariiv. /arxiv. brg/abs/// / 2405. 13576.
Hailey Joren, Chirag Nagpal, Katherine A Heller, and Berk Ustun. Participatory person- alization classification. In Advances Neural Information Processing Systems, vol- ume 36, 2023. URI.
neurips cc ccpppper_pppppppppppppee wesi_ffies//aaaeee htsp//ppp//223////// (tw.//3887766997999999999 7997799999333333 waweeeees--aeeeeCaaaaae Vaper-Conferrecce
Hailey Joren, Charles Marx, and Berk Ustun. Classification with conceptual safeguards. In The Twelfth Inter- national Conference on Learning Representations (ICLR), 2024. 2024/ 2024/postest posstrr/////o 17625.
URL 2024.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodde Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221 2022.
Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarez, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation. arXiv preprint 24x::2409.1294 2024.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin. Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-angmeened generation for knowledge- intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459-944 9474, 2020.
Moxin Li, Wenjie Wang, Fuli Feng, Fenghin Zhu, Qifan Wang, and Tat-Seng Chua. Think twice before assure: Confidence estimation for large language models through reflection on multiple answers. arXiv preprint (.Xiv:2403009972 2024.
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Associ- ation for Computational Linguistics, 12:157-173, 2024. doi: 10.1162/tac__ a__p)oacl__0063...._1 00638. URL https:
//aclanthology, erg/2024. tacl-1.9.
Paul Minciro. Online joint fine-tuning of multi-agent flows. arXiv preprint arXtv:2406.04551 2024.
Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang. Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. Fine-grained hallucination detection and editing for language models. In COLM 2024. 2024.
Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, KaShun Shum, Randy Zhong, Juntong Song, and Tong Zhang. RAGTruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume I: Long Papers). pp. 10862- 118788 Bangkok. Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/w//022..cll acl-ooog------ URL https: : /a//accanntt aclantooooootto wwwpogg...r..o/e. org/ 2024 acl - -long- 585.
Lucas Monteiro Paes, Carol Xuan Long, Berk Ustun, and Flavio Calmon. On the epistemic limits of personalized prediction. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: /ttp////eeeeeeeeeeeeeeeeeeeeeeee net forum? i 1n- INJ.
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Bor Brown, and Yoav Shoham. In-context retrieval- augmentee language models. Transactions of the Association for Computational Linguistics, 11:1316-1331, 2023.
Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slay Petrov, Gaurav Singh Tomar, Iulia Ture, and David Reitter. Measuring attribution in natural language generation models. Computational Linguistics, 49(4):777-84) -40, 2023.
David Rau, Hervé Déjean, Nadezhda Chirkova, Thibault Formal, Shuai Wang, Vassilina Nikoulina, and Stéphane Clinchant. Bergen: A benchmarking library for cetrivval-augmente generation. arXiv preprint MrXiv:2407.0102. 2024.
Dongyu Ru, Lin Qiu, Xiangkun Hu, Tianhang Zhang, Peng Shi, Shuaichen Chang, Jiayang Cheng, Cunxiang Wang, Shichao Sun, Huanyu Li, et al. Ragehecker: A fine-grained framework for diagnosing retrieval- augmented generation. arXiv preprint arXiv:2408.0800 08067, 2024.
12

---

Published as a conference paper at ICLR 2025
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed II Chi, Nathanael Schärli, and Denny Zhou. Large language models can be casily distracted by irrelevant context. International Conference on Machine Learning, pp. 11210-31227. PMLR, 2023a.
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Replug: Retrieval Retrieval-augmented black-box language models. arXiv preprint arXtv:2301.12652 2023b.
Adi Simhi, Jonathan Herzig, Idan Sxpektor, and Yonatan Belinkov. Constructing benchmarks and interventions for combating ballucinations in Ilms. arXiv preprint arXiv:2404.09971 2024.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Is multihop qa in dire condition? measuring and reducing disconnected reasoning. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pp. 8846 8863, 2020.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10: 239-554, 2022.
Tu Vu, Mohit lyyer, Xuczhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Lc, et al. Freshllms: Refreshing large language models with search engine augmentation. arXiv preprint arXiv:2310.0222 2023.
Tu Vu, Kalpesh Krishna, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, and Yun-Hsuan Sung. Foundational autoraters: Taming large language models for better automatic evaluation. arXiv preprint arXiv:2407.1088) 10817, 2024.
Yuhao Wang, Ruiyang Ren. Junyi Li, Wayne Xin Zhao, Jing Liu, and Ji-Rong Wen. Rear: A relevance-aware retrieval-augmented framework for open-domain question answering. arXiv preprint arXiv:2402.17497, 2024a.
Zilong Wang, Zifeng Wang, Long Le, Huaixiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, et al. Speculative rag: Enhancing retrieval augmented generation through drafting. arXiv preprint arXiv:2407.0m 240.02200..,223,, 2024b.
Siye Wu, Jian Xie, Jiangjie Chen, Tinghui 7hu, Kai Zhang, and Yanghua Xiao. How easily do irrelevant inputs skew the responses of large language models? In COLM 2024, 2024.
Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Su. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. In International Conference on Learning Representations (ICLR), 2024.
Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15888 2024.
Zhilin Yang. Peng Qi. Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computa- tional Linguistics, 2018.
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. Do large language models know what they don't t know? In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACI. 2023, pp. 8653-8665, Toronto, Canada, July 2023. Association for Computational Linguisties. doi: 10.18653/v1/2023. 10 URL https: / ac atppa/aaaantttooooooooooo org/ 2023. findings-acll 551.
Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. Making retrieval- language models robust to irrelevant context. In International Conference on Learning Representations (ICLR), 2024.
Tan Yu, Anbang Xu, and Rama Akkiraju. In defense of rag in the of long-context language models. arXiv preprint arXiv:2409.0166 2024.
Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang. R- R-tuning: Instructing large language models to say ii don't know'. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human I Language Technologies (Volune 1: Long Papers), pp. 7106-7132, 2024a.
13

---

Published as a conference paper at ICLR 2025
Jiahao Zhang, Haiyang Zhang, Dongmei Zhang, Liu Yong, and Shen Huang. End-to-end beam retrieval for multi-hop question answering. Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 1718 1731,, Mexico City, Mexico, June 2024b. Association for Computational Linguisties. doi: 10. 12.55/6112224.maacll //aclanthology. crg//024 naacl-long 96.
URL https:
Yujia Zhou, Zheng Liu, Jiajic Jin, Jinn-Yun Nic, and Zhicheng Dou. Metacognitive retrieval retrieval-augmented large language models. In Proceedings the ACM Web Conference 2024, pp. 1453-1463, 2024.
Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, and Tat-Seng Chua. Retrieving and reading: A comprehensive survey open-domain question answering. arXiv preprint 2021.
14

---

Published as a conference paper at ICLR 2025
A SUPPORTING EXPERIMENT INFORMATION
We provide sufficient details to reproduce all of our experiments. We list all of the prompts that we in the next section.
A.1 MODELS
GPT 4o. We use the API-accessible gpt-4o-2024-088-1 model, released on August 6,
2024 (Achiam et al., 2023).
Gemini 1,5 Pro. We use the APl-accessible gemini-1. 1ttpunii-1.------50--- model, released on May 14, 2024 (Gemini Team et al., 2023).
Claude 3,5 Sonnet. We use the only APl-accessible released on June 20, 2024 (Anthropie, 2024).
model,
Gemma 2 27B. We use the publicly available instruction tuned gemma-2-277- model, model, released on Jun 27, 2024 (Gemma Team et al., 2024).
Llama 3.1 8B, We use the publicly available instruction tuned Llama-3 1--.-...111-----ee--e------- model, released on July 23, 2024 (Dubey al., 2024).
Mistral 7B. We use the publicly available instruction tuned Mi aatistal-22 - model, released on May 22, 2024 (Jiang et al., 2023).
FLAMe. We use the published F FLAMe-RM-24B model (Vu et al., 2024).
TRUE-NLI model. Calculated the maximum probability over the chunks in the context. Use threshold of 0.05, where if the maximum probability is higher then this, then we classify 'sufficient context'. The threshold of 0.05 achieved the highest score on our human labeled dataset. We use the 5 xx1 true nli mi iixuuere version of their model (Honovich et al., 2022).
A.2 FINE-TUNING SETTINGS
In our fine-tuning setup, we employed the LoRA adaptation technique (Hu et al., 2022) to fine-tune two models: LLaMA 3.1 8B instruct and fistral-7B-iintruct--000 random subset sampled from the training set of the Musique-Ans dataset or from the development set of the HotPotQA d The prompt template for finetuning is provided in Appendix D.
We used either 2,000-example
For the LoRA parameters, we set the rank to 4 and alpha to 8 for all experiments. The models were fine-tuned over 2 epochs with a batch size of 16 and learning rate of 10-5 We note that the training was not smooth, and different checkpoints led to very different results. To be systematic, we chose the best checkpoint in terms Correct after either 1 or 2 epochs (where for Musique turned out to be after 1 epoch, and for HotPotQA we found that 2 epochs was better).
A.3 DATASETS
We sample 500 examples from HotPotQA and Musique-Are dev sets, following prior work. We use all 'True Premise" questions from FreshQA.
Retrieval for HotpotQA. We adopt the FlashRAG framework (Jin et al., 2024) to implement our Retrieval-Auugeeneer Generation (RAG) process. Our retrieval corpus based on the wiki-18 dataset, utilizing Satfloat/e5-baaeeee from Hugging Face's ' model hub as a Dense Retriever For each query, we retrieved the top 5 documents, which are subsequently concatenated with the query and placed within a prompt template for inference.
To explore advanced retrieval techniques, we also evaluated the REPLUG (Shi et al., 2023b) method. REPLUG enhances the generation quality by prepending each retrieved document individually to the
Available huggi i waw.acc.........o co/mi stral a / Misttal- 78 Instruct-v0..
We use dev set HotPotQA since training set had much different distribution sufficient context examples. Namely, we found the train set to be over 88% sufficient context, while the dev set was only 44%. huuggi ngfacce co/ nt float /e5-/baae-eeeee
15

---

Published as a conference paper at ICLR 2025
input context and ensembling output probabilities across different passes. The REPLUG method is also implemented based on the FlashRAG framework (Jin et al., 2024).
Retrieval for FreshQA We use the urls provided in the FreshQA dataset retrieval for the context. We scraped each url and discarded extra HTML content such as headers, footers, and navigation. We include the title of each webpage the text, convert any included tables markdown, and include the table title immediately before the table in the text. When splitting the tables and text for smaller context windows, we keep tables and sentences intact when possible. large tables that require splitting, we duplicate the table row column headers to include them in each chunk.
16

---

Published as a conference paper at ICLR 2025
Table Fine-tuned (FT) Llama 3.1 8B Instruct and Mistral 3 7B Instruct models. We compare closed book and vanilla RAG with three FT settings, measuring % Correct (% % Abstain (%AA), and % Hallucinate H). Also, "idk" means we change the answer in training samples be "I don't know" instead of the given answer (either for 20% of random examples, or 20% of examples with insufficient context). Best % C for each model/dataset in bold.
Musique
HotPotQA
Model
Variant
RAG
%C
% H
%C
%A
H
Llama Closed Book
2.8
76.4
20.8
18.8
57
24.2
Vanilla RAG
19.6
53.6
26.8
36.8
40.4
22.8
FT GT answer (Data Mix 1)
29.2
31.4
39.4
39,4
27.6
33
idk 20% rand. (Data Mix 2)
26.8
37.2
36
39.2
28.6
32.2
FT idk 20% insuff. (Data Mix 3)
25
38.8
36.2
38
30.4
31.6
Mistral
Closed Book
6.6
29.8
63.6
32
7.6
60.4
Vanilla RAG
28.8
11.8
59.4
46.6
9.2
44.2
FT GT answer (Data Mix
31.4
68.6
43.4
56.6
FT idk 20% rand. (Data Mix 2)
23
1.2
75.8
41.6
0.8
57.6
FT idk 20% insuff. (Data Mix 3)
23
2.2
74.8
41.2
2
56.8
B ADDITIONAL RESULTS
B. FINE-TUNING FULL RESULTS
One aspect of our selective generation framework is that we use FLAMe, 24B model, to provide sufficient context labels. However, we would incur significant overhead we used a 24B model to improve the generation of much smaller L.M.. Instead, we try directly fine-tuning Ilama 3.1 8B and Mistral 7B to increase accuracy with retrieval. Specifically, we experiment with different data mixtures encourage the model to output "I don't 't know" instead of generating an incorrect response.
Fine-tuning Data. We repeat the following process separately for each of the Musique-Ans and HotPotQA datasets create three mixtures of training data with different answers for each dataset. First, we sample 2000 instances. Then, for Data Mix 1, we fine- tune on these instances and keep their given ground truth answer. For Data Mix 2, choose 400 examples (20%) at random and change the answer to "I don't know" before fine-tuning. For Data Mix 3, we instead randomly choose 400 examples (20%) that autorater labels as insufficient context and change their answer "I don't 't know" while keeping the other answers as the ground truth. Our hypothesis is that fine-tuning on Data Mix 2 and 3 should steer the model to abstain more and hallucinate less than with Data Mix
Models, Methods, Metrics. Using the three data mixtures described above, we fine-tune the Llama 3.1 8B Instruct and Mistral 7B Instruct models model with LoRA (details in Appendix A.2). At inference time, we use the standard RAG setup where we add context to the prompt. As baselines, we also evaluate the model without fine-tuning both the closed- book setting (w/o RAG) and the open-book setting (Vanilla RAG). Consistent with the prior experiments, we use an LLM with ground truth answers classify responses as Correct (% C), Abstention (%A), or Hallucination (%
Fine-tuning Results and Discussion. Table shows our experimental results. We verify that the FT variants have higher rate of generating correct answers (% C) compared to closed-book and Vanilla RAG in three of 4 cases. On the other hand, refuting our hypothesis, Data Mix 2 and 3 do not lead to more abstentions than Vanilla RAG. But, they do abstain more than with Data Mix 1, showing the impact of adding "I don't know" in the training set. In general, FT models using RAG output incorrect answers (% at least 31% the time, and often more than they abstain A%.).
B.2 PERFORMANCE BREAKDOWN BY SUFFICIENT CONTEXT
We explore RAG performance by different models for various RAG benchmark datasets. Here, the first column shows performance without RAG (closed-book) while the second column shows performance with RAG (open-booke book). To better understand RAG performance, we use our sufficient context autorater stratify the retrieval augmented generation (RAG) datasets into sufficient and
17

---

Published as a conference paper at ICLR 2025
insufficient context. The third and fourth columns show the performance of the second column stratified by sufficient vs insufficient context respectively.
Categorizing model responses (FreshQA dataset)
Abstain Hallucinate
Correct
0.0%
46.1%
11.7%
1%
Gemini
100%
89.1%
12.7%
1.5 Pro
78.3%
41.2%
40.3%
9.3%
38.2%
8.6%
11.7%
4.0%
17.6%
GPT
79.0%
51.1%
44.1%
53.2%
39.2%
23.1%
18.3%
54.2%
Gemma 27B
11.8%
35.0%
22.8% 8%%
27.5%
53.9%
Without RAG
With RAG
With RAG Suff. Context 77.4% of Dataset
With RAG Insuff. Context 22.6% of Dataset
Figure 5: Correct, hallucination, and abstention fractions across models for dataset FreshQA, stratified by sufficient context. FreshQA includes hand-curated source URLs, which explains the larger percentage of sufficient context (77.4%). FreshQA also specifically explores questions with answers that change based on the question's 's timestamp, which may explain the frequent abstentions without RAG (100% for Gemini 1.5 Pro).
18

---

Published as a conference paper at ICLR 2025
Categorizing model responses (HotpotQA dataset)
Abstain
Hallucinate
Correct
26.0%
33.8%
12.0%
6.5%
16.7%
Gemini 1.5 Pro
8.2%
23.4%
57.8%
67.5%
49.4%
25.8%
26.2%
19.9%
29.0%
10.0%
8.2%
11.5%
GPT 4o
48.0%
65.2%
54.8%
42.8%
7.2%
1.7%
11.9%
Gemma 27B
18.0% 27 2%
64.1%
37.9%
Without RAG
With RAG
With RAG Suff. Context 46.2% of Dataset
With RAG Insuff. Context 53.8% of Dataset
Figure 6: Correct, hallucination, and abstention fractions across models for dataset HotpotQA, stratified by sufficient context. HotpotQA includes questions that are more likely to be answerable without context (e.g., yes or no questions, multiple choice questions, or questions with answers that might be answerable due pre-training). This explains the higher fraction of correct answers without RAG (c.g., 48.0% for GPT 4o).
Table Performance Analysis of RAG Systems Using Human-Annotated Sufficient Context Labels. These tables include results on a curated set of challenging context-dependen questions. Table (a) shows that while larger models generally achieve higher accuracy with sufficient context (present in 54.8% of cases), even top performers exhibit 14-16% eπrо rate. Table (b) reveals that with insufficient context (45.2% of cases), models predominantly abstain from answering (50-73% of instances), though significant hallucination rates (15-40%) persist. These patterns of context- dependent performance and hallucination risk are consistent with our analyses of HotpotQA, FreshQA, and Musique datasets, despite variations in absolute performance due to different task complexities.
(a) Performance with Sufficient Context (54.8% Dataset)
Model
% Correct
% Abstain
% Hallucinate
Gemini 1.5 Pro
84.1
1.6
14.3
GPT 4o
82.5
4.8
12.7
Claude 3.5 Sonnet
85.7
11.1
3.2
Gemini 1.5 Flash
77.8
4.8
17.5
Gemma 27B
71.4
3.2
25.4
Performance with Insufficient Context (45.2% of Dataset)
Model
% Correct
% Abstain
% Hallucinate
Gemini 1.5 Pro
9.6
50.0
40.4
GPT
23.1
61.5
15.4
Claude 3.5 Sonnet
9.6
53.8
36.5
Gemini 1.5 Flash
7.7
73.1
19.2
Gemma 27B
9.6
55.8
34.6
19

---

Published as a conference paper at ICLR 2025
B.3 COMPARISON OF QA EVALUATION METRICS
We compare TWo the LLM-based QA Evaluator (LLMEval) used in the paper with a deterministic lexical matching metric (Contains Answer). The Contains Answer metric labels responses based on whether they contain the exact ground truth answer, while LLMEval uses an LLM to assess semantic correctness.
Table 5 presents model performance across three datasets (FreshQA, Musique, HotpotQA), split by our sufficient context autorater. The results show Contains Answer generally stricter than LLMEval, though both metrics reveal similar patterns in model behavior.
Table 5: Comparison of evaluation metries across models and datasets. We show results for checking whether the response contains one the ground truth answer strings ("Contains"),, where report the % of responses that contain answer. We compare this our LLMEval method that uses an LLM to evaluate the response is correct, abstain, or hallucinated, and we report % correct.
FreshQA
Musique
HotpotQA
Model
Context
Contains
LLMEval
Contains
LLMEval
Contains
LLMEval
Gemini 1.5 Pro
Suff
80.3%
89.1%
60.1% %
83,4%
47.6%
67.5%
Insuff
31.4%
41.2%
33.6%
49.5%
34.2%
49.4%
GPT-4
Suff
84.3%
89.1%
64.6%
83.4%
52.4%
71.9%
Insuff 36.3%
44.1%
44.4%
61.4%
46.1%
59.5%
Gemma 27B
Suff 26.9%
27.5%
10.8%
23.3%
40.7%
64.1%
Insuff 11.8%
6.9%
7.2%
10.1%
22.7%
37.9%
Claude 3.5 Sonnet
Suff 67.9%
73.1%
48.9%
74.0%
46.3%
66.7%
Insuff 26.5%
33.3%
19.9%
40.4%
29.0%
38.3%
The Contains Answer metric exhibits several characteristics when compared to LLMEval:
1. Different formatting affects matching:
Q: What date did the creator of Autumn Leaves die?
Ground Truth: 13 August 1896
Response: August 13, 1896.
Contains Answer: False
LLMEval: Correct
Semantic equivalents are not captured:
Q: What former Los Angeles Lakers majority owner is the father of Jeanie Marie Buss? Ground Truth: Gerald Hatten Buss
Response: Jerry Buss.
Contains Answer: False
LIMEval: Correct
Partial matches can be marked as correct:
Q: What is Amazon Prime Video's most watched premiere ever?
Ground Truth: The Rings of Power
Response: The series explores the forging of the Rings of Power, the rise of Sauron...
Contains Answer: True
LLMEval: Hallucinate
The LLM QA evaluator provides several practical advantages:
20

---

Published as a conference paper at ICLR 2025
Handles variations in model verbosity and formatting
Distinguishes between correct, abstain, and incorrect responses
Enables efficient evaluation across multiple datasets
Our analysis shows two key findings that are consistent across both metrics: LLMs (i) exhibit hallucination even with sufficient context and (ii) struggle to abstain with insufficient context.
21

---

Published as a conference paper at ICLR 2025
C
PROMPTS
C.I SUPFICIENT CONTEXT AUTORATER PROMPT
You are an expert L.M evaluator that excels at evaluating a QUESTION and REFERENCES. Consider the following criteria:
Sufficient Context: IF the CONTEXT is sufficient to infer the answer to the question and IF the CONTEXT cannot be used to infer the answer the question Assume the queries have timestamp <TIMESTAMP>.
First, output list of step-by-step questions that would be used to arrive at label for the criteria. Make sure to include questions about assumptions implicit in the QUESTION. Include questions about any mathematical calculations or arithmetic that would required. Next, answer each of the questions. Make sure to work step by step through any required mathematical calculations or arithmetic. Finally, use these answers to evaluate the criteria. Output the ### EXPLANATION (Text). Then, use the EXPLANATION to output the #### EVALUATION (JSON)
EXAMPLE:
### QUESTION
In which year did the publisher of Roald Dahl's Guide Railway Safety cease exist?
### References
Roald Dahl's Guide to Railway Safety was published in 1991 by the British Railways Board. The British Railways Board had asked Roald Dahl to write the text of the booklet, and Quentin Blake to illustrate it, to help young people enjoy using the railways safely. The British Railways Board (BRB) was a nationalised industry in the United Kingdom that operated from 1963 to 2001. Until 1997 it was responsible for most railway services in Great Britain, trading under the brand name British Railways and, from 1965, British Rail. It did not operate railways in Northern Ireland, where railways were responsibility of the Government of Northern Ireland. ### EXPLANATION
The context mentions that Roald Dahl's Guide to Railway Safety published by the British Railways Board. It also states that the British Railways Board operated from 1963 to 2001, meaning the year it ceased exist was 2001. Therefore, the context does provide precise answer to the question.
### ISON
"Sufficient Context": 1)
Remember the instructions: You are an expert LLM evaluator that excels at evaluating QUESTION and REFERENCES. Consider the following criteria:
Sufficient Context: IF the CONTEXT is sufficient infer the answer to the question and IF the CONTEXT cannot be used to infer the answer to the question Assume the queries have timestamp TIMESTAMP.
First, output list of step-by-step questions that would be used to arrive at label for the criteria. Make sure to include questions about assumptions implicit in the QUESTION Include questions about any mathematical calculations arithmetic that would be required. Next, answer each of the questions. Make sure to work step by step through any required mathematical calculations arithmetic. Finally, use these answers evaluate the criteria. Output the ### EXPLANATION (Text). Then, use the EXPLANATION to output the #### EVALUATION (JSON)
### QUESTION
<question>
### REFERENCES
<context>
22

---

Published as a conference paper at ICLR 2025
C.2 FLAME PROMPT
INSTRUCTIONS:
title: the context sufficient infer the answer the question?
description: In this task, you will be provided with documents and a question. Use one of the following labels under jjudgment":
1. sufficient: The documents are not sufficient to infer the answer to the question. insufficient: The documents are sufficient to infer the answer to question. output fields: judgment
CONTEXT: documents: question: <question>
C.3 LLMEVAL PROMPT
Since the questions in our datasets ask for free form answers, the LLM responses may not exactly match the answers. Hence, we use an LLM to determine: the answers are the same (Correct) or the LLM does not answer the question (Abstain) the answer is incorrect (Hallucinate). We note that prior work has shown that Gemini 1.5 Pro has very high accuracy and correlation with human judgments for this evaluating free form responses (Krishna et al., 2024). Responses resulting in empty strings were classified as "missing," " while variations of "I don't know" were also treated missing. We normalized both the ground truth answers and the model's responses by removing punctuation, converting to lowercase, and eliminating stop words.
=Task===
I need your help in evaluating an answer provided by an LLM against ground truth answers. Your task is to determine if the LLM's response matches the ground truth answers. Please analyze the provided data and make a decision.
=== Instructionss
Carefully compare the "Predicted Answer" with the "Ground Truth Answers". 2. Consider the substance of the answers look for equivalent information correct answers. Do not focus on exact wording unless the exact wording is crucial to the meaning.
3. Your final decision should be based whether the meaning and the vital facts of the "Ground Truth Answers" are present the "Predicted Answer." 4. Categorize the answer as one of the following:
"pperfec":: The answer is completely correct and matches the ground truth.
"acceptable": The answer is partially correct or contains the main idea of the ground truth.
"incorrect": The answer is wrong or contradicts the ground truth.
"missing": The answer is "I don't know". "invalid question", or similar responses indicating lack of knowledge.
===Input Data===
Question: What 1876 battle featured the Other Magpie?
Predicted Answer: The Other Magpie fought in the Battle of the Rosebud.
Ground Truth Answers:
Battle of the Rosebud
===uttt Format===
Provide your evaluation in the following format: Explanation: (How you made the decision) Decision: (One of perfeett", "acceptable". "incorrect", "missing")
Please proceed with the evaluation.
23

---

Published as a conference paper at ICLR 2025
C.4 DATASET QUESTION ANSWER PROMPTS
The CoT prompt instructs the model provide accurate and concise answer based solely the given search results, using an unbiased and journalistic tone. The prompt includes an example question, references, and answer to guide the model's response format. To extract the final answer, we implemented a pattern matching technique on the model's response, specifically targeting the text following "The answer is:" for CoT prompts.
Chain of Thought (CoT)
Write an accurate and concise answer for the given question using only the provided search results (some of which might be irrelevant). Start with an accurate, engaging, and concise explanation based only the provided documents. Must end with "The answer is:". Use an unbiased and journalistic tone.
EXAMPLE:
### Question
<example question>
### References
<example references>
### Answer
<example answer>
### Question
<question>
### References
<references>
### Answer
Answer Only (AO)
Write accurate and concise answer for the given question using only the provided search results (some of which might be irrelevant). Do not say anything other than the answer itself.
EXAMPLE:
### Question
<example question>
### References
<example references>
### Answer
<example answer>
### Question
<question>
#### References
<references>
### Answer
D FINE-TUNING AND RAG PROMPTS FOR LLAMA AND MISTRAL
Finetuning Prompt (FT)
24

---

Published as a conference paper at ICLR 2025
Answer the question based the given document. Only give the answer and do output any other words. The following are given references.
### References
<references>
Please follow the following guideline when formulating your answer: if you are uncertain or don't know the answer, respond with "I don't tt know".
### Question
<question>
### Answer <answer>
Evaluation Without RAG Prompt
Answer the question based your knowledge. Only give the answer and do output any other words. Please follow the following guideline when formulating your answer: you are uncertain or don't 't know the answer, respond with don't know". ### Question
<question>
### Answer
Evaluation With RAG Prompt
Answer the question based the given document. Only give me the and do output any other words. The following are given references.
### References
<references>
Please follow the following guideline when formulating your answer: if you are uncertain or don't know the answer, respond with "I don't 't know".
### Question
<question>
### Answer
25


Reviews:
[{"review_id": "NOXnojZLeF", "reviewer": "Reviewer_9Kme", "summary": "This work analyzes how LLM behavior differs for question answering with and without sufficient context. A context C is considered sufficient to answer a question Q if it supports an answer A in C. The authors make it a point to ensure that the ground truth answer does not matter in the definition of sufficient context.\n\nFirst, the authors collect an evaluation set of 115 (question, context, answer) triplets (from 3 QA datasets) and annotate them for sufficiency. A set of autoraters is evaluated on the task of judging sufficiency. Gemini 1.5 Pro (1-shot) achieves an F1 score of 93% on evaluating sufficiency. For the rest of the analysis, Gemini 1.5 Pro (1-shot) labels examples as having sufficient/insufficient context.\n\nNext, the paper analyzes how context sufficiency differs for different QA datasets. (Question, context) triples are collected for 3 popular QA datasets: FreshQA, HotpotQA, and Musique. Contexts are either gold snippets from the dataset (for Musique) or retrieved with a dense retriever. Results show that sufficiency does not increase significantly after the first 6000 tokens of context. FreshQA has the most sufficient per query while the context is sufficient for only about 45% of the questions from Musique and HotPotQA.\n\nNext, the paper analyzes how context sufficiency affects the accuracy of Retrieval Augmented Generation systems. They study the behavior of 4 API-based QA models.\n1. The accuracy of QA models is higher when given sufficient context than without. However, even without sufficient context, the models answer the query about 35% of the time.\n2. When given sufficient context, models have a low abstention rate and relatively higher error rate.\n3. When given insufficient context, the abstention and hallucination (error) rates both increase.\n\nThe authors show a qualitative breakdown of the questions the QA models answer correctly with insufficient context.\n\nFinally, the paper studies ways to use the \"context sufficiency predictor\" for a well-calibrated QA model. The sufficiency predictor can be used as a signal for selective prediction. This provides some gains over using just the self-reported model confidence especially with a worse QA model. Fine-tuning the QA models to abstain (either on random contexts or contexts predicted to be insufficient) improves the ability to abstain but does not improve overall QA accuracy.\n\n---\n\nScores changed from 8 -> 6. See reasoning [here](https://openreview.net/forum?id=Jjr2Odj8DJ&noteId=fVAH86Vrwz) and [here](https://openreview.net/forum?id=Jjr2Odj8DJ&noteId=a0bQQDrq6s).\n\n--- \n\nFinal Update After Rebuttal\n---\n\nI'm satisfied with the author responses. I am changing my score back to 8. Ideally, this comes with the caveat: I think the core argument of the paper and findings have value, but I believe that the final version of the paper needs detailed discussions based on the author-reviewer discussion.\n- Clear discussions of the different QA benchmarks and their impact. This will address issues of why different findings are not directly comparable and the nuances of why certain QA datasets behave differently.\n- The final version of the paper should include a **quantitative count** of the different ways the QA models are correct given insufficient context. This would also better answer questions about automated metric reliability (rather than anecdotal evidence)", "questions": "1. The autorater is evaluated on single-hop QA datasets like PopQA, FreshQA, Natural Questions, and EntityQuestions. The rest of the paper analyzes QA performance on the FreshQA (in-domain) and potentially out-of-domain HotpotQA and Musique datasets. Can the automatic judgments be trusted out-of-domain?\n2. Following up: since you observe rater error in Table 2, what is the estimated prevalence of this category?\n3. One other potential mechanism of the correct answer given insufficient context is that there are only one/few entities of the expected type (given the question) in the context. Is this mechanism never observed?\n4. What is the context source for FreshQA? Is it the labeled sources? If so, are there no examples of insufficient sources for FreshQA questions in the automated evaluation dataset?\n5. Lines 185-190: Remark about ambiguous queries: It is unclear to me why the given example illustrates how ambiguous queries are handled. My understanding is that ambiguous queries have unclear interpretations, not the context.\n6. Do you plan to release the sufficient context dataset? If not, how do you ensure the following: \"very challenging, including single- and multi-hop questions, as well as adding highly related information in the context even if it is not sufficient\"", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The presentation of the paper is clear and easy to follow.\n- Appropriate evaluations are used at each stage. The sufficiency predictor is first evaluated and then used for later analysis.\n- The paper reports findings of unexpected model behavior with and without sufficient context, which are valuable to the community.\n    - It is interesting that Gemini 1.5 Pro does not abstain very often even on passages that Gemini 1.5 Pro has labeled has having insufficient evidence (as autorater)", "weaknesses": "1. Some details in the paper are vague. While they don't obstruct understanding, they may obstruct reproducibility [see Questions]\n2. While the paper spends a lot of time analyzing the case of correct answer with insufficient context, the opposite case of incorrect answer with sufficient context is equally interesting and warrants exploration\n3. The qualitative evaluation does not attempt to quantify any of the error types. For example, how prevalent is the \"Autorater error\" [see Questions]", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QHUuHyrg7O", "reviewer": "Reviewer_HFFU", "summary": "This work asks why LLMs hallucinate when integrated into RAG systems. They carefully define \"sufficient context\", i.e. whether the retrieved context unambiguously supports some answer for a question, while making sure their definition is appropriate for various types of RAG systems, e.g. multi-hop reasoning. They then develop an \"autorater\" prompt for this quality, which they validate on a set of 115 hand-labeled and challenging data points. Using this autorater, they analyze the performance of an off-the-shelf retrieval system (REPLUG) across datasets and then analyze the performance of different LLMs in RAG, exploring their behavior (correct answers vs. abstaining vs. hallucination) when sufficient context is or isn't available. Overall, they find that models abstain less with RAG and hallucinate whether sufficient context is available or not. They build a small set of reasons answering questions correctly despite insufficient context, e.g. when it's due to models using parameteric knowledge when faced with incomplete multi-hop information. Leveraging their autorater, the authors explore a number of approaches for creating a tradeoff between tuning abstention, e.g. only answering when context is clearly sufficient and the system is confident in high-stakes applications, while relaxing that (to answer more questions) elsewhere.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and easy to follow.\n2. The analyses are genuinely insightful and offer a number of non-trivial findings, as included in my summary.\n3. The autorater, its validation set, the analyses approach, and the method proposed for trading selectivity vs. coverage are all likely to be valuable resources for future work.", "weaknesses": "1. How sensitive is the analysis to the choice of REPLUG and/or E5 as retriever components? The paper glosses over the IR subsystem here in an extreme way. Most of this information is simply not presented. For example, I have to read the results in Figure 2 as essentially an evaluation of REPLUG, but they could also be read as (and appear to be presented as) analyses of the datasets themselves. Is REPLUG able to handle multi-hop queries? I am not aware that it's a multi-hop retriever at all, so the choices here, while still plausible, are lacking in justification.\n\n2. How good is the autorater? The paper includes table 1 but to my knowledge does not directly refer to it; and overall, the paper doesn't seem to offer a convincing discussion of this, given how essential the autorater is to the analysis and methods. Will the 115 data points be released? There's some risk of circularity here. What makes large LLMs so prone to hallucination if one could trust that a simple prompt could reliably discover these cases?\n\n3. Given that the autorater is not perfect, and given the subtle nature of its job (as discussed in the Remarks under the definition of Sufficient Context), how correlated are autorater mistakes with determinations of Correct/Hallucination in Figure 3?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Jz1pHMm787", "reviewer": "Reviewer_pMDo", "summary": "The paper studies the performance of LLMs with retrieval-augmented context. In particular, the paper analyses how often the retrieved context is sufficient for answering a question. It then analyses how the behaviour of LLMs changes depending on whether the retrieved context is sufficient or not, i.e. in how many cases the LLM hallucinates, abstains, or answers correctly. Furthermore, the paper shows that checking for sufficiency can be used to \"teach\" an LLM to abstain (either by directly using the sufficiency prediction in combination with LLM-generated confidence metrics, or by fine-tuning a model to abstain).", "questions": "What can we really conclude without manual validation of the auto-rater predictions?\n\nIs the difference between context sufficiency and NLI based methods really that significant (for practical purposes)?\n\nWhat are the key insights that your analysis has revealed that weren't known from previous work?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 1, "strengths": "Retrieval-augmentation is a popular and promising method to improve the performance of LLMs. Previous work has shown, however, that there are issues with this approach, especially if the retrieved context is not helpful and/or too long. The analysis in the paper contributes to the study of these effects.\n\nThe paper is easy to follow and some of the experimental results may be useful to some practitioners.", "weaknesses": "As the paper acknowledges in the related work section, there is a substantial amount of work that has already analysed the performance of RAG systems in relation to the quality of the retrieved context. The paper claims to study this topic through \"a new lens\", but the specific aims of the paper are not really clear. What is it really that this paper shows that wasn't already known? Perhaps a more explicit formulation of hypotheses/research questions that have not yet been answered could help here. In its current form, the paper reads like a preliminary exploratory analysis of LLM performance, rather than a mature contribution that is ready for publication.\n\nThe notion of \"sufficient context\" is central to this paper, but it is not actually defined in a very precise way. First, the definition (line 157) is rather \"circular\": to determine if a context is sufficient, we need to determine if it allows us to infer the answer, but that actually depends on the capabilities of the LLM. The definition is also confusing, as it talks about \"plausible answers\" rather than the actual answer. I think what is meant is that a context can be sufficient even if the answer it contains is incorrect. In any case, this point needs a lot more discussion. There are further issues, for instance, the definition assumes that an answer can be inferred up to 4 inference hops. Why 4? \n\nTo assess which contexts are sufficient, the paper relies on LLM-generated assessments. This is a key limitation, and the current analysis should be expanded with a human-annotated experiment. Indeed, currently, the paper does not actually study the impact of contexts being sufficient, but rather the impact of an LLM thinking that the context is sufficient. Even if the LLM that predicts sufficiency is different from the one used for generating the answer, this kind of dependency on LLMs may clearly affect the results.\n\nThere are lots of minor issues with the writing and with formulations being imprecise.\n\n* The abstract contrasts the behaviour of closed models with open-source ones. But isn't the contrast rather between larger and smaller models?\n* The example in Remark 2 is of an ambiguous context rather than an ambiguous query.\n* The example in Remark 3 is of an ambiguous query rather than an ambiguous context\n* Lines 225-228: where are these results shown?\n* Table 2: I believe the explanation on the first row should be \"some chance\" and the explanation on the second row should be \"50% chance\"\n* Line 384: sufficient -> insufficient", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9o2vzjF6Fg", "reviewer": "Reviewer_BUto", "summary": "This paper introduces a novel framework for understanding context sufficiency in Retrieval-Augmented Generation (RAG) systems, where retrieval of external context aims to improve Large Language Model (LLM) accuracy and reliability. The authors define “sufficient context” as context providing enough information for a model to answer a query without additional details. They introduce a sufficient context autorater to classify instances based on this metric, allowing for systematic error analysis across several proprietary and open-source LLMs. Key findings suggest that proprietary models (e.g., Gemini, GPT) often generate incorrect answers without abstaining when context is insufficient, while open-source models are more likely to hallucinate or abstain even when context suffices. The authors propose a selective generation technique to reduce hallucinations by leveraging sufficient context signals, showing improvements of up to 10% in response accuracy among Gemini, GPT, and Gemma.", "questions": "1. How is \"sufficient context\" defined for multi-hop reasoning?\nClarify the specific criteria or thresholds the autorater uses, especially for complex multi-hop queries.\n\n2. How does the autorater handle ambiguous or conflicting contexts?\nWhat rules are applied when contexts contain contradictory or unclear information?\n\n3. How does retrieval quality impact context sufficiency and model performance?\nCould improved retrieval reduce hallucination rates by providing more sufficient contexts?\n\n4. What are the computational demands of the selective generation framework?\nIs it feasible for real-time applications, particularly with large models?\n\n5. Is the sufficient context framework applicable beyond QA?\nCould this framework be useful for other retrieval-augmented tasks, like summarization?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Novelty of Sufficient Context Concept: The paper addresses a fundamental gap in RAG literature by formalizing the concept of “sufficient context.” This is essential for evaluating how well LLMs make use of retrieved information, especially when distinguishing between sufficiency and insufficiency.\n\n2. Methodology and Rigorous Analysis: By creating a sufficient context autorater and testing it across proprietary and open-source models, the paper presents a solid empirical foundation. The detailed experiments on multi-hop and open-book QA datasets (e.g., Musique-Ans, HotPotQA, FreshQA) are impressive, particularly in showing how context sufficiency affects model performance.\n\n3. Impact on Hallucination Mitigation: The proposed selective generation framework that combines sufficient context labels with model confidence offers a practical approach to reduce hallucinations. This approach is adaptable across different LLM architectures, which is an important contribution for model deployment.\n\n4. Evaluation Metrics and Generalization: The paper extensively evaluates its autorater and selective generation framework across various datasets and models, which provides strong evidence of generalizability. The careful selection of open and proprietary models, and focus on multi-hop reasoning tasks, enhance the robustness of the conclusions.", "weaknesses": "1. Clarification on the Autorater’s Design and Reliability: The paper could better explain the autorater’s design, particularly regarding criteria used in challenging multi-hop contexts. Additional qualitative examples could help clarify how the autorater discerns sufficiency, especially when dealing with ambiguous or incomplete context.\n\n2. Limitations in Practical Deployment: While the selective generation framework demonstrates significant accuracy improvements, there is limited discussion on computational overhead. For example, implementing a selective generation method based on sufficient context may require high processing power for real-time applications, particularly with large models and datasets.\n\n3. Impact of Retrieval Quality: Although the paper acknowledges that retrieval quality can affect context sufficiency, the experiments do not address how different retrieval strategies impact results. Comparing RAG performance using advanced retrieval techniques, like iterative or corrective retrieval, would strengthen the argument that improving sufficiency is an independent aspect of model improvement.\n\n4. Scope Beyond QA Tasks: The paper focuses on QA, but RAG systems are used across various tasks, including summarization, which the authors briefly mention. A future direction could involve validating sufficient context’s relevance to other NLP tasks, which would enhance the paper’s broader applicability.\n\n5. Explain Model-Specific Findings in More Detail: The differential behavior of models, such as Gemini’s reduced abstention rate and GPT’s lower accuracy without RAG, would benefit from additional discussion. These findings highlight variations in model handling of retrieval, and more insight here could guide further model-specific improvements.\n\n6. Consideration of Confidence Thresholds: The selective generation approach could further explore confidence thresholds to find an optimal balance between abstention and accuracy. Including ablation studies or sensitivity analyses on confidence levels would make this method more practical for different application contexts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)