Optimizing Retrieval for RAG via Reinforcement Learning
Authors: Jiawei Zhou
Lei Chen
Venue: NeurIPS 2025 poster ()
Keywords: Retrieval-augmented Generation
RAG
RL
Primary area: applications
TL;DR: Optimizing Embedding-based Retrieval for RAG via Reinforcement Learning.

Abstract: As retrieval-augmented generation (RAG) becomes more widespread, the role of retrieval is shifting from retrieving information for human browsing to retrieving context for AI reasoning. This shift creates more complex search environments, where relevance is difficult to pre-define. Existing retrievers rely on supervised fine-tuning (SFT) with human labels or synthetic data, resulting in static relevance that struggles to adapt to diverse RAG environments. To address this challenge, we propose R3, a Retrieval framework optimized for RAG through Reinforcement learning (RL). Specifically, we adopt an RL training paradigm that enables the retriever to explore and self-improve within given RAG environments, automating the learning process with minimal manual experimentation or tuning effort. Extensive experiments across diverse tasks demonstrate that \ours improves RAG performance by 5.2% over the original retriever and surpasses state-of-the-art retrievers by 4.9%, while achieving comparable results to LLM-augmented retrieval and RAG systems built on post-trained or instruction-tuned LLMs. It is both efficient and practical, requiring only 4 GPUs and completing training within a single day.

Optimizing Retrieval for RAG via Reinforcement Learning
Jiawei Zhou Lei Chen
The Hong Kong University of Science and Technology
The Hong Kong University of Science and Technology (Guangzhou) {jzhoubu, lei (cwocect ust.chk.....kk...........
Abstract
As retrieval-augmented generation (RAG) becomes more widespread, the role retrieval shifting from retrieving information for human browsing to retrieving context for AI reasoning. This shift creates more complex search environments, where relevance is difficult to pre-define. Existing retrievers rely on supervised fine-tuning (SFT) with human labels or synthetic data, resulting in static relevance that struggles to adapt to diverse RAG environments. To address this challenge, we propose R3, a Retrieval framework optimized for RAG through Reinforcement learning (RL). Specifically, we adopt an RL training paradigm that enables the re- triever to explore and self-improve within given RAG environments, automating the learning process with minimal manual experimentation or tuning effort. Extensive experiments across diverse tasks demonstrate that R3 improves RAG performance by 5.2% over the original retriever and surpasses state-of-the-art retrievers by 4.9%, while achieving comparable results LLM-augmented retrieval and RAG systems built on post-trained or instruction- tuned LLMs. It is both efficient and practical, requiring only 4 GPUs and completing training within single day.
Introduction
Retrieval-augmented generation (RAG) [1, 3] is a well-established paradigm that integrates large language models (LLMs) [4] with information retrieval (IR) [5] to access external knowledge. Recently, RAG has rapidly evolved into foundation for Examples include web-browsing agents [7, 8] that interact with live web content, conversational chatbots [9, 10, 11] that integrate retrieval to enable knowledge-groundee and emotionally aware dialogue, and DeepResearch assistants [12, 13, 14) that autonomously analyze and synthesize information across multiple sources produce comprehensive reports.
wave of advanced Al applications [6].
These RAG applications signify a fundamental shift in the role of retrieval, from search for humans to search for AI. As illustrated in Figure 1, traditional IR searches information for human users, where relevance is pre-defined as superficial similarity to ensure interpretability [15, 16], so that human users can instantly grasp the underlying relations. In contrast, RAG searches contextual knowledge for Al systems, which can efficiently process and reason over vast amounts information and are designed to handle diverse and evolving tasks. The retriever functions within a RAG environment that encompasses tasks, workflows, LLMs, and other components, making relevance more environment-depeedeen. hard pre-define, and necessary explore.
internal component
Existing RAG applications have several retrieval options. Some leverage search engine APIs, which convenient but expensive, difficult to scale, and cannot be further optimized [17]. Others deploy neural retrievers locally. However, directly applying these off-the-shelf retrievers can suffer from gap between IR and RAG. Studies have shown that traditional IR often retrieve semantically related but unhelpful content [18, 19, 20], LLe--grrrtteed documents [21]. or misleading evidence [22],
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Query
Doc
Query
Doc
Environment
Generation
Search
Search
Task
LLM
Workflow
Search for Human
Search for
General tasks
Diverse and Complex tasks Results Processed by Al Systemn Complex Relevance (To Explore)
Results Presented to Human Superficial Relevance (Pre-defined)
Contrastive Learning
Reinforced Contrastive Learning
Signal Source:
Labsled or synthetic data
Interact with environment Relevance that require exploration Reinforcement learning (RL)
Targeted Relevance:
Relevance that be pra-defined Supervised line-tuning (SFT)
Training Paradigm:
Figure I: Comparison of traditional IR setting and RAG setting.
all of which can degrade RAG performance. Furthermore, Cuconasu al. [23, 24] demonstrate that while traditional IR retrieves documents containing the correct answer, these documents do not necessarily improve RAG performance. Additionally, related findings have emerged in the study of LLM faithfulness, where Anthropic [25] and Apple [26] found that chain-of-thought prompts often fail to reflect the actual reasoning process of LLMs,, and even advanced LLMs sometimes fail to follow explicit instructions. These observations reveal a key challenge of context engineering: there no clear guidance on what constitutes effective context across different environments, leaving the process heavily dependent on manual effort.
In addition to the studies mentioned above, we further examine this gap through empirical experi- ments presented in Appendix D, where we evaluate the varying capabilities of different retrievers (unsupervised, supervised, and SOTA) tasks of varying complexity in both IR and RAG settings. In IR, we examine whether the retriever can retrieve documents containing the answer, while in RAG, assess whether the retrieved document can prompt the LLM generate the correct answer. The results reveal two key findings. Finding 1: Better IR performance does not always lead to better RAG performance, especially when task shift occurs. For QA tasks, observe that higher accuracy in IR settings generally correlates with better RAG performance. However, this trend does not hold for non-QA tasks. For example, on PubHealth, vanilla DPRMs outperforms SOTA retriever in RAG settings, despite being weaker in IR metrics. Similarly, on ARC, the unsuper- vised CONTRIEVER achieves the best RAG performance, outperforming its supervised counterparts. Finding 2: Substantial room for improvement in retrieval within the RAG system. We find that although most required knowledge already exists in the datastore, each retriever succeeds on only subset of queries. For example, on NQ. 77% of queries can be answered by at least one retriever via RAG, yet the best single retriever covers only 43%.
To bridge the gap from IR to RAG, we propose R3, a Retrieval framework optimized for RAG via Reinforcement learning. Unlike prior works that learn pre-defined relevance via supervised fine- tuning (SFT) through annotated or synthetic data, R3 aims to explore relevance via reinforcement learning (RL) within the RAG environment. Specifically, R3 follows the standard RAG workflow: it retrieves documents and incorporates them as context for generation. Based on the generated outcomes, the retrieved documents are labeled as positive or negative with respect to the query, and contrastive learning objective is applied to optimize the retriever accordingly. We refer to this paradigm as reinforeed contrastive learning, where contrastive signals arise from interaction with environment rather than manual annotation. Extensive experiments demonstrate that R3 enables retrievers to effectively adapt from IR to RAG, achieving an average 5.2% improvement in 1-shot accuracy and consistently outperforming SOTA retrievers as well as L.M.-empowered retrieval methods.
Our contributions are summarized as follows:
1. We identify and analyze the relevance gap between traditional IR and RAG settings, supported by evidence from prior studies across diverse domains as well as our own empirical findings.
2. We propose R3, which pioneers reinforcement learning optimization for embedding- retriev- in RAG systems.
3. R3 is simple and effective, delivering consistent RAG improvements with modest training costs, consistently outperforming existing SOTA retrievers and achieving improvements comparable to those seen in post-trained or instruction-tuned LLM across several tasks.

---

2
Related Works
The Evolution of Information Retrieval. Information retrieval has long been a core mechanism for various applications. The field began with heuristic approaches such as TF-IDF and BM25 [27], which have been widely adopted in both research and industry. As the field progressed, the advent of neural networks gave rise to neural retrieval [28]. Yet, these models still heavily relied on training data annotated using BM25 and manual validation [29. 30J, leading to superficial relevance. In recent years, however, the rise of RAG has exposed the gap between search for humans and search for AL This shift has prompted efforts to optimize retrieval models specifically for RAG. Early work in this direction focus on learning-free methods, such as using L.MSs or heuristics to determine when to retrieve [31, 32]. performing adaptive retrieval with reasoning [33, 34, 35], or decomposing and refining complex queries [36, 37]. Other research explored learning harning-based approaches, where retrievers are optimized jointly [2, 38] or separately [39, 40] with LLMs, using unsupervised objective like prefix language modeling [40], or supervised objective that maximize question-anse -nnserr generation likelihood [41]. Despite their successes, these methods often come with high computational costs and focus primarily on general relevance, with limited consideration of specific RAG environments.
Beyond conventional RAG systems that search context for LLMs, there are many more complex, real-world environments where retrieval must interact with other intelligent components address emerging challenges. For example, searching multi-modal data to improve multi-modal LLM reasoning [42, 43], retrieving MCP services for agents [44], or searching action modules or planning for robot embodiment [45, 46]. All of these emerging and advanced applications highlight complex environment specific relevance. Our method, in line with this area, provides a solution by efficiently training specialized retrievers tailored to a specific environment for applications.
Contrastive Learning- Contrastive learning (CL) [47, 48] is well-established method used to train bi-encoder retrievers by pulling representations of positive pairs closer and pushing negatives apart [28, 49]. CL relies on contrastive labels, which define the positive or negative relationship between pair-wise data. These relational data (i.c.. positive and negative query-document pairs) can be obtained through annotation [29], extracted from the web [50, 51], or, more recently, synthesized using LLMs [52, 53]. While these contrastive labels provide effective training signals, they are inherently defined at the data level, making them difficult to transfer across different environments when needed. If components such as tasks, LLMs, or workflows within the environment change, these labels often lose effectiveness and require reconstructed. We further introduce Reinforced Contrastive Learning (RCL), which distinguishes itself from conventional CL by constructing contrastive signals on-the-fly in fried-and manner. Simply put, traditional CL involves SFT annotated data to learn pre-defined relevance, while RCL leverages RL to explore proper relevance that suit the specific search environment. Our framework follows RCL paradigm, enabling versatility and automatic generalization to any RAG environment.
3 Methodology
In this section, we introduce R3, a framework that optimizes Retrieval for RAG via Reinforced constrastive learning. While idea is straightforward, it presents several challenges. In the following sections, we detail the problem setup ($3.1), the solution to the first challenge, which addresses the on-policy retrieval issue ($3.2), the solution to the second challenge, which focuses on mitigating the cost of autoregressive generation ($3.3), and the reinforced contrastive learning ($3.4).
3.1 Problem Setup
A RAG framework typically consists of:
Retriever R4 parameterized by 0;
Datastore D containing vast number of documents d;
User query q, and the corresponding answers a, if provided;
RAG environment enc, which refers to everything surrounding the Ra, such LLM G, task instructions, and specific workflows, etc.;
Reward function Reward(-) )) that evaluates the quality of the generated response.
3

---

The downstream RAG pipeline generally follows:
1. Retrieval: retrieve the top-k relevant documents from the D, with relevance function fa:
{d] Ro(g, D,
argmax;Jooq... d) dED
2. Generation: generate response based on the retrieval results and the environment enc.
y = Gene (g. {d]x) e RAGolg env)
3. Evaluation: measure the quality of the generation y. In our setup, we measure whether the generation contains the answer, which can considered as a binary reward:
Reward (ū) = {
ÿ contain a, otherwise.
Optimization Goal. Given a RAG environment, R3 aims to learn the retrieval parameters to maximize the system reward for all training queries:
i
argmax
Reward
b4
argmax
Reward (RAGo(q ene))
4q
Overview. Our work applies reinforcement learning (RL) to learn embedding- retrievers, where downstream rewards used to ensure consistent improvements. A key challenge lies in the interaction cost of retrieval within the environment, given that retrieval is only one small step in the overall RAG pipeline. The main contribution this paper is to make this interaction cost manageable, using various approximations and designed retrieval architectures. We position our work as leveraging these techniques to automate retrieval learning for RAG with stable improvement, aiming to achieve better trade-off between cost and effectiveness in practical applications.
RAG Environment
In-batoh
Positives
Hard Negalives
Prob
Gen
OR
Di
D
Di
Di
Di
Query
Semi-paraaetric Retrievaf
Datastore
Docs
Task
Probability Threshold Preeccoputed Offiins)
LLM
Workflow
1n Prppeesinso peressice
probutilty
Q
0-d1
o-di
Q-00
0.0
Elgi 6ir) fokenezation
Q
0vbi
0-bi
0-0:
0-0:
0:0
Godi
Ekd
Tid.
(d)
ambied
[Bk]]
When Foid x 0.55 Gan pans eva.
Probabiltty
Qr
0.01
0.00
0.00
0.0
0.0
Sg No need rebuld Indox
One forward pass to compute probabaitty
Contrastive Learning of Retnnoor Ro.
tsoeression how couttedyy
Goneration
e (aa
3.
Cor skstent e endroommant as fffencce.
Multill pass generate for avoluation
argmax
Enp elaant
On-poiicy Retrieval
Probabiiity
Probabbitt-ssppraainnn approximated Generation
Reinforced Contrastive Learning
Figure 2: Illustration of the R3 training process.
Training Pipeline. Our training consists of three steps: (i) on-policy retrieval that conduct search during training with its latest parameters; (ii) approximiated LLM generation based on the retrieved documents; (iii reinforced contrastive learning to optimize the retriever. Steps and ii present practical challenges, which we address in the next section. Both are essential to enable effective reinforced contrastive learning. An illustration our framework depicted Figure
3.2 On-Policy Retrieval
In this step, given query q, the on-policy retriever to retrieve the top-k relevant documents. The main challenge in this stage concerns the retrieval index.
Challenge of Index Staleness. As is well known, a bi-encoder retriever consists of query encoder Eº and document encoder Ep. During training, as the parameters update from - 0', query

---

embeddings can be computed on the fly using the updated encoder Eg. since the number of queries limited to the batch size. In contrast, the document datastore typically contains billions or even trillions of documents (i.e., [P] >> 1), making it impractical to recompute document embeddings with the updated encoder F. As result, the index F (D) becomes stale as training progresses 0 >> f' Fundamentally, this challenge stems from the coupling between index and parameters 0.
Prior solutions alleviate index staleness by periodically rebuilding the index [2, 40] or freezing the document encoder during training [41, 38j. While shorter re-index intervals are observed to yield better results [54], longer intervals are often chosen to reduce re-indexing costs. Although these strategies alleviate index staleness to some extent, residual staleness remains and ultimately limits the
effectiveness of retriever training.
Our Solution. We adopt our recently proposed semi-parametric retriever architecture S1DR [55], which is designed address this issue.
Intuition ning leed
Language
Morplig Fokenizattinn (uiee www.rreesosscccsssooooeso Sparse Swweeeilll (Mocabllary Aligh!
T(x)
hild
helel
ViHi OI-T
Transformer Bocel's
Orsagevised
requence
token
One Hot Represertationn
Intial Cirbeddnn offinnss
Frnil Emrbedding
Prosstalyyorr over wodburr
best Mocoad Token Prodioo
olfokens
Tokknn
Training
Align aign + and passage representations during training
Disentangled Vocab Space
@ aige +0000
[Sparse Embedding)
project
project
project
poojett
Entangled Space (Dense Embedding)
Conmrnst
Disentangled (Tokanization)
wed Vocab Space
C D.0
GOCB
enoel
tolenize
tolontoe
tokeritee
Tükerize
toleentee
tallerbee
Quera
Entangled Retriever
Disentangled Retriever kalk Sparse Lexical Retrievent
Semi parametric
Retrieval Methed
Dense Retriever)
Disentangled Retriever (SiDR)
Inference (4 Options for SiDR)
Neural lower
Towert One Toleenizer Tower
praject
project
search
embod
COO 1000
Sakenize
tolenize
Takenbe
tokence
Query
Query
Doc
Query
Doc
Embedded
Tokenized
Embedded Query
Embedded Query
Tokenized Dec
Parametric Search (Option 1)
Alpha Search
Beta Search
Beta Search Re-rank
(Option 2)
[Option
(aka Late-Parametric] (Option 4)
Figure 3: Overview illustration of SIDR.
Revisiting SIDR. Figure 3 illustrates the core idea of SIDR. Intuitively, a language model together with its pre-training head can be viewed as an auto-encoder that encodes one-hot token representations T(x) E {o, 1} 13ll into softmax vocabulary probabilities (zx) E (0, 1) VI to predict masked or next tokens. Inspired by this, SIDR aligns bag-of-token representation with sparse embeddings that, downstream, the embedded query can directly search over tokenized documents, thereby decoupling the index from parameters 0.
Specifically, SIDR builds upon a BERT-based models with three simple modifications to its pre- training head: (i) replacing the softmax with clulp activation; (ii) aggregating token representations via max pooling; and (iii) applying top-k sparsification. Given a sequence X, SIDR embeds into a -dddimensionna sparse embedding which we refer to Ex(X) (0, IVI During training, SIDR applies constrastive learning across three channels: Eo(g) with Ed(d). T(q) with Eo(d), Eg Eg(g) with T(d). In this setup, it succeeds in using Eg (q) to search over bag-of-token index T(D). We expand on SIDR in more detail in Appendix C.
5

---

On-policy Retrieval. During R3 training, we adopt the late parametric mechanism of SıDR (shwon as option 4), which first retrieves the top-m documents using the bag-of-tokens index (D), and then embeds the these documents for re-ranking. This process can be formulated as:
(d)m m (R) T(D), {d]x = (())) Fo (m))
To reduce training costs, we set m = k = 20 in our experiments. Additional ablation studies and comparisons on re-indexing solutions provided in Section 5.1.
3.3 Probabllity-approximiae Generation
Following on- on-policy retrieval, we obtain the retrieved set {dtx for each query, denoted as Dz. The goal of the LLM forward step is divide this set into positive pool Dij and negative pool Dj- based on the RAG outcome. Below, we outline the key challenges involved in this step.
Challenge of Online Autoregressive Generation. During training, the online generation process G computationally expensive and slow due to the overhead of autoregressive decoding.
Solution. While a promising solution is to generate responses on-the-fly and verify them with either coman-in-thee loop in the loop [56] larger proprietary LLM as a judge [57, 58], this approach incurs substantial computational cost, potentially slowing down training and reducing the accessibility of our method. a more practical solution, we pre-compute a probability threshold offline and use it to approximate autoregressive generation online. This relies on the weak assumption that higher conditional probability Ply x) corresponds to a greater likelihood of generating y, where is the input the LLM. Further discussion on this assumption is provided in Appendix F. Below, we present the details of our solution.
Offline Computation of Probability Threshold. We follow the standard RAG pipeline described in Section 3.1. For each training query q. we retrieve the top-100 documents, denoted the initial retrieved set Dg. For each d: E Dy, we compute the joint probability P(y 2) of generating the ground truth response y given the input prompt x, where x consists of the query, one retrieved document, and the instruction.
Prompt (g,
Ply q, q. = Pw x)
II Vt.
Po(4 (c),))
We then generate responses ÿ = (xx). Based on whether ÿ passes the evaluation, we partition the initial retrieved set Dg into positive pool D and negative pool Dj. We i 100 and discard any query for which either pool is empty. We define the probability thresholds for positives and negatives as T and T- respectively.
m PM q, d.d) Ad E D;
To (Py) q, d) Ad E )
Online Identification of Positives and Negatives. During online training, given a query g. we retrieve the top-k documents as the in-training retrieved set D. For each d; E D.y we compute Ply q, de) and elassify it positive negative based on the offline thresholds.
D D Discarded,
if P(y q, d T if P( q, d.) <
d,
else
In plain terms, if using de leads to a higher probability of generating y than any d E D then d, is considered positive. Conversely, if its probability is lower than any d D then d, is considered negative. Otherwise, d, is discarded. If D or D++ is empty, we fall back to the initial set Dj or D
Sampling. In practice, we iterate through the retrieved documents in descending order of retrieval relevance. For each document d, we apply the LLM forward pass to classify as positive, negative, or discard it. This process continues until the first negative document is identified, which we designate the hard negative d- We then randomly sample one positive document d' from D If either

---

dt or d' is unavailable, we fall back to sampling from the initial pool Djj or D respectively. We evaluate alternative sampling strategies Appendix E and find that this strategy consistently achieves the best performance.
Other Details. For closed-set tasks where the response is single token, we directly compare next-token probabilities to classify dz. the gold choice token has a higher probability than other options, d is considered as positive; otherwise, negative, For queries q with multiple gold responses y, each y is treated as a separate entry. If d, succeeds on at least one, it is considered positive; otherwise negative. To avoid redundant computation, we cache all Pyy q, d,) for reuse.
3.4 Reinforeed Contrastive Learning
Following on-policy retrieval and retrieved positive document d' and negative documents d, for the training query q. In the final step. we apply contrastive learning to these pairs.
generation, we obtain the in-training
Our training objective follows SIDR to preserve its compatibility with late parametric retrieval (i.e., retrieving documents from a bag-of-tokens index and then re-ranking). Given a batch B of N samples, each sample contains a query gis a positive document di and a negative document di The semi-parametric contrastive loss is defined as:
elat
Mg Enog Evser (l
log
E
VgeB ell mn)
doo.
The final loss integrates contrastive loss of both parametric and semi-parametric components:
Lyam (g. d) L(Ex Ec()())
Lsemi (g, d) = L(Ea L(d)()),T(d)/ + L(T(g), Es(d))
Tiiaa d) = [upur(g d) + T Pana (g. d)
4 Experiments
4.1
Experimental Setup
Tasks and Datasets. We evaluate R3 five public RAG benchmarks. For free-form generation, we utilize Natural Questions (NQ; Kwiatkowski et al., 2019), TriviaQA (TQA; Joshi et al., 2017), and HotpotQA [60], three well-established open-domain QA datasets. For closed-set generation, employ the PubHealth [61] dataset for fact-checking tasks, and the ARC-Challenge [62] dataset for multiple-choice reasoning. More details can be found in Appendix A.
We exclude long-form generation datasets as we use the probability of continuation to approximate RAG performance, which may not align well with such tasks. Additionally, certain datasets, such as PopQA [63], which only offer a test split, are also excluded.
Evaluation Metrics. Following previous works [64, 63], we use accuracy on the test set as our evaluation metrie. the traditional IR setting, accuracy reflects whether the retrieved documents contain the answer, while in the RAG setting, it is determined by whether the generated output contains the answer. Since our training uses only one document context, whereas prior work typically uses ten, we report accuracy under both 1-shot and 10-shot settings for comparison.
Implementation Details. Our RAG system employs the LLM Llama3-8b [65] with the retriever SIDRMS [55] that trained on MS MARCO dataset [30]. For experiments on NQ, since certain baselines are trained on its training split, we additionally train Ms on NQ before applying our method. By defualt, we use the same English Wikipedia datastore and prompt as those open- sourced by SELF-RAG, detailed in Appendix K. For HotpotQA, we use the official datastore provided with the dataset. During training, we train the retriever for each dataset for 80 epochs, aligning with the training duration used for SIDRMS. We use a batch size of 128 and an AdamW optimizer [66] with a learning rate of 2 x 10-5, The training process is divided into two phases: the first half involves a warm-up phase using initial retrieved positives and negatives, while the second half transitions to in traininng retrieval, using the in in-training positives and negatives. During inference, we set the
7

---

maximum number of generated token to be 100 for free-form generation while 20 for closed-set generation. Our experiments are conducted with NVIDIA GPUs. Both offline RAG preparation and online RAG training take less than one day, depending on the number of queries in the datasets. We leverage vL.LM [67] to accelerate offline generation.
Baselines. We consider the below baselines: (1) Standard RAG: RAG frameworks using Llama3-8B and SOTA retrievers E5 [51] and CONTRIEVERMS [49]. (2) Improving IR for RAG: RAG frame- works with tuning retriever or advanced retrieval strategies enhanced by LLM. Strategy-based methods include ADAPTIVE-RAN RAGG [33] and IR- TR-COT [35], which adapt retrieval based on question complexity or LLM reasoning, as well as DRAGIN [31], FLARE [32], and SEAKR [34], which dynamically retrieve or selectively integrate external information guided by LLMs. For learning- based method, we compare against REPLUG [40], which tunes only the retriever while frozen the L.M. (3) Improving LLM for RAG: RAG frameworks that tune the L.L,, typically requiring more computation than tuning the retriever. We compare with SELF-RAN AG [64], which trains the LLM to decide when to retrieve and to self-verify its answers, and with RA-DIT [41], which jointly tunes both the retriever and the LLM. We include LLM-tuning methods in the comparison to illustrate the rela- tive improvement from tuning the retriever. (4) Transferring to other LLMs: We compare the RAG framework using different LLMs, such as Llamaa--Instru bestral-Instructy [69], along with Ms before and after tuning. This setup is designed to evaluate whether the learned in-context relevance transfers across different LLMs. Further model details are provided in Appendix B.
[65],
sa [68],
4.2 Main Results
Table 1: Main results of R3 and other RAG basclines. Bold: best RAG method by only improving IR. A: improvement or decline; baseline for below methods to compare; f: reproduction by other
works; I: our reproduction; LLM-enhanced retrieval process.
Task Dype
Free-Sras
Cloooksct
Dutaset (-))
TrisisQA
HotpotQA
PubBealth
Mathol (1) Metric (-9)
I-bbtt
L-bbst
10-abet
L-bbt
H-bbot
10abst
Lobhee
104hat
Standard RAG
Limerha 31D8
42.2
99.1
64.
Llbmn3a CONTARIVENN
3655
02.9
Improning
Anorrtve RAGisoon- (15-0000
DRAGIN tehdees (15-bbo)
48.0
SEASKRu (15-bbe)
40.5
-1.
+3.3
42.4
+3.3
FLARR
(15-bbo) [31)
45.0
+3.9
64.8
+3.3
37.2
-1.
RRPP Iife uenn (Soboo [7]]
41.7
-33
40.3
-103
R3
1o0 (Ouss
+5.66
48.1
+6.1
+3.8
+3.
47.1
+8.0
16.0
39.4
Repirstig ELM for RMG
Llann lesoncies S:DR
47.2
(5.0)
54.7
+1266
65.2
+3.2
71.3
+10
(9.9
67.2
13.7
71.8
+6.9
72.3
1-5.2)
75.4
+18.0
SELF-KAGuuai [b]] "C
+3.
61.3
SSuLF-RAGGuaa
15m AG SIDR RA. DiTarwao (5-sbo((-1
16.
18.
51.0
+0.7
640
43.9
75.1
112.6
40.7
[1.66
Transfering R3 other LLM
1. fearutt + SIDR
65.2
12.7
65.2
71.8
75.4
1 1ome Seeraaaeeraaaaa R3
45.3
+3.6
16.5
-38
65.
-0.4
73.8
+0.5
417
61.
-3.
66.1
-5T
71.8
41.5
75.1
-03
35-3 niti 4k di-asinsuus S:DR
46.1
50.7
A
64.6
A
59.2
A
46.1
46.2
A
57,6
84.5
54.7
Ite- do-aanctia
46.5
12.4
-1.
65.
-1.0
+1.
457
-1.
-3.0
$44
88
-0.
812
Minal-lsnaot SIDR
43.9
48.5
71.2
Table 1 presents results of R3 and other RAG methods, with key findings summarized below:
Reinforced contrastive learning effectively improves the retriever within RAG environments. Unlike CONTRIEVERNS and E5, which rely on extensive pre-training and SFT, R3 tunes SIDR with only four GPUs in one day. This efficient setup yields an average improvement of 5.2% over the original SIDRMS and outperforms other SOTA retrievers by 4.9% under the 1-shot setting. PubHealth, the gain reaches 6%, which even instruction-tuner demonstrate that learning retrieval tailored to the specific RAG environment in an RL manner is more effective than SFT with massive paired data that is unaware of the RAG environment. In Appendix I, we show that improving the retriever for RAG can degrade its performance in traditional IR settings, highlighting the gap between IR and RAG.
LLMs fail to match. These results
8

---

Tuning a small retriever can yield improvements on par with tuning a LLM. Reproductions of RALF from other works [72, [1] and our own experiments have shown inconsistent gains. This suggests that, despite substantial training costs, improving RAG through LLM tuning requires heavy customization and lacks generalizability. In contrast, tuning a smaller retriever can achieve comparable or even superior results to RAG-oriented or instruction-tuned 8B LLMs on certain datasets. Importantly, improving retriever complements improving LLM without conflict, offering a more efficient and controllable path for improving RAG systems.
Learned relevance can be transferred other LLMs across domains but not across tasks. Our results show that R3, trained with Llama3-8B, also improves performance when paired with other LLMs, such as across different benchmarks. However, this transferability does not consistently extend to non non-QA tasks. Given that the original S1DR is trained on QA dataset, the current observations suggest that the learned relevance to be robust for transfer across domains within the same task, but not across tasks. This also reflects a broader phenomenon: while different L.MMs may require similar documents for general QA tasks, these requirements diverge significantly for more complex tasks, such as fact-checking and complex reasoning. This highlights the importance of learning retrieval tailored to the specific RAG environment. Rather than viewing this as limitation, we see it as evidence that R3 is capable of capturing environment-speecin relevance.
1iii--3-mii-444-nsstee
and Mistral-Instruct on QA tasks
5
Analysis
5.1
Ablation Study
Compared to prior works, our main differences include (i) construct constrastive on-the-fly (ii) employing contrastive objective instead of KL divergence objective, and (iii) using late parametric to avoid periodic re-indexing. We systematically analyze these factors in this section. Note that our ablation experiments on NQ are conducted with the initial retriever SIDRMs. and thus differs from the main experiment setup.
Contrastive Learning versus Reinforeed Contrastive Learning. In the warm-up stage, we use documents retrieved offline from the initial retrieval pool. During the training stage, we apply RCL using documents retrieved online with the current parameters (i.e., on-policy). We conduct an ablation study on NQ and PubHealth under three settings: [offline-only] uses only the offline retrieved pool offline, which can be considered as conventional CL; [online-only] skips the warm-up and relies solely on online on-policy retrieval; our full method [offline+ +online] uses both offline warm-up up and online on- on-policy retrieval, which refers to RCL with both positive and negative pools guaranteed.
As shown in Figure 4, using either /ofsline-only] or (online-only] alone results in suboptimal perfor- mance, while [ofllinetonline] leads to the best results. These observations demonstrate that both off-policy contrastive labels obtained through conventional CL and on-policy contrastive labels identi- fied via RCL during training can improve the retriever. However, combining both approaches delivers the best results. This supports findings from our prior work [55], which show that a warm-up phase with static contrastive labels helps the retriever initially align with environment-specn -specifc-- relevance, and on- on-policy retrieval further refines it by continuously mining more effective contrastive labels during training. thereby deepening the retriever's 's alignment with the environment.
Contrastive Objective versus KL-Divergence Objective. We also evaluate variant of our method that replaces contrastive learning objective with KL divergence, denoted as [offline+ KL)J. KL divergence has been adopted in prior work [40, 2] to align query-document relevance with generation likelihood. Our results show that while KI. divergence yields initial improvements, these gains quickly platcau and remain consistently lower than those achieved by our contrastive approach. KL-based alignment is easier to implement but provides more rigid and less adaptive learning signal. In contrast, contrastive learning continuously supplies effective positives and negatives based on the current retriever state, enabling ongoing improvement throughout training.
We believe KL divergence underperforms in this context because the generation likelihood distribution of LLMs is inherently hard to capture by bi-encoder. Many studies in knowledge distillation [73, 74] have attempted to use small bi-encoders to approximate fine-grained ranking signals, typically provided by cross-encoders or LLMs,, but have consistently found that such models struggle to capture these signals effectively. This limitation has led to the widespread adoption of the retrieve- then-rerank paradigm in both academic and industrial settings.
9

---

PubHlealth
NQ
PubHealth
39
69
68
Accurscy
38
Accunacy
Accuracy
tea mirk
Ascuraae
37
66
leto sam paras
36-
Offinns
35
Online only
Office Online
Ortri
64
37
65
50
0
75
100
50
Triiiing Progress
Training Progress 19
Training Progress
Training Progress
Figure Ablation of offline and online learning-
Figure Ablation on re-indexing strategies.
Late Parametric versus Periodic Re-indexing, Another key distinction between our method and prior approaches lies in how we address index staleness. While previous works periodically rebuild the index during training, we adopt a late parametric mechanism from SIDR to avoid frequent re-indexing. Below, we compare these strategies for handling index staleness. /R3 (w/o re-rank)] refers to performing retrieval directly on the bag-of-tokens index without subsequent re-ranking. This reduces computational cost but results in suboptimal retrieval accuracy. (R3 (w/o semi-para)] denotes the removal of the entire cemi-parametric design, falling back periodic re-indexing as done in prior work. In our setup, the index is rebuilt three times during training. More details are provided in Appendix H.
As shown in Figure 5, our results indicate that while periodic re-indexing R3 (w/o semi-para) provides some improvement, it is significantly less effective than the semi-parametric design. The late parametric approach not only yields better performance but also reduces training cost and simplifies implementation. Compared to direct retrieval on the bag-of-tokens index, the late parametric design provides higher-qualij, positive and negative examples to improve. This advantage is particularly evident on the PubHealth dataset, where R3 shows much larger improvement over R3 (w/o re-rank).
5.2
Cost-Effectiveness Analysis
The main training cost comes from LLM forward passes to compute probability P(y | q, d) for previously unseen documents. In Table 2, we report the number of such documents processed during training. Each query involves between 14 and 128 unseen documents, depending on the task. We observe positive correlation between the number of processed documents and the performance gain of R3. Notably, PubHealth encounters most unseen documents and yields the largest improvement, suggesting greater gap between the initial and learned retriever.
NQ
TriviaQA
PubHealth
ARC 14
nDoc Improv.
34
18
128
+5.6%
+3.8%
+6.0%
+2.3%
Table 2: Number of documents requiring probability computation and corresponding improvement.
Together with the transferability results in Table 1, this motivates further analysis. In Appendix G, we show that R3 learns I.LM-speccfic relevance on PubHealth. We find that transferability improves initially but then declines steadily during training, even RAG performance continues to rise. This suggests that the retriever becomes more specialized for the target LLM (Llama3-8B), reducing its generalization to other LLMs. This pattern reflects broader trend: while LLMs may share similar relevance preferences for general QA tasks, they might diverge in specialized domains and complex tasks, highlighting the need to tune retrievers for each specific RAG enviroment.
Conclusion
In this work, we demonstrate the relevance gap between IR and RAG. To address this gap, introduce R3, a RAG framework that optimizes retrieval for a given RAG environment RL manner. Our results show that RCL, which constructs on-policy contrastive signals on-the-fly, significantly improves a retriever within a specific RAG environment, outperforming of off-the-shed retrievers and LLM-augmente retrieval, while performing comparably to RAG systems with post-trained or instruction-tunae 8B LLMs. These findings highlight the substantial potential of approach to enhance RAG systems through effective retrieval learning.
10

---

Limitations
Our work explores optimizing retrieval for RAG in a reinforcement learning manner, but several limitations remain. First, the current experimental setup focuses on a vanilla RAG workflow, where only one document retrieved as context. We believe this idea can naturally extend to more complex workflows, as long as the online learning process remains computationally feasible. Second, the current reward function is constrained by its reliance on short answers and string matching. During training, we also observe that performance is sensitive to how positive and negative examples are defined. A more stable and generalizable reward or evaluation function, such as using an L.-M-as- a-Judge, could further improve reliability. Lastly, our current foundation model is based on BERT, which is relatively outdated. We believe that incorporating more advanced L.-M-based encoders and tuning them in eeramparaetttic manner could greatly broaden the applicability and fser-facing potential of this framework.
Broader Impact and Future Work
In this section, we discuss the broader societal impact of our work and its potential influence other areas and advanced applications.
RAG Application Safety. Our work could be deployed in complex environments where retrieval learned to affect Al behavior and safety. The retrieval module can learn to retrieve contextual information, few-shot examples, instructions, existing chain-of-thought prompts that guide LLMs toward more reliable outputs, or conversely be intentionally misused to generate harmful content, mislead users, or surface inappropriate material. We acknowledge this potential for malicious use and encourage responsible deployment and safeguard mechanisms in practice.
Scaling with Learned Sparity. The success of different areas can be unified under the paradigm of Scaling with Learned Sparsity. For example, LLM pre-training leverages Hixture-off-xperee (MoE) to scale up the parameter space, improving effectiveness while sparsely activating modules at inference time ensure efficiency. RAG can be viewed as scaling knowledge coverage by integrating external databases, while sparsely activating specific data partitions through the retrieval function. Similarly, line of sparse retrieval studies [75, 76, 77] can be regarded as scaling the representation space to enhance representational capacity and allivate the limitations [78] of embedding models, while sparsely activating dimensions to guarantee search efficiency.
MoE
RAG Inference Knowledge Space Data Partitions
Sparse Retrieval Representation Learning Representation Space
Area
Pre-training
Scaling
Parameter Space
Sparsely Activate
Neural Modules
Dimensions
Table 3: Overview of scaling with learned sparsity paradigms.
Looking forward, we believe that future AI systems will inevitably require learning to scale along certain axes while simultancously learning sparsity over them. Our two works, SIDR and R3, demonstrate how to learn sparsity and how such sparsity can further be utilized to improve the large-scale complex systems within which they operate, offering preliminary insights in this direction.
Multi-modal RAG. Beyond document retrieval, emerging AI search systems increasingly involve multimodal data such as images and videos. Unlike query-document pairs, where lexical relevance can often serve as a reasonable heuristic, constructing effective relevance pairs for cross-modal data is far more challenging due to the high cost of annotation and the lack of explicit alignment signals. This direction highlights opportunitics for framework that can automatically obtain contrastive labels through interaction or feedback signals, principle that aligns with the learning dynamics explored in our work.
Searching Tools, Skills, MCP, and Agents. Recent progress in agentic frameworks, where AI agents learn to call tools, skills, MCP services, or interact with other agents, has largely relied on online reinforcement learning to manage small pool of available operations. As the pool of callable components continues to grow, learning effective representations of these components for retrieval will become crucial for scalable and efficient decision-making. In such settings, the proposed reinforced contrastive learning offers one possible direction for enabling search and coordination across a large pool of tools, skills, and agents.

---

Acknowledgments
Lei Chen's work is partially supported by National Key Research and Development Program of China Grant No. 2023YFF0725100, National Science Foundation of China (NSFC) under Grant No. U22B2060, Guangdong-Hong Kong Technology Innovation Joint Funding Scheme Project No. 2024A0505040012 2, the Hong Kong RGC GRF Project 1621 (623020, RIF Project 16020-19, AOE Project AoE/E-603/18, Theme- based project TRS T41-603/20R, CRF Project C2004-21G, Key Areas Special Project of Guangdong Provincial Universities 2024ZDZXX006. Guangdong Province Science and Technology Plan Project 2023A0505030011 Guangzhou municipality big data intelligence key lab, 2023A0300012, Hong Kong ITC ITF grants MHX/078/21 and PRP//04/22FX, Zhujiang scholar program 2021JC02X170 Microsoft Research Asia Collaborative Research Grant, HKUST- Webank joint research lab and 2023 HKUST Shenzhen- Hong Kong Collaborative Innovation Institute Green Sustainability Special Fund from Shui On Xintiandi and the InnoSpace GBA.
References
[1] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Retreeval-augmenned generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9559-94999955 2020.
[2] Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In International conference on machine learning. pages 9929-3938. PMLR, 2020.
[3] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Retrieval-augne -uggentted generation for large language models: A survey, arXiv preprint arXiv:
2023.
[4] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint 1.Xiv:2303.18223,, 2023.
[5] Christopher Manning. An introduction to information retrieval. Cambridge university press, 2009.
[6] Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. Agentic retrieval- -auievet--ugmentedd generation: A survey on agentic rag. arXiv preprint arXiv:2501, 09136, 2025.
[7] Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, al. Webagent-rl: Training web agents via end-to-end multi-turn reinforcement learning, arxiv preprint arXiv:2505. 16421, 2025.
[8] Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, et al. Os agents: A survey on milm-based agents for computer, phone and browser use, 2024.
[9] Le Huang, Hengzhi Lan, Zijun Sun, Chuan Shi, and Ting Bai. Emotional rag: Enhancing role-playing agents through emotional retrieval. In 2024 IEEE International Conference on Knowledge Graph (ICKG), pages 120-127. IEEE, 2024.
[10] Zihan Liu, Wei Ping, Rajarshi Roy. Peng Xu, Chankyu Lee, Mohammad Shoeybi, and Bryan Catanzaro. Chaiqa: Surpassing gpt-4 on conversational qa and rag. Advances in Neural Information Processing Systems, 37: 04155416660 5416-15459, 2024.
[11] Mandar Kulkarni, Praveen Tangarajan, Kyung Kim, and Anusua Trivedi. Reinforcement learning for optimizing rag for domain chatbots. arXiv preprint arXiv:2401.0688 06800, 2024.
[12] Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning real-world environments. arXiv preprint ar XiXx::2504.03166 2025.
12

---

[13] Zile Qiao, Guoxin Chen, Xuanzhong Chen, Donglei Yu, Wenbiao Yin, Xinyu Wang, Zhen Zhang, Baixuan Li, Huifeng Yin, Kuan Li, et al. Webresearcher: Unleashing unbounded reasoning capability in long-horizon agents. arXiv preprini arXiv:2509. 13309, 2025.
[14] Zijian Li, Xin Guan, Bo Zhang, Shen Huang. Houquan Zhou, Shaopeng Lai, Ming Yan, Yong Jiang, Pengjun Xie, Fei Huang, al. Webweaver: Structuring web-scale evidence with dynamic outlines for open-ended deep research. arXiv preprint arXiv:2509. 13312, 2025.
[15] Hang Li, Jun Xu, et al. Semantic matching in search. Foundations and Trends in Information Retrieval, 7(5):343-485) 469, 2014.
[16] Avishek Anand, Lijun Lyu, Maximilian Idahl, Yumeng Wang, Jonas Wallat, and Zijian Zhang. Explainable information retrieval: A survey. arXiv preprini arXiv:2211.02405, 2022.
[17] Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, and Jingren Zhou Zerosearch: Incentivize the search capability of Ilms without searching. arXiv preprint ar0iv: 045588 2025.
[18] Siye Wu, Jian Xie, Jiangjie Chen, Tinghui Zhu, Kai Zhang, and Yanghua Xiao. How easily do irrelevant inputs skew the responses of large language models? arXiv preprint arXiv:2404.033: 033302. 2024.
[19] Viktor vorktsri,,k Maria Lysyuk, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, and Alexander Panchenko. Adaptive retrieval without self-knowledge? bringing uncertainty back home. arXiv preprini arXiv:2501. 12835, 2025.
[20] Yuhao Wang. Ruiyang Ren, Junyi Li, Wayne Xin Zhao, Jing Liu, and Ji-Rong Wen. Rear: relevance-aware retrievalaaugmented framework for open-domain question answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5613-5626, 2024.
[21] Sunhao Dai, Yuqi Zhou, Liang Pang. Weihao Liu, Xiaolin Hu, Yong Liu, Xiao Zhang, and Jun Xu. Lims may dominate information access: Neural retrievers are biased towards llm-generated texts. arXiv preprint arXiv:2310.20501, 2023.
[22] Linda Zeng, Rithwik Gupta, Divij Motwani, Diji Yang, and Yi Zhang. Worse than zero-shot? fact-checking dataset for evaluating the robustness of rag against misleading retrievals. arXiv preprint 21X1::2502.16101 2025.
[23] Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. The power of noise: Redefining retrieval for rag systems. In Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval, pages 719-729, 2024.
[24] Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, Fabrizio Silvestri, et al. Rethinking relevance: How noise and distractors impact sirieval-lugmeeeted generation. In CEUR WORKSHOP PROCEEDINGS, volume 3802, pages 95-98. CEUR-WS, 2024.
[25] Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schul- man, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, et al. Reasoning models don't t always say what they think. arXiv preprint arXiv:2505.05410. 2025.
[26] Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity. arXiv preprint arXiv:2506. 06941, 2025.
[27] Stephen Robertson, Hugo Zaragoza, al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends@ in Information Retrieval, 3(4))333-389, 2009.
[28] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 6769-6676669 6781, 2020.
13

---

[29] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453 2019.
[30] Payal Bajaj. Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint 161 1.0.1....966.. 2016.
[31] Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. Dragin: Dynamic retrieval augmented generation based on the information needs of large language models. arXiv preprint vrXiv:2403.1008/ 2024.
[32] Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. Active retrieval augmented generation. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969-7992, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18865/11/2255 eem.pp https:/ ac Lantho org/ 2023. emn monnlp-maan main inaainn.4 495. http:////llaathhoooooooooo
emnlp-main.4995 main.495. URL
[33] Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong C Park. Adaptive-rag: Learning adapt sttrieval-augmmnte large language models through question complexity. arXiv preprint arXiv:2403. 144033 2024.
[34] Zijun Yao, Weijian Qi, Liangming Pan, Shulin Cao, Linmei Hu, Weichuan Liu, Lei Hou, and Juanzi Li. Seakr: Self-aware knowledge retrieval for adaptive retrieval augmented generation. arXiv preprint arXiv:2406. /92/5, 2024.
[35] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain thhthught reasoning for knowledge-intensiv multi-step questions. arXiv preprint 1r 10509, 2022.
[36] Ran Xu, Wenqi Shi, Yuchen Zhuang, Yue Yu, Joyce C Ho, Haoyu Wang, and Carl Yang. Collab- rag: Boosting serieval-augmenter generation for complex question answering via white-box and black box Ilm collaboration. arXiv preprint 0rxXiv25504.04155 2025.
[37] Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, and Jie Fu, Rq-rag: Learning to refine queries for retrieval augmented generation. arXiv preprint 2rXiv:2404.00610, 2024.
[38] Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251): 1-43, 2023.
[39] Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Bean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages
PMLR, 2022.
[40] Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Replug: darieval-auumeett black-box language models. arXiv preprint arXiv: 12652, 2023.
[41] Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Richard James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, et al. Ra-dit: Retrieval-augmenn dual instruction tuning. In The Twelfth International Conference on Learning Representations, 2023.
[42] Jianhao Yuan, Shuyang Sun, Daniel Omeiza, Bo Zhao, Paul Newman, Lars Kunze, and Matthew Gadd. Rag-driver: Generalisable driving explanations with retrieval-augmented in-context learning in multi-modal large language model. arXiv preprint 2rXiv:2402.10822 2024.
14

---

[43] Navve Wasserman, Roi Pony, Oshri Naparstek, Adi Raz Goldfarb, Eli Schwartz, Udi Barzelay. and Leonid Karlinsky, mmal-m--rag: A real-world multi-modal retrieval benchmark. arXiv preprini arXiv:2502. 12342, 2025.
[44] Guozhao Mo, Wenliang Zhong. Jiawei Chen, Xuanang Chen, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han, and Le Sun. Livemephench: Can agents navigate ocean of mep tools? arXiv preprint arxiv 780, 2025.
[45] Quanting Xie, So Yeon Min, Pengliang Ji, Yue Yang, Tianyi Zhang, Kedi Xu, Aarav Bajaj, Rus- lan Salakhutdinov. Matthew Johnson- Johnson-Roberson and Yonatan Bisk. Embodied-rag: General non- parametric embodied memory for retrieval and generation. arXiv preprint arXiv:2409.18313, 2024.
[46] Weiye Xu, Min Wang, Wengang Zhou, and Houqiang Li. P-rag: Progressive retrieval aug- mented generation for planning embodied everyday task. In Proceedings of the 32nd ACM International Conference on Mullimedia, pages 6969-6978, 2024.
[47] Phuc H Le-Khac, Graham Healy, and Alan F Smeaton. Contrastive representation learning: framework and review. Ieee Access, 8: 1193997-113934, 2020.
[48] Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. A survey contrastive self-supervvsed learning. Technologies, 9(1):2, 2020.
[49] Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Towards unsupervised dense information retrieval with contrastive learning. arXiv preprint 1rXiv:2112.09116 2021.
[50] Jiawei Zhou, Xiaoguang Li, Lifeng Shang. Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, Hao Jiang, Zhao Cao, Fan Yu, et al. Hyperlink-induced pre-training for passage retrieval in open- domain question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1135-7146, 2022.
[51] Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXhv:2212.035::
2022.
[52] Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang. Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025.
Junya
[53] Liang Wang, Nan Yang. Xiaolong Huang, Linjun Yang. Rangan Majumder, and Furu Wei. Multilingual e5 text embeddings: A technical report. arXiv preprint arXi::2402.05672, 2024.
[54] Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv preprint arXiv:2007.00808, 2020.
[55] Jiawei Zhou, Li Dong, Furu Wei, and Lei Chen. Semi- Semi-parametric retrieval via binary token index. arXiv preprint arXiv:2405.0192. 2024.
[56] Yu Bai, Yukai Miao, Li Chen, Dawei Wang, Dan Li, Yanyu Ren, Hongtao Xie, Ce Yang, and Xuhui Cai. Pistis-rag: Enhancing retrieval-augmeented generation with human feedback. arXiv preprint arXiv:2407.00072. 2024.
[57] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[58] Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge. arXiv preprint a55552241.15554, 2024.
15

---

[59] Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), pages 1601-1611, 1, 2017.
[60] Zhilin Yang, Peng Qi. Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhut- dinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:/809.0000 180900960,, 2018.
[61] Neema Kotonya and Francesca Toni. Explainable automated fact-checking for public health claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7740-7754, 2020.
[62] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.054577 2018.
[63] Alex Troy Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non non-parametric memories. In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023.
[64] Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511, 2023.
[65] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman. Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
[66] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2018
[67] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng. Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[68] Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et A highly capable language model locally on your phone. arXiv preprint 240.:22444.42119 2024.
al. Phi-3 technical report:
[69] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06822, 2023.
[70] Shengbin Yue, Siyuan Wang, Wei Chen, Xuanjing Huang, and Zhongyu Wei. Synergistic multi-agent framework with trajectory learning for knowledge-intend intensivve tasks. arXiv preprint arXiv:2407.09893. 2024.
[71] Zilong Wang, Zifeng Wang, Long Le, Huaixiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, et al. Speculative rag: Enhancing retrieval augmented generation through drafting. arXiv preprint arXiv:2407.0822.
2024.
[72] Xuanwang Zhang, Yun-Ze Song, Yidong Wang, Shuyun Tang, Xinfeng Li, Zhengran Zeng, Zhen Wu, Wei Ye, Wenyuan Xu, Yue Zhang, et al. Raglab: A modular and research-oriented unified framework for retrieval-augmentt generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 408-418, 2024.
[73] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: survey. International Journal of Computer Vision, 129:1789-1819, 2021.
16

---

[74] Gautier Izacard and Edouard Grave. answering. arXiv preprint arXiv:2012.04584, 2020.
Distilling knowledge from reader to retriever for question
[75] Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. Splade: Sparse lexical and expansion model for first stage ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2288-2292, 2021.
[76] Yang Bai, Xiaoguang Li, Gang Wang, Chaoliang Zhang, Lifeng Shang, Jun Xu, Zhaowei Wang. Fangshan Wang, and Qun Liu. Sparterm: Learning term-based sparse representation for fast text retrieval. arXiv preprint arXiv:2010.00766 2020.
[77] Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Xiaolong Huang, Binxing Jiao. Linjun Yang, and Daxin Jiang. Lexmae: preprint arXiv: 147544 2022.
pretraining for large-scale retrieval. arXiv
[78] Orion Weller, Michael Boratko, Iftekhar Naim, and Jinhyuk Lee. On the theoretical limitations of embedding-based retrieval. arXiv preprint 2xXix:2508.2/066 2025.
[79] Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. Cenet: Extracting high quality monolingual datasets from web crawl data. arXiv preprint arXiv: /91 0911991110099.. 2019.
[80] Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume (Long and Short Papers), pages 4171 1177-4186, 2019.
[81] Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, and Lei Chen. Retrieval- based disentangled representation learning with natural language supervision. arXiv preprint 2022.
2rXiv:2212.07660
[82] Jinming Nian, Zhiyuan Peng, Qifan Wang, and Yi Fang. W-rag: Weakly supervised dense retrieval in rag for open-domain question answering. arXiv preprint arXiv:2408.. 08444, 2024.
[83] Zixuan Ke, Weize Kong, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. Bridging the preference gap between retrievers and LLMs. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10438- 10451, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 1/02011.........lgg.....o URL https: / ac aocclan lan anthoooogy or oogoorggg////oo 2024. acl- 562.
[84] Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with retrieval-augmentee generation in llms. arXiv preprint arXiv:2407.02485, 2024.
A Details of Datasets
We present details of datasets follows.
Natural Questions (NQ; Kwiatkowski et al., 2019) is a widely used open-domain QA dataset constructed from Wikipedia. The questions originate from Google search queries, and the answers are text spans within Wikipedia passages. This dataset consists of queries with one or more answer strings, requiring RAG systems to generate responses based factual knowledge.
TriviaQA (TQA; Joshi et al., 2017) is a challenging QA dataset that comprises question-answer pairs curated by trivia enthusiasts along with independently gathered evidence documents.
HotpotQA [60] is multi-hop question answering dataset that requires reasoning over multiple Wikipedia paragraphs to answer factoid questions.
PubHealth [61] is fact-checking task that focuses on verifying health claims across variety of biomedical topics.
ARC-Challenge [62] is multiple-choice reasoning dataset consisting of science exam questions for grades 3 to 9,
17

---

B Details of Models
B.1 Retrieval Model (IR)
ES [51] is state-of-the-art dense retriever that pre-trained millions of weakly related text pairs from the Web. The unsupervised version of this model is denoted as ES-unsup. This model undergoes further fine-tuning natural language inference (NLJ) datasets, as well as the Natural Questions and MS MARCO datasets, to enhance its capabilities in downstream applications. The fine-tuned version is denoted ES.
widely-used dense retriever pre-trained unsupervised on Wikipedia data and CCNet [79]. The unsupervised version of this model is denoted as CONTRIEVER, It is further fine-tuned on the MS MARCO dataset to enhance its retrieval performance, with the fine-tuned version denoted as CONTRIEVERMS.
CONTRIEVER [49]
DPR [28] is widely used dense passage retriever initialized with BERT-based uncased en- coder [80], and fine-tuned on downstream dataset. Specifically, DPRMs fine-tuned on the MS MARCO dataset, DPRNo the NQ dataset, and DPRTQA the TriviaQA dataset.
SIDR [55] is semi- parametriaaac sparse retriever that supports using both embeddings and tokeniza- tion as index. This nature allows for in in-training retrieval, where the model's 's parameters dynamically update while the retrieval index remains fixed. The model is initialized with a BERT-based uncased encoder [80] and fine-tuned exclusively on single dataset depending on the variant: SIDRMs is fine-tuned on the MS MARCO dataset, SIDRNo on the NQ dataset, and SIDRTQA on the TriviaQA dataset.
All the above retrieval methods initialized with BERT-based encoder, which contains approxi- mately 200 million (0.2B) parameters.
B.2 Large Language Model (LLM)
Llama3gs [65] is variant of the latest Llama3 model series with billion parameters.
LlamaS-Instructeg [65] builds upon the Llama3ss by undergoing a post-training stage in which the model is specifically tuned to follow instructions and align with human preferences improve specific capabilities.
Phi-3-mini-4k -iinii--kkkisttuttsssu [68] is a lightweight widely-used LLM with 3.8 billion parameters, trained on the Phi-3 dataset featuring synthetic and high-quality filtered web data, focused on reasoning and quality.
Mistral-Instructra [69]. We use Mistral-7B-Insttuut--0.333 LLM which is an instruct fine-tuned version of the Mistral-7B-v0.3.
B.3 Relrieval-auummented Generation Framework (RAG)
ADAPTIVE-RO [33] RAG framework that adapts the retrieval strategy based on question complexity where simple questions are answered without retrieval and complex ones trigger multi-step process involving iterative interaction between the retriever and the LLM.
IR-CoT [35] is a RAG framework designed for multi-hop question answering. interleaves retrieval with intermediate steps in chain-of chain-of-though (CoT) reasoning.
FLARE [32] is RAG framework that enhances LLM performance by selectively integrating external knowledge. It monitors token-level generation probabilities to detect uncertainty and triggers retrieval when needed during generation.
DRAGIN [31] is RAG framework that, similar to FLARE, monitors token-level probabilities during generation to detect uncertainty.
SEAKR [34] is RAG framework that reduces hallucinations by leveraging self-aware uncertainty. It monitors the LLM's internal signals and triggers retrieval only when high uncertainty is detected during generation.
For the above four baselines, we adopt the reproductions and reported results from Moskvoretskii et al. [19].
18

---

REPLUG [40] is a RAG framework using GPT-3 and CONTRIEVER, The retriever is specifically trained to use the first 128 tokens of sequence as queries, with the goal of retrieving documents that maximize the probability of generating the subsequent 128 tokens when these retrieved documents are prepended the query.
SELF-RAG [64] is RAG framework designed to improve response quality by enabling on on-demand retrieval and incorporating self-reflection mechanisms.
The reproductions by Wang et [71] and Zhang al. [72], SELF-RA Raee-AAGMaa RAGMittarT and SELF-RAGuamas respectively, involve tuning Mistral-7B and Llama3-8B as base lan- guage models using the open-source data provided by SELF-RAG. Our reproduction. SEL.F-AAGLLama SIDRRMs utilizes the SELF-RAGLIame 8B checkpoint from Zhang et al. [72] LLM, while employing the same retriever SIDRMS and adapting to our downstream setup.
RA-DIT [41] a RAG framework that separately fine-tunes the LLM better utilize retrieved information and the retriever to align with the LLM's preferences.
C Revisiting Semi-parametric Disentangled Retriever (SIDR)
This section can be considered an expanded version of Section 3.2, where we discuss SIDR in more detail.
Intuition
Language ning
Morping (Dee ingrossnttaaee Sparse Eweeedinng (Mecasstary stnghh)
bild
held)
Viri GITE
Erazedding
(Trntformme Bocels
Onsapertied
Iequencce
2017
token
One Hit Retttsetaation
Inttaa Eirbeddnng offonosx
Final Fmmbedding
Probstillyowe over Vadiburrs
hest Mocoed
Afttn SERCsAnCE
offokers
Tokknnx
Tokes Prediton
Training
Algnn wign + query and passage representation during training
Disentangled Vocab Space
6 algi +
C030 Alg
(Sparse Embedding)
project
preject
project
Entangled Space (Dense Embedding)
Corsinat
Veeeeggeeeeee Disentangled Vocab Space (Tokenization)
0.00
0O + takeniie
tolkerize
tekerite
takeniz
takerize
Entangled Retriever
Disentangled Retriever
Semi parametric Disentangled Retriever (SIDR)
Retrieval Methed
Dense Retrieven:
Lexical Retriever
Inference (4 Options for SiDR)
One Toleenizrr Tower
ProJect
S1Rject
search
embod
Jokenize
Wilerize
tokenize
tokenize
Quary
Doc
Query
Embedded
Tokenized
Embedded Query
Tokenized Query
Doc
Parametric Search (Option 1)
Alpha Search (Option 2)
Beta Search
Beta Search Re-rank
[Option
(aka Late-Parametric (Option
Figure Overview illustration of SIDR.
Background: Challenge of Index Staleness. As is well known, the retrieval index is typically the embedding of the datastore, denoted (D). During training, as the Ra parameters update from 0 - 0', the index must be updated accordingly, from Ex(D) - For (D), where [D] > 1. Otherwise, the index becomes stale, leading to mismatches between the query and document
19

---

embeddings. However, frequently re-embedding large datastore is computationally expensive and often unaffordable in practice. Fundamentally, this challenge stems from the coupling between retrieval index and model parameters 0.
Intuition of SIDR. Intuitively, language model (L.M), together with its pre-training head, can be viewed as an auto-encoder. Let's use BERT as an example. Given an input sequence X, a specific token x can be represented as a one-hot vector T(x) E O, 1lV where [V] 30522 is the size of the BERT vocabulary V. These tokens are then processed by the embedding layer to yield d-dimensional hidden state (z) where d= 768. After passing through 12 transformer blocks, the token representations interact with the contextualized token representations, resulting in the final token representation from the last layer, denoted as Hn (z). The H, (z) of a masked token is then further processed by the masked language model head (MLMH) yield a probability distribution (xx) (0, I) over the vocabulary space, used to predict the masked token.
Given sequence X, SIDR embeds it into a -ildimnnsional sparse embedding. denoted as Eg (X) E (0, (0.+ SIDR builds upon a BERT-based model with three simple modifications to its pre- training head: (i) replacing the softmax activation with clulp: (ii) aggregating token representations via max pooling; and (iii) applying top-k sparsification.
Inspired by the auto auto-encoder property, we are curious whether output sparse embedding can be aligned with input tokenization to yield better features. Specifically, during training, SIDR applies contrastive learning across three channels: Es (q) with (d), T(q) with Eg Ed(d), and Eg(g) with (d)). In this training setup, we find that SIDR successfully uses Eg(g) to search over the bag-of-token index T(D), as illustrated in Figure 6 with Option 3 (Beta Search) and Option 4 (Beta Search + Re-rank). This provides an opportunity to use a non-parametric bag-of-token index as a frozen proxy for retrieval during the retriever's 's training loop, thereby bypassing the re-indexing issue.
On- On-policy Retrieval. During R3 training, we adopt the late parametric mechanism of SIDR (shwon as option 4), which first retrieves the top-m documents using the bag-of-tokens index (D), and then embeds the these documents for re-ranking. This process can be formulated as:
{d) m Re( T(D), m) {d}x= Roo (Fo()))) Eo ((d))m))
To reduce training costs, set k 20 in experiments.
Other Applications of SIDR. Besides using the bag-of-tokes index to enable on-policy training in R3 (i.e., (q) searches over 7 (D)), STDR offers additional advantages. For example, it facilitates conventional search, where the sparse embedding of the query searches over the sparse embeddings of the documents, enabling efficient search via inverted indexing and improving interpretability. Addi- tionally, can further reduce online query embedding time by using the bag-of-token representation of the query (i.e., T(g) searches over Eg (D)), a method known as alpha search [81].
D Preliminary Study
In Table 4, we report the performance of several off-the-shelf retrievers across multiple datasets, covering both IR and RAG scenarios. The datasets include QA benchmarks such as NQ and TQA, and non-QA benchmarks such as PubHealth and ARC, as detailed in Appendix A. The evaluated retrievers include the dense retriever DPR, the sparse retriever SIDR, and unsupervised models such as CONTRIEVER and E5-unsup, along with their supervised counterpart. Model details are provided in Appendix B. The evaluation metric is described in Section 4.1, where the IR measure evaluates whether the retrieved document contains the answer, and the RAG measure evaluates whether the retrieved document can prompt the LLM to generate the answer. We benchmark all models using SIDRMS as the base retriever, which also serves as the backbone of our proposed method. The results reveal two key findings.
Finding 1: Better IR performance does not always lead better RAG performance, especially when task shift occurs. For QA tasks, we observe that higher accuracy in IR settings generally correlates with better RAG performance. However, this trend does not hold for non-QA tasks. For example, on PubHealth, a vanilla DPRMs outperforms SOTA retriever in RAG settings, despite being weaker in IR metries. Similarly, on ARC, the unsupervised CONTRIEVER achieves the best RAG performance, outperforming its supervised counterparts.
20

---

Table 4: Accuracy in IR and RAG settings using Llama3-8b with top-1 retrieved document in in-tontext; Bold: best performance; A: improvement or decline compared to Ms:RMs: ÿ: has been trained in-domain.
(-)
NQ
TriviaQA
PubHealth ARC-C
Retriever (1)
IR
A
RAG
A
IR
RAG
A RAG
RAG
Unsupervised Pre-truining
Contriever
23.6
15.5
30.9
335
37.2
18.9
56.6
5.4
61.8
-1.7
58.6 +1.7
E5-unsup
62.9
Supervised on MSMARCO
DPRss
SrDRas 39.1
34.4
56.1
62.0
63.5
56.9
Supervised NQ
DPRsg [43.5
138.5
SiDRyg 349.5
+10.
$42.7
+8.3
47.4
8.7
59.8
2.2
63.5
57.1 +0.2
Supervised TQA
DPRay 32.1
7.0
32
155.4
0..7
161.1
+0.9
63.1
56.7
SiDRtga
30.6
8.5
32.9
-1.5
156.9
+0.8
163.6
+1.6
61.1
-2.4
58.6
+1.7
Pre-training + Supervised on Multiple Datasets
Contrievenus 41.5 +22.4
36.5 (2.11
53.5
-2.6
60.7
-1.3
63.1
-0.4
58.1
411.2
158.0
Upper-hound 65.9
77.6
78.5
80.3
92.1
71.5
Finding 2: Substantial room for improvement in retrieval within the RAG system. We find that although most required knowledge already exists the datastore, each retriever succeeds on only a subset of queries. For example, NQ. 77% of queries can be answered by at least retriever via RAG, yet the best single retriever covers only 43%. This gap reveals significant untapped potential in the datastore and underscores the need for stronger retrieval in RAG.
E
Sampling Strategies
We report the performance using SıDRMs as the initial retriever under the 1-shot setting. The sampling strategy plays crucial role in forming effective contrastive pairs during training. In this section, we compare our current sampling method against two alternatives:
Current (A): Most-relevant negative + randomly selected positive
Alternative (B): Randomly selected negative randomly selected positive
Alternative (C): Most-relevant negative + most-relevant positive
Table 5 reports the RAG performance on NQ and PubHealth under each strategy.
Table 5: Impact of sampling strategies on RAG performance using StDRMs as the initial retriever.
Strategy
NQ
PubHealth
39.8
69.5
B
38.7
68.7
39,2
69.0
Overall, the current strategy (A) achieves the best performance on both datasets. We hypothesize that incorporating the most-relevant negative increases training difficulty and encourages more discriminative learning, while introducing randomness in positive sampling promotes generalization. These findings consistent with observations from prior retrieval learning studies (e.g., DPR), where the choice hard negatives has a substantial impact on performance, whereas positive selection tends to be less critical.
Assumption on Probability and Generation
Given that our training relies on the conditional probability the ground-truth response y given input z, denoted as P(y 2), signal for identifying positive and negative documents, we now
21

---

Table 6: Results of RAG framework using top-1 and top- 10 documents in context, sorted by retrieval relevance and joint probability of responses.
Task Type (-)
Free-form
Closed-sct
Dataset (-)
NQ
TriviaQA
PubHealth
ARC-C
Method (1)
Metries (-+)
I-shot
10-shot
I-shot
10-shot
I-shot
10-shot
I-shot
10-shot
Liama3u StDRas (doc top relevance)
49.1
65,3
67.2
65.2
67.4
58.1
57.3
Llamady SiDRMs (doc with top Ply a²))
85.1
76.2
88.7
84.2
87.4
77.4
95.6
83.6
investigate whether higher values of P(y | I)) correlate with improved RAG performance (i.e., better generation of y)-
For each dataset, we sample 1,000 training instances. For each query, we retrieve the top 100 documents, and evaluate RAG performance using only the top-1 and top-10 documents, selected either by retrieval relevance or by P(y z). The results, presented in Table 6, show that P(y x) is predictive of final RAG accuracy. Moreover, the strong performance obtained when using documents ranked by P(y x) highlights untapped potential of the datastore, a potential that current retrieval methods fail fully leverage.
To our knowledge, using Ply x) to identify positives and negatives is a rough yet resource-efficient solution that could cover most existing knowledge-intensiee tasks, aligning with their evaluation metrics that often utilize string matching. However, may not be suitable for long-form generation, which requires different evaluation strategies. We believe it is possible customize the identification of positive and negative examples based on the specific needs of each task. Ideally, if computational cost is a concern resources are sufficient, a strong proprietary LLM like GPT-4 can be used for identification on-the-fly.
G LLM-Specific Relevance
In Table 2, we observe that PubHealth requires significantly more online probability computations and yields larger improvements compared to other datasets. Combined with the transferability results in Table 1, hypothesize that R3 may succeed in learning LLM-specinf relevance this task. While this enables strong task-specific performance, it also limits the retriever's generalization ability across LLMs. To further examine this hypothesis, we report both effectiveness and transferability metrics throughout the online training process of R3 on PubHealth.
Table 7: Effectiveness and transferability during different stages of online training on PubHealth.
On-policy Training Progress
0%
25%
50%
75%
100%
RAG Effectiveness
65.7
67.7
68.8
69.4
69.5
Transfer to -lama3-8B-Instree
66.7
67.1
67.0
66.1
65.2
Transfer -I-nstuet
49.8
50.4
48.2
47.7
46.7
We observe that while transferability initially increases, it steadily declines as training progresses, task-specific RAG performance continues improve. This trend indicates that the retriever learning relevance function increasingly specialized for the target L.M.M Although this improves retrieval quality for the specific LLM, it comes the cost of generalization to others. This reflects broader phenomenon: while LLMs may share similar relevance preferences for general tasks such as QA, their needs diverge more significantly in specialized domains like fact-checking. These results underscore the value of end-to-end retrieval learning for RAG.
H Late Parametric vs. Periodic Re-indexing
A key distinction between work and prior practices lies in use of the late parametric mechanism to avoid re- re-indexing during training. this section, we systematically evaluate these in-training retrieval approaches.
Baseline. We present ablation studies on different in-training retrieval approaches: (i) R3 employs the late parametric method proposed in SIDR, which uses a bag-of-token index for first-stage retrieval
22

---

and re-ranks the top-20 documents on-the-fly using up-to-date parameters. (ii) R3 (w/o re-rank) employs the bag-of-token index for retrieval, similar to the late parametric method but without the re-ranking process. This setup aims to assess the costs associated with re-ranking during training. (iii) R3 (w/o semi-para) involves periodic re-indexing using the most recently built but outdated index for retrieval, an in in-training retrieval method that commonly used in prior studies. In this setup, we employ MS as the initial retriever. We avoid using SIDRMs. MS. which has high-dimensional embeddings of 30,522, in stark contrast to DPR's s 768 dimensions. This significant discrepancy prevents our GPU cards from allocating the parametric index for SIDRMS, although they manage DPR effectively.
Training, All models undergo the similar training pipeline: they are trained for 80 epochs with the first 40 epochs as a warm-up and the last 40 conducting in-training retrieval. They differ only in their in- mn-training retrieval strategies: both R3 and R3 (w/o re-rank) do not require re- re-indexing; R3 (w/o semi-para) requires rebuilding index at every 15 epochs (around 5k steps), a rebuild interval commonly used in previous research [54], resulting in total of three rebuilds.
Results. We present the RAG accuracy on NQ and PubHealth test splits during in-training retrieval, with results reported every four epochs, as depicted in Figure 7. For the re-ranking setup, signifi- cant improvements are observed in the PubHealth data when re-ranking is employed, whereas the NQ dataset shows only minor improvements. Given that the costs associated with re-ranking are manageable in our setup, we continue to implement it. Regarding re-indexing, our findings indicate that despite requiring significant time and resources, it fails to yield improvements comparable to those of the late parametric approach and significantly lags behind. We attribute this to index staleness, where query embeddings must optimize against outdated document embeddings, rendering the learning process less effective. On the other hand, as presented in the study by Zhou et al. [55], by re-ranking the top-20 retrieved documents, the late parametric method can recover more than 90% of the performance full parametric search across different tasks, representing a minor compromise. This also partially explains why the late parametric approach outperforms periodic re-indexing.
PubHealth
Usit wit Open del winded esaats
29
Accurooy
30
37
0
60
100
40
60
80
100
Thiining Progress
Thininng Progress
Figure 7: RAG accuracy of difffrrent in-training retrieval approaches.
I Inconsistencies between IR and RAG Settings
I.1 Performance Changes in IR Scenarios after Tuning
Table 8: Performance changes before and after tuning the retriever using the R3 approach.
Dataset (-)
NQ
TriviaQA
Method (L) Metrics (-+)
IR
RAG
IR
RAG
Llama3u + SIDRMS
39.1
34.4
56.1
62.0
Llamašsu R3 (SIDRMs)
40.8 (+1.7)
39.8 (+5.4)
53.9 (-2.2)
65.8 (43.8)
Llama3s SIDRNo
49.5
42.7
Llama3ss R3 (SIDRNg)
47.1 (-2.4)
44.1 (+1.4)
We evaluate the performance of our retriever in both IR and RAG scenarios before and after tuning. Note that the retriever we use for NQ is SIDRMs, which does not access the NQ training split, unlike the version used in the main experiments. In IR scenarios, we measure top-1 retrieval accuracy by checking whether the top-1 retrieved document contains the answer. In RAG scenarios, we measure accuracy using a single document the context window, evaluating whether the generated response contains the correct answer.
23

---

Our results indicate that while R3 tunes the retriever to improve RAG performance, it results in inconsistent performance on traditional IR performance, with some degradation observed on certain datasets. This highlights long-standing issue in the IR evaluation pipeline: a document containing the answer does not necessarily imply that it effectively addresses the query, and conversely, document not containing the answer does not mean it is irrelevant or unhelpful.
Our conclusion also aligns with the findings and observations of other research. Cuconasu al. [23] find that including more answer-containnn documents in the context negatively impacts RAG performance. Similarly, Nian al. [82] observe that traditional relevance definitions for IR tasks do not enhance RAG response quality. Additional research emphasizes the need for further learning bridge the preference gap [83] or re-ranking [84] for off-the-shelf retrievers improve RAG performance.
1.2
Case Study
In this section, we present a case study using the NQ dataset where each query has list of answer strings. This case study is designed to further explore the inconsistency issues inherent in RAG implementations. We specifically examine two scenarios: (i) cases where the retrieved document contains the correct answer but fails to produce the correct RAG output, and (ii) instances where the retrieved document does not directly address the query. yet the RAG model manages generate the correct answer nonetheless. To enhance our analysis, we also ask GPT-4 to judge whether the documents address the question, helping readers quickly grasp the key issue.
In Figure 8, we present examples where RAG outputs the correct answer, even though the retrieved document neither contains the answer nor is considered to address the question by GPT-4. In both cases, the document fails to provide the correct answer or relevant clues, yet RÁG is still able to generate the correct response. We believe this is common phenomenon, as LLMs possess wealth of internal knowledge, particularly for public knowledge questions. In general, an incorrect or imperfect retrieved document is insufficient mislead the LLM into producing an incorrect output.
In Figure 9, we present examples where RAG fails to output the correct answer, even though the retrieved document contains the correct answer GPT-4 considers the document addressing the question. In the first case, the document does not address the query, and the LLM tends to extract key phrases, such the title, the response, ignoring the query. In the second case, the document contains information that addresses the query, and the LLM generates the correct answer, but the answer's alias name is not included in the pre-defined answer candidates, leading failure in the RAG scenario. These inconsistencies can be driven by many factors, including the LLM, instruction prompt, evaluation metrics, and relevance. All of these factors are intertwined, and believe that end-to-end data- data-nriven learning is more effective than analyzing their interplay in isolation.
J Examples of Generation
For free-form generation tasks, we assess whether the generation contains any of the given answers. For closed-set generation tasks, we measure whether the generation contains the answers. Below are examples that illustrate how different generations pass (1) or fail (0) the evaluation.
K
Prompt Formats
We demonstrate prompts for different tasks such as OpenQA, fact-checking, and multi-choice reasoning in Figures 12, 13, and 14, respectively.
24

---

Question
Who plays Big Monma in Big Momma's 's House?
Answers
['Ella Witchell', 'Martin Laurence"]
Document
with
Jermaine Dupri, Jagged Edge and Da Brat. Brief clips from "Big Nonma's House" are also included. Bounce with "Bounce with Ne" is single by American rapper Lil' Bow featuring Xscape. It is Lil' Bow Wow's debut single, taken from his debut album "Beware of Dog". The single samples "Love Serenade (Part II)" by Barry white. It spent nine weeks at number the U.S. Rap Chart number 20 on "Billboard" Hot 100. The song featured in the film "Big Momma's 's House" and the trailer of "Hardball".
[1] Nartin Lawrence
GPT4 Judgement
the document does not address the question. The document talks about the song "Bounce with Me" and its connection to the movie Big Homma's House, but doesn't mention who the question is Nartin Lawrence, who plays the
plays Big Momma in the movie. The character of Big Momma in Big Momma's House.
Question
sang "Buddy,
Spare Dime?"
wew.eaecccceeee................. pe..ee...ee...e................. Answers
['Rudy Vallee', "Bing Crosby']
Document
Brother, Can You Spare Dime?
Brother, Can You Spare Dime? "Brother, Can You Spare Dime?", also sung "Buddy, Can You Spare a D'im?",, is one of the best-known American songs of the Great Depression. Written in Spare Dime?" was part of the 1932 musical revue "Americana"; the melody is based on Russian-Jewish lullaby Gorney's mother had sung to him a child. was considered by Republicans be anti-capitalist propaganda, and almost dropped from the show; attempts were made to ban it from the radio.
by lyricist
"Vip" Harburg & composer Jay Gorney, "Brother, Can
Generation
The song "Brother, Can You Spare a D'im?"" was written by E. Y. "Yip" Harburg and Jay Gorney performed by Bing Crosby GPT4 Judgement =esenseepee
The document provides information about the song "Brother, Can You Spare a Dine?" (also referred to "Buddy, Can You Spare Dime?") and its origins, but it does not directly answer the question of who sang it. The document focuses more on the history of the song and its cultural significance.
answer your question: song has performed by several artists, including Bing Crosby, Al Jolson, and Barbra Streisand, among others.
Figure 8: Example RAG output correct answers while the retrieved document does not contain the correct answer, nor does GPT-4 consider the document address the question.
25

---

Question
was The Hitchhiker's Guide the Galaxy made? Answers
sseennnsennaces. ('1978', '2005']
Document
The Hitchhiker's Guide to the Galaxy (radio series) original series Award (1978), The Sony Award (1979), The Society of Authors/Pye Awards Best Programme for Young People (1980) and the Mark Time Awards *Grand Master Award' (Adams) and 'Hall Fame (1998). It was the only radio show awards, in 1979, in the 'Best Dramatic Presentation' category. As result the series, Douglas Adams inducted into the Radio Academy's Hall of Fane. In November 2003, two years after Adams's death and 23 years after the
recipient
number awards including Imperial
to be nominated for the Hugo science fiction
Generation [1] The Hitchhiker's Guide to the Galaxy (radioo series)
aeaasnseeeesaa... ceceeeeeeeeee aeaeesseeeeeee. GPT4 Judgement "*ssass**
No, the document does not directly address the query of when The Witchhiker's Guide to the Galaxy made. It provides information about the radio series, its awards, and recognition but does not specify the year of its creation ar first broadcast.
The original radio series of The Hitchhiker's Guide to the Galaxy first aired in 1978.
Question
Who ordered the Forbidden City to be built? Answers
['Zhu D1?]
Document
History of the Forbidden City site
City Dynasty. After the collapse of the Yuan Dynasty, the Hongwu Emperor of the Ning Dynasty moved the capital from Beijing in the north Nanjing the south, and in 1369 ordered that the Yuan palaces be razed. His Zhu Di was created Prince of Yan with his seat in Beijing. In 1402, Zhu usurped the throne and became the Yongle Emperor. He made Bei beijing ing secondary capital of the Wing
Imperial city during Mongol
The Yongle Emperor ordered the forbidden city to be built. GPT4 Judgement
Yes, the document addresses the query. It mentions that Zhu Di, who later became the Yongle Emperor, ordered the construction the Forbidden City. The passage describes how he moved the capital Beijing and commissioned the building of the Forbidden City 1486.
Figure 9: Example of RAG failing to output the correct answer while the retrieved document contains the correct answer GPTT-4 considers the document as addressing the question.
Question Who the sister for country? Answers
['Rebecca James']
Generation 1
### Explanation:
Rebecca St. James the sister of for king and country. She Christian recording artist and speaker. The Smallbones are the younger brothers of Rebecca St. James, and the the People. They
brothers-in- of Jacob Fink, former bassist for raised with another sister, Libby, and three other brothers, Ben, Dan, and Josh. On Output
Eval (
Generation
Courtney Helm
Explanation:
Caurtney Helm is the sister of for king and country.
Output
Eval :
Figure 10: Examples of generations and their evaluation results for free-form tasks.
26

---

Question
Study: Vaccine for Breast, Ovarian Cancer Has Potential.
True
Generation
True
### Explanation:
The study 15 about a vaccine for breast and ovarian cancer, The vaccine is
Output
Eval(Genrratton):
Generation 2
False
Explanation:
The statement 1s false because the vaccine is not for breast and ovarian cancer.
Output
Eval (Weneeerrattin)
Figure 11; Examples of generations and their evaluation results for close-set tasks.
Prompt (OpenQA)
##d Paragraph:
[1] King & Country (band)
Their father was music promoter, and Joel recalls, "going to these rock concerts, sitting my father''s shoulders, plugging my ears....onessee older, made clear decision to fully lean into it. The Smallbones are the younger brothers of Christian recording artist and speaker Rebecca St. James, the brothers-in-lant of Jacob Fink, former bassist for the band Foster the People They were raised with another sister, Libby, and three other brothers, Ben, Dan, and Josh
in lot ways feel like music
chose and as
aни Instruction:
Who the sister of for king and country?
Response:
Continuation oww.
Rebecca James
asssesnseassees Rebecca St. James
Generation
### Explanation:
Rebecca
the
for king and country. She Christian recording
artist and speaker. The Smallbones are the younger brothers of Rebecca St. James, and the brothers-in-1e of Jacob Fink, former bassist for band Foster the People. They were raised with another sister, Libby, and three other brothers, Ben, and Josh. On Output
Probility: P("Rebecca St. James' pprompt) 0.595
: 1
Figure 12: Example prompt and outcomes of each step for NQ and TQA dataset.
27

---

Prompt (Fact Checking)
Below is an instruction that describes a task. Write a response that appropriately completes request.
### Paragraph:
Gustav Gaudernack
potential of dendritic cells (DCs) and in 2095, Gaudernack's group published results from phase I/II clinical trial prostate
patients using autologous DCs loaded with tumor mRNA as vaccine. This study demonstrates trated that vaccination with autologous DCs transfected with
derived from improved clinical outcome was significantly related to immune responses against the vaccine Furthermore, Gaudernack and colleagues initiated phase I/II clinical trial for treatment of malignant melanoma with autologous tumor-mRNA transfected DC vaccines. These data clearly demonstrated vaccine-speci immune responses with a broad specter of
prostate cancer cell lines was safe and
### Instruction:
Is the following statement correct or not? Say true it's correct; otherwise say false.
нни Input:
Study: Vaccine far Breast, Ovarian Cancer Has Potential
Response:
Continuation
Generation
true
Aиd Explanation: vaccine
study
ovarian cancer. study Output aesennnennenn
P('true' |prompt) 0.116
P('false" |prompt) 0.9 (ienerrettt
Figure 13: Example prompt and outcomes of each step for the Pubhealth dataset.
28

---

Prompt (Nulti-choice Reasoning) Below is instruction that describes task. Write response that appropriately completes the request.
ANd Paragraph:
[1] Rheumatic fever
Rheumatic fever occur following infection of the throat by the bacterium streptococcus pyogenes". If the infection is untreated rheumatic fever can occur up to three percent of people. The underlying mechanism believed to involve the production of antibodies against a person\'s 's own tissues. Due to their genetics, some people are more Likely get the disease include ma malnutrition and poverty. Diagnosis of RF is often based on the presence of signs symptoms combination with evidence recent streptococcal infection. Treating people who have strep
exposed the bacteria than others. Other risk factors
Instruction:
Given four answer candidates, A, B, с and D, choose the best answer choice.
### Input:
Which factor will most likely cause person to develop fever?
A: leg muscle relaxing after exercise
B: bacterial population the bloodstream
C: several viral particles the skin
D: carbohydrates being digested in the stomach
### Response:
Continuation
Generation
Explanation:
The bacteria Streptoccoccus pyogenes is common cause of throat
Output
P('A'
prompt) 0.121
P('B"
prompt) 0.309
P('C'
prompt) 0.861
P('D'
prompt) 0.100
Eval
(G)nerattat::::
Figure 14: Example prompt and outcomes of each step for the ARC-Challenge dataset.
29

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, we have elaborated on the motivation of the paper, and summarized the contributions of the paper in detail.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please refer to our limitations section in Appendix 6, where we discuss computational trade-offs and the limitations of the relatively weak retriever backbone used in our framework.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
30

---

Answer: [NA]
Justification: Our paper is primarily empirical, but we are careful in stating our assumptions. For example, demonstrate the relevance gap between IR and RAG with supporting analysis in Appendix I, and we provide the relationship between probability and generation accuracy with further discussion in Appendix F.
Guidelines:
The answer NA means that the paper does not include theoretical results. the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Yes. We provide dataset details in Appendix A, model architecture Ap- pendix B and C, and training details in Section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
31

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We provide sufficient implementation details to support reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (ht (https://nips...
public/ /wtp//gd///////ee// Seeeee//oeeuubbii C/od/Subeeiieeeeeeeeee seoibbmsssiion aisonoollico ciy for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //ni inips. cc/ public/ htc//giiee Code ss onPolicy cy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer the training details in Section 4.1.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We follow standard RAG evaluation protocols established in prior work and validate results through multiple independent reproductions across different sources, which consistently support our findings.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
32

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report our training cost and runtime details in Section 4.1, with a more detailed analysis provided in Section 5.2.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: https:://ttt::triipppttt httpp//ss..//peiic//tttcctaatrea
Answer: [Yes]
Justification: Our research complies with the NeurIPS Code of Ethies.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer Appendix 6.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
33

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our models are tuned for general QA and reasoning tasks, and do not pose significant risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets and models used in work obtained from publicly available open-source papers. We have cited the original sources and provided detailed references in the Appendix A and B.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
34

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papersw1 thcode. com/ / da tasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We plan to release the trained models and code upon acceptance. Documenta- tion will be provided to ensure reproducibility and responsible
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work does involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
35

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We describe the usage LLMS as a core component of our method Section 3, including details how the LLM interacts with the retriever during training.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components. cecerrrrecccccooeeeceeee/ee////e 2025
Please refer to our LLM policy (https: LLM) for what should or should not be described.
36


Reviews:
[{"review_id": "EONcEcg4pZ", "reviewer": "Reviewer_rFXv", "summary": "This work highlights the importance of finetuning retrievers for RAG. They propose a method called OPEN-RAG that finetunes the retriever using contrastive pairs that are selected by running examples “end-to-end” through the full RAG pipeline. This approach shows improvement over other similar methods across standard RAG benchmarks such as NQ and HotPotQA. The authors also explore both online and offline forms of contrastive learning.", "questions": "Q1: Many practitioners use models like ChatGPT, Claude and Gemini via APIs.How well will this method work? Presumably it can be adapted to API-based RAG systems? The offline computation of probability thresholds is quite clever (starting line 189). Is this something that wouldn’t be possible with APIs? \n\nQ2: In an API scenario, could you use an LLM to judge/rank relevance to pick positive and negative pairs instead of calculating probabilities? While it might be more costly, it might make the method more generalizable\n\nQ3: Does it matter if the training objective follows SiDR with the semi-parametric contrastive loss? Did you consider any other approaches (I guess KL divergence)?\n\nOther comments:\n* The name is very confusing, and does not convey any information about what the method actually does. OPEN-RAG sounds like it is RAG on an open set of questions. The method is also not end-to-end, since it is only optimizing/training the retriever. I really think the authors should consider changing the name to something that conveys a little more information. A good name goes a long way! And a bad name can cause a lot of unnecessary confusion. Even something as simple as “retriever-optimized RAG”\n* Section 1 line 89 states that OPEN-RAG offers “two forms of openness\" - both of these are silly and frankly distracting (it almost sounds like it was AI generated). I’m sure you can come up with a better name.\n* It would be helpful to include the preliminary study mentioned in line 45 in the main text and not in the appendix, if it is indeed very important\n* Typo line 243 “defualt”\n* Typo line 260 “while frozen the computation”", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* This seems like quite a promising method. Finetuning the retriever is also elegant/straightforward and lightweight compared to other methods\n* The authors include ablation studies (online vs. offline, KL divergence vs contrastive loss). The appendix has extensive additional details and experiments\n\nWeaknesses:\n* It is surprising that offline only and online only alone result in suboptimal performance. The fact that the method relies on a mix of online and offline undermines the elegance of the approach. How much of this just depends on hyperparameters?\n* In Table 1, OPEN-RAG shows improvements across 3 different LLMs for NQ, Trivia And HotPotQA but not PubHealth or ARC-C. What is it about these two datasets?\n* Data quality usually plays a crucial role in finetuning. Additional experiments around data quality would make the paper stronger (Figures 3 and 4 are helpful)", "was_revised": true, "final_justification": "The authors have adequately addressed my concerns.", "rebuttal": "We sincerely thank Reviewer rFXv for their thoughtful and constructive review. We would like to address their concerns below:\n\n---\n\n> **W1(1)**: It is surprising that offline only and online only alone result in suboptimal performance. \n\n**A1**: Yes, the suboptimal performance from using offline or online training alone is expected. Offline training can be seen as deploying the retriever into a new environment with pre-labeled data, acting as a \"warm-up\" phase. On the other hand, online training can be considered as a self-improvement process, where the retriever continuously interacts with the environment and refines itself accordingly. The use of a \"warmup\" phase is typically beneficial, as it helps to stabilize the subsequent online training, leading to better overall performance.\n\n---\n\n> **W1(2)**: The fact that the method relies on a mix of online and offline undermines the elegance of the approach. How much of this just depends on hyperparameters?\n\n**A2**: We do not focus extensively on hyperparameters related to the online-offline mixture ratio. Instead, we directly adopt SiDR's ratio, which is an equal mix of offline and online (50%-50%).\n\nRegarding the **elegance** of the approach, we did explore alternatives that use online training only. For example, during online training, we retrieve the top-$k$ (with $k$=20). If we were to gradually increase $k$ from 0 to 20 throughout the training, we could still achieve similar results even without offline warmup. This is because, when k is small (e.g., $k$=2), the top-$k$ retrieved pool is unlikely to contain negatives, prompting us to fall back on the offline negative pool (see `L208`), thus achieving a result similar to offline warmup.\n\nWe ultimately decided to present the work using a mix of online and offline training for the following reasons:\n1. Offline training does not require in-training retrieval, making it more efficient and robust. It is also more likely to transfer stably to new tasks and datastores.\n2. We anticipate that certain users or developers may prefer to use the offline-only to boost their RAG system to a satisfactory level, rather than relying on the more complex offline + online to achieve the utmost performance. Therefore, separating them would make it easier for others to follow up our work.\n\n---\n\n> **W2**: In Table 1, OPEN-RAG shows improvements across 3 different LLMs for NQ, Trivia And HotPotQA but not PubHealth or ARC-C. What is it about these two datasets?\n\n**A3**: Regarding the transferability across different LLMs, we discuss this in `L287-295`. Additionally, it may be related to how these datasets are constructed. For example, the queries in NQ, TQA, and HotpotQA are based on Wikipedia, and the task is designed to work with the knowledge within Wikipedia corpus. In contrast, PubHealth and ARC are not. \n\n- For NQ, TQA, and HotpotQA, the Wikipedia datastore likely contains documents with clear, direct answers. The relationship between queries and answers is so straightforward that even different LLMs can easily extract the same answer from the document, enabling transferability across LLMs.\n\n- However, for PubHealth and ARC, there may not be a document specifically designed to provide a direct answer to the query, and different LLMs may have varying preferences for which documents should be included in the context, making transferability more challenging.\n\n---\n\n> **Q1**: Many practitioners use models like ChatGPT, Claude and Gemini via APIs.How well will this method work? Presumably it can be adapted to API-based RAG systems? The offline computation of probability thresholds is quite clever (starting line 189). Is this something that wouldn’t be possible with APIs?\n\n**A4**: We appreciate the insightful question and thank you for your compliment on our threshold-based approach! Since ChatGPT, Claude, and Gemini APIs are closed source, the applicability of our method depends on whether these APIs can output prompt logits. If they do, our method could be adapted to work with these APIs. If not, we could optimize the retriever using open-source LLMs (e.g., DeepSeek and Qwen) with similar intelligence and scale. As discussed in **A3** above, if the datastore is well-designed and relevant, the learned retriever is likely to transfer effectively.\n\nWe believe that co-training a retriever with these API-based LLMs is a common challenge for any methods aiming to optimize the retriever through interaction with the LLM. In this context, the transferability analysis we provide offers a unique insight for future research, enabling us to handle such scenarios effectively.\n\n---\n\n> **Q2**: In an API scenario, could you use an LLM to judge/rank relevance to pick positive and negative pairs instead of calculating probabilities? While it might be more costly, it might make the method more generalizable\n\n**A5**: It is possible, and we are keen to explore this approach. The only limitation is our limited GPU resources and API token budget. Using an LLM as a judge is similar to a reward model, which has proven effective in training LLMs. This could enable the retriever to self-improve (retrieve → reward → update retriever) and address tasks lacking robust evaluation metrics or require expensive human annotation costs.\n\nIn practice, there are additional considerations. First, beyond the cost, the efficiency of using API LLMs for online evaluation could be challenging. A more practical solution could be deploying a comparable open-source LLM (e.g., DeepSeek-671B) as a local service to guarantee efficiency. Second, while LLM-as-a-judge has its advantages, it also has biases and limitations. Combining different evaluation metrics for online tranining would provide a more balanced solution.\n\n---\n\n> **Q3**: Does it matter if the training objective follows SiDR with the semi-parametric contrastive loss? Did you consider any other approaches (I guess KL divergence)?\n\n**A6**: Thank you for the question! Due to page limits, please refer to our **A3** to `Reviewer rPbd`, where we explain each term of our loss function and the rationale behind it, along with ablation experiments. In Section 5.1 (`L313-326`), we compare contrastive loss with KL-divergence and find that contrastive loss performs better.\n\nWe use an example to illustrate why. Given a query $q$ and its top-5 retrieved documents, where $Gen(q,d)$ measure whether using $d$ can pass the evalution and $Rel(q, d)$ is the relevance between $q$ and $d$.\n\n|        |  \\$d_1\\$  |  \\$d_2\\$  |  \\$d_3\\$  |  \\$d_4\\$  |  \\$d_5\\$  | \n|-------|:----:|:----:|:----:|:----:|:----:|\n| \\$Gen(q,d)\\$   |  1   |  1   |  1   |  0   |  0   | \n| \\$P(y \\mid q, d)\\$  | 0.74 | 0.63 | 0.89 | 0.65 | 0.59 |\n| \\$Rel(q, d)\\$  | 3.7 | 2.9 |  2.4 | 0.9 | 0.02 |\n\nIn this case, online contrastive objective will try to maximize $Rel(q, d_1) \\rightarrow \\infty$ and minimize $Rel(q, d_5) \\rightarrow -\\infty$. However, KL-divergence would try to reduce $Rel(q, d_1)$ and increase $Rel(q, d_5)$, in this way, ensuring the distribution of $Rel(q, d)$ to become more similar to $P(y \\mid q, d)$, which is not the desired outcome.\n\nIn our experience, using KL-divergence leads to rapid convergence in the early stages (typically the first few epochs) with limited improvement. In contrast, the online contrastive objective continues to improve performance over a longer training period (see our `Figure 3`).\n\n---\n\n> **C1**: The name is very confusing, and does not convey any information about what the method actually does. ... I really think the authors should consider changing the name to something that conveys a little more information. A good name goes a long way! And a bad name can cause a lot of unnecessary confusion. Even something as simple as “retriever-optimized RAG”\n\n> **C2**: Section 1 line 89 states that OPEN-RAG offers “two forms of openness\" - both of these are silly and frankly distracting (it almost sounds like it was AI generated). I’m sure you can come up with a better name.\n\n**A7**: Thank you for the suggestion and encouragement! We agree that while \"open\" comes from the initials of \"OPtimize ENd-to-end,\" it may lead to misleading preconceptions. To address this concern, we are considering replacing the title with something like \"(Consistently) Optimizing Retrieval for RAG\" to better highlight the training-inference consistency. Additionally, Section 1, L89 will be restated to focus on contributions beyond just effectiveness, with the term \"openness\" removed.\n\n---\n\n> **C3**: It would be helpful to include the preliminary study mentioned in line 45 in the main text and not in the appendix, if it is indeed very important\n\n**A8**: Thank you for the suggestion. If our paper is accepted with 10-page limitation, we will consider moving this content into the main text to better provide the contextual motivation. We will also correct the typo and ensure everything is checked properly.\n\n---\n\nThank you once again for your constructive review. We hope our response satisfactorily addresses all your questions. Should you have any further doubts or concerns, please feel free to follow up."}, {"review_id": "j3L7oIrqhq", "reviewer": "Reviewer_shNy", "summary": "The paper studies to apply the previously proposed sparse retrieval model, SiDR, to improving the end-to-end training efficiency of RAG systems.", "questions": "1. Why there is only one-shot effectiveness shown in HotpotQA, while all the other datasets show both 1-shot and 10-shot effectiveness?", "limitations": "See Weakness 2 and 3.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The paper offers a method to improve the training efficiency of RAG by proposing a walk-around method without conducting computationally expensive indexing in RAG training. The proposed method is effective while using the retrieval model, SiDR.\nWeaknesses\n1. The biggest concern for me is that the comparison is a bit misleading. As I know, all the other models (e.g., RA-DIT and SELF-RAG) are not fine-tuned on the train set of the targeted datasets (e.g., NQ, TriviaQA, HotpotQA). This means the key point I can learn from the experiment is that fine-tuning RAG (either retriever or LLM) on the targeted dataset can help but the generalization capability is not confirmed. Correct me if I’m wrong. \n2. The previous study has proposed different RAG training strategies to address the issue of index updating for dense retrievers (See top-100 reranking and Query-side fine-tuning in Table 4 in [1]). Although the work provides another solution based on the previously proposed sparse retriever, SIDR. The motivation and contribution are a bit limited since the paper only builds upon a specific sparse retriever, SIDR.\n3. It seems that the obvious improvement is when applying the technique to pre-trained LLMs rather than instruction fine-tuned ones.\n\n[1] Atlas: Few-shot Learning with Retrieval Augmented Language Models; Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Tim  Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, Edouard Grave; arXiv:2208.03299", "was_revised": true, "final_justification": "Thanks for you clarifying and I agree that although the study mainly based on SiDR; the study is still valuable in the community of IR and RAG. I'll increase my score to 4.", "rebuttal": "We thank Reviewer shNy for their thorough and insightful review. We would like to address their concerns below:\n\n---\n\n> **W1**: The biggest concern for me is that the comparison is a bit misleading. As I know, all the other models (e.g., RA-DIT and SELF-RAG) are not fine-tuned on the train set of the targeted datasets (e.g., NQ, TriviaQA, HotpotQA). This means the key point I can learn from the experiment is that fine-tuning RAG (either retriever or LLM) on the targeted dataset can help but the generalization capability is not confirmed. Correct me if I’m wrong.\n\n**A1**: We understand the reviewer may be concerned that our improvement comes solely from access to the supervised training data. We would like to clarify the following:\n\n**Clarification on the train set**. To our knowledge, SELF-RAG leverages the KILT dataset [1] for training, which includes NQ, TQA, and HotpotQA. The baseline using the Llama3-8B instruction-tuned model should be considered as having access to these well-known datasets during its instruction-tuning process. While RA-DIT does not have access to these datasets, it has been trained on various Wikipedia-based QA datasets, which likely benefit performance on NQ, TQA, and HotpotQA as well.\n\n**The role of the training data is not the primary factor**.  Access to supervised training data alone is not the dominant factor. The critical aspect is how training is designed to optimize the retriever for RAG. This is supported by the following observations:\n- (i) standard retriever fine-tuning on supervised data results in less effective RAG performance;\n- (ii) there are other existing strategies for optimizing the retriever for RAG, and even with supervised data, these methods underperform compared to ours (details discussed below). \n\n**Supporting evidence and comparisons**:\n- Standard retriever fine-tuning (Table 3) vs. our method that optimizes the retriever for RAG (Table 1), where both access the training data but OpenRAG leads to significant improvements. This comparison shows that having access to the training data is not the key factor; rather, the crucial aspect is how the data is leveraged to optimize the retriever for the entire RAG system.\n\n- For the \"improving LLM for RAG\" baselines, we recommend comparing our method to SELF-RAG and Llama3-Instruct-8B + SIDR, where OpenRAG consistently outperforms SELF-RAG and matches Llama3-Instruct-8B + SIDR on certain datasets. These baselines are introduced to give the reader a sense of the relative improvement of optimizing the retriever compared to improving the LLM, as noted in L264.\n\n- For the \"improving IR for RAG\" baselines (e.g., RePLUG, RA-DIT's retriever), we suggest reviewing our ablation study in Section 5.1. The key differences between OpenRAG and these models are in the alignment objective (Online Contrastive Learning vs. KL-Divergence) and index refresh strategies (Late Parametric vs. Periodic Re-indexing). These ablation experiments can be viewed as variants of RePLUG using the same LLM, instructions, and **training data**. The results show that, among the methods optimizing the retriever for RAG, OpenRAG introduces several constructive improvements, proving it to be superior and more cost-effective.\n\n---\n\n> **W2**: The previous study has proposed different RAG training strategies to address the issue of index updating for dense retrievers (See top-100 reranking and Query-side fine-tuning in Table 4 in [1]). Although the work provides another solution based on the previously proposed sparse retriever, SIDR. The motivation and contribution are a bit limited since the paper only builds upon a specific sparse retriever, SIDR.\n\n**A2**: Thank you for your detailed question and for pointing out the relevant reference. We would like to address your concern from several aspects below:\n\n**Regarding index staleness issues**. The three index solutions mentioned in the Atlas paper each have inherent limitations:\n1. Full index updates periodically (e.g., RePlug) incurs high costs, and our ablation study in Section 5.1 shows that it is less effective than ours.\n2. Top-100 reranking requires partial index updates. This is incompatible with tools like FAISS for dense indexes and torch.sparse for sparse indexes, making it impractical for large datastores. As a result, few works use this approach due to its inefficiency.\n3. Query-side fine-tuning (e.g., Atlas and RA-DIT) with frozen document encoders and static indices can lead to inconsistent or limited improvements, as outlined in their papers.\n\nOur solution, based on a semi-parametric design, improves effectiveness, reduces training costs, and ensures a straightforward training process, making it easier to deploy for RAG applications. We hope the benefits of our approach are considered alongside its potential limitations.\n\n\n**From a broader perspective**: Our work addresses the optimization of the retriever for the entire RAG system, where index staleness is just one of the challenges we tackle. Other key contributions and insight include:\n\n- Preliminary experiments to highlight the challenges faced by retrievers in the transition from traditional IR to RAG setups.\n- A threshold-based approach to reduce the online LLM autoregressive generation costs, cutting the 100 LLM forward passes (with a maximum of 100 tokens per generation) down to a single forward pass, making the training cost affordable.\n- Exploration of different sampling strategies and the proposal of cache implementations.\n- Validation of the overall effectiveness and transferability of our RAG system.\n- Detailed ablation studies on different co-training strategies and index refresh methods.\n\nWe believe these contributions and analyses will be valuable for future research in this area. While SiDR forms part of our methodology, we hope the broader scope of our contributions will be recognized, as they extend beyond this adoption.\n\n---\n\n> **W3**: It seems that the obvious improvement is when applying the technique to pre-trained LLMs rather than instruction fine-tuned ones.\n\n**A3**: We primarily use pre-trained LLMs because it is very likely that the training data from well-known datasets (e.g., NQ, TQA, HotpotQA) are involved during instruction tuning. Using pre-trained LLMs better aligns with real-world application scenarios, where the online user query is not part of the LLM's training data. \n\nAs shown in Table 1, despite OpenRAG being optimized for Llama3-8B, transferring to Llama3-8B-instruct (the instruction fine-tuned version) still results in improved RAG performance.\n\nTo better address your concern, we also experimented with our method using Llama3-8B-instruct as the LLM, and the results show that our method can also improve performance for instruction-tuned models:\n\n|  | NQ  |  PubHealth | \n|---------------|------|-------|\n| Llama3-8B-instruct + SiDR     | 47.2 | 67.2  |\n| OpenRAG (Llama3-8B-instruct)  | 49.3 (+2.1) | 69.8 (+2.6) |\n\n\n\n---\n\n> **Q1**: Why there is only one-shot effectiveness shown in HotpotQA, while all the other datasets show both 1-shot and 10-shot effectiveness?\n\n**A4**: The difference arises from the use of different Wikipedia datastores (`L243-245`). Specifically, HotpotQA uses a Wikipedia corpus containing 5 million documents, with each document being the first paragraph of a Wikipedia page, ranging from 20 to over 1k words. In contrast, all other datasets use the same Wikipedia corpus, which contains 21 million passages, each 100 words long. Using 10-shot for HotpotQA results in contexts that are too long and exceed the maximum context window of the LLM.\n\n---\n\nThank you once again for your thoughtful review. We hope our response addresses your concerns and questions. If you have any further inquiries, please feel free to follow up.\n\n---\n\n**Reference**\n\n[1] Petroni, Fabio, et al. \"KILT: a benchmark for knowledge intensive language tasks.\" ACL 2023."}, {"review_id": "7qrwjXVZp6", "reviewer": "Reviewer_rPbd", "summary": "This paper presents an end-to-end RAG framework that enhances performance by focusing solely on improving the retriever component. Positive and negative query-document pairs for contrastive learning are selected based on the likelihood of generating the final response. The study also highlights that relevance models used in traditional Information Retrieval (IR) often do not align well with RAG performance. Experimental results on five public RAG benchmarks show that the proposed method outperforms other compared approaches.", "questions": "Please see my questions in the weakness section. The final score may be changed based on the author response. My main concerns are the setting of thresholds and the ablation study on the objective loss.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper introduces an end-to-end optimized RAG framework that enhances RAG performance by tuning the retriever component through online contrastive learning.\n\n2. It demonstrates that higher accuracy in traditional IR settings generally correlates with improved RAG performance for QA tasks, although this correlation does not extend to non-QA tasks.\n\n3. The proposed method outperforms other compared approaches on five public datasets.\n\nWeaknesses:\n1. The settings for the thresholds (\\tau_q^{+/-}) in line 197 are not clearly explained. These thresholds are crucial for determining positive and negative documents during online contrastive learning. The rationale for using the top 100 documents to establish these parameters is unclear. It would be beneficial to show the impact of varying these parameter settings on performance.\n\n2. An ablation study for the final loss function, as mentioned in line 224, is necessary. It would be insightful to observe the performance differences when different components are included in the objective loss. This analysis would help in understanding the contribution of each component to the overall effectiveness of the model.", "was_revised": true, "final_justification": "Based on the authors rebuttal, which solves my concern in this paper, I keep my original rating.", "rebuttal": "We thank Reviewer rPbd for their thorough and insightful review! We would like to address their concerns below:\n\n---\n\n> **W1(1)**: The settings for the thresholds ($\\tau_q^{+/-}$) in line 197 are not clearly explained. These thresholds are crucial for determining positive and negative documents during online contrastive learning. \n\n**A1**: Thank you for the feedback! To clarify the intuition, let's use a simple example. Given a query $q$, we offline retrieve $k=5$ documents {$d_1$, $d_2$, $d_3$, $d_4$, $d_5$} to establish the online thresholds. Note that we use $k=100$ in our experiments, and $k=5$ here for illustration. \n\n|        |  \\$d_1\\$  |  \\$d_2\\$  |  \\$d_3\\$  |  \\$d_4\\$  |  \\$d_5\\$  | \n|-------|:----:|:----:|:----:|:----:|:----:|\n| \\$Gen(q,d)\\$   |  1   |  1   |  1   |  0   |  0   | \n| \\$P(y \\mid q, d)\\$  | 0.74 | 0.63 | 0.89 | 0.65 | 0.59 |\n\nIn this table, $Gen(q,d)$ indicates whether the RAG generation contains the answer (i.e., pass/fail evaluation), and $P(y \\mid q, d)$ refers to the joint probability of ground truth $y$ given $q$ and $d$ (see `L191-193`). Below is the process used to determine positives and negatives online:\n\n- Offline positives = {$d_1$, $d_2$, $d_3$}; Offline negatives = {$d_4$, $d_5$}.\n- Thresholds for online positives $\\tau_q^{+}$ = max{0.65, 0.59} = 0.65; Thresholds for online negatives $\\tau_q^{-}$ = min{0.74, 0.63, 0.89} = 0.63. \n- In practice, we also use $\\tau_q^{+}$ = max{$\\tau_q^{+}$, $\\tau_q^{-}$} and $\\tau_q^{-}$ = min{$\\tau_q^{+}$, $\\tau_q^{-}$} to ensure $\\tau_q^{+}$ > $\\tau_q^{-}$.\n- During online training, a document $d_i$ is considered:\n    - **Positive** if $Prob(y \\mid q, d_i) > \\tau_q^{+}=0.65$. **Rationale**: based on offline $k$ observations, we never see a document $d$ with $Prob(y \\mid q, d)$ greater than $\\tau_q^{+}=0.65$ that fails the evaluation.\n    - **Negative** if $Prob(y \\mid q, d_i) < \\tau_q^{-}=0.63$. **Rationale**: based on the offline $k$ observations, we never see a document $d$ with $Prob(y \\mid q, d)$ smaller than $\\tau_q^{-}=0.63$ that passes the evaluation.\n    - **Discarded** otherwise. **Rationale**: such cases are ambiguous, and classifying them as either positive or negative could risk negatively impacting the training.\n\n---\n\n> **W1(2)**: The rationale for using the top 100 documents to establish these parameters is unclear. It would be beneficial to show the impact of varying these parameter settings on performance.\n\n**A2**: The choice of $k=100$ is primarily driven by cost-effectiveness. If $k$ is too small, we have insufficient observations to determine the threshold and may need to discard more training queries — see `L196`, where we discard queries that have neither offline positives nor offline negatives to guarantee both pools are non-empty for contrastive learning. On the other hand, if $k$ is too large, it would increase the cost and workload for preparation with limited improvement.\n\nTo address your concerns, we conduct further experiments on NQ and PubHealth with varying $k$:\n\n|   | k=5  |  k=20 | k=100 | k=200 |   \n|:--:|:--:|:--:|:--:|:--:|\n| NQ | 42.3 | 46.9  | 47.8 | 47.9 | \n| PubHealth | 64.1 | 65.7  | 69.5| 69.7 |\n\nThe ablation results support our analysis that increasing $k$ from 100 to 200 provides minimal improvement, while performance drops when $k$ is too small (e.g., 5, 20).\n\n---\n\n> **W2**: An ablation study for the final loss function, as mentioned in line 224, is necessary. It would be insightful to observe the performance differences when different components are included in the objective loss. This analysis would help in understanding the contribution of each component to the overall effectiveness of the model.\n\n**A3**: The final loss function is derived directly from SiDR, as we adopt its in-training retrieval mechanism. The loss function consists of two components: $L_{para}$ and $L_{semi-para}$. \n- $L_{para}$ is the standard contrastive learning loss that aligns the query embedding with the document embedding.\n- $L_{semi-para}$ is unique to SiDR:\n    - The first term $L(E_{\\theta}(q), E_{BoT}(d))$ aligns the embedded query with the tokenized document, which guarantees in-training retrieval with bag-of-token index.\n    - The second term $L(E_{BoT}(q), E_{\\theta}(d))$ aligns the tokenized query with the embedded document. Although this term may seem irrelevant, empirical results show that including it provides a beneficial effect.\n\nTo address your concern, we present the following ablation results reported in SiDR’s original work, demonstrating the benefit of the second term:\n\n| Model  | NQ   |  TQA   | WebQ | \n|---------------|------|-------|------| \n| SiDR     | 49.1 | 56.2  | 40.2 | \n| SiDR (w/o second term)  | 48.7 | 56.0  | 38.7 |\n\nWe also conduct ablation studies using our OpenRAG method on both a QA task (NQ) and a non-QA task (PubHealth). The results consistently show that both terms contribute to performance:\n\n| Model  | NQ  |  PubHealth | \n|---------------|------|-------|\n| OpenRAG     | 47.8 | 69.5  |\n| OpenRAG (w/o second term)  | 45.1 | 63.2  |\n\n---\n\n> **C1**: Please see my questions in the weakness section. The final score may be changed based on the author response. My main concerns are the setting of thresholds and the ablation study on the objective loss.\n\n**A4**: We appreciate your thoughtful comments. We hope our responses to W1 and W2 have addressed your concerns. Please feel free to let us know if anything remains unclear. We would greatly appreciate it if you could take these clarifications into account when finalizing your score."}, {"review_id": "MIrGu03i4W", "reviewer": "Reviewer_3W4Q", "summary": "This paper proposes a framework called Open-RAG, which constructs contrastive learning supervision for retriever training by automatically determining whether a given Query-Document pair is a positive match in the context of RAG. This allows the retriever to better align with the downstream needs of the LLM-based QA task, rather than relying solely on semantic similarity, thereby improving task-specific RAG performance.", "questions": "1 In Section 3.2, how is the evaluation conducted to construct the positive pool Dq+ and the negative pool Dq−? I guess it might be based on an exact match, but this should be clearly specified in the main text.\n\n2 As a core design choice of this work, the loss terms described in Section 3.4 are critical. However, I find their respective roles insufficiently explained. A more intuitive explanation would be helpful, especially regarding this confusing point: the component EBoT(q) does not participate in inference and appears only in the loss term. What is the rationale for including it in training?\n\n3 In Appendix D, what are the specific procedures for the IR setting and RAG setting respectively? The descriptions of “Unsupervised Pre-training,” “Supervised on MS MARCO,” “Supervised on NQ,” “Supervised on TriviaQA,” and “Pre-training + Supervised on Multiple Datasets” are unclear. Please clarify what each of these means in practice.\n\n4 In Section 4.1, the statement “Our experiments are conducted with 4 NVIDIA GPUs” should include the specific GPU model for reproducibility and transparency.", "limitations": "See Questions.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n1 The paper presents an end-to-end mechanism to optimize the RAG system in accordance with the requirements of the downstream task. Specifically, it uses generation certainty as a signal to construct contrastive supervision for training the retriever. This is an intuitive, effective, and elegant approach.\n\n2 As an online, lightweight, and end-to-end optimization framework for RAG, the design of Open-RAG offers valuable engineering insights.\n\n3 Open-RAG only requires a one-time offline index construction over the entire corpus, significantly reducing system overhead.\n\n**Weaknesses:**\n\n1 The most novel aspect of this paper—its semi-parametric design—has already been proposed in prior work (SIDR), which diminishes the originality of the contribution.\n\n2 The methodological innovation appears limited. Steps 1 and 3 appear to be directly borrowed from SIDR, leaving only Step 2 as the novel component. As a result, the paper feels more like an application of SIDR to RAG, rather than a substantially new contribution.", "was_revised": true, "final_justification": "I would like to maintain my current score after reviewing the response and the feedback from other reviewers.", "rebuttal": "We thank Reviewer 3W4Q for their detailed and insightful review! We would like to address their concerns below:\n\n---\n> **W1**: The most novel aspect of this paper—its semi-parametric design—has already been proposed in prior work (SIDR), which diminishes the originality of the contribution.\n\n> **W2**: The methodological innovation appears limited. Steps 1 and 3 appear to be directly borrowed from SIDR, leaving only Step 2 as the novel component. \n\n**A1**: We understand the reviewer's concern regarding the originality of our method due to the direct application of SiDR. However, we would like to clarify the following points to demonstrate the contribution of our work.\n\n**Main Contribution Difference**: The primary contribution of our work lies in the optimization of the retriever specifically for RAG workflows. While we leverage SiDR to address the index refreshment issue, this is only one component of improving RAG workflows. There are several other innovations and contributions beyond this. In contrast, SiDR's main contribution is centered on indexing efficiency and cost, with limited empirical experiments focused on RAG.\n\n**Contibutions beyond SiDR**: Please allows us highlight other key contributions.\n-  Section 1 and Appendices B/C/D: We systematically explore and analyze the gap between traditional IR and RAG, identifying current challenges and emphasizing the potential for improving retriever performance in RAG systems.\n- Section 3.3: We propose a threshold-based approach to reduce online LLM autoregressive generation costs, cutting the 100 LLM forward passes (with each generation having a maximum of 100 tokens) down to a single forward pass, making the approach transition from unaffordable to practical.\n- Section 3 and Appendices E/F: We explore different sampling strategies and propose cache implementations to further reduce computation.\n- Section 4: We validate the effectiveness of our overall RAG system and assess its transferability to other LLM systems.\n- Section 5: We conduct ablation studies on different co-training strategies and index refreshment methods.\n\nAll these contributions are original to our work, and we believe they will be valuable for future research in this area. While SiDR forms part of our methodology, we hope that the broader scope of our contributions will be recognized, as they go beyond this adoption.\n\n---\n\n> **Q1**: In Section 3.2, how is the evaluation conducted to construct the positive pool Dq+ and the negative pool Dq−? I guess it might be based on an exact match, but this should be clearly specified in the main text.\n\n**A2**: To clarify, Section 3.2 discusses the retrieval process but does not specifically address how the positive and negative pools are constructed. The details of how we determine the positive and negative documents are provided in Section 3.3. Specifically:\n- For offline positive and negative pools, we evaluate whether the RAG generation contains the answer string (see `L193-195`).\n- For online positive and negative pools, we use a threshold-based approach, which is introduced in Section 3.3 (see `L196-203`).\n- Our evaluation follows prior RAG research (e.g., Self-RAG), where accuracy is used as the metric, based on whether the RAG generation contains the correct answer (see `L236-238`).\n\nWe also provide an example in **A1** of our response to `Reviewer rPbd`, which demonstrates the entire calculation process. We kindly recommend reviewing this for further clarification.\n\n\n\n---\n\n> **Q2**: A more intuitive explanation would be helpful, especially regarding this confusing point: the component EBoT(q) does not participate in inference and appears only in the loss term. What is the rationale for including it in training?\n\n**A3**: Thank you for the insightful question! You're correct that the second term $L(E_{BoT}(q), E_{θ}(d))$ does not appear during inference. Here's a breakdown of the rationale and empirical support for including this term in training:\n\n- **Rationale**: The second term would encourage $E_θ(d)$ to activate lexical tokens from $BoT(q)$. By doing so, it enhances the ability of $E_θ(q)$ to search over $E_θ(d)$, as $E_θ(q)$ typically assigns high term weights to lexical tokens from $BoT(q)$.\n\n- **Empirical support**: In SiDR's rebuttal on ICLR, they empirically demonstrated that this term improves retrieval accuracy. Below are part of their ablation results:\n\n| Model  | NQ   |  TQA   | WebQ | \n|---------------|------|-------|------| \n| SiDR     | 49.1 | 56.2  | 40.2 | \n| SiDR (w/o second term)  | 48.7 | 56.0  | 38.7 |\n| SiDR$_{\\beta}$      | 39.8 | 50.4  | 32.1 | \n| SiDR$_{\\beta}$  (w/o second term)  | 37.1 | 46.8  | 30.4 |\n\nAdditionally, we conducted ablation studies using our OpenRAG method with 1-shot on NQ and PubHealth, where we observed a similar improvement, with the inclusion of the second term providing a noticeable benefit:\n\n| Model  | NQ  |  PubHealth | \n|---------------|------|-------|\n| OpenRAG     | 47.8 | 69.5  |\n| OpenRAG (w/o second term)  | 45.1 | 63.2  |\n\n---\n\n> **Q3**: In Appendix D, what are the specific procedures for the IR setting and RAG setting respectively? The descriptions of “Unsupervised Pre-training,” “Supervised on MS MARCO,” “Supervised on NQ,” “Supervised on TriviaQA,” and “Pre-training + Supervised on Multiple Datasets” are unclear. Please clarify what each of these means in practice.\n\n**A4**: We apologize for any inconvenience caused. As noted in Appendix D (`L696-698`), the details you refer to are already outlined in Section 4.1 and Appendix C. While this information is mentioned, we understand that it might be scattered. To better address your concern, we will consolidate and repeat this information in Appendix D for the convenience of future readers. \n\nHere are the relevant details with references from the paper:\n\n- For Evaluation metric (details in Section 4.1, `L236-238`): \n    - **IR setting** assesses whether the retrieved document contains the answer.\n    - **RAG setting** assesses whether the RAG generated response contains the answer.\n\n- For Retrieval Model Details (details in Appendix C1, `L638-653`): \n    - **Unsupervised Pre-training** refers to a retrieval model that is pre-trained on unsupervised data without any access to any downstream supervised training data (i.e., human-labeled query-document pairs).\n    - **Supervised on MS MARCO / NQ / TriviaQA** refers to models are fine-tuned on the training data of the MS MARCO, NQ, or TriviaQA datasets, respectively.\n    - **Pre-training + Supervised on Multiple Datasets** combines unsupervised pre-training followed by supervised fine-tuning on multiple datasets. Typically, this combination achieves state-of-the-art performance but comes with significant costs.\n\n---\n\n> **Q4**: In Section 4.1, the statement “Our experiments are conducted with 4 NVIDIA GPUs” should include the specific GPU model for reproducibility and transparency.\n\n**A5**: We use 4 NVIDIA A100 GPUs, each with 80GB memory. We will include this detail in revision.\n\n---\n\nWe truly appreciate your comments and suggestions and will incorporate them into our final draft. We hope our response has addressed your concerns. Please feel free to let us know if you have any further questions."}]

Author rebuttal:

Meta-review: This paper proposes OPEN-RAG, a framework that tunes a retriever for a retrieval-augmented generation (RAG) system. The method uses online contrastive learning. The learning signal comes from the final RAG generation outcome, which aligns the retriever's relevance with the downstream task. The authors use a semi-parametric retriever to make this end-to-end training efficient and avoid costly index updates. The experiments show that this approach improves performance on several RAG benchmarks. It is more cost-effective than tuning the large language model itself.

Strengths. It addresses the important problem of relevance mismatch between information retrieval and RAG. Reviewers praised its elegance and practicality. The empirical evaluation is comprehensive, with thorough ablation studies that validate the design choices.

Weaknesses. Limited methodological innovation - the framework builds heavily on the existing SIDR retriever architecture. This led one reviewer to argue that the contribution is more of an engineering application than a fundamental advance. Another reviewer initially questioned the fairness of the baseline comparisons.

Rebuttal & discussion period. The authors were very responsive. 3W4Q raised a strong objection regarding the limited novelty. The authors countered by providing new ablation studies showing that a naive application of SIDR fails and that their specific design is crucial for success. shNy had concerns about unfair comparisons. The authors clarified the training data used by the baselines and provided new experiments to strengthen their claims. The rebuttal successfully convinced 3/4 reviewers to recommend acceptance.

Why accept? We put more weight on the paper's practical impact. It provides an effective and efficient solution to a critical problem for anyone building RAG systems. Yes, the novelty of individual components is limited, but the complete framework delivers clear and significant value to the community.
Decision: Accept (poster)