Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers
Authors: Youmin Ko
Sung Jong Seo
Hyunjoon Kim
Venue: NeurIPS 2025 poster ()
Keywords: retieval-augmented generation
RAG
multi-hop question answering
contrasting layers
question augmentation
Primary area: applications
TL;DR: We propose a novel RAG framework in which a retriever and an LLM cooperate to exchange knowledge, and the retriever utilizes information of both earlier and later layers to accurately rank documents relevant to a question.

Abstract: Since large language models (LLMs) have a tendency to generate factually inaccurate output, retrieval-augmented generation (RAG) has gained significant attention as a key means to mitigate this downside of harnessing only LLMs. However, existing RAG methods for simple and multi-hop question answering (QA) are still prone to incorrect retrievals and hallucinations. To address these limitations, we propose CoopRAG, a novel RAG framework for the question answering task in which a retriever and an LLM work cooperatively with each other by exchanging informative knowledge, and the earlier and later layers of the retriever model work cooperatively with each other to accurately rank the retrieved documents relevant to a given query. In this framework, we (i) unroll a question into sub-questions and a reasoning chain in which uncertain positions are masked, (ii) retrieve the documents relevant to the question augmented with the sub-questions and the reasoning chain, (iii) rerank the documents by contrasting layers of the retriever, and (iv) reconstruct the reasoning chain by filling the masked positions via the LLM. Our experiments demonstrate that CoopRAG consistently outperforms state-of-the-art QA methods on three multi-hop QA datasets as well as a simple QA dataset in terms of both the retrieval and QA performances. Our code is available.

Cooperative Retrieval-Augmentee Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers
Youmin Ko¹, Sungjong Seo', Hyunjoon Kim 2.2-
1 Department of Artificial Intelligence, Hanyang University
*Department of Data Science, Hanyang University
fyouminkk0213, chrisseo2002, hyunjoonkim, waaanayannn.nnngg......mm ac kr
Abstract
Since large language models (LLMs) have tendency to generate factually inaccu- rate output, retrieval-augmented generation (RAG) has gained significant attention key means to mitigate this downside of harnessing only LLMs. However, existing RAG methods for simple and multi-hop question answering (QA) are still prone incorrect retrievals and hallucinations. To address these limitations, propose CoopRAG, a novel RAG framework for the QA task in which retriever and an LLM work cooperatively with each other by exchanging informative knowl- edge, and the earlier and later layers of the retriever model work cooperatively with each other to accurately rank the retrieved documents relevant given query. In this framework, we (i) unroll question into sub-questions and a reasoning chain in which uncertain positions to the question augmented with the sub-questions and the reasoning chain, (iii) rerank the documents by contrasting layers of the retriever, and (iv) reconstruct the reasoning chain by filling the masked positions via the LLM. Our experiments demonstrate that CoopRAG consistently outperforms state-of-the-art QA methods on three multi-hop QA datasets as well as simple QA dataset in terms of both the retrieval and QA performances. Our code available
masked, (ii) retrieve the documents relevant
Introduction
Since large language models (LLMMs) have tendency to generate factually inaccurate output, retrieval- augmented generation (RAG) has gained significant attention as a key means to mitigate this downside of harnessing only L.Mss in various tasks such as knowledge base question answering (KBQA) 4 21
24 79 87), multi-hop question answering (QA) [19 20 86), knowledge graph completion [39 81
20 and recommender systems 2
10 71 777. Recent studies have made various attempts to address
the downside of LLMs: (i) applying the fine-granular late interaction scoring mechanism (3)
using multi-vector representations, (ii) augmenting query with its hypothetical answers query- related concepts by employing LLMs [34 43 [45,, and (iii) allowing LLMs to either generate summaries knowledge graph to connect groups of disparate but related passages, and exploring
multiple documents from the core concepts of the query a structure- structure-augmented augmented manner
However, these approaches are still prone to incorrect retrievals and hallucinations 5 23
26
especially in simple and multi-hop QA. We hypothesize that these limitations stem from the following three reasons.
Corresponding author.
httph://github... http:///www.nngggg/////////goo//
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

First, questions are often too short and limited in information to elicit both the documents most relevant to those questions from the retrieval modules, and high-quality reasoning results from LLMs. Despite the potential of query rewriting methods that form given questions into longer and more helpful queries, their rewriting processes rely heavily on external resourees and their supervisions [14 47). and they do not verify the rewritten queries [2 88], thus being insufficient to fully draw out the internal knowledge of LLMs.
Second, although the exact causes of incorrect retrievals in RAG are not fully understood, one contributing factor may be the contrastive learning objective, which optimizes dense retrievers by pulling the representations of a query and its relevant documents closer in shared vector space while simultaneously pushing the query apart from irrelevant documents [72] [73.. This objective can lead to mass-seeking behavior, causing the retriever to retrieve passages that match superficial patterns in the input query rather than passages that contain critical hints or precise factual answers [54
Empirical studies have shown that retrievers trained with in-batch negatives or heuristic positives on finite data tend to rely on shallow lexical or semantic similarity, rather than retrieving documents that accurately reflect the knowledge encoded in the corpus [65]. From representational perspective, transformer-based retrievers have been observed to capture more syntactic or surface-level information in lower layers, while higher layers may encode more abstract semantic relations ( 63
Third, existing methods fall short in providing LLMs an opportunity to compensate for missing or unconfident knowledge critical to answering the question. Although several existing reasoning 89 enable LLMs to refine their initial outputs, there is lack of literature on an effective way to complete gaps in knowledge in which LLMs are uncertain or lack sufficient information for the final answer.
strategies 46 69 75
To address these issues, we propose novel RAG framework called cooperative RAG for simple and multi-hop question answering in which a retriever and an LLM work cooperatively with each other by exchanging informative knowledge, and the earlier and later layers of the retriever model work cooperatively with each other to accurately retrieve the most relevant documents to given query. In this framework, we (i) unroll question into multiple cub--qestionss and masked reasoning chain, (ii) retrieve the documents relevant to this unrolled question, (iii) rerank the documents by contrasting layers of the retriever, and (iv) reconstruct the reasoning chain by filling masked entities via the LLM. Our experiments demonstrate that CoopRAG consistently outperforms state-of-the -hetthtrrt QA methods on three multi-hop QA datasets HotpotQA [22. 2WikiMultihoQQA 222, and MuSiQue [66] as well as single-hop QA dataset NaturalQuestions 36 in terms of both the retrieval and QA performances. Our retrieval method achieves up to 5.3% improvement on the multi-hop QA datasets and up 35.2% improvement on the single-hop QA dataset over the current state-of-the-are methods 20). CoopRAG using Gemma2-9B outperforms even prior GPT-4o-mini-basee method. Consequently, CoopRAG can create a bidirectional synergy between a retriever and an LLM, i.e., we effectively draw out the internal knowledge of an LLM which will encourage the retriever to provide the documents highly relevant to the query, and the reranking stage effectively harmonizes the internal knowledge of the retriever, and the retrieval results in turn facilitate the confident reasoning of the LLM, thereby enabling accurate retrieval and reasoning for complex questions.
2 Related Work
for RAG. When questions lack sufficient information, LLMs and retrievers are prone to hallucination and inaccurate retrieval, respectively. Various query augmentation methods have emerged to improve document retrieval performance. Several methods [14
47
70 88 extract
keywords from the question to have an LLM generate higher-level concepts, hypothetical answers, pseudo-documents, or sub-questions to augment the original question. In [45 52
LLMs paraphrase
a question for use in retrieval. In contrast, Baleen [33] retrieves documents using the original question, summarizes these documents, refines the question by combining it with the summaries, and then performs a second retrieval. However, the augmented queries in these approaches may include incorrect information due to the hallucinations of LLMs, which consequently impairs retrieval accuracy and reasoning performance.
Dense Retrieval with Pre-trained Language Models. Dense retrieval has emerged to address the limitations of freneere based methods [37 55 56) in capturing semantic relationships by encoding queries and documents as dense vectors and computing their similarity. Queries and

---

Aeroling-Augmented
Ranking Contrasting
Completion
Conaranda
Question
Stean
Diournent
Enooder
Enbuskiigg
Encoder
RaLaa
Top- Documents
Top- Sub-quedtiona
Queation
Preprccees
Toa-n Documents
Quantion: which country town which Haymond Hoterraen bemr?
Reasoning Chain Complicion
Dusstion)
which
which Robensen
Query Unrolling
quesiions)
b.b-quustoon)
Sub ry queetion 1: In wrich trem was Raymond Rebertoen som? EuB question county the town on Hommerteot locaeed in?
Sub oucstio in which loan was Raymond Robersio bon??
Sub queation What courtyy the lown on Hummuriial localed in?
Unseriain Reasoning
linzerse heeseriing Elsers
Maayoood Roterreent bom Hammerfea]]
TRaymond Rebertzor: lao bor in. Hommerfesr]]
DHammerees
SUJNCERTAINE"
TThe county
Raymond Robertsen boml. 1el "ERLLL
Top xt Dosumenss
Docurien: 13 (Tile Homme fool 8-3 Flymmark countty
Generated LLB intenal knowladge
Uncardeetty
Document 23 (TRkk Ragmord Robosttarr) Raymond Rubattsen... is Notwagian
Unroited Question
['Raymoed Robertierr Yosa born if Harmerdea]] Plemmmreeff.. lastlec Firmmrrk courtyl. [The county of the Eown Poymonre Robeetoon uxx bom! hhoway]]
Instech courtry . wndcch snen WWal courtry.... ...? Raymored Robertaan noes
Figure 1: Overview of CoopRAG, which consists of: (i) Question Unrolling, (ii) Unrolling- Augmented Retrieval, (iii) Ranking by Contrasting Layers (RaLa), (iv) Reasoning Chain Completion, and (v) Reasoning
documents are embedded into single vectors to enable efficient similarity computation [5 25 31 [50] 778. Since such compression introduces representational bottlenecks, ColBERT 32 adopts token-level embeddings. Nevertheless, these methods could not capture distinct types of knowledge encoded across different Transformer layers, leading to distorted similarity scores and thus suboptimal retrieval performance.
57
Prructure-Agmennte RAG. Retrieval methods that exploit relationships between documents and the structural properties of knowledge have gained attention. RAPTOR (58) and Proposition [6] divide documents into proposition-level segments, and recursively embed, cluster, and summarize
them to construct hierarchical representations that capture long-range contextual information. SiR-
eRAG [86] and HopRAG 42 paraphrase complex queries using LLMs, and leverage the paraphrased queries to explore logical connections between document chunks within a knowledge graph (KG). GraphRAG [1] and LightRAG (18) leverage LLMs to extract triples from text, and hierarchically construct KGs to maximize semantic connectivity. HippoRAG [99 and HippoRAG2 [20] build KGs by representing noun phrases as nodes and their relations edges. However, these methods incur high construction costs, and often produce overly dense graphs with numerous irrelevant reasoning
paths.
Geasoning-Enhancee Approaches for Complex QA. Reasoning explicitly across multiple steps has been shown to be beneficial for LLMs to solve complex queries through linear reasoning steps (75). multiple branching paths [83], a graph structure [[] CoK [40] prompts LLMs to generate intermediate knowledge, and utilizes external models to validate that knowledge. In CoQ [89 and question decomposition [53], a question is split into answerable sub-questions to deduct a final answer. However, these approaches do not leverage retrieval-augmentatiee to validate knowledge in which LLMs have low confidence. IRCoT (67 is a model-agnostic multi-step retrieval framework that interleaves each reasoning step with document retrieval for mutual enhancement, and can be easily integrated into our method.
Method
3.1
Overview
In this section, we describe a novel RAG framework for question answering. First, we fine-tune a pretrained encoder for retrieval and reranking (Section (36)). Next, for every document, obtain the [CLS] token embedding produced by the fine-tuned encoder, and store this in vector DB in the preprocessing stage.
FigureMilluse iillustrates the overview of our framework inference, which consists of five stages: (1) in the question unrolling stage, an LLM leverages its internal knowledge to decompose question into
3

---

multiple sub- sub-questions and reasoning chain in which uncertain positions are masked (Section 3.2
(2) the question is augmented with the sub-questions and the reasoning chain, a retriever provides top-n documents relevant to the augmented query in unrolling-augmented retrieval (Section 3.3); (3) we rerank the retrieved documents to obtain top-k (k: < n) documents by contrasting layers the retriever model (Section 3.4): (4) the LLM reconstructs the reasoning chain by filling the masked positions with the factual evidence in the top-k documents (Section 3.5); ; (5) the LLM takes the reconstructed reasoning chain, the question, the sub-questions, and the top-k documents input, and generates the answer for the question.
3.2 Question Unrolling
A question often contains limited information to accurately guide both the relevant document retrieval and the correct reasoning of the LLM. In the question unrolling stage, the LLM takes the input question Q as input, and decomposes Q into (i) set S = [$1, $2, 83, .... s| I of sub-questions (ii) an uncertain reasoning chain, i.e., sequence of triples with masked entities. Inspired by reasoning we allow L.M to generate evidence triples that support the step-by-step thinking and the final answer, but the LLM may have low confidence generating entities of some triples. If so, generating such entities and augmenting the original question with these entities could rather lead to wrong retrievals or hallucinations reasoning. Therefore, we guide the LLM to mask out these entities instead of generating them. Finally, the sub-questions and the uncertain reasoning chain are concatenated with the original question to foam an unrolled question U as follows:
chains [33 69,
U Q
S {5), 82, 83, (1S)
R ((ei,ri, e1), (2, r2, (e)...................... (epr), rr, (FIILL)
where || denotes concatenation, and é;, Ti ef represent the head entity, the relation, and the tail entity, respectively, of the i-th triple in the reasoning chain. Each entity stands for either an uncertainty mask (UNCERTAIN) if the LLM lacks confidence; a question-relater concept otherwise. The tail entity of the last triple in the reasoning chain is designated as the placeholder (FILL), which will be substituted with the answer to the original question in the final reasoning step of LLM.
From the following stages, the LLM and the retriever leverage only internal knowledge about which the LLM is certain. We conduct experiments to validate the effectiveness of question unrolling, which will be discussed in Section 4.6 It worth mentioning that unlike question unrolling, (53) decomposes a question into sub-questions, and iteratively calls the LLM to modify these sub- questions, and CoQ [89] generates sub-questions, and answers the sub-questions until the LLM elicits the final answer. The analysis and prompt for question unrolling are provided in Appendix E.I E.4
and Appendix H.I respectively.
3.3 Unrolling-Augmenter Retrieval
In the arrolling-augmeenn retrieval (UAR) stage, we retrieve top-n documents most relevant to the unrolled question from all documents. For this, the fine-tuned encoder computes token-level embeddings of the unrolled question U, and the document embedding of every document D::
qu, 91...... 14lb Normalize(Encoderi 91 92
q01))
do, d d D) Normalize (Encoder((coddrra d1 d2 ... d()))
where Normalize stands for T-2 normalization, 91...., qui represent tokens in the unrolled question U. and di,..., d Di represent tokens in document D. Let qi and d, denote the embedding of the i-th token in U and D respectively. The [CLS] token embeddings of U and D are represented by qu and dg. respectively.
Next, we retrieve set Dr D2, Da) of top-n documents most relevant to U based on the cosine similarity between qo and do for each document. This similarity- search is performed efficiently by Faiss [3).

---

Average
4)o1
((4..4)
(Ai
1 Layer
t Layer
9: %
Mr 4
Controst!
10th Layer
Contras
%
d)
6th Layer
%
4
d
2nd Layer
21
Unrolled Question
[CLS]] Token
1 Tokion
1D Tokon
Document
Figure 2: Illustration of the Ranking by Contrasting Layers (RaLa) process before optimization, corresponding to Equation 3
3.4 Ranking by Contrasting Layers
Motivated by retrieve-reraak-geneee to retain top-k (k < n) documents in this reranking stage. Factual knowledge has generally been shown be localized partieular layers of the Transformer model 7 48 which in turn can provide an opportunity to take advantage of the multiple layers to tackle the inaccurate retrieval of Transformer-bent based encoders.
pipelines 16 85], we rerank the top-n documents Du
Inspired by DoL.a (7). we propose to rerank the top-k documents based on novel ranking method called ranking by contrasting layers (Rala). As shown in Figure RaL.a la leverages the differences in information representation between the lower and higher layers of Transformer encoder. The lower layers focus on syntactic or surface-level information, while higher layers capture more abstract semantic relations. RaLa assigns higher weights to documents with greater similarity differences between these two layers relative the question. While existing retrievers utilize only top-layer embeddings, RaLa enables document retrieval based on semantic relevance through multi-layered representations.
We divide the entire layers into two to four buckets 77. and randomly select one layer from each bucket. Let set C 2, .... [C] of candidate layers be the set of the selected middle layers, and represents the last layer of the encoder. For each document D E DD7 and each layer 7 E C, the
hidden states, i.e., early exit, of the 1-th layer represented d! .d() (.....))).
In the reranking stage, we rerank every document D Dy regarding the unrolled question U by computing the score between them. Every query embedding of U interacts with all document embeddings of D via MaxSim operator, which computes maximum similarity [2) 57), and the outputs of these operators are averaged across all query tokens. Here, we define the similarity between contextualized embeddings of q; and d, as the maximum gap g(g, dy) between the last layer and the premature layer:
score(U, D) avg/el max
(e(0.......Ip)) Ha,, d)), where g(1,,d)) = max ((q,, (q), d d())) IEC
avg(-) denotes the average operation, and (::) represents the cosine similarity between two vectors. However, the cosine similarity applies to every candidate layer to compute the maximum, which may incur substantial computational overhead. Therefore, in practice, we replace the above score with the average maximum similarity in [32 17], which is now multiplied by the importance of how relevant the document is to the query by applying the dynamic layer selection:
5

---

the reasoning 00 max (q: (q.dd) 1111
score,(U, WU, avg avg.=0
je (e (I)
where the gap-aware wy, U and 1) respectively. The comparative performance analysis between Equations (3) and (4) will be presented in Section 4.6. Further comparisons between different scoring strategies are presented in Appendix E.
wU, is defined as g(go, do) in which and do represent the [CLS] token
The motivation for selecting the layer with the highest gap is to ensure that the model would significantly change its output after the selected candidate layer, and thus have a higher chance to include more query-relevant knowledge that does not exist in the early layers before it, which is detailed by the case study Appendix G.1
3.5 Reasoning Chain Completion and Reasoning
Sometimes, the retriever does not provide sufficient information necessary for the LLM to compensate for missing knowledge, in which the LLM has low confidence during step-by-step reasoning. We claim that the retriever providing this knowledge to LLM can contribute to alleviating hallucinations of the LLM.
In the reasoning chain completion stage, the LLM updates the reasoning chain obtained by query unrolling based on the top-k documents. Given the unrolled question U and the top-k documents, the I.LM fills the uncertainty masks (UNCERTAIN) and the final entity (FILL) chain based on clues in the documents. Subsequently, the LLM generates the final answer, given the input question, the sub-questions, the reconstructed reasoning chain, and the top-k documents. The comparison of different LLM invocation strategies is presented in Appendix E.6 The prompts for reasoning chain completion and reasoning are provided in Appendices H.2 and H.3
respectively.
3.6 Difficulty-Aaare Training of Retriever
Training model primarily on easy examples may lead to overfitting to common or superficial patterns, and harder questions usually correspond to ambiguous, rare, or edge-case scenarios [35 44 59]. To tackle this, we fine-tune the pretrained encoder for retrieval through sample-wist loss reweighting that gives more attention difficult questions which may result in wrong document retrievals.
In finetuning, we unroll each question in the training set, and then we associate that unrolled question with positive document and hard negative document by following ColBERT (32] With mini- batch B of size b, let U {U1, UU..... ......,Us } be set of the unrolled questions in a mini-batch, and let D = (D,, D2..... D25) be a set of all documents associated with the unrolled questions, where D, and Do+s denote the positive and hard negative documents for U3 respectively. For each
we compute loss below:
(U, D.)/T)
CB= = Σ av, D.), where C(U,, = -log EDED eXp(score,(U., D)/T)
(5)
U.EU
where the weight ou,, stands for the difficulty of the unrolled question U, ensuring that queries with higher difficulty receive larger penalties. InfoNCE loss LUU,, D:) for Us. where is hyperparameter that controls the importance of negative samples, and score is defined in Equation 4
In our experiments for multi-hop question answering without question unrolling, we observe that the retrieval performance deteriorates as the number of ground-trutt (GT) documents relevant to a question increases, and the number of the sub-questions is proportional to the number of GT documents, which indicates that the LLM tends to decompose a complex multi-hop question into more sub-questions. Hence, we set the weight oz, proportional to the number of sub-questions in U;:
au, log 1 + Sv, I)
(6)
where Sv, represents set of sub sub-questions in U,. In this way, we adjust the importance of sample- wise loss, thereby encouraging the encoder to learn more from challenging cases, which are often more informative and critical for generalization.

---

Experiments
4.1 Experimental Setup
Table Dataset statistics (test sets)
Category
Single-hop QA
Multi-hop QA
Factual QA
NQ
HotpotQA
MuSiQue
2Wiki
FreshQA
SimpleQA
Questions 1,000
1,000
1,000
1,000
553
1,000
Documents 9,633
9,811
11,656
6,119
11,062
20,686
Table shows the statistics of the widely-used datasets adopted in our experiments: HotporQA 82 MuSiQue [66], and 2WikiMultihopQA (2Wiki) [22) for multi-hop QA, and NaturalQuestions
(NQ) [36] for single-hop QA. The test set for each multi-hop QA dataset contains 1,000 samples drawn from HippoRAG, and that for NaturalQuestions consists of 1,000 samples randomly selected from about 27,000 test instances provided by REAL 74 ensure fair comparison with baselines. Futhermore, we evaluate on recent factual QA benchmarks, SimpleQA (76 and FreshQA [68].
which reflect more up-to-date and challenging settings. MPNet 60) is employed as an encoder for retrieval and reranking, and Gemma2 (9B, 27B). Llama3.3 3.B--0B [17] and GPT-4o-mini [51]
are used as LLMs. Notably, Llama3.3-700 denotes the Llama3.3 7-0-...--77-B model throughout all experiments. Detailed experimental setup is described in Appendix [ Baseline methods are explained Appendix In addition to these single-step retrieval methods, we also include the multi-step retrieval method IRCoT 667 baseline. The complete results of multi-step retrieval and QA are provided in Appendices C.3 and C.4 respectively.
4.2 Retrieval Performance Before and After Fine-Tuning
Table 2: Retrieval performance comparison across single-step methods. The best and second-best performances presented in bold and underlined, respectively.
Models
Multi-hop QA
Single-hop QA
HotpotQA
MuSiQue
2Wiki
NQ
R@2
R@5
R@2
R@5
R@2
R@5
R@2
R@S
HippoRAG2 LLlama3.37705
83.5
56.1
74.7
76.2
90.4
45.6
78.0
HippoRAG2 (GPT-4o-mini)
80.5
95.7
53.5
74.2
74.6
90.2
44.4
76.4
SiReRAG (GPT-4o-mini)
80.0
94.8
52.5
64.9
60.6
67.6
42.3
72.5
HopRAG (GPT-4o-mini)
81.1
96.0
53.7
66.8
61.7
70.1
43.9
74.4
CoopRAG (Gemma2-99)
87.9
95.6
59.4
75.5
80.1
96.7
71.6
CoopRAG (Gemma2-277)
88.3
96.6
59.4
75.7
80.8
97.2
72.8
89.5
CoopRAG (Llama3.3-70B)
86.9
96.6
58.2
75.3
80.6
96.3
77.2
90.8
CoopRAG (GPT-40-mini)
88.8
96.8
59.6
75.7
80.4
96.6
80.8
92.1
Table 2 compares the performance of single-step retrieval for ours and the latest basclines. The complete results for all baselines compared with ours are found in Appendix C..] CoopRAG (GPT- 4o-mini) outperforms all of the competitors across every benchmark. On HotpotQA, it achieves the highest Recall @ and Recall @ 5. On NaturalQuestions, it improves upon HippoRAG2 by 35.2% in Recall @2 and 14.1 1% in Recall@ These results show that unrolling-augmente the masked reasoning paths benefits both multi-hop and single-hop questions. CoopRAG (Gemma2- 9B) even surpasses HippoRAG2 with the much larger LLM, i.e., Mamaa.3-70B. demonstrating its effectiveness under limited computational resources. We conduct an analysis demonstrating retrieval efficiency in Appendix
augmented retrieval based on
4.3 QA Performance
Table presents single-step QA performance. CoopRAG (GPT-4o-mini) attains state-of-the-art results on most datasets. Even with Gemma2-9B, it outperforms all baselines, and achieves least 15.2% higher EM on NaturalQuestions, Reranking by contrasting layers and completing the masked

---

Table 3: QA performance comparison single-step methods. The best and second- perfor- mances are denoted in bold and underlined, respectively.
Models
Multi-hop QA
Single-hop QA
HotpotQA
MuSiQue
2Wiki
NQ
EM
EM
EM
EM
Hip (Llama3.3-708)
62.7
75.5
37.2
48.6
65.0
71.0
48.6
63.3
HippoRAG2 (GPT-4o-mini)
56.3
71.1
35.0
49.3
60.5
69.7
43.4
60.0
SiReRAG (GPT-4o-mini)
61.7
76.5
40.5
53.1
59.6
67.9
42.4
58.7
HopRAG (GPT-40-mini)
62.0
76.1
42.2
54.9
61.1
68.3
42.9
59.2
CoopRAG (Gemma2-90) 2-BB)
64.4
78.1
52.2
65.2
70.0
78.1
63.8
72.7
Co (Gemma2-27BB
64.9
79.5
52.8
66.7
71.7
79.0
67.3
75.5
AG (3lama3.3-70BB
64.7
79.0
52.6
66.6
71.2
78.8
70.9
80.3
CoopRAG (GPT-4o-mil -onimi
65.6
78.9
52.3
67.1
71.7
79.2
72.0
82.3
Table 4: QA Performance comparison on SimpleQA and FreshQA datasets. performances are denoted in bold and underlined, respectively.
The best and second-best
Methods
SimpleQA
FreshQA
EM
F1
EM
F1
Correct (f)
Incorrect (L)
Not Attempted (1)
Hi2
48.2
55.0
21.3
29.5
297
HopRAG
50.2
58.2
21.1
28.7
233
275
CoopRAG
58.3
67.6
26.6
35.3
283
250
20
reasoning chain based on the documents retrieved by UAR enhance the QA accuracy. More detailed
analysis on QA performance is included in Appendix C.2
We also evaluate CoopRAG on two recent factual QA benchmarks, SimpleQA and FreshQA, where achieves 16.1% 1% and 26. 1 higher EM than HopRAG, respectvely Since FreshQA includes many sentence-lever and open-form answers, we follow the ChatGPT grader in SimpleQA (76), which labels predictions as Correct if the prediction fully contains the ground truth without contradiction, Incorrect if any contradiction is present, and Not Attempted if the necessary information is missing not contradicted. Under this metric, Co produces 283 correct answers, while HopRAG achieves 233 and HippoRAG2 achieves 225, demonstrating clear improvements in factual correctness.
4.4 Impact of Gap-Aware and Difficulty-Aaare Weights
Table 5: Ablation study the effect of difficulty-wware weights (au and wtr showing retrieval performance with and without each weight.
Category
HotpotQA
MuSiQue
2Wiki
R@2
R@5
R@2
R@5
R@2
R@5
w/o AUADD
84.4
94.3
56.3
72.7
76.1
92.3
wfo og 85.6
87.8
95.2
58.2
74.6
79.9
96.0
wfo WU,D
94.5
57.0
73.8
77.7
93.5
CoopRAG 88.1
95.9
59.6
75.7
81.4
96.6
We conduct an analysis evaluate the impact of the gap- and difficulty-aware weights. Table Fable5sshows retrieval performance with and without each weight. Removing both au and WT,D results in marked decline performance. Recall @2 falls from 8..4% 76.1% 1% 2WikiMulthhopQA. Between the two weights, removing WU,D produces larger performance drop. On MuSiQue, dropping Wo,D reduces Recall @2 from 59.6% to 57. 57.0%, whereas dropping au leads smaller decrease from 59.6% to 58.2%. These results indicate that reweighting the query-document similarity via the gap-aware weight, which contrasts the hierarchical knowledge of LMs, plays more significant role in retrieving correct documents. Nevertheless, removing also degrades performance across all datasets. This demonstrates that the number sub-questions reflecting the question difficulty contributes to retrieval performance gains. We further examine alternative weighting methods in Appendix E.7

---

4.5
Similarity Score Comparison based on Gap-Aware Weight in RaLa
Table 6:
The comparison of the similarity score differences to validate the effectiveness of WU,D.
showing a relative increase of 32.16 in (pos) (rand. neg.).
Weight
Score Difference
HotpotQA
MuSiQue
2Wiki
(pos) (rand. neg.)
0.2951
0.2109
0.2882
who wU,
(pos) (distractor)
0.2161
0,1952
0.2008
(distractor) (rand. neg.)
0.0790
0.0157
0.0874
(pos) - (rand. neg.)
0,3900
0.3847
0.4004
w/ WO,D
(pos) (distractor)
0,2388
0.2139
0.2306
(distractor) (rand. neg.)
0.1512
0.1708
0.1698
We conduct experiment comparing similarity scores to validate the effectiveness of the gap- aware weight in distinguishing positive documents from their distractor and random negative (RN) documents. Distractor documents are harder to distinguish from positive documents than RN documents. For our frameworks with and without the gap-aware weight, Thole three types of differences: (pos) (rand. neg.) is the difference in the average similarity between question-positive document pairs and question- RN document pairs; (pos) - (distractor) is that between question-positive document pairs and question-distractor document pairs; (distractor) (rand. neg.) is that between inestion-distracte document pairs and question-RA RN document pairs. Applying WT.D increases all score differences all datasets, indicating enhanced ability distinguish between positive and negative documents. The relative increase in (pos) (rand. neg.) is 32.16% on HotpotQA, 82.41 441% % on MuSiQue, and 38.93 98.93% on 2WikiMultihopQA. The difference (distractor). (rand. neg.), the relative difficulty of distinguishing distractors from RNs versus positives. substantially increases by 91.39% on HotpotQA, 987.9% on MuSiQue, and 94.28% on 2WikiMulihopPA. These results demonstrate that Ral.a effectively separates truly semantically relevant documents from their distractors, and distractors from RNs, which difficult without RaLa due locally similar keywords in all of these documents.
4.6 Impact of Uncertainty Mask
Table 7: Impact of the uncertainty mask the retrieval and QA performances (Recall @2 and EM)
Category
HotpotQA
MuSiQue
2Wiki
Retrieval (R@2)
QA
Retrieval (R@2)
QA (EM)
Retrieval (R@2)
QA (EM)
(EM)
w/o (UNCERTAIN)
86.9
60.9
55.2
47.2
73.8
65.6
w/ (UNCERTAIN)
88.1
65.6
59.6
52.3
80.4
71.7
Table 8: Entropy decrease ratio based on the uncertainty mask
Category
HotpotQA
MuSiQue
2Wiki
(FILL) Generation w/ & w/o (UNCERTAIN)
5.13
6.32
5.62
Before & After (UNCERTAIN) Generation
8.59
8.88
9.92
We analyze the effect of the uncertainty masks by comparing "w/ (UNCERTAIN)" (CoopRAG) and "w/o (UNCERTAIN)" i.c the variant that does not generate uncertainty masks in question unrolling and does not perform reasoning chain completion, the retrieval and QA performances. Table presents their Recall @ and EMM w/ (UNCERTAIN) improves retrieval performance by 1.4% on HotpoiQA, 8.0% on MuSiQue, and 8.9% on 2WikiMultihopQA. QA performance exhibits similar trends, with EM accuracy increasing across all datasets. These findings demonstrate that the uncertainty masks effectively mitigates errors in uncertain information generated during LLM reasoning, implying that masking out low-confidence tokens prevents negative effects on subsequent inference steps.
In Table X we measure how many times the average entropy decreases at the moment of generating (FILL) token when using (UNCERTAIN) compared w/o (UNCERTAIN), and how many times the average entropy decreases right after (UNCERTAIN) generation compared to the moment of generating

---

(UNCERTAIN). All datasets exhibit substantial entropy differences, with MuSiQue showing 6.32- fold decrease generating (FILL) by using uncertain masks, and 2WikiMulthhoQAA decreasing by factor of 9.92 right after generating uncertain masks. Explicitly marking uncertainty may prevent hallucinations, thus enhancing document retrieval and subsequent reasoning.
4.7 Comparison of Retrieval Performance
Table 9: Single-step retrieval performance comparison of different retrievers HotpotQA, MuSiQue, and 2WikiMulthhpQAA, evaluated with and without fine-tuning, The best and second- best perfor- mances are denoted in bold and underlined, respectively.
Methods
HotpotQA
MuSiQue
2WikiMuliihopQA
R@2
R@S
R@2
R@S
R@2
R@S
Without Fine-tuning
Contriever
57.3
74.8
32.4
43.5
55.3
65.3
ColBERTv2
64.7
79.3
37.9
49.2
59.2
68.2
ReSCORE
68.2
80.9
38.6
49.1
62.2
73.3
Ral.a
73.9
86.1
45.9
60.7
64.6
75.0
With Fine-tuning
Contriever
63.4
80.5
36.8
44.9
68.2
ColBERTv2
78.2
89.3
52.6
68.5
82.1
ReSCORE 82.3
92.6
54.9
72.3
78.8
95.1
RaLa 88.8
96.8
59.6
75.7
80.4
96.6
Table compares the retrieval performance RaLa and recent baselines before and after fine-tuning. Even without fine-tuning, RaLa consistently outperforms Contriever, ColBERTv2, and ReSCORE across all datasets. On HotpotQA, for example, Ral.a achieves an @2 of 73.9%, clearly higher than 68.2% of ReSCORE, with similar advantages observed on MuSiQue and 2WikiMultihopOA.
After fine-tuning, the performance gap becomes more evident. RaLa reaches 88.8% R@2 and 96.8% R@5 on HotpotQA, yielding 7.9% and 4.5% improvements over ReSCORE, respectively, On MuSiQue and 2WikiMultihopO.. RaLa also consistently outperforms ReSCORE, confirming its superior retrieval capability and adaptability across diverse QA benchmarks.
5 Conclusion and Limitation
We propose novel QA framework, where retriever and LLM cooperatively exchange mutually useful context, and multiple layers of the retriever jointly contribute to accurate document reranking. Question unrolling allows the LLM identify positions in which hallucinations are likely to occur, UAR enables LLM to deliver the useful query required to compensate for its uncertain knowledge to the retriever, contrastive reranking enables the retriever to provide appropriate documents that can boost the confidence of L.M,, and LIM can confidently answer the question via reasoning chain completion. Extensive experiments on single-hop and multi-hop QA benchmarks demonstrate that CoopRAG achieves state-of-the-ar thett-rrtt performance in retrieval accuracy and QA quality.
Despite our achievements, pretrained LMs cannot embed long documents due to their sequence length constraints, and like ColBERT, our dense retrieval method might incur rising computational costs for MaxSim operations as the number of tokens grows. Further validation on KBQA [6] 84 and
Nomain-specifee datasets B 28 29), and extending our framework beyond passage-based retrieval to knowledge graph QA remains promising direction for future research.
Acknowledgments and Disclosure of Funding
This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2020-11201333, Artificial Intelligence Graduate School Program (Hanyang University)); and by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. RS RS-2025-25424677
10

---

References
[1] Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi. Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyezyk, and Torsten Hoefler. Graph of thoughts: Solving claborate problems with large language models. Proceedings of the AAAI Conference on Artificial Intelligence, 16821))17777226666611 March 2024. ISSN 2159-5399. doi: 10. 10.1669/aaii.v38811 29720. URL (ttLLtttt//d..cco doi org/ 10. 1609/ awa.aaaaaaa........a w38i16........ 29720
[2] Shuqing Bian, Wayne Xin Zhao, Jinpeng Wang. and Ji-Rong Wen. A relevant and diverse serrivaa--cnhanee data augmentation framework for sequential recommendation. CIKM '22, page 2923-2932, New York, NY, USA. 2022. Association for Computing Machinery. ISBN 978145039236.. doi: 10.1145/351 350..555707. URL http::t/tp///ooio.. 0rg/10.1145/3611808 3557007
[3] Vittorio Castelli, Rishav Chakravarti, Saswati Dana, Anthony Ferritto, Radu Florian, Martin Franz, Dinesh Garg. Dinesh Khandelwal, Scott McCarley, Mike McCawley, Mohamed Nasr, Lin Pan, Cezar Pendus, John Pitrelli, Saurabh Pujar. Salim Roukos, Andrzej Sakrajda, Avirup Sil, Rosario Uceda-Sosa, Todd Ward, and Rong Zhang. The techqa dataset, 2019. URL hattps://aaxii org/ / https//911.0999991 02984
[4] Liyi Chen, Panrong Tong. Zhongming Jin, Ying Sun, Jieping Ye. and Hui Xiong. Plan-on-graph: Self-correcting adaptive planning large language model on knowledge graphs. In A. Glober- son, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang. editors, Ad- vances in Neural Information Processing Systems, volume 37, pages 37665-37691. Curran Asso- ciates, Inc., 2024. URL /ttttps:/pproceeddieee 42556886601666777805447777683663 9b9 Paper- Conference. pdf
[5] Lu Chen, Ruqing Zhang, Jiafeng Guo, Yixing Fan, and Xueqi Cheng. Controlling risk of retrieval- augmented generation: A counterfactual prompting framework. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 2380-2393, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 101186/11/2222.ffna http://..fininng-eeneeee emnlp. 133.. URL lattps://aclanthoolgg 2rrg/0224.0 findings- emnlp
133/
[6] Tong Chen. Hongwei Wang. Sihao Chen, Wenhao Yu, Kaixin Ma, Xinran Zhao, Hongming Zhang, and Dong Yu. Dense X retrieval: What retrieval granularity should we use? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference Empirical Methods in Natural Lan- guage Processing, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: m45..885. URL URL http:://tcccc.tt htt///occlntooooyyoooooooo wrgg:oorg///0 e00024..mlllp.....mmmmmmmm emnlp- maimlinmm....m.. maim.
10. 11168//v120244.mmml
[7] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengeheng Hc. Dola: Decoding by contrasting layers improves factuality in large language models, 2024. URL httt/ttttss/:rrrt org/
abs/2309. 03883
[8] Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio. editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 8493-8502, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 121553//1/2222..ccoooo......
httpss/:: ac/ aclaanhologg org// hetg//022.aaae acl- acl-long....... -o1..111.111////
[9] Wietse de Vries, Andreas van Cranenburgh, and Malvina Nissim. What's so special about BERT's layers? closer look at the NLP pipeline in monolingual and multilingual models. In Trevor Cohn, Yulan He, and Yang Liu, editors. Findings of the Association for Computational Linguisticy: EMNLP 2020, pages 4339-4350, Online, November 2020. Association for Computational Linguistics. doi: 10. 18653/v1/2020 Bedding-eemnl..33 URL httbttts///aaaaatoolloeee wwt.:llooyyyyyyooooooo//e// org/2020. findings- Stndings-emnlp..0
[10] Dario Di Palma. Retrieval-augmented recommender system: Enhancing recommender systems with large language models. In Proceedings of the 17th ACM Conference Recommender Systems, RecSys '23, page 1369-135 1373, New York, NY, USA, 2023, Association for Computing Machinery, ISBN 9798400702419, doi: 10.1 /4////04411.36088868
URL URL httph:tt::/// /ttttps://do..rggge http///o...oroo...///// 1145//3441154//5499999 3608888
[11] Jialin Dong, Bahare Fatemi, Bryan Perozzi, Lin F. Yang. and Anton Tsitsulin. Don't forget to connect! improving rag with graph-based based reranking, 2024. URL httL/httts///axii.. http://aww.....rr///////////// 18441 18414
[12] Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph rag approach to query- focused summarization, 2025. URL https: //arxiva..v.///// org/ httg/bb///0 absp/b///... 16130
11

---

[13] Jinyuan Fang. Zaiqiao Meng. and Craig MacDonald. TRACE the evidence: Constructing knowledge- grounded reasoning chains for retrieval-uugmented generation. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1172-8494, Miami, Florida, USA, November 2024, Association for Computational Linguistics. doi: 10. 10555////2022.fndiiing emn.p.496. URL http//ttt/saaaattooooo// .rgg/ 2024. 200...f.iniii findings- emmlp
496/
[14] Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamic Callan. Precise zero-shot dense retrieval without relevance labels. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61 Annual Meeting of the Association for Computational Linguisties (Volume I: Long Papers), pages 1762- 1777, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v120023..al acl- aclloooo... (tKLL,ttta://acaaatttt
www/oo//////....ooo 2023. acl-long. 99/
[15] Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE: Simple contrastive learning of sentence embed- dings. In Maric-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894-6910, On- line and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.1 18653/v 1/2021 w/w.:/.eemll-pp...... mainlp-main.5522 URL pattps://aclla httph/ooccatttooggggyoooo 222g/2021... emnlp- main. m52m.552/
[16] Michaal Glass, Gactano Rossicllo, Md Faisal Mahbub Chowdhury, Ankita Naik, Pengshan Cai, and Alfio Gliozzo. Re2G: Retrieve, rerank, generate. In Marine Carpuat, Marie-Catheriin de Marnefle, and Ivan Vladimir Meza Ruiz, editors. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2701-2715, Seattle, United States, July 2022, Association for Computational Linguistics. doi: 10.18653/w1/2022.naa. 194. htl.ltttps:/aaaaaatttoo
org/////////// 2022, naacl - main. ma1n.194/
[17] Aaron Grattafiori, Abhimanyu Dubey, and al. The Ilama 3 herd of models, 2024. URL https
aarxiv..rgg/bas/2407.77 21782
[18] Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. Lightrag: Simple and fast retrieval- augmented generation, 2024. URL URL atttps://arxils orggoogg//b abs 2 221004410.000 05779
[19] Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. Hip- porag: Neurobiologically inspired long-term memory for large language models. In A. Glober- son, L. Mackey, D. Belgrave. A. Fan, U. Paquet, Tomezak. and C. Zhang, editors, Ad- vances Neural Information Processing Systems. volume 37. pages 59532 59569, Curran Asso- ciates. Inc., 2024. URL p/ttoee//procceeingg.... neurips. www..ps.ccpppprrrrroooo faapperfiieesppper//0/////// -aperr Conference.rrreeoeeeoeeeeeo pdf
[20] Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi. Sizhe Zhou, and Su. From rag to memory: Non- parametric continual learning for large language models, 2025. URL /ttpss/::///axi http:/awwwii....r/////// abs/2502 14802
[21] Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V. Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson. and Bryan Hooi. G-retriever:
generation for textual graph understanding and question answering. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 132876- 132907. Curran Associates, Inc., 2024. URL htttps::/pproceeeiiiiiee www.snnwwu..ueeeeeeeeee http/appr_ffleeee Pepee
pdf
[22] Xanh Ho. Anh Khoa Duong Nguyen. Saku Sugawara, and Akiko Aizawa. Constructing multi- QA dataset for comprehensive evaluation reasoning steps. In Donia Scott, Nuria Bel, and Chengging Zong. editors, Proceedings of the 28th International Conference on Computational Linguistics, pages 6609-6625. Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics. doi: 1
m80. URI htRl..ttts::/// http:///aaaaatthotoo.oo org/2020..oo.. 000002..col000 coling- ingimmainnnn... 580/
[23] Lei Huang, Weijiang Yu, Weitao Ma. Weihong Zhong, Zhangyin Feng. Haotian Wang. Qianglong Chen, Weihua Peng. Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions Information Systems, 43(2):1-55, January 2025. ISSN 1558-2868. doi: 10.1145/3703155. (RL http://dxxxxa doi. dwi..oo/i/ooo11.. 10 1146/3703155
[24] Wenyu Huang. Guancheng Zhou, Hongru Wang, Pavlos Vougiouklis, Mirella Lapata, and Jeff Z. Pan. Less is more: Making smaller language models competent subgraph retrievers for multi-hop KGQA. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15787-15800 Miami, Florida. USA, November 2024. Association for Computational Linguistics. doi: 18.18653/v 1/2024. 1/2024.iinding........ wwwdiing--emml..22222 em2.p.927. URL Utt/https:/ //c/ aclloooloooooo org/2024 findings- perddings-emnll.9e emnlp.927/
12

---

[25] Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning. Trans. Mach. Learn.
Res., 2022, 2021. SttLLhhtps::// httppp/s/aap..emaatiicchhooooo D :ttp:/ooorps11::9999777 249097975
[26] Andrew Jesson, Nicolas Beltran-Velez, Quentin Chu, Sweta Karlekar, Jannik Kossen, Yarin Gal, John P. Cunningham, and David Blei. Estimating the hallucination rate of generative ai. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Ad- vances in Neural Information Processing Systems, volume 37, pages 31154-31201. Curran Asso- ciates, Inc., 2024. URL https: htpp://///ooceeeengaaa nwwrips. wwwp:per..pp.pcccc/ www.ffllleapppeee wtt.l//ppppr//22//////.a ffle/ (0215ffc66e3370444ttobbbooooo/oe Papeee Conference. Paper-Cofeerrreeeeeeeee pdf
[27] Ziwei Ji. Nayeon Lee, Rita Frieske. Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii Bang. Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Comput. Surv. 55(12), March 2023. ISSN 0360-0300. doi: 10. 145/3571730. URL https://doi. www..o..g....... 1145/ 3571730
[28] Di Jin. Eileen Pan, Nassim Oufattole, Wei-Hung Weng. Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 1 11(14), 2021. ISSN 2076-3417. doi: 10.3390/app1 146421. BtRphttp::/ //wp:/://wwwww// http::/wwwww.pppiwppt com/ 111163317/11111///11
[29] Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A dataset for biomedical research question answering. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Lanuugee Processing (EMNIP-IJCNI.P)) pages 2567-2577 Hong Kong, China, November 2019. Association for Computational Linguisties. doi: 10. 18653/v1/D19 1259.
URL htt.:asss/aaaaaaatolllaeoo
Stg/D19-1259)
[30] Jeff Johnson, Mathhijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transuc- tions on Big Data, 7(3):535 547, 2021. doi: 10. 1091109// 201522925.
[31] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, Online, November 2020. Association for Computational Linguistics. doi: 10. 00018653/112022.emmmm m50n.550. URL htttt:://///// a/abaass//cclathooogg
org/2020. emnlp main. 550/
[32] Omar Khattab and Matei Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over bert. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, page 39-48, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450380164
doi: 10.1145/39727113440000
URL https
//doi.org/10 11tp//.00114/33/77/111 3401075
[33] Omar Khattab, Christopher Potts, and Matei Zaharia. Baleen: robust multi-hop reasoning at scale via condensed retrieval, In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '2,, Red Hook, NY, USA, 2021. Curran Associates Inc. ISBN 1381713845393.
[34] Minsang Kim, Cheoneum Park, and Scung Jun Back. QPaug: Question and passage augmentation for open-domain question answering of L.MMs. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNIP 2024, pages 9024-9042, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 1/2544 Settings-emmlp.527.. URL (tRLhttps:/// http:////....aoooooooog////....e eenlinngs-emnl......
[35] Youmin Ko, Hyemin Yang, Taeuk Kim, and Hyunjoon Kim. Subgraph-aware training of language models for knowledge graph completion using structure- contrastive learning. In Proceedings of the ACM on Web Conference 2025, WWW '25, page 72-85, New York, NY, USA, 2025. Association for Computing Machinery, ISBN 9798400712746,
doi: 10. 041144//666440..7149000
URL https
//doi. //doi.org/10 100.///0...1//311101711111050000 93741413396100009449
[36] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming- Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics. org/ 019-1026/
7:452 466, 2019. doi: 10.1 a 00276.
13

---

[37] Thomas K Landauer, Peter W. Foltz, and Darrell Laham and. An introduuction to latent semantic analysis. Discourse Processes, 25(2-3):259-285 284, 1998. doi: 10. 00.108000163355399995555500 URL https://doi org/10. 000010880000338839995550222
[38]
Chankyu Lec, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shocybi, Bryan Catanzaro, and Wei Ping, Nv-embed: Improved techniques for training Ilms as generalist embedding models, 2025, URL
2405. 17428
[39] Mufei Li. Siqi Miao, and Pan Li. Simple is effective: The roles of graphs and large language models in knowledge-graphe grae---raab-beeddd based retrieval- -augeeete generation. In The Thirleenth International Conference on Learning Represeniations, 2025. URL htttttt//////ereeeeee
[40] Xingxuan Li, Ruochen Zhao, Yew Ken Chia, Bosheng Ding, Shafiq Joty, Soujanya Poria, and Lidong Bing. Chain-of-knowledee knowledge: Grounding large language models via dynamic knowledge adapting over heterogeneous sources, 2024. URL.Itttps:/arriiv.oo wttpaaiwww....aaaaearrrrr abrrg//bb/2305... 13269
[41] Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning, 2023. URL https: h /ttp:///axvvv.v.. org abs/ 2308 03281
[42] Hao Liu, Zhengren Wang. Xi Chen, Zhiyu Li, Feiyu Xiong, Qinhan Yu, and Wentao Zhang. Hoprag: Multi-hop reasoning for logic-aware aenrieval-ngmmented generation, 2025. URL/tttss///arxx org/ abs/2502 12442
[43] Jie Liu and Barzan Mozafari. Query rewriting via large language models, 2024, URL htttp://aaaa: org/ aba/ http//abb/22.... 0900009000000
[44] Peng Lu, Abbas Ghaddar, Ahmad Rashid, Mehdi Rezagholizadeh, Ali Ghodsi, and Philippe Langlais. RW-KD: Sample-wise loss terms re-weighting for knowledge distillation. Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Findings of the Association for Com- putational Linguistics: EMNLP 2021, pages 3145-311 3152, Punta Cana, Dominican Republic, November URL
2021. Association for Computational Linguistics. doi: 10. 10.18653/v12021. findings--mmlpp ottpa://aclanthology wwwpellogyyoo//////e arg/2021- 10002..ffnniii findings-emnann emnlp.2700
[45] Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. Query rewriting in retrieval-augmenntr large language models. In Houda Bouamor, Juan Pino, and Kalika Bali. editors. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303-5315, Singapore. December 2023. Association for Computational Linguistics. doi: 10.18653/1120223..om .em.l..e.mppppppm. m22n.322. URL
org/ 2023. emnlp- main. main.322/
[46] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: iterative refinement with self-feedback. In Proceedings the 37th International Conference on Neural Information Processing Systems, NIPS '33, Red Hook. NY, USA, 2023. Curran Associates Inc.
[47] Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, and Weizhu Chen. retrieval for open-domain question answering. In Chengqing Zong, Fci Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computa- rional Linguistics and the 1thh International Joint Conference Natural Language Processing (Volume I: Long Papers), pages 4089-41 4089-4100, Online, August 2021. Association for Computational Linguistics. doi: org/ 2021 acl- 316/
10. 10.18633/1/20211 10. wow..oog............m.....
[48] Kevin Meng. David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual as- sociations in gpt. In S. Koyejo. S. Mohamed, A. Agarwal, D. Belgrave. K. Cho, and A. Oh, edi- tors, Advances in Neural Information Processing Systems, volume 35. pages 17359 17372. Curran As- neurips. wwtpurissccpppprrraae. /ttpr//ileeeee/pe//////////// sociates, Inc., 2022. URL https: Conference. .dff
Paper
[49] Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang. Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative representational instruction tuning, 2025. http::ttt///arri..rrrr abs/ 2402. 09906)
[50] Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernández Ábrego, Ji Ma, Vincent Y. Zhao, Yi Luan, Keith B. Hall, Ming- Wei Chang, and Yinfei Yang. Large dual encoders are generalizable retrievers, 2021. URL httpst/:::// httppps:///rxiii atg/abs/2112 07899
14

---

[51] OpenAl. Josh Achiam, Steven Adler, and et al. Gpt-4 technical report. 2024. URL https: org/ 0ba/2303..g 00d/2333088774
[52] Wenjun Peng. Guiyang Li. Yue Jiang, Zilong Wang. Dan Ou, Xiaoyi Zeng, Derong Xu, Tong Xu, and Enhong Chen. Large language model based long-tail query rewriting taobao search, 2024. URL
htte//aae///// 2311/03/3333 03758
[53]
Ansh Radhakrishnan, Karina Nguyen, Anna Chen, Carol Chen, Carson Denison, Danny Hernandez, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilè Lukošiūté, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Sam McCandlish, Sheer El Showk, Tamera Lanham, Tim Maxwell, Venkatesa Chandrasekaran, Zac Hatfield-Dee Doddds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, and Ethan Perez. Question decomposition improves the faithfulness of gener-gee reasoning, 2023. URL https: / //tpxvaaxvvvv/// www.:ogg/////aeeeeeoo 11768
[54]
Ori Ram, Liat Bezalel, Adi Zicher, Yonatan Belinkov, Jonathan Bcrant, and Amir Globerson. What are you token about? dense retrieval as distributions over the vocabulary. In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 2481-2498, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 1/253/3///23.. 2023. aacllllng 140/
URL httttp::/aaaaaahllllaaaa org
[55] S. E. Robertson and S. Walker. Some simple effective approximations to the 2-Poisson model for proba- bilistic weighted retrieval, page 345-354. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1997. ISBN 1558604545.
[56] Gerard Salton and Christopher Buckley. Term-weighting approaches in automatic text retrieval. Inf. Process. Manage., 24 August 1988. ISSN 0306-4573. doi: 00.101660006-457748889990----000 URL http:/tttt/:/ooooootttt http:///daa..rrr..1..oo1//006660 4573 (88) 90021-0
[57] Keshav Santhanam. Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. ColBERTv2: Effective and efficient retrieval via lightweight late interaction. In Marine Carpuat, Marie-Cathering de Marneffe, and Ivan Vladimir Meza Ruiz, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3715-3734, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10. 1...6663/11/2222.mmdmmmmaa
URL httph:://accanttholggeooo wwwpeehoooooooog//2222 or2/2022 saaacl-main.
[58] Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. Raptor: Recursive abstractive processing for tree-organized retrieval, 2024. URL htttt:///arxxi org/ abs/2401. 18059
[59] Minscok Son, Inyong Koo, Jinyoung Park, and Changick Kim. Difficulty- balancing margin loss for long-tailed recognition. Proceedings of the AAAI Conference on Artificial Intelligence, (9(1))200522- 20530, Apr. 2025. doi: 10. wtt.//aai.v9999..3... article/vies//4200
URL https:///oo
index.ndee.... .thp/ /
[60] Kaitao Song. Xu Tan, Tao Qin. Jianfeng Lu, and Tie- Liu. Mpnet: masked and permuted pre-training for language understanding, In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS 200 Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546
[61] Alon Talmor and Jonathan Berant, The web as knowledge base for answering complex questions. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the A Association for Computational Linguistics: Human Languag Technologies, Vol- ume (Long Papers). pages 641-151, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10. 1e118863/v1/ 1/N18-1II1--- 1059. /ttt:tt:: actp//olonnttooooyyy wwwphhoooooooooooooooooooo 1ttg/N88-1000 1059/
[62] Gemma Team, Morgane Riviere, Shreya Pathak, and al. Gemma 2: Improving open language models at practical size, 2024. URL http:/ttsss////aasaa /aaxivv wttpw//.wrr/a////////////// 20110/000000101808801111111
[63] Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NI.P pipeline. In Anna Korho- nen, David Traum, and Lluís Marquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593-4601, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 11111885////11191111 1452. URL http::/:///caaaatoooooooooooooo org//19-1452
[64] Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. BEIR: heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-ffth Confer- ence on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL
15

---

[65] Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fithh Confer- ence on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https://openreviice htt.r//ew....eoorro/aaoooodoo.a.
[66] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539-554, 2022, doi: (0.1162/tac_a a_005, https:/tts///acaaannee tacl- 1.31/
www.oog..oooooooooooo 2xgg/022
[67]
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of- thouttt reasoning for knoeledge-intensiee multi-step questions. Anna Rogers. Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 10014-10037, Toronto, Canada, July URL https?
2023, Association for Computational Linguisties. doi: 10. /22/33//112222.alloooo
/aaclanthology.oo. .og. 2222/222..222 acl-long..50. 557/
[68] Tu Vu, Mohit lyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoe Le, and Thang Luong. FreshLLMs: Refreshing large language models with search engine augmentation. In Lun-Wa Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 13697- 13720, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.183/////20244 1.0...iin/.....nnniiiin ac1.813. URLhttps //aclanthology. wrg. 2024. foss.findingg findings-acl. 813/
[69] Jianing Wang, Qiushi Sun, Xiang Li, and Ming Gao. Boosting language models reasoning with chain- of-knowledge prompting. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers). pages 4958-4988888 -8881,991111111 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: /tRLLattpp://accaattolllggg org 2024. song. 271/
10. 00118833/1122222 12m.://////2224..cooooo...2222
[70] Liang Wang, Haonan Chen, Nan Yang, Xiaolong Huang, Zhicheng Dou, and Furu Wei. augmented generation, 2025. Utt/sttts://arxxi org. abs abw.ab/////// 2501 14342
[71] Shijie Wang. Wenqi Fan, Yue Feng, Xinyu Ma, Shuaiqiang Wang. and Dawei Yin. Knowledge graph retrieval-augmenteed generation for llm-based recommendation, 2025. httppt://////xxi org/abs/ 2221.
[72] Shiqi Wang. Yeqin Zhang, and Cam-Th Tu Nguyen. Mitigating the impact of false negatives in dense retrieval with contrastive confidence regularization. In Proceedings of the Thirty-Eihhth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence, AATT2//AATT24AAAMM AAAI Press, 2024. ISBN 978-1 978-1-57735-88 -777755-887- doi: 10.1609/aaii v38117.29888 URL https://doi org/ 10. 1609/ aaai. ww8i17.11...11 29885
[73] Shuai Wang and Guido Zuccon. Balanced topic aware sampling for effective dense retriever: A repro- ducibility study. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '23, page 2542 2551, New York, NY, USA. 2023. As- sociation for Computing Machinery. ISBN 9781450394086.
doi: 011144//33961883591915 URL
https://ddi. org/ 10. 10111114/3555 3539313396111111 3.539911
[74] Yuhao Wang, Ruiyang Ren, Junyi Li, Xin Zhao, Jing Liu, and Ji-Rong Wen. REAR: A relevance- aware retreeval-angmented framework for open-domain question answering. In Yaser Al-Omaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5613-5626, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. N0.18653w112222..mmm 2twp:/00244.mmmmmmmmmm.33.. 321. https: http://aalamathlooo/o .1:12....llpppp. emnlp main. 321/
org/2024.
[75] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian lchter, Fei Xia, Ed H. Chi, Quoc V. Lc, and Denny Zhou Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088 1088.
[76] Jason Wei, Nguyen Karina, Hyung Won Chung. Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus, Measuring fhort-form factuality in large language models, 2024. URL https: / //trxivv.v///////////ee////aalla org/ hbs///b////0 /bbs/111.... 04368
16

---

[77] Ning Wu, Ming Gong, Linjun Shou, Jian Pei, and Daxin Jiang. Ruel: Retrieval-augmentee user representa- tion with edge browser logs for sequential recommendation. In Proceedings the 32nd ACM International Conference on Information and Knowledge Management. CIKM '23, page 487 1-8888. ACM, October 2023, doi: 10.1 00.144//883788.661544
URL/tttp://xx doi org htgg//0.1145/// /1111/11///888... 3615498
[78] Jiahao Xu, Wei Shao, Lihui Chen, and Lemao Liu. SimCSE++: Improving contrastive learning for sentence embeddings from two perspectives. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12028- 12040, Singapore, December 2023. Association for Computational Linguistics. doi: (0.18633/v1/202/ emnlp- m37n. URL httptt:://acaaaatololieee
wwg///////.... emm main. 737/
[79] Zhentao Xu, Mark Jerome Cruz, Matthew Guevara. Tie Wang. Manasi Deshpande. Xiaofeng Wang. and Zheng Li. Retrieval -aummeteed generation with knowledge graphs for customer service question answering. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, page 2905 2909, ACM, July 2024, doi: 10. 10.1145/362677.36661377 URL (ttt:://://dr dww. org/ 10011145//3 2444772 260.//6.66772...777
[80] Jiaqi Xue, Mengxin Zheng, Yebowen Hu, Fei Liu, Xun Chen, and Qian Lou. Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models, 2024. URL https://axxiv
oentraaaionno
org/abs/2406
[81] Rui Yang. Boming Yang. Aosong Feng, Sixun Ouyang. Moritz Blum, Tianwei She, Yuang Jiang, Freddy Lecue, Jinghui Lu, and Irene Li. Graphusion: A rag framework for knowledge graph construction with a global perspective, 2025. httpe://:://axxi org/ htab////11... 17600
[82] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Ellen Riloff. David Chiang. Julia Hockenmaier, and Jun' ichi Tsujii, editors. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369-2380, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10. 10618865//DD1 12599. URL https://aclanthol.oom org/D18- 1259/
[83] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: deliberate problem solving with large language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc.
[84] Wen-tau Yih, Matthew Richardson, Christopher Meek, Ming-Wee Chang, and Jina Suh. The value of semantic parse laheling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume Short Papers), pages 201-206, 2016.
[85] Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with retrieval--ugnmentee generation in llms, 2024. URL
https://arxiv. org/ aba///a//////// 2407502485575
[86] Nan Zhang, Prafulla Kumar Choubey, Alexander Fabbri, Gabriel Bernadett- Mitra, Caiming Xiong, and Chien-Sheng Wu. Sirerag: Indexing similar and related information for multihop reasoning, 2025. URL htttps://axxiv. http://axxxvv..oo///////2// wttp:///////.....///e52g
Bernadett-Shapir. Shapiro, Rui Zhang, Prasenjit
[87] Ruilin Zhao, Feng Zhao, Long Wang, Xianzhi Wang, and Guandong Xu. Kg-cot: -hain-of-thought prompting of large language models over knowledge graphs for knowledge-wware question answering. In Kate Larson, editor, Proceedings of the Thirty-Third International Joint Conference Artificial Intelli- gence, IJCAI-24, pages 6642 6650. International Joint Conferences on Artificial Intelligence Organization. 2024. doi: 10.24963/iicaii2224774. URL httpp:://ooi org 10. 24963/ htt.:/a..aa...2/.... Main Track.
[88] Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen, Heng- Cheng, Ed H. Chi, Quoe V Le, and Denny Zhou. Take a step back: Evoking reasoning via abstraction in large language models, 2024. URL https: /ttps://arxiv... org/ 2310. 2210001171
[89] Wang Zhu, Jesse Thomason, and Robin Jia. Chain-of-questions training with latent answers for robust multistep question answering. Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8845-8860, Singapore, December 2023, Association for Computational Linguistics. doi: URL htRpssttp:::22 : http://aaaaannnnyo...ooo org/ 222g.2023..2 emnlp-maaaee main..47n777
1/2023.2222mm emmnlp 547.
17

---

[90] Xiangrong Zhu, Yuexiang Xie, Yi Liu, Yaliang Li, and Wei Hu. Knowledge graph-guided retrieval augmented generation. In Luis Chiruzzo, Alan Ritter, and Lu Wang. editors, Proceedingy of the 2025 Conference of the Nations of the Americas Chapter of Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2912-994,, Albuquerque, New Mexico, April 2025, Association for Computational Linguistics. ISBN 979-8 8 19176-189-6. URL https://
aclanthologyy. www.llooo.ooogoooooooooo org/2026 maacl-lore 449/
18

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The introduction lays out three main claims and explains each detail. The abstract briefly introduces these claims. These are about unrolling questions into masked sub-questions and reasoning chains to pinpoint uncertain information, re-ranking retrieved documents through layer-wise contrastive comparison, and reconstructing the full reasoning chain by filling in masked positions via the LLM. All of these points are addressed the main text. in the main text
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be pereeived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much results can he expected to generalize to other setrings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The main limitations of the work, as well as future directions that might address some of these limitations, are layed out in the discussion portion of the paper.
Guidelines:
The answer NA means that paper has no limitation while the answer means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well well-speciffiation, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures hecause fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicaa,le, authors should discuss possible limitations their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve integrity the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
19

---

We provide correct and complete proofs for each theoretical result, illustrated in the experiments and appendix sections.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross wross-rcferenced
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by
formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Datasets, models, and hyperparameters used in implementing proposed algorithms are all described in detail. See Section 4.1 lnd Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments. No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on contribution, reproducibility can accomplished various ways. example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results. access to a hosted model (e.g.. in the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. . a large language model), then there should either be a way to access this model for reproducing the results a way to reproduce the model (e.g.. with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may he that access to the model is limited in some way (e.g.. registered users). hut it should be possible for other rescarchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We already provide the code and the preprocessed datascts for reproducing the experiment results in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
20

---

Please see the NeurlPS code and data submission guidelines btttps::/nips.cc/puut
for more details.
While we encourage the release of code and data, we understand that this might not be possible.
"No" acceptable answer. Papers cannot be rejected simply for not including code, unless this central to the contribution (e.g.. for a new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines cestps:/niia cc/pullic/ guides/Code
for more details.
The authors should provide instructions on access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for new proposed method and baselines. If only a subset experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer. etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the details of our experiments including prompts for generation.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix. or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification: We performed training and inference five times for each setting, computed the and standard deviation, and added this information to the table caption. See Appendix C.T and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some paramcter, or overall run with given experimental conditions).
The method for calculating the error bars should he explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be carcful not to show in tables or figures
symmetric error bars that would yield results that are of range (c.g. negative error rates). error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
21

---

Answer: [Yes]
Justification: We discuss local computing resources we utilize training time Appendix addition, we discuss our model efficiency in Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform. every respect, with NeurIPS Code of Ethics
Answer:
[Yes]
Justification: We reviewed the NeurlPS Code of Ethics and made sure that our paper conforms to it in every respect
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., if there is special consideration due laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts the work performed?
Answer: [NA]
Justification:
We not anticipate our work to have any meaningful positive or negative socictal
impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed. If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploy- technologies that could make decisions that unfairly impact specific groups), privacy considcrations, and security considerations.
The conference expects that many papers applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
foundational research and not tied to particular
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
22

---

Answer: [NA]
Justification: such models or datasets are involved.
Guidelines:
The answer means that the paper poses no such risks.
Released models that have high risk misuse or dual-use should be released with necessary safeguards to allow for controlled use of model, for example by requiring users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code. data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite the original papers, such those describing the datasets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwwprspwpphaeeee www.wwtthhhe....... com/ datasete has curated licenses for some datasets. Their licensing guide help determine the license of a dataset.
For existing datasets that ro-packaged, both the original license and the license of the derived asset (if it has changed) should he provided.
If this information not available online, the authors encouraged reach out the asset's creators.
13. New assets
Question: Are new assets introduced the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All of the code and data assets released alongside our paper are appropriately documented for reproducibility
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the dctails of the dataset/code/moddl as part of their sub- missions via structured templates. This includes details about training. license, limitations.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create
anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We did not employ crowdsourcing or involve any human subjects in our rescarch.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
23

---

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least minimum wage country data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: We did not employ crowdsourcing involve any human subjects in our research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained TRB approval, you should clearly state this the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere to the NeurlPS Code of Ethics the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review.
16. Declaration LLM usage
Question: Does the paper describe the usage of LLMs if it an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: use L.MSs editing purposes, such as paraphrasing and correcting typos.
Guidelines:
answer NA means that core method development this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy http:///eaaaapeeee////o///eeeeee for what should or should not be described.
24

---

Appendices
In this supplementary material, we provide further details on the following topics:
Appendix A
Implementation Details
Appendix
Baselines
Appendix
Overall Performance
Single-step Retrieval Performance
+
Single-step QA Performance
Multi-step Retrieval Performance
+
Multi-step QA Performance
Appendix D Efficiency
D.I
Retrieval Efficiency
D.2
Reasoning Efficiency
Appendix
Analysis of CoopRAG
F.1
Impact of Question Unrolling with Varying Similarities
Impact of the Number of Sub-questions
Impact of the Length of the Reasoning Chain
Performance Comparison Different Question Unrolling Methods
B.5
Impact of Contrastive Reranking Strategies
E.6
Impact of Separating Reasoning Steps on Model Performance
Impact of Alternative Weighting Methods Based on Sub-Quee tub-Questions and Reasoning Chains
E
Comparison of Retrieval Performance by Loss Function
M
Complexity and Latency Analysis
Scalability Analysis
Appendix
Hyperparameter Sensitivity
Impact of Mini-batch Size
Impact Temperature in InfoNCE Loss
Impact Bucket Size
Appendix C
Case Study
G.I
Example of RaLa
G.2
Example of End-to-End Process
Example of Retrieval Error
Example of Reasoning Error
Appendix H:
Prompt
Question Unrolling
Reasoning Chain Completion
QA Reasoning
1
Multi-step Key Extraction
25

---

A Implementation Details
In our weighted InfoNCE loss, the temperature parameter 7 is set to 0.05. We select the best performing batch size of 40. The maximum sequence length is configured as 512, and the bucket size is set 3. For in-batch training, select random negative sample and distractor for each question. Details of the hyperparameter sensitivity experiments are provided Appendix F
We employ MPNet [60] as an encoder, and Gemma2 (9B, | L. and GPT-4o-mini [5]] as LLMs. Training is performed on two NVIDIA A6000 GPUs, and inference on a single A6000 GPU. For every multi-hop QA dataset, the encoder is trained for 5 epochs, requiring about 5 hours for HotporQA, 2 hours for MuSiQue, and 8 hours for 2WikiMulthopQAA (2Wiki). For NaturalQuestions (NQ), the encoder trained for epochs, taking approximately hours. Following previous work [9 20), retrieval performance is assessed using Recall 2 and Recall@ while QA performance is evaluated with Exact Match (EM) and Fi-score.
B Baselines
We adopt three types of comparative approaches: The classic retrievers BM25 [55]. Contriever 25
ColBERTv2 [57]. Proposition [6). and GTR 50] (ii) Large embedding models which perform on
BERT leaderboard (64), including CTEE-Qwen277B-mnstruuc [41), GritLM-7B (49, and NY-Embed-v2
(iii) Atructure-augmenter RAG approaches, including RAPTOR (58), HippoRAG [19, HippoRAG2
[9]
SiReRAG [6). and HopRAG 42
C Overall Performance
C.1 Single-step Retrieval Performance
Table 10:
Overall retrieval performance of single-step methods. The best and second- -bstt perfor- mances are presented in bold and underlined, respectively. We use different random seeds for each run, and conduct five runs to report the mean with a maximum standard deviation of +0.3.
Models
Multi-hop QA
Single-hop QA
HotpotQA
MuSiQue
2Wiki
NQ
R@2
R@5
R@2
R@5
R@2
R@5
R@2
R@5
Simple Baselines
BM25
57.3
74.8
32.4
43.5
55.3
65.3
28.2
56.1
Contriever
58.4
75.3
34.8
46.6
46.6
57.5
29.1
54.6
GTR (TS-base)
59.3
73.9
37.4
60.2
67.9
35.0
63.4
Proposition (ColBERTv2)
63.9
78.1
37.8
50.1
55.9
64.9
33.1
62.2
ColBERTv2
64.7
79.3
37.9
49.2
59.2
68.2
36.8
64.3
Large Language Models
TLEE--we2-77-Insttee GritL.M-BB NV-Embed-v2 (7B)
75.8
48.1
63.6
66.7
74.8
44.7
79.2
92.4
49.7
65.9
67.3
76.0
46.2
76.6
84.1
94.5
52.7
69.7
67.1
76.5
45.3
75.4
Structure-augmeeneer RAG
RAPTOR (Llama3.3-70B)
76.8
86.9
47.0
$7.8
58.3
66.2
40.3
RAPTOR (GPT-4o-mini)
78.6
90.2
49.1
61.0
58.4
66.0
40.5
69.4
HippoRAG (Llama3.3-70BB
60.4
77.3
41.2
53.2
71.9
90.4
21.3
44.4
HippoRAG (GPT-4o-mini)
60.1
78.5
41.8
52.4
68.4
87.0
21.6
45.1
HippoRAAG2 (Llama3.3-70B)
83.5
96.3
56.1
74.7
76.2
90.4
45.6
78.0
HippoRAG2 (GPT-4o-mini)
80.5
95.7
53.5
74.2
74.6
90.2
44.4
76.4
Ai (GPT-4o-mini)
80.0
94.8
52.5
64.9
60.6
67.6
42,3
72.5
HopRAG (o-T-4o-mini)
81.1
96.0
53.7
66.8
61.7
70.1
43,9
74.4
Co (Gemma2-99B
87.9
95.6
$9.4
75.5
80.1
96.7
71.6
88.9
CoopRAG (Gemma2-277)
88.3
96.6
59.4
75.7
80.8
97.2
72.8
89.5
AG (Llama3.3-7006
86.9
96.6
58.2
75.3
80.6
96.3
77.2
90.8
CoopRAG (GPT-4o-mini)
88.8
96.8
59.6
75.7
80.4
96.6
80.8
92.1
We compare the performance of single-step retrieval across several benchmark datasets. Table 10 shows that structure-aus augmented RAG consistently outperforms simple baselines. and achieves the best results on all datasets. CoopRAG (GPT-4o-mini) records the highest scores every benchmark except 2WikiMultihop. in which
26

---

CoopRAG (Gemma2-277) leads. On NaturalQuestions, CoopRAG (GPT-4o-mini) improves Recall by 34.6% and Recall @5 by 15.5% over the previous state-of-the-art, GritLM-7B. Although NaturalQuestions requires only one document inference, over 71% of questions sub-questions, indicating that question unrolling benefits both single-hop and multi-hop questions. Our methods with different LLMs, i.c. Gemma2-98. Gemma2-22 2-2na and Llama3.3-700 achieve superior performance across all datasets, and CoopRAG (Gemma2-9B) excels despite its smaller number of parameters, demonstrating the effectiveness of method in resource-constrained settings.
dataset are decomposed into two or more
C.2
Single-step QA Performance
Table 11: Overall QA performance comparison across single-step methods. The best and second-best performances presented in bold and underlined, respectively.
Models
Multi-hop QA
Single-hop QA
HotpotQA
MuSiQue
2Wiki
NQ
EM
Fl
EM
FI
EM
F1
EM
F1
Llama3.3-70B
BM25
$2.0
63.4
20.3
28,8
47.9
51.2
44.7
59.0
Contriever
51.3
62.3
24.0
31.3
38.1
41.9
45.0
58.9
GTR (T5-base)
50.6
62.8
25.8
34.6
49.2
52.8
45.5
59.9
GTE- wwwe------nn-nteeeeeeeee
58.6
71.0
30.6
40.9
55.1
60.0
46.6
62.0
GritI.M--B
60.7
73.3
33.6
44.8
55.8
60.6
46.8
61.3
NV-Embed-v2 (7B)
62.8
75.3
34.7
45.7
57.5
61.5
RAPTOR
56.8
69.5
20.7
28.9
47.3
52.1
36.9
50.7
GraphRAG
55.2
68.6
27.3
38.5
51.4
58.6
30.8
46.9
LightRAG
2.0
2.4
0.5
1.6
9.4
11.6
8.6
16.6
HippoRAG
$2.6
63.5
26.2
35.1
65.0
71.8
43.0
55.3
HippoRAG2
62.7
75.5
37.2
48.6
65.0
71.0
48.6
63.3
GPT-4o-mini
RAPTOR
50.6
64.7
27.7
39.2
39.7
48.4
37.8
54.5
GraphRAG
$1.4
67.6
27.0
42.0
45.7
61.0
38.0
55.5
LightRAG
9.9
20.2
2.0
9.3
2.5
12.1
2.8
15.4
HippoR
46.3
60.0
24.0
35.9
59.4
67.3
37.2
55.2
HippoRAG2
56.3
71.1
35.0
49.3
60.5
69.7
43.4
60.0
SiReRAG
61.7
76.5
40.5
53.1
59.6
67.9
42.4
HopRAG
62.0
76.1
54.9
61.1
68.3
59.2
Co (Gemma2-99)
64.4
78.1
52.2
65.2
70.0
78.1
63.8
72.7
Co (Gemma2-277B)
64.9
79.5
52.8
66.7
71.7
79.0
67.3
75.5
CoopR AG (llama3.3-770B
64.7
79.0
52.6
66.6
71.2
78.8
70.9
80.3
CoopRAG (GPT-40-mini)
65.6
78.9
52.3
71.7
79.2
72.0
82.3
We compare the performance of single-step QA across all the benchmark datasets. As shown in Table 3 CoopRAG (GPT-4o-mini) achieves state-of-the-art performance most datasets. In particular, when using Gemma2-9B. CoopRAG achieves 15.2% higher EM NaturalQuestions than the previous state-of-the of-the-ar method. These results demonstrate that reranking by contrasting layers and completing the masked reasoning chain based on the documents retrieved by murolling-augmente retrieval enhance the QA accuracy.
C.3 Multi-step Retrieval Performance
IRCoT 67 is representative framework for multi-step retrieval and QA. However. it employs simple structure which the LLM performs chain-of-thouge reasoning over the question and candidate documents, augments the question until judges problem solved. Moreover, its basic prompting design requires a lot of examples, i.c., 13-15 shots. To overcome these limitations, we propose the KeyExtract method: the LLM first evaluates whether LLM can infer the answer from the initially retrieved candidate documents; if LLM cannot, LLM extracts key sentence from those documents. and appends it to the unrolled question for iterative small LLMs such
re-retrieval. KeyExtract operates effectively with only three shots, and can be applied as Gemma2-9B.
demonstrate effectiveness the KeyExtract method, we compare multi-step steli--tep retrieval performance across various methods and datasets. Table 12 presents the results against baselines. Our approach outperformed existing methods by wide margin on all datasets. In particular, KeyExtract + CoopRAG (GPT-4o-min) miniiiiiinn........... achieves 34.6% improvement Recall @ and 17.1% improvement in Recall@5 HotpotQA compared to IRCoT +
27

---

Table 12: Overall retrieval performance comparison across multi-step methods. The best and second- best performances are denoted bold and underlined, respectively. We used different random seeds for each run and, for each dataset and evaluation metric, conducted five runs to report the mean, with a maximum standard deviation of +0.21.
Models
HotpotQA
MuSiQue
2Wiki
@@2
R@5
R@2
R@5
R@2
R@5
Simple Baselines
IRCoT BM25 (Default)
65.6
79.0
34.2
44.7
61.2
75.6
+RCoT+ Contriever
65.9
81.6
39.1
52.2
51.6
63.8
IRCoT + ColBERTv2
67.9
82.0
41.7
53.7
64.1
74.4
Structureauugmnnted RAG
IRCoT HippoRAG (Contriever)
65.8
82.3
43.9
93.4
IRCoT + HippoRAG (ColBERTv2)
67.0
83.0
45.3
57.6
75.8
93.9
IRCoT Co (GPT-4o-mini)
87.3
90.9
62.2
74.9
81.0
95.9
KeyFxtract CoopR AG (Gemma2-9B)
88.6
93.9
62.9
76.8
81.8
97.2
Key AG (GPT-4o-mini)
90.2
97.2
64.5
78.6
83.6
97.6
HippoRAG (ColBERTy2), the latest structure-aware RAG model. On MuSiQue, our method improves Recall @2 and Recal 5 by 42.3 % and 36.5%, respectively, and 2WikiMultihopQA by 10.3% and 3.9%, respectively. Notably, applying KeyExtract yiclds an average performance gain of 3.31% the IRCoT approach. These findings demonstrate KeyExtract retrieves relevant documents more effectively for questions requiring complex reasoning, even with fewer examples. The prompt for KeyExtract is Appendix H.4
C.4
Multi-step QA Performance
Table 13: Overall QA performance comparison across multi-step methods. The best and second-best performances denoted in bold and underlined, respectively.
Models
Reader (LLM)
HotpotQA
MuSiQue
2Wiki
EM
F1
FM
F1
EM
F1
IRCoT + ColBERTv2
GPT-4o-mini
45.5
58.4
19.1
30.5
35.4
45.1
IRCoT + HippoRAG
GPT-3.5-turbo
45.7
59.2
21.9
33.3
47.7
62.7
IRCoTT + CoopRAG
GPT-4o-mini
64.3
75.5
52.3
65.9
72.2
77.7
KeyExtract + CoopRAG
Gemma2-9B
64.9
77.9
52.5
66.1
71.5
78.9
KeyExtract + CoopRAG
GPT-4o-mini
66.7
79.2
53.8
68.6
72.2
78.2
We compare the multi-step QA performance, As shown in Table CoopRAG outperforms all existing methods by substantial margin. We observe that applying IRCoT CoopRAG using the same baseline setup results in a least 40.7% improvement EM on HotporQA compared the baselines. Notably, combining KeyExtract with Co (GPT-40-mini) yields an additional gain over 5.3%. This trend holds across other datasets as well. On MuSiQue, our model achieves an EM score 145.7% higher than the previous state-of-the-art, i.e. IRCoT HAp These results demonstrate that CoopRAG is highly compatible to multi-step QA frameworks such as IRCoT.
D Efficiency
D.1 Retrieval Efficiency
Table 14]cpp4cs the efficiency of different retricval methods, evaluating search latency and accuracy (R @ for CoopRAG versus HippoRAG2.. For a fair comparison, all experiments are conducted by using GPT-4o mini. The results show that CoopRAG retrieves faster on HotpotQA and MuSiQue. Although CoopRAG is slightly slower 2WikiMulihhoOA. the difference negligible. Importantly, despite the matching exceeding retrieval speed of HG2, CoopRAG delivers substantially higher accuracy across all datasets. On HotpotQA, we achieve R@2 88.1% compared to 80.5% for HippoRAG2 (+7.6% on MuSiQue, R@2 2 59.6% versus 53.5 (+6.15 ()) and on 2WikiMulthopQAA R@2 80.4% versus 74.6% (+5.8%). These results demonstrate that our approach significantly improves retrieval accuracy without sacrificing the retrieval speed.
28

---

Table 14: Comparison of retrieval efficiency between our model and HippoRAG2
Model
HotpotQA
MuSiQue
2Wiki
Time (8)
R@2
Time (s)
R@2
Time (s)
R@2
HippoRAG2
2.25
80.5 88.1
2.11
53.5
2.34
74.6
CoopRAG
2.04
1.98
59.6
2.39
80.4
D.2 Reasoning Efficiency
Table 15: Comparison of the number of LLM calls per question across preprocessing, retrieval, and reasoning stages for CoopRAG, HippoRAG2, and HopRAG.
Methods
Retrieval
QA
LLM calls per question
Recall
Recall
EM
F1
Preprocessing Retrieve
Reasoning
HippoRAG2
80.5
95.7
56.3
71.1
HopRAG
81.1
62.0
76.1
14.96
1
CoopRAG (unified)
88.8
96.8
63.1
76.6
1
CoopRAG
88.8
96.8
65.6
78.9
2
To demonstrate reasoning efficiency of CoopRAG, we compare the number of LLM calls required by CoopRAG and state-of-the-art baselines, HippoRAG2 and HopRAG, across preprocessing. retrieval, and inference stages. As shown in Table 15] HippoR AG2 and HopRAG make 4 and 12 calls per question during preprocessing, while CoopRAG requires none. This difference arises because hoth baselines repeatedly invoke the LLM for each document generate triples. During retrieval, HopRAG incurs additional average 14.9 calls per question due to its graph-based iterative triple extraction method. the inference stage, CoopRAG makes two calls per question. one for reasoning chain completion and another for reasoning. Despite this, CoopRAG AG achieves to 16.5% higher EM compared to the baselines. Coop (Unified), which combines the two reasoning steps into single call, also outperforms both basclines. In summary, CoopRAG demonstrates clear efficiency and effectiveness, requiring fewer LLM calls while achieving superior performance.
E Analysis
E.1
Impact of Question Unrolling with Varying Similarities
HotpotOA
MuSiQue
2Wik/MuttinpOOA
0.5
RecaligS
0.8
0.
0.8
8.7
Question
0.57
0.53
0.61 grou
0.10
0.55
3.50
Score group
Score group
Figure 3: Mean @ against the score between questions and positive documents. We sort all of the question-positive document pairs by their similarity scores, divide them into equal-sized groups, and measure the average similarity scores and average @ for each group.
We confirm that question unrolling, which enhances the original question using the internal knowledge of LLM, significantly increases similarity with the positive, i.e., ground-truth, document compared to the original question. Figure shows the difference in retrieval performance according to the similarity between question and its ground truth document. The retrieval performance for using the unrolled question consistentl outperforms that for using the original question. As the similarity decreases, their performance gap increases. This indicates that when the question is structured and enhanced by effectively utilizing the internal knowledge, the retriever can accurately retrieve highly relevant documents.
29

---

E.2
Impact of the Number of Sub-questions
HotpotQA
MuSiQuc
2WikiMult
0.0
0.9
Recallgss
Recalli55
0.8
0.8
0.8
Original Question
Original Quest
Orig nat Quesion
U Uulllle Question
Urrolled Quuslian
Uneciled Quuasion
the number sub questions
The number of sub quessisssooss
The number sub qubqquestion
Figure 4: Effectiveness of question unrolling with respect to sub-question complexity.
We investigate the effect of the number of uub-questions on retrieval performance to analyze the effectiveness of question unrolling. Figure [ presents the how average performance varies with the number of sub-questions. where "Original question" refers the CoopRAG variant without using question unrolling. and "Unrolled Question" refers to CoopRAG. Across all datasets, CoopRAG consistently outperforms the variant. As the number of sub-questions increases, the performance for the original question declines, whereas that CoopR remains stable. On MuSiQue, the performance gap between Coop AG and the variant reaches approximately 30%, i.c., the largest observed difference, when there are four cuub-uustions. This finding demonstrates that question unrolling becomes increasingly beneficial for more complex questions. The results suggest that relying solely on the original question makes difficult to retrieve appropriate documents for questions requiring complex reasoning, and that our question unrolling method effectively overcomes this limitation.
E.3 Impact of the Length of Reasoning Chain
HotpotQA
MusiQue
2WikiMultihopOO
1.0
1.0
0.9
Recall3
Recaligs
67
Original Question
Original Question
Orignal Queston Unvillld Questio on
Uniolled Question
Unrolled Question
3.0
0.6
number of tripies reasoning chain
The number of triples in reesoning chain
The number of triples in reasoning chein
Figure 5: Effectiveness of Question Unrolling with respect to reasoning chain complexity (number of triples).
We conduct an additional experiment to analyze the impact of the length of reasoning chain on retrieval performance. Figure 5 the how average performance varies with the number of triples in the reasoning chain, where "Original question" refers the CoopRAG variant without using question unrolling, and "Unrolled Question" refers to CoopRAG. Across all datasets, CoopRAG consistently outperforms the variant "Original question".. As the number of triples increases, the performance of the variant declines sharply, while the performance of CoopRAG remains stable. On HotpotQA, the performance gap between the original and unrolled questions widens approximately 20%, when six triples included in the reasoning chain. On MuSiQue, similarly large gap of about 35% appears when seven triples are used. These findings indicate that the more complex the reasoning process, the greater the benefit of question unrolling. The results demonstrate that for questions requiring complex reasoning chains, relying solely on the original question makes it difficult to retrieve appropriate documents, demonstrating that question unrolling effectively overcomes this limitation.
30

---

Table 16: Ablation study on the effect of question unrolling. The best and second-best performances are denoted in bold and underlined. respectively.
Category
HotpotQA
MuSiQue
2Wiki
R@2
R@5
R@2
R@5
R@2
R@5 5
MQ MQ SQ
71.6
82.8
43.7
56.5
71.5
88.6
80.3
90.9
$1.6
67.9
77.7
89.9
MQ + RC
86.4
93.9
57.7
73.3
78.7
92.9
MQ SQ RC
88.1
95.9
59.6
75.7
80.4
96.6
E,4 Performance Comparison of Different Question Unrolling Methods
Table 16 shows retrieval results for different question unrolling techniques: MQ stands for the CoopRAG variant without question unrolling. MQ + SQ stands for the variant that decomposes an input question into only sub subb-uuettonss MQ RC decomposes
input question into only masked reasoning chain, and MQ SQ denotes CoopRAG, For HotpotQA, MQ + SQ increases Recall MQ by 12%. MQ + SQ RC further increases Recall to 88. %,, with 23% gain over MQ. MuSiQue, MQ yields Recall@ 43.7% MQ+ + SQ RC raises it to 59.6%, achieving a 36% improvement. Notably. MQ RC alone moves Recall from 51.6% 59.6%. A similar pattern appears 2WikiMultihopAA where MQ yields 71.5% and the full combination reaches 80.4%, resulting in 12.19 increase. Across all datasets, MQ SQ improves performance MQ. and MQ SQ
enhances MQ SQ further. These findings demonstrate that decomposing main question into sub questions and explicitly representing the reasoning steps with uncertain reasoning chains improves the document retrieval accuracy for complex questions.
E.5 Impact of Contrastive Reranking Strategies
Table 17: The effect of different contrastive reranking strategies retrieval and QA performance within CoopRAG.
Strategy
HotpotQA
MuSiQue
2Wiki
R 2
R@5
R@2
R@5
R@2
R@5
Contrasting token embeddings Contrasting similarity scores Optimization based on Wi,D
86.6
94.4
52.2
69.9
77.1
91.2
88.4
96.6
60.0
76.2
81.8
96.7
88.1
95.9
59.6
75.7
81.4
96.6
sCore(U, D) avg.=0 max
(q: a(")))
where, 1" = arg max
||a50) 181EL
He (0.2.....II))
Table 17 compares retrieval performance under RaLa framework using three contrastive reranking strategies: contrasting token embeddings in Equation]? contrasting similarity scores in Equation and the optimization based on the gate-aware weight Wo,D in Equation 4 According to the table. contrasting similarity scores. ie.. Equation achieves the best performance, demonstrating that reflecting fine-grained information change through token-level similarity differences highly effective. In contrast, contrasting token embeddings yiclds the lowest performance. Although effectively captures token-wise information diffcrences, fails to preserve the overall contextual semantics of the document. The contrasted embedding vector converges toward zero if key tokens exhibit minimal change in embedding, offering little contribution to scoring. Ultimately, we adopt the optimization strategy, i.e., Equation 4 Although this strategy shows slightly lower performance than contrasting similarity scores, offers several advantages. First, relying solely on the [CLS] token allows encoder to effectively retain the document's global semantics. Second, this strategy more efficient than contrasting similarity scores by reducing the training time by more than factor of four. Considering both effectiveness and efficiency, the optimization strategy is the most practical choice. In summary, this strategy delivers near-best performance while greatly enhancing computational efficiency.
E.6 Impact of Separating Reasoning Steps on Model Performance
We compare QA performance between: (1) unifying reasoning chain completion and reasoning steps in single LLM call, and (2) separating them into independent calls. evaluate both small (Gemmaa-9y and Gemma2-277 277),, and larger API-based LLMs (GPT-4o-mm and GPT-o3). Across all LLMs in Table 18] the separated-cal approach consistently outperforms the unified-call call approach. Notably, the performance drop for the unified-aa becomes more pronounced model size decreases. For the smallest LLM,
LLMs
31

---

Table 18: QA performance under unified separate strategies for reasoning chain completion and reasoning for different LLMs.
LLM-call Strategy
LLM
HotpotQA
EM
F1
Unified
Gemma2-9B
58.7
72.7
Separated
Gemma2-9B
64.2
77.8
Unified
Gemma2-27B
59.9
75.5
Separated
Gemma2-27B
64.9
79.5
Unified
GPT-4o-mini
63.1
76.6
Separated
GPT-4o-mini
64.7
78.8
Unified
GPT--33
70.2
83.3
Separated
GPT-o3
71.1
82.8
i.c., Gemma2-9B, using the unified-call results in over an 6% reduction in both EM and F1 compared the separated-cee call. These results suggest that compact LMs incur greater cognitive load when processing complex reasoning in single step, leading to degraded performance. CoopRAG makes three LLM calls in total, which is the same HippoRAG2 (20 and fewer than HopRAG [2]
E.7 Impact of Alternative Weighting Methods Based on Sub-Questions and Reasoning Chains
Table 19: Retrieval performance comparison using the number of sub-questions and the length of reasoning- as weights for CoopRAG.
Diffiuult-Aware Weight
HotpotQA
MuSiQue
2Wiki
R@2
R@5
R@2
R@5
R@2
R@5
w/o difficulty-awwe aware weight
87.8
95.2
58.2
74.6
79.9
96.0
Number of sub-questions
88.1
95.9
59.6
75.7
81.4
96.6
Reasoning chain length
87,8
95.8
76.4
81.4
96.5
We compare CoopRAG and its two variants: (I) removing the difficulty-aware weight from CoopRAG, i.e. w/o am,, (2) using the number of sub-questions a difficulty-aware weight, i.e., CoopRAG, and (3) using the length of masked reasoning chain
difficulty-waarc weight. Table T9) shows their retrieval performances. Applying these weights during training consistently improves performance across all datasets compared w/o ou, indicating that reweighting by the question complexity enhances the retrieval capability the encoder. Both reweighting approaches achieve similar overall performance, though subtle differenees emerge. On HotpotQA and 2 MuikiiulaihopOA, CoopRAG achieves Recall @ of 88.1 % and 81 1.4% respectively, exceeding 87.8% and matching 81.4% obtained by applying the reasoning chain length, respectively. Overall, CoopRAG is slightly superior on HotpotQA, whereas reweighting by reasoning chain length produces higher Recall and Recall@ on MuSiQue than CoopRAG. As a result, the optimal reweighting strategy may vary with dataset characteristics. Both CoopRAG and the variant using the reasoning chain length reflect the question complexity effectively. as evidenced by their comparable performance.
E.8
Comparison of Retrieval Performance by Loss Function
Table 20: Comparison of retrieval performance using different loss functions (Recall @ @2).
Loss
HotpotQA
MuSiQue
2Wiki
Cross-entropy InfoNCE
77.5
30.9
63.3
88.1
59.6
80.4
We conduct experiments comparing InfoNCE loss adopted by CoopRAG and cross-entropy loss analyze the impact various functions on retrieval performance. Table 20 shows Recall@2 across the three datasets. The experimental results demonstrate that CoopRAG using InfoNCE loss achieves significantly higher performance than the variant using cross-entropy loss on all datasets. The performance differences are substantial. These findings indicate that contrastive learning such InfoNCE loss provide more accurate retrieval results.
32

---

E.9
Complexity and Latency Analysis
Table 21: Latency comparison between HippoRAG2 and RaLa. We report average latency (in seconds) for retrieval, QA, and total end-to-end processing.
Methods
HotpotQA
MuSiQue
Retrieval
QA
Total
Retrieval
QA
Total
HippoRAG2
2.25
1.96
4,21
2.11
1.81
3.92
Co
2.04
1.86
3.90
1.98
1.65
3.63
We analyze the computational complexity and latency of RaLa in comparison with HippoRAG2 contextualize the practical trade-offss our method. RaLa adopts a suriever-rerankee pipeline. The time complexity for retrieval O(N d), where the numher of documents and is the embedding dimension. For reranking. the naive time complexity without our optimization, as deseribed in Equation BJ Section is ( La d (CI). where L, and La denote the query and document lengths, respectively, and |C] is the bucket size. By applying optimization strategy in Equation 4 only the [CLS] tokens compared per bucket, which reduces the complexity to (L,, La d) and makes asymptotically identical that ColBERT. All reported results are obtained using this optimized version.
As shown in Table 21 RaLa consistently achieves lower latency than HippoRAG2 across both datasets. On Ho RaLa reduces retrieval latency from 2.25s to 2.04s and QA latency from 1.96s to 1.86s, leading to an overall reduction in total latency from 4.21s to 3.90s. similar trend is observed MuSiQue, where RaLa achieves 1. 1.98s retrieval lateney and 1.65s QA lateney, compared 2.1 Is and 1.81s for HippoRAG2, reducing the total latency
3.92s to 3.63s. These results show that RaLa not only delivers stronger retrieval QA performance but also processes queries more efficiently.
E.10 Scalability Analysis
Table 22: Retrieval performance of HippoRAG2. HopRAG, and CoopRAG when varying the candi- date size from 10,000 to 60,000 documents. Performance gain (%) denotes the relative improvement of CoopRAG over the best baseline. The best and second- best performances denoted in bold and underlined, respectively.
Methods
Candidate size
10000
20000
30000
40000
50000
60000
HippoR HopRAG
95.7
86.5
56.3
33.2
85.6
77.1
58.8
43.4
38.9
CoopRAG
96.8
88.6
81.9
68.7
59.6
56.8
Performance Gain (%)
0.83
2.43
4.73
16.84
37.33
46.02
We evaluate the scalability of CoopRAG by varying the candidate size in Wikipedia from 10,000 60,000 docu- ments. As shown Table CoopRAG achieves consistently higher retrieval performance than HippoR AG2 and HopRAG across all candidate sizes. With 10,000 candidates, CoopRAG shows marginal gain of 0.83% over the best baseline, but the advantage becomes more substantial the candidate pool grows. At 40,000 candidates, CoopRAG outperforms baselines by more than 16%, and at 60,000 candidates the gap widens 46.02%. These results highlight that CoopRAG is less affected by the presence of distracting negatives and preserves strong recall even large-scale retrieval settings.
F Hyperparameter Sensitivity
F.1
Impact of Mini-batch Size
We conduct experiments HotpotQA, MuSiQue, and 2 2WikiMultihoQQA evaluate the impact of the mini- batch size on retrieval performance. Figure shows Recall@2 mini-batch sizes of 30, 40, 50 across datasets. A mini- batch size of 40 yields the highest Recall 2, 88.1% on HotpotQA, 59.6% on MuSiQue, and 80.4% on 2WikiMulthooQA. We further observe that performance drops sharply when mini-bate batch size exceeds 50.
33

---

HotpetOA
MuSiQue
2WiiMMutthhppo
90
60-
80-
Recalia2
Rocalig?
75-
57
40 Batch
50
size
Figure 6: Hyperparameter sensitivity analysis batch size, showing Recall @2 for batch sizes of 30, 40, and 50 across datasets (HotporQA, MuSiQue, and 2WikiMulchhoOAA)
HatpotOJ
MuSique
ZWIcMMultihpOOA
$0.0
$7.5
Becalig?
Recaligo
62.5
75-
80.0
17.5
70
Figure 7: Hyperparameter sensitivity analysis on temperature in InfoNCE loss (Recall @2 for values of 0.01, 0.05, 0.1, and 0.2).
F22 Impact of Temperature in InfoNCE Loss
We conduct experiments AotpotQA, MuSiQue and 2WikiMultihopOA analyze the effect of the temperature hyper-parameter r InfoNCE loss on retrieval performance. Figure / shows Recall@2 for the values of 0.01, 0.05, 0.1 and 0.2 across the datasets. The highest Recall@2 on HotpoiQA, MuSiQue and 2WikiMulihopQA is yielded when 0.05. Performance declines sharply once t reaches 0.2.
F.3 Impact of Bucket Size
Table 23: Effect bucket size on retrieval performance
Bucket Size
HotpotQA
MuSiQue
2Wiki
R@2
R@5
R@2
R@5
R@2
R@5
87.7
94.5
58.8
75.3
80.1
95.2
95.9
59.6
75.7
80.4
96.6
88.1
96.2
59.9
75.5
80.6
97.0
12
88.3
96.3
60.1
75.9
81.0
97.3
We conduct comparative experiments to analyze the impact of bucket size retrieval performance. MPNet with 12 layers used as encoder, and the number of layers each bucket is determined by dividing the total number layers by the bucket size. Table 23 the experimental results with varying bucket sizes. bucket size of 12, which utilizes all layers for contrasting, achieves the highest performance across all three datasets. However. there is trade-off' between the retrieval accuracy and the computational efficiency. Raising the bucket size from 4 12 yields only marginal performance improvement, while increasing the training time 4.6 times. Similarly. increasing the bucket size from to results negligible accuracy gain but times longer training time. To identify an optimal balance between performance and efficiency, we evaluated various bucket size configurations. and ultimately selected bucket size of 4, which offers near best performance while keeping computational resource usage reasonable level.
34

---

RoLa Contrasting Scores
Question:
When was the American laayer and pollticion which contisled in 2014 gubernator ui election born?
11
2.092
0133
0317
Jason Carter (pelitiiann)
American lawyer and Carter August 1975, Demoortic nommoo
3.00
0367
0.399
Georgls govertor the 2014 eiection He icetto incumberr Nathon Dasl by mirgin of taling 44
bean from the siste Gounge,. Curter furner somatur und
the voto to ueats 52.5 Varter ia glontsoon tome Uoorge poeemor and USS Prosstert lemey Usa tex.
0253
0.23
Dktchomas potiticionn DMtatomm 2010 piatiomm of and resuarin the govarmment. le max detected by Honr Fallin On Decemoer 26, 2013, Broadon denouncen that no would chollengs incumbent Uovernn nor latinn in the 2014 gubemaatonal election. Howeeess he withdrew run for the U.S. Seenate epecist siection rrigened by the redirmne of Tom Oobum.
uniil Brogrin
0.359
0443
(petitolant,
ticlan from the Arkansss. hoxa axnced U.S. Rage Represartation for Artanssa's corgressonal aisliciiio 2001 2013. was t1e Demoorizz nomince for Covernor Arkanoas in the 2014 election and lrot Awwillunctirnn by margin of 50Nis 45W.
066
Figure 8: Illustrative example of RaLa. For each document, the heatmap displays the scores obtained by applying RaLa between the final, i.e., 12th, layer and premature add-numbered layers (1, 3, 5, 7, 9, 11). An input question, ground-truth document, and two negative documents are described to the right.
G
Case Study
G.1
Example of RaLa
We conduct case study to demonstrate the effect of RaLa. Figure 8 illustrates heatmap of the score, Equation between the question and the three documents, i.e., Doc1, Doc2, Doc3, to the right, where the x-axis of the
heatmap represents document indices, and the y-axis represents the odd--ummbered-layer., i.e. 1, 5, 7, 9, and 11. Document 1 is the ground-truth document, whereas Documents 2 and 3 are negative. Note Document I provides decisive clues by specitying "Georgia govemnr in the 2014 election" along with the birthdate August 1975. In contrast, Document contains the 2014 gubernatorial election, but the politician in this document withdrew in the election, and Document 3 shares similar context with the question: "Democratic nominee for Governor of Arkansas in the 2014 election."
We confirm that Ral aa effectively highlights representational differences between the question and the documents, enabling clear identification of the correct document. Without Ral.a, the average MaxSim scores at the final layer are 0.7666 for Document 1, 0.8110 for Document 2, and 0.7915 for Document 3, indicating higher surface-level similarity for the negative documents. This occurs because the core keywords the question appear in the negative documents, e.g. "politician" appears in all of the three documents, and "2014 gubernatorial election" appears only in Document 2. In the heatmap of RaLa, score increases are larger in the lower layers, e.g.. Document I reaches the highest score of 0.565 at layer 1, while the score for layer 11 is 0.052. As result, RaLa moves beyond surface-level keyword matching by contrasting semantic differenees between the final layer and earlier layers, thereby effectively distinguishing the truly relevant document among the three documents.
G.2 Example of End-to-End Process
Figure 2 illustrates the reasoning chain completion and reasoning stages. It clearly demonstrates how the input and output formats evolve throughout the progression of our approach.
G.3 Example of Retrieval Error
We compare and evaluate the retrieval performance before and after applying uncertainty masks. We quantita- tively analyze changes in accuracy and relevance of the top-n retrieved documents. As previously mentioned Section 4.6 we observed that hallucinations may occur in the LLM's reasoning chain generation when produces uncertain entities instead of masking out entities it considers uncertain. For example, as shown in Figure 10 we can see differences in the retrieved documents when the LLM marks uncertain entities with (UNCERTAIN) versus when it does not. This difference in the retrieved documents leads to hallucinations during the final reasoning process, thus encouraging the L.M produce incorrect answers.
G.4 Example of Reasoning Error
As shown in Figure IT we focus on analyzing how uncertainty masks affect the reasoning process. This case illustrates the reasoning process for a question comparing the death dates of film directors. With uncertainty masks, the LLM marks uncertain information, i.c., the directors' death dates, with (UNCERTAIN) tags, and accurately retrieves the relevant documents, and fills in this information during the reasoning chain completion
35

---

Case Study: And-to-End Process
Question:
Which film has the director who died later, 45 Calibre Echo or Bons Baisers De Hong Kong?
SUB Q1: Who directed the film 45 Calibre Echo?
SUB_ Q2: Who directed the film Bons Baisers Hong Kong?
SUB Q3: What was the date of death for the director 45 Calibre Echo??
SUB Q4: What was the date of dcath for the director of Bons Baisers De Hong Kong?
Uncertain Reasoning Chain:
[["45 Calibre Echo", "was directed by", "Bruce M. Mitchell")] [" ("NUUEERTAIN)" "was directed by", "(UNCERTAIN)"", ("(UNCERTAIN)", "died on", "(UNCERTAIN]"". ["Yvan Chiffre", "died on", "(UNCERTAIN)"), ["Between the directors of tuo films", "the one who later is", "[FILL)"]]
Top-5 Retrieved Documents:
Document[1] (Titl: 45 Calibre Echo) 45 Calibre Echo a 1932 American western film directed by Bruce M. Mitchell and starring Jack Perrin, Ben Corbett and Elinor Fair.
[ocument[2] (Title: Bons Baisers de Hong Kong) Bons Baisers Hong Kong also known as From Hong Kong with Love is a 1975 French film directed by Yvan Chiffre. It is a parody of James Bond movies featuring Les Charlots with scenes shot in Hong Kong. Mickey Rooney featured in the film as well Bernard Lee and Lois Maxwell, stars of the James Bond films who appeared as M and Miss Moneypenny, respectively.
filmed the Shaw Brothers studios in Hong Kong.
Document[3] (Titl: Yvan Chiffre) Yvan Chiffre March 1936 27 September 2016 was French director, producer, and stunt coordinator. He is the father of Philippe Chiffre, Romain Chiffre and the grandfather of Cesar Chiffre.
Document[4] (Title: Bruce M. Mitchell) Bruce M. Mitchell November 16, 1883 September 26, 1952 an American film director and writer active during the silent film era from 1914 to 1934, With the advent of sound films in the 1930s, Mitchell abandoned directing and became an actor, appearing mainly in bit roles.
55c (Title: Won in the Clouds) Won the Clouds 1928 American silent film directed by Bruce M. Mitchell and starring Al Wilson. Like many actors the silent film cra, Wilson did not survive the transition to" talkies", with" Won in the Clouds", one of his last films.
Reconstructed Reasoning Chain:
[["45 Calibre Echo", "was directed by", "Bruce M. Mitchell"), ["Bons Baisers de Hong Kong", "was directed by", "Yvan Chiffre"], ["Bruce K. Mitchell", "died on", "September 26, 1952"), ["Yvan Chiffre", "died on", "27 September 2016"], ["Between the directors of the films", "the who died later is", "Yvan Chiffre"]]
Final Answer: Bons Baisers De Hong Kong Ground Truth: Bons Baisers De Hong Kong
Figure 9: Case study of the CoopRAG end-to-end process
stage. This results in the correct answer "Bons Baisers De Hong Kong." In contrast, without uncertainty masks,
the LLM generates incorrect death dates for Bruce M. Mitchell "December 31, 2020" and Yvan Chiffre as "May 2000", thus ultimately producing the wrong answer "Bruce M. Mitchell". This elearly demonstrates how uncertainty masks play crucial role preventing hallucinations, enabling accurate reasoning of LLMs.
36

---

Case Study: Effectiveness of Uncertainty Mask Retrieval
Question:
Who was in charge the place where Castricum is located?
SUB Q1: What province is Castricum located in?
SUB Q2: What is the capital of the province where Castricum is found?
SUB Q3: Who is the King's Commissioner of North Holland?
Ground Truth: Johan Remkes
Number of Golden Does: 2
<With Uncertainty Mask>
Uncertain Reasoning Chain:
[["Castricure cumr", "is located in", "North Holland province"),
["North Holland's capital", "is", "Haarlem"],
["The King's Commissioner of North Holland", "is", "<UNGERTAIN>"") ["The person in charge of the place where Castricum is located", "is", "<FILL>"]]
Top-5 Retrieved Documents:
Document[1] (Title: Castricum) Castricum seaside town in the province of North Holland. It draws fair share tourists who mainly come to visit beach nearby dune landscape. [...]
D (Title: North Holland) The capital and seat of the provincial government is Haarlem, and the provinces largest city is the Netherlands capital Amsterdam. The King's Commissioner of North Holland Johan Remkes, serving since 2010. [...]
Document[3] (Title: Dutch Republic) In 1579 number of the northern provinces of the Low Countries signed the Union of Utrecht, in which they promised support other in their defence against the Spanish army. [...]
Document[4] (Titl: Dutch Republic) During Republic, any person who wished hold public office had to conform to the Reformed Church and take an oath this effect. [...]
[ocument[5] (Title: Capital of the Netherlands) Although the proper legal status of Amsterdam capital of the Netherlands of recent date, city has been uniformly recognised as capital ever since 1814. [...]
Final Answer: Johan Remkes
<Without Uncertainty Mask
Generated Reasoning Chain:
[["Castricum", "is associated with", "castra"), ["Castra", "means", "Roman military camp"], ["Roman military camp", "was ruled by", "Nero Augustus Caesar"], ["The person charge of the place where Castricum is located", "is", "Nero Augustus Caesar"]]
Top-5 Retrieved Documents:
Castricum) Castricum a seaside town in the province of North Holland. It draws in a fair share of tourists who mainly come to visit the beach and nearby dune landscape. [...]
Document[L] (Titl:
Document(2) 21 (Title Forged from the Love Liberty) Patrick S. Castagne composed the words and music of the National Anthem in 1962. [...]
Document[3] (Title: Last Supper (del Castagno)) The Last Supper 14451450 fresco by Italian Renaissance artist Andrea del Castagno, located in the refectory of the convent of SantApollonia, now the "Museo di Cenacolo di SantApollonia", and accessed through a door on Via Ventisette Aprile at the with Santa Reparata, in Florence, region of Tuscany. [...]
Documentf4 4] (Title: Galicia (Spain)) The Roman legions first entered the area under Decimus Junius Brutus in 137136 BC, but the country only incorporated into the Roman Empire by the [...]
D5cument(5] (Titl: Saint Peter) According to Christian tradition, Peter was crucified in Rome under Emperor Nero Augustus Caesar. [...]
Final Answer: Nero Augustus Caesar
Figure 10: Case study of the uncertainty masks retrieval, illustrating differences in retrieved documents when the LLM marks uncertain entities with (UNCERTAIN) versus when does not.
37

---

Case Study: Effectiveness of Uncertainty Mask on Reasoning
Question:
Which film has the director who died later, 45 Calibre Echo or Bons Baisers De Hong Kong?
SUB Q1: Who directed the film Calibre Echo?
SUB Q2: Who directed the film Bons Baisers De Hong Kong?
SUB Q3: What was the date of death for the director of 45 Calibre Echo'
SUB Q4: What was the date of death for the director of Bons Baisers De Hong Kong?
Ground Truth: Bons Baisers Hong Kong
<iith Uncertainty Mask>
Uncertain Reasoning Chain:
[["45 Calibre Echo", "was directed by", "Bruce M. Mitchell"], [" (UNCERRTAIN)" "was directed by", "(UNCERTAIN)"),
("(UNCERTAIN)", "died on", "(UNCERTAIN;").
["Yvan Chiffre", "died on", "(UNCERTAIN))) ["Between the directors of the two films", "the one who died later is", "[FILL)"]]
Reconstructed Reasoning Chain:
[["45 Calibre Echo", "was directed by", "Bruce M. Mitchell"], ["Bons Baisers De Hong Kong", "was directed by", "Yvan Chiffre"], ["Bruce X. Mitchell", "died on", "September 26, 1952"], ["Yvan Chiffre", "died on", "27 September 2016"], ["Between the directors of the two films", "the one who died later is", "Yvan Chiffre"]]
Final Answer: Bons Baisers Hong Kong
<Without Uncertainty Mask
Generated Reasoning Chain:
[["45 Calibre Echo", "was directed by", "Bruce M. Mitchell"], ["Bons Baisers De Hong Kong", "was directed by", "Yvan Chiffre"], ["Bruce X. Mitchell", "died on", "December 31, 2020"], ["Yvan Chiffre", "died on", "May 15, 2000"], ["Between the directors of the two films", "the one who died later is", "Bruce M. Mitchell"]]
Final Answer: Bruce M. Mitchell
Figure 11: Case study of the uncertainty masks in reasoning
38

---

H
Prompt
H.1
Question Unrolling
Prompt: Question Unrolling
You are an assistant specialized multi-hop question answering and logical decomposition. Your task to analyze complex questions, break them down into reasoning steps, and create structured representations of the reasoning chain.
Follow these steps exactly when processing question:
1. Upon receiving question, determine the number of hops (reasoning steps) needed to answer it.
Utilize all information you know to the fullest extent in your reasoning processes.
Draw your existing knowledge without external search tools.
Provide brief logical explanation of how the question can be broken down into sequential reasoning
steps.
This explanation should help understand the reasoning path without revealing specific factual answers.
3. Decompose the original question into a series independent sub-questions that follow the logical reasoning path.
Each sub-question must be answerable with single piece of information.
4. Create structured reasoning chain using triples in the format [Head, Relation, Tail]".
Draw specific facts, relationships, and entities from your knowledge clearly define each component.
The final triple's tail should always be "(FILL)" to represent the answer to the original question.
*#Imporannt Constraints
When creating sub-questions:
You may expressions from the Original Question.
not use pronouns placeholders (like "it", "this person", etc.). Always use clear, specific and fully spelled-out entity names.
Each sub-question must be completely self-contained and independently answerable without requiring context from other sub-questions.
If the Original Question involves comparing elements, separate sub-question that explicitly asks for this comparison using the full names of the entitics heing compared.
If you are uncertain about an entity or lack confident knowledge about it, replace the entity with "(UNCERTAIN)"
Fininal Output #ormat*
Hop Count: [number]
Reasoning Structure: (brief explanation]
Sub-questions: ("Sub-question 1", "Sub-question 2", "Sub-question 3", ...]
Triple Reasoning Chain: [["Triplel_head", Striplei_relation: "Triplei_tail"), ("Triple2_heaal "Iriple2_relation", "Triple2.taill],,
""TriploN_head" "Tripleti_eeatton" "(FILL)"]]
Figure 12: Prompt for Question Unrolling.
39

---

H.2
Reasoning Chain Completion
Prompt: Reasoning Chain Completion
You are Triple Verification agent designed to precisely complete reasoning chains for Multi-ha hop Question- Answering. Your task is to examine the provided documents, main question, sub-questions, and reasoning chain containing placeholders marked as "(FILL)" or "(UNCERTAIN)"
Follow these steps exactly when processing question:
For each placeholder ("(FILL)" or "(UNCERTAIB)" or else), strictly replace with the exact phrase or word explicitly found in the provided documents. Do not paraphrase or introduce synonyms. Ensure every placeholder is replaced using only verbatim text extracted from the documents. If the reasoning chain is incomplete additional triples necessary for accurate reasoning, explicitly add triples by strictly using exact phrases words found in the provided documents.
2. Your completed reasoning chain must contain exclusively verbatim terms from the documents. Do not include introductory phrases, explanations, any additional commentary.
The final output should strictly follow the triple list format provided below without any deviations: [["Triple1, "errippe__headd,,. "Triplet_relationn"" reelation, "riipee_ttai"""
@TTripe2_heaad Drriple__relation" "Triple2_tail"",
["TripleN_hear
"Tripleti_relatine "[PILL)"]]
#*Input #ormat*s
DOCUMENTS:
[context]
MAIN QUESTIONS: [question]
SUB SOBSUUESSION: [sub_quest] ]
BAASONING_CHINY
[chain]
**Output *ormat**
Reconstructed Reasoning Chain:
[Reconstructed reasoning chain formatted exactly the triple list shown above, with all "(FILL)" "(UNCERTAIN) placeholders accurately replaced using exact phrases or words from the provided documents, any necessary new triples explicitly included]
Figure 13: Prompt for Reasoning Chain Completion.
40

---

H.3 Reasoning
Prompt: QA Reasoning
You are Multi-hop Question-Answen inference agent specialized in generating concise and factual answers based strictly on provided documents and a fully completed reasoning chain.
1. Given the documents, the main question, sub-questions, and the reconstructed reasoning chain, carefully infer the most concise and essential answer to the main question.
Your answer must strictly use phrases or words explicitly present either the completed reasoning chain or directly in the provided documents. Do not use synonyms, paraphrasing, or external knowledge.
3. Verify your inferred answer explicitly using the provided documents ensure its accuracy and factual correctness before finalizing.
4. Do not generate the answer in full sentence. Return only the most concise and essential term as the answer, avoiding any appended descriptions, subtitles, variations. If the question is binary (e.g.. "yes" or "no"), respond explicitly with "yes" or "no" without any additional explanation.
5. Indicate your answer precisely within the delimiters provided below.
**Input *ormat*
DOCUMENTS:
[context]
MAIN_QUE QUESTIONS:
[question]
SUB QUESTIONS: [ub__question]]
REASONING_CHATE
[chain]
**Output +
GENERATED ANSWER:
((ANS)) [Your Answer Here] ((ANS))
Figure 14: Prompt for QA Reasoning.
41

---

H.4 Multi-step Key Extraction
Prompt: Multi-step Key Extraction
You are an expert reasoning agent for multi-hop question answering, using Beam Retrieval framework iteratively select and analyze the most relevant document at each step. Your task is to iteratively select the most helpful document among the provided documents in order to deduce the final answer. At each iteration, will receive top-10 10 documents in the following fixed format:
Document [i]] (Title: [title]) [text]
For each iteration, you must follow these steps:
1. Analyze all 5 documents and select the document that contains the most relevant information.
2. From the selected document, choose single "Key Sentence" that best contributes to deducing the answer.
3. Output tuple in the exact format below:
**Output **rmat**
([i), "Key Sentence"). So the answer is: [Final Answerl
"[i]" the index of the selected document.
Key Sentence" must be exactly the sentence consider most informative.
"[Final Answer]" should be the final deduced answer if it fully determined; if the final answer cannot be deduced, output False (i.e. So the answer is: False).
Output format must "(ii. "Key Sentence"). So the answer is: [Final Answer]".
Figure 15: Prompt for Multi-step Key Extraction.
42


Reviews:
[{"review_id": "tZ1CYFuXAc", "reviewer": "Reviewer_S8QW", "summary": "The paper proposes a retrieval-augmented generation approach called CoopRAG for multi-hop QA and single hop QA tasks. \n\nThe proposed approach consists of multiple modules in sequence: (a) retriever (b) re-ranker, and (c) (generative) language model. The main novelty is in the design and training methodology of the individual components and jointly. Given a query, first the language model uses its knowledge to decompose a question into multiple sub-questions, along with reasoning knowledge triples required to answer the question. To emphasize the LM’s uncertainty in the reasoning chain, it is allowed to output masked tokens with placeholder tokens. Given this information, the retriever selects the top-n documents from candidate pool which is fed to re-ranker module to further select the most relevant top-k documents (K < N). The re-ranker is trained using a contrastive layers approach while the retriever training loss is modified to give importance to difficult examples. The answer is generated given the re-ranked documents along with the incomplete steps of the reasoning chain.\n\nExperimental results on three multi-hop and single-hop datasets are state-of-the-art. A set of ablation studies highlights the importance individual design decisions such as training methodology and LM prompt design for the reasoning chain.", "questions": "Please see the Weakness section above.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The proposed system consists of well designed multiple sub-components. The LM is encouraged to expand the query by asking sub-questions, provide a reasoning chain, and using accurate set of retrieved documents to first complete the reasoning trace and answer the query. A couple of novel ideas are presented to improve the retrieved set of documents - First, by contrastively using the transformer layers in ranking and giving more weight to the difficult examples during model training process. These design choices lead to accuracy improvements in end evaluations.\n* Good performance results: The effectiveness of the proposed system design of CoopRAG is shown in obtaining new state-of-the-art results across four hard datasets on both their retrieval and end QA task, which is quite significant. They have compared the method against a range of recent baselines.\n\n\nWeaknesses:\n* My main point of concern is that the method is not really tested at scale. As shown in Table 1 (Sec 4.1), the candidate size of the evidence pool is quite smaller (on the range of 10000 documents). To really understand how these methods would perform in real-world QA tasks, ideally, the candidate size of Wikipedia should be considered and evaluated against. On smaller candidate size, its easy to get a very high retrieval recall as there aren’t many strong distracting negatives.\n* More evaluation datasets needed: The datasets such as HotpotQA and Natural Questions are now somewhat old and so there are chances that their reference answers won’t be conflicting when provided with updated knowledge bases such as Wikipedia. It would be nice to have some results on more recent datasets such as SimpleQA.", "was_revised": false, "final_justification": "", "rebuttal": "# **Dear Reveiwer S8QW**   \nThank you very much for your thoughtful and constructive feedback. We truly appreciate your time and valuable comments. We have carefully addressed each of your suggestions, and we will incorporate all feedback and new results into the final version. If you find our responses satisfactory, we kindly request that you reconsider the evaluation of our work and, if possible, raise the score above the middle range. Your support would be greatly appreciated. If you have any further questions or suggestions, we would be happy to provide clarification. Thank you again for your consideration.\n\n----------------\n\n###  **W1) My main point of concern is that the method is not really tested at scale. As shown in Table 1 (Sec 4.1), the candidate size of the evidence pool is quite smaller (on the range of 10000 documents). To really understand how these methods would perform in real-world QA tasks, ideally, the candidate size of Wikipedia should be considered and evaluated against. On smaller candidate size, its easy to get a very high retrieval recall as there aren’t many strong distracting negatives.** \n\n| Models                | Candidate |  Size |       |       |       |       |\n| --------------------- | -------------- | ----- | ----- | ----- | ----- | ----- |\n|                       | 10000          | 20000 | 30000 | 40000 | 50000 | 60000 |\n| HippoRAG2             | 95.7           | 86.5  | 78.2  | 56.3  | 41.7  | 33.2  |\n| HopRAG                | 96.0           | 85.6  | 77.1  | 58.8  | 43.4  | 38.9  |\n| CoopRAG               | 96.8           | 88.6  | 81.9  | 68.7  | 59.6  | 56.8  |\n| Performance  Gain (%) | 0.83           | 2.43  | 4.73  | 16.84 | 37.33 | 46.02 |\n\n**Table1.** Retrieval performance  of CoopRAG and main baselines at different Wikipedia candidate scales.\n\n| Models    | Candidate |  Size |       |       |       |       |\n| --------- | -------------- | ----- | ----- | ----- | ----- | ----- |\n|           | 10000          | 20000 | 30000 | 40000 | 50000 | 60000 |\n| HippoRAG2 | 2.25           | 2.72  | 2.88  | 3.56  | 3.89  | 4.32  |\n| HopRAG    | 13.3           | 15.3  | 16.0  | 20.2  | 24.3  | 29.6  |\n| CoopRAG   | 2.01           | 2.28  | 2.53  | 2.79  | 3.05  | 3.26  |\n\n**Table2.**  Latency  of CoopRAG and main baselines at different Wikipedia candidate scales.\n\nWe assess the scalability of CoopRAG by varying the candidate size in Wikipedia. Table 1 shows that as the number of candidates increases, CoopRAG exhibits much less performance degradation than the state-of-the-art baselines HippoRAG2 and HopRAG. With 10,000 documents, CoopRAG achieves a 0.83% gain over HopRAG, which increases to 46.02% at 60,000 documents. Furthermore, we also measure the retrieval time as the number of candidates increases, as shown in Table 2. CoopRAG consistently demonstrates faster retrieval times than all baselines, regardless of candidate size.Thus, CoopRAG remains more robust than other baselines even in large candidate settings.\n\n----------------\n\n### **W2) More evaluation datasets needed: The datasets such as HotpotQA and Natural Questions are now somewhat old and so there are chances that their reference answers won’t be conflicting when provided with updated knowledge bases such as Wikipedia. It would be nice to have some results on more recent datasets such as SimpleQA.**\n| Models    | SimpleQA  |      |\n| --------- | -------------------------- | ---- |\n|           | EM                         | F1   |\n| HippoRAG2 | 48.2                       | 55.0 |\n| HopRAG    |  50.2                       | 58.2 |\n| CoopRAG   | **58.3**                       | **67.6** |\n\n**Table 3.** Performance comparison on the SimpleQA dataset, where the number of documents is 47,229.\n\nWe have conducted an additional experiment comparing CoopRAG with the two recent baselines, HippoRAG2 and HopRAG, on the SimpleQA dataset. As shown in Table 3, CoopRAG outperforms the baselines in both EM and F1 scores, achieving 16.1% higher EM than HopRAG. These results demonstrate that CoopRAG generalizes well to recent and challenging datasets, thereby supporting its robustness beyond historical benchmarks."}, {"review_id": "75CqWXbreC", "reviewer": "Reviewer_pyhp", "summary": "The paper introduces CoopRAG, a framework for RAG in QA. CoopRAG aims at addressing issues in existing RAG systems, especially in multi-hop QA setup, by fostering cooperation between the retriever and large language model. The core innovations include: 1/ decomposing the original question into sub-questions and a reasoning chain with masked uncertain elements, 2/ using the unrolled question to retrieve documents, leveraging the internal knowledge of the LLM, 3/ reranking retrieved documents by contrasting representations from different layers of the retriever.  Experimental results on three multi-hop QA datasets (HotpotQA, 2WikiMultihopQA, MuSiQue) and one QA dataset (NaturalQuestions) show that CoopRAG outperforms state-of-the-art QA methods in both retrieval and answer accuracy.", "questions": "see Strengths And Weaknesses", "limitations": "No concerns", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The paper is technically sound, and the proposed framework yields promising results. However, the complexity of the approach is a key limitation that requires deeper discussion: \n1) While query decomposition is a well-established method for improving RAG performance, it introduces overhead through multiple LLM calls and relies heavily on the model’s reasoning capabilities. The effectiveness of question unrolling and reasoning chain completion similarly depends on the LLM's strength. An ablation or analysis on the LLM’s impact would strengthen the claims.\n\n2) The experimental setup could also be more rigorous. The use of historical benchmarks is a limitation and make it hard to understand the generalization capabilities of this approach. Using more recent and challenging datasets (e.g., SimpleQA, FreshLLM) would provide a more robust evaluation. \n\n3) Comparisons with baselines should include complexity and latency metrics to better contextualize the practical trade-offs of the proposed method. Finally, I would encourage authors to add details on how they implemented the QA performance metrics.", "was_revised": true, "final_justification": "I thank the authors for their work and effort in addressing my concerns. I do think that the paper would benefit from another round of review given the amount of changes that are needed to incorporate the feedback. I'll keep my score to 3.", "rebuttal": "# Dear Reviewer pyhp,\nThank you very much for your thoughtful and constructive feedback. We truly appreciate your time and valuable comments. We have carefully addressed each of your suggestions and will incorporate all feedback and new results into the final version. If you find our responses satisfactory, we kindly request that you reconsider the evaluation of our work and, if possible, raise the score above the middle range. Your support would be greatly appreciated. If you have any further questions or suggestions, we would be happy to provide clarification. Thank you again for your consideration.\n\n----------\n\n### **W1) While query decomposition is a well-established method for improving RAG performance, it introduces overhead through multiple LLM calls and relies heavily on the model’s reasoning capabilities. The effectiveness of question unrolling and reasoning chain completion similarly depends on the LLM's strength.**\n| Models                  | Retrieval |          | Reasoning |      |\n| ----------------------- | --------- | -------- | --------- | ---- |\n|                         | Reacll@2  | Reacll@2 | EM        | F1   |\n| HippoRAG2 (GPT-4o-mini) | 80.5      | 95.7     | 56.3      | 71.1 |\n| HopRAG (GPT-4o-mini)    | 81.1      | 96.0     | 62.0      | 76.1 |\n| CoopRAG (Gemma3-1B)     | 84.9      | 96.2     | 64.0      | 77.9 |\n| CoopRAG (Gemma2-2B)     | 83.7      | 96.4     | 62.4      | 76.6 |\n| CoopRAG (Llama3.3-70B)  | 86.9      | 96.6     | 64.7      | 79.0 |\n\n**Table 1.** Performance comparison of CoopRAG with varying LLM scales.\n\nTo assess the robustness of CoopRAG under limited reasoning capacity, we have conducted additional experiments on the HotpotQA dataset using small-scale LLMs such as Gemma3-1B and Gemma2-2B. As shown in Table 1, CoopRAG outperforms two state-of-the-art baselines, HippoRAG2 and HopRAG, both powered by GPT-4o-mini, in terms of both retrieval and reasoning. Notably, **the performance gap for CoopRAG between using Llama3.3-70B and using the small LLMs is marginal**. These results demonstrate that CoopRAG’s performance gains are not merely due to the reasoning power of large LLMs, but stem from its uncertainty-aware query decomposition and reasoning chain completion strategies. This confirms that CoopRAG remains robust and effective even when operating with smaller language models.\n\n| Models            | Retrieval Performance |          | QA Performance |      | LLM Calls per question |          |           |\n| ----------------- | --------------------- | -------- | -------------- | ---- | ---------------------- | -------- | --------- |\n|                   | Recall@2              | Recall@5 | EM             | F1   | Preprocessing          | Retrieve | Reasoning |\n| HippoRAG2         | 80.5                  | 95.7     | 56.3           | 71.1 | 4                      | 1        | 1         |\n| HopRAG            | 81.1                  | 96.0     | 62.0           | 76.1 | 12                     | 14.96    | 1         |\n| CoopRAG (Unified) | 88.8                  | 96.8     | 63.1           | 76.6 | 0                      | 1        | 1         |\n| CoopRAG           | 88.8                  | 96.8     | 65.6           | 78.9 | 0                      | 1        | 2         |\n\n**Table 2.** Comparison of LLM Call Counts and Model Performance\n\nFurthermore, we compare the number of LLM calls required by CoopRAG and the state-of-the-art baselines, i.e.,  HippoRAG2 and HopRAG, across preprocessing, retrieval, and inference stages. As shown in Table 2, HippoRAG2 and HopRAG require 4 and 12 LLM calls per question during preprocessing, respectively, while **CoopRAG needs no preprocessing and thus requires no LLM calls at this stage**. Notably, both HopRAG and HippoRAG2 repeatedly invoke the LLM for each document to generate triples, which leads to a substantial increase in the number of LLM calls during preprocessing. During retrieval, HopRAG further incurs an average of 14.9 LLM calls per question because it uses a graph-based iterative triple extraction method. In the inference stage, CoopRAG calls the LLM twice per question. One call is for reasoning chain completion, and the other is for reasoning. Despite this, CoopRAG achieves up to 16.5% higher EM compared to the baselines. **CoopRAG (Unified), which combines the two reasoning steps into a single LLM call, also outperforms both baselines**. In summary, CoopRAG is more efficient and more effective, requiring fewer LLM calls to achieve superior performance.\n\n----------\n\n### **W2) The experimental setup could also be more rigorous. The use of historical benchmarks is a limitation and make it hard to understand the generalization capabilities of this approach. Using more recent and challenging datasets (e.g., SimpleQA, FreshLLM) would provide a more robust evaluation.**\n\n| Models    | SimpleQA |(Document: 47,229)      |      | FreshQA  | (Document: 11,602)     |      |      |\n| --------- | --------------------------- | ---- | ---- | -------------------------- | ---- | ---- | ---- |\n|           | EM                          | F1   |      | EM                         | F1   | Correct | Incorrect | Not Attempted |\n| HippoRAG2 | 48.2                        | 55.0 |      | 21.3                       | 29.5 | 135     | 387       | 31            |\n| HopRAG    | 50.2                        | 58.2 |      | 21.1                       | 28.7 | 147     | 361       | 45            |\n| CoopRAG   | 58.3                        | 67.6 |      | 26.6                       | 35.3 | 283     | 250       | 23            |\n\n**Table 3.** Performance comparison on the SimpleQA and FreshQA dataset.\n\nWe have conducted experiments on the SimpleQA [1] and FreshQA [2] datasets, comparing CoopRAG with the two stage-of-the-art baselines. As shown in Table 3, CoopRAG outperforms the baselines in both EM and F1 scores. CoopRAG achieves 16.1% higher EM than HopRAG on SimpleQA and 26.1% higher EM on FreshQA, respectively. Since FreshQA contains many sentence-level answers, we also evaluate factual correctness using the ChatGPT grader proposed in the SimpleQA paper. Answers are graded as ‘Correct’ if the prediction fully contains the ground truth answer without contradiction, ‘Incorrect’ if there exists any contradiction (even partial or hedged), and ‘Not Attempted’ if necessary information is missing but not contradicted. For both inference and grading, we use GPT-4o-mini for all models. Importantly, while CoopRAG achieves an EM of 26.6% (147 out of 553 questions), the ChatGPT grader finds 283 answers factually correct. This highlights that EM may underestimate the actual factual correctness\n\nCompared to HopRAG, which produces only 147 factually correct answers, CoopRAG yields over 92% more factually correct responses. These results demonstrate that CoopRAG generalizes even better to recent and challenging datasets, highlighting larger performance gains on such benchmarks.\n\n----------\n\n### **W3) Comparisons with baselines should include complexity and latency metrics to better contextualize the practical trade-offs of the proposed method. Finally, I would encourage authors to add details on how they implemented the QA performance metrics.**\n\n| Models     | HotpotQA |      |      | MuSiQue |      |      |\n|------------|----------|------|------|---------|------|------|\n|            | Retrieval| QA   | Total| Retrieval| QA   | Total|\n| HippoRAG2  | 2.25     | 1.96 | 4.21 | 2.11    | 1.81 | 3.92 |\n| CoopRAG    | 2.04     | 1.86 | 3.90 | 1.98    | 1.65 | 3.63 |\n\n**Table 4.** Latency (seconds) of HippoRAG2 and CoopRAG on HotpotQA and MuSiQue. Lower values indicate higher efficiency.\n\nFirst, CoopRAG uses a retriever and reranking pipeline. The time complexity for retrieval is $O(N \\cdot d)$ where $N$ is the number of documents and $d$ is the embedding dimension. For reranking, the time complexity of RaLa without our optimization as in Equation 3 of Section 3.4 is $O(L_q \\cdot L_d \\cdot d \\cdot \\vert B \\vert)$, where $L_q$ is the query length, $L_d$ is the document length, $\\vert B \\vert$ is the bucket size. According to our optimization strategy in Equation 4 of Section 3.4, only $[CLS]$ tokens are compared per bucket, reducing the complexity to $O(L_q \\cdot L_d \\cdot d)$, which is asymptotically identical to that of ColBERT. In all our experiments, we use this optimized version. As shown in Table 4, CoopRAG achieves lower latency than HippoRAG2 for retrieval, reasoning, and end-to-end processing, while also delivering better retrieval and QA performances. We will provide comprehensive efficiency comparisons across all datasets in the efficiency section, Appendix D.\n\nSecond, we compute all metrics following HippoRAG2 under the same evaluation setting. We report Recall@k for retrieval performance and use EM and F1 scores to evaluate QA performance. We will add a description of how we conduct experiments for both our model and the baselines, including implementation details and evaluation procedures, in Appendix A. \n\n----------\n\n### **Reference**\n[1] Jason Wei, Nguyen Karina, Hyung Won Chung, and etc. 2024. Measuring short-form factuality in large language models. arXiv:2411.04368 [cs.CL] https://arxiv.org/ abs/2411.04368  \n[2] Tu Vu, Mohit Iyyer, Xuezhi Wang, and etc. 2023. FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation. arXiv:2310.03214 [cs.CL] https://arxiv.org/abs/2310.03214"}, {"review_id": "mUaEUcxdwr", "reviewer": "Reviewer_h3Qx", "summary": "This paper presents CoopRAG, a novel RAG framework in which a language model first decomposes the input question into sub-questions and a masked reasoning chain. A retriever then retrieves and reranks documents based on this unrolled query, and finally, the language model fills in the masked components to generate the answer. \nFor reranking, the authors propose RaLa, a method that computes the maximum difference of cosine similarity between the query and the final layer and a lower layer. his is based on the hypothesis that higher transformer layers capture semantic similarity, while lower layers primarily encode surface-level or stylistic features. The retriever, based on MPNet, is further trained with difficulty-aware weighting to emphasize challenging examples and prevent overfitting to easy questions.\nExperimental results across several QA benchmarks, including HotpotQA, 2WikiMultihopQA, and Natural Questions, demonstrate the effectiveness of CoopRAG in both retrieval accuracy and final answer quality.", "questions": "- Are other baselines also training their retrieval systems on the target dataset?", "limitations": "The authors included a limitation section, such as long-context capabilities and further evaluations on KBQA.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n- A new training methodology for retrieval systems based on previous findings about transformers' internal representation differences: To my knowledge, I am not aware of retrieval training methods that use the same approach (contrasting the last and lower layers).\n\n- Strong results on short-form QA datasets: The experimental results show that the proposed methods outperform HippoRAG2, HopRAG, and SiReRAG.\n\n**Weaknesses**\n\n- Technical novelty: The proposed framework (CoopRAG) seems like a combination of multiple incremental ideas—decomposing/enhancing queries using LLMs before retrieving documents to improve retrieval results has been widely studied in RAG. While I think RaLa is an interesting new idea, the evaluation focuses on comparisons with other recent RAG systems that do not use trained retrievers, making me wonder whether RaLa is indeed better than other retrieval training objectives.\n\n- **Baselines**: Related to the point above, to my knowledge (correct me if I’m wrong), all baseline methods do not train their retrieval systems on the target datasets by default, as their main focus is more on inference-time algorithms and interactions between the retriever and the LM. In contrast, the proposed method trains retrievers on the target datasets. This makes me wonder whether the authors could compare model performance with methods that also train retrieval models on the target datasets.\n\n- **Evaluations**: Evaluations are primarily conducted on simple, short-form QA datasets (synthetic multi-hop and simple single-hop questions). These datasets are often more easily decomposed, so I was curious how the proposed \"unrolling\" strategies would perform on more natural, complex questions, such as those in SimpleQuestions or natural search queries (e.g., Search Arena queries). Given that intermediate reasoning chains assume knowledge-triple formats, I am afraid this method may not generalize well beyond entity-centric, factoid questions.", "was_revised": true, "final_justification": "I appreciated the authors' response, and the new results addressed my concerns about evaluations (baselines and additional datasets). That being said, I think the technical novelties are somewhat limited, and thus my final score is 4.", "rebuttal": "# Dear Reviewer h3Qx, \nThank you very much for your thoughtful and constructive feedback. We truly appreciate your time and valuable comments. We have carefully addressed each of your suggestions and will incorporate all feedback and new results into the final version. If you find our responses satisfactory, we kindly request that you reconsider the evaluation of our work and, if possible, raise the score above the middle range. Your support would be greatly appreciated. If you have any further questions or suggestions, we would be happy to provide clarification. Thank you again for your consideration.\n\n----------\n\n### **W1) Technical novelty: The proposed framework (CoopRAG) seems like a combination of multiple incremental ideas—decomposing/enhancing queries using LLMs before retrieving documents to improve retrieval results has been widely studied in RAG. While I think RaLa is an interesting new idea, the evaluation focuses on comparisons with other recent RAG systems that do not use trained retrievers, making me wonder whether RaLa is indeed better than other retrieval training objectives.**\n\nMany prior studies have explored decomposing or enhancing queries using LLMs before retrieval in order to improve retrieval quality. However, these approaches do not consider the uncertainty that may arise in the answers or outputs of LLMs, which can introduce incorrect or misleading information into the retrieval process. CoopRAG addresses this limitation by guiding the LLM to identify and mask uncertain spans within the query. This prevents unreliable information from influencing retrieval results and ensures that only trustworthy parts of the reasoning chain are used for document retrieval. Furthermore, CoopRAG introduces a cooperative reasoning mechanism. After the retrieval, the LLM revisits the masked reasoning chain and fills them in, using factual evidence from the retrieved documents. This two-stage process improves not only retrieval accuracy but also reasoning robustness by grounding uncertain parts of the query in verifiable information.\n\nRegarding your second point, we have demonstrated through extensive experiments that RaLa outperforms other retrieval training objectives. Detailed results of this comparison are provided in our response to W2.\n\n----------\n\n### **W2) Baselines: Related to the point above, to my knowledge (correct me if I’m wrong), all baseline methods do not train their retrieval systems on the target datasets by default, as their main focus is more on inference-time algorithms and interactions between the retriever and the LM. In contrast, the proposed method trains retrievers on the target datasets. This makes me wonder whether the authors could compare model performance with methods that also train retrieval models on the target datasets.**\n\n| Models        | HotpotQA |        | MuSiQue |        | 2WikiMultihopQA |        |\n| ------------- | -------- | ------ | ------- | ------ | --------------- | ------ |\n|               | R@2      | R@5    | R@2     | R@5    | R@2             | R@5    |\n| **Without Training** |        |        |         |        |                 |        |\n| Contriever    | 57.3     | 74.8   | 32.4    | 43.5   | 55.3            | 65.3   |\n| ColBERTv2     | 64.7     | 79.3   | 37.9    | 49.2   | 59.2            | 68.2   |\n| ReSCORE       | 68.2     | 80.9   | 38.6    | 49.1   | 62.2            | 73.3   |\n| CoopRAG       | 73.9     | 86.1   | 45.9    | 60.7   | 64.6            | 75.0   |\n| **Fine-tuning**      |        |        |         |        |                 |        |\n| Contriever    | 63.4     | 80.5   | 36.8    | 44.9   | 60.6            | 68.2   |\n| ColBERTv2     | 78.2     | 89.3   | 52.6    | 68.5   | 71.6            | 82.1   |\n| ReSCORE       | 82.3     | 92.6   | 54.9    | 72.3   | 78.8            | 95.1   |\n| CoopRAG       | 88.8     | 96.8   | 59.6    | 75.7   | 80.4            | 96.6   |\n\n**Table 1.** Performance Comparison of Retrievers Before and After Fine-tuning\n\nAs shown in Table 1, we compare the performance of RaLa and other retrievers before and after fine-tuning. For a fair comparison, all retrievers receive only the question as input, and question unrolling is not applied. The results show that RaLa consistently outperforms other recent retriever models across all datasets. In response to your comment, we additionally include ReSCORE [1], a recent fine-tuned retriever, for comparison. Notably, after fine-tuning, RaLa outperforms the state-of-the-art retriever ReSCORE, achieving a 7.8% higher Recall@2 on HotpotQA, which demonstrates its strong retrieval capability and adaptability.\n\n----------\n\n### **W3) Evaluations: Evaluations are primarily conducted on simple, short-form QA datasets (synthetic multi-hop and simple single-hop questions). These datasets are often more easily decomposed, so I was curious how the proposed \"unrolling\" strategies would perform on more natural, complex questions, such as those in SimpleQuestions or natural search queries (e.g., Search Arena queries). Given that intermediate reasoning chains assume knowledge-triple formats, I am afraid this method may not generalize well beyond entity-centric, factoid questions.**\n\n| Models    | SimpleQA |(Document: 47,229)      |      | FreshQA  | (Document: 11,602)     |      |      |\n| --------- | --------------------------- | ---- | ---- | -------------------------- | ---- | ---- | ---- |\n|           | EM                          | F1   |      | EM                         | F1   | Correct | Incorrect | Not Attempted |\n| HippoRAG2 | 48.2                        | 55.0 |      | 21.3                       | 29.5 | 135     | 387       | 31            |\n| HopRAG    | 50.2                        | 58.2 |      | 21.1                       | 28.7 | 147     | 361       | 45            |\n| CoopRAG   | 58.3                        | 67.6 |      | 26.6                       | 35.3 | 283     | 250       | 23            |\n\n\n**Table 2.** Performance comparison on the SimpleQA and FreshQA datasets.\n\nBoth SimpleQuestions and Search Arena Queries are not compatible with the document-based QA setting and our evaluation setup. SimpleQuestions is a dataset of the KGQA task that requires reasoning over knowledge graphs rather than retrieving from text corpora. Search Arena is a human preference dataset where multiple LLM responses to a query are compared by human judgement, and ground truth answers are not available.\n\nInstead, we evaluated CoopRAG on two recent factual QA benchmarks: SimpleQA [2] and FreshQA [3]. As shown in Table 2, CoopRAG outperforms the state-of-the-art baselines HippoRAG2 and HopRAG on both datasets in terms of EM and F1. CoopRAG achieves 16.1% and 26.1% higher EMs than HopRAG on SimpleQA and FreshQA, respectively. Since FreshQA includes many sentence-level answers, we also use the ChatGPT grader from the SimpleQA paper to assess factual correctness. With this grader, CoopRAG produces 283 factually correct answers out of 553 queries, compared to only 147 by exact match, which is 92% more correct answers than HopRAG. These results demonstrate that CoopRAG generalizes well to recent and challenging QA benchmarks, showing even greater performance gains in these settings.\n\n\n### **Reference**\n[1] Dosung Lee, Wonjun Oh, Boyoung Kim, and etc. 2025. ReSCORE: Label-free Iterative Retriever Training for Multi-hop Question Answering with Relevance-Consistency Supervision. In Proceedings of the 63rd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 341–359. https://aclanthology.org/2025.acl-long.16/  \n[2] Jason Wei, Nguyen Karina, Hyung Won Chung, and etc. 2024. Measuring short-form factuality in large language models. arXiv:2411.04368 [cs.CL] https://arxiv.org/ abs/2411.04368  \n[3] Tu Vu, Mohit Iyyer, Xuezhi Wang, and etc. 2023. FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation. arXiv:2310.03214 [cs.CL] https://arxiv.org/abs/2310.03214"}, {"review_id": "J3mBrQDeCz", "reviewer": "Reviewer_aR4a", "summary": "This paper introduces CoopRAG to address inaccurate retrievals and hallucinations in Retrieval-Augmented Generation (RAG) for simple and multi-hop QA by enabling a retriever and an LLM to cooperate via mutual information exchange and layer-contrastive reranking. CoopRAG first unrolls a question into sub-questions and a masked reasoning chain, then performs Unrolling-Augmented Retrieval, applies Ranking by Contrasting Layers (RaLa) for reranking, and finally completes the reasoning chain via LLM-driven entity filling. Experiments on benchmarks show the effectiveness of the proposed approach.", "questions": "See the Weakness.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:  \n1. The topic of the paper is interesting and the proposed approach is novel.  \n2. The proposed approach show good performance on benchmark datasets.  \n3. The paper is easy to follow.\n\n\nWeaknesses:  \n1. It would be better if the authors can provide case studies or visualizations to illustrate the effectiveness of the proposed approach.  \n2. It is suggested to perform an error analysis with failed case so that future studies can know how to improve it.   \n3. The proposed model is evaluated with four LLMs (i.e., Gemma2-9B, Gemma2-27B, Llama3.3-70B, and GPT-4o-mini). It is recommended to also evaluate the baselines on the same group of LLMs and report the results in Table 2.", "was_revised": true, "final_justification": "Thanks for the author's responses.\n\nI would suggest that the authors include the case studies and error analyses in the main body of the paper if space is allowed.\n\nAfter carefully reading the author's response and other reviews, I raised the Quality and Clarity scores but kept the final rating unchanged.", "rebuttal": "# Dear Reviewer aR4a,\nThank you very much for your thoughtful and constructive feedback. We truly appreciate your time and valuable comments. We have carefully addressed each of your suggestions and will incorporate all feedback and new results into the final version. If you find our responses satisfactory, we kindly request that you reconsider the evaluation of our work and, if possible, raise the score above the middle range. Your support would be greatly appreciated. If you have any further questions or suggestions, we would be happy to provide clarification. Thank you again for your consideration.\n\n--------\n\n### **W1) It would be better if the authors can provide case studies or visualizations to illustrate the effectiveness of the proposed approach.**\nWe provide four case studies, which are briefly summarized below and described in more detail in Appendix G.\n- **Example of RaLa in Appendix G.1**: We visualize layer-wise similarity as a heatmap and show that RaLa better distinguishes the correct document by reducing surface-level bias compared to standard scoring.\n- **End-to-end Process in Appendix G.2**: Figure 9 presents a case study of the end-to-end CoopRAG process, showing the prompts used at each stage from question decomposition to final answer generation.\n- **Example of Retrieval Error in Appendix G.3**: A case study demonstrates that using uncertainty masks improves retrieval by focusing on reliable information in the reasoning chain, resulting in more accurate and factual answers.\n- **Example of Reasoning Error in Appendix G.4**: We demonstrate that incorporating uncertainty masks into the reasoning chain enables the model to explicitly mark uncertain spans, which are later resolved using retrieved documents. This process prevents reasoning errors and improves factual accuracy.\n\n------------\n\n### **W2) It is suggested to perform an error analysis with failed case so that future studies can know how to improve it.**\n\n| Error Type        | HotpotQA | MuSiQue | 2WikiMultihopQA |\n|-------------------|----------|---------|-----------------|\n| Retrieval Error   | 8.96%    | 24.30%  | 10.70%          |\n| Dataset Error     | 38.30%   | 40.60%  | 39.40%          |\n| Reasoning Error   | 52.74%   | 35.10%  | 49.90%          |\n\n**Table 1.** Error type distribution (%) for HotpotQA, MuSiQue, and 2WikiMultihopQA in CoopRAG.\n\nAs shown in Table 1, we categorize errors into three types: retrieval error, dataset error, and reasoning error. Retrieval error refers to cases where some or all key documents are missing from the retrieved set, with MuSiQue showing the highest proportion (24.30%) since it typically requires more supporting documents per question.\n\nDataset error refers to cases (excluding retrieval errors) where the ground truth does not provide sufficient information to answer the question, or where the answer is expressed differently in the document (e.g., \"John Fitzgerald Kennedy\" vs. \"J. F. Kennedy\"). We measured these rates by providing the question, ground truth documents, and answer to GPT-4o-mini, resulting in 38.30% for HotpotQA, 40.60% for MuSiQue, and 39.40% for 2WikiMultihopQA. \n\nAll remaining errors are classified as reasoning errors, which mainly arise from the LLM's generative limitations or hallucinations due to long context.\n\n-------\n### **W3) The proposed model is evaluated with four LLMs (i.e., Gemma2-9B, Gemma2-27B, Llama3.3-70B, and GPT-4o-mini). It is recommended to also evaluate the baselines on the same group of LLMs and report the results in Table 2**\n\nAs shown in Appendix C.1 and C.2, CoopRAG consistently outperforms state-of-the-art baselines when evaluated with the same LLMs. On NaturalQuestions, CoopRAG (GPT-4o-mini) improves Recall@2 by 34.6% and Recall@5 by 15.5% over GritLM-7B, and achieves 15.2% higher EM than HippoRAG2 when using Gemma-2-9B. For QA performance on 2WikiMultihopQA, CoopRAG outperforms HippoRAG2 by 9.53% in EM with Llama3.3-70B and by 17.3% with GPT-4o-mini. These results demonstrate CoopRAG’s clear advantage in both retrieval and QA tasks across different LLMs."}]

Author rebuttal:

Meta-review: - This paper presents Cooperative Retrieval-Augmented Generation for Question Answering, where a retriever and an LLM work cooperatively by exchanging informative knowledge, and the earlier and later layers of the retriever model work cooperatively to rank the retrieved documents relevant to a given query
- The paper proposes some interesting ideas about contrasting LLM layers for retriever training, has solid and promising results. There were several concerns (such as not evaluated at scale, missing more recent eval benchmarks), but they have been largely addressed after rebuttal. The remaining holdback is around limited novelty and whether the authors will be able to incorporate all the feedback into paper revision.
- Overall, this looks like a borderline accept to me
Decision: Accept (poster)