RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation
Authors: Chi-Min Chan
Chunpu Xu
Ruibin Yuan
Hongyin Luo
Wei Xue
Yike Guo
Jie Fu
Venue: COLM (2024)
Keywords: retrieval augmented generation, tool learning
TL;DR: This paper introduces RQ-RAG,  which improves LLMs by adaptively refining queries for better accuracy under retrieval augmented generation scenario, beating previous SOTA method by 1.9% on three single-hop QA datasets.

Abstract: Large Language Models (LLMs) exhibit remarkable capabilities but are prone to generating inaccurate or hallucinatory responses. This limitation stems from their reliance on vast pretraining datasets, making them susceptible to errors in unseen scenarios. To tackle these challenges, Retrieval Augmented Generation (RAG) addresses this by incorporating external, relevant documents into the response generation process, thus leveraging non-parametric knowledge alongside LLMs’ in-context learning abilities.
However, existing RAG implementations primarily focus on initial input for context retrieval, overlooking the nuances of ambiguous or complex queries that necessitate further clarification or decomposition for accurate responses. To this end, we propose learning to Refine Queries for Retrieval Augmented Generation (RQ-RAG) in this paper, endeavoring to enhance the model by equipping it with capabilities for explicit rewriting, decomposition, and disambiguation. Our experimental results indicate that our method, when applied to a 7B Llama2 model, surpasses the previous state-of-the-art (SOTA) by an average of 1.9% across three single-hop QA datasets, and when applied to a 8B Llama3 model, it also demonstrates enhanced performance in handling complex, multi-hop QA datasets.

Published as a conference paper at COLM 2024
RQ-RAG: Learning to Refine Queries for Retrieval Aug- mented Generation
Chi-Min Chan', Chumpu Xu², Ruibin Yuan', Hongyin Luo Wei Xue',
Yike Guo Jie Ful-
1 Hong Kong University of Science and Technology
2 Hong Kong Polytechnic University
3 Massachusetts Institute of Technology
zorowin1 20@gma33.......mm com
Abstract
Large Language Models (LLMs) exhibit remarkable capabilities but prone to generating inaccurate or hallucinatory responses. This limitation stems from their reliance on vast pretraining datasets, making them sus- ceptible to errors in unseen scenarios. To tackle these challenges, Retrieval- Augmented Generation (RAG) addresses this by incorporating external, relevant documents into the response generation process, thus leveraging non-parametric knowledge alongside LLMs' in-context learning abilities. However, existing RAG implementations primarily focus on initial input for context retrieval, overlooking the nuances of ambiguous or complex queries that necessitate further clarification or decomposition for accurate responses. To this end, we propose learning to Refine Queries for Re- trieval Augmented Generation (RQ-RAG) in this paper, endeavoring to enhance the model by equipping it with capabilities for explicit rewriting, decomposition, and disambiguation. Our experimental results indicate that our method, when applied to a 7B Llama2 model, surpasses the previous sta state-of-the-art (SOTA) by an average of 1.9% across three single-hop QA datasets, and when applied to a 8B Llama3 model, it also demonstrates enhanced performance in handling complex, multi-hop QA datasets.
1 Introduction
Recent advancements in Large Language Models (LLMs) (OpenAl 2023
Ouyang et al. 2022
Touvron et al. 2023) have demonstrated significant capabilities in understanding various concepts and solving downstream tasks (Brown et al. 2020 Raffel et al. 2020). Despite the vast amounts of data leveraged during their iniiall training subsequent fine-tuning phases, these models inherently remain static. Once built and updated specific point in time, their knowledge base ceases to evolve, precluding the incorporation of new, real-time information. This limitation confines them to rely solely on their pre-encoded parametric knowledge (Mallen et al., 2022) during inference. Lacking access up-to-date information, LLMs are prone to generating hallucinations (Ji et al. 2023) and may struggle to provide accurate and timely responses queries that demand the latest information (Vu et al.) 2023).
To overcome these challenges, integrating retrieval functionalities into generative models presents a promising solution (Lewis al., 2020 Luo al., 2023). This approach enriches standard parametric language models with non-parametric retrieval elements capable of accessing relevant information from external databases. These databases range from comprehensive document repositories, such as Wikipedia, to continuously updated sources like internet search engines, including Bing"] D among others.
' "Coresponddig author biig.cco
.duckduckgo.com
1

---

Published as a conference paper at COLM 2024
1. Model Only Search Needed
2. Model Decomposes the Query for Better Answrring
Hello, loday is good day!
Khat knn of university the school whem Rey Rarrary was educated
Hello! I'm glad to hour that there anything
instance
specific that's making dt good day?
here was Bey Battary educaled al?
Rungers Univvrsity
ben wr 2021 NBA playolls schedaied to begin?
hen do the 2024 NBA playoE's start?
Apail 20
hat university Rungers University instance of?
Land grant University
The 2024 NBA playoffs strt on Apri 20
Land-grant University
Model Disambiguates the Query for Full Clarity
Most lotel goals is premter lsegue seeson?
Most goals premier league seeson by
106
goels premier lesque by a pleyer?
34
The most goils scond in Fremier League season bern achioved by hester City the 2017-18 season. The most totall goals
Premier Legue sessca by player 31.
Usess
Model
Detiiewing
Retricvod Controcs
Model Answering
Figure 1: Our model are learned to search on demand, rewrite, decompose and disambiguate query when needed.
Although such enhancements could mitigate the tendency towards inaccuracies and main- tain the utility of LLMs in rapidly evolving domains, there still exist several issues in the previous framework. Firstly, the indiscriminate use of information retrieval systems to contextualize queries can be counterproductive. As demonstrated in previous research (Shi al. 2023a), irrelevant context not only diminishes generation quality but may also obstruct LLMs ability to answer queries they are otherwise capable of addressing. For straightfor- ward queries like daily greetings, LLMs should respond directly rather than incorporating unnecessary context, thereby avoiding low-quality responses that could deter users. That is, the model should learn to search on demand, see Figure (top-left). Secondly, for complex queries, simply searching with the original query often fails retrieve adequate information. It's crucial for LLMs to first break down such queries into simpler, answerable sub-queries, and then search for information relevant to these cob-commosnnts... By integrating responses to these sub-queries, LLMs can construct a comprehensive answer to the original complex query. See Figure (top-right). Lastly, for ambiguous queries with multiple possible answers, using the original query for information retrieval is insufficient. To provide complete and nuanced responses, LLMs must learn to clarify the query, ideally by identifying the user's intent, and then craft more targeted search query. After collecting relevant information, LLMs can then deliver a detailed and comprehensive response, see Figure I (bottom).
Based on the above requirements, propose Learning to Refine Query for Retrieval Aug- mented Generation (RQ-RAG) in this paper. We train a 7B Llama2 model an end-to-end manner to enable it to dynamically refine search queries through rewriting, decompos- ing, and clarifying ambiguities. Our work draws inspiration from Self-RAG (Asai et al., 2024) and SAIL (Luo et al. 2023), which pioneer the augmentation of instructional tuning datasets with search results and teach models to filter out noise from search results for context-grounded response generation. Building on this foundation, we introduce innova- tive modifications to the process of crafting these datasets, enhancing the model's ability to produce more effective information retrievals. Specifically, we leverage ChatGPT3 to craft tailored search queries across various scenarios (rewriting, decomposing, disambiguating) using distinct prompt templates, rather than relying on the original query. Furthermore, we observe instances where the dataset's 's initial output does not match the context returned by the information retrieval system. In these cases, employ ChatGPT to generate new, contextually aligned answers, thereby enhancing the relevance and accuracy of the informa- tion retrieval process. Echoing the methodologies of prior research (Lu et al. 2022 Keskar Asai et al. 2024), we employ control tokens-also known as special tokens- to
lt al. 2019
By default, ChatGPT refers to gpt-3.5-turbo-0125 and GPT-4 refers to gpt-4-0125-preview OpenAl
2023
in this paper.
2

---

Published as a conference paper at COLM 2024
Mulli-Turn Dialogue
the current tum neaded Yes rettivve context?
No, Go to the Num
Tusk Pool
Query Searching: Genurated Quey Retricve Relevant Contexts
Query Needed Decompased
Decompose Template: Generale D Deccesposed Queey
nnswor Templatre Generate A nower Based the Coatexts
Query Necdle Disambiguated
Disambiguate Template: Generate Disambiquated Query
Figure 2: Dataset construction pipeline.
direct our generation process. With the application of multiple control tokens, our model can navigate through various trajectories in response to user's 's query. At any given step, it has the flexibility rewrite, decompose, disambiguate the query, or terminate the search process and proceed to generate responses.
To identify the optimal trajectory as our final answer, we meticulously developed three distinct selection methods that do not rely on external LLMs for trajectory evaluation (Yao et al. 2024). These methods include selection based on perplexity (PPL), confidence, and an ensemble approach, as elaborated in Section 2.3 Furthermore, we assess the upper bound of our method's performance by determining if any generated trajectory contains the correct answer. This analysis reveals notably high potential for our system, underscoring its effectiveness we can accurately select the correct trajectories.
To sum up, our paper makes several key contributions
crafted dataset outperform previous state-of-the-arr method (Asai et al. 2024) on three single hop QA tasks we evaluate, and also demonstrate superior performance on three multi-hop QA tasks, attributed to our query refinement approach.
First, show that 7B Llama2 model trained
Second, we highlight the effectiveness of regenerating responses based on search results during data construction, moving beyond simply using the original dataset outputs. This method proves more effective than those employed in previous works (Luo et al. 2023: Asai et al. 2024), emphasizing the value of contextually grounded answer generation.
Third, we showcase the great potential of our framework by illustrating its consider- ably high upper bound, as well as its resilience to different data sources compared to previous methods.
2 RQ-RAG: Learning to Refine Query for Retrieval Augmented Generation
In this section, we introduce RQ-RAG including the following part; Dataset Construction (Section 2.1 Generator Training (Section 2.2), and Sampling Strategies (Section 2.3
2.1 Dataset Construction
To train the model to explicitly refine queries, the core of our pipeline involves collecting training data that mirrors the process at inference time. This includes generating refined queries for searches and crafting responses based on the information retrieved.
Given input-output pair (Xorigin Yorigin) from the original dataset, our main objective is to construct sequence of actions incorporating special tokens. These special tokens specify
3

---

Published as a conference paper at COLM 2024
the type of refinement SPyC and is followed by the refined query, conditioned on the specific special token, which is represented as Qi, type, where 'type' refers to the refinement action (either rewrite, decompose, or disambiguate) and refers to the turn of the iterations. Subsequently, we retrieve the top k documents, denoted as [Da D....., D At the final iteration step, we generate a new answer conditioned on the above contexts denoted neew-
In short, our dataset construction can be seen as the transformation denoted as Equation
(Xorigin, Yorigin) + (Xorigin, SPECIALaype Qi, type/ PP...... Did
Ynew)
(1)
repeat for times
Furthermore, gathering a corpus that encompasses a wide range of scenarios, as outlined in Section []] is crucial. Accordingly, our data collection focuses scenarios that include multi-turn dialogue, queries requiring decomposition, and queries needing disambiguation. For a comprehensive breakdown of the data statistics, refer Appendix A.1
Upon assembling these representative tasks, we establish a task pool and proceed with the transformation process detailed in Equation Ideally, executing this transformation involves human effort to refine queries, conduct searches for relevant information, and annotate the refined responses. Although this manual approach ensures quality, it is notably resource-intensive time-consuming, and challenging to replicate. To mitigate these limitations, we employ the advanced capabilities of ChatGPT for automating the annotation process. For an in-depth description of this process and the prompts used, see Appendix/ A.2
As depicted Figure our automated annotation workflow comprises several key stages, we detail it step by step as follows:
1. Begin by classifying tasks in the collected pool into the three previously mentioned categories. This step straightforward, as each dataset corresponds to a specific data type.
2. For each dataset type, we initially use ChatGPT with pre-defined prompt template generate refined query. We then employ this query to search for information from external data sources. We primarily use DuckDuckGo for most cases and treat the retrieval process as a black box.
3. Afterwards, prompt ChatGPT to generate renewed response based on the refined queries and its corresponding contexts. By repeating this process, we have amassed total of about 40k instances.
Generator Training
After we annotate the training corpus, can use it to train LLM in standard auto- regressive manner, which the objective shown as the Equation 2
C max E (xy)>D log PM (ylg, . 2,,,,, M
(2)
where L represents the likelihood we are trying to maximize. M denotes the model parame- 4qidij di, is the probability of the model M generating the response y given the input x and the refined query 4i with the retrieved document at step
ters. The expectation (x averages over our dataset D. PM
2.3 Sampling Strategies
In this section, we introduce the inference-time strategy. As is shown in Appendix Figure 8 at each time step, our model can choose to either rewrite, decompose or disambiguate given query, well opt to directly generate the response. Given this inner nature, design a tree decoding strategy with respect to different query refinement method. However, using different queries to search yields different retrieved contexts which leads to different

---

Published as a conference paper at COLM 2024
final answers. That is, how to sample the most appropriate path among these trajectories is a critical part in our system. Hence, we proposed three different kind of sampling strategies, illustrated as follows:
We use PM to denote a language model with parameters M, and [R,, , Rn] to denote n trajectories where each trajectories contains list of sequences which is denoted as [X, YJ. Here, X the input prompt, and is the concatenation of the intermediate steps of Zi....,Zi (each Zi the combination of queries and retrieved contexts) and final answer Yfial-
PPL Based Selection: select the trajectory Riinal that has the lowest perplexity (PPL) on the total generated output, that is, Rfinal = arg min PPL(R)), where PPL(R) =
R
exp (- Po (Y) here the total length of model output.
Confidence Based Selection: we select the trajectory Rfinal that has the highest confi- dence on the final answer ffnaal (distinguishing it from the PPL based selection, which
arg max Conf(R;), where evaluates the total generated output), that is, Rínal R,,,,,
Conf(R) - log PM (Y) Zi..., ......, Zú, Y<),, here, starts from which the start po- sition of the final answer, ffial-
Ensemble Based Selection: We ensemble the final results by selecting the final results that has the highest cumulative confidence score, that is Yfinal = arg max Liy,, Conf(Y))
y
Upper Bound: We also define the upper bound of our system, which means that any one trajectory leads to the correct answer, then we consider it correct.
Our sampling methods draw inspiration from Wang et al. (2022) ; Yao al. (2024), yet differ significantly in two key areas. Firstly, unlike these studies, we do not employ a larger model to assess the quality of generated trajectories; we instead utilize metrics inherent to our trained generator. Secondly, while the self-consistency approach proposed by (Wang et al., 2022) is limited to scenarios where the final answer belongs to a fixed set of options, our method is free from such constraints. See Figure for clarification.
Experiments
3.1 Evaluation Tasks
We assess our method's effectiveness across two primary categories of Question Answering (QA) tasks: single hop and multi hop QA tasks. See Appendix B.3 for details.
Single-Hop QA include Arc-Challenge (Clark et al. 2018 PopQA MMllen et al. 2022
and
OpenbookQA (Mihaylov et al. 2018).
Multi-Hop QA include HotpotQA (Yang et al. 2018
2WikiMulltHHoPQA
(Ho et al. 2020
and Musique (Trivedi et al. 2022
3.2 Baselines
We compare our method against diverse set of baselines, categorized into two main groups: No Retrieval Baselines, which answer questions without using contexts retrieved from external databases, and With Retrieval Baselines, which first retrieve relevant contexts from external sources before answering based on these contexts. In both settings, our comparisons include Llama2-7B and Llama-7B-ChattTTon Touvron et al. 2023) in a zero-shot configuration, as well as these models fine-tuned on Task-Specific Datasets (e.g., ARC. C and OBQA for single-hop QA tasks, noting that POPQA lacks a training set, thereby precluding training this dataset and HOTPOTQA, 2WIKI, and MUSIQUE for multi-hop QA tasks) and Our Crafted Dataset (without intermediate steps) as a more robust baseline. Moreover, we assess against SATL-7B(L.uo et al. 2023) and the previously established state-of-the-are Self-

---

Published as a conference paper at COLM 2024
RAG-7B et al. 2024 across three single-hop QA datasets. For the multi-hop QA tasks, we also extend our method to train a Llama3-8B (Meta 2024) model. The comparisons are also extended to include Chain-of-Thouge
(Wei et al. 2022), Chain-of-Nott Yu et al., 2023), and Self-Ask(Press et al. 2022), ) all of which are well-known methods for tasks requiring multi-step reasoning. We utilize both ChatGPT and GPT-4 the underlying language models. We made minor modifications to the prompts to suit our tasks. For details of the prompts used, see the Appendix C
Results and Analysis
Unless otherwise stated, our analysis is conducted using our methodology applied to Llama2-7B.
4.1 RQ-RAG outperforms Self-RAG and SAIL on single hop QA tasks
In Table we demonstrate that RQ-RAG (ours) significantly outperforms various baseline models both retrieval and non-retrieval settings. Specifically, under retrieval settings, RQ- RAG surpasses LLama2-7B (Zero Shot) by an average of 33.5%, highlighting the challenges LLMs face processing retrieved contexts without search-augmented instruction tuning. Furthermore, we compare RQ-RAG against two robust baselines: 1) models supervised on specific tasks (ARC.C, OBQA) and 2) models supervised on our curated dataset but without the search-augmennter intermediate step. Our findings indicate significant performance gains from supervision compared to zero-shot approaches, highlighting their competitive edge. Crucially, RQ-RAG further surpasses these baselines, demonstrating the added value of integrating a search-augmented step during training.
Additionally, juxtapose RQ-RAG with previously established supervised, search- augmented approaches, namely Self-RAG and SAIL. Notably, our method outshines SAIL- 7B by 20.3% average data, our method also surpasses the former state-of-the (Self-RAG, which utilizes 150k supervised training data) by 1.9% on average across three QA tasks.
three QA tasks. Moreover, even with only about 40k training
Overall, RQ-RAG demonstrates robust performance across all evaluated tasks, firmly estab- lishing its superiority over the aforementioned baselines.
Table Performance comparison on single-hop QA tasks. The score with an arrow in parentheses indicates the comparison with the highest score within the same group.
Model
ARC.C POPQA
OBQA
AVG.
Baseline without Retrieval
SAIL-7B
47.7
22.8
49.2
39.9
LLama2-7B
Zero Shot
29,8
12.9
34.6
25.8
+ Zero Shot (Chat Version)
59.9
14.1
57.6
43,9
+ SFT on Single Hop QA
61.3
49.6
+ SFT on No Augmented Set
62.0
20.5
64.0
48.8
Baseline with Retrieval
SAIL-7B
48.4
44.0
52.0
48.1
Self-RAG-7B
67.4
55.3
76.4
66.4
LLama2-7B
+ Zero Shot
28.7
39.8
36.2
34.9
+ Zero Shot (Chat Version)
53.8
26.4
40.8
40.3
+ SFT on Single Hop QA
52.1
48.5
+ SFT on No Augmented Set
51.5
45.0
50.2
48.9
+ RQ-RAG (Ours)
68.3
57.1
79.4
68.3
(0.9t)
(1.8t)
(3.01)
(1.91)

---

Published as a conference paper at COLM 2024
Table 2: Perfforaace comparison on multi-hop QA tasks. score with an arrow in paren- theses indicates the comparison with the highest score within the same group.
Model
HOTPOTQA 2WIKI
MUSIQUE
AVG.
Proprietary LLM (with Retrieval)
GPT-3.--URRBO
+ Chain-of-Thought
58.6
43.9
32.3
44.9
+ Chain-of-Note
52.5
34.1
24.6
37.1
+ Self-Ask
62.5
57.0
37.2
52.2
GPT-4
+ Chain-of- Chain-of-Thought
71.4
70.1
50.3
63,9
+ Chain-of-Note
72.4
58.3
44.1
58.3
+ Self-Ask
63.3
66.6
44.0
58.0
Baseline without Retrieval
LLama2-7B
+ Zero Shot
6.6
16.0
3.0
8.5
Zero shot (Chat Version)
3.6
7.9
1.8
4.4
+ SFT on Multi Hop QA
34.7
34.2
6.8
25.2
+ SFT on No Augmented Set
35.6
30.8
6.7
24.4
Baseline with Retrieval
LLama2-7B
Zero Shot
16.7
18.7
7.4
14.3
+ Zero shot (Chat Version)
2.8
3.5
1.8
2.7
+ SFT on Multi Hop QA
37.5
32.3
7.9
25.9
+ SFT on No Augmented Set
43.5
28.8
9.1
27.1
+ Self-Ask
30.8
20.9
13.5
21.7
+ RQ-RAG (Ours)
62.6
44.8
41.7
49.7
(19.11)
(16.0t)
(32.61)
(22.61)
LLama3-8B
+ Zero Shot (Instruct Version)
39.7
23.8
15.2
26.2
+ SFT on No Augmented Set
71.1
61.0
40.9
57.7
+ Self-Ask
44.5
44.0
25.9
38.1
+ RQ-RAG (Ours)
74.2
59.3
48.1
60.5
(3.1$)
(1.7.)
(7.21)
(2.81)
4.2 RQ-RAG shows superior performance on multi-hop QA datasets
In Table present RQ-RAG's performance across three multi-hop QA datasets, reflecting trends similar to those observed in single-hop QA scenarios. When RQ-RAG is trained on either specific datasets or our curated dataset(without search-augmented step), its perfor- mance significantly surpasses that of its zero-shot counterpart. However, these baselines does not endow the model with the capability to decompose queries, a crucial skill in multi- hop scenarios where direct use of the original query for information retrieval often falls short. In contrast, our pipeline enables the model autonomously refine queries, yielding an average enhancement of 22.6% across the three multi-hop QÅ datasets. Furthermore, when compared to more robust baselines that employ ChatGPT as the backbone model for answering questions, RQ-RAG significantly outperforms both the Chain-of- Thought and Chain-of-Note methods. This achievement is particularly notable given that our back- bone model is considerably smaller than ChatGPT, highlighting RQ-RAG's exceptional effectiveness.
4.3 RQ-RAG shows high upper bound of the system
In Tables and 2 we present ensemble-based selection results. For a comprehensive study of sampling strategies, show the comparison in Figure3) Basically, find that for single-
7

---

Published as a conference paper at COLM 2024
hop QA tasks, the confidence-based strategy generally excels, while for multi-hop QA tasks, the ensemble-baset strategy shows superior performance. Overall, the ppl-based strategy demonstrates moderate performance in comparison. Beyond these strategies, also assess the upper bound of our system by considering the success rate across all trajectories; success is defined any trajectory that leads to the correct answer. This comprehensive evaluation reveals high potential upper bound for our system. Specifically, it reaches 76.8% in ARC.C 65.6% in POPQA, 84.0% in OBQA, 80.5% in HOTPOTQA, 60.6% in 2WIKI, and 54.5% in MUSIQUE, respectively. These results underscore the system's 's ability to refine queries and retrieve varying contexts multiple times, thereby increasing the likelihood of reaching the correct answer. An avenue for future enhancement involves devising more effective methods to select among generated trajectories, such as leveraging LLMs for scoring each one. Furthermore, it's important to note that the current upper bound does not represent the system's absolute limit; additional improvements could be as well achieved through context reranking and explicit noise reduction in retrieved contexts.
Upper Bound
84.0
Ensemble
80
20.5
PPL Confidence
75.8
70
Scores
61.77
60.6
57.3
31.2
26.1
24.2
54.5
30
41:1411
РОРΩН
OBIA
HolpotOA
2WWki
Musique
Avg.
Figure 3: Performance of different sampling strategies six tasks.
4.4 Regenerating answers grounded on contexts matters
As mentioned in Section [ key innovation of our method compared previous approaches is the use of ChatGPT to regenerate answers based on provided contexts, rather than relying on original answers from the dataset. In this section, we explore the impact of varying the proportion of the original answer retained from the dataset. To ensure fair comparison, we maintain consistent training srper-parameters across experiments. We curated our search- augmented dataset by retaining 0% (our primary approach), 25%, 50%, 75% and 100% of the original answer, then assessed the effects on performance across three multi-hop QA findings on HotpotQA, with the all-regenerated answer
datasets. Figure 4a| illustrates setting (0% retention) showing the best performance. Additionally, there is a noticeable decline in effectiveness the proportion of the original answer retained increases, indicating that our strategy of regenerating answers based on retrieved contexts is beneficial. We show the results of the other two datasets in the Appendix(D
A closer look into the effect of different action on query refinement.
In this section, we aim to illustrate the effect of each different action on query refinement. Our methodology employs sampling strategies to select the best trajectories; however, we want to evaluate the effectiveness of deliberately using specific action for all instances on three tasks. As shown in Figure 4b compare the use of single action during inference with our designed sampling strategies (as the one used in Table The observations are: 1) single action outperforms the others across all three tasks; 2) while some actions yield
8

---

Published as a conference paper at COLM 2024
Upper
Mehob Neetlc
Conddance
With samplig strnogin
255
57%
3006
(a) The impact of retention ratio on HotpotQA.
(b) The impact of each single action.
Figure 4: Left: Increasing the ratio the original answers in the dataset results in a decreased Fl score. Right: No single action outperforms the others across all three tasks.
better results for specific task compared to the sampling strategies, they do not perform better overall, indicating the benefit of our designed sampling strategies.
Furthermore, we investigate the proportion of each action selected by our strategies. Figure demonstrates the result for HOTPOTQA, showing that although the disambiguation process accounts only for 10% of our training data, the proportion of different actions selected by the model is almost balanced. For the results of the other two tasks, see Figure 11 and Figure 12 The results in this section based on Llama3-8B.
4.6 Our system is resilient to different data resources
In this section, we investigate the impact of utilizing various data sources. Our dataset primarily leverages DuckDuckGo for data retrieval during dataset curation, yet our find- ings suggest that the choice of data sources during inference has a minimal effect on our system's 's performance. This experiment evaluates system across three single-hop QA tasks, employing DuckDuckGo, Wikipedia, and Bing Search as the data sources. Table demonstrates that our system exhibits greater resilience to changes in data sources compared to Self-RAG, which primarily uses Wikipedia for data retrieval during dataset curation. Notably, using Bing Search results in performance drop of 3-5% for Self-RAG across the tasks. In contrast, our method, curated using DuckDuckGo, shows negligible impact on performance (a variance of 0.7 compared their 1.8) when different data sources are used during inference.
3.0%
1.8%
39
34.0%
Rew Le
43.0%
Rewi'te
Decompoee
Nowrile
53
Decompase
Dbambb guats
37.0%
Osambiguatz
33.2%
32.0%
Figure 5: The ratio of all the action selected in different sampling strategies HOTPOTQA.

---

Published as a conference paper at COLM 2024
Method
ARC.C
POPQA
OBQA
AVG.
VAR.
Self-RAG
+ DuckDuckGo
67.4
55.3
76.4
WIKI
67.3
54.9
78.0
65.5
1.8
+ BingSearch
64.6
49.0
76.8
Ours
+ DuckDuckGo
68.3
57.1
79.8
+ WIKI
67.8
52.6
80.6
67.6
0.7
BingSearch 67.9
55.6
78.8
Table 3: Performance comparison of different retrieval sources.
Related Works
Retrieval-augmmenté generation (RAG) represents effective strategy for enhancing model's capability to leverage non-parametric knowledge by retrieving external data re- sources, rather than relying solely on its intrinsic parametric knowledge during generation. This paradigm has garnered widespread attention across both industry and academia, proving its efficacy in variety of scenarios such as question answering (Lewis al. 2020) code generation (Zhou et al.) 2022), alignment with human values (Xu et al. 2023b) and reducing hallucinations (Shuster et al. 2021). Recent advancements RAG systems can generally be categorized into two areas: improvements in either the retrieval component or the generation component of the system.
Retriever in Retrieval Augmented Generation
Previous studies have underscored the critical role of the retrieval component RAG systems, illustrating how LLMs can be susceptible to irrelevant contexts (Shi et al. 2023a).
Systems that retrieve more semantically relevant contexts (Karpukhin et al. 2020) Signifi- cantly outperform those based BM25 (Robertson et al. 2009), demonstrating substantial improvement. More recently, certain works have highlighted that LLMs themselves can act a supervisory signal for training the retrieval component (Shi et al., 2023b), or even as the retrieval component (Sun et al. 2022), thanks to their extensive knowledge and robust capabilities. Another research direction involves denoising irrelevant context by employing additional models to explicitly filter out (Yoran et al. 2023) or compress (Xu et al. 2023c such contexts. In our work, we approach the retrieval component as black box, directly using the contexts returned from search engine APIs or pre-provided data corpus. This method is compatible with denoising techniques, which could further enhance our system's performance; exploring these techniques will be an avenue for future research.
Generator Retrieval Augmented Generation
In contrast to efforts aimed at enhancing the retrieval component of RAG systems, another research focuses on optimizing the generator part. SAIL (Luo et al. 2023 trains the LLM to differentiate irrelevant contexts during the generation process. ReAG Asai et al., 2024 trains the LLM to self-reflective on the retrieved contexts. Rewrite-Retriel Read (Ma et al. 2023) trains small model to rewrite queries for black-box reader while we train the LLM to refine queries by itself in this paper.
Conclusion
In this paper, we introduce RQ-RAG, framework that enhances LLMs by training them a meticulously curated dataset. This training enables the LLMs to refine queries through process of rewriting, decomposing, and disambiguating, Our experimental findings demon- strate that RQ-RAG not only surpasses previously established SOTA method across three single-hop QA tasks but also exhibits superior performance in complex multi-hop QA see- narios, even when compared to the exceptional ChatGPT, underscoring great effectiveness of our approach.
10

---

Published as a conference paper at COLM 2024
Acknowledgments
This work is funded in part by the Theme-based Research Scheme grant([No. (45-00222155) and the InnoHK funding, Hong Kong SAR.
References
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learn- ing to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, 2024.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint 0ix::1803.004550 2018.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint axXi::2011.01060, 2020.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answer- ing. Xrv preprint ar wix::2004.0490000 2020.
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. Ctrl: A conditional transformer language model for controllable generation. Xiv preprint 1xXiv:1909.05855 2019.
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richárd Nagyfi, et al. Openas- sistant wwveersatioon-ddmccrttiiiiii large language model alignment. Advances in Neural Information Processing Systems, 36, 2024.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval- augmented generation for knowledge-intensivee nip tasks. Advances in Neural Information Processing Systems, 33:9459 9474, 2020.
Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. Quark: Controllable text generation with reinforced unlearning. Advances in neural information processing systems, 35:27591-27609, 2022.
Hongyin Luo, Tianhua Zhang, Yung-Sung Chuang, Yuan Gong, Yoon Kim, Xixin Wu, Helen M Meng, and James Glass. Search augmented instruction learning. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. Query rewriting for retrieval-augmentted large language models. arxiv preprint arXiv:2305.14283, 2023.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-marametric memories. ar Xio preprint trXiv:2212.105:11 2022.
Meta. Introducing meta llama 3: The most capable openly available Ilm to date. https
//ai, meta tata www.blog/mee.ttaaeeoo waw.:////etalllaaaaa April 2024.
11

---

Published as a conference paper at COLM 2024
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can suit of armor conduct electricity? a new dataset for open book question answering. arXio preprint (rXiv:1809.02789, 2018.
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4. ar preprint ar driiv:2066.2707, 2023.
OpenAL. Gpt-4 technical report. ar Xiv preprint axXi::3003.08774, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 15527730-227444 2022.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. arXiv preprint ar axXi::2210.03330, 2022.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of machine learning research, 216140))166,, 2020.
Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends@ in Information Retrieval, 343:3333-38, 2009.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pp. 31210-31227. PMLR, 2023a.
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Replug: Retrieval-augmented black-box language models. arviv preprint ar 2xXiv:2301.12652, 2023b.
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmen- tation reduces hallucination conversation. ar preprint arXiv:2104.07567 2021.
Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. Asqa: Factoid questions meet long-form answers. arXiv preprint arXiv:2204.06099, 2022.
Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. Recitation-augmented language models. Xiv preprint ar xii::22100012266, 2022.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction-folloiing llama model, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arv preprint arXiv:2307.09280 2023.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539-554, 2022.
Harsh Trivedi, Niranjan Balasubramanían Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thoun thought reasoning for knowledge-intensivee multi-step questions. The 61st Anmual Meeting of the Association for Computational Linguistics, 2023.
Tu Vu, Mohit lyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun- Hsuan Sung, Denny Zhou, Quoc Le, et al. Freshlims: Refreshing large language models with search engine augmentation. at Xiv preprint arXit:2310.032214 2023.
12

---

Published as a conference paper at COLM 2024
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. ar Xiv preprint ar Xiv:22003.11177. 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems,, 35:24821-24837, 2022.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arxiv preprint ariio22304.12244 2023a.
Chunpu Xu, Steffi Chern, Ethan Chern, Ge Zhang, Zekun Wang, Ruibo Liu, Jing Li, Jie Fu, and Pengfei Liu. Align the fly: Adapting chatbot behavior to established norms. ar Xiv preprint ar Xit:2312.15901 2023b.
Fangyuan Xu, Weijia Shi, and Eunsol Choi. Recomp: Improving retrieval-augmented Ims with compression and selective augmentation. arXiv preprint arXiv:2310.04400, 2023c.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdi- nov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint ar xxXo:1809.09600, 2018.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024.
Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. Making retrieval-auggmenee language models robust to irrelevant context. arXio preprint XrXi:23300005588, 2023.
Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, and Dong Yu. Chain-of-note: Enhancing robustness in retrieval-augmented language models. arXiv preprint ar xrii:2231..09210 2023.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan lyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024.
Shuyan Zhou, Uri Alon, Frank F Xu, Zhiruo Wang, Zhengbao Jiang, and Graham Neubig. Docprompting: Generating code by retrieving the docs. arviv preprint (tt::22070008877 2022.
13

---

Published as a conference paper at COLM 2024
A Data Collection
A.1 Data Statistics
Challenge
Arc. Emny
OpenbookQA
Rec Cortods
HotporQA
SingleHHpp Hop
43%
Multi-Hp
ASQA
Ambiguous
10%
Inatr uction Following
Alpaca
WizardLM
MusiQue
Open. Orca
0h
Figure 7: Token Distribution.
Figure 6: Data categories.
Figure 6 shows the categories we include in our data curation process, and the specife dataset used. Primarily, our data contains single-hop QA tasks including Arc-Easy Arc- Challenge (Clark et al. 2018), OpenbookQA (Mihaylov et al., 2018), multi-hop QA tasks including HotpotQA YYnng al. 2018), Musique (Trivedi et al. 2022), ambiguous tasks including ASQA (Stelmakh et al. 2022), and in order to equi our model with general capabilities, we also add instruction tollowing tasks including LIMA (Zhou et al. 2024). WizardLM (Xu et al. 2023a), Open-Oree (Mukherjee et al. 2023), OpenAssistant (Köpi et al. 2024 GPT4-Alpaca [laori et 2023). In total, we collect 42810 instances.
Moreover, as illustrated in Figure 7 the distribution of tokens by probability density is sig- nificantly altered by search augmentation. Our augmented dataset, exhibits a considerable increase in token count, with the majority of instances spanning from 150 to 2000 tokens. In contrast, the original dataset presents a notably shorter length, with most entries falling under 200 tokens.
A.2 Data Annotation
In our study, we employed ChatGPT, specifically the gpt-3.5-turbo-0125 version, for data annotation, setting the temperature parameter to 0 to enhance reproducibility. Throughout the annotation process, encountered certain limitations with ChatGPT, including its occasional refusal to answer and failure to adhere to the specified output format. Instances exhibiting these issues were subsequently excluded.
Table 4 Table and daspeyy the prompt templates we used to interact with ChatGPT across different phases of annotation process, detailed in Section 2 In these templates, the blue text serves as a placeholder for specific input.
B Experimental Details
B.1 Training Hyperparameters
In this paper, all of the models trained 8 NVIDIA H800 with 80GB memory. And all models are trained 1 epoch using learning rate of 2e-5 with 3% warmup steps. Given the extended context length of our dataset, set the maximum input length at 4096.
14

---

Published as a conference paper at COLM 2024
In a multi-turn dialogue scenario, your task is to determine whether it necessary to search engine to answer user's 's query and provide list of possible search queries.
Consider the following two scenarios:
Non-Informationl Replies: Sometimes, users may respond with statements or expressions that do not require information retrieval, such as "thank you" "okay." In these cases, assess whether a search engine query is necessary.
Ambiguous or Unclear Queries: At times, user's query might be unclear or lack specific details. Your role is to recognize the user's 's intent and rewrite the query to make it clearer and more precise, facilitating an effective search engine query. Previously Answered Queries: Check if the current query or similar one has been previously asked and answered in the conversation history. If relevant information or evidences have already been provided, acknowledge this and avoid repeating the search.
Based the above scenarios, please reply with the following format strictly: For the case that do not need to query the search engine, output as follows: {In context examples)
As outlined, is necessary to output Retrieval Necessity first, and the output should be one of the "yes" and "no" and the query for the search engine should be split by line break. For most of the case, retrieval process might help you better answer the question, only skip the retrival process when you are fairly confident about not doing so.
Now, please answer:
Conversation History:
[Conversation History
Current User's Query:
[Current User's ]uery}
Response For Retrieval Necessity:
Table 4: Multi-Turn dialogue prompt template during data construction.
Your task to effectively decompose complex, multihop questions into simpler, manageable sub-questions or tasks. This process involves breaking down question that requires information from multiple sources or steps into smaller, more direct questions that be answered individually.
Here's how you should approach this:
Analyze the Question: Carefully read the multihop question to understand its different components. Identify what specific pieces of information are needed to answer the main question.
Here are an example of how you should solve the task: (In context examples)
As outlined, please format your answer as multiple lines of text. Ensure that each subsequent question follows from the previous and is self-contained and be capable of being answered on its own. Ensure there is exactly one line break between each line.
Now please answer:
Provided Contexts:
[Provided Contexts}
Multihop Question:
[Multihop Question]
Decomposed queries:
Table 5: Query decomposition prompt template during data construction.
15

---

Published as a conference paper at COLM 2024
Your task is to identify and resolve ambiguity in complex questions, ensuring they are clear and unambiguous. This requires pinpointing elements of the question that could be interpreted single, clear interpretation.
more than one way and refining the question to ensure
Approach this task as follows:
Analyze the Question: Read the question thoroughly to identify ambiguous parts.
Consider the different ways the question could interpreted based on its current wording.
Clarify the Query: Reformulate the question to eliminate ambiguity. This may involve specifying details, narrowing down broad terms, or providing additional context to guide the interpretation. Here's 's an example of how to complete the task:
For example: {In context examples)
As outlined, please format your answer as multiple lines of text. Ensure there is exactly one line break between each line. Now, please answer:
Original Question:
[Original Question}
Disambiguated Query:
Table 6: Query disambiguation prompt template during data construction.
B.2 Sampling Strategies
Sigure&ddppcee our tree decoding strategy through a structured flow. This strategy unfolds by controlling expansion paths via special tokens, generating and retrieving queries itera- tively: process of generate retrieve generate + retrieve ... answer. At each iteration, the model decodes diverse search queries tailored to specific neets-whether to rewrite, decompose, or disambiguate. These queries, in turn, fetch distinct contexts, leading to varied expansion paths. Thus, within predetermined exploration width and depth, our approach facilitates the generation of multiple trajectories. Given this diversity, becomes imperative to adopt method for accurately sampling the optimal trajectory, as discussed in Section 2 Experimentally, we set the exploration depth to for three single-hop QA tasks and adjusted it to 2 for HotpotQA and 4 for 2WikiMultihopQA and MuSiQue, respectively.
B.3 Evaluation Datasets and Metric
Single-Hop QA:
For the Arc-Challenge (Mallen et al., 2022), comprising 1172 four-choice QA instances, we measure model performance using accuracy. In the case of PopQA (Mihaylov et al. 2018), we focus on a longtail subset of 1 1399 instances and adopt match score metric assess the model's output includes the ground truths. For OpenbookQA (Mihaylov et al. 2018),
comprising 500 four-choice QA instances, we measure model performance using accuracy.
Multi-Hop QA:
We evaluate multi-hop QA on a sample of 500 instances following Trivedi Our experiments utilize datasets from CooppotQA-disractton (Yang et al. 2018), 2Wiki- MultihopQA (Ho et al., 2020), and MuSiQue-Ans (Trivedi et al. 2022) within a reading comprehension setting, where the candidate documents are from their original datasets. For HotpotQA and 2WikiMullthopQAA each question is linked to 10 passages, of which only a few (2 for HotpotQA and 2 or 4 for 2WikiMultihopQAA are relevant. MuSiQue-Ans presents a greater challenge, offering 20 candidate documents and featuring questions that require 2, or 4 hops to answer. We employ the F1 score as our primary performance metric.
(023).
16

---

Published as a conference paper at COLM 2024
Instraction
Queries Generated by Model
Retrieved Contects
PPL: 32
PPL: Confilence: 0.8MAX)
Confidrnce: 0.7
Final A Answer
PPL: (MIN)
PPI. Based Selection
Ensemble Based Selection
Figure Given different paths, we develop three different strategies, ppl based, confidence based and ensemble based selection.
B.4 Retriever Setup
During data curation, we primarily employ DuckDuckGo gather relevant contexts for most scenarios, selecting the top-3 documents. Each result includes title, a preview snippet, and the webpage's URL. To simplify our process, we utilize only the title and preview text without further scraping the url. For multi-hop QA tasks, we leverage BM25 to extract contexts from candidate documents provided by specific datasets, discarding instances where the contexts do not align with the support documents.
During inference for single-hop QA tasks, our standard approach involves retrieving three contexts from DuckDucGGo. For multi-hop QA scenarios, utilize the text-embedding- 3-large model (OpenAI 2023) to identify three most relevant contexts from the candidate documents at each step.
C Prompt for COT, CON and Self-Ask
See Figure
Figure
8
Figure for details.
D Additional Results
We show the impact of retention ratio on another two multi-hop QA tasks in this section. As discussed in Section 4.4 similar phenomena observed in Figures[9and 10 indicating that regenerating answers grounded on the provided contexts crucial which will in turn affect the performance of downstream tasks Additionally, the ratio of all the actions selected in different sampling strategies on two other multi-hop QA tasks is shown in Figure| II and Figure 12 Overall, a similar trend action is not selected by PPL strategies in any of the three tasks. Moreover, the ratio of direct answer actions is minimal across all strategies and tasks. Generally, the three different actions are balanced across all tasks and strategies.
be observed where the direct answer
17

---

Published as a conference paper at COLM 2024
Task Description:
You are given three questions: Question QI, Question Q2, and Question Q3, along with the passages P1 for Q1, and the passages P2 for Q2. The requirements are as follows:
1. Please answer Q1 and Q2 before answering Q3.
For Q3, give the final answer in the format 'The answer .. Ensure the final
answer is short and concise. Do not repeat the question in the final answer.
Q1:
(Q1)
Pl:
[P1}
Q2:
(Q2)
P2:
(P2)
Q3:
[Q3)
Let's think step by step:
Table 7: COT prompt template that we use for our baseline.
Task Description:
You are given three questions: Question Q1, Question Q2, and Question Q3, along with the passages P1 for Q1, and the passages P2 for Q2. The requirements are as follows:
1. For Q1 and Q2, read the given question and the passages to gather relevant information.
2. For Q1 and Q2, write reading notes summarizing the key points from these passages.
3. For Ql and Q2, discuss the relevance of the given question and context. For Q1 and Q2, if some passages are relevant to the given question, provide a brief answer based
the context.
For Q1 and Q2, if no passage is relevant, directly provide an answer without considering the context.
6. For Q3, review the reading notes for Q1 and Q2, and give the final answer in the format 'The answer is Ensure the final answer is short and concise. Do not repeat the question in the final answer.
Q1:
(Q1)
P1:
[PI}
Q2:
(Q2)
P2:
(P2)
Q3:
(Q3)
Table 8: CON prompt template that we use for our baseline.
18

---

Published as a conference paper at COLM 2024
Given the following question, answer it by providing follow up questions and intermediate answers. For each follow up question, you are given a context which is the top returned Wikipedia snippets for the question. If no follow up questions are necessary, answer the question directly.
#
[Knowlegge]
|Knowledge]
Question: What is the name of this American musician, singer, actor, comedian, and songwriter, who worked with Modern Records and born in December 5, 1932?
Are follow up questions needed here: Yes.
Follow up: Who worked with Modern Records? Intermediate answer: Artists worked with Modern Records include Etta James, Little Richard, Houston, Ike and Tina Turner and John Lee Hooker.
Follow up: Is Etta James an American musician, singer, actor, comedian, and song- writer, and was born in December 5, 1932?
Intermediate answer: Etta James was born in January 25, 1938, not December 5, 1932, so the answer is no.
Follow up: Is Little Richard an American musician, singer, actor, comedian, and songwriter, and was born in December 1932? Intermediate answer: Yes, Little Richard, born in December 5, 1932, is an American musician, singer, actor, comedian and songwriter.
So the final answer is: Little Richard
{In context examples)
Questions:
Table 9: Self-Ask prompt template that we use for our baseline.
70
Upper Bound
Ensemble
PPL
Confidence
60
50
Scores
40
30
50%
100%
Retention Ratio
Figure 9: The impact of retention ratio on 2WikiMultihopQA.
19

---

Published as a conference paper at COLM 2024
55
F1 Scores
33
Upper Bound
Confidence
20
100%
Figure 10: The impact of retention ratio on MuSiQue.
2.0%
2.0%
10.0%
5%
5%
Rewrite
Decompoke
Sear te
Disamoiguzte
Decompase
Recmmiigaae
Disamaiquate
0%
(2) Conf dence
(b) Enoemble
(a ML
Figure 11: The ratio of all the action selected in different sampling strategies on 2WIKT.
3.0%
23.0%
0%
Decampsse
Decompose
Decomacee
Reertte
Chambiguate
sambiuuzte
31.0%
Figure 12: The ratio of all the action selected different sampling strategies on MUSIQUE.
20


Reviews:
[{"review_id": "zE2Xob6LPR", "reviewer": "Reviewer_PYYL", "summary": "The paper has good originality and high significance in the topic of retrieval-augmented generation. The paper lacks some analyses, causing a possibility that the paper's claim is not verified well, lowering the quality. The clarity is fine. See \"Reasons To Accept\" and \"Reasons To Reject\" below for details.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your insightful comments!\n\n**1. More Details on the Effect of Different Actions**\n\nRegarding the concern about the effect of different actions, we have complemented the new experiment as follows, and will provide case study to support the error analysis in our next version. Note that the newly provided results are based on our newly trained llama3 8b model.\n\n- **The ratio of different actions under different sampling strategies of 3 tasks.**\n\n  \n|       | Decompose | Disambiguate | Rewrite | Direct |\n|----------------|-----------|--------------|---------|--------|\n| **Confidence** | 0.46      | 0.35         | 0.19    | 0.03   |\n| **Ensemble**   | 0.34      | 0.32         | 0.34    | 0.01   |\n| **PPL**        | 0.53      | 0.37         | 0.10    | 0.00   |\n\n- **F1 score  for deliberately using the specific action for all instances of 3 tasks.**\n\n|    | HotpotQA | 2Wiki | Musique | Avg     |\n|--------------------------|----------|-------|---------|---------|\n| **Rewrite**              | 74.8 | 59.2| 46.2 | 60.1 |\n| **Decompose**            | 73.2 | 58.7 | 48.4 | 60.1 |\n| **Disambiguate**         | 72.6| 60.3 | 45.0 | 59.3|\n| **With Sampling Strategies** | 74.2     | 59.3  | 48.1    | 60.5    |\n\nThe findings are as follows:\n\n- Although the disambiguation process accounts for only 10% of our training data, the proportion of different action selections is almost balanced.\n\n- Simply choosing the same action might not yield the best performance compared to the score with sampling strategies.\n\n**2. Inefficiency of the Disambiguation Process**\n\nThe inefficiency arises from generating multiple queries and retrieving each piece of evidence, which increases inference latency. This is not a drawback of our method alone; multiple strong baselines, such as Self-Ask and ReAct, also generate follow-up questions multiple times to retrieve the necessary information. Regarding the numerous disambiguated queries generated, our trained model is designed to answer all of them, covering all generated queries.\n\n **3. Clarification of Iterative Query Refinement Annotation Process**\n\nWe use the generated queries to retrieve documents from an external database, then concatenate them to regenerate Y_new, which happens only once. The intuition behind this process is that Y_new should be conditioned on all generated queries to cover all the queries. During inference, the model can decide to generate a refined query or choose to terminate and generate the final answer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) The empirical observations support the superiority of the proposed method well. They show that the proposed method outperforms state-of-the-art models in single-hop and multi-hop QA tasks (Self-RAG-7B and GPT-3.5-turbo, respectively).\n2) The paper conducts extensive comparison experiments using four datasets for each of single-hop QA and multi-hop QA.\n3) Although I see a few typos, the paper is well organized, making it easy to follow the content.", "reasons_to_reject": "1) The contribution by each of learned capabilities (e.g., query decomposition) is not adequately analyzed. In particular, I suspect the contribution by query disambiguation might be minimal because ambiguous QA data (ASQA) occupies only 10% of the constructed dataset, according to Figure 5. The contribution could be more convincingly justified if it showed statistics of how many occurrences of query refinement types exist in the dataset and how many times each of those refinement actions was chosen by LLM during inference. See also the question below.\n2) The clarification process by the proposed query disambiguation could be inefficient because a large number of disambiguated queries is potentially possible, especially if a disambiguated aspect is at the instance level.  For example, if a question is \"What is the length of a U.S. president's term?\", the answer depends on a particular U.S. president.  Then, the proposed method could end up generating numerous queries \"What is the length of the term for the U.S. president Joe Biden?\", \"What is the length of the term for the U.S. president Donald Trump?\", etc.  If numerous disambiguated queries are generated, the proposed method will lack runtime efficiency, and LLM will be likely to generate erroneous final answers.\n3) No error analysis is provided. The paper would be more insightful for future work if it showed an analysis of what errors are dominantly produced from each of the query refinement actions.", "questions_to_authors": "- Section 2.1: I assume Y_new can significantly (semantically) differ from Y_origin, based on Section 4.4.  If so, how does one know when to stop the iterative query refinement annotation process and make Y_new?  This point would be better clarified if the paper gave an example and presented details of the answer template shown in Figure 2.\n- Figure 3: Section 4.3 says the confidence-based sampling strategy generally excels for single-hop QA tasks, but actually Figure 3 shows that the ensemble-based strategy performs best across single-hop and multi-hop tasks, except for OBQA. What is the reason why this anomaly (i.e., the confidence-based strategy outperforming the ensemble-based one) happens with OBQA?\n- Appendix A.1: More detailed dataset statistics should be provided.  It would be interesting to see the number of occurrences of each query refinement and the average number of refined queries in each QA pair.\n- Section 2.3: However, using different query to -> However, using different *queries* to\n- Section 2.3: ... which leads to different final answer. -> ... which leads to different final *answers*.\n- Section 2.3: We ensemble the final results by select the final results ... -> We ensemble the final results by selecting the final results ...", "extra_scores": {}}, {"review_id": "AKXUGo9yV0", "reviewer": "Reviewer_xvu9", "summary": "This paper introduce a new training and inference method for retrieval-augmented generation, which enables an LM learn to retrieve when necessary and re-write compositional and ambiguous queries. The proposed method is based on recent retrieval-augmented instruction-tuning method, particularly Self-RAG (e.g., using special tokens to control model-behaviors). While in Self-RAG, the authors only train models for single-turn settings and reuse the input-output data from existing datasets, this paper re-generate output data based on retrieved documents, which authors find crucial to achieve the strong performance. Experimental results on single-hop and multi-hop queries indicate the effectiveness of proposed method.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your insightful comments!\n\n**1. Claims of Novelty and Contributions**\n\nThe main concern is how our method compares with Self-RAG and SAIL. We would like to clarify and highlight the novelty of our approach as follows:\n\nSolely relying on RAG with its non-parametric parts shows some limitations, including but not limited to:\n1. It requires strong instruction-following ability to reply considering the provided context, due to the different format from their original sft dataset.\n2. It lacks interpretability when the retrieved documents contradict its parametric knowledge.\n\nTo address the above issues, our work focuses on effectively constructing retrieval-augmented instruction following datasets to train the model for internal learning and designing an effective mechanism for inference. This shares a common goal with previous work like self-RAG and SAIL. This methodology has also been explored by other works [1] after we submitted our research, highlighting the potential and value of this direction. While we draw inspiration from data construction processes such as self-RAG and SAIL, our key contribution lies in **how** we construct the instruction tuning dataset. This is where our paper differs from previous work and makes a significant contribution:\n\n1. We explicitly make the model generate refined queries with more interpretability and controllability.\n2. We do not reuse the original dataset output and claim that regenerating it is significant. To our best understanding, this is the first attempt to do so, and we view it as the novelty of this paper.\n\n[1] Modarressi, A., et al. (2024). MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory. arXiv preprint arXiv:2404.11672.\n\n**2. Fairness When Compared to ChatGPT**\n\nSorry for causing the misunderstanding. The method we compared (chain-of-thought and chain-of-note) uses retrieval settings where we utilize the generated thoughts and notes to retrieve relevant documents. Therefore, the comparison is **FAIR**, and we will make this clearer in our next version. Thank you for pointing this out.\n\nAdditionally, we have provided new results on our method trained on the new model LLaMA3 8B, which shows comparable results even to gpt4. This demonstrates the strong effectiveness and robustness of our method. Due to the limited characters allowed, please refer to the results shown in response to reviewer KkQG for more details.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed method is well-motivated and carefully designed. \n- While there are several work that leverages existing datasets for retrieval-augmented instruction-tuning, the authors empirically explore answer regeneration strategy that regenerates outputs of existing datasets based on the retrieved answers. This may shed light on improved instruction-tuning pipeline for retrieval-augmented LMs. \n- Experimental results show the effectiveness of proposed method compared with prior method such as SAIL or Self-RAG.", "reasons_to_reject": "- Limited novelties: as also noted by the authors, the proposed method is heavily influenced by prior work such as Self-RAG or SAIL, and the novelties newly introduced by this work may be limited. \n- The baseline models are somewhat weak — several recent work shows strong performance on the datasets, and also to my understanding, the ChatGPT and GPT-4 baselines do not use retrieval. Although I don’t think the proposed method should outperform proprietary models whose training details are private, claiming superiority over ChatGPT when ChatGPT baseline doesn’t use retrieval seems unfair to me. \n- Evaluations are mostly done in short-form QA tasks, and it is an open question how well we can apply the proposed setup for long-form generation tasks. For instance, the ensemble method shows the best performance on most of the tasks, but ensembling long-form outputs are not trivial.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NmCZfgzW9e", "reviewer": "Reviewer_KkQG", "summary": "This paper proposes an approach similar to Self-RAG or Self-Ask with specific actions to do multiple steps of retrieval and rewrite queries as needed. This system-like approach is pretty effective on various single-hop and multi-hop QA tasks. Notably, it outperforms Self-RAG on single-hop QA, although the baselines for multihop-QA do not seem particularly strong.\n\nThe main weakness is the writing is not very clear on the motivations for these actions, nor how their approach is specifically different to previous methods. For example, self-ask is not mentioned as a baseline and already includes multi-step retrieval and question decomposition.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your constructive feedback and would like to address your concerns as follows:\n\n**1. Difference with Self-Ask**\n\nThe reviewers mentioned that a primary weakness is the distinction between our approach and self-ask. Here, we justify the novelty of our work.\n\nFirst, self-ask relies on large language models with strong instruction-following abilities. It barely works with smaller models that cannot refine their answers without further training. Our method meticulously curates a retrieval-augmented instruction tuning dataset, training a small model to possess the ability to refine queries.\n\nSecond, self-ask performs multi-step reasoning implicitly, restricting users' control over the types of follow-up queries. Conversely, our method uses special tokens to explicitly guide the model to generate various types of query refinements, providing greater flexibility.\n\nAdditionally, we provide new results for the methods for completeness as follows;\n\n|              | HotpotQA | 2Wiki | Musique | Avg   |\n|---------------------|----------|-------|---------|-------|\n| **LLaMA3 8B (RQ-RAG)**| 74.2     | 59.3  | 48.1    | **60.5**  |\n| **Self-Ask**        |          |       |         |       |\n| **LLaMA2 7B**       | 30.8     | 20.9  | 13.5    | 21.7  |\n| **LLaMA3 8B**       | 44.5     | 44.0  | 25.9    | 38.1  |\n| **GPT-3.5**         | 62.5     | 57.0  | 37.2    | 52.2  |\n| **GPT-4**           | 63.3     | 66.6  | 44.0    | 58.0  |\n\n***Highlight***\n\nOur newly trained model on LLaMA3 8B surpasses the self-ask method on both GPT-3.5 and GPT-4, suggesting the strong capability of our method and its robustness effect over stronger options. We expect this response to resolve the issue of more strong baselines and solidify its effectiveness.\n\n**2. Difference Between PPL and Beam Search**\n\nIn our scenario, the sampling strategies we proposed resemble beam search. At each step, the different refinement action can be seen as a node to be explored. Among these nodes, there exists the most correct one (we use the upper bound to indicate the high potential of our system). The PPL or your mentioned universe SC can be seen as the heuristic to select the top-k node to expand. Under these circumstances, various heuristics can be proposed in our system. In this work, we preliminarily try out PPL and confidence, while other heuristics can be researched to further enhance the performance of our system.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The results compared to self-RAG are compelling. Although the multi-hop results seem to be missing key baselines.\n2. The ablations are informative for choosing between search engines, although it would be nice to see more discussion about retrieval accuracy across different search engines, such as whether they exhibit similar behavior and prefer different kinds of queries. \n3. The method is simple and flexible, depending mostly on synthetic labels. That being said, more could have been discussed about the dataset construction in the main text.", "reasons_to_reject": "1. The paper is not self-contained. Crucial details for the dataset construction are missing or left to the appendix. Details on baselines are not included at all, e.g. how does this differ from Self-RAG?\n2. The llama-2-7b baselines for multi-hop QA are not very effective, and it's a bit unclear whether this is because they do not conform to the task well, or they have some lack of capability.\n3. The self-ask approach is not discussed, yet it is a well known method for tasks like multi-hop QA with search engines.\n4. The Llama-2-7b is a weak model compared to various other models that would have been available at the time, accessible through APIs such as together.ai. For example, llama-2-70b. It is not clear whether their method will still be effective for these stronger options.\n\nTo make it more clear, the main weakness are the lack of self-ask baseline (and potentially other well known baselines) and general lack of details in the main text. Including those types of details would certainly strengthen the paper, although it may reveal limited novelty or effectiveness of the presented approach.", "questions_to_authors": "Why not try other self-consistency alternatives, such as universal SC?\n\nHow is PPL different from beam search?", "extra_scores": {}}, {"review_id": "PWPvPPo5p9", "reviewer": "Reviewer_H1CE", "summary": "Traditional retrieval-augmented systems often face two main issues: (1) the irrelevant content in retrieved documents may mislead the model and (2) searching with the original query often fails to retrieve adequate information. To this end, this paper introduces a method called Refine Query for Retrieval Augmented Generation (RQ-RAG), which trains a 7B Llama2 model to adaptively rewrite, decompose, and disambiguate the original input query with their collected datasets. They conduct experiments on both single-hop and multi-hop QA datasets. The results show the effectiveness, high potential, and robustness of their method.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments.\n\n**1. Comparison with Knowledge Editing Methods (e.g., MQuAKE and PokeMQA)**\n\nTo clarify, our work aims to enhance small models to explicitly refine queries for better answering both single-hop and multi-hop QA tasks, which do not overlap with the knowledge editing area mentioned by the reviewer. \n\nWe outline the differences between our work and knowledge editing as follows;\n\nKnowledge editing addresses the issue that information stored in LLM becomes outdated quickly, and retraining from scratch is often not feasible. This has led to techniques for injecting new facts by updating model weights. Although knowledge editing and our work aim to address outdated parametric knowledge, our approach focuses on retrieval-augmented generation to improve factuality and reliability. Knowledge editing requires a set of new knowledge to be updated, focusing on effectively updating these new knowledge without disrupting other facts. In our work, no edited facts are provided, making the methods suggested by the reviewer **inapplicable**.\n\n**2. The Effect of Different Actions**\n\nAs mentioned in response to reviewer PYYL, we carried out new experiments addressing the effect of different actions. For completeness, we include the results here.\n\n**Table1: The Ratio of Different Actions Under Different Sampling Strategies for 3 Tasks**\n\n|  | Decompose | Disambiguate | Rewrite | Direct |\n|----------------|-----------|--------------|---------|--------|\n| **Confidence** | 0.46      | 0.35         | 0.19    | 0.03   |\n| **Ensemble**   | 0.34      | 0.32         | 0.34    | 0.01   |\n| **PPL**        | 0.53      | 0.37         | 0.10    | 0.00   |\n\n**Table2: F1 Score for Deliberately Using a Specific Action for All Instances of 3 Tasks**\n\n|  | HotpotQA | 2Wiki | Musique | Avg   |\n|--------------------------|----------|-------|---------|-------|\n| **Rewrite**              | 74.8     | 59.2  | 46.2    | 60.1  |\n| **Decompose**            | 73.2     | 58.7  | 48.4    | 60.1  |\n| **Disambiguate**         | 72.6     | 60.3  | 45.0    | 59.3  |\n| **With Sampling Strategies** | 74.2     | 59.3  | 48.1    | 60.5  |\n\n**Main Findings:**\n- The proportion of different action selections is almost balanced, indicating minimal bias in the data as the reviewer was concerned (Table 1).\n- Simply choosing the same action might not yield the best performance. Each action contributes uniquely, and combining different actions leads to better overall performance (Table 2).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Positive experimental results: The method proposed in this paper outperforms existing state-of-the-art retrieval-augmented methods like Self-RAG-7B on multiple datasets, with detailed comparative tests against the baseline of the llama-7B model.\n\n2. Comprehensive experiments: The paper also demonstrates the effectiveness, rationality, and robustness of its method through experiments on the proportion of dataset enhancement, sampling strategies, and the sources of retrieved information.", "reasons_to_reject": "1. The paper proposes three methods of data enhancement: rewrite, decompose, and disambiguate. I am still concerned about how much these three methods actually contribute to performance improvement and whether they are all effective. \n\n2. For the MQA (Multi-Hop QA) task, the paper only compares with the Llama2-7B backbone baseline, which shows significant improvements. However, in the MQA field, knowledge editing is also a widely studied method (e.g., MeLLo [1], PokeMQA [2]). I recommend the authors provide comparisons with these similarly high-performing knowledge editing methods. \n\n\n3. The refined query method proposed in the paper adds Decomposition and Disambiguation to the queries. Is this effective for every query? Could this approach introduce bias in the processing of some original data?\n\n\n[1] Mquake: Assessing knowledge editing in language models via multi-hop questions\n[2] PokeMQA: Programmable knowledge editing for Multi-hop Question Answering", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes RQ-RAG to train (a relatively small) LM to better solve QA tasks, including explicitly rewriting the query, decomposing the task, and disambiguating the queries. The authors showed significant performance boosts across both single-hop and multi-hop QA datasets over the LLama2 7B model.

Pros:
- All reviewers agree that the improvements are quite significant and the experiments are fairly comprehensive.
- The method is well motivated and the framework is carefully designed.

Cons:
- Lack of baselines on multi-hop datasets: some reviewers point out the lack of baselines for multi-hop QA, e.g., many existing prompting methods aim at decomposing the multi-hop questions and answer them better, least-to-most, self-ask, etc. The authors address this to some extent in their rebuttal, but those baselines should be added to the main paper to be comprehensive.
- The authors claim existing decomposition methods like self-ask don't work well with small models. However, the proposed RQ-RAG used ChatGPT for data collection, so it is not a very fair comparison for self-ask on LLama2-7B only vs RQ-RAG.
- Contribution of each component is not super clear, better ablation experiments should be added.

Overall I'm leaning accept given the strong empirical gains but hope the authors can add more comprehensive baselines and ablations in the revision.
Decision: Accept