Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generation
Authors: Pengyue Jia, Derong Xu, Xiaopeng Li, Zhaocheng Du, Xiangyang Li, Yichao Wang, Yuhao Wang, Qidong Liu, Maolin Wang, Huifeng Guo, Ruiming Tang, Xiangyu Zhao
Accepted at: acl25

Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generation

Pengyue Jia1*, Derong Xu1,2*, Xiaopeng Li1*, Zhaocheng Du3, Xiangyang Li3 Yichao Wang3, Yuhao Wang1, Qidong Liu1, Maolin Wang1, Huifeng Guo3 Ruiming Tang2†, Xiangyu Zhao1†
1City University of Hong Kong,
2University of Science and Technology of China, 3Huawei Noah’s Ark Lab
Correspondence: xianzhao@cityu.edu.hk

Abstract
The reranker and generator are two critical components in the Retrieval-Augmented Generation (i.e., RAG) pipeline, responsible for ranking relevant documents and generating responses. However, due to differences in pretraining data and objectives, there is an inevitable misalignment between the documents ranked as relevant by the reranker and those required by the generator to support query-specific answers. To bridge this gap, we propose RADIO, a novel and practical preference alignment framework with RAtionale DIstillatiOn. Specifically, we first propose a rationale extraction method that leverages the reasoning capabilities of Large Language Models (LLMs) to extract the rationales necessary for answering a query. Subsequently, a rationale-based alignment process is designed to rerank documents based on the extracted rationales and fine-tune the reranker to better align the preferences. Extensive experiments conducted on three tasks across four datasets demonstrate the effectiveness and transferability of our approach. Our code is released online1.

1 Introduction
Large Language Models (LLMs), pretrained on massive datasets, have demonstrated exceptional reasoning and text generation capabilities, as evidenced by prior research (Zhao et al., 2023; Ni et al., 2025; Wang et al., 2023b; Xu et al., 2024; Fu et al., 2023). These models also adhere to the scaling laws, exhibiting improvements in performance and intelligence as the number of parameters increases (Kaplan et al., 2020). Retrieval-augmented generation (RAG) (Xu et al., 2025a,b) builds upon these capabilities by integrating information retrieval mechanisms (Zhao et al., 2018a,b; Liu et al., 2024) with generative models, such as LLMs. This approach not only mitigates the problem of hallucination in text generation but also enhances the system’s adaptability to dynamically evolving information needs, making it a robust solution for tasks requiring both accuracy and contextual relevance (Gao et al., 2023).

However, RAG pipelines typically assemble components (e.g., the reranker and generator (Fan et al., 2024)) that have been pretrained separately. Due to differences in their pretraining data and optimization objectives, these components often exhibit varying preferences, which can impact the overall effectiveness of the system. Specifically, pretrained rerankers (Xiao et al., 2023) are designed to evaluate the relevance between queries and documents. However, the documents identified as relevant under this criterion may not provide the necessary support for reasoning to derive an accurate answer to the query. Bridging this gap between the reranker’s relevance measurement and the generator’s reasoning requirements presents a significant challenge that must be addressed to improve the RAG pipeline’s performance.

Recent studies try to address this gap by training a bridge model (Ke et al., 2024), using LLM-based scores (Zhang et al., 2024) or combining both LLM-based and retrieval-based scores (Dong et al., 2024) to fine-tune RAG components. Additionally, while some other methods are not explicitly designed for this problem, they can indirectly contribute to bridging the gap. These approaches can use response quality (Ma et al., 2023) or perplexity distillation (Izacard et al., 2023; Shi et al., 2023) as signals to fine-tune the reranker. Despite showing promise, these approaches face critical limitations: their alignment signals rely solely on the surface-level connection between the query/answer and document, failing to capture the deeper reasoning processes or more complex relationships involved.

To address the above limitation, we propose RA-
DIO, a novel and practical preference alignment framework with rationale distillation in RAG. RADIO leverages rationale as a signal to bridge the reranker’s relevance measurement with the generator’s reasoning requirements for response generation. First, to efficiently extract the rationales needed to answer a query, we use the query and its ground truth answer as context and generate the rationales with LLMs. Second, to mitigate the preference misalignment between the reranker and generator while ensuring the solution remains practical, we rerank the documents based on the extracted rationales and fine-tune the reranker. This step distillates rationales from generators to rerankers, and aligns the reranker with the generator’s information needs for answering the query effectively.

RADIO effectively addresses the preference inconsistency between RAG components by first generating a comprehensive rationale and then fine-tuning the reranker based on the extracted rationale. This approach considers the deeper reasoning behind answers. We evaluate RADIO on three tasks across four datasets: Open-domain QA (NQ (Kwiatkowski et al., 2019) and TriviaQA (Joshi et al., 2017)), Multi-choice questions (MMLU (Hendrycks et al., 2020)), and Multi-hop QA (Musique (Trivedi et al., 2022)). The results validate the superiority of our method compared to other state-of-the-art baselines. Our contributions can be summarized as follows:

• We propose RADIO, a novel and practical framework designed to address the preference misalignment of different components in RAG pipelines.

• We introduce rationale distillation within the RAG framework, which is an effective approach that leverages explicit textual rationales as signals to align the preferences of different components in RAG.

• Extensive experiments are conducted on three tasks across four datasets to demonstrate the effectiveness and transferability of RADIO.

2 Related Work

Retrieval-Augmented Generation. Large language models (LLMs) have demonstrated groundbreaking performance across numerous tasks (Jia et al., 2025) but still face challenges such as hallucination and outdated knowledge (Gao et al., 2023). To address these issues, retrieval-augmented generation (RAG) has been introduced (Fan et al., 2024; Zhang et al., 2025; Jia et al., 2024a). RAG retrieves relevant information from external knowledge bases and incorporates it as contextual input to the generator (LLM), enhancing the accuracy and reliability of the generated responses. The typical RAG pipeline can be divided into several key components: query rewriter, retriever, reranker, and generator. The query rewriter (Wang et al., 2023a) modifies and expands (Jia et al., 2024b; Li et al., 2023a) the original query to improve retrieval recall, ensuring more relevant documents are retrieved. The retriever (Chen et al., 2024) fetches relevant documents based on the query. Dense retrievers generally outperform sparse retrievers in this step. To integrate contextual information more effectively and identify documents more relevant to the query, rerankers (Moreira et al., 2024a) with larger models and greater complexity are introduced to reorder the retrieved documents compared to retrievers. Finally, the generator—usually a powerful LLM such as GPT-4 (Achiam et al., 2023) or Llama (Touvron et al., 2023)—uses the query and the top-k documents from the reranker to generate the final response. In this work, we address the issue of preference misalignment among different components within the RAG pipeline. We aim to leverage rationale as a signal to align these preferences and enhance the overall performance of the RAG system. It is worth noting that RADIO and methods like ReAct (Yao et al., 2022) are not mutually exclusive and can be used together to complement each other. While RADIO enhances the ranking of retrieved documents at each step, ReAct focuses on task-solving through reasoning and action.

Preference Alignment. To further improve LLMs, preference alignment is often performed after the initial pretraining phase (Jiang et al., 2024). Approaches such RLHF (Ouyang et al., 2022) and DPO (Rafailov et al., 2024) are proposed to align the output of LLMs more closely with human preferences. DPO transforms tasks into classification problems, achieving high computational efficiency and strong performance. In the context of RAG, several works (Ke et al., 2024) can be transformed to address the challenge of preference alignment between RAG components. REPLUG (Shi et al., 2023) improves RAG pipelines involving blackbox LLMs by using the probability of the LLM generating the correct answer as a signal to determine document importance. Similarly, RRR (Ma et al., 2023) uses metrics based on the quality
of the LLM’s generated response as a signal to evaluate a document’s utility. On the other hand, ARL2 (Zhang et al., 2024) prompts LLMs to generate self-guided relevance labels for fine-tuning retriever, and DPA-RAG (Dong et al., 2024) introduces a bidirectional alignment strategy to mitigate preference inconsistencies in RAG pipelines. In this work, we focus on optimizing the reranker within RAG. Our goal is to enable the reranker to effectively identify supportive documents well-suited for the generator, whether black-box or open-source, facilitating the production of accurate outputs. In addition, BGM (Ke et al., 2024) trains a bridge model between retriever and LLMs to transform the retrieved information into the format LLM’s prefer. Our method is theoretically compatible with BGM and could be combined to jointly enhance performance without any conflict.

3 Methodology

In this section, we detail rationale distillation in RAG. Specifically, we first demonstrate the task definition of RAG in Section 3.1. Then we give an overview of our proposed framework in Section 3.2, introduce the rationale extraction method in Section 3.3, and detail the rationale-based alignment in Section 3.4 and optimization in Section 3.5.

3.1 Task Definition

To address the hallucination problem and enhance adaptability to dynamic information of LLMs, RAG systems have been proposed. These systems enhance generative models by introducing additional contextual information retrieved based on a given query \( q \). Specifically, when a query \( q \) is input into the RAG pipeline, the retriever \( \mathcal{R}_{\text{retriever}} \) first retrieves relevant documents by calculating similarity scores and top-\( k_1 \) selection. The process can be formalized as follows:

\[
\mathcal{D}_{\text{retriever}} = \{ d_i \mid d_i \in \text{Top-}k_1(\text{score}_{\text{retriever}}(q, d)) \}
\]

where \( q \) and \( d \) are the query and document, \( \text{score}_{\text{retriever}} \) denotes the score function in retriever, \( d_i \) means the \( i \)-th document in corpus and \( \mathcal{D}_{\text{retriever}} \) is the documents set output by retriever \( \mathcal{R}_{\text{retriever}} \).

To eliminate contextually irrelevant noise and provide more precise contextual information, the initially filtered documents will be further reranked by the reranker:

\[
\mathcal{D}_{\text{reranker}} = \{ d_j \mid d_j \in \text{Top-}k_2(\text{score}_{\text{reranker}}(q, d)) \}
\]

where \( \mathcal{D}_{\text{reranker}} \) is the documents selected by reranker, \( \text{score}_{\text{reranker}} \) denotes the score function in reranker. Note that the \( j \)th document \( d_j \) is in \( \mathcal{D}_{\text{retriever}} \) (i.e., \( d_j \in \mathcal{D}_{\text{retriever}} \)) and \( k_2 \) is the number of documents selected by reranker, which is smaller than \( k_1 \) used by the retriever.

Finally, the documents filtered by the reranker, along with the original query, will be fed into the generator as contextual information to help generate the final response:

\[
\hat{y} = \mathcal{G}(q, \mathcal{D}_{\text{reranker}})
\]

where \( \hat{y} \) is the generated response and \( \mathcal{G} \) denotes the generator.

It is worth noting that the documents selected by the reranker directly influence the generator’s input. Therefore, in this work, we aim to align the preferences of the reranker and generator to enhance their consistency. This alignment improves the overall accuracy of the RAG system’s responses.

3.2 Framework Overview

The overview of RADIO is depicted in Figure 1. RADIO is consisted of two phases: rationale extraction (Figure 1(a)) and rationale-based alignment (Figure 1(b)).

In the rationale extraction process, we combine the query with its ground truth answer and input
them into LLMs to generate precise rationales. Using the correct answer as context in the prompt improves the accuracy of the LLM’s rationale generation, ensuring that the generated rationale closely aligns with the requirements for deriving the correct answer.

In the rationale-based alignment process, our goal is to use the generated rationale to guide rerankers, enabling it to select documents that better support the generator in answering the query. Specifically, we leverage the generated rationale as a signal to rerank documents. The reranked documents will be used to fine-tune the reranker, addressing the preference misalignment between the reranker and the generator. By aligning these components, the process ensures that the selected documents are not only contextually relevant but also optimally supportive for the generator’s reasoning and response generation.

3.3 Rationale Extraction

Rationales are critical components of LLM reasoning processes and have been shown to enhance the accuracy of LLM-generated responses significantly. This perspective is supported by existing works such as Chain-of-Thoughts (CoT (Wei et al., 2022)). Existing work (Shi et al., 2023; Ma et al., 2023; Dong et al., 2024) has primarily focused on the initial relationships between queries and documents or indirect relationships between answers and documents, while overlooking rationales, a crucial intermediary component in the reasoning process. Motivated by this, we aim to extract rationales as signals to align the preferences of different components in the RAG pipeline.

To accurately extract the rationale necessary for answering a query and deriving the correct answer, we combine the query with the ground truth answer as contextual information of LLMs, as shown in Figure 1(a). The prompt template used for this process is as follows: "You are a professional QA assistant. Given a question and the ground truth answer, you can output the rationale why the ground truth answer is correct. Question: {question}. Answer: {answer}. Rationale: ". The generation process can be formalized as:

\[
r = \text{LLM}(q, a)
\]

where \( q \) and \( a \) are the query and answer, \( r \) deontes the generated rationale.

By doing so, we effectively bridge the gap between the query and the answer by generating the necessary rationale. This rationale accurately supports the reasoning process required to derive the correct answer from the query.

3.4 Rationale-based Alignment

Given the extracted rationale \( r \), a key challenge lies in effectively and efficiently utilizing it to improve preference consistency within the RAG pipeline. In this section, we propose a rationale-based alignment approach, where the rationale serves as a signal to fine-tune the reranker. This enables the reranker to identify and prioritize supportive documents that facilitate the generator in producing accurate responses. Specifically, we first use the retriever \( \mathcal{R}_{\text{retriever}} \) to retrieve \( k_1 \) relevant documents based on the query:

\[
\mathcal{D}_{\text{retriever}} = \mathcal{R}_{\text{retriever}}(q, \mathcal{C})
\]

where \( \mathcal{D}_{\text{retriever}} \) is the document set retrieved by \( \mathcal{R}_{\text{retriever}} \) based on query \( q \) and corpus \( \mathcal{C} \), and \( |\mathcal{D}_{\text{retriever}}| = k_1 \).

Next, to facilitate the comparison of similarity between different documents and the rationale, we use a text encoder to convert both the documents and the rationale into dense vectors.

\[
e_i^{\text{document}} = \text{Encoder}(d_i)
\]
\[
e^{\text{rationale}} = \text{Encoder}(r)
\]

where \( e_i^{\text{document}} \) and \( e^{\text{rationale}} \) denote the representations of ith document and rationale. \( d_i \) is the ith document and \( r \) represents the extracted rationale.

Then, we calculate the semantic similarity between each document and the rationale. Here, we use cosine similarity, denoted as \( \text{sim}(\cdot) \). The calculated scores indicate the degree to which each document supports generating the correct answer, with higher scores reflecting stronger support. We also linearly interpolate the score of documents with their retrieval score in the retrieval stage by weighted score sum.

\[
s_i^{\text{rationale}} = \text{sim}(e_i^{\text{document}}, e^{\text{rationale}})
\]
\[
s_i^{\text{retriever}} = \text{score}_{\text{retriever}}(q, d_i)
\]
\[
s_i' = \alpha s_i^{\text{rationale}} + (1 - \alpha)s_i^{\text{retriever}}
\]

where \( s_i^{\text{rationale}} \), \( s_i^{\text{retriever}} \), \( s_i' \) represent the rationale similarity score, retrieval score, and final score for ith document. \( \text{score}_{\text{retriever}}(\cdot) \) is the score function in retriever and \( \alpha \) is a hyperparameter used for integration. Note that we apply min-max normalization
in this work to both rationale score and retrieval score before integration.

Next, we rerank the documents based on their scores. Following the previous sampling method Top-k shifted by N (Moreira et al., 2024b), we select the top-ranked document as the positive sample and then shift by n documents and sample m negative samples from the subsequent documents to construct positive-negative pairs for fine-tuning the reranker. This process can be represented as:

\[
d_{pos} = d_i, \text{ where } i = \arg\max_s s_i \tag{11}
\]
\[
\{d_{neg}\} = \text{Sample}_m(\{d_i|\text{rank}(d_i) > n\}) \tag{12}
\]

where \( d_{pos} \) and \( d_{neg} \) are the sampling positive and negative documents, \( \text{Sample}_m(\cdot) \) denotes a sampling operation that selects m negative documents from the set of documents ranked lower than n.

3.5 Optimization
Following BGE embedding (Xiao et al., 2023) and QA Ranking Benchmark (Moreira et al., 2024a), we use InfoNCE as our optimization objectives to fine-tune reranker:

\[
f(q, d) = \exp(\phi(q, d)/\tau) \tag{13}
\]
\[
L = -\log \frac{f(q, d^+)}{f(q, d^+) + \sum_{i=1}^N f(q, d_i^-)} \tag{14}
\]

where \( d^+ \) and \( d^- \) represent the positive and negative document, \( \tau \) is the temperature parameter, and \( N \) denotes the number of negative documents.

4 Experiments

4.1 Datasets and Metrics
Following previous work (Shi et al., 2023; Zhang et al., 2024), we conduct experiments on three tasks across four datasets to evaluate RADIO with other methods: Open-domain QA (NQ (Kwiatkowski et al., 2019) and TriviaQA (Joshi et al., 2017)), Multi-choice questions (MMLU (Hendrycks et al., 2020)), and Multi-hop QA (Musique (Trivedi et al., 2022)). Detailed dataset descriptions are given in Appendix A.1. Following previous work (Ma et al., 2023; Shi et al., 2023), we report EM and F1 scores for QA datasets and EM for MMLU.

4.2 Baselines
To verify the effectiveness of RADIO, we conduct experiments with the following baseline methods: Base (Xiao et al., 2023), Atlas (Izacard et al., 2023), REPLUG (Shi et al., 2023), Trainable rewrite-retrieve-read (RRR) (Ma et al., 2023), ARL2 (Zhang et al., 2024), and DPA-RAG (Dong et al., 2024). The detailed introduction of baselines is given in Appendix A.2.

4.3 Backbone Rerankers
To validate the generality and adaptability of RADIO, we select three different rerankers as the backbone models for our experiments: gte-base (Li et al., 2023b), gte-large (Li et al., 2023b), and bge-reranker-base (Xiao et al., 2023). A detailed introduction is given in Appendix A.3.

4.4 Implementation Details
We implement RADIO on FlashRAG (Jin et al., 2024), a Python library for efficient RAG research. In the RAG pipeline, we take e5-base-v2 (Wang et al., 2022) as the retriever, and Meta-Llama-3.1-8B-Instruct (Touvron et al., 2023) as the generator. We sample 20,000 instances from NQ and TriviaQA to construct the fine-tuning dataset and fine-tune rerankers separately. For document sampling, we set the shift n in Top-k shifted by N method as 3, and sample 6 negative samples from the subsequent documents. To ensure a fair comparison, the sampling index is fixed and remains unchanged across methods. In the RAG pipeline, we set the number of documents selected by retriever and reranker (i.e., \( k_1 \) and \( k_2 \)) as 20 and 5. The retrieval corpus we used in experiments is Wikipedia (2018, December). For fine-tuning the reranker, we tune the training epochs from 1 to 5 and the integration hyperparameter \( \alpha \) from 0.0 to 1.0. We use Adam (Kingma, 2014) optimizer with a learning rate 6e-5 and a weight decay of 0.01. The prompts we used in experiments are given in Appendix A.4. We provide an analysis of the impact of fine-tuning sample size on performance in Appendix A.5, and we also give a detailed analysis on the shift parameter n and the negative samples m in Appendix A.6.

4.5 Main Results

4.5.1 Open-domain QA
To evaluate the effectiveness of RADIO and its transferability across different rerankers, we conduct experiments on the NQ and TriviaQA datasets. The results are presented in Table 1. A case study is given in Appendix A.7. From these results, we can draw the following conclusions:

• Compared to the Base method, most experimen-
Table 1: Overall experiments. “**” indicates the statistically significant improvements (i.e., two-sided t-test with \( p < 0.05 \)) over the best baseline. For all metrics, higher is better. \( \Delta \) represents the relative improvement of RADIO over Base method.

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="6">NQ</th>
    <th colspan="6">TriviaQA</th>
  </tr>
  <tr>
    <th>gte-base</th><th>FI</th><th>EM</th><th>gte-large</th><th>FI</th><th>EM</th><th>bge-reranker-base</th><th>FI</th><th>EM</th><th>gte-base</th><th>FI</th><th>EM</th><th>bge-reranker-base</th><th>FI</th><th>EM</th>
  </tr>
  <tr>
    <td>Base</td><td>0.2931</td><td>0.4046</td><td>0.2798</td><td>0.3935</td><td>0.3371</td><td>0.4603</td><td>0.5449</td><td>0.6374</td><td>0.5495</td><td>0.6414</td><td>0.6114</td><td>0.7120</td>
  </tr>
  <tr>
    <td>Atlas</td><td>0.3338</td><td>0.4587</td><td>0.3418</td><td>0.4677</td><td>0.3521</td><td>0.4832</td><td>0.5823</td><td>0.6752</td><td>0.6004</td><td>0.6972</td><td>0.6083</td><td>0.7063</td>
  </tr>
  <tr>
    <td>REPLUG</td><td>0.3257</td><td>0.4484</td><td>0.2607</td><td>0.3670</td><td>0.3427</td><td>0.4753</td><td>0.5679</td><td>0.6578</td><td>0.5248</td><td>0.6171</td><td>0.6032</td><td>0.7004</td>
  </tr>
  <tr>
    <td>RRR</td><td>0.3374</td><td>0.4608</td><td>0.3299</td><td>0.4578</td><td>0.3438</td><td>0.4754</td><td>0.5801</td><td>0.6716</td><td>0.5358</td><td>0.6237</td><td>0.6099</td><td>0.7091</td>
  </tr>
  <tr>
    <td>ARL2</td><td>0.3413</td><td>0.4688</td><td>0.3515</td><td>0.4804</td><td>0.3568</td><td>0.4885</td><td>0.6079</td><td>0.7086</td><td>0.6107</td><td>0.7120</td><td>0.6137</td><td>0.7149</td>
  </tr>
  <tr>
    <td>DPA-RAG</td><td>0.3391</td><td>0.4674</td><td>0.3385</td><td>0.4710</td><td>0.3462</td><td>0.4793</td><td>0.6080</td><td>0.7076</td><td>0.6097</td><td>0.7119</td><td>0.6149</td><td>0.7169*</td>
  </tr>
  <tr>
    <td>RADIO (Ours)</td><td><b>0.3512*</b></td><td><b>0.4790*</b></td><td><b>0.3565*</b></td><td><b>0.4850*</b></td><td><b>0.3665*</b></td><td><b>0.4917*</b></td><td><b>0.6084</b></td><td><b>0.7095*</b></td><td><b>0.6128*</b></td><td><b>0.7137*</b></td><td><b>0.6154</b></td><td><b>0.7151</b></td>
  </tr>
  <tr>
    <td>\( \Delta \)</td><td>\( \uparrow 19.82\% \)</td><td>\( \uparrow 18.39\% \)</td><td>\( \uparrow 27.41\% \)</td><td>\( \uparrow 23.25\% \)</td><td>\( \uparrow 8.72\% \)</td><td>\( \uparrow 6.82\% \)</td><td>\( \uparrow 11.55\% \)</td><td>\( \uparrow 11.31\% \)</td><td>\( \uparrow 11.52\% \)</td><td>\( \uparrow 11.27\% \)</td><td>\( \uparrow 10.65\% \)</td><td>\( \uparrow 10.40\% \)</td>
  </tr>
</table>

Table 2: Experimental results on MMLU. EM is reported as the evaluation metric. The source dataset used to fine-tune rerankers is the Open-domain QA dataset NQ. \( \Delta \) represents the relative improvement of RADIO over Base method.

<table>
  <tr>
    <th>Method</th>
    <th>Humanities</th>
    <th>Social</th>
    <th>STEM</th>
    <th>Other</th>
    <th>ALL</th>
  </tr>
  <tr>
    <td>Base</td><td>0.4089</td><td>0.6867</td><td><b>0.5147</b></td><td>0.6650</td><td>0.5502</td>
  </tr>
  <tr>
    <td>Atlas</td><td>0.3985</td><td>0.6935</td><td>0.5074</td><td>0.6563</td><td>0.5447</td>
  </tr>
  <tr>
    <td>REPLUG</td><td>0.4102</td><td>0.6854</td><td>0.5065</td><td>0.6590</td><td>0.5473</td>
  </tr>
  <tr>
    <td>RRR</td><td>0.4079</td><td>0.6913</td><td>0.5116</td><td>0.6572</td><td>0.5484</td>
  </tr>
  <tr>
    <td>ARL2</td><td>0.4147</td><td>0.7016</td><td>0.5106</td><td>0.6630</td><td>0.5540</td>
  </tr>
  <tr>
    <td>DPA-RAG</td><td>0.4157</td><td>0.701</td><td>0.5078</td><td>0.6652</td><td>0.5541</td>
  </tr>
  <tr>
    <td>RADIO (Ours)</td><td><b>0.4172</b></td><td><b>0.7013</b></td><td>0.5080</td><td><b>0.6717</b></td><td><b>0.5562</b></td>
  </tr>
  <tr>
    <td>\( \Delta \)</td><td>\( \uparrow 2.03\% \)</td><td>\( \uparrow 2.13\% \)</td><td>\( \downarrow 1.30\% \)</td><td>\( \uparrow 1.01\% \)</td><td>\( \uparrow 1.09\% \)</td>
  </tr>
</table>

tal settings achieve better results, demonstrating the necessity of preference alignment within the RAG pipeline.

• Compared to other baseline methods, RADIO consistently achieves superior performance across all datasets and reranker backbone configurations. The results validate the effectiveness of using rationales as signals for preference alignment in RAG pipeline.

• On TriviaQA, methods such as RRR and REPLUG show performance declines relative to the base method when using rerankers gte-large and bge-reranker-base. This indicates that these methods are sensitive, limiting their applicability. In contrast, RADIO demonstrates robust adaptability to different rerankers, achieving significant performance improvements across all three rerankers.

• As the reranker becomes larger or more powerful (e.g., progressing from gte-base to gte-large and further to bge-reranker-base), the performance ranking of models fine-tuned with RADIO aligns with the reranker’s inherent capabilities. This suggests that RADIO’s performance gains are sustainable and scalable with stronger rerankers, providing an avenue to further explore the upper performance limits of RAG pipelines.

4.6 Task Generalization Analysis

4.6.1 Multi-choice Questions
We also conduct experiments on MMLU. Since MMLU is a multiple-choice dataset, we report the EM metric (Ma et al., 2023). Additionally, following previous work (Yu et al., 2023), we fine-tune the reranker bge-reranker-base using open-domain QA as the source task and evaluate its performance on the MMLU dataset. Table 2 shows the results of fine-tuning reranker with NQ dataset. The results of fine-tuning reranker with TriviaQA are given in Appendix A.8. We can draw the following conclusions:

• From the metrics corresponding to the ALL category, RADIO demonstrates consistent improvements over Base. This highlights the effectiveness and generalization of RADIO, as it successfully adapts to multi-choice question tasks even when fine-tuned on the Open-domain QA tasks.

• Analyzing the results by question category, RADIO shows more significant improvements over the Base method in the Humanities and Social Sciences categories, with average gains of 2.03% and 2.13%, respectively. However, it exhibits a slight negative effect in the STEM category. This may be due to the fine-tuning datasets (NQ and TriviaQA), which are Open-domain QA datasets with distributions more similar to humanities and social sciences but markedly different from STEM subjects.

• Compared to other baseline methods, RADIO achieves top performance in the vast majority of
Table 3: Transferability analysis across generators. EM is reported as the metric for MMLU dataset.

<table>
  <tr>
    <th rowspan="2">Generator</th>
    <th rowspan="2">Method</th>
    <th colspan="2">NQ</th>
    <th colspan="5">MMLU</th>
  </tr>
  <tr>
    <th>EM</th>
    <th>F1</th>
    <th>Humanities</th>
    <th>Social</th>
    <th>STEM</th>
    <th>Other</th>
    <th>ALL</th>
  </tr>
  <tr>
    <td rowspan="2">Llama3.1-8b-instruct</td>
    <td>Base</td>
    <td>0.3371</td>
    <td>0.4603</td>
    <td>0.4089</td>
    <td>0.6867</td>
    <td><b>0.5147</b></td>
    <td>0.6650</td>
    <td>0.5502</td>
  </tr>
  <tr>
    <td>RADIO (ours)</td>
    <td><b>0.3665</b></td>
    <td><b>0.4917</b></td>
    <td><b>0.4172</b></td>
    <td><b>0.7013</b></td>
    <td>0.5080</td>
    <td><b>0.6717</b></td>
    <td><b>0.5562</b></td>
  </tr>
  <tr>
    <td rowspan="2">qwen2.5-14b-instruct</td>
    <td>Base</td>
    <td>0.3310</td>
    <td>0.4484</td>
    <td>0.5439</td>
    <td>0.8200</td>
    <td>0.7206</td>
    <td>0.7541</td>
    <td>0.6906</td>
  </tr>
  <tr>
    <td>RADIO (ours)</td>
    <td><b>0.3518</b></td>
    <td><b>0.4753</b></td>
    <td><b>0.5598</b></td>
    <td><b>0.8229</b></td>
    <td><b>0.7250</b></td>
    <td><b>0.7631</b></td>
    <td><b>0.6995</b></td>
  </tr>
  <tr>
    <td rowspan="2">gpt4o-mini</td>
    <td>Base</td>
    <td>0.3607</td>
    <td>0.4880</td>
    <td>0.6485</td>
    <td>0.8362</td>
    <td><b>0.6784</b></td>
    <td>0.8005</td>
    <td>0.7300</td>
  </tr>
  <tr>
    <td>RADIO (ours)</td>
    <td><b>0.3742</b></td>
    <td><b>0.5086</b></td>
    <td><b>0.6548</b></td>
    <td><b>0.8372</b></td>
    <td>0.6768</td>
    <td><b>0.8010</b></td>
    <td><b>0.7321</b></td>
  </tr>
</table>

Table 4: Experimental results on Musique. EM and F1 are reported as the metrics. The source datasets used to fine-tune rerankers are Open-domain QA datasets NQ and TriviaQA. Δ represents the relative improvement of RADIO over Base method.

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="2">NQ (Source Dataset)</th>
    <th colspan="2">TriviaQA (Source Dataset)</th>
  </tr>
  <tr>
    <th>EM</th>
    <th>F1</th>
    <th>EM</th>
    <th>F1</th>
  </tr>
  <tr>
    <td>Base</td>
    <td>0.0662</td>
    <td>0.1297</td>
    <td>0.0662</td>
    <td>0.1297</td>
  </tr>
  <tr>
    <td>Alllas</td>
    <td>0.0658</td>
    <td>0.1336</td>
    <td>0.0541</td>
    <td>0.1184</td>
  </tr>
  <tr>
    <td>REPLUG</td>
    <td>0.0608</td>
    <td>0.1269</td>
    <td>0.0559</td>
    <td>0.1188</td>
  </tr>
  <tr>
    <td>RRR</td>
    <td>0.0658</td>
    <td>0.1315</td>
    <td>0.0592</td>
    <td>0.1207</td>
  </tr>
  <tr>
    <td>ARL2</td>
    <td>0.0641</td>
    <td>0.1309</td>
    <td>0.0674</td>
    <td>0.1330</td>
  </tr>
  <tr>
    <td>DPA-RAG</td>
    <td>0.0629</td>
    <td>0.1307</td>
    <td>0.0690</td>
    <td>0.1337</td>
  </tr>
  <tr>
    <td>RADIO (Ours)</td>
    <td><b>0.0699</b></td>
    <td><b>0.1371</b></td>
    <td><b>0.0695</b></td>
    <td><b>0.1347</b></td>
  </tr>
  <tr>
    <td>Δ</td>
    <td>+5.59%</td>
    <td>+5.71%</td>
    <td>+4.98%</td>
    <td>+3.86%</td>
  </tr>
</table>

metrics, demonstrating its superiority and state-of-the-art capability.

4.6.2 Multi-hop QA
We also conduct experiments on the multi-hop dataset Musique with reranker bge-reranker-base, and the experimental results are shown in Table 4. Based on the experimental results, we can draw the following conclusions:

• First, RADIO achieves significant performance improvements in all evaluation metrics (EM and F1), demonstrating its task generalization ability in multi-hop reasoning tasks. On the NQ source dataset, RADIO achieves an EM value of 0.0699 and an F1 value of 0.1371, with improvements of 5.59% and 5.71% over the Base method, respectively. On the TriviaQA source dataset, RADIO’s EM value is 0.0695, and F1 value is 0.1347, showing improvements of 4.98% and 3.86% over the Base method.

• Secondly, compared to other baseline methods, RADIO demonstrates a more balanced and consistent advantage across both the NQ and TriviaQA source datasets. Notably, when using NQ as the source dataset, only RADIO shows a positive improvement in the EM metric, while all other baseline methods experience varying degrees of negative degradation.

4.7 Transferability Analysis across Generators
We conduct experiments on two datasets of different tasks, NQ and MMLU, using three different generators (Llama3.1-8b-instruct (Touvron et al., 2023), qwen2.5-14b-instruct (Yang et al., 2024), and gpt4o-mini (Achiam et al., 2023)) with bge-reranker-base fine-tuned on NQ to validate the transferability of our method, as shown in Table 3. We can find that (1) RADIO maintains its effectiveness across different generators, consistently enhancing the performance of the original RAG pipeline. This demonstrates RADIO’s strong transferability with various generators. (2) Comparing different generators reveals that RADIO’s performance gains are more pronounced with smaller, less capable generators. Specifically, when the generators are Llama3.1, Qwen2.5, and GPT4o-mini, RADIO achieves EM improvements of 8.72%, 6.28%, and 3.74%, respectively, and F1 improvements of 6.82%, 6.00%, and 4.22%. This is because as the generator’s capability increases and approaches the upper performance limits of the RAG pipeline, further enhancing the pipeline becomes increasingly challenging, resulting in a smaller improvement.

4.8 Ablation Study
To explore the specific impact of rationale and retrieval score, we design the following variants with reranker bge-reranker-base:

• w/o ALL: Base reranker without fine-tuning. Do not introduce rationale or retrieval score.

• w/o Retrieval: Ranking documents and fine-tuning reranker only based on the rationale scores.
Table 5: Ablation study.

<table>
  <tr>
    <th>Dataset</th>
    <th>Metrics</th>
    <th>w/o ALL</th>
    <th>w/o Retrieval</th>
    <th>RADIO</th>
  </tr>
  <tr>
    <td rowspan="2">NQ</td>
    <td>EM</td>
    <td>0.3371</td>
    <td>0.3587</td>
    <td><b>0.3665</b></td>
  </tr>
  <tr>
    <td>F1</td>
    <td>0.4603</td>
    <td>0.4858</td>
    <td><b>0.4917</b></td>
  </tr>
  <tr>
    <td rowspan="4">MMLU</td>
    <td>Humanities (EM)</td>
    <td>0.4089</td>
    <td>0.4168</td>
    <td>0.4172</td>
  </tr>
  <tr>
    <td>Social (EM)</td>
    <td>0.6867</td>
    <td>0.6981</td>
    <td><b>0.7013</b></td>
  </tr>
  <tr>
    <td>STEM (EM)</td>
    <td><b>0.5147</b></td>
    <td>0.5109</td>
    <td>0.508</td>
  </tr>
  <tr>
    <td>Other (EM)</td>
    <td>0.665</td>
    <td>0.6666</td>
    <td><b>0.6717</b></td>
  </tr>
  <tr>
    <td colspan="2">ALL (EM)</td>
    <td>0.5502</td>
    <td>0.5548</td>
    <td><b>0.5562</b></td>
  </tr>
</table>

![Four line plots showing EM and F1 performance on NQ and MMLU datasets as a function of alpha](page_180_670_1087_377.png)

Figure 2: Hyperparameter analysis on NQ and MMLU.

• RADIO: Fine-tuning reranker based on both rationale and retrieval scores.

Table 5 shows the results of ablation study on NQ and MMLU, where we can derive the following findings: (1) Both the rationale score and retrieval score contribute positively to RADIO’s performance, with the rationale score demonstrating a stronger positive impact compared to the retrieval score. (2) RADIO outperforms both w/o ALL and w/o Retrieval, while w/o Retrieval surpasses w/o ALL. This indicates that the rationale score and retrieval score are not conflicting but rather complementary. Their integration provides a more robust signal for document ranking, which effectively aids in fine-tuning the reranker. (3) In the STEM category of the MMLU dataset, w/o ALL outperforms both w/o Retrieval and RADIO. This could be attributed to the fact that the training dataset (NQ) contains questions with distributions more similar to humanities and social sciences, leading to trends in the Humanities, Social Sciences, and Other categories that differ from STEM category.

4.9 Hyperparameter Analysis

Figure 2 visualizes the performance of RADIO across different integration coefficients \( \alpha \) on NQ and MMLU. The x-axis represents the integration coefficients \( \alpha \) and the y-axis represents the evaluation metrics EM (red) or F1 (blue). For the MMLU dataset, we present results for two representative categories: Humanities and STEM. The trends for other categories align with those observed in Humanities. Complete experimental results are provided in Appendix A.9 for reference. From the figure, we can draw the following conclusions: (1) As \( \alpha \) increases, the metrics on the NQ dataset and most categories of the MMLU dataset exhibit a trend of first rising and then falling, with the optimal range for \( \alpha \) being between 0.3 and 0.7. This demonstrates the complementary nature of rationale scores and retrieval scores, which together form an optimal signal for preference alignment. (2) When \( \alpha = 0 \), the RAG performance is suboptimal because document scoring relies entirely on retrieval scores, focusing solely on query-document relevance while ignoring whether the document supports the generator in answering the query. Conversely, when \( \alpha = 1 \), the performance is still not optimal, as it completely disregards retrieval relevance, leading to a mismatch between the fine-tuning dataset and the training data, which negatively affects model performance. (3) The trends for STEM differ from those of other MMLU categories, showing an opposite pattern. This is likely due to the significant distributional differences between STEM and Humanities/Social Sciences, resulting in a "seesaw effect" as observed in the figure. It is worth noting that this phenomenon is also reflected in REPLUG (Shi et al., 2023), where the improvement in the STEM category is weaker compared to other categories.

5 Conclusion

In this paper, we propose a novel and practical preference alignment framework, RADIO, with rationale distillation in retrieval-augmented generation. First, we introduce a rationale extraction method to extract the rationales necessary for answering queries with LLMs. Next, a rationale-based alignment is proposed to rerank documents based on extracted rationales and fine-tune rerankers. Extensive experiments on three tasks across four datasets are conducted to validate the effectiveness of our proposed method against state-of-the-art baselines and demonstrate its strong transferability. We also release our code online to facilitate the following research and ease of reproduction.
6 Limitations

First, compared to other methods, our approach RADIO requires additional time in the rationale extraction stage to generate rationales. Since different samples are independent of one another, we can reduce generation time by employing parallel processing to mitigate this issue. Secondly, the MMLU experimental results reveal that the composition of fine-tuning datasets can affect RADIO’s effectiveness. This issue can be addressed by designing task-specific fine-tuning datasets for different downstream tasks or large-scale general fine-tuning datasets.

Acknowledgments

This research was partially supported by Research Impact Fund (No.R1015-23), Collaborative Research Fund (No.C1043-24GF), and Huawei (Huawei Innovation Research Program, Huawei Fellowship).

References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shaymal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Jianly Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Preprint, arXiv:2402.03216.

Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Zhicheng Dou, and Ji-Rong Wen. 2024. Understand what llm needs: Dual preference alignment for retrieval-augmented generation. arXiv preprint arXiv:2406.18676.

Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6491–6501.

Zichuan Fu, Xiangyang Li, Chuhuan Wu, Yichao Wang, Kuicai Dong, Xiangyu Zhao, Mengchen Zhao, Huifeng Guo, and Ruiming Tang. 2023. A unified framework for multi-domain ctr prediction via large language models. ACM Transactions on Information Systems.

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.

Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251):1–43.

Pengyue Jia, Qingpeng Cai, Xiangyu Zhao, Ling Pan, Xin Xin, Jin Huang, Weinan Zhang, Li Zhao, Dawei Yin, and Grace Hui Yang. 2025. Agentir: 2nd workshop on agent-based information retrieval. ACM SIGIR.

Pengyue Jia, Yiding Liu, Xiaopeng Li, Xiangyu Zhao, Yuhao Wang, Yantong Du, Xiao Han, Xuetao Wei, Shuaiqiang Wang, and Dawei Yin. 2024a. G3: an effective and adaptive framework for worldwide geolocalization using large multi-modality models. Advances in Neural Information Processing Systems, 37:53198–53221.

Pengyue Jia, Yiding Liu, Xiangyu Zhao, Xiaopeng Li, Changying Hao, Shuaiqiang Wang, and Dawei Yin. 2024b. Mill: Mutual verification with large language models for zero-shot query expansion. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2498–2518.

Ruili Jiang, Kehai Chen, Xuefeng Bai, Zhixuan He, Juntao Li, Muyun Yang, Tiejun Zhao, Liqiang Nie, and Min Zhang. 2024. A survey on human preference learning for large language models. arXiv preprint arXiv:2406.11191.

Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, and Zhicheng Dou. 2024. Flashrag: A modular toolkit for efficient retrieval-augmented generation research. arXiv preprint arXiv:2405.13576.

Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611.

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.

Zixuan Ke, Weize Kong, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024. Bridging the preference gap between retrievers and llms. arXiv preprint arXiv:2401.06954.
Diederik P Kingma. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453–466.

Xiaopeng Li, Lixin Su, Pengyue Jia, Xiangyu Zhao, Suqi Cheng, Junfeng Wang, and Dawei Yin. 2023a. Agent4ranking: Semantic robust ranking via personalized query rewriting using multi-agent llm. arXiv preprint arXiv:2312.15450.

Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjin Xie, and Meishan Zhang. 2023b. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281.

Qidong Liu, Xian Wu, Xiangyu Zhao, Yuanshao Zhu, Zijian Zhang, Feng Tian, and Yefeng Zheng. 2024. Large language model distilling medication recommendation model. arXiv preprint arXiv:2402.02803.

Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query rewriting for retrieval-augmented large language models. arXiv preprint arXiv:2305.14283.

Gabriel de Souza P Moreira, Ronay Ak, Benedikt Schifferer, Mengyao Xu, Radek Osmulski, and Even Oldridge. 2024a. Enhancing q&a text retrieval with ranking models: Benchmarking, fine-tuning and deploying rerankers for rag. arXiv preprint arXiv:2409.07691.

Gabriel de Souza P Moreira, Radek Osmulski, Mengyao Xu, Ronay Ak, Benedikt Schifferer, and Even Oldridge. 2024b. Nv-retriever: Improving text embedding models with effective hard-negative mining. arXiv preprint arXiv:2407.15831.

Wei Ni, Kaihang Zhang, Xiaoye Miao, Xiangyu Zhao, Yangyang Wu, Yaoshu Wang, and Jianwei Yin. 2025. Zeroed: Hybrid zero-shot error detection through large language model reasoning. arXiv preprint arXiv:2504.05345.

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida-Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36.

Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023. Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652.

Devendra Singh, Siva Reddy, Will Hamilton, Chris Dyer, and Dani Yogatama. 2021. End-to-end training of multi-document reader and retriever for open-domain question answering. Advances in Neural Information Processing Systems, 34:25968–25981.

Hugo Touvron, Thibaut Lavril, Gautier Izardc, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.

Harsh Trivedi, Niranjana Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multi-hop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554.

Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533.

Liang Wang, Nan Yang, and Furu Wei. 2023a. Query2doc: Query expansion with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9414–9423.

Maolin Wang, Yao Zhao, Jiajia Liu, Jingdong Chen, Chenyi Zhuang, Jinjie Gu, Ruocheng Guo, and Xiangyu Zhao. 2023b. Large multimodal model compression via efficient pruning and distillation at antgroup. arXiv preprint arXiv:2312.05795.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. Preprint, arXiv:2309.07597.

Derong Xu, Pengyue Jia, Xiaopeng Li, Yingyi Zhang, Maolin Wang, Qidong Liu, Xiangyu Zhao, Yichao Wang, Huifeng Guo, Ruiming Tang, et al. 2025a. Align-grag: Reasoning-guided dual alignment for graph retrieval-augmented generation. arXiv preprint arXiv:2505.16237.

Derong Xu, Xinhang Li, Ziheng Zhang, Zhenxi Lin, Zhihong Zhu, Zhi Zheng, Xian Wu, Xiangyu Zhao, Tong Xu, and Enhong Chen. 2025b. Harnessing large language models for knowledge graph question answering via adaptive multi-aspect retrieval-augmentation.
In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 25570–25578.

Derong Xu, Ziheng Zhang, Zhenxi Lin, Xian Wu, Zhihong Zhu, Tong Xu, Xiangyu Zhao, Yefeng Zheng, and Enhong Chen. 2024. Multi-perspective improvement of knowledge graph completion with large language models. arXiv preprint arXiv:2403.01972.

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671.

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.

Zichun Yu, Chenyan Xiong, Shi Yu, and Zhiyuan Liu. 2023. Augmentation-adapted retriever improves generalization of language models as generic plug-in. arXiv preprint arXiv:2305.17331.

Lingxi Zhang, Yue Yu, Kuan Wang, and Chao Zhang. 2024. Arl2: Aligning retrievers for black-box large language models via self-guided adaptive relevance labeling. arXiv preprint arXiv:2402.13542.

Yingyi Zhang, Pengyue Jia, Xianneng Li, Derong Xu, Maolin Wang, Yichao Wang, Zhaocheng Du, Huifeng Guo, Yong Liu, Ruiming Tang, et al. 2025. Lsrp: A leader-subordinate retrieval framework for privacy-preserving cloud-device collaboration. arXiv preprint arXiv:2505.05031.

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223.

Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2018a. Deep reinforcement learning for page-wise recommendations. In Proceedings of the 12th ACM conference on recommender systems, pages 95–103.

Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, and Dawei Yin. 2018b. Recommendations with negative feedback via pairwise deep reinforcement learning. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1040–1048.
A Appendix

A.1 Dataset Descriptions
The detailed descriptions of baselines are given as follows:

• Natural Questions (NQ): NQ contains real user questions compiled from Google search, with corresponding answers identified from Wikipedia by human annotators.

• TriviaQA: TriviaQA dataset comprises trivia questions paired with answer annotations and supporting evidence documents, such as web pages and Wikipedia articles. It is designed to assess a model’s ability to retrieve and comprehend textual evidence for open-domain question answering.

• Massive Multitask Language Understanding (MMLU): MMLU is a comprehensive evaluation dataset comprising 57 categories of questions, which are grouped into four broad domains: Humanities, Social Sciences, STEM, and Other. In this paper, we report evaluation metrics based on these categories.

• Musique: MuSiQue is a multi-hop question answering dataset created using a bottom-up approach to ensure connected reasoning, with 25,000 2–4 hop questions.

A.2 Baselines
Following is the introduction of baselines:

• Base (Xiao et al., 2023): The reranker model is used off-the-shelf without any fine-tuning.

• Atlas (Izacard et al., 2023): A pretrained retrieval-augmented language model designed for knowledge intensive task. We choose the EMDR² (Singh et al., 2021) as the reward to rerank documents and fine-tune rerankers.

• REPLUG (Shi et al., 2023): REPLUG seeks to fine-tune the retriever to enhance RAG pipelines that include black-box LLMs. It achieves this by using the query and document as contextual inputs and leveraging the probability of the LLM generating the correct answer as the importance score. This idea is also reflected in the PDist method in Atlas (Izacard et al., 2023).

Table 6: Prompts for QA datasets.

<html>
<table>
  <tr>
    <th>System Prompt</th>
    <td>Answer the question based on the given document. Only give me the answer and do not output any other words. The following are given documents.<br>{reference}</td>
  </tr>
  <tr>
    <th>User Prompt</th>
    <td>Question: {question}<br>Answer:</td>
  </tr>
</table>
</html>

• Trainable rewrite-retrieve-read (RRR) (Ma et al., 2023): RRR optimizes the query rewriter using the evaluation metrics of the final RAG output as a reward, which is used to fine-tune rerankers in our pipeline, enhancing the overall effectiveness of RAG.

• ARL2 (Zhang et al., 2024): ARL2 introduces a method to use LLMs as supervisor to generate self-guided relevance labels (e.g., “Not relevant”, “relevant”) for fine-tuning retriever.

• DPA-RAG (Dong et al., 2024): DPA-RAG proposes a knowledge preference pipeline to dual-align rerankers and generators in RAG. It combines document importance from the LLM’s perspective with the importance determined during the retrieval stage.

A.3 Detailed Introduction to Backbone Rerankers
To validate the generality and adaptability of RA-DIO, we select three different rerankers as the backbone models for our experiments: (1) gte-base (Li et al., 2023b): a reranker model proposed by Alibaba DAMO Academy, with 109M parameters and 768 embedding dimensions. (2) gte-large (Li et al., 2023b): the larger version of gte-base, with 335M parameters and 1024 embedding dimensions. (3) bge-reranker-base (Xiao et al., 2023): a powerful cross-encoder architecture reranker proposed by Beijing Academy of Artificial Intelligence, with 278M parameters.

A.4 Prompts
In this section, we detail the prompts we used in the experiments. For Open-domain QA datasets (NQ and TriviaQA) and Multi-hop QA dataset (Musique), the prompts are as shown in Table 6. For Multi-choice dataset MMLU, the prompts are shown in Table 7.
Table 7: Prompts for Multi-choice datasets.

System Prompt: Answer the question based on the given document. Only give me the option (A/B/C/D) and do not output any other words. The following are given documents.
{reference}
User Prompt:
Question: {question}
Answer:

<table>
  <tr>
    <th>Sample Num.</th>
    <th>EM</th>
    <th>F1</th>
  </tr>
  <tr>
    <td>0</td>
    <td>0.3371</td>
    <td>0.4603</td>
  </tr>
  <tr>
    <td>4,000</td>
    <td>0.3526</td>
    <td>0.4843</td>
  </tr>
  <tr>
    <td>8,000</td>
    <td>0.3634</td>
    <td>0.4896</td>
  </tr>
  <tr>
    <td>12,000</td>
    <td>0.3582</td>
    <td>0.4842</td>
  </tr>
  <tr>
    <td>16,000</td>
    <td>0.3604</td>
    <td>0.4889</td>
  </tr>
  <tr>
    <td>20,000</td>
    <td>0.3665</td>
    <td>0.4917</td>
  </tr>
</table>

Table 8: Experimental results on NQ with different numbers of fine-tuning samples.

Table 9: Hyperparameter analysis on the number of negative samples \( m \).

<table>
  <tr>
    <th>Shift (\( n \))</th>
    <th>Negative Samples \( m \)</th>
    <th>EM</th>
    <th>F1</th>
  </tr>
  <tr>
    <td>3</td>
    <td>2</td>
    <td>0.3615</td>
    <td>0.4886</td>
  </tr>
  <tr>
    <td>3</td>
    <td>4</td>
    <td>0.3601</td>
    <td>0.4888</td>
  </tr>
  <tr>
    <td>3</td>
    <td>6</td>
    <td>0.3665</td>
    <td>0.4917</td>
  </tr>
  <tr>
    <td>3</td>
    <td>8</td>
    <td>0.3604</td>
    <td>0.4895</td>
  </tr>
</table>

Table 10: Hyperparameter analysis on the shift \( n \) in Top-k shifted by \( N \)

<table>
  <tr>
    <th>Shift (\( n \))</th>
    <th>Negative Samples \( m \)</th>
    <th>EM</th>
    <th>F1</th>
  </tr>
  <tr>
    <td>1</td>
    <td>6</td>
    <td>0.3654</td>
    <td>0.4918</td>
  </tr>
  <tr>
    <td>2</td>
    <td>6</td>
    <td>0.3554</td>
    <td>0.4826</td>
  </tr>
  <tr>
    <td>3</td>
    <td>6</td>
    <td>0.3665</td>
    <td>0.4917</td>
  </tr>
  <tr>
    <td>4</td>
    <td>6</td>
    <td>0.3579</td>
    <td>0.4868</td>
  </tr>
  <tr>
    <td>5</td>
    <td>6</td>
    <td>0.3587</td>
    <td>0.4887</td>
  </tr>
</table>

A.5 Sample Efficiency Analysis of Fine-tuning

To understand the impact of fine-tuning samples on RAG performance, we conduct additional experiments to evaluate RADIO’s performance under a few-shot learning setup using varying amounts of fine-tuning samples (0, 4,000, 8,000, 12,000, 16,000, and 20,000 instances) on the NQ dataset with bge-reranker-base. The results are shown in Table 8.

We observe that even with a relatively small number of fine-tuning samples (e.g., 4,000 or 8,000), RADIO yields substantial improvements over the zero-shot baseline. Notably, with 8,000 samples the EM score increases by 7.8% and the F1 score by 6.4%, clearly demonstrating the model’s capacity to benefit from limited task-specific data. Moreover, as the number of fine-tuning instances increases, the overall performance tends to improve, suggesting that additional annotated data can further enhance the retrieval and generation capabilities of the system. Although there are minor fluctuations—such as a slight dip in performance at 12,000 and 16,000 samples—these variations may stem from differences in data distribution or optimization dynamics.

A.6 More Analysis on Training Hyperparameters

To understand the specific impact of two training hyperparameters (i.e., the shift \( n \) in Top-k shifted by \( N \) and the number of negative samples \( m \)) on performance, we conduct the following experiments on NQ based on bge-reranker-base:

• Varying \( m \): We fix \( n = 3 \) and vary \( m \) over the values 2,4,6, and 8.

• Varying \( n \): We fix \( m = 6 \) and vary \( n \) over the values 1,2,3,4, and 5.

The detailed experimental results are presented in Table 9 and Table 10, respectively. From the results, we can find the following points: (1) Impact of Negative Samples \( m \): Variations in the number of negative samples \( m \) have a relatively modest effect on RADIO’s performance. The results suggest that the model is robust to changes in \( m \), as only marginal performance differences are observed when \( m \) is varied from 2 to 8. (2) Impact of the shift parameter \( n \): In contrast, the shift parameter \( n \) in the Top-k shifted by \( N \) mechanism plays a more critical role. Our experiments show that smaller values of \( n \) yield better performance, while larger \( n \) values lead to a notable drop in both EM and F1 scores. We hypothesize that as \( n \) increases, the task of mining hard negatives becomes more challenging, which in turn hampers the fine-tuning process of RADIO.
Example 1: Query: which state is the richest state in nigeria
Ground Truth: ['Lagos State']
Document Reranked by RADIO
1. Niger State is a state in central Nigeria, and it is the largest state in the country in terms of land area. 2. Introduction to Niger State. 3. Lagos State is the most economically important state in the country, especially the city of Lagos. 4. Lagos is Nigeria's largest city. Lagos State is a significant financial center, and if it were a country, it would be the fifth-largest economy in Africa. 4. Ugbo Kingdom is a town in the Ijale Local Government Area of Ondo State in the southwestern part of Nigeria. Ugbo Kingdom is ranked as the richest king in Nigeria. 5. Rivers State, also known simply as Rivers, is one of the 36 states of Nigeria.

Response Given by RADIO: ✔️ Lagos State

Document Reranked by DPA-RAG
1. Niger State is a state in central Nigeria, and it is the largest state in the country in terms of land area. 2. Ugbo Kingdom is a town in the Ijale Local Government Area of Ondo State in the southwestern part of Nigeria. Ugbo Kingdom is ranked as the richest king in Nigeria. 3. Introduction to Niger State. 4. Kano is located in the northern part of Nigeria and is an important commercial and economic center of Nigeria. 5. The official language of Nigeria is English. The religious composition of Nigeria is roughly divided into Christians in the southern region and Muslims in the northern region.

Response Given by DPA-RAG: ✗ Can not give a answer.

Example 2: Query: who is the highest selling r&b artist of all time
Ground Truth: ['Rihanna', 'Michael Jackson']
Document Reranked by RADIO
1. R. Kelly is recognized as one of the best-selling music artists in the US. 2. Record sales definitions. Michael Jackson's 1982 album "Thriller" is still the best-selling album of all time. 3. Kelly is considered one of the most successful R&B artists of the past 25 years. 4. 1990s in Music. Janet Jackson, Michael Jackson, Whitney Houston, and Mariah Carey are among the best-selling music artists, particularly in the 1990s when they brought contemporary R&B to the global stage. 5. 2000s in Music. Destiny's Child, composed of Beyoncé, Michelle Williams, and Kelly Rowland, is the most successful female R&B group in history, selling over 50 million records globally in the 2000s.

Response Given by RADIO: ✔️ Michael Jackson

Document Reranked by DPA-RAG
1. R. Kelly is considered one of the most successful R&B artists of the past 25 years. 2. R. Kelly was one of the most successful male R&B artists of the 1990s and one of the best-selling music artists of all time. 3. 1990s in Music. R&B had a huge surge as other hip-hop artists such as Nas, Sean Combs, and The Notorious B.I.G. 4. 2000s in Music. Destiny's Child, composed of Beyoncé, Michelle Williams, and Kelly Rowland, is the most successful female R&B group in history, selling over 50 million records globally in the 2000s. 5. Rhythm and Blues.

Response Given by DPA-RAG: ✗ R. Kelly

Figure 3: Case study on NQ dataset.

A.7 Case Study

To intuitively illustrate the effectiveness of RADIO, we select examples from the NQ dataset to compare the documents reranked by RADIO with those reranked by a novel baseline, DPA-RAG, as well as their responses. In Figure 3 Example 1, the query asks, "Which state is the richest state in Nigeria?" RADIO successfully ranks information about Lagos State’s economic and financial status, which relates to the correct answer, among the top-3 documents. In contrast, DPA-RAG fails to identify documents relevant to answering the query, and cannot provide a valid response. In Figure 3 Example 2, the query is, "Who is the highest-selling R&B artist of all time?" RADIO prioritizes documents containing information about the correct answer, Michael Jackson, and effectively highlights key terms such as "R&B" and "best-selling." However, DPA-RAG misinterprets the query’s constraints, retrieving documents that either overlook the R&B artist specification or fail to consider the time span, resulting in an incorrect response. These examples demonstrate that RADIO enhances RAG by providing a more efficient and accurate reranking. It selects contextually appropriate documents, enabling the generator to infer correct answers.

A.8 More Results on MMLU

In this section, we give the complete experimental results on MMLU. Specifically, we use the NQ dataset and TriviaQA dataset as source dataset to fine-tune rerankers and evaluate them in MMLU. The results are shown in Table 11.

A.9 More Results on Hyperparameter Analysis

Figure 4 illustrates the trend of RADIO’s performance across various MMLU categories as the integration coefficient \( \alpha \) increases. For Humanities, Social Sciences, and Other, the trends are consistent: as \( \alpha \) increases, the performance metrics first improve and then decline. However, the STEM category shows a unique pattern, with metrics initially decreasing as \( \alpha \) grows, followed by an improvement. This divergence may be attributed to the fine-tuning dataset (NQ), which shares a closer distribution with Humanities, Social Sciences, and Other categories, while differing significantly from the STEM category.
Table 11: Experimental results on MMLU. EM is reported as the metric. The source datasets used to fine-tune rerankers are Open-domain QA datasets NQ and TriviaQA.

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="5">MMLU (Source Dataset NQ)</th>
    <th colspan="5">MMLU (Source Dataset TriviaQA)</th>
  </tr>
  <tr>
    <th>Humanities</th>
    <th>Social</th>
    <th>STEM</th>
    <th>Other</th>
    <th>ALL</th>
    <th>Humanities</th>
    <th>Social</th>
    <th>STEM</th>
    <th>Other</th>
    <th>ALL</th>
  </tr>
  <tr>
    <td>Base</td>
    <td>0.4089</td>
    <td>0.6867</td>
    <td><b>0.5147</b></td>
    <td>0.6650</td>
    <td>0.5502</td>
    <td>0.4089</td>
    <td>0.6867</td>
    <td><b>0.5147</b></td>
    <td>0.6650</td>
    <td>0.5502</td>
  </tr>
  <tr>
    <td>Atlas</td>
    <td>0.3985</td>
    <td>0.6935</td>
    <td>0.5074</td>
    <td>0.6563</td>
    <td>0.5447</td>
    <td>0.3966</td>
    <td>0.6822</td>
    <td>0.4868</td>
    <td>0.654</td>
    <td>0.5364</td>
  </tr>
  <tr>
    <td>REPLUG</td>
    <td>0.4102</td>
    <td>0.6854</td>
    <td>0.5065</td>
    <td>0.6590</td>
    <td>0.5473</td>
    <td>0.3977</td>
    <td>0.6744</td>
    <td>0.4821</td>
    <td>0.6466</td>
    <td>0.5323</td>
  </tr>
  <tr>
    <td>RRR</td>
    <td>0.4079</td>
    <td>0.6913</td>
    <td>0.5116</td>
    <td>0.6572</td>
    <td>0.5484</td>
    <td>0.4132</td>
    <td>0.6926</td>
    <td>0.5005</td>
    <td>0.6501</td>
    <td>0.5464</td>
  </tr>
  <tr>
    <td>ARL2</td>
    <td>0.4147</td>
    <td>0.7016</td>
    <td>0.5106</td>
    <td>0.6630</td>
    <td>0.5540</td>
    <td>0.4012</td>
    <td><b>0.6951</b></td>
    <td>0.5011</td>
    <td>0.6639</td>
    <td>0.5462</td>
  </tr>
  <tr>
    <td>DPA-RAG</td>
    <td>0.4157</td>
    <td>0.701</td>
    <td>0.5078</td>
    <td><b>0.6652</b></td>
    <td>0.5541</td>
    <td>0.4189</td>
    <td>0.6932</td>
    <td>0.5062</td>
    <td><b>0.6746</b></td>
    <td>0.5552</td>
  </tr>
  <tr>
    <td>RADIO (Ours)</td>
    <td><b>0.4172</b></td>
    <td><b>0.7013</b></td>
    <td>0.5080</td>
    <td><b>0.6717</b></td>
    <td><b>0.5562</b></td>
    <td><b>0.4230</b></td>
    <td>0.6942</td>
    <td>0.5090</td>
    <td>0.6678</td>
    <td><b>0.5559</b></td>
  </tr>
</table>

![Five line plots showing EM metrics on Humanities, Social Sci., STEM, Other, and All datasets as a function of alpha.](page_1012_1342_1027_180.png)

Figure 4: More results of hyperparameter analysis on MMLU.

Reviewer paper summaries: This paper proposes **RADIO**, a rationale distillation framework designed to address preference misalignment in Retrieval-Augmented Generation (RAG) pipelines. The approach leverages Large Language Models (LLMs) to extract rationales supporting the correctness of answers, which are then used to fine-tune the reranker for improved alignment with the generator. Extensive experiments on multiple datasets and tasks demonstrate the effectiveness and transferability of the approach.

This paper introduces RADIO, a novel framework designed to address the misalignment between rerankers and generators in Retrieval-Augmented Generation (RAG) pipelines. RADIO leverages Large Language Models (LLMs) to extract rationales from query-answer pairs and uses these rationales to fine-tune rerankers, ensuring better alignment with the generator’s reasoning needs. The framework is evaluated across three tasks (Open-domain QA, Multi-choice QA, Multi-hop QA) on four datasets, demonstrating significant improvements over state-of-the-art baselines. Key contributions include the rationale distillation mechanism and extensive empirical validation.
Summary of strengths: 1. **Comprehensive Evaluation**: Experiments are conducted across various datasets (NQ, TriviaQA, MMLU, Musique) and tasks (open-domain QA, multiple-choice, multi-hop QA), showcasing broad applicability.
2. **Strong Empirical Results**: RADIO consistently outperforms baseline methods across multiple settings, with significant improvements in metrics like EM and F1.
3. **Practical Implementation**: The method is computationally efficient since it only fine-tunes the reranker, making it feasible for real-world deployment.
4. **Clear Ablation Studies**: The paper provides thorough ablation studies and hyperparameter analyses, highlighting the effectiveness of the rationale-based signal.

1. Novel Alignment Strategy: RADIO introduces explicit rationales as a bridge between relevance and reasoning, addressing a critical gap in RAG pipelines.
2. Empirical Rigor: The paper conducts thorough experiments across diverse tasks/datasets (NQ, TriviaQA, MMLU, Musique) and evaluates multiple reranker backbones, showcasing robustness.
3. Practical Impact: The open-source code and clear ablation studies (e.g., Appendix A.8) facilitate reproducibility and further research.
4. Generalization: RADIO’s effectiveness across different generators (Llama, Qwen, GPT4o-mini) highlights its adaptability.
Summary of weaknesses: 1. **Method Simplicity**: The approach primarily focuses on fine-tuning the reranker without considering improvements for the retriever or generator, which limits the overall novelty.
2. **LLM Dependency**: The rationale extraction heavily relies on LLMs, and the paper does not explore methods to ensure the quality or validity of these rationales.
3. **Data Bias Concern**: The model shows inconsistent performance on STEM tasks, highlighting potential data bias in the fine-tuning dataset.

1. Rationale Generation Dependency: The rationale extraction relies on ground-truth answers (Line 223-224, Section 3.3), which may not be feasible in real-world scenarios where answers are unknown.
2. Computational Cost: The LLM-based rationale generation step introduces latency (Lines 635-636, Section 6). While parallel processing is suggested, the paper does not quantify the overhead or propose optimizations.
3. STEM Performance: RADIO underperforms in STEM tasks (Lines 463-464, Section 4.6.1), attributed to dataset bias. However, the paper lacks targeted solutions for domain-specific alignment.
Comments, suggestions and typos: 1. The paper could benefit from a deeper discussion on how rationale quality affects final performance and whether alternative strategies for rationale generation were considered.

**Typos**
- *Desctiptions* → *Descriptions* (Appendix A.1)
- *Deontes* → *Denotes* (Section 3.3)
- Ensure consistent spacing and punctuation across sections, particularly in equations and figure descriptions.

1. Rationale Optimization: Explore unsupervised or weakly supervised rationale generation to reduce reliance on ground-truth answers (Section 3.3).
2. Efficiency Enhancements: Investigate lightweight alternatives for rationale generation (e.g., smaller LLMs or retrieval-based methods) to mitigate computational costs (Section 6).
3. Domain-Specific Fine-Tuning: Propose strategies to align rationales with STEM reasoning patterns, such as incorporating subject-specific corpora (Section 4.6.1).
Overall assessment scores: 3

4
Metareview score: 3
Metareview: This paper introduces RADIO, a framework designed to improve preference alignment between rerankers and generators in Retrieval-Augmented Generation (RAG) systems. The approach leverages rationale distillation, using large language models (LLMs) to extract rationales from query-answer pairs, which are then used to fine-tune rerankers (note that this method does not assume access to any ground-truth documents). In other words, this is a work about using LLM to generate better reranker training data. The method is evaluated across multiple tasks and datasets, demonstrating consistent improvements over strong baselines.
Summary of reasons to publish: • Novelty and Relevance: The paper addresses a well-motivated problem in RAG pipelines, namely the misalignment between document rerankers and downstream generators. The proposed rationale-based alignment is a clear and original contribution that fills a gap in current approaches.

• Empirical Validation: Extensive experiments are conducted on diverse tasks (open-domain QA, multi-choice QA, multi-hop QA) and datasets (NQ, TriviaQA, MMLU, Musique), with results showing robust gains across different reranker architectures and generator models.

• Reproducibility and Impact: The authors provide open-source code and detailed ablation studies, supporting reproducibility and facilitating further research. The framework’s generalization across various generators (including Llama, Qwen, and GPT4o-mini) highlights its practical applicability.