Bridging the Preference Gap between Retrievers and LLMs (2024)

Abstract: Large Language Models (LLMs) have demonstrated superior results across a wide range of tasks, and Retrieval-augmented Generation (RAG) is an effective way to enhance the performance by locating relevant information and placing it into the context window of the LLM. However, the relationship between retrievers and LLM in a RAG is still under-investigated. Most existing work treats the retriever and the LLM as independent components and leaves a gap between retrieving human-“friendly” information and assembling a LLM-“friendly” context. In this work, we examine a novel bridge mechanism. We validate the ranking and selection assumptions of retrievers in the context of RAG and propose a framework that chains together supervised and reinforcement learning to train a bridge model that optimizes the connection between the retriever and the LLM. Empirical results demonstrate the effectiveness of our method in both questionanswering and personalized generation tasks.

heading: 1 Introduction
text: Large language models (LLMs) such as GPT-4 (OpenAI, 2023) and PaLM 2 (Anil et al., 2023), have demonstrated impressive performance on a wide variety of tasks. Retrieval-augmented generation (RAG), which retrieves knowledge items from an external data source and puts it into the context window of LLMs, has produced significantly enhanced results in many NLP tasks (Khandelwal et al., 2020; Borgeaud et al., 2022; Izacard et al., 2022; Yasunaga et al., 2023).
However, most works on RAG study retrievers and LLMs separately. On one hand, most retrievers are designed to be human-friendly, usually based on the general belief in classic information
∗ The work was done during internship at Google Research.
†The work was done as a visiting researcher at Google Research.
retrieval literature that ranking is paramount, as humans typically read from top to bottom (Robertson, 1977). On the other hand, LLMs exhibit preferences different from humans and yield accurate results only when the information in the prompt aligns with these preferences. This discrepancy leads to sub-optimal design in current RAG systems, a phenomenon we term preference gap. This gap manifests in various aspects. For example, the general belief in ranking may not align with LLM’s preferences due to the self-attention mechanism of Transformers, which can focus on any token regardless of its position. Another aspect is selection; while humans can easily disregard irrelevant information in a context, it has been shown that LLMs are highly sensitive to irrelevant content (Shi et al., 2023a). There likely exist more aspects that further diverge the LLM’s preference from that
10438
of humans, e.g., repetition. Repeated information is generally considered detrimental for retrieval systems (Xia et al., 2017), but it may be useful to LLMs for weighting the relevance of context items.
We empirically investigate this preference gap, focusing specifically on ranking and selection. As shown in Fig. 1, when we randomize the ordering of top-5 retrieved items (in our case, passages), the performance of RAG only varies by around 1% 1. However, the variation exceeds 5% when the LLM is only presented with the top-1 passage under each order (therefore it encounters different selections of information). This indicates the general belief in ranking does not always apply to LLMs, and that the selection of information could be more crucial. This finding confirms the existence of the preference gap between retrievers and LLMs, and it is critical to bridge this preference gap to enhance the performance of RAG. To the best of our knowledge, this is a novel insight that may guide future designs of RAG systems.
Existing work has tried to finetune the LLMs to align with the retriever or adjust the retriever to align with the LLM. However, finetuning LLMs, especially at the scale of GPT-4 or Palm 2, is often expensive. Similarly, it is difficult to adjust production-level retrievers such as Google or Bing. Even when the retriever is adjustable, existing efforts often focus on re-ranking the retrieved results and fail to address other aspects of preference such as selection or repetition. Instead, we propose a novel and practical framework called BGM (Bridging the Gap between retrievers and LLMs), which keeps the retriever and LLM fixed and trains a bridge model in between. The bridge model aims to transform the retrieved information into a format that LLMs prefers and can effectively work with.
Without loss of generality, we structure the bridge model as a sequence-to-sequence (seq2seq) model, which allows dynamically selecting items, re-ranking them, and potentially broader operations like repeating some of them in the retrieval-augmented prompt. Training such a bridge model is challenging as there are usually no ground truth labels on ideal item sequences for
1We note that this finding is different from (Liu et al., 2023), where a “loss in the middle” phenomenon is observed, meaning that the LLM is better at using the relevant information at the beginning or end of its input context. This is probably due to different numbers of retrieved items. Liu et al. (2023) used 20 documents, whereas we use 5 passages. Regardless, both findings indicate that there exists a preference gap between the retrievers and LLMs, which we aim to bridge.
retrieval augmentation. Existing work has tried to derive supervisory signals for ranking from RAG’s downstream task performance, such as perplexity distillation (Izacard et al., 2022). Nevertheless, these methods only provide pointwise supervisory signals for each item independently. Directly applying the same idea to obtain sequential supervision is infeasible, since it would require feeding all possible item sequences into the LLM to obtain perplexity. We developed a greedy search approach to solve this problem (Sec. 4.1). Moreover, we find sequential supervision can be too sparse to effectively train such a seq2seq model (Table 5). To address this issue, we employed reinforcement learning (RL) on the SL trained bridge model, regarding the downstream task performance as the reward and the bridge model as a policy model. Chaining SL and RL provides increased supervision from the downstream task. It also offers the flexibility to explore more advanced strategies, such as repetition, in forming the optimal passage sequence.
It is important to note that BGM transcends the limitations of traditional re-rankers. Typically, rerankers most often overlook the selection process (i.e., they simply return the top-k results) and score passages independently, without considering collective properties such as result diversity. In contrast, BGM addresses both ranking and selection jointly, and enhances collective aspects like diversification. Furthermore, BGM often reduces the API cost of the LLM, and decreases its overall runtime. This is because BGM significantly reduces the number of tokens fed into the final LLM by selecting relevant passages only. This approach is similar in spirit to a multi-stage retriever (i.e., a retriever plus re-ranker), which is more cost-effective.
Our experiments reveal that BGM can enhance the performance of various downstream tasks, such as Question Answering (QA) and personalized generation, across a spectrum of datasets, from public QA and amazon reviews to private email conversations. Notably, the modified passages retrieved by BGM surpass the performance of strong retrievers and baseline reranking models. This underscores the significance and promise of the “bridge” approach in the realm of RAG. In summary, our contributions can be summarized as follows:
• We empirically establish the existence of the preference gap between retrievers and LLMs, and introduce BGM to address this gap.
• We propose a seq2seq bridge model to jointly
accomplish reranking and selection, adapting the retrieved information to be LLM-friendly. We employ a SL and RL training scheme to optimize this adaptation process.
• We evaluate BGM with diverse tasks, including QA and text generation, with publicly available and personalized datasets. The evaluation underscores the effectiveness of BGM in bridging the preference gap and improving RAG performance in downstream tasks.

heading: 2 Related Work
text: Retrieval-augmented Generation (RAG). Augmenting LLMs with relevant information retrieved from various knowledge sources is proven effective across numerous NLP tasks, including language modeling (Borgeaud et al., 2022; Khandelwal et al., 2020; Shi et al., 2023b), question answering (Lewis et al., 2020; Izacard et al., 2022; de Jong et al., 2023; De Jong et al., 2023; Shi et al., 2023b; Guu et al., 2020; Izacard and Grave, 2020; Xu et al., 2023), fact versification (Lewis et al., 2020) and text generation (Lewis et al., 2020). Specifically, RAG utilizes input as a query and comprises two main components: (1) a retriever retrieves a set of items from a side corpus. Particular items may vary across different tasks, including documents, passages, or even tokens. In this study, we focus on retrieving passages; and (2) a LLM incorporates the retrieved items, as additional information in the input context, and makes final predictions.
A fundamental question in this process arises regarding the disparate preferences between LLMs
and retrievers, as LLMs performing optimally only when their preferences are satisfied. Bridging the preference gap is crucial. Depending on which components are subject to updates, this challenge can be categorized into three families.
Finetuning retrievers and LLMs jointly. This is the most widely used setting of RAG (Izacard et al., 2022; Khandelwal et al., 2020; Wu et al., 2022; Guu et al., 2020; Lewis et al., 2020). However, most prior work is based on relative small LMs (< 1B). For example, Altas (Izacard et al., 2022) finetunes LLM (T5 (Raffel et al., 2020a)) and retriever (Contriever (Izacard et al., 2021)) jointly by leveraging the LLM to provide supervisory signal to train the retriever. RAG (Lewis et al., 2020) uses a tunnable query encoder and DPR (Karpukhin et al., 2020) as retriever, BART as LLM, and design an end-to-end schema to train the query encoder and the LLM.
Finetuning LLMs only. Updating retrievers is not always desirable as it is costly and requires the document index to be periodically updated. To bridge the preference gap, it is also possible to only update the LLMs. FiD (Izacard and Grave, 2020) takes the retrieved documents and query as input, finetunes the LLM to adapt to the external information. Similarly, Lummen (De Jong et al., 2023) and Glimmer (de Jong et al., 2023) improve FiD via adding reranker and pre-encoding memeory.
Finetuning retrievers only. Although above systems have shown improves results, they are not always applicable in practice. Many LLMs (especially larger ones) are only available as black-box APIs. Given this constraint, a natural approach is to only update the retrievers to ensure they can retrieve passages that are more compatible with LLMs. REPLUG (Shi et al., 2023b) adapts a similar idea as Atlas but fix the LM. RECOMP (Xu et al., 2023) trains a compressors to summarize the retrieved document from retriever. However, this family of models is incapable of performing any sample-level selection and can only choose top passages by setting a fixed threshold.
Unlike the existing approaches, BGM does not fine-tune the LLM or the retriever and instead employs a bridge model in between (Fig. 2).
RL for information retrieval. Before the LLM era, RL has been used in information retrieval (IR) (Xia et al., 2017; Wei et al., 2017; Zeng et al., 2018; Xu et al., 2020). The core approach was to frame the IR problem as a Markov Decision Process and
apply an RL algorithm to solve it. Typically, an IR task would be structured to determine which document to select for a ranking position, using ranking metrics such as DCG as the reward. None of these existing studies explore the application of RL in the context of RAG.
In the LLM era, RL has been used in query rewriting for retrieval (Wu et al., 2021; Nogueira and Cho, 2017; Adolphs et al., 2021), where a black-box retriever is assumed. This is a different problem from RAG. Bacciu et al. (2023) suggest using RL to fine-tune retriever in the RAG context in their opinion paper, not supported by experiments. Their work does not recognize the importance of bridging the gap between retrievers and LLMs, nor does it specify what the bridge model should be.

heading: 3 Problem Formulation
text: Retriever. Given an input x, the retriever aims to retrieve a ranked list of passages from a corpus D = {di}mi=1 that are relevant to x. In this work, we assume a typical scenario of employing a frozen dense retriever. Typically, a dual encoder architecture is applied, where an encoder is used to encode both the input context x and the passage d. Specifically, the encoder maps each passage to an embedding E(d). The similarity between input and passage embedding is computed by their cosine similarity,
s(d, x) = cos(E(d),E(x)). (1)
The top-k passages that have the highest similarity scores when compared with the input x are retrieved in this step,
(dretr.j ) k j=1 = Top-K({s(d, x)}mi=1). (2)
Bridge Model for RAG. The retrieved top-K passages provide richer information about the original input/query x to help the LLM to make a better prediction on downstream tasks. A bridge model B adapts the retrieved passages to a sequence of passages that is LLM-friendly. As mentioned in the Sec. 1, the bridge model is a seq2seq model. It takes all the retrieved passages (dretr.j ) k j=1 as well as the query x as input, and outputs the adapted passages (dbdr.j ) n j=1,
(dbdr.j ) n j=1 = B ( x, (dretr.j ) k j=1 ) . (3)
This formulation is general enough as the seq2seq model automatically considers ranking by
generating the next token based on the preceding one, selection by placing the end-of-sentence token in the appropriate position, and repetion by generating the same passage ID (as explained in the following paragraph). Note that n may be smaller or larger than k due to selection and repetition.
Before concatenating the query and passages as bridge model’s input, we prepend each passage with a unique sentinel token as its passage ID, e.g., [query][id1]dretr.1 [id2]d retr. 2 . In this way, the model only needs to generate the passage IDs instead of the actual passage content, which is much more efficient and avoids making unfaithful changes to the retrieved passages. We then convert the obtained passage IDs to the corresponding passages for downstream processing.
Retrieval-augmented generation with bridge. We concatenate adapted passages from the bridge model, (dbdr.j ) n j=1, with the input x, and fed the resulting long sequence into the LLM as context to obtain the output for downstream tasks.
Algorithm 1: Synthesis SPS Input: (dretr.j )nj=1, R(·) Output: (dsilv.j )sj=1
1 dsilv. ← (); 2 Rsilv. ← R(dsilv.); 3 Rbest ← −∞; 4 while true do 5 for c← dretr.1 to dretr.n where c 6∈ dsilv. do 6 d′ ← add_to_seq(dsilv., c); 7 Rcur ← R(d′); 8 if Rcur > Rbest then 9 cbest ← c;
10 Rbest ← Rcur;
11 if Rbest > Rsilv. then 12 dsilv. ← add_to_seq(dsilv., cbest); 13 Rsilv ← Rbest; 14 Rbest ← −∞; 15 else 16 break;

heading: 4 Training the Bridge Model
text: In Eq. 3, we format the bridge model as seq2seq model. As mentioned in Sec. 1, its training is challenging due to the lack of supervision for passage sequences, the infeasibility of directly applying existing methods that provide only point-wise su-
pervision, and the sparsity of sequential supervision. For effective training, we propose to chain supervised learning (SL) and reinforcement learning (RL), where SL aims to reduce the search space of RL and provides a reasonably good initial model that does ranking and selection. RL aims to optimize the policy model, i.e., bridge model, for the downstream task. Fig. 3 shows an overview.

heading: 4.1 Supervised Learning
text: To conduct SL, the ground-truth passage sequence is required for each query. Existing approaches, which focus on obtaining the relevance score of each passage (see Section 1), are not applicable as we need to determine which combination of passages is most effective for the downstream tasks. To address this, we propose to synthesis silver passage sequence (SPS) by selecting only the useful passages. This is done by greedy search that incrementally selects the next passage that can maximize the downstream task performance.
Synthesising SPS using greedy search. We denote the downstream task performance when using a given passage sequence for RAG as R(·). For the edge case where the passage sequence is empty, R(∅) simply denotes the task performance without retrieval augmentation (i.e., no passage used). We start from an empty SPS dsilv. = ∅, and iteratively
add the next best candidate passage to the sequence, measured based on the resulted task performance R(dsilv.). We stop until no improvement can be made to R(dsilv.). Algorithm 1 shows the pseudocode for synthesising SPS. The training with the SPS is achieved by applying cross-entropy loss.

heading: 4.2 Reinforcement Learning
text: Although SL can already help training the bridge model, it is still ineffective – we observe using SL alone results in mixed performance (see Table 5). This is attributed to sparse supervision (see Sec. 1) and the lack of end-to-end training on downstream results.
To address these issues, we apply RL to continue the training of the bridge model. In SL, we only consider permutations or deletions in the SPS, while RL can accommodate more complex manipulations that an optimal passage sequence might require, such as repetition. Additionally, RL provides enhanced supervision beyond the silver sequences through the reward from sampled passage sequences. Using the performance of downstream task as the reward, the bridge model is trained in an end-to-end manner. Specifically, our task can be formulated as an RL problem – Reward is the performance of the downstream task, usually measured based on certain ground-truth labels, e.g., Ex-
tract Match or BLEU. Policy model is the bridge model that need to be trained. Action space is restricted to passage IDs (Sec. 3) as we are interested in organizing rather than revising the retrieved passages. In training, the reward objective can be optimized by any off-the-shelf RL algorithm, e.g., proximal policy optimization (PPO).

heading: 5 Experiments

heading: 5.1 Datasets and Baselines
text: Datasets. We consider four datasets, ranging from popular QA datasets to personalized generation datasets. We also include one dataset that contains private email conversations (Avocado Email), which is unlikely to be included in the LLM’s pretraining datasets. This will further help us investigate the effectiveness of our proposed BGM model, as the LLM will have to rely on the retrieved passages. The summary of statistics is given in Table 1.
Open-domain QA. We conduct evaluations on two open-domain QA datasets: Natural Questions (NQ) (Kwiatkowski et al., 2019) and HotpotQA (Yang et al., 2018). They both consist of questions, answers collected from Wikipedia and the Web. HotpotQA is a multi-hop QA dataset which requires findings and reasoning over multiple passages to answer the question. The candidate passages are retrieved from Wikipedia pages.
Personalized Generation. We follow (Li et al., 2023) to construct the personalized generation datasets. In this task, the query includes the start and necessary properties (e.g., title) of a document authored by a user. The objective is to complete the document as if it were finished by the same user. The candidate passages for this task are retrieved from documents previously authored by this user. This includes datasets from two domains: Avocado Email (Email) (Oard et al., 2015) and Amazon Book (Book) (Ni et al., 2019). An example from Book is given in Fig. 4.
Baselines. We consider the state-of-the-art baselines: (1) GTR (Ni et al., 2021) a widely recognized retriever that operates independently of LLMs; (2) a variant of GTR, termed Random, in which the order of passages retrieved by GTR is randomized; (3) Point-wise score ranking (PSR) (Izacard et al., 2022). This is a variant from (Izacard et al., 2022) where we substitute the decoder with our LLM and apply perplexity distillation. This approach can be regarded as utilizing a reranker as a bridge model, but lacking the capabil-
ity for dynamic selection; and (4) Additionally, we include a non-retrieval baseline, Naive, where no retrieval augmentation in the generation process.

heading: 5.2 LLM and Hyperparamters
text: We select the T5-XXL (11B) (Raffel et al., 2020b) model as our bridge model for most experiments. In the supervised learning stage, the T5-XXL model is fine-tuned with a base learning rate of 0.001. A linear warmup scheduler is used for the first 1,000 training steps. Additionally, the square root normalized decay of the learning rate is applied. The model is trained until its performance converges on the validation set. Decoding is performed using beam search with a beam size of 4. We use the PaLM2-S model (Anil et al., 2023), a new state-of-the-art LLM, as our LLM. It adopts temperature sampling as the decoding strategy. The parameters of PaLM2-S are frozen, and we set the temperature to 0 to make the output deterministic. We use Exact-Match (EM) and BLEU metrics, depending on the dataset. It is also used as the reward to train the bridge model in the RL stage. The K in “Top-K” in Eq. 2 is set to 5.

heading: 5.3 Evaluation Results and Analysis
text: Superiority of BGM. Table 2 reports the overall performance. We can see that
(1) The proposed BGM outperforms all the 4 baselines in all 4 datasets. This clearly indicates
BGM is effective in adapting the retrieved passages. It’s noteworthy that Random and GTR perform similarly, suggesting that ranking has less impact than selection of passages does.
(2) Compared to the Naive approach, BGM shows significant improvement overall. An exception is observed in the Book dataset, where the improvement of BGM is less pronounced. This suggests that retrieval is not always essential in this context. This also explains why the Naive approach outperforms other baselines (except BGM) on the Book dataset. It’s important to note that BGM still manages to show improvement by dynamically deciding whether and how many passages to select. For example, in the Book dataset, for around half of the samples (approximately 25,000), BGM selects no passages for augmentation.
(3) Compared to the GTR approach, BGM demonstrates substantial improvement. Compared to HotpotQA, NQ shows smaller improvement, as most instances need only one passage, which both GTR and BGM can successfully include. Conversely, the HotpotQA shows a more substantial improvement, indicating that HotpotQA may be more sensitive to irrelevant passages.
(4) Compared to the PSR approach, BGM again demonstrates significant improvement. This indicates that pure reranking alone is not sufficient for the bridge model. Selection must also be taken into account. It’s notable that PSR performs similarly to GTR, further suggesting that reranking alone has limited impact.
Understanding BGM. Next, we study a few research questions about BGM and RAG.
(1) Can we perform effectively passage selection for RAG by simply thresholding PSR? We conducted an ablation experiment using various thresholds for PSR, as shown in Table 3. “TopK” in rows 3 to 6 denotes selecting only the top-k passages from PSR reranked passages. The results vary but are consistently lower than BGM’s. This suggests that a naive manual threshold applied to
the reranking model is insufficient to achieve the desired objectives. Therefore, it is necessary to consider both reranking and dynamic selection simultaneously.
(2) How does different SPS (sliver passage sequence) used for SL affect BGM performance? In Table 2, we employed greedy search passage sequences as SPS for SL (Sec. 4.1). Here, we explore whether this approach is superior to others. Our ablation experiment in Table 4 involved using various types of SPS. The first row, labeled “GTR”, indicates the use of GTR’s retrieved passages as SPS. Similarly, “PSR” refers to using PSR reranked passages as SPS. In the final row, we used greedy search passage sequences, representing the proposed version of BGM. The results demonstrate that the quality of SPS significantly affects downstream task performance. While using PSR as SPS shows improvement over GTR on three datasets, Greedy (BGM) further enhances performance and achieves the best results. Identifying potentially better SPS is left for future work.
(3) How helpful is RL to BGM? BGM integrates SL and RL, and we aim to assess the effectiveness of each component. An ablation experiment is detailed in Table 5. We can observe that BGM, when operating with only SL, performs significantly worse than the full BGM model in NQ, HotpotQA, and Email, and in some cases, it even underperforms GTR. This suggests that SL alone is inadequate, highlighting the necessity of incorporating RL. Note that removing SL is likely ineffective, leading to a poorly initialized policy model
for RL with an excessively large search space. (4) How does the size of bridge model affect the final performance? In Table 2, we utilized Flan-T5-XXL (11B) as the bridge model, which is already smaller than the PaLM2-S. However, we are curious about the feasibility of using an even more lightweight LM as the bridge model. The ablation study shown in Table 6 presents the outcomes of employing bridge models of various sizes. It is evident that all three sizes (large, XL, and XXL) surpass the performance of the setup without a bridge model (i.e., GTR), with the largest size yielding the best results. This demonstrates that a bridge model is beneficial even at a smaller scale, and that larger sizes lead to further improvements.
(5) How does the size of LLMs affect the final performance? In Table 2, Palm2-S was used as the LLM. We are interested in evaluating the effectiveness of the bridge model with different sizes of LLMs. We conducted experiments using Palm2XXS in Table 7. It is important to note that the smaller LLM struggles with personalized generation datasets (i.e., Email and Book), resulting in BLEU scores lower than 1%. We opted not to report these results as they may not accurately reflect the trend. However, observations from NQ and HotpotQA suggest that BGM significantly outperforms all baselines by a large margin. This indicates that BGM is effective even with a smaller LLM.
(6) Can a bridge model generalize to different datasets and LLMs? Our experiments above demonstrate the effectiveness of the bridge model when trained “in-domain” (i.e., trained and tested on the same dataset) using PaLM2-S as the LLM. A more ambitious goal is to extend this performance to new datasets or LLMs without extra training. We conducted ablation experiments to investi-
gate bridge model generalizability: Table 8’s upper section shows the results across different datasets. Row 2 shows the performance of the bridge model when trained exclusively on the NQ dataset and then tested on three other unseen datasets. Similarly, row 3 shows training BGM solely on the Email dataset and testing it on the other three. In all cases, performance falls short of BGM’s when trained and tested on the same datasets. This is expected, given the lack of techniques for dataset generalization, a topic we leave for future work.
In the lower section of Table 8, we present the results of BGM when tested on Palm2-XXS, but trained on Palm2-S. Comparing with results of BGM both trained and tested on Palm2-XXS (see row 1 of the bottom section), the mismatch between the training and testing LLMs leads to a significant decline in performance. This suggests that BGM’s ability to generalize across different LLMs is currently limited. Addressing this is considered an important direction for future research.
(7) Case Studies. We provided examples of GTR, PSR, and BGM in Table 9 in Appendix for the NQ dataset 2. For question I, both GTR and
2We find interesting cases from the Avocado Email dataset a well. Specifically, we find that BGM tends to select previous emails/passages that are relevant in terms of writing style and targeting audience, instead of emails that are topically relevant. However due to data license restrictions, we are unable to show these emails as examples.
PSR yield the same incorrect output, even though both include a relevant passage for RAG. Only BGM provides the correct answer, indicating that additional irrelevant context can be noisy and detrimental to RAG’s performance. In question II, none of the candidate passages contain the answer (they discuss FaZe Clan and the number of subscribers, but do not identify who has the most subscribers). GTR and PSR provide incorrect answers, as their additional context is unhelpful. In contrast, BGM opts not to select any passages and answers the question using its own memory, resulting in the correct answer. This demonstrates that retrievalaugmented processes are not always necessary, and BGM is capable of handling such cases.

heading: 6 Conclusion
text: This paper demonstrates the need to bridge the preference gap between retrievers and LLMs, which has various levels of impact on ranking and selection in RAG systems. We propose a bridge model, BGM, to adapt the output of a frozen retriever for frozen LLMs, formatting the task as a seq2seq problem. BGM chains supervised and reinforcement learning, for dense supervisions and end-to-end training. Our extensive experiments validate BGM’s effectiveness.

heading: 7 Limitations
text: While effective, BGM has certain potential limitations. Firstly, BGM is limited in terms of generalization across datasets and domains, as shown in Sec. 5.3. Secondly, the silver passage sequences for supervised learning is synthesized using greedy search. Although Sec. 5.3 demonstrates its effectiveness, it is worth exploring other synthesizing approaches. We leave these explorations to the future work, as bridging the preference gap is an critical but new problem in RAG.

heading: 8 Ethics Statement
text: This paper proposes a novel bridge model that can effectively bridge the preference gap between retrievers and Large Language Models (LLMs). We do not anticipate any negative consequences for individuals as a result of this research. In the event of system failure, the primary outcome would be suboptimal performance of an LLM, and we do not foresee notable ethical implications.

heading: Future Work
text: We leave these explorations to the future work, as bridging the preference gap is an critical but new problem in RAG