Uplift- RAG : Uplift-Driven Knowledge Preference Alignment for Retrieval-Augmented Generation (2025)

Abstract: Retrieval-augmented generation (RAG) has proven effective in enhancing the knowledge coverage of large language models (LLMs) and mitigating hallucinations by incorporating external retrieved documents. However, documents deemed relevant by the retriever are not necessarily helpful for answer generation, and including misleading information can even degrade performance. Existing efforts to estimate document utility often rely on the downstream generation performance, which conflates the influence of external documents with the intrinsic knowledge of the LLM, thereby obscuring the actual contribution of the retrieved content. To address this, this paper proposes Uplit-RAG, a uplift-driven knowledge preference alignment framework for RAG. Specifically, we first propose an uplift-based definition of document utility that quantifies each document’s marginal benefit over the LLM’s internal knowledge. We then optimize the reranker with three alignment objectives to identify and prioritize documents based on their uplift. This enables dynamic selection of documents that address the LLM’s knowledge gaps, going beyond fixed top-k selection, while reducing reference redundancy and the computational overhead of the LLM’s input. Extensive experiments demonstrate the effectiveness of Uplift-RAG.

## introduction retrieval-augmented generation (rag), which enhances large language models (llms) with access to retrieved documents relevant to the question @xcite @xcite , has substantially improved the performance of llms in handling knowledge-intensive tasks while effectively mitigating hallucination issues. a typical rag system comprises several components that work collaboratively to support effective response generation @xcite , where the retrieval subsystem constitutes the architectural cornerstone for providing reference documents. however, the retrieval component is typically optimized for query-document relevance metrics, and usually fails to address the essential requirement of the generator for utility-driven document selection @xcite , where the retrieved content must not only be topically related but also actionable for refining the response accuracy of the llm. as shown in figure 1 , our analysis reveals that among documents ranked by standard relevance-based methods, only 17.81% actually assist the generator in correcting its potential errors, while 45.20% paradoxically degrade performance by introducing conflicting evidence that overrides the correct internal knowledge of the llm. the remaining 37% exhibit neutral effects. these results highlight a fundamental misalignment between the retrieval objective and the need of llm for knowledge supplementation: retrieval models maximize document recall under lexical/semantic similarity metrics, whereas llms require documents that fill specific knowledge voids in their parametric memory. in some cases, seemingly relevant documents may even conflict with the internal knowledge of the llm, damaging its performance. bridging this knowledge preference gap is critical to improving the effectiveness of rag system. the community has made preliminary attempts to address this preference misalignment, but three fundamental challenges remain inadequately resolved. first, defining and quantifying document utility remains an open problem. existing approaches often rely on the downstream generator's performance as a proxy for utility @xcite @xcite . this method, however, suffers from a critical credit assignment problem: it fails to disentangle the retrieved document's actual contribution from the llm's inherent parametric knowledge or its reasoning capabilities. second, aligning retrieval mechanisms with such a nuanced utility definition poses a significant hurdle. current retrievers are predominantly optimized for topical relevance. integrating a utility dimension requires a paradigm shift where the system must not only identify relevant documents but also assess their potential to complement the generator's knowledge and rectify its potential inaccuracies. third, the inherently adaptive nature of document utility complicates its modeling. the usefulness of a particular piece of information is not static; it is query-specific and model-dependent. therefore, a one-size-fits-all approach, such as retrieving a fixed top-k set of documents, is suboptimal, leading to the inclusion of noisy or redundant information, and increasing llm's computational overhead without guaranteeing performance gains. toward this end, this paper proposes uplift-rag. we first propose a novel uplift-based approach to accurately define and quantify document utility. unlike prior methods relying solely on relevance or entangled utility metrics as supervision signals, our approach leverages uplift, a measure capturing the genuine incremental benefit each document provides to the generator. specifically, we prompt the llm to independently generate responses using each retrieved document, evaluating the quality of these responses against ground truth labels and comparing them to the baseline performance without external documents. the resulting performance differential constitutes the uplift of each document. to mitigate the discrepancy between the retrieval objective and the need of the llm for knowledge supplementation, we then introduce a lightweight reranker that utilizes the computed uplift scores to effectively integrate utility considerations into the retrieval process. concretely, we incorporate a point-wise loss to directly identify documents beneficial to the generator. to ensure effective prioritization, we further integrate pair-wise and list-wise losses, enabling the reranker to rank documents by their uplift accurately. by jointly optimizing these objectives, the reranker aligns closely with the specific knowledge needs of the generator. in addition, our reranker selectively identifies and includes only those documents that genuinely address the llm's immediate knowledge gaps, thereby avoiding the retrieval of redundant or misleading content, and reducing unnecessary computational overhead. we conduct extensive experiments on three qa datasets, demonstrating the effectiveness and robustness of our method. notably, our upper bound analysis of uplift-rag demonstrates its maximum achievable improvement of 25%-105% (as shown in figure 2 ), along with a 50%-60% reduction in token costs, highlighting its promising capability. in summary, our main contributions are as follows: ## conclusion in this paper, we propose uplift-rag, a novel uplift-driven preference alignment framework for rag. uplift-rag first computes the uplift of each document by comparing response quality with and without that document. subsequently, we leverage uplift as the alignment signal to achieve three alignment objectives, enabling the reranker to simultaneously possess both classification and ranking capabilities. extensive experiments demonstrate that uplift-rag not only reduces computational resources but also improves the overall performance.