SEER : Self-Aligned Evidence Extraction for Retrieval-Augmented Generation (2024)

Abstract: Recent studies in Retrieval-Augmented Generation (RAG) have investigated extracting evidence from retrieved passages to reduce computational costs and enhance the final RAG performance, yet it remains challenging. Existing methods heavily rely on heuristic-based augmentation, encountering several issues: (1) Poor generalization due to hand-crafted context filtering; (2) Semantics deficiency due to rule-based context chunking; (3) Skewed length due to sentence-wise filter learning. To address these issues, we propose a model-based evidence extraction learning framework, SEER, optimizing a vanilla model as an evidence extractor with desired properties through self-aligned learning. Extensive experiments show that our method largely improves the final RAG performance, enhances the faithfulness, helpfulness, and conciseness of the extracted evidence, and reduces the evidence length by 9.25 times. The code will be available at https://github.com/HITsz-TMG/SEER.

## introduction recent years have witnessed the prevailing winds of retrieval-augmented generation (rag), which is a popular paradigm for improving the performances of large language models (llms) in various downstream tasks, such as question answering, making the output more reliable @xcite @xcite , interpretable @xcite , and adaptable @xcite . traditional practices @xcite often involve providing top-retrieved passages as the input context to llms without discrimination. however, imperfect retrieval systems frequently yield irrelevant content. furthermore, indiscriminately feeding all retrieved content to llms will cause input redundancy, im-corresponding author. ideally, llms should be grounded on supporting content that is both highly helpful to address user input and sufficiently concise to facilitate inference speed. however, it is practically impossible for imperfect retrieval systems to achieve such an ideal grounding solely @xcite . in fact, top-retrieved passages usually compose supporting and distracting content, inflicting a heavy blow on llms trained with high-quality corpora to generate the correct output. this motivates us to develop an evidence extractor, that aims at extracting supporting content while filtering out distracting content. recently, a pioneering study, filco @xcite , attempts to retrieve chunking document content with sentence precision via three filters, i.e., strinc, lexical, and cxmi. then, it trains a context filtering model, using context filtered by the above three measures as ground truth. despite effectiveness, current context-filtering methods have several limitations: (1) hand-crafted context filtering. manually designed contextfiltering measures typically require domain knowledge, which can hardly be adaptable to diverse downstream tasks with limited supervision. (2) disruptive chunking on context. the use of chunking strategies may be ineffective as rule-based splitting on context usually cannot preserve its original semantics and often produces semantically deficient text blocks. (3) skewed distribution in length. the length of supporting content in topretrieved passages may vary largely across different samples. hence, learning to filter context sentencewise is biased toward skewed length distribution. given these limitations, an interesting question arises: now that heuristic-based augmentation foot_0 suffers from several issues, can we develop a model-based augmentation method free of the above problems? inspired by the recent success of self-alignment @xcite @xcite , self-aligned learning utilizes the model to improve itself and aligns its response with desired properties, which can mitigate the heavy reliance on hand-crafted context filtering, rule-based context chunking, and sentence-wise filter learning. given the extracted evidence, a question arises again: how to evaluate the quality of evidence properly? in principle, the evidence should be faithful (i.e., avoiding intrinsic hallucination) to the retrieved passages @xcite , helpful in addressing the user input @xcite , and concise to facilitate the inference speed @xcite . figure 1 shows three representative scenarios: (1) when the evidence only favors faithfulness, llms may generate an incorrect answer; (2) when the evidence further favors helpfulness but lacks conciseness, llms' attention may be distracted by noise; (3) when the evidence favors all three criteria, llms can generate confidently with low computational costs. in this paper, we propose a model-based evidence extraction learning framework, seer, self-aligned evidence extraction for retrieval-augmented generation. specifically, it consists of three primary stages: (1) evidence extraction: to mitigate the issues above, we propose extracting diversified evidence with semantic consistency and varying length through response sampling, offering sufficient preference data for alignment. (2) expert assessment: for each extracted evidence, we construct a quadruple, quadqare, made up of query, answer, passage, and evidence. then, we devise three experts to assess the quality of each extracted evidence w.r.t. three primary criteria. given these scores, we propose smoothing cov-weighting, which explicitly leverages the statistics to estimate their relative weighting and result in the cov-weighted scores. (3) self-alignment: with a ranking list of extracted evidence and their smoothing cov-weighted scores, a question remains: how to optimize extraction preference with the ranking position? to this end, we propose a listwise-aware lambda preference optimization method, lpo, assigning each preference pair with a listwise-aware weight scaled by the gain in reciprocal rank from swapping the position of two evidence @xcite @xcite . it is worth mentioning that seer is a criterionagnostic framework and can employ any off-theshelf expert. in this work, we use faithfulness, helpfulness, and conciseness, which are regarded as three primary criteria for assessing the quality of evidence @xcite @xcite . our main contributions can be summarized as four-folds: ## methodology figure 3 depicts the overall framework of seer, composing three key stages: (1) evidence extrac-tion ( §3.1), which extracts evidence via response sampling. (2) expert assessment ( §3.2), which assesses the quality of evidence. (3) self-alignment ( §3.3), which aligns the extractor with extraction preference. the learning algorithm of our proposed method can be seen in appendix d in algorithm 1. ## conclusion this work explores the method that learns to extract high-quality evidence to assist model generation and reduce computational cost. different from previous works heavily relying on heuristics, we introduce a novel evidence extraction learning framework, seer, which utilizes the model to calibrate its extraction preference via self-alignment. to this end, we first probe into model extraction preferences via response sampling, then assess the quality of extracted evidence via experts, and finally optimize the vanilla model as an evidence extractor via self-alignment. extensive experiments show that seer considerably improves the final rag performance. moreover, it can extract more faithful, helpful, and concise evidence, and also shows higher robustness against data noise issues.