Semantic Contribution-Aware Adaptive Retrieval for Black-Box Models (2025)

Abstract: Retrieval-Augmented Generation (RAG) plays a critical role in mitigating hallucinations and improving factual accuracy for Large Language Models (LLMs). While dynamic retrieval techniques aim to determine retrieval timing and content based on model intrinsic needs, existing approaches struggle to generalize effectively in black-box model scenarios. To address this limitation, we propose the Semantic Contribution-Aware Adaptive Retrieval (SCAAR) framework. SCAAR iteratively leverages the semantic importance of words in upcoming sentences to dynamically adjust retrieval thresholds and filter information, retaining the top- \alpha % most semantically significant words for constructing retrieval queries. We comprehensively evaluate SCAAR against baseline methods across four long-form, knowledge-intensive generation datasets using four models. Our method achieved the highest score on each dataset with GPT-4o. Extensive experiments also analyze the impact of various hyperparameters within the framework. Our results demonstrate SCAAR’s superior or competitive performance, showcasing its ability to effectively detect model retrieval needs and construct efficient retrieval queries for relevant knowledge about problem-solving in black-box scenarios. Our code is available on https://github.com/linqinhong/SAC.

## introduction large language models (llms) demonstrate impressive capabilities in various natural language processing tasks such as question-answering (qa), abstractive summarization, and machine translation @xcite . the emergence of prompt tuning and in-context learning @xcite @xcite facilitates llms to generate convincing and human-like responses, enabling llms to be integrated into aipowered intelligent assistants to support human 1 * corresponding author reasoning and decision-making processes @xcite . however, when confronting time-dependent and complex reasoning tasks, llms demonstrate reasoning inconsistencies and factual inaccuracies during response generation, which is referred to as the hallucination of llms @xcite . retrieval-augmented generation (rag) @xcite alleviates the hallucination issue by incorporating relevant knowledge into the context during the reasoning, enhancing the model's reasoning ability @xcite . the conventional rag framework implements a single retrieval operation upon a question and leverages the retrieved knowledge to assist the response generation @xcite . however it shows limited performance in longform generation and tasks requiring multi-step reasoning. this limitation stems from single-step retrieval, which only retrieves knowledge relevant to the initial question, neglecting the potential need during the iterative generation process. recent work focuses on the problem of when and what to retrieve during the generation process of llms. ircot @xcite triggers retrieval at the end of each sentence, and toolformer @xcite triggers retrieval when seeing named entities. meanwhile, adaptive retrieval has received increasing attention. the advantage of the adaptive retrieval lies in its ability to decide whether to trigger retrieval and determine the query for retrieval in accordance with status of the model. adaptive retrieval avoids unnecessary retrieval overhead and reduces the interference caused by wrong retrievals, improving the quality of the query and the retrieved content. recent work has explored different implementations of adaptive retrieval. flare @xcite uses the probability of the generated tokens to determine whether to retrieve and uses the model's current generation as the query, treating low-confidence tokens as hallucinations. dragin @xcite proposes an attention-based dynamic retrieval determination criterion assigns different significance values to content words and stopwords when building the query for retrieval. seakr @xcite proposes a retrieval determination criterion based on self-aware uncertainty. these methods effectively enhance rag, but they rely on models' hidden states and can't work with black-box models. moreover, the query formulation schemes of these methods revolve around the entire sentence or part of speech, which lacks dynamism. in this work, we focus on threshold adaptive weighting schemes that work in black-box scenarios and retrieval problem construction schemes based on these weights. following the definition in previous works @xcite @xcite , in a black-box scenario, we can only obtain the probabilities information corresponding to the tokens returned by the model, and cannot obtain other content. we propose semantic contribution-aware adaptive retrieval (scaar) as shown in figure 1 , which adopts an encoder model to compute the semantic contribution value of each token. the semantic contribution values are then leveraged to dynamically adjust the retrieval threshold and filter lowimportance words in the query for retrieval. we compared our scaar against white-box adaptive retrieval approaches and static retrieval approaches on four knowledge-intensive datasets. experimental results show that scaar achieves a performance comparable to adaptive white-box retrieval approaches, indicating that scaar can effectively capture the value of each token and determine 'when to retrieve' in black-box settings. additionally, the construction of contributionbased queries in scaar outperforms existing approaches, indicating that scaar can better determine 'what to retrieve'. the contributions of our paper are as follows: ## conclusion in this paper, we propose an adaptive rag framework incorporating a dynamic weight adjustment mechanism based on semantic contribution and a percentile-filtered query construction method for black-box scenarios. extensive experiments demonstrate the effectiveness of our framework. furthermore, ablation study results show the contributions of individual pipeline components to the enhanced performance.