C trl A : Adaptive Retrieval-Augmented Generation via Inherent Control (2025)

Abstract: Retrieval-augmented generation (RAG) has emerged as a promising solution for mitigating hallucinations of large language models (LLMs) with retrieved external knowledge. Adaptive RAG enhances this approach by enabling dynamic retrieval during generation, activating retrieval only when the query exceeds LLM’s internal knowledge. Existing methods primarily focus on detecting LLM’s confidence via statistical uncertainty. Instead, we present the first attempts to solve adaptive RAG from a representation perspective and develop an inherent control-based framework, termed CtrlA. Specifically, we extract the features that represent the honesty and confidence directions of LLM and adopt them to control LLM behavior and guide retrieval timing decisions. We also design a simple yet effective query formulation strategy to support adaptive retrieval. Experiments show that CtrlA is superior to existing adaptive RAG methods on a diverse set of tasks. Honesty steering can effectively make LLMs more honest and confidence monitoring is a promising indicator of retrieval trigger.

## introduction retrieval-augmented generation (rag; @xcite ) has proven effective in mitigating hallucination by integrating external knowledge into llms. early efforts often employ single-round, indiscriminate retrieval, resulting in over-reliance on external knowledge and incomplete retrieval @xcite . to solve the issues, adaptive @xcite @xcite ) has emerged, which enables dynamic retrieval during generation, activating retrieval only when the query exceeds llm's internal knowledge @xcite . ♣ the first two authors contributed equally. ♠ corresponding authors. 1 https://github.com/hsliu-initial/ctrla the key challenges in arag involve determining what and when to retrieve @xcite @xcite . the design of what aspect depends on the construction of when aspect, making arag's primary focus the issues related to when aspect. for the when aspect, recent arags leverage the ability that llms are aware of their uncertainty @xcite @xcite , utilizing this characteristic to determine retrieval timing by assessing confidence level of their knowledge @xcite . they primarily focus on detecting uncertainty in the llm's outputs to signal retrieval, relying on factors such as output probabilities @xcite , entropy of output @xcite or internal states @xcite , or verbal feedback @xcite . from a statistical standpoint, uncertainty and confidence are conceptually equivalent, both reflect the degree of certainty in a model's predictions @xcite @xcite . thus, uncertainty can act as a proxy for confidence when determining retrieval timing. we revisit the assumptions underlying these uncertainty-based methods. first, they presume that llm's output aligns with its internal knowledge @xcite @xcite , i.e., llms can accurately reflect internal knowledge in outputs or they are honest. however, llms often navigate a trade-off between honesty and helpfulness, balancing between discerning its limitations and generating user-satisfied plausible content @xcite . when the output diverges from internal knowledge, indicating low honesty, they only detect intended output rather than internal knowledge. second, they equate uncertainty with llm's confidence, 2 which may not always apply to llm behavior. for instance, an llm may frequently respond with "i don't know" or "insufficient information," suggesting low uncertainty, yet retrieval should still occur. moreover, semantically equivalent answers can be expressed in various ways in free-form generation, which may lead to high uncertainty @xcite . however, retrieval is unnecessary in this scenario. based on this analysis, we emphasize that both honesty and confidence of llms are crucial for accurate retrieval timing. however, current arags struggle to address them due to the limitations of statistical uncertainty. we propose to solve arag from a representation perspective @xcite @xcite , developing an efficient and unified framework that seamlessly tackles the requirements of honesty and confidence. our core idea involves extracting features corresponding to honesty and confidence directions from llms and using them to control llm behavior and guide retrieval timing decisions simultaneously. we devise an inherent control-based adaptive rag (ctrla). to steer llm toward honesty and monitor its confidence, we extract features aligned with the directions of honesty and confidence within llm's representation space. by adjusting the honesty direction-a process we refer to as honesty steering-we can shift llm's representation space to promote more honest outputs. simultaneously, confidence is quantified by measuring the projection of current representation onto the confidence feature, a method we call confidence monitoring. honesty steering helps llm recognize its limitations and suppress the generation of fabricated, plausible information. confidence monitoring, in turn, enhances the precision of retrieval timing. experiments verify the effectiveness of ctrla, revealing that adjusting the directions of llm's internal states enhances its honesty, while confidence monitoring reliably signals when to trigger retrieval, optimizing the balance between retrieval and internal knowledge use. ## related work early rag efforts @xcite @xcite @xcite relied on single-round, indiscriminate retrieval, increasing computational costs and degrading performance @xcite . to address it, arag emerged, enabling dynamic retrieval during generation when the query exceeds llm's internal knowledge @xcite @xcite . previous implementations utilized static rules, such as prior sentences @xcite , sliding windows @xcite , and in-context learning @xcite @xcite . recent arags leverage llms' self-awareness of uncertainty to optimize retrieval timing by assessing confidence levels through internal states @xcite , likelihoods @xcite @xcite , or verbal feedback @xcite @xcite . this enhances retrieval timing and balances external and internal knowledge use. however, uncertaintybased arags face challenges with llm honesty and confidence, crucial for accurate retrieval timing. ctrla solves these issues from a representation perspective, enhancing control over honesty and confidence to improve the effectiveness. ## experiment setup datasets and evaluation. for short-form qa, we select popqa @xcite aqa @xcite . for long-form qa, we use asqa @xcite and biography generation (bio; min et al. 2023). for multi-hop qa, we follow su et al. (2024) to choose 2wiki-multihopqa (2wmqa; ho et al. 2020) and hot-potqa (hqa; yang et al. 2018). for short-form qa, we report the accuracy. for asqa, we report str-em, rouge-l (r-l; lin 2004), mauve (mau; pillutla et al. 2023), em, and f1. bio is evaluated by factscore (fs; min et al. 2023). for multi-hop qa, we report em and f1. we also evaluate 500 test samples (v04082024) of freshqa (vu et al., 2024) and report both relaxed and strict accuracy scores. more details in appendix b.2 and b.3. implementation and retrieval setup. we select the mistral-7b (jiang et al., 2023a) as the backbone of ctrla and adopt the greedy decoding strategy for all experiments. the λ for honesty steer is set to 0.3 and τ for confidence monitoring is set to 0.0. by default, we use bm25 and bge (xiao et al., 2024) as our retriever and use the 2018 english wikipedia corpus as the document source following jiang et al. (2023b) and asai et al. (2024). for popqa and bio, we follow self-rag (asai et al., 2024) to retrieve additional information from the web to mitigate coverage limitations in the wikipedia corpus. for the multihop qa task, we only use bm25 as the retriever. for freshqa, we only retrieve from the web to obtain supporting documents. more details in appendix b.4 and b.5. baselines. we compare ctrla with (1) singleround rag (sr-rag), which retrieves documents before generation; (2) fix-sentence rag (fs-rag; trivedi et al. 2023), which triggers retrieval every sentence and the previous sentence is used as query; (3) fix-length rag (fl-rag; ram et al. 2023), which triggers retrieval every n tokens and the previous token window is used as query; (4) query-decompose rag (qd-rag; press et al. 2023; khattab et al. 2023), which prompts llms to generate follow-up queries and trigger retrieval for each query; (5) adaptive rags: flare (jiang et al., 2023b), self-rag @xcite , dra-gin @xcite , seakr @xcite , rq-rag @xcite and qc-rag @xcite . for ( 1 )-( 4 ), we reimplement them under the same setting as ctrla. more details about the baselines are in appendix b.6. ## conclusion this paper introduces ctrla, a lightweight framework that optimizes retrieval timing in adaptive rag by leveraging representation features for honesty and confidence. ctrla regulates llm behavior, monitors internal states to assess retrieval needs, refines search queries, and handles refusal outputs. evaluations show it consistently outperforms baselines, underscoring the efficacy of its honesty-and confidence-driven approach.