CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Control

Authors: Liu Huanshuo, Hao Zhang, Zhijiang Guo, Jing Wang, Kuicai Dong, Xiangyang Li, Yi Quan Lee, Cong Zhang, Yong Liu
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Retrieval-Augmented Generation, Large Language Model

Abstract:
Retrieval-augmented generation (RAG) has emerged as a promising solution for mitigating hallucinations of large language models (LLMs) with retrieved external knowledge. Adaptive RAG enhances this approach by enabling dynamic retrieval during generation, activating retrieval only when the query exceeds LLM's internal knowledge. Existing methods primarily focus on detecting LLM's confidence via statistical uncertainty. Instead, we present the first attempts to solve adaptive RAG from a representation perspective and develop an inherent control-based framework, termed CtrlA. Specifically, we extract the features that represent the honesty and confidence directions of LLM and adopt them to control LLM behavior and guide retrieval timing decisions. We also design a simple yet effective query formulation strategy to support adaptive retrieval. Experiments show that CtrlA is superior to existing adaptive RAG methods on a diverse set of tasks, the honesty steering can effectively make LLMs more honest and confidence monitoring is a promising indicator of retrieval trigger. Our anonymous codes are submitted with the paper and will be publicly available.

Decision: reject
Reviewer scores: 3, 5, 5, 5

Reviews and discussion:
<thread>
creation date:
1730070104680

thread_type:
official_review

<thread_object>
creator: reviewer_nbtc

<content>
summary:
This paper approaches the problem of adaptive retrieval for RAG, in which “when and what to retrieve” is dynamically determined during generation. Motivated by the limitation of existing uncertainty-based approaches, this paper proposed to consider both “honesty” and “confidence” as important factors to guide adaptive retrieval. Three techniques are introduced: (1) shifting LLM representation to the honest direction, (2) projecting the representation onto a confidence feature and (3) a revised query generation technique based on the already-generated content. Experiments using several LLMs on short-form and long-form generation tasks verify the effectiveness of the proposed approach compared to existing adaptive retrieval methods.

strengths:
* Although the adaptive retrieval framework has been introduced for several years, it is still an open problem for how to build strong adaptive retrieval strategies catered to modern LLM. The research question proposed by the paper is thus timely. 

* The empirical evaluation is thorough. A number of LLMs are evaluated with the proposed method. The authors consider several major benchmarks across different generation formats. The empirical results can serve as good reference points for future works on adaptive retrieval. 

* The proposed method is intuitive and can outperform well-known existing baselines such as FLARE and DRAGIN.

weaknesses:
My major concern of this paper is that its proposal and contribution are not aligned with the key problem the paper is motivated by.

* This paper contains many moving parts, but most of them are (1) disconnected and (2) irrelevant to the key challenge of adaptive retrieval. The core motivation of this paper is that uncertainty-based adaptive retrieval is insufficient for determining when retrieval should be triggered. However, the proposed honesty steering and query formulation methods aim to generate higher quality RAG responses instead of solving the problem of adaptive retrieval. In addition, the proposed retrieval triggering criteria is still fundamentally an uncertainty-based method. 

* The proposed method does not seem to solve the issue in the motivating example, where adaptive retrieval fails to trigger when the model confidently says “I don’t know”. From my understanding, the proposed method will also fail in this case.

* Algorithm 1 is incomplete. Specifically, an outer loop that calls algorithm 1 is required. The input to the outer loop is the query and the document, and the output is the final generated response, instead of a single output segment. 

* More citations to the adaptive RAG literature are required in section 3. Specifically, 3.1 and 3.2.4 should cite FLARE [1], as the approaches are similar. In addition, decoding-stage feature monitoring works such as [2] and [3] should be cited in the confidence monitoring section, 

[1] Active Retrieval Augmented Generation. Jiang et al., EMNLP 2023. 

[2] Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation. Wu et al., EMNLP 2024.

[3] Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps. Chuang et al., EMNLP 2024.

questions:
(please also see weaknesses)

* What is this paper’s definition of honesty? Can you further explain how it connects with adaptive retrieval in the proposed framework? 

* I would like to see a full ablation on the proposed techniques, where only one technique is ablated at a time and compared to the performance of all the techniques together.

* How do you determine the hyperparameters of the proposed method and the baselines (such as the retrieval threshold)? Do you always use an in-domain validation dataset to tune the parameters? Are the hyperparameters kept the same across all test sets?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730293735926

thread_type:
official_review

<thread_object>
creator: reviewer_kzpg

<content>
summary:
This paper tackles an issue in Adaptive Retrieval-Augmented Generation by focusing on retrieval timing to reduce hallucinations, and using interpretable methods to analyze honesty and confidence. The method shows promising experimental results. However, the motivation behind some methods feels unclear, with Honesty Steering and Search Query Formulation seeming unrelated to the core goal and lacking novelty. Additionally, the experimental setup lacks clarity on segment definitions. The reliance on an outdated linear representation assumption raises concerns about applicability in modern LLMs.

strengths:
This paper proposes to address Adaptive Retrieval-Augmented Generation by focusing on retrieval timing to reduce hallucinations. It takes an interpretable approach by analyzing internal mechanisms like honesty and confidence, providing clearer insights into model behavior. Experimental results show meaningful improvements, suggesting that the proposed method contributes effectively to retrieval-augmented generation tasks.

weaknesses:
1.	The paper doesn’t explain why the methods are crucial for Adaptive Retrieval-Augmented Generation (ARAG). The section Confidence Monitoring as Retrieval Trigger seems directly relevant to ARAG, while the section Honesty Steering and Search Query Formulation seems to be unrelated to the main goal. The Honesty Steering part has already been introduced in similar work[1], which makes the technical contribution of the paper incremental. The methods aren’t presented in a clear, motivating way, making the paper feel disjointed.

2.	In the experiments, it’s unclear how the "segments" are defined, leaving readers without a solid understanding of the setup. Also, the ablation study misses the marks that specifically show if Confidence Monitoring is driving the improvements. Without clear evidence, it’s hard to conclude that the approach tackles the core ARAG issues.

3.	The paper builds on a linear representation assumption, referencing work from Templeton et al. (2024) which is based on early RNN-based findings[2]. Given the evolution of model architectures in the LLM era, this assumption seems to be oversimplified.


[1] Li, Kenneth, et al. "Inference-time intervention: Eliciting truthful answers from a language model." Advances in Neural Information Processing Systems 36 (2024).
[2] Linguistic regularities in continuous space word representations." Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies. 2013.

questions:
See the above comments.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730337791573

thread_type:
official_review

<thread_object>
creator: reviewer_s1gu

<content>
summary:
This paper proposes CtrlA, an adaptive RAG framework to extract honesty and confidence features and use them to steer the generation. CtrlA uses contrastive prompts to instruct the LLM with corresponding behaviors (honest/dishonest/confident/unconfident), and then extracts the hidden representations of the whole statement to form the honesty/confidence vectors with PCA. It uses a linear combination to guide the generation towards the honest/confident direction. Experimental results and analyses demonstrate that CtrlA performs well in reasoning tasks.

strengths:
- The proposed method is interesting and effective.

- The paper is well-written and easy to follow.

weaknesses:
- The method requires accessing the internal structure of LLMs, which may not be available for some competitive commercial LLMs such as o1 and limits its applicability. 

- The approach relies heavily on PCA and detailed layer-wise feature extraction, potentially leading to high computational costs and processing time, especially with larger models. The principle component selection process is also not discussed since the first principle component might not represent the required direction.

- The method involves intricate processes such as feature extraction, honesty steering, and confidence monitoring, which require substantial computational resources and expertise to implement. No token expenses are reported.

- The query formulation strategies lack clarity in handling ambiguous or noisy outputs. The methods for masking unconfident tokens and refining search queries are insufficiently detailed, making it challenging to understand how they ensure the retrieval of relevant documents. Additionally, there's a potential risk of losing important context when masking tokens, which could affect retrieval accuracy.

- Modifying the representation space can inherently harm the quality of generated content, which successfully answers the question but fails to produce a plausible and fluent sentence, as illustrated in Fig 3b.

- The experimental results are not comprehensively discussed. Fig 2 and 3 show different effects of $\lambda$ on accuracy, but no further discussion is provided for such contradiction except the dataset category.

questions:
- Why does honesty steering always improve accuracy in closed-domain QA? Generally, the result should be similar to open-domain tasks.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730441606102

thread_type:
official_review

<thread_object>
creator: reviewer_ftss

<content>
summary:
This paper proposes an adaptive RAG framework, CTRLA, which focuses on dynamically activating retrieval to supplement LLMs only when needed. Instead of relying solely on statistical confidence measurements, the paper introduces a representation-based approach to extract "honesty" and "confidence" features that guide the retrieval process. This framework targets issues of hallucination and efficiency in RAG models by enabling more accurate timing of retrievals. The experiments demonstrate that CTRLA outperforms other adaptive RAG methods, showcasing its effectiveness in retrieval timing and content accuracy.

strengths:
1. CTRLA introduces a method for adaptive retrieval, focusing on honesty and confidence features within the LLM's representation space, a departure from conventional uncertainty-based approaches.
2. The experiments reveal CTRLA’s effectiveness in generating accurate and relevant responses, with enhanced performance in both short-form and long-form QA tasks.
3. The approach optimizes retrieval timing, resulting in fewer unnecessary retrievals.
4. The CTRLA framework is lightweight and does not heavily depend on external fine-tuning, potentially making it applicable across various LLMs and use cases.
5. The figures in the paper are very exquisite.

weaknesses:
1. The experiments cover several QA tasks, and additional datasets that may not be memorized by the LLM could have further validated the framework's robustness, especially across rapidly changing topic datasets.
2. There are many similar works described as "knowledge conflicts", but the author seems to miss them both for introducing and as baselines, such as 2.1 Xie, Jian, et al. "Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts." arXiv preprint arXiv:2305.13300 (2023). 2.2 Jin, Zhuoran, et al. "Tug-of-war between knowledge: Exploring and resolving knowledge conflicts in retrieval-augmented language models." arXiv preprint arXiv:2402.14409 (2024).
3. Does this multi-step pipeline approach increase the computational burden? What is the overall complexity?
4. Since CTRLA relies on inherent LLM honesty and confidence features, the effectiveness of the framework may vary significantly depending on the LLM's underlying architecture.
5. This paper is based on a strong assumption that "honesty" and "confidence" features can be linearly separated within the LLM's latent space. Does the independence between the two still hold in complex contexts?
6. The extraction of "honesty" and "confidence" is based on the parameterized knowledge of the model itself. How to ensure the fairness of this process? Because this is the basis for the following work.

questions:
See above
</content>
</thread_object>

</thread>
<thread>
creation date:
1732189479881

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
Dear Reviewers:

Thank you for your thorough and constructive feedback on our manuscript. We sincerely appreciate your time and effort in reviewing our work. We have carefully addressed all the comments and suggestions, which have significantly improved the quality of our paper.

Below is the summary of the major revisions we have made during the rebuttal:

1. Extended Experimental Analysis
   - Conducted and analyzed additional experiments on Honesty Steering with different coefficients using the TruthfulQA dataset (Section 5.2)
   - Enhanced our findings with detailed analysis (lines 374-375, 385-386, 402-404, 429-431)
   - Revised visual representation of results (Figure 2)
2. Enhanced Theoretical Framework and Algorithm Description
   - Added explanations and references about output segment processing (Section 3.1, line 137,  line 142)
   - Improved presentation of Algorithm 1 (Section 3.2.4 lines 245-262) and Algorithm 3 (Appendix B 3.2 lines 1089-1112 )
   - Clarified the role of confidence monitoring (Section 3.2.3, lines 212-213)
   - Added comprehensive explanation for handling LLM's refusal responses (Section 3.2.3, lines 220-225; Section 3.3, line 280)
3. Strengthened Related Work and Connections
   - Expanded the related work section on Knowledge Conflicts (Appendix B.1, lines 934-956)
   - Elaborated on the relationship and distinctions between Knowledge Conflicts and Adaptive RAG

These revisions have been carefully implemented to address the reviewers' concerns while maintaining the paper's clarity and scientific rigor. We believe these changes have substantially improved the manuscript's quality and hope they meet your expectations.

Thank you again for your valuable feedback. We look forward to your reply and are happy to answer any further questions.

Best,

Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
In this paper, the authors propose an adaptive RAG framework to address hallucination and efficiency issues. Reviewers found the idea interesting and appreciated the lightweight nature of the framework. However, they raised questions about its novelty and whether the underlying assumption—that confidence and honesty can be linearly separated in the latent space—is universally applicable, particularly for complex tasks and when using closed-source LLMs. Concerns were also expressed regarding the computational complexity of the proposed approach.