Adaptive RAG Guided by Self-Evaluation
Authors: Zihan Zhu, Minqi Xiang, Ping Nie, Zhonghua Yu


Reviewer paper summaries: This paper investigates the self-evaluation capabilities of LLMs in deciding the need for RAG across various sizes and architectures, shedding light on the efficacy of model self-evaluation in facilitating autonomous Adaptive Retrieval-augmented Generation.
Summary of strengths: 1. Conduct a comprehensive evaluation of the intrinsic self-adaptive capabilities of  LLMs for retrieval-augmented generation.

2. The proposed adaptive accuracy metric offers a robust evaluation for ARAG by accounting for both answer correctness and generation efficiency.

3. Writing is clear and easy to follow.
Summary of weaknesses: 1. Limited technical contribution. Using the self-evaluation capabilities of LLMs to perform adaptive-RAG is not novel, there have been many related studies explored this, such as [1, 2]. Therefore, the technical contribution of this paper is limited.

2. Weak baselines. There are only two basic baselines: fixed retrieval@1 and without retrieval. It is necessary to compare with the popular adaptive RAG frameworks such as Self-RAG to show the effectiveness of the proposed method.

3. Limited improvement. Table 1 shows that the improvement brought by the proposed method is limited, and in most cases, the most basic baseline retrieval@1 is even worse.

[1] Are Large Language Models Good at Utility Judgments?

[2] When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation
Comments, suggestions and typos: Add the introduction to related work on LLMs’ self-verification, especially their applications in adaptive RAG [1,2]

[1] Are Large Language Models Good at Utility Judgments?

[2] When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation
Overall assessment scores: 2.5