When Do LLM s Need Retrieval Augmentation? Mitigating LLM s’ Overconfidence Helps Retrieval Augmentation (2024)

Abstract: Large Language Models (LLMs) have been found to have difficulty knowing they do not possess certain knowledge and tend to provide specious answers in such cases. Retrieval Augmentation (RA) has been extensively studied to mitigate LLMs’ hallucinations. However, due to the extra overhead and unassured quality of retrieval, it may not be optimal to conduct RA all the time. A straightforward idea is to only conduct retrieval when LLMs are uncertain about a question. This motivates us to enhance the LLMs’ ability to perceive their knowledge boundaries to help RA. In this paper, we first quantitatively measure LLMs’ such ability and confirm their overconfidence. Then, we study how LLMs’ certainty about a question correlates with their dependence on external retrieved information. We propose several methods to enhance LLMs’ perception of knowledge boundaries and show that they are effective in reducing overconfidence. Additionally, equipped with these methods, LLMs can achieve comparable or even better performance of RA with much fewer retrieval calls.

## introduction recently, large language models (llms) such as chatgpt have demonstrated remarkable performance across various nlp tasks @xcite @xcite @xcite , sometimes even outperforming humans. however, unlike humans, they have been found to have difficulty perceiving their factual knowledge boundaries, i.e., knowing what they know and what they do not know @xcite . when llms cannot answer a factual question, they should acknowledge it instead of providing a specious answer, especially in safety-critical fields like healthcare. ## perception of factual knowledge boundaries in llms in this section, we select a broad range of representative llms and use the vanilla prompt (see figure 3 in appendix §a.1) to test their qa performance and the perceptual level of factual knowledge boundaries. instead of indirectly characterizing the llms' perception of their knowledge boundaries as done by @xcite , we measure this by precise metrics and show the degree of overconfidence and conservativeness. we can observe the overall results in ## correlation between certainty and reliance on external information under retrieval augmentation, we need to know when llms show uncertainty to a question, whether they will leverage the provided external information. in this section, we investigate whether llms tend to rely on the documents when expressing uncertainty and how the models' confidence levels affect their reliance. ## alignment enhancement as discovered in section §4, the poor perception of knowledge boundaries in llms is mainly caused by their overconfidence. therefore, we enhance the perception of knowledge boundaries by mitigating overconfidence. this can be done from two perspectives: urging llms to be prudent and improving their ability to provide correct answers. ## adaptive retrieval augmentation our work focuses on determining when to conduct retrieval rather than triggering retrieval all the time and enhancing the ability of llms to leverage a document of unknown quality. in this section, we introduce the methods proposed in section §6 to adaptive retrieval augmentation. ## conclusion in this paper, we explore the direction of effective and efficient adaptive retrieval augmentation by enhancing llms' perception of their knowledge boundary. first, we propose several metrics to quantitatively measure llms' perception of knowledge boundaries and find that overconfidence is the primary reason for the unsatisfactory perception of knowledge boundaries. to see whether llms' certainty has an impact on their reliance on external knowledge, we investigate and find that the more llms are uncertain about their internal knowledge, the more they rely on external knowledge. we further probe to enhance the llms' perception from two perspectives and find that equipped with these methods, llms can achieve comparable or even better performance of retrieval augmentation with much fewer retrieval calls.