SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented
  Generation
Authors: name: Zijun Yao; name: Weijian Qi; name: Liangming Pan; name: Shulin Cao; name: Linmei Hu; name: Weichuan Liu; name: Lei Hou; name: Juanzi Li
Published: 2024-06-27 14:38:33

Abstract: This paper introduces Self-aware Knowledge Retrieval (SeaKR), a novel
adaptive RAG model that extracts self-aware uncertainty of LLMs from their
internal states. SeaKR activates retrieval when the LLMs present high
self-aware uncertainty for generation. To effectively integrate retrieved
knowledge snippets, SeaKR re-ranks them based on LLM's self-aware uncertainty
to preserve the snippet that reduces their uncertainty to the utmost. To
facilitate solving complex tasks that require multiple retrievals, SeaKR
utilizes their self-aware uncertainty to choose among different reasoning
strategies. Our experiments on both complex and simple Question Answering
datasets show that SeaKR outperforms existing adaptive RAG methods. We release
our code at https://github.com/THU-KEG/SeaKR.