Query-Level Uncertainty in Large Language Models
Authors: name: Lihu Chen, name: Gaël Varoquaux

It is important for Large Language Models to be aware of the boundary of
their knowledge, the mechanism of identifying known and unknown queries. This
type of awareness can help models perform adaptive inference, such as invoking
RAG, engaging in slow and deep thinking, or adopting the abstention mechanism,
which is beneficial to the development of efficient and trustworthy AI. In this
work, we propose a method to detect knowledge boundaries via Query-Level
Uncertainty, which aims to determine if the model is able to address a given
query without generating any tokens. To this end, we introduce a novel and
training-free method called Internal Confidence, which leverages
self-evaluations across layers and tokens. Empirical results on both factual QA
and mathematical reasoning tasks demonstrate that our internal confidence can
outperform several baselines. Furthermore, we showcase that our proposed method
can be used for efficient RAG and model cascading, which is able to reduce
inference costs while maintaining performance.