Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus (2023)

Abstract: Large Language Models (LLMs) have gained significant popularity for their impressive performance across diverse fields. However, LLMs are prone to hallucinate untruthful or nonsensical outputs that fail to meet user expectations in many real-world applications. Existing works for detecting hallucinations in LLMs either rely on external knowledge for reference retrieval or require sampling multiple responses from the LLM for consistency verification, making these methods costly and inefficient. In this paper, we propose a novel reference-free, uncertainty-based method for detecting hallucinations in LLMs. Our approach imitates human focus in factuality checking from three aspects: 1) focus on the most informative and important keywords in the given text; 2) focus on the unreliable tokens in historical context which may lead to a cascade of hallucinations; and 3) focus on the token properties such as token type and token frequency. Experimental results on relevant datasets demonstrate the effectiveness of our proposed method, which achieves state-of-the-art performance across all the evaluation metrics and eliminates the need for additional information.

## introduction large language models (llms) have garnered substantial attention for their remarkable performance across various domains, such as finance @xcite , medicine @xcite , and education @xcite . these models exhibit an extraordinary capacity to generate natural language texts with high levels of coherence, fluency, and informativeness. nevertheless, a significant obstacle confronting llms is the risk of producing hallucinations @xcite , which refers to the generated text that is untruthful or nonsensical @xcite . hallucinations are a common occurrence in almost all applications @xcite , thus undermining the reliability and trustworthiness of llms, especially in scenarios where accuracy and veracity are essential. existing studies on hallucination detection for llms can be broadly divided into two categories: (i) retrieval-based methods @xcite , which evaluate the veracity of the given text against knowledge bases, and (ii) sampling-based methods @xcite , which assess information consistency between the evaluated text and additional sampled responses from the same llm. however, retrieval-based methods depend heavily on the external knowledge that may not always be accessible. and sampling-based methods require multiple responses from the llm for the information consistency verification or model training, making these methods costly and inefficient. to address the above issues, we propose a novel reference-free, uncertainty-based method to detect hallucinations in llms that are factually incorrect according to the world knowledge. the proposed method relies exclusively on the llm output text, thereby eliminating the need for additional resources like sampled responses from llm or external knowledge, as well as further training based on such data. our basic idea is to use a proxy language model for calculating the probability of each token in a given text. based on the calculated probability, we can compute the hallucination score at both token and sentence level using uncertainty-based metrics @xcite , where tokens and sentences with high hallucination scores are identified as candidate hallucinated contents. our assumption is that a powerful enough llm should assign a low probability to tokens that make up hallucinated information, since @xcite using a naive proxy model can hinder the focus on hallucination itself: 1) considering all tokens within the given text may introduce noise; 2) the hallucinated tokens might be assigned high probabilities (green bar) due to the overconfidence problem; 3) factual tokens may receive low probabilities (red bar) due to the underconfidence problem. (b) to strengthen such focus, we imitate how humans perform factuality checking from three aspects: 1) focus on the informative keywords; 2) focus on the preceding words by propagating the uncertainty through attention weights; 3) focus on the token properties by providing entity type before each named entity. the above method serves as a base framework, which can be limited by the inherent characteristics of the prediction probability from a naive proxy model. such a model functions as a general probability estimator, its predictions reflect syntactic, semantic and other sources of information, which can hinder the focus on hallucination itself as illustrated in figure 1a . firstly, the proxy model ignores varying degrees of informativeness, which may introduce noise. secondly, the probabilities assigned by lms are general and can deviate from factuality confidence in different contexts. for instance, the proxy model can be overconfident if the historical context contains surface tokens that are correlated with a hallucinated token, or the historical context features exposure bias @xcite due to the auto-regressive nature of generative process. one example is shown in figure 1a , where hallucinated tokens "2012 summer olympics" are assigned high probabilities. in addition, a proxy model can be underconfident if there are many plausible choices of topic directions to continue a context, despite that the hallucination involves different tokens within only one topic direction. one example is shown in figure 1a , where the factual token "1992" received a low probability due to the competitors like "west" and "coral". to strengthen the focus on hallucination, we take inspiration from human factuality checking, which can include at least three specific considerations as depicted in figure 1b: • focus on the informative keywords: the keywords that express salient information will be extracted for the calculation of hallucination scores at both sentence-level and passage-level. in summary, our primary contribution is that we introduce a novel reference-free, uncertainty-based approach for detecting hallucinations in llms. our approach does not require additional sampled responses or external knowledge bases, making it simple and cost-effective. experimental results demonstrate that our proposed method achieves state-of-the-art performance on the wikibio gpt-3 dataset across various models with different scales, and shows effectiveness in detecting hallucinations within summaries generated by small models. ## methods a proxy model is utilized in our method for uncertainty assessment in cases where token-level probabilities are inaccessible, such as gpt-3 @xcite . although previous work by @xcite has demonstrated the ineffective performance of using a proxy model, we attribute it to the uncertainty metrics employed. these metrics, such as the average entropy and average loss for all tokens in the sentence, are insufficiently aligned with human evaluation. we believe this issue stems from the inherent disparities in how models and humans perceive and assess information, thus limiting the capability of the uncertainty-based approach for hallucination detection. to mitigate this problem, we imitate how humans perform factuality checking from three aspects, which will be discussed in following sections. ## conclusion in this paper, we propose a reference-free, uncertainty-based method for detecting hallucinations in llms. the proposed method aims to imitate human factuality checking by considering three aspects: focus on informative keywords, focus on preceding words and focus on token properties. our experimental results empirically demonstrate the effectiveness of the proposed method for hallucination detection at both sentence and passage level, without requiring any external knowledge or training data. we have also analyzed how each of the three focus mechanisms impacts the overall performance when using different proxy models as the backbone. the results on xsumfaith and frank datasets further showcase the potential capability of the proposed method for detecting hallucinations produced by small models. we hope our work can contribute to the field of llm research and help improve the reliability and factuality of llms.