Hallucination Detection for Generative Large Language Models by B ayesian Sequential Estimation (2023)

Abstract: Large Language Models (LLMs) have made remarkable advancements in the field of natural language generation. However, the propensity of LLMs to generate inaccurate or non-factual content, termed “hallucinations”, remains a significant challenge. Current hallucination detection methods often necessitate the retrieval of great numbers of relevant evidence, thereby increasing response times. We introduce a unique framework that leverages statistical decision theory and Bayesian sequential analysis to optimize the trade-off between costs and benefits during the hallucination detection process. This approach does not require a predetermined number of observations. Instead, the analysis proceeds in a sequential manner, enabling an expeditious decision towards “belief” or “disbelief” through a stop-or-continue strategy. Extensive experiments reveal that this novel framework surpasses existing methods in both efficiency and precision of hallucination detection. Furthermore, it requires fewer retrieval steps on average, thus decreasing response times.

## introduction in the era of information overload and the proliferation of misleading or false information, automatic fact checking has become an essential tool for verifying the veracity of claims and combating misinformation. large language models (llms), such as gpt-4 (openai, 2023),palm @xcite and llama @xcite , have made significant advancements in the field of natural language generation (nlg). however, the inherent tendency of llms to generate inaccurate or non-factual content, commonly referred to as "hallucinations" @xcite , continues to present a significant challenge. one previous work utilizes the sampled-based approach for detecting hallucination (manakul 1 our code is available at https://github.com/ xhwang22/hallucinationdetection . ## method in this paper, we propose a framework for detecting hallucinations of llms by leveraging external evidence. given a claim generated by the llms, we first decompose it into subclaims that contain basic knowledge and proceed to evaluate the veracity of each subclaim individually. subsequently, we employ a search engine to retrieve web documents for each subclaim as external evidence. rather than predefining the number of documents to retrieve, we adopt a dynamic retrieval process that involves retrieving one document at one step and determining the need for additional retrieval based on the content of the retrieved document. then, we utilize a specific classification model to compute the entailment score between each document and subclaim which represents the extent to how well the evidence supports a given claim. the entailment score is processed into features, and a naive bayes classifier (nbc) is applied to calculate the probability of the subclaim's veracity based on these features. next, we adopt a stop-or-continue strategy, employing bayesian risk decision to choose among three options: (1) stopping the retrieval process and classifying the subclaim as faculty, (2) stopping the retrieval process and classifying the subclaim as a hallucination, (3) or continuing to retrieve additional documents and make a choice in the next step. after making the decision to stop the retrieval process and determine the veracity or reaching the maximum retrieval times, we obtain the probability that the subclaim is faculty. finally, by aggregating the probabilities of the subclaims' veracity, we evaluate the overall veracity of the original claim and assess its potential as a hallucination. in the following subsections, we discuss in more detail our method. in section 3.1, we demonstrate the process of claim decomposition. in section 3.2, we describe details of using a search engine to retrieve web documents. section 3.3 explains how the entailment score is calculated and processed into features required by the naive bayes classifier (nbc). in section 3.4, we provide a specific description of the stop-or-continue strategy's details. finally, in section 3.5, we elaborate on how the veracity probabilities of subclaims are aggregated. ## experiments we conducted experiments on the hallucination detection classification task. first, we compared our framework with selfcheckgpt to evaluate the effectiveness of our approach in hallucination detection at both the sentence-level and passage-level. next, we performed a comparative experiment using a fixed number of retrieved documents as external evidence to validate the efficiency of our framework in improving retrieval efficiency and reducing the number of external evidence. finally, we conducted ablation study to evaluate the necessity of the claim decomposition step. ## conclusions in this study, we propose a framework that utilizes bayesian sequential analysis for detecting hallucinations in large language models (llms). we consider the retrieved documents as a sequence and employ a stop-or-continue strategy to make in-formed decisions on whether to retrieve additional information for a given case. by applying bayesian sequential analysis, our framework achieves competitive results on annotated gpt-3 responses, surpassing sample-based methods and retrieval strategies with a fixed number of search documents. additionally, we validate the practical applicability of our framework by conducting searches for relevant documents on the web, demonstrating the feasibility of our approach in corroborating the claims made by llms with real-world information.