Long-form Hallucination Detection with Self-elicitation (2025)

Abstract: While Large Language Models (LLMs) have exhibited impressive performance in generating long-form content, they frequently present a hazard of producing factual inaccuracies or hallucinations. An effective strategy to mitigate this hazard is to leverage off-the-shelf LLMs to detect hallucinations after the generation. The primary challenge resides in the comprehensive elicitation of the intrinsic knowledge acquired during their pre-training phase. However, existing methods that employ multi-step reasoning chains predominantly fall short of addressing this issue. Moreover, since existing methods for hallucination detection tend to decompose text into isolated statements, they are unable to understand the contextual semantic relations in long-form content. In this paper, we study a novel concept, self-elicitation, to leverage self-generated thoughts derived from prior statements as catalysts to elicit the expression of intrinsic knowledge and understand contextual semantics. We present a framework, SelfElicit, to integrate self-elicitation with graph structures to effectively organize the elicited knowledge and facilitate factual evaluations. Extensive experiments on five datasets in various domains demonstrate the effectiveness of self-elicitation and the superiority of our proposed method.

## introduction large language models (llms), pre-trained on massive text corpora and fine-tuned to follow human instructions @xcite @xcite , have shown remarkable performance to generate longform content that consists of multiple coherent sentences @xcite . however, there remains a concern regarding their tendency to generate factual (external) hallucinations @xcite , producing sentences that appear plausible but are factually unsupported @xcite . this * corresponding author: haishuai wang issue undermines their reliability in real-world scenarios where factually accurate responses are expected @xcite . for example, a modelgenerated non-factual statement, "gliclazide can be taken at any time of the day, regardless of whether it is on an empty stomach or after meals", could mislead patients into taking medication at inappropriate times, as the medication is actually recommended to be taken with food @xcite . an important strategy to alleviate hallucinations is to detect hallucinations after generation @xcite @xcite . the detection of hallucinations in large language models has been approached through various methods, such as retrieval-based techniques @xcite @xcite @xcite and probe-based approaches @xcite @xcite . however, these methods rely heavily on external databases or training corpora, which may not always be available. consequently, many studies have shifted toward leveraging the intrinsic capabilities of pre-trained, off-the-shelf llms. the central challenge lies in effectively eliciting models' internal knowledge for hallucination detection. for example, @xcite and @xcite prompt the models to verbally express knowledge through chain-of-thought reasoning. @xcite ; @xcite ; @xcite prompt the model to generate statements from various perspectives and quantify their semantic consistencies. other works @xcite @xcite ask the model to answer verification questions corresponding to the factoids. while theoretically insightful, these multi-step elicitation procedures often encounter limitations in practice. they either tend to express knowledge with low relevance or are prone to accumulated inaccuracies when extracting triples or constructing verification questions, therefore requiring labor-intensive, meticulous design of prompts or samples to ensure accuracy and performance. additionally, the long-form content generated by llms consists of multiple semantically related sentences that exhibit logical relationships @xcite , such as coherence, comparison, and causality. for instance, the preceding statement, "gliclazide is an oral hypoglycemic medication" and the subsequent statement, "it is suitable for adult type 2 diabetes patients whose blood sugar cannot be adequately controlled by diet alone", demonstrate logical coherence and progression. the first statement introduces the category and function of the medication, while the second statement further elaborates on its specific medical application. however, existing long-form hallucination detection methods @xcite @xcite generally decompose the long-form text into isolated statements for individual fact-checking (figure 1(a) ), overlooking the contextual semantic relationships and limiting their long-form understanding. incorporating contextual information into models (figure 1(b) ) can provide a more natural, coherent chain of meanings, enhancing the understanding and evaluation of successive statements. in this work, we present selfelicit, an integrated framework designed to effectively elicit a model's intrinsic knowledge and utilize semantic relations for hallucination detection in long-form content. specifically, it evaluates the factuality of each statement and elicits relevant knowledge through reflection conditioned on the evaluation. these elicited thoughts are then incorporated into subsequent evaluations as contextual information (figure 1(c )), forming an iterative process where the evaluation and elicitation interact, namely self-elicitation. to mitigate hallucinations arising during the self-elicit process, we integrate a knowledge hypergraph with self-elicitation to facilitate knowledge retention, deduplication, and resolution of inconsistencies. extensive experiments on five datasets in various domains demonstrate that self-elicitation can act as an effective catalyst to improve both the factuality and diversity of models' knowledge expression and our framework outperforms existing methods for long-form hallucination detection. to sum up, our contributions include: a knowledge hypergraph is integrated to organize the elicited knowledge and alleviate hallucination snowballing. ## methodology figure 2 provides an overview of our framework. given long-form content to be fact-checked, we first extract important entities and statements corresponding to factoids to be checked. we then present the framework along with a knowledge hypergraph to iteratively evaluate the factuality of each statement via ①retrieving contextual information from the graph, ②evaluating the factuality based on calibration, ③reflecting for intrinsic knowledge elicitation, ④resolving inconsistencies suggesting induced hallucinations and ⑤updating the graph to retain the elicited thoughts. ## conclusion in this paper, we have investigated the task of detecting hallucinations from long-form content. ex-isting methods predominantly fall short of comprehensively elicitating the intrinsic knowledge of models or overlook the semantic relationships within long-form content. to address these issues, we present a novel framework, selfelicit, which uses self-generated thoughts from prior statements to elicit the models' intrinsic knowledge and synergize self-elicitation and contextual understanding in a unified diagram. extensive experiments on five datasets with various domains with modern large language models have shown the effectiveness and superiority of the proposed framework.