Simple Factuality Probes Detect Hallucinations in Long-Form Natural Language Generation (2025)

Abstract: Large language models (LLMs) often mislead users with confident hallucinations. Current approaches to detect hallucination require many samples from the LLM generator, which is computationally infeasible as frontier model sizes and generation lengths continue to grow. We present a remarkably simple baseline for detecting hallucinations in long-form LLM generations, with performance comparable to expensive multi-sample approaches while drawing only a single sample from the LLM generator. Our key finding is that LLM hidden states are highly predictive of factuality in long-form natural language generation and that this information can be efficiently extracted at inference time using a lightweight probe. We benchmark a variety of long-form hallucination detection methods across open-weight models up to 405B parameters and demonstrate that our approach achieves competitive performance with up to 100x fewer FLOPs. Furthermore, our probes generalize to out-of-distribution model outputs, evaluated using hidden states of smaller open-source models. Our results demonstrate the promise of hidden state probes in detecting long-form LLM hallucinations.

## introduction modern large language model (llms) applications increasingly rely on frontier models producing tens of thousands of tokens: users generate codebase-scale edits with specialized coding assistants (openai, 2025a), create multi-page technical reports with advanced models like @xcite , or employ reasoning models such as @xcite and gemini-2.5-pro (deepmind, 2025) as proof assistants. while these models generally produce accurate content, they remain susceptible to hallucinations, particularly as context lengths and conversation turns increase. for users to safely leverage these frontier capabilities while mitigating potential harms, mod-els could accompany their generations with finegrained factuality scores, enabling users to identify which portions of the output are trustworthy and which require verification. detecting hallucinations in long-form generations, however, presents unique challenges compared to short-form settings. unlike evaluating the likelihood of a single short phrase being false, long-form texts contain numerous interdependent claims spanning multiple paragraphs or documents. current approaches to long-form hallucination detection rely primarily on sampling-based methods, which assess claim confidence through semantic entropy @xcite , self-consistency @xcite , or graph uncertainty @xcite . while effective for paragraph-length outputs, these approaches become computationally prohibitive for frontier models with hundreds of billions of parameters generating tens of thousands of tokens, requiring multiple generation calls to increasingly expensive reasoning models. a promising alternative lies in leveraging an llm's internal representations of factuality and confidence. recent work has demonstrated that llm activations-specifically, their per-token hidden states-encode information about answer correctness in short-form settings @xcite . however, no prior research has systematically investigated whether this information could be extracted for long-form hallucination detection, to assign finegrained confidence scores to interdependent claims in extended generations. we address this crucial gap by exploring whether hidden states capture hallucination signals in longform generations and whether lightweight probes trained on these states can enable efficient, claimlevel hallucination detection. specifically, we train linear and tree-based probes to map an llm's hidden state representation of an atomic claim directly to its likelihood of hallucination. these factuality probes are inexpensive to train, interpretable by design, and enable a pipeline for detecting long-form hallucinations with just a single generation call to the llm-dramatically reducing computational overhead compared to sampling-based methods. our empirical evaluations demonstrate that our approach achieves comparable hallucination detection performance (measured by auroc) to established sampling-based baselines while reducing computational resources by two orders of magnitude. moreover, we show robust scalability across models from 3b to 405b parameters, with detection performance scaling log-linearly in model size. remarkably, probes trained on smaller open models retain strong performance when assessing outputs from significantly larger, proprietary models, despite the substantial disparity in model sizes. to summarize, our key contributions are as follows: code to reproduce our results can be found at: https://github.com/jthh/fact-probe . ## fine-grained hallucination detection with factuality probes frontier llms produce tens of thousands of tokens per output in applications such as editing codebases, writing research reports, and answering technical questions @xcite . to provide users with a fine-grained understanding of the reliability of such long-form generations, we aim to accurately report claim-level scores of factuality. unlike prior approaches drawing many samples from the generator llm, we seek an efficient approach that scales to modern frontier models. the core empirical finding that enables our approach is that llm hidden states encode rich rep-resentations of claim-level factuality. leveraging this, we propose an approach to report fine-grained factuality scores at inference time by applying efficient probes on llm hidden states. we refer to these probes as factuality probes. representing fine-grained confidence scores in long-form generations. our high-level procedure for providing fine-grained factuality scores at inference time is shown in figure 1 . first, the generator llm generates a single long-form generation. next, we apply an (often smaller) auxiliary lm to decompose the long-form generation into atomic claims, and to associate each claim to a token span in the long-form generation. then, we can obtain estimates of the factuality of each atomic claim using a factuality probe, and attribute this factuality to the corresponding span, providing users with a visual representation of confidence as in the green and red paragraph in figure 1 . ## experiment setup let k denote the sample size for sampling-based methods. we define the model producing generations as π gen and the auxiliary model used for claim breakdown, filtering, and possibly entailment assessment as π aux . for evaluation, we sample a minimum of 25 topics from the longfact objects dataset @xcite for training our factuality probes and 30 entities from the wikipedia-based dataset used by @xcite for testing all methods. detailed sample statistics are provided in ## experiment we evaluate factuality prediction methods on several π gen including the llama 3 series models (llama3.2-3b, llama3.1-8b, llama3.1-70b, llama3.1-405b) and gemma2-9b, and utilize gpt-4o-mini as π aux throughout. we choose logistic regression (lr) and xgboost as the candidate classifiers f in our fact probing method. to test method robustness under data domain shift, we train probes on longfact labels and evaluate them on factscore labels. additionally, we test robustness to out-of-distribution (ood) models by applying probes trained on the activations of π small on the generations of π gen or π aux , e.g., closed-source model generations @xcite . we train probes on the model's hidden states to predict fact labels, categorizing them as either supported or not supported. the inputs are the hidden representations of a single token-either the first token (ft), last token (lt), or second-last token (slt) of a single hidden layer-or the concatenated hidden representations of 5-layer groups. all claims, regardless of topic, are pooled together, and the probe is trained with 3-fold stratified cross-validation. overall, the statistics for training and testing data are presented in ## conclusion we introduce factuality probes, a method that leverages llm hidden representations to efficiently assess the factuality of generated claims. our lightweight linear probes achieve comparable or superior performance to current approaches while requiring up to 100× fewer computational resources, making real-time factuality assessment practical. these probes generalize effectively across both subject domains and model architectures, even evaluating claims from closed-source models when trained on open-weight architectures. efficient factuality assessment methods like ours will be essential for responsible ai deployment.