Steer LLM Latents for Hallucination Detection
Authors: Seongheon Park
Xuefeng Du
Min-Hsuan Yeh
Haobo Wang
Yixuan Li
Venue: ICML 2025 poster (2025)
Keywords: hallucination detection
Primary area: deep_learning->large_language_models

Abstract: Hallucinations in LLMs pose a significant concern to their safe deployment in real-world applications. Recent approaches have leveraged the latent space of LLMs for hallucination detection, but their embeddings, optimized for linguistic coherence rather than factual accuracy, often fail to clearly separate truthful and hallucinated content.
To this end, we propose the **T**ruthfulness **S**eparator **V**ector (**TSV**), a lightweight and flexible steering vector that reshapes the LLM’s representation space during inference to enhance the separation between truthful and hallucinated outputs, without altering model parameters.
Our two-stage framework first trains TSV on a small set of labeled exemplars to form compact and well-separated clusters.
It then augments the exemplar set with unlabeled LLM generations, employing an optimal transport-based algorithm for pseudo-labeling combined with a confidence-based filtering process.
Extensive experiments demonstrate that TSV achieves state-of-the-art performance with minimal labeled data, exhibiting strong generalization across datasets and providing a practical solution for real-world LLM applications.

Steer LLM Latents for Hallucination Detection
Seongheon Park Xuefeng Du Min-Hsuan Yeh
Haobo Wang²
Yixuan Li"
Abstract
(a) Pre-trained embeddings
(b) Steered embeddings by TSV
Hallucinations in LLMs pose a significant con- cern to their safe deployment real-world appli- cations. Recent approaches have leveraged the latent space of LLMs for hallucination detection, but their embeddings, optimized for linguistic co- herence rather than factual accuracy, often fail to clearly separate truthful and hallucinated con- tent. To this end, we propose the Truthfulness Separator Vector (TSV), a lightweight and flexi- ble steering vector that reshapes the L.LM's rep- resentation space during inference enhance the separation between truthful and hallucinated out- puts, without altering model parameters. Our two- stage framework first trains TSV on small set of labeled exemplars to form compact and well- separated clusters. It then augments the exemplar set with unlabeled LLM generations, employing optimal transport- -bssed algorithm for pseudo- labeling combined with confidence-batte filter- ing process. Extensive experiments demonstrate that TSV achieves state-of- of-the-art performance with minimal labeled data, exhibiting strong gen- cralization across datasets and providing a practi- cal solution for real-world LLM applications.
hulludnatied
Halluchatod
TSV
Pre-trained LLM
Pre-trained LLM
Figure I. T-SNE visualization (Van der Maaten & Hinton, 2008) of the last-token embeddings from the final layer of LLaMA-3. 1-B the TruthfulQA test set. (a) Pre trainaind model's embeddings exhibit significant overlap, whereas (b) adding TSV to latent states of intermediate LLM layer effectively separates the embeddings of truthful and hallucinated data.
consistent with user prompts but also possess the ability to detect hallucinations and alert users when they occur.
Recent work has explored leveraging the latent space of LLMs to identify hallucinations (Burns et al., 2023; Azaria & Mitchell, 2023; Marks & Tegmark, 2024; Yin et al., 2024a; Du et al., 2024; Chen et al., 2024a; Li et al., 2024; Kossen et al., 2024; Orgad et al., 2025). These approaches typically rely on the embeddings of off-the-shelf L.MSs to classify outputs as truthful or hallucinated. However, pre- trained LLMs are optimized for linguistic coherence using next-token prediction objective, often prioritizing fluency and syntactic correctness over factual accuracy (Radford et al., 2019). As result, their internal representations, while powerful for general text generation, can fail to provide a clear separation between truthful and hallucinated content (see real-world example in Figure la). This motivates key question:
1. Introduction
Large language models (LLMs) have demonstrated remark- able capabilities in natural language understanding and gen- eration, showcasing their potential as general-purpose task solvers (Zhao et al., 2023). Despite their success. LLMs can generate hallucinated nutputs-Ssatemeent statements that appear plausible but factually inaccurate or unsupported. Such hal- lucinations can undermine user trust and lead potentially harmful consequences, especially in high-stake applications (Zhang et al., 2023; Pal al., 2023). Therefore, to be truly trustworthy, an LLM must not only generate text that is
How can we shape the latent space of an LLM for hallucination detection?
'Department of Computer Sciences, University of Wisconsin- Madison "School of Software Technology, Zhejiang University. Correspondence to: Yixuan Li essaronni@@ss.wisee
Instead of fine-tuning the LLMs,, which is computationally expensive and alters the model's parameters (Gekhman et al., 2024), we propose learning a lightweight vector, called Truthfulness Separator Vector (TSV). As illustrated in Fig- ure 1b, this learnable vector is introduced during inference
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Steer LLM Latents for Hallucination Detection
and adjusting the internal representations of the LLM en- hance the separation between truthful and hallucinated gen- erations, without modifying the model's parameters. TSV focuses on reshaping the latent space for classifying halluci- nated responses, a fundamentally different objective from mitigating hallucinated generations (Li et al., 2024; Chen et al., 2024b; Marks & Tegmark, 2024). To the best of our knowledge. this the first exploration of steering represen- tations for hallucination detection.
of various design choices in TSV and validate its scala- bility across larger LLMS and diverse datasets. These findings provide systematic understanding of leverag- ing steering vector and limited labeled data for halluci- nation detection, paving the way for future research.
2. Related Works
Hallucination detection has emerged as a critical area of research, addressing safety concerns of LLMs and their de- ployment in real-world applications (Huang al., 2023). A plethora of works address hallucination detection by design- ing uncertainty scoring functions. For instance, logit-based methods utilize token-level probability as an uncertainty score (Ren et al., 2022; Malinin & Gales, 2021; Kuhn et al., 2023), verbalized methods prompt LLMs to express their un- certainty in human language (Lin et al., 2022b; Xiong et al., 2024), and consistency-baud evaluating the consistency across multiple responses (Man- akul et al., 2023; Chen et al., 2024a). Recently, internal state-based methods such as HaloScope leverage hidden activations to identify hallucination subspace (Du et al., 2024). However, these approaches often rely on default L.LM embeddings that do not inherently separate truthful and hallucinated data. In contrast, our method aims to shape the latent space through learnable steering vector for enhanced separation between the two types of data.
Learning TSV is appealing yet challenging due to the lack of large-scale human-labeled datasets with truthfulness an- notations for LLM generation, which are costly and time- intensive to create. To overcome this, we propose a two- stage training framework, In the initial stage, a small ex- emplar set of labeled data is used to guide the learning process. The objective in this stage is to encourage the steered embeddings to form compact clusters around class prototypes, representing truthful and hallucinated genera- tions. In the second stage, we augment the training data by leveraging unlabeled LLM generations, which are freely available for deployed L.LM systems through user queries and interactions (Du et al., 2024). To assign pseudo-labels to these unlabeled samples, we propose an optimal transport- based algorithm, which aligns unlabeled data embeddings with class prototypes by minimizing transport costs while accounting for the imbalanced class proportions. Further- more, a confidence-bass include only the most reliable pseudo-labeled samples in the training process. Together, these stages enable TSV to effectively separate truthful and hallucinated representations while significantly reducing the reliance on human labeling.
methods assess uncertainty by
sample selection is then used to
On the other hand, supervised methods leverage labeled data to train the classifier, assuming that pre-trained LLMs encode the truthfulness of responses within their internal states (Azaria & Mitchell, 2023; Marks & Tegmark, 2024). However, these methods require extensive labeling efforts.
Extensive experiments demonstrate the strong performance of our method across diverse datasets. On the challenging TruthfulQA benchmark (Lin et al., 2022a), our approach achieves a significant + +12.8 improvement in hallucination detection accuracy (AUROC) compared to state-of-the-ee -rrt--r methods. Notably, our method reaches performance com- parable to the fully-supervised upper bound (e.g., 84.2% vs. 85.5% on TruthfulQA). while using small labeled exemplar set with only 32 examples. TSV also exhibits strong generalization capabilities, maintaining competitive performance when applied to unseen datasets. Our key contributions are summarized as follows:
In contrast, our method performs hallucination detection with minimal human supervision, which is more practical for real-world applications.
A Attvaation engineering enables control over the LLM gen- eration during inference, applying task-specific steering vec- tors into the model's s internal representation (Im & Li, 2025). For example, several studies mitigate hallucination by shift- ing activations along the truthful direction identified by ana- lyzing activation differences between contrastive pairs (Li et al., 2024; Chen et al., 2024b; Marks & Tegmark. 2024). Concurrently, representation fine-tuning methods introduce learning task-specific interventions on linear subspaces of hidden representations (Wu et al., 2024) or sparse subsets of attention heads (Yin et al., 2024b).
1.
We propose the Truthfulness Separator Vector (TSV), a lightweight approach to separate truthful and hallu- cinated representations without fine-tuning the LLMs, which is largely unexplored in hallucination detection.
Our approach differs in the following key aspects: (1) We learn a steering vector specifically for hallucination detec- tion, focusing on separating representations rather than mit- igating hallucinated generations, and (2) while previous methods rely on large labeled datasets, our method achieves strong performance under minimal human supervision.
2. We develop optimal transport-based pseudo-labeling framework with confidence-busse sample selection to leverage unlabeled LLM generations effectively.
3. We demonstrate TSV's superior performance and per- form in-depth ablation studies to evaluate the impact
2

---

Steer LLM Latents for Hallucination Detection
Exemplar set Dg
Pre-trained LLM
Training objective C
Follsiittied Truthfu
000
[3) Exemplar sot
Unlabeled dataset Du
(1)
Confident data
sugmentation
OOOOOOO
loboling
selection
Truthfulness score S(x)
Gradient update
TSV
O Unlabeled data
Truthful data
Hallucinated data
Truthful protatype
* Hallucinated prototype
Initial training phase
Augmented training phase
Figure 2. Overall framework. In the initial training phase, Truthfulness Separator Vector (TSV) is trained an exemplar set. After initial training. (1) we assign soft pseudo-labels to unlabeled data, (2) select confident pseudo-labeled samples, and (3) augment the exemplar set with selected samples. Finally, retrain TSV with the augmented set. Best viewed in color.
3. Problem Setup
responses è' can be annotated with ground labels e; e C (truthful, hallucinated). This forms the labeled exem- plar set: Dg = ((epronp + (1), ..))..... (premp: @ ē cN)). where N is the total number of labeled exemplars. In this paper, will show that N can be kept very small (e.g.. 32) to minimize annotation costs while still providing valuable guidance for the learning process.
Definition 3.1 (Hallucination detector). We define the trathful distribution Preee as the joint distribution over pairs of the input prompts and their corresponding trutiful gener- ations. Let V denote a vocabulary space of a causal LLM, where each individual token is denoted as x V. Given an input prompt Xprmpp (I1,.... (......;,))))) and a model gen- eration x (Ia+1,..., In+mm, the task of hallucination detection aims to learn binary predictor G X + {0, 1}:
4. Method
Overview. Since LLMs do not inherently produce op- timal embeddings to separate truthful and hallucinated data, our framework introduces learnable vector, named Truthfulness Separator Vector (TSV), designed to enhance this separation within the representation space of the L.M.M As illustrated in Figure 2, TSV added into the latent states of the model during inference, which avoids the compu- tational overhead associated with retraining or fine-tuning the model. In what follows, describe how to learn TSV using unlabeled data and small exemplar set.
-fu 1 if (Xpromps + x) ~ F Fine 00,
G G(Xprompty 8)
otherwise
where Xprmpp x (I1...., Enn, Inn Ent1.... Intm) repre- sents the ordered concatenation of the prompt Xprompt and the generation X.
Following the practical setup in recent work HaloScope (Du et al., 2024), we utilize unlabeled LLM generations in the wild, which can be collected in vast quantities through user interactions with LLMs. This data can be freely collected for any deployed L.LM system, yet often contains a mixture of truthful and hallucinated content. Formally,
4.1. How to learn TSV? Initial training phase
TSV is defined single trainable vector v E TRd, which can be plugged into pre-trained LLMs after the generation is without compromising their original language capabilities. Given a sequence of tokens (e.g., input prompt and generation pair), we add v to h(), which represent the d-dimensional latent states at an intermediate layer J:
Definition 3.2 (Unlabeled data). We define the unlabeled pairs of input prompt Xprompr and LLM generation in the wild & to be the following mixture of distributions:
Puntabeled = (1 - (T))))Pmmm + T.P #rhallucination
h (1)
h ( + Av,
where a E [0,1] 1] is the fraction of hallucinated generation.
where is a hyperparameter which controls the strength of the steering, and is shared across all token positions. This intervention affects the embeddings in subsequent layers + 1, 1,.... L via the non-linear transformations inherent in LLM architecture. The last-token embedding at the final layer after applying TSV is:
The unlabeled dataset, Du [(xprommp (X*), xprompp )) is independently and identically sampled from the mixture distribution IP IFuntabeled. Here, M is the total number of unlabeled samples, and the tilde symbolizes the uncertain nature of the generation.
Pinath( (1) + Xv) Φr. o &1.11... o 1 + Av),
Exemplar set. In addition to the unlabeled data, we incor- porate a small, practical- snnetiicl-to-annttaal to-annttae set of labeled exem- plars to guide the learning of ballucination detector. Specif- ically, pairs of input prompt of prompt and L.---geeertedd
where & indicates the non-linear transformation in layer of the transformer model. In Section 5.3, we perform ablations on different layers of applying TSV.
3

---

Steer LLM Latents for Hallucination Detection
Training objective of TSV. To effectively detect halluci- nations, it is crucial to establish clear decision boundary between truthful and hallucinated data. To this end, we propose a training objective that learns TSV to separate em- beddings between two classes C = (truthful, hallucinated). This is achieved by performing maximum likelihood esti- mation (MLE) on the exemplar set Dr:
where is the decay rate, and F" Σx alein denotes
E, vtclyy)
the mean of the normalized embeddings from class
4.2. How to learn TSV? Augmented training phase
While we demonstrate that leveraging few labeled ex- amples helps hallucination detection (Section 5.3), these examples may not fully capture the diversity inherent in the truthful and hallucinated data distributions. To address the limitation, we propose to further incorporate unlabeled training data to augment the learning process.
De
arg max II p(cs Piinal (b." Av)),
(3)
where i is the index of training sample in De.
Label assignment via optimal transport. Assigning labels (truthful vs. hallucinated) to unlabeled data is non-trivial task, particularly because we aim to generate pseudo-labela that align with the class distribution of L.M generations, which are naturally imbalanced (Hu et al., 2024). To this end, we propose leveraging Optimal Transport (OT) (Villani et al., 2009), which provides a principled approach to label assignment. This approach aligns unlabeled data embed- dings with class prototypes by minimizing transport costs while respecting the imbalanced class proportions. Given unlabeled dataset Du with M samples, the optimization problem is formulated as:
To realize the MLE objective, we need to explicitly model the probability distribution p(c; | (h + Av)). In particular, we model the last-token embeddings at the final layer using a hyperspherical distribution with the unit norm, where truthful and hallucinated data each form distinct clus- ters. This modeling aligns with the structure of embeddings typically observed after the RMSNNrm layer in practical Transformer models (Dubey et al., 2024; Yang et al., 2024), where the norms of the embeddings are similar but direc- tions can vary (see verification in Appendix G). This can be naturally characterized by the von Mises-Fisher distribu- tion, a classical probability distribution in directional statis- tics (Mardia& & Jupp, 2009), which is analogous to spherical Gaussian distributions for features with unit norms. Under this model, the class conditional probability given by:
min Q=(o.1jaxa
Σ Emme log P Pm,c cH(Q)
REC
1
s.t. Q12
IM,
M
exp (Fit T r*)
Q T 1M =
p(c 1x) -===== Ee exp rx)
(4)
where 1M E R.M denotes an M M-dimensional vector of ones, Qm,c 37 (g)))r)))) represents an entry of the matrix E R Mx2 for assigned joint pseudo-label probabilities, and P m,, ((rr)))) denotes entry of P Mx2 for joint probabilities estimated by our model after initial training, where p(c|r)) ) is computed with Eq. 4. The first constraint ensures that for each unlabeled sample, the total probability mass of being assigned to two classes adds up to 1. The second constraint ensures that the number of samples assigned each class matches the expected class probability distribution e IR2. Here, (Q) = =-E; Quj log Qıj is the entropy function, and c is a hyperparameter controlling the smoothness of the assignment, which we set to 0.05. The entropy regularization term enables the computationally efficient Sinkhorn algorithm (Cuturi, 2013) to solve the problem. The minimizer of Eq. can be expressed as: (5)
where r" = P (h + Pl + Av)ll2, repre- sents the normalized last-token embedding the final layer, Me E 12d is the class prototype for class c, 2 0 is the con- centration parameter controlling how tightly the distribution is clustered around the mean direction Me..
Empirical loss function. Under the probability model defined above, our MI_E objective in Eq. 3 equivalent to minimizing the negative log-like likelihooho over the exemplar set Dr. This encourages embeddings within each class to cluster tightly around their respective class centroids:
Del
1
L = DE
g(e rT") log p(c rY)
1=1 CEC
where q(- rY) denotes the target label distribution, which can be either ground-truth or laeudo-label.
Q diag (a: /
(8)
where a E R and B E TR.2 are scaling coefficient vectors ensuring that the resulting Q forms a valid probability ma- trix. These scaling coefficients are determined iteratively using the following updates:
Prototype update. In practice, the prototype vector fe can be efficiently updated using exponential moving aver- age (Wang et al., 2022):
1M M p1/e3'
W
He < normalizelayu. + +(1 - o)F"),
(6)
x
B 8+
(Plls)Ta'
(9)
4

---

Steer LLM Latents for Hallucination Detection
We use the class distribution of the exemplar set a proxy for w, assuming missing completely at random (MCAR) scenario, which is natural assumption for data collected in real-world settings (Van Buuren, 2018).
datasett-TrutffuQAAA (Lin et al., 2022a), TriviaQA (Joshi et al., 2017), and NQ Open (Kwiatkowski et al., 2019); and domain-specific QA datasct-SciQ (Welbl et al., 2017), For evaluation, 25% of the QA pairs from each dataset are reserved for testing. Consistent with Du et al. (2024), 100 QA pairs are used for validation, while the remaining samples simulate the unlabeled training dataset. We ran- domly sample N = 32 pairs from TruthfuQA, and 64 pairs from the other datasets to construct an exemplar set, with K = 128 used for all experiments. Implementation details are provided in Appendix B. Models. We evaluate our method using two families of widely adopted open-source LLMs which provide accessi- 70b (Dubey et al., 2024), and Qwen-2.5-7b & 14b (Yang et al., 2024), By default, we used greedy sampling for the generation.
Confident data selection. Since pseudo-labed predicted for the unlabeled data may be incorrect and thus introduce noise into the learning process, we propose selecting only the most "confident" pseudo-labeled samples from the unla- beled dataset Du. which are most likely to be correct. We measure the model's predictive uncertainty using the cross- entropy between the assigned pseudo-label distribution q and the model's predicted distribution p. Specifically, for each unlabeled sample r,, we define:
ble internal representations: L.LMA-3.1--8
Ω -- {-- Σ - (t) r() i E IDu <EC
(10)
where ID denotes the index set of Du. We then select K samples from Dy to form the subset Ds:
Baselines. We evaluate our approach against a diverse set of 11 baseline methods, including existing state-of- the-art. The baselines are categorized as follows: (1) logit-based methods (Ren et al., 2022). Length- Normalized Entropy (LN-entropy) (Malinin Gales, 2021) and Semantic Entropy (Kuhn al., 2023); (2) consistency- based methods-Lexiess Similarity (Lin et al., 2024), Self- CKGPT (Manakul et al., 2023) and EigenScore (Chen et al., 2024a): (3) verbalized -eetbads-eerbaizee (Lin et al., 2022b) and Self-evaluation (Kadavath et al., 2022); and (4) Beahoos-Conttrst--Ce
Ds {Dt j TopK (-1,)),
(11)
where TopK denotes the indices of the K samples with the lowest uncertainty values, and D is j-th data in Dr.
Exemplar set augmentation. Finally, we augment the original training dataset DF by incorporating the selected samples D3 along with their pseudo-labels:
DE + DE Dss
(12)
internal state-based (CCS) (Burns et al., 2023), HaloScope (Du et al., 2024), and SAPLMA (Azaria & Mitchell, 2023). To ensure a fair com- parison, all methods are evaluated on the same test dataset, using their default experimental configurations as specified in the respective literature.
Search
The learning process described in Section 4.1 is then re- peated using the augmented dataset until convergence. We summarize the full algorithm in Appendix A.
4.3. Inference-time hallucination detection
During inference, we leverage the learned class prototypes He to perform hallucination detection. Specifically, we compute the truthfulness score as the normalized probability of test input's 's embedding vector r being assigned to the truthful class. The scoring function is defined as:
Evaluation. Following previous works (Kuhn et al., 2023; Du et al., 2024), we evaluate the performance with the area under the curve of the receiver operator characteristic (AUROC). We consider the generation truthful when the similarity score between the generation and the reference answer is larger than pre-defined threshold (e.g.. 0.5). Following Lin et al. (2022a), we utilize BLEURT (Sellam et al., 2020) to measure the similarity. Additionally, we show that our method is robust when evaluated using GPT- 4o (Hurst et al., 2024) in Appendix E.2.
S(x') exp (NS Festt
(13)
Ee exp (K(p)rYss))
Based on the scoring function, the hallucination detector is Gx(Xue) = 1 11 (S(ee) 2 <}. where indicates the truthful class and 0 indicates otherwise. The task can be seamlessly switched back to the original text generation by simply removing TSV v, restoring the model's initial generation capabilities without additional modifications.
5.2. Main results
In Table 1, we compare TSV with competitive hallucination detection methods from the literature. TSV demonstrates state-of-the- performance, significantly outperforming other methods on both the LLaMA-3.1 and Qwen-2.5-7b models. We show that unsupervised methods often strug- gle with inconsistent performance across different models and data distributions as the representations in LLMs are not inherently aligned with the hallucination detection task,
5. Experiments
5.1. Setup
Datasets. We evaluate our method on four generative
question-answering (QA) tasks: three open-domain QA
5

---

Steer LLM Latents for Hallucination Detection
Table Main results. Comparison with competitive hallucination detection methods different datasets. "Single sampling" indicates whether the approach requires multiple generations during inference. For method, the mean and standard deviation are computed
across three different random seeds. denotes methods trained on fully labeled datasets. values are percentages (AUROC), and the
best results are highlighted in bold.
Model
Method
Single Sampling
TruthfulQA
TriviaQA
SciQ
Open
Perplexity
71.4
76.3
52.6
50.3
LN-Entropy
X
62.5
55.8
57.6
52.7
Semantic Entropy
X
59.4
68.7
68.2
60.7
Lexical Similarity
X
49.1
71.0
61.0
60.9
FigenScore
X
59.6
SeIFCKGPT
X
57.0
80.2
67.9
60.0
LLaMA-3.1-8b
Verbalize
50.4
51.1
53.4
50.7
Self-evaluation
67.8
50.9
54.6
52.2
CCS
62.6
HaloScope
70.6
76.2
76.1
62.7
SAPLMA +
78.2
83.7
77.3
62.8
TSV (Ours)
2:0.2
84.0
85.8 0.4
76.1 76.1 1:077
TSV (Ours)
85,5 L0.1
87.2 87.2
88.6 0.1
78.0 - 1.0.2
Perplexity
65.1
50.2
53.4
51.2
LN-Entropy
X
66.7
51.1
52.4
54.3
Semantic Entropy
X
66.1
58.7
65.9
65.3
Lexical Similarity
X
49.0
63.1
62.2
61.2
EigenScore
53.7
61.3
63.2
57.4
SelFCKGPT
61.7
62.3
58.6
63.4
Qwen-2.5-7b
Verbalize
60.0
54.3
51.2
51.2
Self-evaluation
73.7
50.9
53.8
52.4
CCS
67.9
53.0
51.9
51.2
HaloScope
73.4
65.7
SAPLMA +
81.7
82.0
81.5
67.9
TSV (Ours)
87.3 +7.3
+0.4
79.8
82.0=0.4
73.8
T (Ours)
88.7 +0.1
84.2 +0.5
84.8
76,2 +0.3
making them less reliable for safety-critic applications. In contrast, our method achieves robust and superior perfor- mance across both models and all four datasets. In particular, TSV outperforms HaloScope 13.6% TruthfulQA with LLaMA-3.1-88 While both methods use the same valida- tion set and unlabeled data, HaloScope relies on default LLM embeddings. By contrast, our method leverages small exemplar set and shapes the latent space to better align with the hallucination detection task, enabling signif- icantly improved performance while remaining practical. Our method is also computationally efficient at the infer- stage with complexity of O(m²), where is the number of generated tokens. In contrast, some logit and consistency-basee methods require multiple sampling, re- sulting in a higher complexity of O (AA²), where A can be over 10 in practice. Qualitative results are in Appendix F, and experiments with larger models (LLaMA-3..) Qwen-2.5-114) are provided in Appendix E.1.
TruthfulQA, emphasizing the importance of shaping the la- tent space and the label-efficiency of our method. We further evaluate our method by comparing it with a fully supervised upper bound (TSV*). Specifically, all unlabeled data is notated with ground-truth truth labels, and TSV is trained this fully labeled dataset. We then compare our default setting (with a small exemplar set) to this supervised oracle on the same test set, using the AUROC metric to measure perfor- mance. Our evaluation, based on the LLaMA-3. 1-bb model, demonstrates that our method with 32 examples achieves hallucination detection AUROC of 84.2% on TruthfulQA, closely matching the performance of the fully supervised oracle (AUROC: 85.5%). These results underscore that our approach can achieve reliable hallucination detection accuracy with small labeling costs, offering an effective and efficient alternative to fully-supervised approaches.
&
5.3. Ablation studies
How does the steering location affect performance? We investigate the impact of the location where TSV is applied on overall performance using LLaMA-3.1 1-B-..... In Figure 3a, we present the effects of factors on performance: (I) the index of the layer, and (2) the component of the multi- head attention (MHA) architecture where TSV is applied.
Comparison with fully supervised methods. We com- pare our approach with fully supervised method SAPLMA which trains a binary classifier using the de- fault embeddings, fully labeled as truthful or hallucinated. As shown in Table 1, with only 32 labeled examples, TSV outperforms SAPLMA with full supervision by 6.0% on
6

---

Steer LLM Latents for Hallucination Detection
85
90
90
80
85
TruthfuiQA
TruthfulgA
TriviaA
TriviaQA
AUROC (%)
70
75
AUROC
AUROC
65
70
70
60
05
63
55
MLP Attention Residual
60
60
50
10 15
20
25
30
50
0.1
0.5
50
16 32
64
Steering layer (2)
Steering strength (A)
# of exemplars (M)
(a) Effect of the steering location
(b) Effect of the steering strength (A)
(c) Effect of the numher of exemplars (N)
Figure 3. (a) Effect of steering location (layer index and MHA components) TruthfulQA performance, effect of steering strength À
(Section 4.1) 4.1,, and (c) effect of the number of labeled exemplars. All results are reported as AUROC using LLAMA-3.1-88
In particular, the MHA can be conceptually expressed as:
accuracy (PL ACC) on selected unlabeled generations and hallucination detection perfor- mance (HD AUROC) on the test dataset. Results are reported based LLaMA 3.1-8b.
Table 2. Camparison on
G+1 f; + QfAttt((f;,
(14)
where f; represents the output of the i-th transformer block, (f;) denotes the output of the self-attention module in the i-th block, and Q; is the weight of the feedforward layer. We train and apply TSV at three distinet locations within the MHA architecture: (1) residual stream f, (2) MLP output QAttt(f), (f). and (3) attention output (ft We find that ap- plying TSV in the early-middle layers (e.g., 4th- layers) the most effective for guiding representations the hal- lucination detection task. Performance improves as TSV is applied from the top layers towards the early-middle layers but gradually declines in later layers. Moreover, the choice of location within MHA shows minimal impact on perfor- mance. Our findings suggest that tuning the layer position likely more critical than the specific MHA location for effectively separating representations in the hallucination detection task.
K
Dataset
32
64
128
256
512
Metric
PI. ACC (%)
100
98.4
95.3
89.8
81.8
TruthfulQA
HD AUROC (%)
83.5
84.0
84.2
84.7
84.2
PL ACC (%)
100
91.2
89.1
87.9
87.0
TriviaQA
AUROC (%)
78.3
82.8
84.0
82.2
81.0
small number of labeled exemplars, which are practical to obtain. However, when the number of labeled exemplars is too small (N = 8), the performance becomes suboptimal.
Pseudo-labeling accuracy and the number of selected unlabeled data. We analyze the effect of the number of selected unlabeled samples, K, for augmenting the training data. In Table 2, we report (1) the pseudo-labeling accu- racy on selected unlabeled generations (PL ACC), and (2) the overall hallucination detection performance on the test dataset (HD) AUROC). Our optimal transport- based pseudo- labeling achieves near-perfect accuracy up to K = 64, with a gradual decline as K increases further. The hallucina- tion detection performance peaks at K = 128 and decreases thereafter. This trend indicates that while our learning frame- work is relatively robust to the number of selected samples, including too many false- colse-positive samples introduce noise into the learning process, potentially affecting perfor- mance.
How does the steering strength affect the performance? To better understand the characteristics of TSV, we vary the steering strength À E {0.1, 0.5, 10) and analyze its effect on the model's performance, as demonstrated in Fig- ure 3b. The results show that performance improves with moderate steering strength (e.g., = 5), but declines as increases further. A small À does not provide sufficient signal to meaningfully separate representations in the final layer, while large Ả disrupts the representation space by dominating it, resulting in suboptimal performance.
How does number of exemplars affect the performance? In Figure 3c, we examine the impact of the number of labeled exemplars on performance. We evaluate N E (8, 16, 32, 64) and compare them the fully-supervised upper bound (FS), where all samples in Du are labeled with ground truth. Our results indicate that small exemplar set is effective for modeling the truthfulness distribution when N = (32, 64), achieving performance almost comparable to the fully superveue oracle. This demonstrates that re- liable hallucination detector can be designed using only a
Can TSV generalize across data distributions? While TSV shows superior performance, we are also interested in its capability to generalize across different data distribu- tions. As shown Figure 4, we evaluate the generalization capability of TSV using LLaMA-3. A.1.1-18) model by learning it from source in-distribution (ID) dataset, directly apply- ing it to different target out-of-distribution (OOD) datasets, and computing the corresponding hallucination detection scores. The results demonstrate the robust transfcrability of our approach across diverse datasets, specifically achiev-
7

---

Steer LLM Latents for Hallucination Detection
TrutifaiAA
Treetd Hall lalarded
Tushia Hatuciaate
84.2
79.8
80.0
67.9
Source
75.7
$4.0
$1.0
70.2
520
75
78.2
76.6
85.0
71.2
74
15
Open
76.9
72.0
75.0
76.1
buthuuiQA TIVvSQAA
5dQ
Open
8:
0.5
Target
Truthultees score
Trutifateess
(a) Scores HaloScope
(b) Scores Ours
Figure 4. Generalization results on
datasets.
Figure 5. Score distributions for HaloScope our method.
Table Component analysis. TSV: Truthfulness Separator Vector,
Table 4, Performance comparison with PEFT methods. % Params is calculated by dividing the number of trainable parameters by the total number of parameters in the base LLM.
IT: Initial Training phase, and AT: Augmented Training phase.
Component
Dataset
Index
TSV
IT
AT
TruthfulQA
TriviaQA
SciQ
NQ Open
Model
Method
Trainable Parameters
Datasets
(a)
x
52.2
50.8
50.8
Params
Params
TruthfulQA
TriviaQA
(b)
x
52.0
50.2
57.1
52.1
(c)
80.9
80.8
82.0
71.2
LoRA
3.4M
0,0424
83.6
82.0
Llama 3.1-8b
LoReFT
32K
0.0004 %
77.5
76.0
Ours
84.2
84.0
85.8
76.1
Ours
4K
0.00005 %
84.2
84,0
LoRA
2.5M
0,0331 %
85.9
76.0
ing hallucination detection AUROC of 79.8% on Triv- iaQA when TSV is learned from TruthfulQA, exhibiting performance close to that obtained directly from TriviaQA (84.0%). This strong transferability highlights TSV's poten- tial for real-world L.M applications, effectively detecting hallucinations even under domain shifts.
Qwen2.5-7h
LoReFT
28K
0.0004
81.5
79.3
3.6K
0.00005%
87.3
79.8
Our method achieves superior performance while utilizing 8 x to 800x fewer parameters, demonstrating that TSV can effectively shape representations for the hallucination de- tection task while significantly reducing computational and annotation costs. Training time is detailed in Appendix H.
Component analysis. In Table 3, we present ablation sults for the components of our approach using LLaMA- 3.1-8b model. Comparing (a) and (b), which update the class prototypes without using TSV, we observe that train- ing performance remains close to 50%, and even with the augmented training phase, performance does not improve. In contrast, comparing (a) and (c), we find that incorpo- rating TSV improves AUROC by 28.7% TruthfulQA. This demonstrates that shaping representations with TSV is critical for hallucination detection, as it makes the repre- sentations more separable. Further, comparing (c) with our full approach, we see that the augmented training phase enhances performance by an additional 3.3% on Truth- fulQA, achieving the best performance among all config- urations. Unlike (a) and (b), this highlights that the aug- mented training phase is effective only when supported by well-structured representations and accurate pseudo-labeis underscoring the importance of learning TSV. Overall, inte- grating all components achieves the best performance across all datasets, indicating that each component effective for addressing the hallucination detection task.
Visualization of truthfulness score distributions. Figure 5 visualizes the score distributions for HaloScope (Du et al., 2024) and our method on TruthfulQA based on LLaMA-3.L 8b model. Our approach demonstrates a more distinct sepa- ration between truthful and hallucinated data distributions. This enhanced differentiation is attributed to the effective- ness of shaping latent space with TSV, which contributes to more reliable detection performance than other methods.
6. Conclusion
In this work, we tackle the challenge of hallucination de- tection in LLM by introducing the Truthfulness Separator Vector (TSV), lightweight and modular approach that re- shapes the latent space during inference to enhance the sep- aration between truthful and hallucinated outputs without al- tering the model's parameters. Through a two-stage training framework that combines a small labeled exemplar set with unlabeled LLM generations, TSV achieves superior perfor- mance while minimizing reliance on human labeling and computational cost. Our experiments demonstrate TSV's ef- fectiveness, achieving state-of-the-art accuracy with strong generalization across datasets. This work not only advancee the state of hallucination detection but also lays the ground- work for scalable and practical solutions to improve the reliability of L.MMs in real-world applications.
Computational efficiency of TSV. To evaluate the cost- efficiency of our method, we compare TSV with parameter- efficient fine-tuning (PEFT) approaches in Table 4. Specif- ically, we train LoRA (Hu et al., 2022) and LoReFT (Wu et al., 2024) using our training framework, leveraging a small labeled exemplar set along with the unlabeled dataset.
8

---

Steer LLM Latents for Hallucination Detection
Impact Statement
Cuturi, M. Sinkhorn distances: Lightspeed computation of optimal transport. In NeurlPS, 2013.
Ensuring the reliability LLM is paramount as they are increasingly integrated into high-stakes applications like healthcare, law, and education. This work tackles the eritical challenge of hallucination detection, which iden- tifies factually inaccurate outputs for enhanced user trust. We propose a practical method that minimizes computa- tional and labeling costs while enabling a plug-and-ple approach for pre-trained LLMs. This research not only ad- vances the technical landscape of hallucination detection but also lays the groundwork for scalable and reliable Al systems, fostering broader trust and adoption of LLMs in critical domains. Our study does not involve human sub- jects, complies with all legal and ethical standards, and we do not anticipate any potential harmful consequences resulting from our work. Code is available at: ht ats: //github. com
Du, X., Xiao, C., and Li, Y. Haloscope: Harnessing un- labeled Ilm generations for hallucination detection. In NeurIPS, 2024.
Duan, J., Kong, F., Cheng, H., Diffenderferr, J., Kailkhura, B., Sun, L., Zhu, X., Shi, X., and Xu, K. Truthprint: Miti- gating Ivim object hallucination via latent ruthful--uided pre-intervention. arXiv preprint arXiv:2503. 10602, 2025.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21788. 2024.
Gekhman, Z., Yona, G., Aharoni, R., Eyal, M., Feder, A., Reichart, R., and Herzig, J. Does fine-tuning llms on new knowledge encourage hallucinations? In EMNLP, 2024.
Acknowledgement
Hu, E. J., Shen, Y., Wallis, P., Allen- Z., Li, Y.. Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. In ICLR, 2022.
We gratefully acknowledge Maxim Khanov, Shawn Im, and Shrey Modi for their valuable comments on the draft. Seongheon Park, Xuefeng Du, Min-Hsuan Yeh and Yix- uan Li are supported in part by the AFOSR Young Inves- tigator Program under award number PA9550-23-1-0183 National Science Foundation under awards IIS-2237037 and IIS2331669 Alfred P. Sloan Fellowship, and Schmidt Sciences Foundation.
Hu, X., Zhang, Y., Peng, R., Zhang, H., Wu, C., Chen, G., and Zhao, J. Embedding and gradient say wrong: A white-box method for hallucination detection. In EMNLP, 2024.
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng. W., Feng. X., Qin, B., et al. A survey hallucination in large language models: Principles, taxon- omy, challenges, and open questions. ACM Transactions on Information Systems, 2023.
References
Azaria, A. and Mitchell, T. The internal state of an Ilm knows when it's lying. In EMNLP Findings, 2023.
Burns, C., Ye, H., Klein, D., and Steinhardt, J. Discovering latent knowledge in language models without supervision. In ICLR, 2023.
Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. Unsupervised learning of visual features by contrasting cluster assignments. In NeurIPS, 2020.
Im, S. and Li, Y. A unified understanding and evaluation of steering methods. arXiv preprint arXiv:2502.02716, 2025.
Chen, C., Liu, K., Chen, Z., Gu, Y., Wa, Y., Tao, M., Fu, Z., and Ye, J. Inside: Llms' internal states retain the power of hallucination detection. In ICLR, 2024a.
Jiang, N., Kachinthaya, A., Petryk, S., and Gandelsman, Y. Interpreting and editing vision-language representations mitigate hallucinations. In ICLR, 2025a.
Chen, J., Zhang, T., Huang, S., Niu, Y., Zhang, I., Wen, L., and Hu, X. Ict: Image-object cross-level trusted intervention for mitigating object hallucination in large lasion-language models. In CVPR, 2025.
Jiang, Z., Chen, J., Zhu, B., Luo, T., Shen, Y., and Yang, X.
Devils in middle layers of large vision- models: Interpreting, detecting and mitigating object hallucina- tions via attention lens. In CVPR, 2025b.
Chen, Z. Sun, X., Jiao, X., Lian, F., Kang, Z., Wang, D., and Xu, C. Truth forest: Toward multi-scale truthfulness in large language models through intervention without tuning. In AAAI, 2024b.
Joshi, M., Choi, E., Weld, D., and Zettlemoyer, L. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. ACL, 2017.

---

Steer LLM Latents for Hallucination Detection
Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z-, DasSarma, N., Tran-Johnson, E., et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221, 2022.
Mardia, K. and Jupp, E. Directional statistics. John
Wiley & Sons, 2009.
Marks, S. and Tegmark, M. The geometry of truth: Emer- gent linear structure in large language model representa- tions of true/false datasets. In COLM, 2024.
Kossen, J., Han, J., Razzak, M., Schut, I.,, Malik, S., and Gal, Semantic entropy probes: Robust and cheap hallucination detection in llms. arXiv preprint (rXiv:2406.15922 2024.
Orgad, H., Toker, M., Gekhman, Z., Reichart, R., Szpek- tor, I., Kotek, H., and Belinkov, Y. Lims know more than they show: On the intrinsic representation of Ilm hallucinations. In ICLR, 2025.
Kuhn, L.. Gal, Y., and Farquhar, S. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natu- ral language generation. In ICLR, 2023.
Pal, A., Umapathi, L. K., and Sankarasubbu, M. Med- halt: Medical domain hallucination test for large language models. In CoNLL, 2023.
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K. et al. Natural questions: a benchmark for question answering research. In TACL, 2019.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., al. Pytorch: An imperative style, high-performance deep learning library. In NeurlPS, 2019.
Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-time intervention: Eliciting truthful answers from language model. In NeurIPS, 2024.
Qiao, Y., Duan, H., Fang, X., Yang. J., Chen, L., Zhang, S., Wang, J., Lin, D., and Chen, K. Prism: A framework for decoupling and assessing the capabilities of vims. In NeurIPS, 2024.
Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W. X., and Wen, J.-R. Evaluating object hallucination in large vision- language models. In EMNLP, 2023.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I,, et al. Language models are unsupervised multitask learners. OpenAl blog, 2019.
Lin, S., Hilton, J., and Evans, Truthfulqa: Measuring how models mimic human falsehoods. In ACL, 2022a.
Ren, J., Luo, J., Zhao, Y., Krishna, K., Saleh, M., Lakshmi- narayaman, B.. and Liu, P. J. Out-of-distribution detection and selective generation for conditional language models. In ICLR, 2022.
Lin, S., Hilton, J., and Evans, O. Teaching models to express their uncertainty in words. In TMLR, 2022b.
Lin, Z., Trivedi, S., and Sun, J. Generating with confidence: Uncertainty quantification for black-box large language models. In TMLR, 2024.
Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T., and Saenko, K. Object hallucination image captioning. In EMNLP, 2018
Liu, F., Lin, K., Li. L., Wang, J., Yacoob, Y., and Wang, L. Mitigating hallucination in large multi-modal models via robust instruction tuingg ICLR, 2024.
Sellam, T., Das, D., and Parikh, A. Bleurt: Learning robust metrics for text generation. In ACL, 2020.
Liu, H., Li, C., Wu, Q., and Lee, J. Visual instruction tuning. In NeurIPS, 2023.
Sun, Z., Shen, S., Cao, S., Liu, H., Li, C., Shen, Y., Gan, C., Gui, I.,, Wang, Y.-X., Yang, Y., Keutzer, K., and Dar- rell, T. Aligning large multimodal models with factually augmented RLHF. In ACL Findings, 2024.
Liu, S., Ye, H., Xing, L., and Zou, J. Reducing hallucina- tions in vision-language models via latent space steering. In ICLR, 2025.
Tong, P., Brown, E., Wu, P., Woo, S., IYER, A. J. V., Akula,
Loshchilov, Decoupled weight decay regularization. ICLR, 2019.
S. C., Yang, S., Yang, J.. Middepogu, M.. Wang, Z., et al. Cambrian-1: fully open, vision-centric exploration of multimodal 1mm.. In NeurIPS, 2024.
Malinin, A. and Gales, M. Uncertainty estimation in autore- gressive structured prediction. In /CLR, 2021.
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaci, Y.. Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Manakul, P., Liusie, A., and Gales, M. Selfcheckgpt: Zero- resource black-box hallucination detection for generative large language models. In EMNLP, 2023.
10

---

Steer LLM Latents for Hallucination Detection
Van Buuren, S. Flexible imputation of missing data. CRC press, 2018.
Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:230..1052, 2023.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 2008
Villani, C. al. Optimal transport: old and new. Springer, 2009.
Wang. H., Xiao, R., Li, Y., Feng, L., Niu, G., Chen, G., and Zhao, Pico: Contrastive label disambiguation for partial label learning. In ICLR, 2022.
Welbl, J. Liu, N. F., and Gardner, M. Crowdsoure- ing multiple choice science questions. arXiv preprint arXiv: 1et.i:7077000000 2017.
Wu, Z., Arora, A., Wang, Z.., Geiger, A., Jurafsky, D., Man- ning, C. D., and Potts, C. Reft: Representation finetuning for language models. In NeurlPS, 2024.
Xiong, M., Hu, Z., Lu, X., Li, Y., Fu, J., He, J., and Hooi, B. Can Iims express their uncertainty? an empirical evaluation of confidence elicitation in 1lms. In ICLR, 2024.
Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F, Wei, H., et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024.
Yang, L., Zheng, Z. Chen, B., Zhao, Z., Lin, C., and Shen, C. Nullu: Mitigating object hallucinations in large vision- language models via halluspace projection. In CVPR, 2025.
Ych, M.-H., Kamachee, M., Park, S., and Li, Y. Can your un- certainty scores detect hallucinated entity? arXiv preprint arXiv:2502.11945 2025.
Yin, F., Srinivasa, J., and Chang, K.-W. Characterizing truthfulness in large language model generations with local intrinsic dimension. In ICML, 2024a.
Yin, F.. Ye, X., and Durrett, G. Lofit: Localized fine-tuning Ilm representations. In NeurIPS, 2024b.
Zhang, Y., Li, Y., Cui. L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al. Siren's 's song in the ai ocean: survey on hallucination in large language models. arXiv preprint arXiv:2309.012199 2023.
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, 7., et al. A survey of large language models. arXiv preprint arXiv:2303.1822: 2023.
Zheng, L., Chiang, W.-L., Sheng. Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. Judg- ing an-as-a-judge with mt- nt--bench and chatbot arena. In NeurlPS, 2023.

---

Steer LLM Latents for Hallucination Detection
Appendix
Contents
Algorithms
13
Overall training framework
13
A.2 Sinkhorn algorithm
13
Implementation Details and Hyperparameters
13
B.1 Implementation details (ours)...
13
B.2 Hyperparameters
14
B.3 Implementation details (baselines)
14
More Details of the Benchmarks
15
Additional Related Works
15
Ablation Studies
16
E.1
Scalability larger language models
16
E.2 Evaluation results with GPT-4o
16
E.3 Design choices for the class distribution
17
E,4 Results with LLaMA-2-chat-7b
17
E.5 Robustness to pseudo-labe noise
17
E,6 Class distribution mismatch.
18
Qualitative Results
18
Embedding Norms
18
Compute Resources and Time
19
H.1 Software and hardware
19
H.2 Training and inference time
19
I Limitations and Future Work
19
12

---

Steer LLM Latents for Hallucination Detection
A. Algorithms
A.1. Overall training framework
Algorithm Overall training framework
Parameters: Minitial Raugmeemedd 1, K
Input: Exemplar set DE, unlabeled dataset Du
Initialize TSV v and class prototypes with random weights.
Apply TSV to the intermediate layer I: h(") &- h + Av Initial training phase
1: for = 1 to Minitiall do
2: Compute the loss L(DF)
Equation (5)
3: Update v with a gradient step
4: Update class prototypes je using EMA
Equation (6)
5: end for
2. Augmented training phase
1: Compute pseudo-labels for Du using the Sinkhorn algorithm
Equations (7) to (9)
2: Select confident samples Dy from Du
Equations (10) and (11)
3: Augment exemplar set: DE Dr. U Ds-
Equation (12)
4: for = 1 to Paugmemred do
5: Compute the loss L(Dp)
Equation (5)
6: Update with gradient step
7: Update class prototypes j4 using EMA
Equation (6)
8: end for
A.2. Sinkhorn algorithm
Algorithm 2 Sinkhorn algorithm for entropic-regularizzed optimal transport
Parameters: Riter
Input: Unlabeled dataset Du, class distribution w, cost matrix log P
Initialize 3 <- 12
1: for 1 to Fliter do
2:
1 1M
M p1/e3
B (P1/e)Ta
Equation (9)
4: end for Return diag(a))*/
Equation (8)
B. Implementation Details and Hyperparameters
B.1. Implementation details (ours)
Following Kuhn et al. (2023), we generate the likely answer using beam search with beams. Class prototypes μt, and TSV v are randomly initialized, and trained in two stages: 20 epochs using only the exemplar set, followed by additional 20 epochs after augmentation. Training is performed using the AdamW optimizer (Loshchilov, 2019). with learning rate of 5e-03 and batch size of 128. We set steering strength À to 5, the concentration parameter of the vMF distribution to 10, and the EMA decay rate to 0,99. The number of iterations in the Sinkhorn algorithm is 3, and the regularization parameter is set to 0.05, following Caron et al. (2020). The hyperparameters are tuned based on testing performance on the validation set. The steering location for each model is detailed Appendix B.2. For generating responses, we utilize the following input prompt:
13

---

Steer LLM Latents for Hallucination Detection
Input prompt for generating responses
Prompt:
Answer the question concisely:
Q: [question]
A:
B.2. Hyperparameters
Table 5, Steering layer index for LLaMA-3.1 3.1-88 and Qwen-2.5-7b.
Datasets
Model
TruthfulQA
TriviaQA
SciQ
NQ Open
LLaMA-3.1-8b
Qwen-2.5-7b
11
Table 6. Hyperparameter search space. The hyperparameters used in method are underlined.
Hyperparameters
Search space
Steering MHA component
('mlp', 'attn', 'res")
Steering strength (A)
{0.1, 1, 5, 10}
Optimizer
{SGD, Adam, AdamW}
Learning rate
(le-04, 2e-04, 5e-04, le-03, 2e-03, Se-03, le-02)
Batch size
{32, 64, 128}
Initial training epochs (Ninitial)
{5, 10, 20, 40}
Augmented training epochs (Paugmented)
{5, 10, 20, 40}
EMA decay rate (a)
(0, 0.5, 0.9, 0.95, 0.99, 1}
Concentration parameter (K)
{0.1, 1, 5, 10, 100)
The steering layer index for applying TSV is provided Table 5. We select the steering layer index based on the model's performance on the validation set for each dataset, and we consistently apply TSV to the residual stream of MHA component. The search space of hyperparameters outlined in Table The training configuration is determined using the performance on the TruthfulQA dataset with LLaMA-3.1-88 and is uniformly applied across all experiments.
B.3. Implementation details (baselines)
For Perplexity (Ren al., 2022), we evaluate the average perplexity score based on the generated tokens. For baselines requiring multiple generations (Malinin Gales, 2021; Kuhn al., 2023; Lin al., 2024; Manakul et al., 2023; Chen et al., 2024a), utilize multinomial sampling generate 10 samples (A = 10) per question, setting the temperature to 0.5, and adhering to the default configurations outlined in the original paper. For Verbalize (Lin et al., 2022b), we implement the following prompt:
Verbalized
Prompt:
Q: {question]
A: [answer)
The proposed answer is true with confidence value (0-100) of
The generated confidence value is directly utilized as the uncertainty score during testing. For the Self-evaluation (Kadavath al., 2022), adhere the approach outlined in the original paper and use the following prompt:
14

---

Steer LLM Latents for Hallucination Detection
Self-evaluation
Prompt:
Q: {question]
A: [answer}
Is the proposed answer:
(A) True
(B) False
The proposed answer is:
In line with the original paper, evaluate hallucination detection performance by using the log probability of the output token "A" as the uncertainty score. We implement SAPLMA (Azaria & Mitchell, 2023) using MLP classifier consisting of three hidden layers with decreasing numbers of hidden units (256, 128, and 64). Each layer employs ReLU activations, consistent with the original paper. We set the LoRA² (Hu et al., 2022) rank to 8, to 32, and the dropout rate to 0.1. We use the AdamW W optimizer with learning rate of 5e-04. For LoReFT (Wu et al., 2024), we set the rank to 4 and apply to the same layer as ours and all input positions to ensure consistency with ours.
C. More Details of the Benchmarks
We evaluate method on four publicly available generative question-answering (QA) tasks: TruthfulQA (Lin et al., 2022a), TriviaQA (Joshi et al., 2017), SciQ (Welbl et al., 2017), and NQ Open? (Kwiatkowski et al., 2019). TruthfulQA focuses on assessing model's truthfulness and robustness in generating false or unsupported responses; we use its generation track with 817 QA pairs. TriviaQA includes fact-based questions from trivia websites, making it useful for testing factual accuracy; we use the deduplicated validation split of the re.noconteat subset, comprising 9,960 QA pairs. SciQ is a domain-specific dataset with science-related QA pairs, suitable for evaluating hallucinations in specialized domains, and we its validation split with 1,000 QA pairs. NQ Open, with 3,610 QA pairs in its validation split, challenges models open- open-domain reasoning and general knowledge. Together, these datasets provide comprehensive benchmark for evaluating hallucination detection across diverse tasks.
D. Additional Related Works
Hallucination in Large Vision-Language Models (LVLMs). Leveraging the progress in LLMs, Large Vision- LanguaLe Models (LVLMs) (Liu al., 2023; Zhu al., 2023; Tong et al., 2024; Qiao et al., 2024) have demonstrated strong capabilities in interpreting and reasoning about real-world visual content. Despite these advancements, fundamental challenge remains in object hallucinations (Rohrbach et al., 2018: 8: Li et al., 2023), where the model incorrectly mentions objects that not present in the image, often producing outputs that appear plausible but are factually inaccurate.
A growing body of work aims to detect and mitigate object hallucinations in LVLMs by using external models (Liu et al., 2024) or fine-tuning (Sun et al., 2024), but these approaches are often computationally expensive and resource-intensive. Inspired by activation engineering techniques LLMs, recent approaches (Jiang al., 2025a;b; Chen al., 2025; Duan al., 2025) instead leverage the latent representations within LVLMs address object hallucinations in more efficient and interpretable manner. For instance, Nullu (Yang al., 2025) extracts low-rank subspaces of the differences between truthful and hallucinated features, and further edits the LVLM's weights to mitigate object hallucinations. VTI (Liu et al., 2025) proposes to steer latent representations during inference to improve the alignment between vision features and textual outputs, thereby reducing object hallucinations.
'htrps//hugginngface
http://wuugniiiiie.e.ccddassnnnn col/lataeee/mmanddjjohhitttaaaaa qa
"hups://uugingface covllanseet//lllenniiccg
open
15

---

Steer LLM Latents for Hallucination Detection
E. Ablation Studies
E.1. Scalability to larger language models
Table Hallucination detection results on larger LLMs.
LLaMA-3.1-70%
Qwen-2.5-14b
Method
TruthfulQA
SciQ
TruthtulQA
SciQ
Perplexity
52.3
53.5
64.7
76.0
CCS
63.1
55.5
69.4
80.0
HaloScope
67.4
60.3
74.6
78.9
SAPLMA
70.2
58.4
83.1
83.8
Ours
76.6
75.0
83.6
89.7
We evaluate our method on larger LLMs, including the LLaMA-3.1-700 and Qwen-2.5-14b models, to illustrate its scalability. Specifically, we apply TSV to the residual stream of the 31st layer in LLaMA-3.1-706 and the 23rd layer in Qwen-2.5-14b.. Results in Table demonstrate that our approach consistently outperforms four strong baselines including the fully-supervised method (SAPLMA** + while also improving upon the performance achieved with smaller LLMs. For instance, the SciQ dataset, approach achieves an AUROC of 89. 997% with the Qwen-2.5-144 model, compared to 82.0% with the Qwen-2.5-7b model, reflecting performance gain of 7.7%.
E.2. Evaluation results with GPT-4o
Table 8. Hallucination detection results using labels generated by GPT-4o.
LLaMA-3.1-4 8bb
Qwen-2.5-7b
Method
TruthfulQA
SciQ
TruthfulQA
SciQ
Perplexity
65.8
41.3
56.3
55.3
CCS
59.5
52.7
58.6
50.3
HaloScope
63.3
66.9
63.4
77.0
SAPLMA +
70.7
72.6
67.7
79.6
TSV (Ours)
75.1
81.6
87.2
85.6
In main paper, generation is considered truthful if its BLEURT score with the gold standard answer exceeds predefined threshold. In addition to the threshold-based evaluation using BLEURT (Sellam et al., 2020), we employ GPT-4o (Hurst et al., 2024) to determine the truthfulness of the generation, under the paradigm of LLM-as-a-judge (Zheng et al., 2023). Specifically, we assess the truthfulness of L.M--generatee responses by determining their semantic equivalence to the given gold standard answers. Table 8, we demonstrate that our method consistently outperforms competitive baselines, highlighting its robustness across different approaches for determining the truthfulness of the model generation. We utilize the following input prompt:
Input prompt for GPT-4o based evaluation
Prompt:
You are tasked with evaluating whether generated answer is truthful compared to a provided list of gold standard answers. Your response should be "Yes" the generated answer is consistent with any of the gold standard answers, and "No" otherwise. Provide clear justification for your decision. Question: {question}
Gold Standard Answers: {gold_stann sstandardd .answess)
Generated Answer: (generated Lanswer)
Response Format:
Answer: [Yes/No]
Justification: [Explain briefly why the answer is correct or incorrect.]
We consider generation truthful when the answer "Yes" and hallucinated when the answer "No."
16

---

Steer LLM Latents for Hallucination Detection
Table 9. Hallucination detection results using different
LLaMA-3.1-88
Qwen-2.5-7b
Method
TruthtulQA
SciQ
TruthfulQA
SciQ
Uniform
83.2
84.0
87.0
81.2
Estimation
83.7
83.9
87.3
80.6
Oracle
84.3
85.0
87.5
82.6
Ours
84.2
85.8
87.3
82.0
E.3. Design choices for the class distribution w
We ablate the various design choices for the class distribution w the unlabeled dataset when formulating the optimal transport problem in Equation (7). We evaluate the following configurations: (1) uniform class distribution (Uniform), (2) an estimated class distribution obtained via pseudo-labeling with nearest neighbor classification (Estimation), (3) the ground-truth class distribution of the unlabeled dataset (Oracle), and (4) the class distribution derived from the exemplar set (Ours). In Table our proposed design choice achieves performance comparable to the Oracle approach. Notably, the robustness to design choices of w appears to stem from our confident data selection procedure in Equation (11), which plays an important role in ensuring stable performance across different configurations. Additionally, we demonstrate that the pseudo-labeling approach also effective, highlighting the scalability and adaptability of the algorithm.
E.4. Results with LLaMA-2-chat-7b
Table 10. Experiment results with I.MMA-2-chat-7hh All results are directly copied from HaloScope.
Method
TriviaQA TruthfulQA
Perplexity
56.77
72.13
LN-Entropy
61.51
70.91
Semantic Entropy
62.17
73.21
Lexical Similarity
55,69
75,96
EigenScore
51.93
73.98
SeIfCKGPT
52.95
73,22
Verbalize
53.04
52.45
Self-evaluation
51.81
55.68
CCS
61.27
60.73
HaloScope
78.64
77.40
TSV (Ours)
80.93
85.20
We evaluate our method using the LLaMA-2-chat-7b model (Touvron et al., 2023), following the experimental setup outlined in HaloScope (Du al., 2024). Specifically, apply TSV to the residual stream of the 9th layer and adopt the same training configurations in the main experiments. Our results demonstrate that TSV is effective even when applied to legacy models such as ALaMA-2-chat-71 showcasing the versatility and robustness of our approach.
E.5. Robustness to pseudo-label noise
Table 11. Rohustness to pseudo-lahel noise I.IMMA-.1--hh
Pseudo-label Noise Ratio (%)
(no flipping, original)
10
15
20
25
Hallucination Detection AUROC (%)
84.2
83.8
82.6
82.2
81.3
We evaluate the impact of pseudo-label noise our method's performance. Specifically, we use the same set of selected unlabeled examples from TruthfulQA and systematically introduce noise by flipping some of the correct pscudo-labels. As shown in Table 11, the ballucination detection AUROC on LLaMA-3.1 3.bbbb gradually decreases as noise increases, but the performance remains relatively robust, with only a modest drop (from 84.2% to 81.3%) even under 25% label noise. This demonstrates the robustness of our approach to pseudo-labeling errors.
17

---

Steer LLM Latents for Hallucination Detection
E.6. Class distribution mismatch
Table 12. Ablation study class distribution mismatch using LLaMA-3.1-8b.
Method
TruthfulQA
SciQ
Aligned (Ours)
84.2
85.8
Uniform
82.8
82.5
Reversed
82.1
82.7
We analyze the impact of class distribution mismatch between the assumed class distribution (i.e., the exemplar set used for guidance) and the actual class distribution in the unlabeled data. To simulate this, we manually construct exemplar sets under three scenarios: (1) distribution aligned with the unlabeled generations, (2) uniform distribution across classes, and (3) reversed distribution relative the unlabeled data. As shown in Table 12, our method exhibits slight performance degradation under mismatched conditions, yet remains competitive across both datasets. This suggests that while alignment between exemplar and target class distributions is beneficial, our method is reasonably robust under class distribution mismatch.
F. Qualitative Results
Test input x'
GT
s(x)
Test input x'
GT
S(x)
Q: Can people turn into pillars of salt?
9: Which singer had a 2008 number one bit
A: No, people cannot salt.
pillars
0.8491
song A: Rihanna.
0.9031
Mhat the biggest city the U.S.
x
what is the offepring and tigress called? liger.
1ion
with only one word in its name?
0.1431
0.7931
A: los Angeles.
Mhat happens watermelon seeds? you
giraffe
0.1002
roys-r-us uses as itt' mascot? Toya-r-ua
0.1201
waternelons.
gizaffe
its' mascot.
TruthfulQA
(b) TriviaQA
Figure 6. Qualitative examples from (a) TruthfulQA and (h) TriviaQA, We compare the truthfulness scores S(x') across different test inputs x'. green checkmark indicates ground truth labeled as truthful, while cross denotes ground truth labeled as hallucinated.
We present qualitative examples of the model's truthfulness score, S(x'), for various input query and generated text pairs, Using questions sampled from (a) TruthfulQA and (b) TriviaQA, generate responses with the LLaMA-3.1188 model. As illustrated in Figure our approach accurately assigns scores that align with the truthfulness of the answers, demonstrating the effectiveness of the method.
G. Embedding Norms
We model the last-token embeddings at the final layer using a hyperspherical distribution with unit norm. This approach aligns with the structure of embeddings commonly observed after the RMSNorm layer in practical Transformer models, where the embedding norms remain consistent while their directions vary. These characteristies be naturally characterized by the von Mises-Fisher (vMF) distribution, which we employ represent the probability distribution in the MLE objective in Equation (3). To validate our modeling, we visualize the L2 norms of the last-token embeddings at the final layer for the pre- pre-trained LLaMA-3.1 --11-- 1-8 (Figure 7) and Qwen-2.5-7b (Figure 8). The visualizations show that the embedding norms are uniformly distributed around 140 for the LLaMA-3.1 model and around 300-330 for the 2.5572 model, supporting the validity of our modeling approach.
18

---

Steer LLM Latents for Hallucination Detection
TruthfulQA
TriviaOA
SciQ
NQ Open
162 143
163
160
16]
143
140
1.0
L2 norm
120
100
norm
1.20
80
Data
Data
Data
Data
Linear fitting
Linear fitting
Linear fitting
Linear fitting
10: 203 325 400 539
605
700 900
2010
8200 1000
2002 17200
200
400
500
200
1200
200
2003
1519
2005
2015
3595 2010
Figure L.2 norms last token embeddings the final layer from LLaMA-3.1-8b
TruthfulQA
TriviaQA
SciQ
NQ Open
602
400
253
250
La nom
300
250
5 220
Data
Dala
Data
Data
Linear fitting
Linear fitting
Linear fitting
Linear fitting
10:
203
325 40p 519 00
700 503
2010
2100 1000
2002 17700
200
400
500
200
1200
200
1007
15/0
1005
2015
3:92 2010
Figure L.2 norms of last token embeddings at final layer from Qwen-2.5-7h.
H. Compute Resources and Time
H.1. Software and hardware
We conducted all experiments using Python 3.8.15 and PyTorch 2.3.1 (Paszke et al., 2019) on NVIDIA A100 GPUs. For evaluation with GPT-4o, we utilized the OpenAl API.
H.2. Training and inference time
Based on tracked runs, the estimated total training and inference time is notably low: approximately 0.1 GPU-hours for LLMMA-3.1-8b and Qwen-2.5-77, 0.2 GPU-hours for Qwen-2.5-14b, and I GPU-hours for 1LLMA-3311706 These highlight the computational efficiency of our approach, achieving practical training and inference time even for large-scale models.
To further contextualize this, we compare the wall-clock time for training and inference computed the same split of TruthfulQA with LLaMA-3.1-8hh as shown in Figure 9. We evaluate three hallucination detection methods requiring training: HaloScope (Du et al., 2024), SAPLMA (Azaria & Mitchell, 2023), and TSV (Ours); and one training-fre free method: Semantic Entropy (Kuhn et al., 2023).
85.
Semantic Entropy
AUROC (%)
HaloScope
SAPLMA
All methods are tested using the same software and hardware setup. and runtime is measured after completing the sampling process. While TSV incurs slightly higher computational costs compared HaloScope, it achieves a significant performance improvement of 13.6%. Furthermore, TSV demonstrates superior performance compared to the fully-supervised method: SAPLMA & achieving both lower computational and annotation costs. Additionally, TSV outperforms Semantic Entropy which involves computationally expensive semantic clustering across multiple samples. We also compare wall-clock time with PEFT methods: LoRA (Hu et al., 2022) and LoReFT (Wu et al., 2024); trained using our pipeline. Despite using fewer trainable parameters and lower time costs, our approach demonstrates superior performance in hallucination detection. These results demonstrate TSV's effectiveness as a high-performen hallucination detection method that balances detection performance, computational efficiency, and annotation costs, offering flexibility across different cost budgets.
70
LoRA
TSV [Ours)
65.
10
2 3x
Wall-clock time (log-scale)
Figure 9. AUROC and wall-clock time for training and inference.
I. Limitations and Future Work
Fine-grained hallucination detection. While our method focuses on sentence-level hallucination detection, practical applications often demand identifying hallucinated spans at the token or phrase level to provide more interpretable
19

---

Steer LLM Latents for Hallucination Detection
explanations. Achieving this requires adapting TSV to reason over hidden states finer granularity across token positions, which poses technical challenges. A promising future direction is to focus salient commonn sources of hallucinations (Yeh et al., 2025>-and apply TSV before and after each entity span. By measuring shifts in hidden representations or detection scores, one could potentially localize hallucinations in a fine-grained way.
Long-form QA. This work focuses on short-form QA, which remains a challenging setting. We adopt this setup to ensure fair comparisons with existing benchmarks. However, real-world applications often require complex, long-form answers. natural extension is to decompose long-for generation into multiple short QA pairs and verify each pair individually. This reframes the task hallucination detection over a of short QA pairs, where TSV can be directly applied.
20


Reviews:
[{"review_id": "BkOmAMVXda", "reviewer": "Reviewer_iT9L", "summary": "The paper introduces the Truthfulness Separator Vector (TSV), a lightweight and flexible steering vector designed to reshape the latent space of Large Language Models (LLMs) during inference to enhance the separation between truthful and hallucinated outputs without modifying model parameters. TSV is trained using a two-stage framework that first learns from a small set of labeled exemplars and then augments this set with pseudo-labeled LLM generations using an optimal transport-based algorithm.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> A1. Steering vector for LLMs\n\nThank you for your insights! While it is true that the steering vectors have been explored in LLMs, our key contribution lies in how we specifically design them for hallucination detection, as recognized by you and Reviewers iT9L and tv5M. Sections 4.2 and 4.3 illustrate the novel algorithm that leverages both a few labeled and unlabeled LLM generations with an optimal transport-based pseudo-labeling framework for training. To our knowledge, this is the **first work** to formulate hallucination detection in this way. \n\nMoreover, we emphasize that **our contribution lies not only in the specific method but also in the perspective shift we offer to the hallucination detection community**, which has largely relied on fixed pre-trained embeddings—despite their limitations. Our work challenges this status quo by demonstrating that actively steering the latent space—even with a lightweight vector—can substantially improve detection performance. We believe this perspective shift is **fundamentally significant, as it opens up a new design space** and encourages the community to look beyond pre-trained embeddings toward representation shaping as a core principle.\n\n> A2. Evaluation with other metrics\n\nThank you for the suggestion! In addition to AUROC, we report AUPRC, F1 score, precision, and recall at the generation level, consistent with prior works. We include results on TruthfulQA with LLaMA-3.1-8b, where we found that **our method maintains a consistent advantage across all metrics**.\n\n| Method  | AUROC | AUPRC|  F1 | Precision (Pos) | Precision (Neg) | Recall |\n|-|--|-|-|-|-|-|\n| HaloScope  | 70.2       | 56.2 |64.0  | 55.6| 78.2| 75.3|\n| SAPLMA |  78.2       | 63.3  |64.7       |  57.7| 85.8| 73.4|\n| **Ours** |  **84.2**       | **76.2**  | **70.9**       | **64.5**|**88.5**| **78.9**\n\n> A3-1. Robustness to pseudo-label noise\n\nThank you for bringing this up! As requested, we include the robustness analysis of pseudo-label noise for the selected unlabeled data ($K=128$). Specifically, with the same selected dataset on TruthfulQA, we gradually add more noise to their pseudo labels by manually flipping some of the correct labels. The resulting relationship between the pseudo-label noise ratio and the hallucination detection AUROC on LLaMA-3.1-8b is shown as follows, where our method remains relatively robust under increasing noise conditions.\n\n| Pseudo-label Noise Ratio (%)  |5 (no flipping, original)  |10 |15 | 20 | 25|\n|-|-|-|-|-|-|\n| Hallucination Detection AUROC (%)| 84.2 |83.8| 82.6 | 82.2 | 81.3|\n\n> A3-2. Quality of exemplar set, and how does the content of exemplars impact the results?\n\nAnother insightful point! To understand the quality, we randomly sample the exemplar set using different seeds and annotate them w.r.t. factuality. Manual inspection confirms that the exemplars are high-quality and cover diverse content. The TruthfulQA dataset contains 817 questions across 38 categories with an imbalanced distribution, making it a meaningful testbed for examining how content affects performance. To further verify, we construct exemplar sets on LLaMA-3.1-8b using three strategies, with imbalance ratio $\\gamma$ defined as the ratio of the most to least frequent category counts.\n\n| Setting  | AUROC | AUPRC|  F1 |\n|-|-|-|-|\n| Uniform ($\\gamma=1$) |     84.9    | 77.0 |  71.0    | \n| Random (Ours, $\\gamma=5$) |   84.2  | 76.2 | 70.9 |\n| Biased ($\\gamma=10$)|   83.7    | 75.0 |  69.3    | \n\n **Even in cases where the selected exemplars are biased toward a particular content, the test performance remains consistent.** This reinforces the robustness of our approach.\n \nWe emphasize that our approach achieves strong performance with a very small labeled exemplar set (e.g., 32), **which makes manual curation and quality control highly feasible in practice**. Since the cost of labeling such a small set is minimal, our work offers a reliable and efficient solution for hallucination detection in real-world applications.\n\n>A4. Hyperparameters\n\nWe emphasize that TSV requires minimal hyperparameter tuning, making it practical and easy to deploy. Key hyperparameters (e.g., steering strength) are **lightweight to tune** and we provide default settings (see Table 6 in Appendix) that **generalize well across all models and datasets**. \n\nIn fact, all experiments in our paper use hyperparameters selected using the validation set of TruthfulQA with LLaMA-3.1-8b, and these settings are **applied uniformly without re-tuning to all experiments**, while still achieving state-of-the-art performance.\n\nMoreover, our ablation in Figure 3b demonstrates that TSV’s performance is **not overly sensitive** to the choice of hyperparameters. The robustness of TSV is further supported by our generalization results in Figure 4, which show consistent performance across diverse data distributions, even with fixed hyperparameters. This significantly reduces the burden of hyperparameter tuning in real-world use.", "claims_and_evidence": "N/A", "methods_and_evaluation_criteria": "Strength:\n\n1. The paper presents a novel and lightweight steering vector method that enhances hallucination detection in LLMs without modifying model parameters. \n\n2. Unlike prior work that relies on costly fine-tuning or extensive labeled datasets, TSV reshapes the latent space dynamically during inference while maintaining model flexibility.\n\n3. The paper provides rigorous experimental validation, demonstrating state-of-the-art hallucination detection on multiple benchmark datasets.\n\nWeakness:\n\n1. A trainable steering vector is not a novel concept for LLMs. The main idea of TSV is to model the binary classification problem of hallucination detection using the von Mises-Fisher distribution and then treat the mean direction of each cluster as a trainable steering vector. While the overall performance is promising, this approach essentially combines existing methods in a new problem setting rather than introducing a fundamentally new technique.\n\n2. The paper primarily uses AUROC as the evaluation metric for hallucination detection. Could you also provide precision (both positive and negative classes), recall, and F1 scores at either the generation level (non-hallucination vs. hallucination) or the token level?\n\n3. TSV heavily relies on the quality of pseudo labels. It would be helpful to include a failure case analysis and assess its robustness to label noise. Additionally, how is the quality of the exemplar set evaluated, and how does the content of exemplars impact the final results?\n\n4. Although TSV avoids fine-tuning, it still requires hyperparameter tuning, which to some extent reduces its practicality in plug-and-play scenarios across diverse models.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "See above.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "## update after rebuttal\n\nMost of my concerns have been addressed. I would like to keep the current positive score for this work.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GlOYGbChHE", "reviewer": "Reviewer_tv5M", "summary": "This paper introduces the Truthfulness Separator Vector (TSV), a lightweight approach for hallucination detection in LLMs that reshapes the model's latent space during inference without modifying its parameters. The method employs a single trainable vector added to an intermediate layer, trained through a two-stage framework that first uses a small labeled exemplar set (as few as 32 examples) and then incorporates pseudo-labeled unlabeled data using an optimal transport algorithm with confidence-based filtering. Experiments across multiple datasets (TruthfulQA, TriviaQA, SciQ, NQ Open) demonstrate state-of-the-art performance, achieving +12.8% AUROC improvement on TruthfulQA while requiring only 0.00005% of model parameters (4K for LLaMA-3.1-8B). The approach shows strong generalization across datasets and model families (LLaMA and Qwen) at various scales (7B-70B parameters), performing comparably to fully supervised methods with significantly less labeled data and 8-800× fewer parameters than parameter-efficient fine-tuning alternatives.\n\n## update after rebuttal\nMost of my concerns have been addressed. I will remain my positive score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable feedback and insightful questions. Below, we provide detailed responses to each of your points. All the added experiments are performed with LLaMA-3.1-8b.\n\n>A1. Longer generations\n\nGreat point. In this work, we focus on short-form QA (phrase and sentence-level) which remains challenging, as evidenced by suboptimal performance reported in prior literature. We adopt this setting to ensure fair comparison with existing benchmark studies. That said, we fully agree that extending to long-form generation is a critical direction for real-world applications. We view our current work as a necessary step toward this goal.\n\nOne natural solution is to decompose the long-form generation into multiple QA pairs and verify each pair individually. This reframes the problem as hallucination detection over a set of QA pairs, where TSV can be applied. We believe this warrants a deeper investigation on its own, and we appreciate you highlighting it!\n\n\n> A2. Deployment in practice\n\nTSV is a lightweight, plug-and-play hallucination detection module that can be integrated with any production LLMs. We first perform a standard forward pass using the original model parameters (without TSV) to generate a response. Then, during a second pass, we apply TSV at an intermediate layer and compute a truthfulness score using the steered representation. It can be flexibly toggled on/off, preserving the original model behavior when hallucination detection is not required. We will clarify these deployment steps more explicitly in the revision.\n\n> A3. Class distribution mismatch\n\nThank you for bringing up this point! We manually construct exemplar sets under three different scenarios: (1) distribution aligned with the unlabeled generations, (2) uniform distribution, and (3) distribution reversed from the unlabeled generations. We observe a slight performance decline in scenarios with distribution mismatches, but the overall performance remains competitive.\n\n|Setting|TruthfulQA|SciQ|\n|-|-|-|\n|Aligned (Ours)|84.2|85.8| \n|Uniform|82.8|82.5| \n|Reversed|82.1|82.7| \n\n> A4. Extension to the span-level hallucination detection\n\nGreat question! We agree this is an exciting direction, which requires localizing exactly which tokens or phrases are hallucinated. This is particularly challenging, as it requires adapting TSV to separate hidden states at different token positions.\n\nOne promising direction is to identify salient entities—often the primary source of hallucinations—and apply TSV before and after each entity span, then measure the resulting shift in hidden states or detection scores to infer potential hallucinations at the corresponding span. These challenges require deeper investigation. But again, we agree this could significantly increase practical utility and will consider it in future work.\n\n\n> A5. Ablation on LoRA\n\n\nTSV is more effective for hallucination detection as it directly shapes the representation space while being more parameter-efficient. Additionally, we applied selective LoRA updates to specific model components within the same layer as TSV, and found these targeted adjustments still underperformed TSV, as shown below.\n\n|Method|TruthfulQA|SciQ|\n|-|-|-|\n|LoRA (q_proj) |72.0|72.2|  \n|LoRA (k_proj) |71.6|74.1|  \n|LoRA (v_proj) |76.7|76.8|  \n|LoRA (o_proj) |75.2|72.4|\n|**Ours**|**84.2**|**85.8**|\n\n> A6. Applying TSV to multiple locations\n\nWe appreciate your insights. We applied TSV to multiple layers or MHA components of LLaMA-3.1-8b on TruthfulQA and observed slight performance gains. However, a single-layer TSV already performs strongly, suggesting it is sufficient for effective hallucination detection and highlights the efficiency of our approach.\n\n|Layer|AUROC|MHA Components|AUROC| \n|-|-|-|-|\n|7, 13|84.2|Res, MLP |82.1|\n|8, 14|84.9|Res, Attn |83.3|\n|9, 12|84.7|MLP, Attn |85.2|\n|Ours (9th layer)|84.2|Ours (Residual)|84.2|\n\n> A7. Performance on the quantized model\n\nThank you for the great question. We experiment with an 8-bit quantized LLaMA-3.1-8b model$^1$ and observe a slight drop in performance. However, the overall results remain strong, indicating that our method is robust to quantization.\n\n|Model|TruthfulQA|SciQ|\n|-|-|-|\n|8-bit|84.1|84.9|\n|16-bit|84.2|85.8|\n\n$^1$ https://huggingface.co/docs/transformers/main/en/quantization/bitsandbytes\n\n\n> A8. Transferability for different models\n\nThank you for the suggestion! While we demonstrate that TSV performs well across various models and scales, we believe full transferability is unlikely. This is because differences in architecture, training framework, and pretraining data across models (e.g., LLaMA vs. Qwen) lead to **fundamentally different representation spaces**, making direct application of TSV challenging. However, a promising future direction could be to train an adapter that aligns the representations between models of different architectures or scales.", "claims_and_evidence": "The primary claims are well-supported by empirical evidence:\n- Superior performance claim: The authors demonstrate substantial improvements over state-of-the-art methods across multiple datasets. - The reported AUROC improvements (e.g., +12.8% on TruthfulQA compared to previous methods) are significant and convincing.\n- Minimal labeling requirement claim: The paper shows that their approach performs nearly as well with just 32 labeled examples as with full supervision. The ablation studies in Figure 3c provide sufficient evidence for this claim.\n- Lightweight and flexible design claim: The comparison with PEFT methods in Table 4 clearly demonstrates TSV's parameter efficiency, using 8-800× fewer parameters than alternatives while achieving superior performance.", "methods_and_evaluation_criteria": "The proposed methods are appropriate for the hallucination detection problem.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experimental designs are generally sound.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths: \n- The paper addresses a critical problem in LLM deployment with a pragmatic approach that doesn't require model fine-tuning.\n- The component analysis provides valuable insights into the contribution of each part of the system.\n\nWeakness:\n- The paper doesn't fully explore potential limitations of the approach, such as its behavior with longer-form content beyond QA pairs.\n- While the method shows good generalization across datasets, there's limited discussion of how it might behave across different model architectures or scales beyond the tested models.", "other_comments_or_suggestions": "Consider explaining in more detail how the method might be deployed in practice alongside a production LLM system.", "questions_for_authors": "1. How might the TSV approach scale to much longer generations beyond QA pairs? Many real-world hallucinations occur in longer contexts where there might be a mix of truthful and hallucinated content within a single generation.\n2. Have you explored whether the TSV learned from one model architecture (e.g., LLaMA) transfers to a different architecture (e.g., Qwen)? This would strengthen the claim about generalization capabilities.\n3. The optimal transport algorithm assumes a class distribution based on the exemplar set. How sensitive is the method to mismatches between the assumed and actual class distributions in real-world data?\n4. Could the TSV approach be extended to finer-grained hallucination detection (e.g., identifying specific hallucinated spans within otherwise truthful text)? This would significantly increase its practical utility.\n5. The results in Table 4 show TSV outperforming LoRA despite using far fewer parameters. Is this advantage primarily due to the low-data setting, or is there something inherently more effective about the TSV approach for hallucination detection? Would LoRA perform better if updates were restricted to specific components (e.g., only the o_proj weight in the 4th layer) rather than applied broadly across the model?\n6. The ablation studies examine different locations for applying TSV, but have you explored applying TSV to multiple locations simultaneously? For example, can the vector be added to multiple positions within the MHA architecture or to multiple layers? This might further enhance the separation capability.\n7. How is performance affected when using quantized models (e.g., 4-bit or 8-bit quantization) compared to fp16 models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4nb8ohSdPf", "reviewer": "Reviewer_EMXm", "summary": "The paper proposes a method named Truthfulness Separator Vector (TSV) to detect the hallucinations in LLMs. The TSV is a lightweight vector that reshapes the LLM's latent space during inference without altering model parameters. A two-stage training framework is used to explore TSV in a similar manner to a semi-supervised manner, which makes sense to me. Extensive experiments demonstrate that TSV performs well with minimal labeled data, exhibiting strong generalization across datasets.\n\n------------------------After rebuttal--------------\n\nMost of my concerns are well addressed, and therefore, I change my score to 4. \nHowever, I still think the steering method mentioned in A5 is worth trying, as it would further strengthen this paper's contributions to the community.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your thoughtful and valuable comments. Below, we provide detailed responses to each of your questions and comments.\n\n> A1. Optimal separability of the representation\n\nThank you for your insightful question! We acknowledge that there can indeed be failure cases where the representations of truthful and hallucinated contents overlap. For instance, in Table 1, the performance on the NQ Open dataset (76.1%) is relatively lower compared to other datasets, suggesting weaker separability. This is likely due to the inherently challenging QA task, which can make perfect separation harder.\n\n**We'd like to emphasize that our goal is not to guarantee perfect separability, but rather to improve the separability relative to the pre-trained model.** _This improvement holds consistently across all datasets_, even if the final representation space is not perfectly disentangled. As a result, our method enables more reliable deployment compared to prior approaches that rely on fixed pre-trained embeddings.\n\n> A2. $\\epsilon$ in Eqauation (7)\n\nThe value of $\\epsilon$ is defined in **Appendix  L602** where it is set to 0.05, following [1] that demonstrated its effectiveness. For improved clarity, we will move this detail to the main paper. Thank you for pointing this out!\n\n [1] Caron et al., \"Unsupervised Learning of Visual Features by Contrasting Cluster Assignments,\" NeurIPS 2020\n \n> A3. Different pseudo-label assignment strategies\n\nWe appreciate the reviewer for highlighting this important point. \n \nOur motivation for introducing the optimal transport (OT) is that we aim to generate pseudo-labels that align with the class distribution of unlabeled LLM generations, which is inherently **imbalanced**. Traditional pseudo-labeling methods [2,3] can suffer from class imbalance, leading to biased predictions—either toward majority classes or, in some cases, unstable predictions favoring minority classes. This leads to a **mismatch between the class distribution of pseudo-labels and the true underlying distribution**, which can degrade both pseudo-label quality and overall detection performance.\n \nIn contrast, our OT-based clustering approach introduces cluster-wise regularization through a constrained optimization framework, **ensuring that cluster sizes align with the underlying or estimated class distribution**. \n\nIn comparison, we evaluate two pseudo-labeling baselines—nearest centroid [2] and confidence thresholding [3] on LLaMA-3.1-8b using AUROC. We will include these comparisons and discussions in the main paper.\n\n| Method  | TruthfulQA | SciQ | \n|----|------------|------------|\n|Nearest centroid [2]|    82.2   | 81.8 |\n| Confidence thresholding [3]|    83.1   | 81.6\n| **Ours** |  **84.2**       |    **85.8**  |\n\n\n [2] Rebuffi et al., \"iCaRL: Incremental Classifier and Representation Learning,\" CVPR 2017\n\n [3] Sohn et al., \"FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence,\" NeurIPS 2020 \n\n \n> A4. References on LVLM latent space steering\n \nThank you for highlighting these interesting works in the LVLM field! We agree that comprehensively studying latent feature steering in LVLMs is also crucial, offering valuable insights that can inform both communities and help shape future directions. We are happy to include the following relevant works from the LVLM literature. We will also ensure a more thorough literature review to identify any additional related studies.\n\n[4] Liu et al., \"Reducing Hallucinations in Vision-Language Models via Latent Space Steering,\" ICLR 2025\n\n[5] Yang et al., \"Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection,\" CVPR 2025\n\n[6] Chen et al., \"ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object\nHallucination in Large Vision-Language Models,\" CVPR 2025\n\n[7] Duan et al., \"TruthPrInt: Mitigating LVLM Object Hallucination Via Latent Truthful-Guided Pre-Intervention,\" ArXiv 2025\n\n \n> A5. Can TSV mitigate hallucination?\n  \nThank you for the interesting question. TSV is designed to learn a representation space specifically for detection (i.e., classification), which is fundamentally different from mitigation (i.e., generation). As a result, it is not straightforward to directly apply a detection-trained TSV to a mitigation task.\n\nHowever, TSV offers a plug-and-play mechanism for pre-trained LLMs. In practice, one can first apply TSV to classify truthful vs. hallucinated outputs from unlabeled generations in the wild, and then leverage these predictions to learn a steering vector for the mitigation task [8]. **This perspective opens up a promising direction for hallucination mitigation under unsupervised/limited-label settings, where extensive supervision is often impractical**.\n\n[8] Li et al., \"Inference-Time Intervention: Eliciting Truthful Answers from a Language Model,\" NeurIPS 2023", "claims_and_evidence": "Supporting by pervious study in Haloscope [NeurIPS'24], I think the two-stage method to explore the truthfulness separator vector can work in hallucination detection tasks.", "methods_and_evaluation_criteria": "Overall, the paper is well-written and easy to follow. The proposed hallucination detection pipeline makes sense. In all the evaluated datasets, the truthful and untruthful latent representations can all be separated well. Are there failure cases when the latent representations overlap? In other words, does the separation assumption for truthful and untruthful features always hold?", "theoretical_claims": "The $\\epsilon$ in (7) seems not be defined.", "experimental_designs_or_analyses": "I think most of the experimental designs are sound and valid. While I think it will be better to discuss and briefly evaluate different methods for pseudo-label assignments. The current experiments seem to lack a proper baseline model.", "relation_to_broader_scientific_literature": "This research intersects with the broader scientific literature on LLM reliability. It builds on existing hallucination detection methods by addressing the limitation of default embeddings. Demonstrating TSV's effectiveness across datasets validates the approach of steering latent spaces, providing a new direction for improving LLM trustworthiness in real-world applications.", "essential_references_not_discussed": "I think most of the listed related works are quite essential for understanding the contributions of this paper.\n\nWhile there are some other works in Large vision language models (LVLMs) that seem to use a similar approach of studying latent features in LVLM between positive and negative samples to adjust the model's behavior: \ne.g.\n\n[1] Reducing hallucinations in vision-language models via latent space steering, 2024.\n\n[2] Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection, 2024.\n\nThese papers are only on arXiv, from what it seems, but I still suggest searching for related papers using feature steering in hallucination-related issues in LVLM fields.", "other_strengths_and_weaknesses": "I suggest that the authors further strengthen the argument that using label assignment via optimal transport is necessary for the proposed method, which is a key contribution of this paper. Generally, there can be a series of techniques that can achieve the label assignment for TSV. Why can using other methods hinder the overall detection performance? \n\nProviding a brief discussion will make it easier for readers to make the main contribution.", "other_comments_or_suggestions": "See previous parts.", "questions_for_authors": "Does the learned TSV be used to steer activation for hallucination mitigation, as in [1]?\n\n[1] Inference-time intervention: Eliciting truthful answers from a language model.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a lightweight approach for LLM hallucination detection which reshapes the model's latent space during inference without modifying its parameters. Reviewers acknowledged the contribution and strong performance of the proposed method, while initially raised concerns regarding its difference to prior work and missing important experiments.

After the rebuttal, the authors addressed most of the concerns, and all reviewers agreed to accept this paper. AC read all the reviews, author rebuttals, and the paper, and believes this is a strong paper and recommends for acceptance.
Decision: Accept (poster)