Venue: NAACL 2025

# Token-Level Density-Based Uncertainty Quantification Methods for

Eliciting Truthfulness of Large Language Models

Artem Vazhentsev2,3 Lyudmila Rvanova4,6 Ivan Lazichny3

Alexander Panchenko2,3 Maxim Panov1 Timothy Baldwin1,5 Artem Shelmanov1

###### Abstract

Uncertainty quantification (UQ) is a prominent approach for eliciting truthful answers from large language models (LLMs). To date, information-based and consistency-based UQ have been the dominant UQ methods for text generation via LLMs. Density-based methods, despite being very effective for UQ in text classification with encoder-based models, have not been very successful with generative LLMs. In this work, we adapt Mahalanobis Distance (MD) - a well-established UQ technique in classification tasks - for text generation and introduce a new supervised UQ method. Our method extracts token embeddings from multiple layers of LLMs, computes MD scores for each token, and uses linear regression trained on these features to provide robust uncertainty scores. Through extensive experiments on eleven datasets, we demonstrate that our approach substantially improves over existing UQ methods, providing accurate and computationally efficient uncertainty scores for both sequence-level selective generation and claim-level fact-checking tasks. Our method also exhibits strong generalization to out-of-domain data, making it suitable for a wide range of LLM-based applications.

languageresourceLanguage

and proven to be useful for selective text classification (Vazhentsev et al., 2022, 2023a). Despite being computationally lightweight, these techniques often outperform more resource-intensive methods, such as deep ensembles (Lakshminarayanan et al., 2017) and Monte Carlo dropout (Gal and Ghahramani, 2016; Tsymbalov et al., 2018). Unfortunately, the reported performance of density-based scores for text generation so far has been notably low (Vashurin et al., 2024).

Recent work has demonstrated that the internal states of LLMs carry a lot of information about their uncertainty (Azaria and Mitchell, 2023; Chen et al., 2024; He et al., 2024b; CHWang et al., 2024; Vazhentsev et al., 2024). These techniques train a supplementary model on top of the activations of LLM internal layers. However, they often rely on simplistic features and fail to incorporate more advanced, well-established density-based UQ methods, limiting their ability to capture uncertainty.

In this work, we address this gap by adapting density-based techniques for the UQ of LLMs and propose a new supervised method based on density-based features. Specifically, we adapt one of the most robust methods for UQ in the classification tasks, namely Mahalanobis Distance (MD; Lee et al. (2018)), and train a linear regression on top of the MD scores from various layers of the LLM. These features are supplemented with a probability of the generated sequence. Figure 1 illustrates the scheme of the proposed supervised UQ method. Our extensive experimental evaluation demonstrates that the proposed method provides substantial improvement over the state of the art.

Our key **contributions** are as follows.

* We conduct a comprehensive investigation of density-based UQ methods for LLMs. While previous research (Vashurin et al., 2024) has indicated that sequence-level density-based methods are ineffective, we propose a token-level adaptation of MD that is on par with or better than state-of-the-art UQ techniques.
* We propose a new computationally efficient supervised method for UQ in LLMs using layer-wise density-based scores as features to improve uncertainty estimation without sacrificing the performance.
* We conduct a vast empirical investigation that demonstrates the effectiveness of the proposed methods for sequence-level selective classification across eleven datasets and claim-level fact-checking.

## 2 Related Work

Many effective UQ methods, such as deep ensembles (Lakshminarayanan et al., 2017) and Monte Carlo (MC) dropout (Gal and Ghahramani, 2016), require sampling multiple predictions from a model, which leads to substantial computational and memory overheads. A key challenge in UQ is developing techniques that balance effectiveness with computational efficiency. Among the most promising approaches in this regard are density-based methods (Lee et al., 2018; Liu et al., 2020; van Amersfoort et al., 2020; Kotelevskii et al.,

Figure 1: An illustration of the proposed method. After each decoder layer, the embeddings of each generated token are extracted. Subsequently, we compute the Mahalanobis distance for each token and layer and then average over all tokens in the generated sequence. Finally, we train a linear regression on the PCA decomposition of the calculated features with the addition of sequence probability to predict the uncertainty of the generation.

 

2022; Yoo et al., 2022). These methods leverage latent representations of instances to model the training data distribution, then estimate how likely a new instance belongs to that distribution. Lee et al. (2018) propose to use Mahalanobis Distance (MD) as a measure of uncertainty for out-of-distribution detection in computer vision tasks. Podolskiy et al. (2021) adapt MD to out-of-distribution in text classification tasks. Vazhentsev et al. (2022, 2023b) show that it also provides high performance in selective text classification.

For LLMs, Fomicheva et al. (2020) and Kuhn et al. (2023) proposed UQ methods that sample multiple predictions and leverage their diversity. In the context of black-box LLMs, where we have no access to the logits or embeddings of a model, Fomicheva et al. (2020) propose the use of lexical dissimilarity of sampled texts as a measure of uncertainty. Lin et al. (2023) leverage a similarity matrix between responses for deeper analysis of the diversity of the sampled generations. Some methods also combine sampling diversity measures with the probability of each generation (Kuhn et al., 2023; Duan et al., 2024; Nikitin et al., 2024; Cheng and Vlachos, 2024; Chen et al., 2024; Vashurin et al., 2025).

Recently, it was demonstrated that MD is an efficient approach for out-of-distribution detection in sequence-to-sequence models (Vazherntsev et al., 2023b; Ren et al., 2023; Darrin et al., 2023). However, for selective generation tasks, density-based methods so far have substantially underperformed compared to trivial baselines (Vashurin et al., 2024).

Supervised methods are another research direction for UQ of LLMs. Azaria and Mitchell (2023) demonstrate that the internal states of the model contain information about uncertainty, and propose to train a multi-layer perceptron over the hidden LLM representation to predict the truthfulness of the model responses. He et al. (2024b) enhance this idea by training a deep neural network with recurrent and convolutional layers. Furthermore, this method uses embeddings from all layers and incorporates features based on the probability and the dynamics of the generated tokens through layers. In contrast to these methods, we employ a simple linear model, but focus on more accurate feature extraction from internal layers, using well-established density-based UQ methods.

## 3 Background on Density-Based Methods

Recently, Mahalanobis distance (MD) and Robust Density Estimation (RDE) were adapted (Vazhentsev et al., 2023b; Ren et al., 2023) to the text generation task by considering the marginal distribution of the training dataset.

Following the assumption of a Gaussian distribution of training instance representations, the MD method (Lee et al., 2018) calculates a centroid of the training data \(\mu\) and the empirical covariance matrix \(\Sigma\). For a given instance \({\bf x}\), the uncertainty score is defined as the Mahalanobis distance:

\[U^{\text{MD}}({\bf x},l)=(h_{l}({\bf x})-\mu)^{T}\Sigma^{-1}(h_{l}({\bf x})- \mu),\]

where \(h_{l}({\bf x})\) is a hidden representation extracted from the layer \(l\).

RDE (Yoo et al., 2022) operates within the reduced dimensionality of \(h_{l}({\bf x})\) via the kernel PCA decomposition. To ensure the robustness of the covariance matrix, it uses the Minimum Covariance Determinant estimate (Rousseeuw, 1984). Finally, the uncertainty score is computed as the Mahalanobis distance in the reduced dimensionality.

Ren et al. (2023) proposed a modification of MD - Relative Mahalanobis Distance (RMD). It takes into account a background contrastive MD score. The score aims to assess how close the test instance is to the in-domain training data compared to the background data. The uncertainty score based on RMD is given by the following equation:

\[U^{\text{RMD}}({\bf x},l)=U^{\text{MD}}({\bf x},l)-U^{\text{MD}}_{0}({\bf x}, l),\]

where \(U^{\text{MD}}_{0}({\bf x},l)\) is a Mahalanobis distance computed with the centroid \(\mu^{0}\) and the empirical covariance matrix \(\Sigma^{0}\) calculated using the background dataset, such as C4 (Raffel et al., 2020).

For the sequence-to-sequence tasks, it was proposed to use the last encoder and decoder layer for extracting hidden representation of the model (Vazhentsev et al., 2023b; Ren et al., 2023). In contrast, recent research (Azaria and Mitchell, 2023; Chen et al., 2024) indicates that the middle layers of the model may be more suitable for decoder-only models.

## 4 Proposed Method: Token-Level Mahalanobis Distance

To define the method, we assume access to training data consisting of a set of prompts paired with 

LLM responses, each accompanied by an assessment of its correctness. The assessment can be based on ground truth answers (as in tasks like question-answering, machine translation, or summarization) or through alternative means, such as human annotation or another big LLM.

Layer-Wise Uncertainty Score Embedding extraction.First of all, we need to extract embeddings of instances in the training dataset. We note that previous works use sequence-level embeddings, which are essentially an average of token-level embeddings. Recent studies Azaria and Mitchell (2023); Chen et al. (2024) note that sequence embeddings might be useless for UQ with LLMs and propose to use embeddings of the last or the first generated token, as they encode useful information for estimating the truthfulness of the entire generation. We acknowledge that this property may not always hold, as the informative tokens are likely to vary depending on the specific task. In our method, we first compute individual token-level uncertainty scores and then aggregate them into a sequence-level score.

Embedding selection.To construct a covariance matrix and centroid for MD, a model training set is required. However, unlike standard text classification tasks, where training sets are typically limited and accessible during the development of an ML-based application, the pre-training data for general-purpose LLMs is extremely large and usually not publicly available. Moreover, even if this data were available, LLM performance on it would likely be not homogeneous and could be low for certain tasks. Therefore, to construct the parameters for MD, we propose selecting a subset of token embeddings from high-quality LLM responses.

From the responses generated in the training set, we select a subset of token embeddings that correspond to responses that meet a defined correctness criterion. Let \({\mathcal{T}}\) be a training set of input prompts and \(|{\mathcal{T}}|=N_{|{\mathcal{T}}|}\). For each prompt \({\mathbf{x}^{j}}\in{\mathcal{T}}\), the model generates a response as a sequence \(\tilde{\mathbf{y}}^{j}=\mathbf{t}_{1}^{j},\ldots,\mathbf{t}_{N_{j}}^{j}\), where \(N_{j}\) is a length of the \(j\)-th generation and \(\mathbf{t}_{i}^{j},i\in[1,\ldots,N_{j}]\) is an \(i\)-th token in the response. We define a set of selected tokens as \({\mathcal{D}}=\{\mathbf{t}_{i}^{j}:{\mathcal{Q}}(\tilde{\mathbf{y}}^{j})>\tau, i\in[1,\ldots,N_{j}],j\in[1,\ldots,N_{|{\mathcal{T}}|}]\}\), where \({\mathcal{Q}}(\cdot)\) is a quality metric and \(\tau\) is a given threshold. Then \({\mathcal{E}}_{l}=\{h_{l}(\mathbf{t})\colon\mathbf{t}\in{\mathcal{D}}\}\) is the set of selected token embeddings. The correctness criterion helps filter out low-quality responses. Depending on the dataset in the experiment, exact match and AlignScore are employed as quality metrics. The correctness criterion used for token selection is described in Section 5.1.

Layer-wise scores.For each layer \(l=1,\dots,L\) of the model, we compute the covariance matrix \(\Sigma_{{\mathcal{E}}_{l}}\) and the centroid \(\mu_{{\mathcal{E}}_{l}}\) using the set of selected token embeddings \({\mathcal{E}}_{l}\). For each token from the generated sequence \(\tilde{\mathbf{y}}^{k}=\mathbf{t}_{1}^{k},\ldots,\mathbf{t}_{N_{k}}^{k}\), we compute the layer-wise MD as follows:

\[U^{\text{MD}}(\mathbf{t}_{i}^{k},l)\!=\!\big{(}h_{l}(\mathbf{t}_{i}^{k})-\mu_{ {\mathcal{E}}_{l}}\big{)}^{T}\Sigma_{{\mathcal{E}}_{l}}^{-1}\big{(}h_{l}( \mathbf{t}_{i}^{k})-\mu_{{\mathcal{E}}_{l}}\big{)}.\]

For the token-level RMD, we additionally compute the background covariance matrix \(\Sigma_{l}^{0}\) and the background centroid \(\mu_{l}^{0}\) using the embeddings of all generated tokens for the input prompts from some background dataset.

Finally, the uncertainty score of the entire generated sequence \(\tilde{\mathbf{y}}^{k}\) is the _Average Token-level Mahalanobis Distances (ATMD)_ over \(\mathbf{t}_{i}^{k},i=1,\ldots,N_{k}\) (for RMD, we designate it as ATRMD).

### Linear Regression on Layer-Wise Scores

The ATMD and ATRMD scores can be computed on various layers. Azaria and Mitchell (2023) indicate that the best-performing layer may vary depending on the generation task. To effectively integrate information from multiple layers, we propose training a regression model on top of the layer-wise scores.

For a generation \(\tilde{\mathbf{y}}^{k}\), we construct a vector of features based on ATMD or ATRMD: \(f^{*}(\tilde{\mathbf{y}}^{k})=[U^{*}(\tilde{\mathbf{y}}^{k},1),\ldots,U^{*}( \tilde{\mathbf{y}}^{k},L)]\) (we use \(*\) instead of ATMD or ATRMD). To learn the uncertainty of the generation, we define target variables as negative values of a quality metric for generations \(\tilde{\mathbf{y}}^{k}\): \(\mathbf{q}^{k}=-{\mathcal{Q}}(\tilde{\mathbf{y}}^{k})\). We note that the features \(f^{*}(\tilde{\mathbf{y}}^{k})\) might be highly correlated with each other (a multicollinearity problem; Shrestha (2020)), which makes linear models to overfit Chan et al. (2022). To make our features more robust, we use top \(N=10\) components from the PCA decomposition of feature vectors: \(\tilde{f^{*}}(\tilde{\mathbf{y}}^{k})=\operatorname{PCA}_{N}\bigl{(}f^{*}( \tilde{\mathbf{y}}^{k})\bigr{)}\).

We train the machine learning model \(G(\cdot)\) to predict an uncertainty score as follows:

1. Split the entire training dataset \({\mathcal{T}}\) into two parts \({\mathcal{T}}_{1}\) and \({\mathcal{T}}_{2}\).
2. Using \({\mathcal{T}}_{1}\), construct \({\mathcal{E}}_{l},l\in[1,\ldots,L]\) and fit layer-wise covariance matrices and centroids. ATRMD also fits layer-wise background covariance matrices and background centroids 

3. For each generation \({\tilde{\mathbf{y}}^{k}},k=1,\ldots,|\mathcal{T}_{2}|\) for the prompts from \(\mathcal{T}_{2}\), compute features \(\hat{f}^{*}({\tilde{\mathbf{y}}^{k}})\) and targets \(\mathbf{q}^{k}\).
4. Train the machine learning model \(G^{*}(\cdot)\) to predict the targets \(\mathbf{q}^{k}\) using the features \(\tilde{f}^{*}({\tilde{\mathbf{y}}^{k}})\), \(k=1,\ldots,|\mathcal{T}_{2}|\). In our work, we use linear regression models as \(G^{*}(\cdot)\).
5. Re-estimate layer-wise parameters of the distribution using the entire training dataset \(\mathcal{T}\).

Finally, the supervised uncertainty score for a test generation \({\tilde{\mathbf{y}}^{k}}\) based on token-level MD or RMD, namely SATMD or SATRMD is:

Following He et al. (2024b), we also experiment with adding the sequence probability \(P({\tilde{\mathbf{y}}^{k}}\mid\mathbf{x}^{k})\) as an additional feature to the features vector: \(\tilde{f}^{*}{}_{prob}({\tilde{\mathbf{y}}^{k}})=[\tilde{f}^{*}({\tilde{ \mathbf{y}}^{k}});P({\tilde{\mathbf{y}}^{k}}\mid\mathbf{x}^{k})]\), and get

### Hybrid Score

In addition, we explore Hybrid Uncertainty Quantification (HUQ; Vazhentsev et al. (2023a)), which empirically combines multiple uncertainty scores. Using HUQ, we combine sequence probability \(U_{1}({\tilde{\mathbf{y}}^{k}})=1-P({\tilde{\mathbf{y}}^{k}}\mid\mathbf{x}^{ k})\) and the proposed SATMD or SATRMD scores: \(U_{2}({\tilde{\mathbf{y}}^{k}})=U^{S^{*}}({\tilde{\mathbf{y}}^{k}})\). The hyperparameters of HUQ are tuned on the \(\mathcal{T}_{2}\) dataset. A detailed description of the HUQ method is given in Appendix B.

## 5 Experiments

### Experimental Setup

For the experimental evaluation, we employ the LM-Polygraph framework Fadeeva et al. (2023); Vashurin et al. (2024). We consider two tasks: (1) sequence-level selective generation Ren et al. (2023), in which we can "reject" untruthful generations based on provided uncertainty; (2) claim-level fact-checking Fadeeva et al. (2024), where we aim to identify nonfactual claims in long generations, consisting of several claims.

Metrics.To evaluate the quality of UQ methods on the selective generation task, we use the standard Prediction Rejection Ratio (PPR) metric Malinin and Gales (2021); Vashurin et al. (2024). This metric measures the correctness of the ranking of generations based on uncertainty relative to a specified quality metric. PRR computes the area under the Prediction Rejection (PR) curve, which is constructed by sequentially rejecting the most uncertain generation and calculating the average quality for all stored generations at each possible threshold. Subsequently, this area is normalized by scaling between the PR curve for the random selection and oracle selection. A higher value of the PRR corresponds to a better quality of selective generation. Following previous work Vashurin et al. (2024), we use ROUGE-L, Accuracy, and AlignScore Zha et al. (2023) as text generation quality metrics.

For claim-level fact-checking, we follow previous work Fadeeva et al. (2024) and consider this task as a binary classification problem. We utilize the ROC-AUC and PR-AUC metrics, where nonfactual claims represent a positive class.

Models.For selective generation, we experiment with two state-of-the-art LLMs in their size: Llama@8b v3.1 Dubey et al. (2024) and Gamma 9b v2 Riviere et al. (2024). For fact-checking, we utilize Mistral 7b v0.1 Naruct Jiang et al. (2023). The inference hyperparameters are presented in Table 9 in Appendix F.

Datasets.We consider several text generation tasks, including text summarization (TS), question-answering (QA) with long free-form answers, QA based on reading comprehension, QA with short free-form answers, and multiple-choice QA. Dataset statistics are presented in Table 8 in Appendix E. For TS, we utilize XSum Narayan et al. (2018), SamSum Gliwa et al. (2019), and the CNN/DailyMail See et al. (2017) dataset. For QA with long free-form answers, we use PubMedQA Jin et al. (2019), MedQUAD Abacha and Demner-Fushman (2019), TruthfulQA Lin et al. (2022), and GSM8k Cobbe et al. (2021). For reading comprehension, we use CoQA Reddy et al. (2019) and SciQ Welbl et al. (2017). For QA with short free-form answers, we use TriviaQA Joshi et al. (2017). The last three datasets represent the common benchmarks for evaluating UQ methods in previous work Kuhn et al. (2023); Duan et al. (2024); Lin et al. (2023). For multiple-choice QA, we utilize MMLU Hendrycks et al. (2021), which is a common dataset for evaluating LLMs.

UQ baselines.In an experimental evaluation, we compare the proposed methods against several UQ baselines, including trivial yet robust information-based methods such as Maximum Sequence Probability (MSP) and Perplexity Fonicheva et al. ( 

2020), and consistency-based methods considered state-of-the-art for LLMs (Vashurin et al., 2024): Lexical Similarity based on ROUGE-L (Fomicheva et al., 2020), black-box methods (DegMat, Eccentricity, EigValLaplacian; Lin et al. (2023)), Semantic Entropy (Kuhn et al., 2023), and Shifting Attention to Relevance (SAR; Duan et al. (2024)). Furthermore, to ensure the robustness of the proposed methods, the suite of baselines in our experiments also includes methods that utilize model internal states: Factoscope (He et al., 2024b), SAPLMA (Azaria and Mitchell, 2023), and EigenScore (Chen et al., 2024). The first two are supervised methods, while EigenScore is unsupervised. Following the previous works (Azaria and Mitchell, 2023; Chen et al., 2024), we use embeddings from the middle layer of the model for the latter two methods. For the methods that require sampling, we sample five generations for each input text.

Configuration of ATMD/ATRDMInDomain experimental evaluation on the SciQ, CoQA, TriviaQA, MMLU, and GSM8k datasets, we select token embeddings used to construct the covariance matrix and centroids for ATMD and ATRMD from generations that are fully accurate according to the exact match criterion. On other datasets, we utilize generations with AlignScore greater than 0.3. Raw ATMD/ATRMD denotes a corresponding method without selecting embeddings using the correctness criterion.

### Results

Layer-wise comparison of density-based methods.Figure 2 presents the layer-wise comparison of various sequence-level density-based approaches for selective generation for the Llama 8b v3.1 and Gemma 9b v2 models. These results demonstrate the presence of robust patterns across the majority of datasets and models.

Consistent with the findings of Vashurin et al. (2024), we observe that in most cases, density-based methods that use sequence-level embeddings (MD, RMD, and RDE) yield PRR scores that are close to or below zero, indicating performance comparable to random selection. Only for GSM8k, these methods provide meaningful uncertainty scores, but they still do not outperform the basic MSP baseline. Furthermore, we see that using sequence-level embeddings derived from internal layers does not improve the performance of density-based methods; they usually perform better when using embeddings from the last layer.

MD that uses token-level embeddings performs consistently better than the MD based on sequence-level embeddings for all datasets except SamSum,

\begin{table}
\begin{tabular}{l|c c c c c c c c c c c c c c c c c} \hline \multirow{2}{*}{**COQM-Model**} & \multicolumn{5}{c|}{**XMM**} & \multicolumn{5}{c|}{**Attention**} & CNN & \multicolumn{5}{c|}{**Database**} \\  & & & & & & & & & & & & & & & & & & \\  & **biggCC** & **Aggression** & **BRECAL** & **Agression** & **BRECAL** & **Aggression** & **BRECAL** & **Aggression** & **BRECAL** & **Aggression** & **BRECAL** & **Aggression** & **BRECAL** & **Aggression** & **BRECAL** & **Aggression** & **BRECAL** & **Aggression** & **BRECAL** & **Aggression** & **BRECAL** & **Aggression** & **BRECAL** \\ \cline{2-16}  & 170 & 160 & 200 & 250 & 600 & 500 & 217 & 100 & 401 & 500 & 200 & 300 

where all methods perform similarly to each other. Moreover, density-based methods that compute MD using token-level embeddings from internal layers outperform those that rely on embeddings from the top layers. While for SamSum and MMLU with the Gemma 9b v2 model, ATMD achieves the best performance using the last layer embeddings, for all other cases the best performance is achieved by using embeddings from the middle layers. This finding is consistent with previous research Azaria and Mitchell (2023); Chen et al. (2024).

Using the selection of correct generations from the training dataset for fitting the covariance matrix and centroid is key to achieving good performance of the methods based on token-level MDs. ATMD/ATRMD consistently outperform raw ATMD/ATRMD. The highest difference was observed on the MedQUAD and TruthfulQA datasets, where using selection improved PRR by 0.2-0.3.

Comparison of sequence-level aggregationsTables 1 and 5 in Appendix A.1 present the comparison of various sequence-level supervised approaches for selective generation for the Llama 8b v3.1 and Gemma 9b v2 models. The results demonstrate that SATMD and SARTMDM provide stable and robust performance, which is often superior or equal to the performance of the MD/RMD using the embeddings from the best layer. As anticipated, the incorporation of MSP as an additional feature or combining it using HUQ significantly improved the performance of SATMD/SARTMDM. While on average by mean rank, the best performance across various datasets was achieved by SATMD+MSP, for XSum and SciQ, HUQ-SATRMD exhibited a slight improvement. It is also noteworthy that using RMD led to a consistent improvement in performance compared to the original MD for all variants of the supervised method.

Main results on the selective generation tasksThe main results on the selective generation tasks for the Llama 8b v3.1 and Gemma 9b v2 models are presented in Tables 2 and 6 in Appendix A.2. In the summarization task, our supervised methods outperform all the baselines on XSum and SamSum. HUG-SATRMD achieves the best performance on the XSum and SamSum datasets in terms of PRROUGE-L. For SamSum, the SARTMD+MSP method significantly outperforms other methods in terms of PRR-AlignScore. For the CNN dataset, the proposed methods demonstrate the second-best results in terms of PRR-ROUGE-L, but they substantially fall behind unsupervised UQ techniques in terms of PRR-AlignScore.

In the QA tasks with long answers (PubMedQA, MedQUAD, TruthfulQA, and GSM8k), SARTMD+MSP consistently demonstrates the best performance, with a notable margin over best supervised and unsupervised techniques, while HUQ-SATRMD ranks second. In the reading comprehension task, HUQ-SATRMD is the best-performing method. Meanwhile, on the MMLU dataset, SARTMD+MSP is the most effective method, significantly outperforming HUQ-SATRMD and other state-of-the-art baselines.

Considering QA with short answers on CoQA, we observe that HUQ-SATRMD performs on par with the MSP baseline, while on the SciQ dataset performs the best with a large margin. On TriviaQA, SARTMD+MSP outperforms the MSP baseline, underperforming only sampling-based methods that require much more computation time.

Overall, we can conclude that HUQ-SATRMD is the most effective method for summarization and reading comprehension tasks, where it significantly outperforms state-of-the-art unsupervised UQ methods. For all other QA datasets, including those with long answers and tasks requiring internal knowledge, the best performance is demonstrated by SARTMD+MSP.

\begin{table} \end{table}
Table 2: Main results on selective generation tasks. PRR† for Llama 8b v3.1 model for various tasks for the considered sequence-level methods. Warmer color indicates better results.

 

Out-of-domain generalization.Table 3 presents a comparison of various sequence-level methods for the selective generation task for the Llama 8b v3.1 model in the out-of-domain settings. We train and evaluate supervised methods using a leave-one-out approach: train on all datasets except one, which is left for testing. For each evaluation dataset, the training set is composed of 400 instances sampled from each of the remaining datasets. We use the negative AlignScore generation quality metric as a target for all considered datasets in this setting.

The performance of supervised methods in the out-of-domain setting shows a significant decline compared to the in-domain setting. Despite this, HUQ-SATRMD achieves the best results according to the mean rank, outperforming unsupervised state-of-the-art methods across the majority of datasets and metrics, except MSP on SamSum and MedQUAD in terms of PRR-ROUGE-L. Notably, when testing on the MMLU dataset, the training data consists of texts from summarization tasks and free-form QA, which differ significantly from the multiple-choice QA format used in MMLU. Nevertheless, the strong performance on MMLU demonstrates the potential of our supervised method HUQ-SATRMD for broad generalization.

Other supervised methods, including SATRMD+MSP and the baselines, show significantly poorer results in the out-of-domain setting. SATRMD+MSP underperforms MSP on several datasets, including SamSum, MedQUAD, and TruthfulQA. SAPLMA and Factoscope are not able to provide meaningful uncertainty scores, lagging significantly behind unsupervised UQ methods.

Fact-checking results.Table 4 presents a comparison of various claim-level methods for the fact-checking task using the Mistral 7b v0.1 Instruct model. The baseline supervised method SAPLMA performs similarly to a random choice. Our method SATRMD provides meaningful uncertainty scores slightly outperforming Maximum Claim Probability (MCP). We note that CCP, like MCP, is also based on the probabilities derived from the model output but demonstrates better performance than MCP. Consequently, we combine CCP with SATRMD to provide more effective claim-level fact-checking. The results demonstrate that HUQ-SATRMD achieves the best results in terms of ROC-AUC, outperforming CCP by 3.4%, while in terms of PR-AUC, SATRMD+CPP is the best, outperforming CCP by 2.6%. These results demonstrate that the proposed SATRMD methods are effective not only for sequence-level uncertainty quantification but also for estimating uncertainty on the claim level.

Impact of training data size.Figure 5 in Appendix A.3 illustrates the dependency of the performance of supervised UQ methods on the size of the training data. As expected, the optimal results on all datasets are achieved when the maximum number of training instances is used. Nevertheless, for all datasets, except SamSum and MedQUAD, the results obtained with 200-500 training instances are only slightly lower than with 2,000-5,000 instances. Furthermore, even with fewer than 200 training instances for MedQUAD, GSM8k, and MMLU, HUQ and SATRMD+MSP are able to substantially outperform the MSP method. These results demonstrate the robustness of the proposed methods with respect to the size of the training dataset.

Impact of the correctness threshold.Figure 3 presents the dependence of the performance of the SATRMD+MSP and HUQ-SATRMD methods on the correctness threshold used for the embedding selection for computing the centroid and covariance matrix for MD.

The results demonstrate that the proposed methods are generally not sensitive to the correctness threshold and consistently show high performance. However, for the MedQUAD dataset, we can see

\begin{table}
\begin{tabular}{l|c c} \hline \multirow{2}{*}{**UP Method**} & \multicolumn{2}{c}{**Mistral Tb.**} \\  & **ROC-AUC** & **PR-AUC** \\ \hline Maximum Climb Probability & 629 & 271 \\ RTNet & 638 & 276 \\ CTC & **716** & **358** \\ SARTML & 489 & **166** \\ \hline SATRMD & 447 & 275 \\ HIO-SATRMD & **278** & **410** \\ SATRMD+CCP & **279** & **448** \\ \hline \end{tabular}
\end{table}
Table 4: Results in fact-checking. ROC-AUC\(\uparrow\) and PR-AUC\(\uparrow\) for the MIST 7b v0.1 Instruct model for fact-checking for the considered claim-level methods. Warmer color indicates better results.

\begin{table}
\begin{tabular}{l r r} \hline \multirow{2}{*}{**Up Model**} & \multicolumn{2}{c}{**Mutual Tb.**} \\  & **ROC-AUC** & **PR-AUC** \\ \hline Maximum Climb Probability & 629 & 271 \\ RTNet & 638 & 276 \\ CTC & **716** & **358** \\ SARTML & 489 & **166** \\ \hline SATRMD & 447 & 275 \\ HIO-SATRMD & **278** & **410** \\ SATRMD+CCP & **279** & **448** \\ \hline \end{tabular}
\end{table}
Table 3: Out-of-domain generalization. PRR\(\uparrow\) for Llama 8b v3.1 for selective generation tasks for the considered sequence-level methods in the out-of-domain setting. Warmer color indicates better results.

 

the results with a threshold of 0.3 are significantly better than those with other thresholds. Specifically, lower thresholds (e.g., 0.1) result in selecting the embeddings corresponding to incorrect instances, while higher thresholds (e.g., 0.8) exclude some embeddings associated with correct instances. Both scenarios result in suboptimal estimation of the centroid and covariance matrix, leading to a slight degradation in overall performance.

**Impact of the number of PCA components.** Figure 4 illustrates the impact of the number of PCA components used for the features of linear regression on the performance of SATRMD+MSP and HUQ-SATRMD methods. The best performance is achieved with 10 or 20 components for most datasets. Only for CNN and TruthfulQA, using just 2 components yields slightly better results than using more. Overall, these results indicate that our choice of 10 components is well-balanced on average across multiple datasets. We also observe that results with more than five PCA components remain stable across all datasets, showing minimal variation. Therefore, methods based on RMD are not sensitive to the precise choice of the number of PCA components.

Computational efficiency.Table 7 in Appendix C summarizes the average runtime per instance for each UQ method, along with the percentage overhead compared to standard LLM inference. State-of-the-art UQ methods that require sampling from the LLM multiple times (Semantic Entropy, SAR, Lexical Similarity) introduce a huge computational overhead (315-700%). In contrast, the proposed methods HUQ-SATRMD and SATRMD+MSP introduce minimal overhead (5.3-7.6%), which makes them much more practical.

## 6 Conclusion

We have introduced a series of new supervised UQ methods based on layer-wise features derived from the Mahalanobis distance. We show that calculating MD over token-level embeddings yields much better results than previous attempts that leverage sequence-level embeddings. Training a linear regression on top of the layer-wise scores allows us to produce even better uncertainty scores and outperform the state-of-the-art supervised and unsupervised UQ methods in selective classification across eleven datasets and in claim-level fact-checking. We also show that the proposed methods are computationally efficient and have the potential for generalization, which makes them useful in real-world LLM-based applications.

In future work, we aim to improve the generalization capabilities of the supervised UQ methods on out-of-domain data by investigating new features and a more robust training pipeline.

Figure 4: Dependency of PRR\({}^{\uparrow}\) of the SATRMD+MSP and HUQ-SATRMD methods on the number of the PCA components for the features of linear regression for the Llama 8b v3.1 model. Higher values indicate better results.

Figure 3: Dependency of PRR\({}^{\uparrow}\) of the SATRMD+MSP and HUQ-SATRMD methods on the correctness threshold for the embedding selection for the centroid and covariance matrix for MD for the Llama 8b v3.1 model. Higher values indicate better results.

 



### Limitations

Our approach is supervised, which means that its performance depends on the quality and size of the data available for supervision. We evaluated the robustness of the approach to dataset variation, which demonstrates that the method does not significantly degrade its quality compared to the target dataset. Nevertheless, we observe certain performance drops; thus, the resulting UQ method should be used with care beyond the supervision domain.

We did not test our method on very large LLMs, such as LLAMA 3 70b, as we were limited to using 7-9b models due to constraints in our computational resources.

## Ethical Considerations

In our study, we focused on open-source LLMs and datasets that are not designed to produce harmful content. However, LLMs can still generate potentially harmful texts that may impact various groups. Uncertainty quantification techniques offer a way to enhance the reliability of neural networks and can even be used to detect harmful outputs, though this is not our focus.

Although our proposed method shows substantial performance improvements, it may sometimes incorrectly flag safe and accurate generated text as having high uncertainty. While we explicitly benchmarked the method on robustness to the task change, its applicability across various tasks remains limited.

## Acknowledgments

We thank the anonymous reviewers for their valuable suggestions, which significantly contributed to improving this paper.

## References

* B. Abacha and D. Demner-Fushman (2019)A question-entailment approach to question answering. BMC Bioinform.. External Links: Link Cited by: SS4.
* A. Azaria and T. Mitchell (2023)The internal state of an llm knows when it's lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, pp. 967-976. External Links: Document Cited by: SS5.
* J. Baan, N. Daheim, E. Ilia, D. Ulmer, H. Li, R. Fernandez, B. Plank, R. Sennrich, C. Zerva, and W. Aziz (2023)Uncertainty in natural language generation: from theory to applications. arXiv preprint arXiv:2307.15703. Cited by: SS1.
* S. Ch-Wang, B. Van Durme, J. Esner, and C. Kedzie (2024)Do androids know they're only dreaming of electric sheep?. In Findings of the Association for Computational Linguistics ACL 2024, Bangkok, Thailand and virtual meeting, pp. 4401-4420. External Links: Link, Document Cited by: SS1.
* J. Chan, S. Leow, K. Bea, W. Khuen Cheng, S. Phoong, Z. Hong, and Y. Chen (2022)Mitigating the multicollinearity problem and its machine learning approach: a review. Mathematics10, pp. 1283. Cited by: SS1.
* C. Chen, K. Liu, Z. Chen, Y. Gu, Y. Wu, M. Tao, Z. Fu, and J. Ye (2024)INSIDE: llms' internal states retain the power of hallucination detection. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, External Links: Link Cited by: SS1.
* Y. Chen, Q. Fu, Y. Yuan, Z. Wen, G. Fan, D. Liu, D. Zhang, Zhixu Li, and Y. Xiao (2023)Hallucination detection: robustly discerning reliable answers in large language models. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Vol. , pp. 245-255. External Links: ISBN 978-1-4503-387X, Link, Document Cited by: SS1.
* J. Cheng and A. Vlachos (2024)Measuring uncertainty in neural machine translation with similarity-sensitive entropy. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), St. Julian's Malta, Stockholm, Sweden, pp. 2115-2128. External Links: Link, Document Cited by: SS1.
* K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman (2021)Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: SS1.
* M. Darrin, P. Piantaniola, and P. Colombo (2023)RainProof: an umbrella to shield text generator from out-of-distribution data. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, pp. 5831-5857. External Links: Link, Document Cited by: SS1.
* J. Duan, H. Cheng, S. Wang, A. Zavalny, C. Wang, R. Xu, B. Kailkura, and K. Xu (2024)Shifting attention to relevance: towards the predictive uncertainty quantification of free-form large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand, pp. 5050-5063. External Links: Link, Document Cited by: SS1.
* A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman,

Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aboo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Himsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corine Wong, Cristian Canton Ferrer, Cyrus Nikolaids, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Neil, Greggior Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johntsun, Joshua Saxe, Junteng Jia, Kalyan Vasdden Ahvala, Kartikeya Upsani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. 2024. The llama 3 herd of models. _arXiv preprint arXiv:2407.21783_.
* Dairi et al. (2022) Nouha Dairi, Sivan Milton, Mo Yu, Osmar Zaiane, and Siva Reddy. 2022. On the origin of hallucinations in conversational models: Is it the datasets or the models? In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5271-5285, Seattle, United States. Association for Computational Linguistics.
* Fadeeva et al. (2024) Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, and Maxim Panov. 2024. Fact-checking the output of large language models via token-level uncertainty quantification. In _Findings of the Association for Computational Linguistics: ACL 2024_. Association for Computational Linguistics.
* Fadeeva et al. (2023) Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveto Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, and Artem Shelmanov. 2023. LM-Polygraph: Uncertainty estimation for language models. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_, pages 446-461.
* Farquhar et al. (2024) Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. _Nature_, 630(8017):625-630.
* Feng et al. (2024) Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidishisa Balachandran, and Yulia Tsvetkov. 2024. Don't hallucinate, abstain: Identifying LLM knowledge gaps via multi-LLM collaboration. In _Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 14664-14690, Bangkok, Thailand. Association for Computational Linguistics.
* Fomicheva et al. (2020) Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frederic Blain, Francisco Guzman, Mark Fishel, Nikolos Aletras, Vishrav Chaudhry, and Lucia Specia. 2020. Unsupervised quality estimation for neural machine translation. _Transactions of the Association for Computational Linguistics_, 8:539-555.
* Gal and Ghahramani (2016) Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In _Proceedings of The 33rd International Conference on Machine Learning_, volume 48 of _Proceedings of Machine Learning Research_, pages 1050-1059, New York, New York, USA. PMLR.
* Geng et al. (2024) Jiahui Geng, Fengu Cai, Yukia Wang, Heinz Koeppl, Preslav Nakov, and Iryna Gurevych. 2024. Survey of confidence estimation and calibration in large language models. In _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 6577-6595, Mexico City, Mexico. Association for Computational Linguistics.
* Gliwa et al. (2019) Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019. SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization. In _Proceedings of the 2nd Workshop on New Frontiers in Summarization_, pages 70-79, Hong Kong, China. Association for Computational Linguistics.
* He et al. (2024a) Jianfeng He, Linlin Yu, Shuo Lei, Chang-Tien Lu, and Feng Chen. 2024a. Uncertainty estimation on sequential labeling via uncertainty transmission. In _Findings of the Association for Computational Linguistics: NAACL 2024_, pages 2823-2835, Mexico City, Mexico. Association for Computational Linguistics.
* He et al. (2020) Jianfeng He, Xuchao Zhang, Shuo Lei, Zhiqian Chen, Fanglan Chen, Abdulaziz Alhamadani, Bei Xiao, and Chang-Tien Lu. 2020. Towards more accurate uncertainty estimation in text classification. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_, pages 8362-8372. Association for Computational Linguistics.
* He et al. (2024b) Jinwen He, Yujia Gong, Zijin Lin, Cheng'an Wei, Yue Zhao, and Kai Chen. 2024b. LLM factoscope: Uncovering LLMs' factual discernment through measuring inner states. In _Findings of the Association for Computational Linguistics ACL 2024_, pages 10218-10230, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics 

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
* Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lelio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothee Lacroix, and William El Sayed. 2023. Mistral 7b. _arXiv preprint arXiv:2310.06825_.
* Jin et al. (2019) Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. PubMedQA: A dataset for biomedical research question answering. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2567-2577, Hong Kong, China. Association for Computational Linguistics.
* Joshi et al. (2017) Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1601-1611, Vancouver, Canada. Association for Computational Linguistics.
* Kotelevskii et al. (2022) Nikita Kotelevskii, Aleksandr Artemenkov, Kirill Fedyanin, Fedor Noskov, Alexander Fishkov, Artem Shelmanov, Artem Vazhentsev, Aleksandr Petiushko, and Maxim Panov. 2022. Nonparametric uncertainty quantification for single deterministic neural network. In _Advances in Neural Information Processing Systems_.
* Kuhn et al. (2023) Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. In _The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023_.
* Lakshminarayanan et al. (2017) Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In _Advances in Neural Information Processing Systems_, volume 30. Curran Associates, Inc.
* Lee et al. (2018) Kimin Lee, Kubok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In _Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montreal, Canada_, volume 31, pages 7167-7177.
* Lin et al. (2022) Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 3214-3252, Dublin, Ireland. Association for Computational Linguistics.
* Lin et al. (2023) Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2023. Generating with confidence: Uncertainty quantification for black-box large language models. _Transactions on Machine Learning Research_.
* Liu et al. (2020) Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balajii Lakshminarayanan. 2020. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. In _Advances in Neural Information Processing Systems_, volume 33, pages 7498-7512. Curran Associates, Inc.
* Malinin and Gales (2021) Andrey Malinin and Mark J. F. Gales. 2021. Uncertainty estimation in autoregressive structured prediction. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
* Manakul et al. (2023) Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 9004-9017.
* Min et al. (2023) Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Tyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FACTScore: Fine-grained atomic evaluation of factual precision in long form text generation. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 12076-12100.
* November 4, 2018_, pages 1797-1807. Association for Computational Linguistics.
* Nikitin et al. (2024) Alexander Nikitin, Jannik Kossen, Yarin Gal, and Pekka Martinen. 2024. Kernel language entropy: Fine-grained uncertainty quantification for llm s from semantic similarities. _arXiv preprint arXiv:2405.20003_.
* OpenAI et al. (2020) OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altaman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcon, Paul Ballesccu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jakke Berdine, Gabriel Bernatet-Shapiro, Christopher Bernard, Lenny Bogdonoff, Ole Boiko, Madeline Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittainy 

Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Aity Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simon Posada Fishman, Juston Forte, Isabella Schrof, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gonitoj-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidelberg, Chris Hesse, Alan Hickey, Wade Hickery, Peter Hoeschelle, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhu Jin, Denny Jin, Shinno Jomoto, Billie John, Heewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lae, Anna Makaniu, Kim Malfa'cian, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, and Luke Metz et al. 2024. Gpt-4 technical report. _Preprint_, arXiv:2303.08774.
* Podolskiy et al. (2021) Alexander Podolskiy, Dmitry Lipin, Andrey Bout, Ekaterina Artemova, and Irina Piontkovskaya. 2021. Revisiting mahalanobis distance for transformer-based out-of-domain detection. In _Proceedings of the AAAI Conference on Artificial Intelligence_, pages 13675-13682.
* Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. _J. Mach. Learn. Res._, 21:140:1-140:67.
* Reddy et al. (2019) Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: A conversational question answering challenge. _Transactions of the Association for Computational Linguistics_, 7:249-266.
* Ren et al. (2023) Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter J Liu. 2023. Out-of-distribution detection and selective generation for conditional language models. In _The Eleventh International Conference on Learning Representations_.
* Rivierre et al. (2018) Morgane Rivierre, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Leonard Hussenat, Thomas Mesward, Bobak Shahriari, Alexandre Rame, Johan Ferret, Pietru Duya Tafit, Abe Friessen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor, Jean-Bastian Grill, Behnam Neyshabur, Olivier Bachem, Alanna Walton, Aliakesei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Cohen, Anthony Laforge, Antonia Paterson, Benjamin Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chinutu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette-Choo, Danila Sinopallikov, David Weinberger, Dimple Vijaykumar, Dominika Rogozinska, Dustin Herbison, Eliasa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Elthyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Klimczak-Plucinska, Harleen Batra, Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Iping Zhou, Joan Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost van Amersoort, Josh Gordon, Josh Lipschitz, Josh Newman, Ju-yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sedhia, Kish Greene, Lars Lowe Sjosund, Lauren Usui, Laurent Sifre, Lena Heuermann, Lictia Lago, and Lilly McNelius. 2024. Gamma 2: Improving open language models at a practical size. _arXiv preprint arXiv:2408.00118_.
* Rousseeuw (1984) Peter J Rousseeuw. 1984. Least median of squares regression. _Journal of the American statistical association_, 79(388):871-880.
* See et al. (2017) Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1073-1083, Vancouver, Canada. Association for Computational Linguistics.
* Shelmanov et al. (2021) Artem Shelmanov, Evgenii Tsymbalov, Dmitri Puzyrev, Kirill Fedyanlin, Alexander Panchenko, and Maxim Panov. 2021. How certain is your transformer? In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 1833-1840, Online. Association for Computational Linguistics.
* Shrestha (2020) Noora Shrestha. 2020. Detecting multicollinearity in regression analysis. _American Journal of Applied Mathematics and Statistics_, 8:39-42.
* Tsymbalov et al. (2018) Evgenii Tsymbalov, Maxim Panov, and Alexander Shapeev. 2018. Dropout-based active learning for regression. In _Analysis of Images, Social Networks and Texts: 7th International Conference, AIST 2018, Moscow, Russia, July 5-7, 2018_, pages 247-258 

Joost van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. 2020. Uncertainty estimation using a single deep deterministic neural network. In _Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event_, volume 119 of _Proceedings of Machine Learning Research_, pages 9690-9700. PMLR.
* Vashurin et al. (2024) Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev, Lyudmila Rvanova, Akim Tsvigun, Daniil Vasilev, Rui Xing, Abdelrahman Boda Sadallah, Kirill Grishchenkov, Sergey Petrakov, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov, and Artem Shelmanov. 2024. Benchmarking uncertainty quantification methods for large language models with lm-polygraph. _arXiv preprint arXiv:2406.15627_.
* Vashurin et al. (2025) Roman Vashurin, Maiya Goloburda, Preslav Nakov, Artem Shelmanov, and Maxim Panov. 2025. Cocoa: A generalized approach to uncertainty quantification by integrating confidence and consistency of llvm outputs. _arXiv preprint arXiv:2502.04964_.
* Vazhentsev et al. (2024) Artem Vazhentsev, Ekaterina Fadeeva, Rui Xing, Alexander Panchenko, Preslav Nakov, Timothy Baldwin, Maxim Panov, and Artem Shelmanov. 2024. Unconditional truthfulness: Learning conditional dependency for uncertainty quantification of large language models. _arXiv preprint arXiv:2408.10692_.
* Vazhentsev et al. (2022) Artem Vazhentsev, Gleb Kuzmin, Artem Shelmanov, Akim Tsvigun, Evgeni Tysplakov, Kirill Fedyanin, Maxim Panov, Alexander Panchenko, Gleb Gusev, Mikhail Burtsev, Manvel Avetisian, and Leonid Zhukov. 2022. Uncertainty estimation of transformer predictions for misclassification detection. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 8237-8252, Dublin, Ireland. Association for Computational Linguistics.
* Vazhentsev et al. (2023a) Artem Vazhentsev, Gleb Kuzmin, Akim Tsvigun, Alexander Panchenko, Maxim Panov, Mikhail Burtsev, and Artem Shelmanov. 2023a. Hybrid uncertainty quantification for selective text classification in ambiguous tasks. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 11659-11681, Toronto, Canada. Association for Computational Linguistics.
* Vazhentsev et al. (2023b) Artem Vazhentsev, Akim Tsvigun, Roman Vashurin, Sergey Petrakov, Danil Vasilev, Maxim Panov, Alexander Panchenko, and Artem Shelmanov. 2023b. Efficient out-of-domain detection for sequence to sequence models. In _Findings of the Association for Computational Linguistics: ACL 2023_, pages 1430-1454, Toronto, Canada. Association for Computational Linguistics.
* Wang et al. (2022) Yuxia Wang, Daniel Beck, Timothy Baldwin, and Karin Verspoor. 2022. Uncertainty estimation and reduction of pre-trained models for text regression. _Transactions of the Association for Computational Linguistics_, 10:680-696.
* Welbl et al. (2017) Johannes Welbl, Nelson F. Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. In _Proceedings of the 3rd Workshop on Noisy Usergenerated Text_, pages 94-106, Copenhagen, Denmark. Association for Computational Linguistics.
* Xiao and Wang (2021) Yijun Xiao and William Yang Wang. 2021. On hallucination and predictive uncertainty in conditional language generation. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 2734-2744, Online. Association for Computational Linguistics.
* Xin et al. (2021) Ji Xin, Raphael Tang, Yaoliang Yu, and Jimmy Lin. 2021. The art of abstention: Selective prediction and error regularization for natural language processing. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 1040-1051, Online. Association for Computational Linguistics.
* Yoo et al. (2022) KiYoon Yoo, Jangho Kim, Jiho Jang, and Nojun Kwak. 2022. Detection of adversarial examples in text classification: Benchmark and baseline via robust density estimation. In _Findings of the Association for Computational Linguistics: ACL 2022_, pages 3656-3672, Dublin, Ireland. Association for Computational Linguistics.
* Zha et al. (2023) Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. 2023. AlignScore: Evaluating factual consistency with a unified alignment function. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 11328-11348, Toronto, Canada. Association for Computational Linguistics.
* Zhang et al. (2019) Xuchao Zhang, Fanglan Chen, Chang-Tien Lu, and Naren Ramakrishnan. 2019. Mitigating uncertainty in document classification. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 3126-3136, Minneapolis, Minnesotaota. Association for Computational Linguistics.

 



## Appendix A Additional Experimental Results

### Comparison of Sequence-Level Aggregations

### Dependency on the Size of the Training Dataset

Figure 5 presents the results when varying the size of the training dataset for the supervised methods. We train the linear regression model on the training datasets of size: 100, 200, 500, 1000, 2000, and additionally on a training dataset of 5000 instances for SciQ and MMLU. Since the TruthfulQA dataset consists of only 817 instances, of which we use 409 instances as the test subset, we train linear regression on the training datasets of sizes: 100, 200, and 408.

## Appendix B Hybrid Uncertainty Quantification

We combine the sequence probability \(U_{1}(\tilde{\mathbf{y}}^{k})=1-P(\tilde{\mathbf{y}}^{k}\mid\mathbf{x}^{k})\) with the SATMD and SARTMD methods \(U_{2}(\tilde{\mathbf{y}}^{k})=U^{\mathsf{S*}^{*}}(\tilde{\mathbf{y}}^{k})\). For a given \(\mathcal{T}_{1}\) and \(\mathcal{T}_{2}\) from Section 4.2, and trained SARTMD method, we fit HUQ hyperparameters on the \(\mathcal{T}_{2}\) set.

Following Vazhentsev et al. (2023a), we define the set of in-distribution instances from \(\mathcal{T}_{2}\) as follows: \(\mathcal{T}_{\text{ID}}=\left\{\mathbf{x}\in\mathcal{T}_{2}\colon U_{2}( \mathbf{x})\leq\delta_{\min}\right\}\). We define the set of arbitrary in-distribution instances using \(\left\{\mathbf{x}\colon U_{2}(\mathbf{x})\leq\delta_{\min}\right\}\) and ambiguous in-distribution instances \(\mathcal{X}_{\text{IDA}}=\left\{\mathbf{x}\!\in\!\mathcal{X}_{\text{ID}}\colon U _{1}(\mathbf{x})>\delta_{\max}\right\}\) using \(\delta_{\min}\), \(\delta_{\max}\) are thresholds selected on the \(\mathcal{T}_{2}\) dataset.

To make different uncertainty scores comparable, we define a ranking function \(R(\mathbf{u},\mathfrak{D})\) as a rank of \(\mathbf{u}\) over a sorted dataset \(\mathfrak{D}\), where \(\mathbf{u}_{1}>\mathbf{u}_{2}\) implies \(R(\mathbf{u}_{1},\mathfrak{D})>R(\mathbf{u}_{2},\mathfrak{D})\). We compute the total



uncertainty \(U_{\text{T}}(\mathbf{x})\) as a linear combination \(U_{\text{T}}(\mathbf{x})=\)\((1-\alpha)R(U_{2}(\mathbf{x}),\mathcal{T}_{2})+\alpha R(U_{1}(\mathbf{x}),\mathcal{ T}_{2})\), where \(\alpha\) is a hyperparameter selected on the \(\mathcal{T}_{2}\) dataset. As a result, we define HUQ as follows:

\[U_{\text{HUQ}}(\mathbf{x})=\begin{cases}R(U_{1}(\mathbf{x}),\mathcal{T}_{\text {ID}}),\forall\mathbf{x}\in\mathcal{X}_{\text{ID}}\setminus\mathcal{X}_{\text {AID}},\\ R(U_{1}(\mathbf{x}),\mathcal{T}_{2}),\forall\mathbf{x}\in\mathcal{X}_{\text{ AID}},\\ U_{\text{T}}(\mathbf{x}),\forall\mathbf{x}\notin\mathcal{X}_{\text{ID}}.\end{cases}\]

## Appendix C Computational Efficiency

## Appendix D Computational Resources

All experiments were conducted on a cluster with 6 NVIDIA H100 GPUs. The total time for all conducted experiments for all models across all datasets is approximately 400 GPU hours.
 



## Appendix F Inference Hyperparameters

\begin{table}
\begin{tabular}{l|c|c|c|c|c|c|c|c|c|c} \hline \hline
**Dataset** & **Task** & **Max Input Length** & **Generation Length** & **Temperature** & **Top-p** & **Do Sample** & **Beams** & **Repetition Penalty** \\ \hline XSum & TS & & & 56 & & & & & & \\ SamSum & TS & & & 128 & & & & & & \\ CNN & & & & 128 & & & & & & \\ PubMedQA & & & & 128 & & & & & & \\ MedQUAD & QA & & & 128 & & & & & & \\ TruthfulQA & Long answer & & & 128 & 1.0 & 1.0 & False & 1 & 1 & 1 \\ GSM8k & & & & 256 & & & & & \\ CoQA & QA & & & 20 & & & & & \\ SciQ & Short answer & & & 20 & & & & & \\ Trivia & MMLU & MCQA & & 3 & & & & & \\ \hline \hline  & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \\ \end{tabular}
\end{table}
Table 9: Text generation hyperparameters for all LLMs used in the experiments.