Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities

Authors: alexander  nikitin
jannik  kossen
yarin  gal
pekka  marttinen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 16:57:56

Abstract: Uncertainty quantification in Large Language Models (LLMs) is crucial for applications where safety and reliability are important. In particular, uncertainty can be used to improve the trustworthiness of LLMs by detecting factually incorrect model responses, commonly called hallucinations. Critically, one should seek to capture the model's semantic uncertainty, i.e., the uncertainty over the meanings of LLM outputs, rather than uncertainty over lexical or syntactic variations that do not affect answer correctness.
To address this problem, we propose Kernel Language Entropy (KLE), a novel method for uncertainty estimation in white- and black-box LLMs. KLE defines positive semidefinite unit trace kernels to encode the semantic similarities of LLM outputs and quantifies uncertainty using the von Neumann entropy. It considers pairwise semantic dependencies between answers (or semantic clusters), providing more fine-grained uncertainty estimates than previous methods based on hard clustering of answers. We theoretically prove that KLE generalizes the previous state-of-the-art method called semantic entropy and empirically demonstrate that it improves uncertainty quantification performance across multiple natural language generation datasets and LLM architectures.

Main paper: 
 introduction: 
 Large Language Models (LLMs) have demonstrated exceptional capabilities across a wide array of natural language processing tasks [58, 66, 69] . This has led to their application in many domains, including medicine [11] , education [32] , and software development [40] . Unfortunately, LLM generations suffer from so-called hallucinations, commonly defined as responses that are "nonsensical or unfaithful to the provided source content" [26, 18, 52] . Hallucinations pose significant risks when LLMs are deployed to high-stakes applications, and methods that reliably detect them are sorely needed.A promising direction to improve the reliability of LLMs is estimating the uncertainty of model generations [36, 13, 51, 44, 23] . For instance, high predictive uncertainty is indicative of model errors or hallucinations in settings such as answering multiple-choice questions [30] . This allows us to prevent harmful outcomes by abstaining from prediction or by consulting human experts. However, the best means of estimating uncertainty for free-form natural language generation remains an active research question. The unique properties of LLMs and natural language preclude the application of established methods for uncertainty quantification [20, 39, 45, 59, 55] .A particular challenge is that language outputs can contain multiple types of uncertainty, including lexical (which word is used), syntactic (how the words are ordered), and semantic (what a text means). For many problems, semantic uncertainty is the desired quantity, as it pertains directly to the accuracy of the meaning of the generated response. However, measuring the uncertainty of the generation via token likelihoods conflates all types of uncertainty. To address this, Kuhn et al. [36] have recently introduced semantic entropy (SE), which estimates uncertainty as the predictive entropy of generated texts with respect to clusters of identical semantic meaning (we discuss this in more detail in Sec. 2).[inp] Which scientists contributed to probability theory?Figure 1 : Illustration of Kernel Language Entropy (KLE). We here show a version of KLE called KLE-c, which operates on semantic clusters. Given an input query and two different LLMs, we sample 10 answers from each model a 1 , . . . , a 10 and a ′ 1 , . . . , a ′ 10 and cluster them by semantic equivalence into clusters C 1 , . . . , C 3 and C ′ 1 , . . . , C ′ 3 . For the sake of the example, we assume that the numbers and sizes of clusters, as well as individual cluster probabilities, are all equal p(C i |inp) = p(C ′ i |inp) for all i. Then, semantic entropy would yield identical uncertainties for both LLMs. However, uncertainty should be lower for LLM 2 because semantic "similarity" between the generations is much higher; i.e., the model is fairly confident that "Kolmogorov" and "Laplace" are good answers. KLE, explicitly accounts for the semantic similarity between texts using a kernel-based approach. Semantic kernels provide an effective way to encode the semantic similarity between answers, enabling the method to correctly identify that LLM 2 's outputs should be assigned lower uncertainty (see right).A critical limitation of SE is that it captures semantic relations between the generated texts only through equivalence relations. This does not capture a distance metric in the semantic space, which would allow one to account for more nuanced semantic similarity between generations. For instance, it separates "apple" as equally strongly from "house" as it will "apple" from "granny smith" even though the latter pair is more closely related. In this paper, we address this problem by incorporating a distance in the semantic space of generated answers into the uncertainty estimation.We propose Kernel Language Entropy (KLE). KLE leverages semantic similarities by using a distance measure in the space of the generated answers, encoded by unit trace positive semidefinite kernels. We quantify uncertainty by measuring the von Neumann entropy of these kernels. This approach allows us to incorporate a metric between generated answers or, alternatively, semantic clusters into the uncertainty estimation. Our approach uses kernels to describe semantic spaces, making KLE more general and better at capturing the semantics of generated texts than the previous methods. We theoretically prove that our method is more expressive than semantic entropy, meaning there are cases where KLE, but not SE, can distinguish the uncertainty of generations. Importantly, our approach does not rely on token likelihood and works for both white-box and black-box LLMs.Our work makes the following contributions towards better uncertainty quantification in LLMs:• We propose Kernel Language Entropy, a novel method for uncertainty quantification in natural language generation (Sec. 3), • We propose concrete design choices for our method that are effective in practice, for instance, graph kernels and weight functions (Sec. 3.2), • We prove that our method is a generalization of semantic entropy (Thm. 3.5), • We empirically compare our approach against baselines methods across several tasks and LLMs with up to 70B parameters (60 scenarios total), achieving SoTA results (Sec. 5).We release the code and instructions for reproducing our results at https://github.com/ AlexanderVNikitin/kernel-language-entropy. 
background: 
 Uncertainty Estimation. Information theory [49] offers a principled framework for quantifying the uncertainty of predictions as the predictive entropy of the output distribution:EQUATIONwhere Y is the output random variable, x is the input, and H(Y |x) is a conditional entropy which represents average uncertainty about Y when x is given. Uncertainty is often categorized into aleatoric (data) and epistemic (knowledge) uncertainty. Following previous work on uncertainty quantification in LLMs, we assume that LLMs capture both types of uncertainty [30] and do not attempt to disambiguate them, as both epistemic and aleatoric uncertainty contribute to model errors.UQ in sequential models. Let S ∈ T N be a sequence of length N , consisting of tokens, s i ∈ T , where the set T denotes a vocabulary of tokens. The probability of S is then the joint probability of the tokens, obtained as the product of conditional token probabilities:EQUATIONInstead of Eq. ( 2), the geometric mean of token probabilities has proven to be successful in practice [50] . Using Eq. ( 1) and ( 2), we can define the predictive entropy of a sequential model. Definition 2.1. The predictive entropy for a random output sequence S and input x isEQUATIONwhere the sum is taken over all possible output sequences s.A downside of naive predictive entropy for Natural Language Generation (NLG) is that it measures uncertainty in the space of tokens while the uncertainty of interest lies in semantic space. As an illustrative example, consider two sets of n answers, S i and S ′ i sampled from two LLMs with equivalent token likelihood p(S i |x) = p(S ′ i |x) as a response to the question "What is the capital of France?" [36] . Suppose the answers from the first LLM are various random cities ("Paris", "Rome", etc.), and those from the second LLM are paraphrases of the correct answer "It is Paris". Naive predictive entropy computation can give similar values, even though the second LLM is not uncertain about the meaning of its answer. Kuhn et al. [36] have proposed semantic entropy to address this problem.We first define the concept of semantic clustering. Semantic clusters are equivalence classes obtained using a semantic equivalence relation, E(•, •), which is reflexive, symmetric, and transitive and should capture semantic equivalence between input texts. In practice, E is computed using bidirectional entailment predictions from a Natural Language Inference (NLI) model, such as DeBERTa [22] or a prompted LLM, that classifies relations between pairs of texts as "entailment," "neutral," or "contradiction". Two texts are semantically equivalent if they entail each other bi-directionally. Semantic clusters are obtained by greedily aggregating generations into clusters of equivalent meaning. We can now define semantic entropy. Definition 2.2. For an input x and semantic clusters C ∈ Ω, where Ω is a set of all semantic clusters, Semantic Entropy (SE) is defined asSE(x) = - C∈Ω p(C | x) log p(C | x) = - C∈Ω      s∈C p(s | x)   log   s∈C p(s | x)      . (4)In practice, it is not possible to calculate C p(C | x) log p(C | x) because of the intractable number of semantic clusters. Instead, SE uses a Rao-Blackwellized Monte Carlo estimatorEQUATIONwhere C i are M clusters extracted from the generations and p′ (C i | x) is a normalized semantic probability, p ′ (C i | x) = p(Ci|x) / i p(Ci|x), which we refer to as p(C i |x) in the following for simplicity. SE can be extended to cases where token likelihoods are not available by approximating p(C i |x) with the fraction of generated texts in each cluster, p(C i |x) ≈I(Si ∈ Ci) /N. We refer to this variant as Discrete Semantic Entropy [16] . 
kernel language entropy: 
 This section introduces Kernel Language Entropy (KLE), our novel approach to computing semantic uncertainty that accounts for fine-grained similarities between generations for better uncertainty quantification. We introduce two variants of KLE: the first, simply called KLE, operates directly on the generated texts, and the second, KLE-c operates on the space of semantic clusters.Motivating Example. Figure 1 illustrates the advantages of KLE (to be precise, the KLE-c variant) over other methods such as SE. Imagine querying two LLMs such that the outputs of LLM 1 are all semantically different and those of LLM 2 are semantically similar but not equivalent. For simplicity, we assume an equal amount of clusters between LLMs and equal likelihoods of clusters p(C i |inp) = p(C ′ i |inp). SE would not distinguish between those cases and, thus, would misleadingly predict equal uncertainty. KLE on the other hand, will correctly assign lower uncertainty to the outputs of LLM 2 , its kernels accounting for the fact that LLM 2 produces semantically similar outputs.Before introducing KLE, we recall the definition of a positive semidefinite (PSD) kernel.EQUATIONFor a finite set X , a PSD kernel is a PSD matrix of the size |X |.Next, we define semantic kernels, denoted K sem , as unit trace 1 positive semidefinite kernels over the finite domain of generated texts. Unit trace PSD matrices are also called density matrices. These kernels should, informally speaking, capture the semantic similarity 2 between the texts such that K(s 1 , t [72] ) is defined asEQUATIONIt can be shown that VNE(A) = n i -λ i log λ i where λ i , 1 ≤ i ≤ n are the eigenvalues of A. Within this definition, we assume 0 log 0 = 0. This reformulation shows that VNE is, in fact, the Shannon entropy over the eigenvalues of a kernel.We can now define Kernel Language Entropy, as the VNE of a semantic kernel. Definition 3.3 (Kernel Language Entropy). Given a set of LLM generations S 1 , . . . , S N , an input x, and semantic kernel K sem over these generations and input, we define Kernel Language Entropy (KLE) as the von Neumann entropy of a semantic kernel K sem :EQUATIONThe von Neumann entropy has the following properties, which are aligned with the overarching goal of measuring the uncertainty of a set of generations.Proposition 3.4 (Properties of the von Neumann Entropy [5] ). The VNE of a unit trace positive semidefinite kernel has the following properties:1. The VNE of a kernel with only one non-zero element is equal to 0.2. The VNE is invariant under changes of basis U :VNE(K) = VNE(U KU ⊤ ).3. The VNE is concave. For a set of positive coefficients α i , k i=1 α i = 1, and density matrices K i , it holds thatVNE k i=1 α i K i ≥ k i=1 α i VNE(K i ).1 Kernels with Tr[K] = 1 are called unit trace kernels. 2 Or more broadly semantic relatedness, including antonymy, meronymy, as well as semantic similarity [7] .Let us briefly discuss the practical implications of these properties. Property 1 states that if an LLM outputs a single answer (for KLE) or a semantic cluster (for KLE-c), the VNE is zero, indicating high certainty. Property 2 is significant as it allows the VNE to be calculated in practice as the Shannon entropy of the diagonal elements of an orthogonalized kernel, which can be interpreted as a disentangled representation of a semantic kernel. Property 3 states that entropy is concave, meaning that the entropy of a combined system is greater than or equal to the entropy of its individual parts, a common requirement for entropy metrics. The intuition behind our use of the VNE for LLMs also relates to its origins in quantum information theory. ▷ Eq. ( 8)The VNE in Quantum Information Theory. In quantum information theory, the states of a quantum system (or pure states) are defined as unit vectors in C N . However, experiments often result in statistical mixtures of pure quantum states, represented as density matrices.The VNE is used to evaluate the entropy of the mixed states. Analogously, we can think of KLE as considering each answer as a mixture of pure "semantic meanings", measuring the entropy of this mixture. We refer the reader to Aaronson [1] for further background reading on the VNE and quantum information theory.Instead of defining semantic kernels directly over individual model generations, we can also apply KLE to clusters of semantic equivalence.We call this variant of our method KLE-c. Although KLE is more general than KLE-c for non-trivial clusterings, KLE-c can provide practical value as it is cheaper to compute and more interpretable due to its smaller kernel sizes.Algorithm. Algorithm 1 provides a generic description of the steps required to compute KLE. We describe the practical details for defining and combining semantic kernels later in Sec. 3.2.Computational Complexity. The computational complexity of KLE is approximately identical to SE which requires sampling from an LLM N times and running the entailment model O(N 2 ) times. Additionally, KLE requires O(N 3 ) elementary operations for kernel and VNE calculation. The actual cost of this is negligible in comparison to the forward passes through the LLM or entailment model.This section describes a practical approach for constructing semantic kernels over LLM generations or semantic clusters. Concretely, we apply NLI models to construct semantic graphs over the LLM outputs and then borrow from graph kernel theory to construct kernels from these graphs. A similar notion of semantic graphs derived from NLI models was proposed by Lin et al. [44] for black-box LLM uncertainty quantification.Graph Theory Preliminaries. First, let us recall the basics of graph theory. A graph is a pair of two sets G = (V, E), where V = {1, . . . , n} is a set of n vertices and E ⊆ V × V is a set of edges. A graph is called weighted when a weight is assigned to each edge, and the weight matrix W ij contains weights between nodes i and j. For unweighted graphs, we can use a binary adjacency matrix to encode edges between nodes. The degree matrixD is a diagonal |V | × |V | matrix with D ii = |V | j=1 W ij . The graph Laplacian is defined as L = D -W .L is a positive semidefinite matrix, and eigenvalues of L are often used to study the structure of graphs [10, 71] .Semantic Graph. We define semantic graphs as graphs over LLM generations (G sem ) or semantic clusters (G sem-c ). For G sem , edges can be defined as a function of NLI predictions in both directions:W ij = f (NLI(S i , S j ), NLI(S j , S i )), where NLI are the predicted probabilities for entailment, neutral, and contradiction for S i and S j . For example, f could be the weighted sum over the predicted probabilities for entailment and neutral classes. For G semc-c , the weights between the clusters are computed by summing the entailment predictions over the generations assigned to the clusters, W ij = s∈Ci t∈Cj f (NLI(s, t), NLI(t, s)).Graph Kernels. When a semantic graph is obtained, KLE calculates graph kernels over semantic graph nodes to compute a distance measure. Since graphs are discrete and finite, any positive semidefinite matrix would be a kernel over the graph. However, we seek kernels that exploit knowledge about the graph structure. We, therefore, adopt Partial Differential Equation (PDE) and Stochastic Partial Differential Equation (SPDE) approaches to graph kernels [34, 6, 57] . If u ∈ R n is a signal over the nodes of a graph, the heat kernel is a solution to the partial differential equation ∂u /∂t + Lu = 0 and the Matérn kernel is a solution to the stochastic differential equation,( 2ν /κ 2 + L) ν 2 u = w,where w is white noise over the graph nodes and L is the graph Laplacian defined above. The corresponding solutions to these equations are:EQUATIONThese kernels allow for the incorporation of a distance measure that reflects the graph's locality properties (right part of Fig. 2 ). For example, the Taylor series of the heat kernel can be shown to be equal to a sum of powers of random walk matrices. Both kernels have hyperparameters: lengthscales t in the heat kernel, κ in Matérn kernels, and ν in the Matérn kernel, often interpreted as smoothness. The scaled eigenvalues of the Matérn kernel converge to the eigenvalues of the heat kernel [6] when ν goes to infinity. Matérn kernels provide more flexibility at the cost of the additional parameter. Note that any kernel can be normalized into a unit trace kernel via K(x, y) ← K(x, y)(K(x, x)K(y, y)) -1/2 /N , where N is the size of K. We refer to [34, 57, 6] for further background reading on graph kernels.Kernel Hyperparameters. We propose two ways to select the hyperparameters of the heat and Matérn kernels: either by maximizing the validation set performance or by selecting parameters from what we call Entropy Convergence Plots, illustrated in Fig. 2 . We obtain these plots by defining a set of progressively denser graphs G 1 ≺ . . . ≺ G K . These can be obtained by starting from a graph without edges and a fixed number of vertices and adding new edges either randomly or by filling in the adjacencies of each node sequentially. We then plot the VNE against the number of edges in the graphs G i . We analyze the von Neumann entropy over these plots to avoid pathologies connected to the fact that for large lengthscales, the VNE converges rather quickly, and such behavior should generally be avoided. For all remaining values, we can either choose hyperparameters randomly from the range of non-collapsing hyperparameters or rely on prior domain knowledge.Kernel Combination. KLE offers the additional flexibility of combining kernels from various methods (e.g., multiple NLI models, different graph kernels, or other methods). For example, we can combine multiple kernels using convex combinations, K = P i=1 α i K i , whereP i=1 α i = 1.The semantic kernels used in KLE are more informative than the semantic equivalence relations used in SE [36] . The next theorem shows that KLE can recover SE for any semantic clustering. Theorem 3.5 (KLE and KLE-c generalize SE). For any semantic clustering, there exists a semantic kernel over texts K sem (s, s ′ ) such that the VNE of this kernel is equal to semantic entropy (computed as in Eq. ( 5)). Moreover, there exists a semantic kernel over clusters K sem (c, c ′ ) such that the VNE of this kernel is equal to SE. Proof Sketch. For any semantic clustering, we consider a kernel with a block diagonal structure. Each block corresponds to a semantic cluster, and cluster likelihoods are normalized by the size of the cluster, p(Ci|x) /mi. This is a valid semantic kernel and the KLE for this kernel equals the SE. Thm. B.1 and Thm. B.2 in the Appendix contain the detailed proofs.The proof of Thm. 3.5 shows that the block diagonal semantic kernels used with KLE can recover semantic entropy for any clustering. However, there are other kernels available that allow KLE to be more expressive than SE. Comparing KLE and KLE-c, we find that KLE is more general than KLE-c for any non-trivial clustering. 
related work: 
 In the context of machine learning, the VNE has been studied theoretically [4] , applied to GAN regularization [33] , and the exponential of the VNE has been used for effective rank and sample diversity analysis [64, 19] .The first attempts at estimating the entropy of language date back to the 1950s [65] , and today, techniques for uncertainty quantification are widely used in natural language processing. For instance, Desai and Durrett [15] and Jiang et al. [28] presented calibration techniques for classification tasks. Xiao and Wang [73] empirically showed that, for various tasks, including sentiment analysis and named entity recognition, measuring model uncertainty can be used to improve performance. Calibration techniques have also been applied in machine translation tasks to improve accuracy [37] .Malinin and Gales [50] discussed the challenges of estimating uncertainty in sequential models. Several previous works have queried LLMs to elicit statements about uncertainty, either via finetuning or by directly including previous LLM generations in the prompt [30, 9, 54, 43, 53, 21, 63, 68, 12, 74, 36] . Zhang et al. [76] studied UQ for long text generation. Quach et al. [61] used conformal predictions to quantify LLM uncertainty, which is orthogonal to the approach we pursue here. Yang et al. [75] have shown that Bayesian modeling of LLMs using low-rank Laplace approximations improves calibration in small-scale multiple-choice settings. Lin et al. [44] extended the work of Kuhn et al. [36] on semantic entropy by introducing the use of the Laplacian of semantic graphs and applying spectral graph analysis for UQ in black-box LLMs. Aichberger et al. [2] proposed a new method for sampling diverse answers from LLMs, and Liu et al. [47] proposed improving calibration by adding an extra linear layer; more diverse sampling strategies and better calibration could improve KLE as well.There are a variety of ways besides model uncertainty to detect hallucinations in LLMs such as querying external knowledge bases [17, 42, 70] , hidden state interventions [77, 24, 46] , using probes [8, 41, 48] , or applying fine-tuning [31, 67] . KLE is complementary to many of these directions and focuses on estimating more fine-grained semantic uncertainty. It can either be used to improve these approaches or be combined with them sequentially. 
experiments: 
 Datasets and Models. Our experiments span over 60 dataset-model pairs. We evaluate our method on the following tasks covering different domains of natural language generation: general knowledge (TriviaQA [29] and SQuAD [62] ), biology and medicine (BioASQ [35] ), general domain questions from Google search (Natural Questions, NQ [38] ), and natural language math problems (SVAMP [60] ). We generally discard the context associated with each input for all datasets except SVAMP, as the tasks become too easy for the current generation of models when context is provided. We use the following LLMs: Llama-2 7B, 13B, and 70B [69] , Falcon 7B and 40B [3] , and Mistral 7B [27] , using both standard and instruction-tuned versions of these models. As the NLI model for defining semantic graphs or semantic clusters, we use DeBERTa-Large-MNLI [22] .Baselines. As baseline methods, we compare KLE with semantic entropy [36] , discrete semantic entropy [16, 36] , token predictive entropy [50] , embedding regression [16] , and P(True) [30] . For embedding regression, we train a logistic regression model on the last layer's hidden states to predict whether a given LLM answer is correct. KLE Kernels. We propose to use the following two semantic kernels with KLE: K HEAT and K FULL . Both are obtained from the weighted graph W ij = w NLI ′ (S i , S j ) + w NLI ′ (S j , S i ), where w = (1, 0.5, 0) ⊤ is a weight vector. Here, we assume that NLI ′ returns a one-hot prediction over [36] 0.75 ± 0.04 0.90 ± 0.01 0.71 ± 0.03 0.46 ± 0.03 0.66 ± 0.03 0.65 ± 0.03 0.63 ± 0.03 0.61 ± 0.03 0.77 ± 0.03 0.79 ± 0.02 PE [50] 0.69 ± 0.04 0.90 ± 0.01 0.67 ± 0.03 0.44 ± 0.03 0.65 ± 0.03 0.65 ± 0.03 0.59 ± 0.03 0.58 ± 0.03 0.61 ± 0.03 0.73 ± 0.03 P(True) [30] 0.86 ± 0.03 0.92 ± 0.01 0.78 ± 0. (entailment, neutral class, contradiction). K HEAT is a heat kernel over this graph. We further propose K FULL = αK HEAT + (1α)K SE , where α ∈ [0, 1] and K SE is a semantic entropy kernel. We ablate these kernel choices in our experiments below.Evaluation metrics. Following previous work, we evaluate uncertainty methods by measuring their ability to predict the correctness of model responses, calculating the Area under the Receiver Operating Curve (AUROC). Further, uncertainty metrics can be used to refuse answering when uncertainty is high, increasing model accuracy on the subset of questions with uncertainty below a threshold. We measure this with the Area Under the Accuracy-Rejection Curve (AUARC, [56] ).The rejection accuracy at a given uncertainty threshold is the accuracy of the model on the subset of inputs for which uncertainty is lower than the threshold; the AUARC score computes the area under the rejection accuracy curve for all possible thresholds.Sampling. We sample 10 answers per input via top-K sampling with K = 50 and nucleus sampling with p = 0.9 at temperature T = 1. To assess model accuracy, we draw an additional low-temperature sample (T = 0.1) and ask an additional LLM (Llama 3 8B Instruct) to compare the model response to the ground truth answer provided by the datasets. We evaluated the accuracy-checking performance of Llama 3 8B Instruct by comparing its assessments with human raters, finding 90% agreement across 100 cases. We also compared its evaluations with GPT-4 evaluations on the TriviaQA dataset, using answers generated by Llama-2-70B-chat, and observed a 95% agreement.Statistical significance. We assess statistical significance in two ways. First, we run a large number of experimental scenarios (60 model-dataset pairs), and second, for each experimental scenario, we also obtain confidence intervals over 1000 bootstrap resamples. We note that standard errors in each scenario are more representative of the LLM and the dataset rather than the method. Therefore, our main criterion for comparing the methods is based on the fraction of experimental cases where our method outperforms baselines (assessed with a binomial statistical significance test).KLE outperforms previous methods. We compare the performance of UQ methods over 60 scenarios (12 models, five datasets). Figure 3 shows the heatmaps of pairwise win rates. We observe that both our methods, KLE(K HEAT ) and KLE(K FULL ), are superior to the baselines. Furthermore, Table 1 shows the detailed results for the two largest models from our experiments, Llama 2 70B Chat and Falcon 40B Instruct. The results show that for the largest models, our method consistently achieves best results compared to baselines. In KLE hyperparameters can be selected without validation sets. We compare the strategies of hyperparameter selection from Sec. 3.2: entropy convergence plots and validation sets (100 samples per dataset except for SVAMP, where we used default hyperparameters). We observe that default hyperparameters achieve similar results as selecting hyperparameters from validation sets and conclude that choosing default hyperparameters from entropy convergence plots is a good way to select hyperparameters in practice. In Fig. 4 , we compare the two strategies for selecting hyperparameters, and see that the ranking of the methods remains stable and the pairwise win-rates are similar for both methods.0.2 0.4 0.6 Average pairwise win rate Many design choices outperform existing methods, the best is KLE(K HEAT ). Next, in Fig. 4 , we compare several design choices for KLE: choosing a kernel (heat or Matérn), using KLE-c, combining kernels via a weighted sum or product, and using the probabilities returned by DeBERTa for edge weights. The superscript indicates the type of a graph: no superscript indicates a weighted graph as described above, DB means weights are assigned using probabilities from DeBERTa, and C means a weighted graph over clusters (KLE-c). The subscript indicates the semantic kernels: SE stands for a diagonal kernel with semantic probabilities (K SE ), HEAT and MATÉRN for the type of kernel (K HEAT and K MATÉRN ), and ⋆ for the best of Heat and Matérn kernels. We observe that even though all design choices outperform SE, the heat kernel over a weighted semantic graph, KLE(K HEAT ), was overall the best. Additionally, we notice that the methods based on token likelihoods are performing better for non-instruction-tuned models, and we can practically recommend including semantic probabilities (e.g., use variations of K FULL ) if KLE is used in non-instruction-tuned scenarios (see Fig.SE K DB FULL K c + K SE K full K c • K SE K c heat K DB K matérn K heat Method Selection Validation DefaultKLE is better in practice because it captures more fine-grained semantic relations than SE. The performance of KLE improves over SE because in complex free-form language generation scenarios, such as those studied here, LLMs can generate similar but not strictly equal answers. SE assigns these to separate clusters, predicting high entropy. By contrast, our method can account for semantic similarities using the kernel metric in the space of meanings over generated texts, and predict reduced uncertainty if necessary. We give a detailed illustrative example for which KLE provides better uncertainty estimates than SE from the NQ Open dataset in Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: Yes, we attach the source code and instructions for reproducing the results in the supplementary materials. We will release the source code and instructions online.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
discussion: 
 Measuring semantic uncertainty in LLMs is a challenging and important problem. It requires navigating the semantic space of the answers, and we have suggested a method, KLE, that encodes a similarity measure in this space via semantic kernels. KLE allows for fine-grained estimation of uncertainty and is an expressive generalization of semantic entropy. We provided several specific design choices by defining NLI-based semantic graphs and kernels, and studying kernel hyperparameters. We have evaluated KLE across various domains of natural language generation, and it has demonstrated superior performance compared to the previous methods. Our method works both for white-and black-box settings, enabling its application to a wide variety of practical scenarios. We hope to inspire more work that moves from semantic equivalence to semantic similarity for estimating semantic uncertainty in LLMs.Broader Impact. Our work advances the progress toward safer and more reliable uses of LLMs. KLE can positively impact areas that involve using LLMs by providing more accurate uncertainty estimates, which can filter out a proportion of erroneous outputs.Limitations. One limitation of the proposed method is that it requires multiple samples from an LLM, which generally increases the generation cost. However, in safety-critical tasks, the potential cost of hallucination should outweigh the cost of sampling multiple answers, so reliable uncertainty quantification via KLE should always be worthwhile. Additionally, we study semantic kernels derived from NLI-based semantic graphs, but other semantic kernels warrant investigation, such as kernels on embeddings. Moreover, the NLG landscape is highly diverse, and the method should be carefully evaluated for other potential applications of LLMs, such as code generation. Lastly, our method estimates uncertainty using predictive entropy, as commonly done in Bayesian deep learning. However, in applications where confidence estimates are important, alternative methods should be considered.Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic SimilaritiesA Background A.1 Linear Algebra Definition A.1. For a set X ̸ = ∅, a symmetric function K : X × X → R is called a positive- semidefinite kernel if for all n > 0, x i ∈ X , α i ∈ R n i=1 n j=1 α i α j K(x i , x j ) ≥ 0. (A.1)For a finite set X , a positive semidefinite kernel is a positive semidefinite matrix of the size |X |. Lemma A.2. For a block diagonal matrixA =        A 11 0 0 . . . 0 0 A 22 0 . . . 0 0 0 A 33 . . . 0 . . . . . . . . . . . . . . . 0 0 0 . . . A nn       eigenvalues are all eigenvalues of the blocks A ii combined, or equivalentlydet (A -λI) = 0 ⇔ n i=1 det(A ii -λI) = 0Proof. Notice, that a block diagonal matrix can be decomposed into the following product: , where I ii are the identity matrices of the same size as A ii .EQUATIONBy using the product rule for determinants, we obtain det (A -λI) = 0⇔ n i=1 det(A ii -λI) = 0.Lemma A.3 (Horn and Johnson [25] ). An all-ones matrix J of size n has eigenvalues {n, 0, . . . , 0 n-1 }.Throughout the text, we often refer to the notion of equivalence relation. We remind readers of the definition of equivalence relation here. Example. We visualize an example from the NQ dataset in Fig. In this section, we provide additional experimental results.Hardware and Resources. We ran Llama 2 70B models on two NVIDIA A100 80GB GPUs, and the rest of the models on a single NVIDIA A100 80GB. The generation process took from one to seven hours (depending on a model) for each experimental scenario, and the evaluation additionally took roughly four hours per scenario which can be further optimized by reducing the number of hyperparameters. The project spent more resources due to other experiments. Our experimental pipeline first generates the answers for all the datasets and then computes various uncertainty measures. We did not recompute generations, but in each experimental run we only evaluated uncertainty measures.Licenses. We release our code under a clear BSD-3-Clause-Clear. The datasets used in this paper are released under CC BY 2.5 (BioASQ; [35] ), Apache 2.0 (TriviaQA; [29] ), CC BY-SA 4.0 (SQuAD; [62] ), MIT (SVAMP; [60] ), and CC BY-SA 3.0 (NQ; [38] ).In Fig. D .1, we show samples from each dataset we used in the experimental evaluation of our method.Who played the gorilla in the cadbury advert? 1945In what year of 20th century, did Harvard release an important document about education in America?What city, Chile's second largest, suffered an 8.8 earthquake in 2010?Question: Correct Answer: Real-world applications often involve deploying models with varying degrees of performance, and a robust UQ method should provide reliable uncertainty estimates for all of them. By demonstrating the efficacy of our method across a wide variety of models, we validate its applicability in diverse scenarios. This highlights that our approach can be confidently used in practical settings where model performance can fluctuate.TriviaFurthermore, we investigate the performance of UQ methods by splitting the set of experimental scenarios into instruction-tuned and non-instruction-tuned models. We visualize the splits in Fig. D .5. Interestingly, our approach significantly outperforms the existing methods when evaluated with instruction-tuned models, and only marginally outperforms when evaluated on non-instruction-tuned models. We can hypothesize that non-instruction-tuned models are better calibrated, and thus methods based on token-likelihoods perform well whereas instruction-tuning worsens calibration. This hypothesis is also supported by comparison of SE and DSE (DSE significantly outperforms SE on an instruction-tuned split, when AUROC is measured).We provide a detailed comparison of our method with previous uncertainty quantification measures. In , we show the results for a wide range of models across five datasets for non-instruction-tuned and instruction-tuned models, respectively. We want to note that ER has failed for Llama 2 13B (non-instruction-tuned version) for all datasets except BioASQ because training datasets for ER contained samples of only one class. We have assigned zero scores to the failed cases.In Supplementary Note 2, Farquhar et al. [16] analyze the accuracy of various NLI models. They report that DeBERTa shows an average agreement of 0.8 with human raters, compared to an agreement of 0.87 between human annotators. We hypothesize that using a more advanced but computationally expensive NLI model, such as GPT-4, could improve the semantic kernel and, consequently, enhance uncertainty estimation using KLE.We resort to the 6-level model of the structure for text analysis proposed in [14] to extensively describe aspects of language beyond semantics. This model distinguishes four basic notions for text analysis: medium of transmission, grammar, semantics, and pragmatics. Medium of transmission is irrelevant to the study of language model outputs (however, it becomes relevant for multimodal foundation models that can, for instance, answer a request either with a text or an image); grammar is further divided into the syntax and morphology of the text and semantics into semantics and discourse. Another dimension is pragmatics, or how the text is used. In this work, we focus only on the semantics of the text. However, the method can be extended to other aspects of text analysis. For instance, one can design syntactic or pragmatic kernels. We leave the study of other kernel modalities to future works. some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We provide all the necessary details in Sec. 5 and additional details in App. D.• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We assess statistical significance on two levels. First, we run a large number of experimental scenarios (60 model-dataset pairs), and second, for each experimental scenario, we also obtain confidence intervals with 1000 resamples. We would like to additionally comment that standard errors in each scenario are more representative of the LLM and the dataset rather than the method. Therefore, our main criterion for evaluating the methods is the proportion of experimental cases where our method outperforms others (assessed with a binomial statistical significance test). We discuss it in more detail in Sec. 5.• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We discuss the licenses in App. D. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes] Justification: We document our source code in the README.md file.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification:Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification:Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. 
 
 Appendix:  


Decision: This paper provides an improvement over Kuhn et al. (ICLR, 2023) "Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation", which performs hard clustering of the generated answers and computes the entropy of the clusters, by modifying the notion of semantic entropy to consider inter-cluster interactions. This is done by considering instead the von Neumann entropy (VNE) with respect to a graph kernel, where the graph edge weights encodes the similarity between the clusters. 

Most reviewers agree that this is a simple but well motivated and theoretically sound approach, pointing out as weaknesses the modest empirical improvements, the requirement of multiple samples to compute uncertainty, and the need to improve the structure of the paper. While this seems a relatively small contribution on top of Kuhn et al. and it has its limitations, the proposed approach seems solid and addresses an important problem with the original definition of semantic entropy by Kuhn et al. The rebuttal is convincing and provides additional results which will improve the paper. I am leaning to recommend acceptance.

Reviews: review: paper_summary: paper_summary: This paper is highly motivated by and heavily draws from Kuhn et al. (ICLR, 2023) “Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation.” In Kuhn’s original paper they propose an unsupervised way to calculate the semantic uncertainty of LLMs by (1) generating a set of outputs from an LLM, (2) clustering these outputs by semantic equivalence via an NLI model, and (3) plugging in these clusters to an equation for “semantic entropy.” 

This paper proposes an extension of the strict clusters of Kuhn et al.~by creating a “semantic graph” and calculate the “graph kernel” for the uncertainty metric. Thus, the authors attempt to measure the semantic entropy *between* clusters rather than just *within* clusters (the latter which is the approach of the previous work, Kuhn et al. (2023)). 

The authors provide a lot of math and theory trying to justify that their generalization from Kuhn et al. is meaningful. However, the empirical experiments show modest to no performance gains from their generalization on real-world QA datasets and LLMs.
strength_weakness: strength_weakness: 1. **The claims are not backed by strong enough evidence.**  

On line 59, the authors claim their approach achieves SoTA results. However, the empirical results (Table 1) are little to modest gains over baselines. Additionally, the bolding in Table 1 is misleading. Results should only be bolded if the method’s confidence interval is non-overlapping the confidence intervals of other methods (which they are not). For example, in Table 1 for AUROC on BioASQ, $0.88 \pm 0.03$ for KLE (K_FULL) is overlapping with P(True) at $0.86 \pm 0.03$. Likewise for SVAMP AUROC with $0.77 \pm 0.02$ for KLE (K Heat) and ER at $0.75 \pm 0.02$. This also makes me skeptical of how the authors are calculating the “win rates” for the other figures as well. 

2. Additionally the authors do not investigate the **error/accuracy of intermediate models** that the experimental set-up relies upon.

A key part of the authors pipeline is using DeBERTa-Large-MNLI to predict whether LLM outputs entail one another and create their semantic graph (line 262). However, DeBERTa-Large-MNLI is also an imperfect model. What was the accuracy of this model on the domains/datasets in the authors’ empirical pipeline? Are there ways to propagate uncertainty from this intermediate NLI model downstream to the final uncertainty calculations of other LLM’s outputs? 

3. Additionally, see the questions below.
questions: questions: 1. In the abstract, you motivate this work by saying “by detecting factually incorrect model responses, commonly called hallucinations.” Yet, you never actually show that you detect hallucinations (which require world knowledge that is external and more complex than the “semantic uncertainty” you’re actually targeting). How do you justify this disconnect? 

2. On line 25 you cite two works that you use to justify “As LLM predictions tend to be well-calibrated.” However there is an *enormous* body of literature that shows the opposite. I would recommend hedging on this statement or providing citations of the opposite (non-calibration) findings.  

3. Figure 1 lacks some details and motivation. In this example, I would not expect the named entity outputs, (e.g., “Laplace” or “Kolmogorov and Laplace”) to have any other lexical variants that have similar semantic meaning because these are named entities. In this figure, what does the little boxes next to “Semantic Kernels” represent? I would recommend explaining this more (e.g., I think you’re implying that blue means more similarity between particular clusters?) 

4. In Figure 3’s caption, please explain why only “values larger than or equal to 0.62” correspond to a p-value less than 0.05. 

5. I would recommend explaining how you are calculating AUPRC and why it is important. I know Kuhn et al. also use this metric but you do not make clear what this metric is in your standalone work. From re-reading Kuhn et al., I took away that they calculated an “entropy score” to predict whether a model’s answer to a question is correct. They vary the threshold of the entropy score for correctness prediction to get AUROC.
limitations: limitations: Addressed.
paper_summary: paper_summary: In this work, the authors propose Kernel Language Entropy, which shares a similar concept to semantic uncertainty but additionally considers semantic similarity. Based on this proposed theory, they design graph kernels and weight functions to estimate LLM uncertainty. Furthermore, they demonstrate that their method generalizes semantic entropy. The empirical results confirm the effectiveness of the proposed method across various tasks and LLMs.
strength_weakness: strength_weakness: 1. KLE requires iterative sampling from the LLM, which is computationally costly. This leads to delayed responses and limits its applicability.

2. It seems that this method can only estimate the confidence regarding whether the LM will correctly answer the query, but it cannot predict the confidence for a given answer. For example, in scenarios like ranking candidate answers, we can use P(True) or PE to estimate answer confidence, which KLE cannot do.
questions: questions: 1. Which sample do you choose as the final answer among multiple samples, the answer in the biggest cluster or just a random one? I did not find this detail (perhaps I missed it), but it is important.

2. Since the answer correctness is determined by an 8b model, I am curious about how reliable the predicted label is. From my previous experience, even GPT-3.5 is not capable of reliably estimating the correctness of model responses, especially when the responses are phrases. Tian et al. [1] also noted a similar issue, as seen in their Appendix C.

[1]. Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback.

3. Can this method be applied to long-form responses? Considering that long-form responses typically consist of multiple claims.

4. Missing the following related work:

Language Models (Mostly) Know What They Know

LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses
limitations: limitations: The authors acknowledge the issue of computational cost. Investigating ways to elicit confidence from the LLM itself could be a possible direction for addressing this.
paper_summary: paper_summary: The paper introduces the method "Kernel Language Entropy", capturing semantic similarities of output sequences via semantic kernels and subsequently estimating uncertainty using the von Neumann entropy.
strength_weakness: strength_weakness: The primary area for improvement is the paper's structure. The paper is quite difficult to follow due to definitions and expressions not being clearly contextualized, for instance:
- **Sections and Headings**: Section 3 is inconsistent in its use of subsections and subheadings. It begins with a motivating example (subheading), followed by the formal definition of PSD kernels without a subheading. The definition of semantic kernels is embedded in the text without a subheading, making it less prominent than the practical approach for constructing semantic kernels, which has a dedicated subsection at the end of the section. Also, VNE is given more prominence than the main concept, KLE (starting with the definition of KLE (subheading) as the VNE, followed by deriving its properties, would improve coherence).
- **Kernels**: In Section 3.1, two explicit kernels ($K_{heat}$ and $K_{Matérn}$) are introduced. In Section 5, the authors then propose to use $K_{heat}$ and $K_{full}$, with $K_{full}$ being a weighted sum of $K_{heat}$ and $K_{SE}$. The previously proposed $K_{Matérn}$ is no longer considered in the main experiments, while $K_{SE}$ has not been introduced before. It only becomes clear in Section B of the Appendix that $K_{SE}$ is equal to Semantic Entropy, which is referred to as $SE$ in Section 5. Also, the importance of the weighting factor for $K_{full}$ is not discussed. In general, the criteria for choosing between kernels are unclear (and not all kernels consistently outperform the baselines across datasets).
questions: questions: - Is the rather small model Llama 3 8B Instruct capable of evaluating the correctness of an output sequence? Have you considered other (statistics-based) metrics to evaluate the correctness?
- Have the authors considered utilizing a regression model that directly assigns a single value to the semantic similarity instead of unintuitively having to aggregate the three classes of the NLI model?
limitations: limitations: The authors adequately addressed limitations.
paper_summary: paper_summary: The authors propose a novel Kernel Language Entropy (KLE) method for uncertainty estimation in white- and black-box LLMs. KLE defines positive semidefinite unit trace kernels to encode the semantic similarities of LLM outputs and quantifies uncertainty using the von Neumann entropy. It considers pairwise semantic dependencies between answers (or semantic clusters), providing more fine-grained uncertainty estimates than previous methods based on hard clustering of answers. We theoretically prove that KLE generalizes the previous state-of-the-art method called semantic entropy and empirically demonstrate that it improves uncertainty quantification performance across multiple natural language generation datasets and LLM architectures.
strength_weakness: strength_weakness: 1.  The motivation is clear in Fig 1, but why choose the kernel to solve the problem, this part is not very clear.

2. The proposed method looks like it depends on the sampling times, which is expensive. How is the performance when we sample few times?
questions: questions: 1. How do I get the final answer based on 10 samplings? For example, in fig 1, what could be the final answer of LLM2, majority vote?
limitations: limitations: the authors adequately addressed the limitations

Comments: title: Response
Response
comment: Thanks for the detailed response. I changed my score to a 4.
Dear reviewer S5q9, Thank you for your comment and for engaging in the discussion!

>Per my comment "Results should only be bolded if the method’s confidence interval is non-overlapping the confidence intervals of other methods (which they are not)", in each of the 5 datasets, how many times did your method have better point estimates and non-overlapping CIs with the baselines' for both metrics?

We observe that the standard error from bootstrap for each evaluation is consistently large across all method-dataset pairs, and more related to the experimental setup than differences between the methods. Precisely because the CIs are overlapping in individual cases, we repeated the experiment for 60 model-dataset pairs. Our experimental setup follows a recent paper (https://www.nature.com/articles/s41586-024-07421-0, [2]), where the authors specifically emphasize:
“We report the raw average score across held-out evaluation datasets without standard error because the distributional characteristics are more a property of the models and datasets selected than the method. **Consistency of relative results across different datasets is a stronger indicator of variation in this case.**” 

Additionally to evaluating such consistency with the binomial test (Fig. 3 and 4, main text), we’ve included relative mean gain in the reported metrics per dataset in the rebuttal PDF (Tab. 1).

Finally, we report a comparison in each of the 5 datasets separately, when _both metrics are considered simultaneously and the mean estimate from the bootstrap_ is used for comparison. The table shows the numbers of wins, ties (one metric is better, another is worse), and losses of our method compared to the two strongest baselines (the metrics were strongly correlated and therefore highly consistent with each other). Each cell contains (#wins / #ties / #losses).

| 	| SQUAD | SVAMP | NQ | TriviaQA | BioASQ | Total wins |
| -------- | ------- | ------- | ------- | ------- | ------- | ------- |
| SE | 5/2/5 | 5/1/6 | 10/1/1 | 11/1/0 | 9/0/3 | 42.5 ( p ≤ 0.001) |
| P(True) | 10/1/1 | 9/1/2 | 9/2/1 | 10/1/1 | 6/2/4 | 47.5 (p < 0.00001) |

The p-value was calculated using the sign test by splitting the ties between wins and losses [1]. 

In summary, _while the results for a single dataset may not always be conclusive, we in the end observe very strong evidence that our method performs overall the best by repeating the experiment over 60 experimental scenarios_.

We hope that we have addressed your remaining concern!


## References
See the references from our response above.
Thanks for the detailed response. I will keep my positive score.
Thanks for the detailed response. All of my concerns are addressed. And after reading other reviews, I will keep my score.
Thank you for the rebuttal. Addressing the mentioned points in the final version of the paper will indeed improve its clarity and overall quality. I have no further questions and am updating my score.
> Statistical significance is achieved primarily by repeating the experiment with many different models and datasets

"Statistical significance" has a precise, technical meaning as in [Berg-Kirkpatrick 2012](https://aclanthology.org/D12-1091.pdf). 

Per my comment "Results should only be bolded if the method’s confidence interval is non-overlapping the confidence intervals of other methods (which they are not)", in each of the 5 datasets, how many times did your method have better point estimates and non-overlapping CIs with the baselines' for *both* metrics?