Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space

Authors: xin  qiu
risto  miikkulainen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 18:07:02

Abstract: With the widespread application of Large Language Models (LLMs) to various domains, concerns regarding the trustworthiness of LLMs in safety-critical scenarios have been raised, due to their unpredictable tendency to hallucinate and generate misinformation. Existing LLMs do not have an inherent functionality to provide the users with an uncertainty/confidence metric for each response it generates, making it difficult to evaluate trustworthiness. Although several studies aim to develop uncertainty quantification methods for LLMs, they have fundamental limitations, such as being restricted to classification tasks, requiring additional training and data, considering only lexical instead of semantic information, and being prompt-wise but not response-wise. A new framework is proposed in this paper to address these issues. Semantic density extracts uncertainty/confidence information for each response from a probability distribution perspective in semantic space. It has no restriction on task types and is "off-the-shelf" for new models and tasks. Experiments on seven state-of-the-art LLMs, including the latest Llama 3 and Mixtral-8x22B models, on four free-form question-answering benchmarks demonstrate the superior performance and robustness of semantic density compared to prior approaches.

Main paper: 
 introduction: 
 Large language models (LLMs) have revolutionalized many domains, such as conversational agents [34] , code generation [41] , and mathematical discovery [39] . Given their ability for general reasoning and adaptability to new tasks, LLMs are utilized increasingly in safety-critical applications, including healthcare [42] and finance [50] . However, existing LLMs have an unpredictable tendency to hallucinate [30] , leading to misleading information and risky behaviors. Responses are generated without quantitative indicators for their uncertainty/confidence, making it difficult to evaluate how trustworthy they are. As a result, concerns have been raised about their safety [54] , hindering a deeper utilization of LLMs in risk-sensitive domains [5] .Although significant resources have been invested in LLM development, leading to a rapid pace in new model releases, only little progress has been made in building an uncertainty quantification framework for LLMs. An ideal outcome of such a system would be a quantitative metric associated with each response that can be used as an uncertainty/confidence indicator. Users can then build on this metric to evaluate the trustworthiness of LLM responses, e.g., establish an automatic system that triggers a warning if the response confidence is below a pre-defined threshold.Following this line of thought, several techniques have been proposed in the literature to extract the uncertainty/confidence score from LLMs. In addition to the baselines that directly ask the LLM itself to evaluate its own answers [44, 21] , one further step was to integrate traditional uncertainty estimation/calibration methods into LLMs [6, 51, 53] . However, due to the nature of these traditional methods, they only work on classification problems, not free-form natural language generation (NLG) tasks, which are more general and challenging. Another direction was to fine-tune the original model [25] or train an additional layer or classifier [3, 28] to output uncertainty/confidence indicators for the responses. The main drawback is that these approaches are not "off-the-shelf" for new tasks and models: additional task-specific training labels of the ground-truth confidence are needed, and the training needs to be done in a model-specific manner, limiting their applicability.Most importantly, prior work still treats LLM outputs as traditional auto-regressive predictions [29] , i.e., the generated responses are simply handled as sequences of tokens/words, considering only their lexical uncertainty/confidence. However, due to the unique nature of free-form NLG, tokens that are lexically different may be semantically similar. In most LLM applications, decisions depend on the semantics of responses, and the same semantics can be stated using different words or sentence structures, leading to different lexical tokens. Therefore, uncertainty/confidence in semantic space is a more essential indicator for trustworthiness of LLM responses than lexical uncertainty/confidence.Semantic entropy [22] is the state-of-the-art (SOTA) technique in semantic uncertainty [27] . However, its current design has two intrinsic limitations. First, the returned uncertainty score is prompt-wise, i.e., the semantic entropy is calculated for each prompt, instead of each response. Considering that LLMs can generate diverse responses for the same prompt, using the same uncertainty score for different responses is problematic [26] . Second, semantic entropy only considers semantic equivalence, which is a binary one-cut measurement, i.e., it only returns whether two responses are considered semantically equivalent or not, without reporting how semantically different the responses are. It does not make use of the more fine-grained semantic differences among the responses, which encode information that can make uncertainty quantification more precise.To fill these gaps, a framework is developed in this paper for a new uncertainty/confidence metric, semantic density (SD), that can quantify the confidence of LLM responses in semantic space. Semantic density analyzes the output probability distribution from a semantic perspective and extracts a confidence indicator analogous to probability density. The proposed semantic density metric has the following advantages: (1) The returned confidence metric is response-wise, making it possible to evaluate the trustworthiness of each specific response; (2) it takes the fine-grained semantic differences among responses into account, which makes uncertainty quantification more precise; (3) it does not need any further training or fine-tuning of the original LLM; it is an "off-the-shelf" tool that can be directly applied to any pre-trained LLMs without modifying them; and (4) it does not pose any restrictions on the problem type; in particular, it works for general free-form generation tasks.The performance of the semantic density metric was compared with six existing uncertainty/confidence quantification methods designed for LLMs across four question-answering benchmark datasets. All the approaches were tested on seven SOTA LLMs, including the latest Llama 3 and Mixtral-8x22B models. Semantic density performed significantly better than the alternatives across the board, suggesting that it forms a promising foundation for evaluating the trustworthiness of LLM responses. The source codes for reproducing the experimental results reported in this paper are provided at: https://github.com/cognizant-ai-labs/semantic-density-paper. 
related work: 
 In the LLM literature, the terms "uncertainty" and "confidence" are used in a mixed manner. A number of studies [51, 25, 12, 4, 16] treat uncertainty and confidence as two facets of a single concept, i.e., lower confidence on one particular response corresponds to higher uncertainty (or lower certainty). Other studies try to further differentiate "uncertainty" from "confidence" [26] and only use "uncertainty" to describe the entire output distribution instead of a specific response [22, 1] . Both perspectives fall in the same research area of "uncertainty quantification/estimation" [17, 26, 9, 16] , and the goals of most existing uncertainty/confidence metrics are indeed the same: to provide a quantitative indicator of the trustworthiness of LLM responses. For better coverage and clarity, this paper uses "uncertainty quantification" as a general term to describe work related to the assessment of uncertainty or confidence of LLMs, and "uncertainty/confidence" to refer to multiple metrics with mixed term definitions. The proposed semantic density is thus an indicator of response-wise "confidence".Although the main focus of the LLM community is still on developing new models with better performance, a number of studies aim at measuring uncertainty/confidence in LLMs. This section summarizes their basic ideas and potential limitations, which are then targeted in the development of semantic density.The first direction is to ask the LLM to evaluate the uncertainty/confidence of its own responses. Tian et al. [44] performed an empirical study showing that the inherent conditional probabilities are poorly calibrated in existing LLMs with RLHF (reinforcement learning from human feedback), and that verbal confidence estimates provided by the LLMs are better calibrated. Kadavath et al. [21] developed an approach where the LLM was asked to evaluate the correctness of its own answer, in the form of the probability "P(True)" that its own answer is correct. An additional "value" head was also trained to predict "P(True)", but it turned out not to generalize well to out-of-distribution datasets. In general, the performance of model self-evaluation is not as good as other more advanced uncertainty quantification methods [22] .A second direction is to integrate traditional uncertainty quantification methods into LLMs. The effectiveness of temperature scaling [13] for calibrating the output token probabilities of LLMs was verified in Desai and Durrett [6] and Xiao et al. [51] . Ye et al. [53] utilize conformal prediction to quantify the uncertainty of LLMs. However, these approaches are limited to NLP classification tasks; in contrast, the proposed semantic density can be applied to the more general and challenging free-form generation tasks.A third direction is to perform supervised learning, either by fine-tuning the original LLM or adding an additional layer/classifier, to create uncertainty/confidence indicators. For example Lin et al. [25] fine-tuned the GPT-3 model to verbally express its own confidence level, and Azaria and Mitchell [3] trained an additional classifier to return the truthful probability of each generated response, based on the hidden layer activations of the LLM. Liu et al. [28] proposed the LitCab framework, in which a single linear layer is trained over the LLM's last hidden layer representations to predict a bias term, and the model's logits are then altered accordingly to update the response confidence. Since these methods are model-specific and need additional task-specific training labels, they cannot be readily applied to new models and tasks. In comparison, as an unsupervised method, semantic density is "off-the-shelf" for any new models and tasks, without the need for additional data or modifications to the original LLMs.A common limitation of all the above methods is that they only consider the lexical information, and do not take into account the semantic relationships between responses. Yet semantics is critical in analyzing LLM outputs. As a fourth direction, semantic entropy [22] is a SOTA technique that quantifies semantic uncertainty for LLMs. It works by grouping the generated samples based on their semantic equivalence, and then generating an entropy-based indicator as an uncertainty metric. Although its performance is promising compared to the other approaches, as discussed in Section 1, it has two intrinsic limitations: (1) The generated semantic entropy is prompt-wise instead of response-wise, and (2) only a one-cut equivalence relationship is considered during semantic analysis. The second issue was considered in a recent follow-up [33] that tries to improve semantic entropy. However, it did not resolve the first issue, and the proposed uncertainty metric is still prompt-wise, limiting its utility when different responses are sampled given the same prompt. The proposed semantic density improves over these two aspects by providing a response-specific confidence indicator and analyzing the semantic relationship in a fine-grained manner.Besides the above four major directions, uncertainty quantification for LLMs has been explored from other angles as well. Lin et al. [26] tested several simple baselines, among which a straightforward measurement of semantic dispersion is robust in evaluating selective response generation. Similarly, Manakul et al. [30] proposed a framework with several variants that use sampling consistency for detecting hallucinations. These two studies assume a very restricted condition in which the original sampling likelihoods of each response are not used; thus, the uncertainty/confidence information extracted by these methods is limited. Duan et al. [7] proposes a mechanism to shift attention to more relevant components at both token and sentence levels for better uncertainty quantification. Their approach was applied to prompt-wise uncertainty metrics only. The study by Ling et al. [27] focused on a specific in-context learning setting, aiming to decompose the uncertainty of LLMs into that caused by demonstration quality and that caused by model configuration. These experiments were limited to classification problems. Similarly, Hou et al. [15] decomposed the uncertainty into data uncertainty and model uncertainty in a prompt-wise approach. Xiao and Wang [52] studied the connections between hallucination and predictive uncertainty, showing that higher uncertainty is positively correlated with a higher chance of hallucination. This study validates the importance of a reliable uncertainty measurement in detecting hallucinations of LLMs. Finally, Huang et al. [17] perform an explorative study using simple baselines on uncertainty measurement for LLMs, highlighting the need for more advanced uncertainty quantification methods developed exclusively for LLMs. This is the goal for the current paper as well. 
methodology: 
 This section first defines the LLM uncertainty quantification problem, then describes the design principles and technical details of each algorithmic component, and concludes with a summary of the entire framework. The semantic space defined in Section 3.2 forms a theoretical foundation for semantic density. It can be implemented either explicitly through an embedding model or implicitly through an inference model; the latter is the approach used in this paper (details in Section 3.5).Given a pre-trained LLM, an input prompt x, and an output sequencey = [y 1 , y 2 , • • • , y L ],where L is the number of tokens in y, the target is to produce a confidence metric that is positively correlated with the probability of y to be true. Note that this metric should be response-wise, i.e., it is calculated for a specific y given x. The metric can be used as a quantitative indicator for whether a specific response y can be trusted.Theoretically, a semantic space can be any metric space such that a distance function is properly defined to measure the semantic similarity between any two output responses, given the input prompt. Note that such a space is prompt-specific, i.e., each prompt results in a specific semantic space in which the distance function measures the contextual semantic similarity between two responses, treating the prompt as a common context. More concretely, an oracle semantic space S is assumed to be a Euclidean space where each point is a D-dimensional vector that represents a contextual embedding of response y given prompt x:EQUATIONwhere v ∈ R D , and E(•|•) is an encoder that generates text embeddings with the following properties:1. All the generated embedding vectors are normalized to have a norm of 1 2 :EQUATIONWhereas most existing text embedding models normalize the output vectors to have a norm of 1 [32] , they are rescaled to 1 2 without changing their direction to make it simpler to integrate them into the kernel function (as explained in Section 3.4).2. Given a prompt x and two resulting responses y i and y j , with v i = E(y i |x) and v j = E(y j |x), the following constraints exist for three extreme cases:EQUATION2 , if y i and y j are semantically irrelevant given context x 1, if y i and y j are semantically contradictory given context x.(3) Given the norm requirement in Eq. 2, the above three cases also correspond to v i = v j , v i ⊥ v j and v i = -v j , respectively. Note that ||v i -v j || is not restricted to the above three values. It can be any value within [0, 1], depending on the semantic similarity between y i and y j given x.3. Given a prompt x and three resulting responses y i , y j and y k , with v i = E(y i |x), v j = E(y j |x)andv k = E(y k |x), ||v i -v j || < ||v i -v k ||, if y i is semantically closer to y j than to y k , given context x. (4)Given the semantic space S defined in Section 3.2, the underlying probability distribution from which the LLM samples in S provides critical information: If a response is semantically close to many highly probable samples, it should be more trustworthy compared to a response that is semantically distant from the major sampling possibilities. A classical technique for estimating probability density is kernel density estimation (KDE) [40, 36] . However, the standard KDE only works for continuous variables, whereas the LLM outputs are discrete, i.e., sequences of tokens selected from a finite vocabulary. One possible way to extend KDE to accommodate LLM outputs is to build a density estimator asEQUATIONwhere x is the input prompt, y * is the target response, i.e., the response that needs confidence estimation, and v * = E(y * |x). In total, M i=1 n i reference responses are sampled to facilitate the density estimation, where M is the number of unique samples. Each y i represents a unique sample; n i is the number of occurrences of y i during sampling, f i = ni M i=1 ni is the relative frequency of y i during sampling, and K(•) is a kernel function.The design of Eq. 5 is similar to an early variant of KDE [37] that was used to handle integer data. However, it has the drawback that it incorporates no knowledge about the sampling probabilities for each y i . It thus requires a large number of samplings, including a sufficient number of duplicated results, to obtain the relative frequency as an empirical approximation of the sampling probability. This cost becomes prohibitive for LLMs given how expensive LLM inference generally is.In contrast with the inherently unknown probability distributions in standard KDE, the output token probabilities can be explicitly calculated in LLM sampling, and with this information, a more sampleefficient estimator can be developed. Given a prompt x and a resulting response y * , the semantic density of y * is defined asEQUATIONwhere v * = E(y * |x), andv i = E(y i |x) for i = 1, 2, • • • , M. The M unique responses y i are the reference responses based on which the semantic density of y * is estimated. K(•) is a kernel function which will be specified in Section 3.4, and p(y i |x)(for i = 1, 2, • • • , M ) is the probability for the orig- inal LLM to generate sequence y i given x. That is, p(y i |x) = Li j=1 p(y i,j |y i,1 , y i,2 , • • • , y i,j-1 , x), where L i is the number of tokens in y i and p(y i,j |•) is the conditional probability to generate token y i,j . Note that in cases where p(y * |x) is available, y * can also be used as one of the M reference responses.One advantage of the semantic density estimator of Eq. 6 is that each result y i only needs to be sampled once; their relative frequency f i can then be estimated asf i = p(yi|x) M i=1 p(yi|x). Given a sampling budget of M reference responses, it is therefore desirable that these M samples are unique (duplications will be removed before calculating Eq. 6) and have high sampling probabilities, so that they can cover more sampling regions in the semantic space. In the current implementation, diverse beam search [46] , which tends to generate diverse and highly probable responses, is used to sample the M unique reference responses.In practice, length-normalized probability [31, 29] is usually used to correct the length bias in sequence probability. Moreover, temperature scaling [13, 6, 51] is a simple yet effective method for calibrating the token probabilities during sampling. Both methods can be seamlessly integrated into semantic density: The p(y i |x) in Eq. 6 can be replaced with L i p(y i |x), and the temperature changed during sampling to calibrate each p(y i,j |•).In a standard KDE setup, a commonly used kernel for multi-variate cases is the Epanechnikov kernel [8, 11] , which was proved to be the most efficient in terms of asymptotic mean integrated squared error [47] . Its original form isEQUATIONwhere D is the dimension of vector v, Γ(•) is the gamma function, ||v|| is the 2-norm of v, and 1 condition equals 1 if the condition is true, 0 otherwise.In the semantic density estimator use case, one drawback of the original Epanechnikov kernel is that the normalization coefficientΓ(2+ D 2 ) π D 2changes with the dimension D of v. As a result, semantic densities calculated using embeddings with different dimensionalities are incomparable. This issue may limit the flexibility in selecting embedding methodologies for semantic density calculation. However, the normalization coefficient can be removed to make the kernel function simpler and more flexible, without affecting the performance of confidence measurement. The kernel function of Eq. 6 thus becomesEQUATIONAlthough the resulting kernel function does not meet the normalization requirement in standard KDE, it fits confidence estimation well. As long as the norm requirements in Eq. 2, 3 and 4 are fulfilled, any embedding models can be used to generate v, regardless of the embedding dimensionalities. The outcome of the kernel function is always within [0, 1], and a kernel value of 1, 1 2 , and 0 correspond to semantically equivalent, irrelevant, and contradictory responses, respectively. As a result, the semantic density is also within [0, 1], with 1 as the highest semantic density a response can obtain, indicating that all the reference responses are semantically equivalent to it; analogously, it obtains a semantic density of 0 when all reference responses are semantically contradictory to it. This consistency in the value range makes practical applications of semantic density convenient: Practitioners can set a fixed threshold on semantic density to detect unreliable responses.Although most of the existing text-embedding models work in the semantic space defined in Section 3.2, they do not perform well in measuring semantic similarities [32] . Moreover, they can only consider input texts as a whole instead of doing a contextual encoding on part of the input, i.e., they can only obtain E(x + y) instead of E(y|x), where x + y means a concatenation of x and y.The natural language inference (NLI) classification model [14] has proven to be effective in analyzing the semantic relationship between LLM responses with the prompt as context [22] . Given a pair of texts, an NLI model performs a classification task and outputs the probabilities for them to be semantically equivalent ("entailment" class), irrelevant ("neutral" class), or contradictory ("contradiction" class). Given the output class probabilities, the expectation of ||v * -v i || can be obtained asEQUATION2EQUATIONwhere p c (y * , y i |x), p n (y * , y i |x) and p e (y * , y i |x) are the probabilities for y * and y i to be semantically contradictory ("c" for "contradiction" class), irrelevant ("n" for "neutral" class) and equivalent ("e" for "entailment" class), respectively, given context x. During implementation, each response y will be concatenated with its prompt x (with prompt placed before the response) to form one text, i.e., x + y. Each input of the NLI model will then be a pair of these texts, analyzing the semantic relationship between two responses given the prompt. The expected value of ||v * -v i || 2 can then be used in Eq. 8 to obtain the kernel function output.Algorithm 1 describes how the semantic density metric is deployed on a given task and model. The procedure consists of four main steps, i.e., sampling the reference responses, analyzing semantic relationships, calculating the kernel function, and calculating the semantic density. calculate kernel function value using the expectation of ||v * -vi|| 2 , given by:K(v * -vi) = (1 -E(||v * -vi|| 2 ))1 E(||v * -v i ||)≤1Step 4: Semantic Density Calculation: 7: calculate semantic density:SD(y * |x) = 1 M i=1 L i √ p(y i |x) M i=1 L i p(yi|x)K(v * -vi)one beam in each group, and thus only M inferences need to be done by the original LLM. The second step requires another M or 2M inferences by the NLI classification model, depending on whether the relationship analysis is performed in a bi-directional manner. Considering the fact that NLI models are usually significantly smaller than LLMs (e.g., the Deberta-large-mnli model [14] used in the implementation in this paper only has 1.5 billion parameters), the computational cost is therefore mainly determined by the LLM inferences in the first step.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] Justification: There are no new theorems proposed in this work. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experiments: 
 This section first evaluates the performance of semantic density by comparing it with six existing uncertainty/confidence metrics over various LLMs and benchmarks. After that, two empirical studies are performed to investigate the robustness of semantic density when the number of reference responses and sampling strategy for target response are varied.Following the usual evaluation approach in the literature [22] , the uncertainty/confidence metric is used as a quantitative indicator of how likely the response is going to be correct. Uncertainty values above a threshold are taken as incorrect while those below are taken as correct (and vice versa for confidence scores). For each threshold, the true positive rate vs. false positive rate is then measured. The area under this curve, namely area under receiver operator characteristic curve (AUROC), is calculated for each uncertainty/confidence metric. The AUROC score equals the probability that a randomly chosen incorrect response has a higher uncertainty than a randomly chosen correct response (and vice versa for confidence). A perfect uncertainty/confidence metric would have an AUROC score of 1 while a random metric would have 0.5. As an additional performance metric, the area under the Precision-Recall curve (AUPR) is also calculated. The average AUPR scores over two setups, i.e., using either correct or incorrect samples as the positive class, are reported. Note that a higher semantic density corresponds to a higher confidence.The performance of semantic density (SD) was compared with six existing LLM uncertainty quantification methods: semantic entropy (SE) [22] , P(True) [21] , degree (Deg) [26] , length-normalized likelihood (NL) [31] , length-normalized entropy (NE) [29] and predictive entropy (PE) [21] . These methods were applied to seven SOTA open-source LLMs: Llama-2-13B [45] , Llama-2-70B [45] , Llama-3-8B [2] , Llama-3-70B [2] , Mistral-7B [18] , Mixtral-8x7B [19] and Mixtral-8x22B [43] .Each LLM was tested on four free-form question-answering datasets commonly used in the literature: CoQA [38] , TriviaQA [20] , SciQ [48] and Natural Questions (NQ) [23] . For each question, 10 responses were generated using group beam search and used as reference responses in calculating SD, SE, Deg, NE, and PE (note that P(True) and NL do not need reference responses). Each unique response among these 10 will also be used as a target response, i.e., the response that needs an uncertainty/confidence estimation, in calculating the AUROC scores of uncertainty/confidence metrics. Detailed experimental configuration and parametric setup is described in Appendix A.1.Table 1 and Table A2 (in Appendix A.4) show the AUROC and AUPR scores of each uncertainty/confidence metric across different models and datasets, with the best entry in each configuration highlighted in boldface. SD performs best in 26 out of 28 cases for AUROC and 27 out of 28 cases for AUPR, demonstrating that it is reliable and robust as a confidence metric for LLM responses. For AUROC, in two cases it is outperformed by Deg. After investigation, the inherent sequence likelihood returned by the original LLM was badly calibrated in these two cases. Deg is the only method that ignores the likelihood information during its calculation, making its performance unaffected by this negative factor. However, for the other 26 cases, SD is able to utilize the likelihood information to its advantage and outperform Deg. Appendix A.5 includes the results of another variant of SE [10] , which is comparable to the original version. SD outperforms it in all the test cases.To confirm that the observed performance differences in Table 1 are statistically significant, a paired t-test (paired by LLM and dataset) was performed between SD and the other metrics (Table A1 in The experiments reported above use the same setup as Kuhn et al. [22] to evaluate the correctness of responses: If the Rouge-L [24] between one response and the reference answer is larger than 0.3, then the response is deemed as correct. In order to investigate the influence of the correctness-checking criterion, the threshold of Rouge-L was varied between 0.1 and 1.0; the resulting performance of different uncertainty/confidence metrics are shown in Figure A1 . SD consistently outperforms other methods under the different Rouge-L thresholds. Moreover, when the Rouge-L threshold increases, the AUROC scores generally increase for SD whereas they decrease for most other methods. Since a higher Rouge-L threshold means stricter correctness checking, the experimental performance gain of SD may be even larger if such checking is further improved (e.g. by using a more capable LLM).To further evaluate the generalizability of SD in other types of tasks, an empirical study using a summarization task (DUC-2004 [35] ) was performed. Appendix A.6 shows the performance comparisons among different uncertainty/confidence metrics. SD performs best in most test cases, demonstrating good generalizability.Two additional empirical studies were performed to evaluate the robustness of semantic density when the number of reference responses varies or the sampling strategy for target response changes.In the first study, the number of reference responses was reduced from 10, which is a standard setup for existing methods [22] , to one, which is the extreme minimum case. Figure 1 shows the resulting AUROC scores, covering the same four datasets and seven LLMs. Although performance indeed decreases with fewer reference responses, the decrease is minor as long as the number of references is at least four. This result suggests that semantic density can provide reasonable performance even with a very limited budget for reference sampling.In real-world applications, users may have different preferences when generating responses: Some may prefer a greedy sampling strategy while others may need diverse responses. The second study thus investigated how each uncertainty/confidence metric performs when the target response is sampled using different such strategies. The diverse beam search method inherently utilizes different strategies for each beam group: The first group performs a greedy beam search while later groups encourage more diverse responses. Following Section 4.1, the AUROC scores were calculated for target responses from each group separately, and the results averaged over the four datasets.As the results in Figure 2 show, semantic density exhibits consistently good AUROC scores across different beam groups. Thus, it is robust against both more greedy and more diverse sampling strategies, thus covering a range of possible user preferences. In contrast, other approaches either perform consistently worse compared to semantic density across different beam groups, or their performance is unstable when the sampling strategy changes. Figure 2 : AUROC scores over the different beam groups. The plots show the average AUROC scores over the four datasets for the different beam groups in diverse beam search. A smaller group index corresponds to the group with a more greedy generation strategy, while the group with a larger index tends to be more diverse during response generation. Each subfigure corresponds to one of the seven LLMs, as indicated by the subfigure heading. Each curve represents one uncertainty/confidence metric indicated by the legend at the lower right. Semantic density exhibits consistently better and stable performance across different groups, compared to other methods.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: The detailed description about each algorithmic step of the proposed semantic density is provided in Section 3.6. The detailed guidelines about the experimental configurations are provided in Appendix A.1. Source codes are provided in the supplementary material (as an attached zip file). Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Since the proposed method intends to improve the trustworthiness of LLMs, there is no obvious negative societal impacts. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
discussion and future work: 
 In terms of the broader societal impact of this work, the proposed semantic density provides a general way to evaluate the trustworthiness of responses generated by LLMs. This ability should have a positive impact on real-world applications that are safety-critical, such as healthcare and finance. Practitioners can utilize semantic density as an off-the-shelf indicator to filter out unreliable responses.One limitation of semantic density is that it needs access to the output probabilities of generated tokens, which may not be available in some proprietary LLMs. In such a case, the more expensive variant in Eq. 5 can be considered as an alternative. Since semantic density does not require any further access to the internal states or weights of the original LLMs, it is still widely applicable. Another limitation is that most responses in the current experiments are at the sentence level. However, an extension of semantic density to long-paragraph responses should be feasible. Following the existing solutions [28, 10] , a long response can be decomposed into sentence-level claims [28] or factoids [10] , and semantic density then applied to estimate the confidence of each claim/factoid. The framework for measuring semantic density is modular, and therefore its main components can be extended in future work. First, new sampling strategies that explicitly encourage a better coverage of semantic space can be developed to generate reference responses. Such extensions should improve the reliability of semantic density further. Second, text embedding methods that can measure contextual semantic similarity between responses more reliably will be helpful as well. Third, kernel functions specifically designed for the semantic space should allow the utilization of semantic relationships more efficiently. Fourth, more precise methods for calibrating inherent token probabilities will form a more reliable base for calculating semantic density. 
conclusion: 
 This paper proposes semantic density as a practical new metric for measuring the confidence of LLM responses. It overcomes the limitations of existing approaches by utilizing the semantic information in an efficient and precise manner. It is response-specific, "off-the-shelf", and applicable to free-form generation tasks. Experimental comparisons with six existing uncertainty/confidence metrics across seven SOTA LLMs and four benchmark datasets suggest that it is accurate, robust, and general, and can therefore help deploy LLMs in safety-critical domains. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This work poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: All the models, datasets and source codes used in this work have been properly cited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: This section provides the details of the experimental setup for reproducing the results presented in Section 4. The source codes for reproducing the reported experimental results are provided at: https://github.com/cognizant-ai-labs/semantic-density-paper.Base LLMs: For all the seven base LLMs, the open-source versions in Huggingface Transformers library [49] were used in the experiments.More specifically, the following versions are used: meta-llama/Llama-2-13b-hf, meta-llama/Llama-2-70b-hf, meta-llama/Meta-Llama-3-8B, meta-llama/Meta-Llama-3-70B, mistralai/Mistral-7B -v0.1, mistralai/Mixtral-8x7B-v0.1, and mistralai/Mixtral-8x22B-v0.1. When running diverse beam search in generating the responses, the generate() function is used with diversity_penalty=1.0, a default temperature of 1.0, and the num_beam_groups equals the number of beams so that each group has exactly one beam.Correctness Checking: Following the setup in Kuhn et al. [22] , for all the reported experimental results except Figure A1 , a response is considered to be correct if the Rouge-L [24] between it and any of the reference answers is larger than 0.3, after trimming the redundant continuations.The details of the datasets are as follows:• CoQA: The coqa-dev-v1.0 version is used with 1596 questions randomly selected for the experiments, using the Huggingface datasets.train_test_split function with a seed value of 10. The prompt format follows the same setup as in Kuhn et al. [22] .• TriviaQA: The dataset is loaded with the Huggingface datasets API. 1705 questions were randomly selected from the validation split for the experiments, using the Huggingface datasets.train_test_split function with a seed value of 10. A 10-shot prompt format is used following the setup in Kuhn et al. [22] .• SciQ: The test split from https://github.com/launchnlp/LitCab is used in the experiments. It contains 990 questions. A 10-shot prompt format is used following the setup in Kuhn et al. [22] .• NQ: The test split from https://github.com/launchnlp/LitCab is used in the experiments. 1800 questions were randomly selected using the Huggingface datasets.train_test_ split function with a seed value of 10. A 10-shot prompt format is used following the setup in Kuhn et al. [22] .• DUC-2004: All the 500 samples in task 1 [35] is used in the experiments. The dataset is downloaded from https://www.kaggle.com/datasets/sandeep16064/duc2004. The following prompt is used for all the experiments: "This is a bot that summarizes the following paragraph using one sentence. \n Paragraph: {document} \n Summary:".Uncertainty/Confidence Metrics: The same exact target response and reference responses are used for all the tested methods. For SD, SE, and Deg, the microsoft/deberta-large-mnli model from Huggingface Transformers library [49] is used as the NLI classification model. Following Kuhn et al. [22] , the probabilities for "contradiction", "neutral" and "entailment" are averaged bidirectionally. The parametric setup for the uncertainty metrics tested in Section 4 is summarized as follows:• semantic density (SD): The same exact steps as described in Algorithm 1 were implemented, with a fixed temperature of 0.1 applied to rescale the value of each token probability during postprocessing.• semantic entropy (SE): The original implementation and parametric setup from https://github.com/lorenzkuhn/semantic_uncertainty was used.• P(True): The original few-shot prompt format from Kadavath et al. [21] is used.• degree (Deg): The "entailment" probability returned by the NLI model (averaged bidirectionally) is used as the similarity between two responses, which is then used as the diagonal element in the degree matrix.• length-normalized likelihood (NL): The original form as in Murray and Chiang [31] was implemented.• length-normalized entropy (NE): The implementation from https://github.com/lorenzkuhn /semantic_uncertainty was used.• predictive entropy (PE): The implementation from https://github.com/lorenzkuhn/semantic _uncertainty was used.Compute Resources: All the experiments in Section 4 were running on an AWS P4de instance with 96 Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz, 1152GB memory and 8 NVIDIA A100 (80GB). The GPU memory required to run the experiments is dependent on the base LLMs, as detailed below:• Llama-2-13B: ∼30GB GPU memory.• Llama-2-70B: ∼140GB GPU memory.• Llama-3-8B: ∼20GB GPU memory.• Llama-3-70B: ∼140GB GPU memory.• Mistral-7B: ∼20GB GPU memory.• Mixtral-8x7B: ∼110GB GPU memory.• Mixtral-8x22B: ∼300GB GPU memory.The exact computation time was affected by many factors, e.g., the current workload of the machine, the prompt length of the question, the generated response length, whether the cache option is turned on to store the model states of LLMs, etc.Table A1 shows the p-values of the paired t-tests as described in Section 4.1. Figure A1 shows the performance changes of the uncertainty/confidence metrics with different Rouge-L thresholds when checking the correctness of responses. SD performs the best overall across these different criteria.During this evaluation, each uncertainty/confidence metric was used as a quantitative detector in two cases: 1) detecting incorrect responses, 2) detecting correct responses. The AUPR scores for both cases were calculated. Table A2 shows the average AUPR scores for them. Again, SD performs the best overall.Table A3 shows the performance of a follow-up variant of semantic entropy [10] . The performance of this variant is comparable with the original semantic entropy, with several cases worse than the original version. The proposed semantic density outperforms this variant in all test cases. The Rouge-L threshold for checking response correctness is set as 0.1, 0.3, 0.5, 0.7, 0.9 and 1.0, respectively. Each subfigure corresponds to one combination of base LLM and dataset, indicated by the subfigure heading. Each curve corresponds to the performance of one uncertainty/confidence metric identified in the legend at the very top and bottom. For each subfigure, the horizontal axis indicates the Rouge-L threshold, and the vertical axis indicates the AUROC score. Overall, semantic density exhibits best performance under different Rouge-L thresholds. The experimental performance of SD increases when the correctness-checking criterion becomes stricter (a higher Rouge-L threshold), while most other methods show opposite trends. Justification: There are several discussions about the limitations of the proposed semantic density in the current manuscript: (1) The second paragraph of Section 5 discusses the potential limitation regarding the applicability in some restricted cases; (2) the third paragraph in Section 4.1 discusses a negative factor that may impair the performance of the proposed method; (3) section 4.2 studies other factors that may impact the performance of semantic density, and found it to be robust to these factors; (4) the computational efficiency is considered in algorithmic design (the second last paragraph of Section 3.3) and discussed in Section 3.6; (5) scalability to LLMs with different sizes are empirically evaluated in Section 4.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 


Decision: The paper proposes an approach based on kernel density estimation techniques in the semantic space for uncertainty quantification in text generation for large language models. The paper evaluated on question answering data sets, and during the rebuttal process, added results on summarization data sets as well. The reviewers agreed that the approach is principled and the emprical results are strong across benchmarks.

One concern from reviewer koQE is that semantic density lacks a solid theoretical foundation within the context of current uncertainty quantification. However, I think in practical terms, calibration to quantify the confidence of a sampled answer (e.g., at temperature 0) is useful, and the evaluation metric of AUROC also reflects this goal.

Reviews: review: paper_summary: paper_summary: The paper proposes an approach for estimating uncertainty of LLM outputs. The proposed approach begins by sampling a diverse set of responses, analyses their equivalence using a NLI classification model, and then computes the semantic density using a kernel density estimate.
strength_weakness: strength_weakness: - The paper is somewhat vague in distinguishing itself from the closest prior work on Semantic Entropy - this makes its specific contribution unclear.
- Some details of the implementation of semantic density are unclear.
questions: questions: - Sec 3.5: Please describe how the NLI model is used for computing p_c, p_n and p_e. NLI models typically accept pairs of texts, but these probabilities seem to contain three: y_*, y_i and x.
- Do I understand correctly that embedding models (discussed in Sec 3.2) are never explicitly used in Semantic Density? Instead the NLI model is relied upon for establishing equivalence between texts and for the kernel function? If yes, the current presentation comes across as a bit confusing - please consider rewriting parts of the paper to make this more explicit.
- As I understand, the proposed approach only estimates uncertainty rather than explicitly calibrating the underlying model - and yet the proposed uncertainty estimation method helps improve predictive performance in Table 1. Am I understanding correctly that the underlying models are somewhat well calibrated to begin with? To put it another way, are there generation tasks where one might expect SD to not be as reliable? For example, a summarization task. If this is a reasonable understanding, please consider discussing where SD is unlikely to work well.
- The differences highlighted against Semantic Entropy in Lines 105-107 are vague or inaccurate - SE considers responses in computing the SE(x) value so it is unclear how it is "prompt-wise", that being said, what are the advantages of obtaining uncertainty for a specific output and prompt pair? What does a "one-cut equivalence relationship" mean? Please consider highlighting differences more clearly and discussing the specific advantages/disadvantages that the differences result in.
limitations: limitations: NA
paper_summary: paper_summary: This paper addresses the problem of LLM’s lack of uncertainty metric for the response it generates. The authors propose to use semantic density to quantify such uncertainty, as it is not restricted to any specific downstream task. In particular, the approach samples reference responses, analyzes semantic relationships, and then calculates the semantic density. Experiments on four QA benchmarks of several Llama and Mistral models are conducted, including the latest Llama 3 13B and Mistral-8x22B, to demonstrate the effectiveness and the robustness of semantic density compared to prior metrics.
strength_weakness: strength_weakness: * Only some of the llama and mistral models are tested
* The results shown in Table 1 seem to be quite close for all the models (on each task), which are not so indicative for system comparison
* No further analysis/discussions on different model sizes (7B vs. 70B), nor different architecture (MoE vs. Non-MoE)
questions: questions: In Figure 1, different models have inconsistent performance across different tasks. Do you have any explanations?
limitations: limitations: Yes
paper_summary: paper_summary: The paper proposes a novel framework for quantifying uncertainty in LLM's responses. The authors introduce the concept of "semantic density" (SD), which measures uncertainty from a probability distribution perspective in semantic space, applicable to any pre-trained LLM without additional training. Experiments on various state-of-the-art LLMs and benchmarks demonstrate that SD outperforms existing methods in accuracy and robustness. The paper presentation is overall easy to read.
strength_weakness: strength_weakness: 1. The comparison with existing methods, although extensive, may not cover all possible alternatives. There are recent or lesser-known methods that also warrant consideration.
2. While the method shows promise for free-form question-answering tasks, its applicability and performance in other types of tasks (e.g., summarization, translation) are not explored, which limits the generalizability of the findings.
questions: questions: 1. The literature review is not comprehensive, there are lots of related works [1, 2, 3] not mentioned or adopted in the experiment.  
2. The AUROC is not enough the show the strength of the proposed model. It would be more beneficial to show Area under precision-recall curve for the misclassification detection task as well.  
3. The prompt construction can have more details. In Sec. 3.2, it's worth denoting the final prompt structure, since the in-context learning examples and prompt structures can make the final result different.
4. The dataset are mostly QA tasks, it seems the model can also handle other NLG tasks. Can you elaborate more on this?

[1] Ling, Chen, et al., "Uncertainty Quantification for In-Context Learning of Large Language Models." (NAACL 2024)   
[2] Fadeeva, Ekaterina, et al. "LM-polygraph: Uncertainty estimation for language models." (EMNLP 2023)    
[3] Duan, Jinhao, Hao Cheng, Shiqi Wang, Chenan Wang, Alex Zavalny, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. "Shifting attention to relevance: Towards the uncertainty estimation of large language models." ACL (2024).
limitations: limitations: N/A
paper_summary: paper_summary: The paper introduces the metric "Semantic Density" to quantify the uncertainty of LLMs by measuring the distance of response embeddings in a semantic space.
strength_weakness: strength_weakness: - **Theory**: Semantic Density is theoretically not well grounded in uncertainty quantification theory, but proposed as an ad-hoc solution for practical application. So, although it seems intriguing to assign an uncertainty score response-wise rather than prompt-wise, uncertainty theory suggests that the (aleatoric semantic) uncertainty has to be quantified prompt-wise, no matter what ends up being sampled as the response [1]. However, since the true uncertainty score for a given prompt is unknown, the performance of an uncertainty measure is usually evaluated by how well the scores align with the correctness of the most-likely response. 
- **Experiments**: According to the experimental setup in line 897, an answer is considered to be correct if its Rouge-L to any of the reference answers is larger than 0.3 for all datasets. However, since the considered datasets usually result in short responses, a fixed threshold of 0.3 is not sufficient. For instance, in the TriviaQA question *What is the last Grand Slam tennis tournament played in a calendar year?* the reference answer *The US Open* and the sampled response *The Australian Open* achieve a Rouge-L of approximately 0.66. This highlights the fact that considering multiple thresholds is essential to assess the performance of uncertainty metrics.

---
[1] L. Aichberger, K. Schweighofer, M. Ielanskyi, and S. Hochreiter. Semantically Diverse Language Generation for Uncertainty Estimation in Language Models. arXiv preprint arXiv:2406.04306, 2024.
questions: questions: - What do the authors exactly mean by Semantic Density rebuilding the output probability distribution from a semantic perspective? (line 58)
- In line 277, the authors write "After investigation, the inherent sequence likelihood returned by the original LLM was badly calibrated in these two cases.". What is meant by badly calibrated sequence likelihoods? Why didn't the authors account for this by recomputing the sequence likelihood?
- There exists independent work (Kernel Language Entropy) that is closely related to Semantic Density [2]. The authors should consider discussing this in their paper.
- It would be insightful to additionally report the performance of existing uncertainty estimation methods when decreasing the number of reference responses instead of only reporting Semantic Density with different models (section 4.2)
- Semantic Density uses diverse beam search to sample responses, requiring hyperparameters such as diversity penalty and number of groups. How have these hyperparameters been chosen? Is the method sensitive to the specific sampling strategy? An ablation study on the performance of Semantic Density compared to existing uncertainty estimation methods would be insightful.
- Have the authors considered using sampling strategies other than diverse beam search? For instance, Aichberger et al. (2024) introduced sampling semantically diverse sequences, which could be utilized to sample unique reference responses [1]. 

---
[2] A. Nikitin, J. Kossen, Y. Gal, and P. Marttinen. Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities. arXiv preprint arXiv:2405.20003, 2024.
limitations: limitations: The authors did not address the limitations of their work, only the limitations of existing uncertainty estimation methods.

Comments: title: Response to Reviewer koQE
Final remarks
Reply to Reviewer rbw2
Response to Reviewer koQE
Message to Reviewer koQE
Message to Reviewer kNB4
Thank you
Follow-up comment to Reviewer kNB4
Follow-up response to the general rebuttal
Response to Reviewer koQE
Detailed results for Rouge-L threshold of 1.0 and two ablations studies
Detailed results for Rouge-L threshold of 0.7 and 0.9
Detailed results for Rouge-L threshold of 0.1 and 0.5
Follow-up response to Reviewer koQE:
comment: Thanks for your final remarks. We will make sure to discuss and clarify this point in a careful way in the revised version, based on our insightful discussions. Thank you again for your time and effort!
I want to emphasize once more that, while the method empirically performs very well, it lacks a solid theoretical foundation within the context of current uncertainty quantification. The prevailing theory suggests that uncertainty quantification should focus on evaluating the predictive distribution (over classes, semantic clusters, etc.), irrespective of the sampled output. To draw a parallel with the typical classification setting, one also doesn't assign different uncertainty estimates to different classes sampled from the predictive distribution – to evaluate the performance of an uncertainty estimate, one uses the argmax to determine wether the model "knows" the correct class (i.e. should be certain) or "doesn't know" the correct class (i.e. should be uncertain). This principle can be similarly applied to uncertainty estimates in NLG. It should be carefully elaborated on in the updated version of the paper.
Thank you for spending time reading our rebuttal.
Many thanks for your reply. We will include the line plot as you suggested in the revision.
Thank you for providing the detailed results. As a side note, these 20 tables could have been effectively summarized in a line plot (x-axis: correctness threshold, y-axis: AUROC). Overall, I have decided to increase my score to 5.
Since we won't be able to post any replies after the discussion period (ending soon), please let us know if you have any further questions. Thank you!
Since we will not be able to post any further responses after the discussion deadline tomorrow, please feel free to let us know if you have any feedback about our rebuttal, including those newly added experiments as suggested by you. Thank you for your attention!
Thank you for the additional results and the responses.
As per the request by Reviewer koQE, we have now added the detailed results for all the newly added experiments, including the one suggested by your that adds a new variant of semantic entropy. Please see our follow-up comment in the general rebuttal above for detailed results. Thank you.
Due to the space limitation of initial rebuttal, we did not include all the detailed results for the newly added experiments. As per the request by Reviewer koQE, we have now added the detailed results for all the newly added experiments. The detailed results for experiment 3, 4 and 5 are added in the comments replying to Reviewer koQE. The detailed results for experiment 6 are directly included below:

## Results for the modified version of semantic entropy [1]:

| AUROC         | CoQA  | TriviaQA | SciQ  | NQ    |
|---------------|-------|----------|-------|-------|
| Llama-2-13B   | 0.587 | 0.726    | 0.576 | 0.576 |
| Llama-2-70B   | 0.570 | 0.696    | 0.596 | 0.555 |
| Llama-3-8B    | 0.606 | 0.736    | 0.587 | 0.582 |
| Llama-3-70B   | 0.571 | 0.694    | 0.582 | 0.563 |
| Mistral-7B    | 0.572 | 0.734    | 0.613 | 0.568 |
| Mixtral-8x7B  | 0.567 | 0.699    | 0.576 | 0.612 |
| Mixtral-8x22B | 0.562 | 0.690    | 0.592 | 0.565 |

Its performance is comparative to the original semantic entropy [2], and the proposed semantic density still outperforms it consistently across all the base LLMs and tasks.
 
[1] Farquhar, Sebastian and Kossen, Jannik and Kuhn, Lorenz and Gal, Yarin, “Detecting hallucinations in large language models using semantic entropy”, Nature 630, 625–630, 2024

[2] Lorenz Kuhn and Yarin Gal and Sebastian Farquhar, “Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation”, The Eleventh International Conference on Learning Representations (ICLR), 2023
Thanks for your comment. The motivation of most summarization benchmarks are to evaluate a model’s ability in compressing the information, so the DUC summarization task used here requires the language models to summarize a paragraph within a predefined length limit, usually no more than a single long sentence. The current NLI model we used (Deberta-large-mnli) still works reasonably in such tasks. However, we think you did bring up a good point for future work in this research direction, i.e., how should we quantify the uncertainty of a long paragraph? Recent solutions to this scenario focuses on decomposing the long paragraph into pieces of “claims” [1] or “factoid” [2], each one being at sentence level, then we can apply existing uncertainty quantification methods to each of them. This strategy makes sense since it is indeed more informative if we could quantify the uncertainty for each part when the response is a long paragraph containing multiple claims. However, it would also be interesting if we can further develop a NLI model or embedding model that can properly measure the overall semantic similarity between two long paragraphs, or we can directly ask another LLM to make this overall similarity evaluation quantitatively. This discussion may be a little out-of-scope, but we will add this discussion in the future work section. Thanks again for your inspiring comment.

[1] Xin Liu, Muhammad Khalifa, and Lu Wang. “LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses”. In The Twelfth International Conference on Learning Representations (ICLR), 2024
 
[2] Farquhar, Sebastian and Kossen, Jannik and Kuhn, Lorenz and Gal, Yarin, “Detecting hallucinations in large language models using semantic entropy”, Nature 630, 625–630, 2024
## Results for Rouge-L threshold of 1.0

### **CoQA**
| Rouge-L = 1.0 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.804|0.548|0.513|0.636|0.798|0.429|0.472|
|Llama-2-70b-hf|0.807|0.546|0.464|0.615|0.799|0.421|0.461|
|Meta-Llama-3-8B|0.767|0.495|0.528|0.730|0.685|0.420|0.515|
|Meta-Llama-3-70B|0.801|0.563|0.644|0.617|0.788|0.387|0.467|
|Mistral-7B-v0.1|0.815|0.561|0.557|0.651|0.784|0.413|0.460|
|Mixtral-8x7B-v0.1|0.811|0.565|0.487|0.635|0.799|0.420|0.471|
|Mixtral-8x22B-v0.1|0.809|0.537|0.484|0.612|0.786|0.419|0.471|
### **TriviaQA**
| Rouge-L = 1.0 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.893|0.546|0.551|0.756|0.914|0.465|0.509|
|Llama-2-70b-hf|0.895|0.559|0.560|0.708|0.903|0.466|0.490|
|Meta-Llama-3-8B|0.855|0.507|0.608|0.714|0.838|0.413|0.417|
|Meta-Llama-3-70B|0.843|0.540|0.602|0.679|0.828|0.425|0.446|
|Mistral-7B-v0.1|0.868|0.559|0.542|0.753|0.772|0.447|0.446|
|Mixtral-8x7B-v0.1|0.867|0.568|0.495|0.709|0.844|0.487|0.502|
|Mixtral-8x22B-v0.1|0.865|0.571|0.533|0.674|0.860|0.496|0.547|
### **SciQ**
| Rouge-L = 1.0 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.890|0.574|0.490|0.788|0.880|0.407|0.503|
|Llama-2-70b-hf|0.888|0.676|0.577|0.732|0.861|0.453|0.526|
|Meta-Llama-3-8B|0.881|0.602|0.544|0.734|0.881|0.464|0.652|
|Meta-Llama-3-70B|0.876|0.645|0.556|0.716|0.883|0.456|0.450|
|Mistral-7B-v0.1|0.861|0.574|0.556|0.737|0.806|0.412|0.397|
|Mixtral-8x7B-v0.1|0.856|0.608|0.553|0.721|0.851|0.493|0.604|
|Mixtral-8x22B-v0.1|0.857|0.620|0.587|0.708|0.863|0.490|0.592|
### **NQ**
| Rouge-L = 1.0 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.845|0.487|0.662|0.760|0.734|0.349|0.500|
|Llama-2-70b-hf|0.853|0.494|0.469|0.776|0.721|0.418|0.480|
|Meta-Llama-3-8B|0.813|0.429|0.418|0.703|0.819|0.409|0.595|
|Meta-Llama-3-70B|0.883|0.569|0.738|0.791|0.838|0.467|0.523|
|Mistral-7B-v0.1|0.815|0.502|0.392|0.732|0.737|0.427|0.530|
|Mixtral-8x7B-v0.1|0.847|0.535|0.531|0.769|0.730|0.473|0.547|
|Mixtral-8x22B-v0.1|0.859|0.526|0.391|0.772|0.773|0.504|0.655|

## Ablation study where the number of reference responses changes for Semantic Entropy

### **CoQA**
| sample number | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---------------|---|---|---|---|---|---|---|---|---|----|
|Llama-2-13b-hf|0.600|0.605|0.616|0.626|0.630|0.635|0.633|0.633|0.636|0.633|
|Llama-2-70b-hf|0.605|0.624|0.625|0.629|0.628|0.625|0.623|0.622|0.623|0.621|
|Meta-Llama-3-8B|0.571|0.579|0.585|0.598|0.601|0.600|0.606|0.609|0.608|0.599|
|Meta-Llama-3-70B|0.604|0.597|0.599|0.604|0.607|0.608|0.609|0.609|0.608|0.608|
|Mistral-7B-v0.1|0.572|0.596|0.612|0.617|0.624|0.626|0.626|0.628|0.630|0.627|
|Mixtral-8x7B-v0.1|0.594|0.609|0.619|0.621|0.620|0.625|0.625|0.624|0.625|0.626|
|Mixtral-8x22B-v0.1|0.591|0.603|0.614|0.617|0.616|0.615|0.614|0.614|0.614|0.614|
### **TriviaQA**
| sample number | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---------------|---|---|---|---|---|---|---|---|---|----|
|Llama-2-13b-hf|0.542|0.649|0.672|0.679|0.679|0.676|0.675|0.673|0.673|0.672|
|Llama-2-70b-hf|0.645|0.670|0.691|0.686|0.685|0.683|0.676|0.677|0.675|0.677|
|Meta-Llama-3-8B|0.656|0.674|0.685|0.685|0.682|0.680|0.674|0.669|0.665|0.662|
|Meta-Llama-3-70B|0.585|0.615|0.645|0.653|0.663|0.666|0.667|0.666|0.666|0.663|
|Mistral-7B-v0.1|0.674|0.694|0.703|0.707|0.704|0.703|0.699|0.693|0.692|0.690|
|Mixtral-8x7B-v0.1|0.600|0.644|0.678|0.682|0.685|0.687|0.687|0.688|0.687|0.685|
|Mixtral-8x22B-v0.1|0.615|0.655|0.677|0.688|0.692|0.693|0.692|0.691|0.688|0.686|

## Ablation study where the diversity penalty (dp) of diverse beam search changes for semantic density (using Mistral-7B)

| AUROC    | dp=0.2 | dp=0.5 | dp=1.0 |
|----------|--------|--------|--------|
| CoQA     | 0.784  | 0.785  | 0.788  |
| TriviaQA | 0.861  | 0.863  | 0.866  |
| SciQ     | 0.765  | 0.766  | 0.771  |
| NQ       | 0.678  | 0.678  | 0.680  |
## Results for Rouge-L threshold of 0.7:

### **CoQA**
| Rouge-L >= 0.7 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.796|0.577|0.569|0.675|0.767|0.494|0.509|
|Llama-2-70b-hf|0.802|0.576|0.544|0.662|0.780|0.493|0.503|
|Meta-Llama-3-8B|0.759|0.537|0.574|0.764|0.667|0.481|0.526|
|Meta-Llama-3-70B|0.788|0.570|0.665|0.654|0.754|0.456|0.485|
|Mistral-7B-v0.1|0.804|0.581|0.612|0.687|0.759|0.481|0.501|
|Mixtral-8x7B-v0.1|0.802|0.583|0.547|0.672|0.768|0.486|0.508|
|Mixtral-8x22B-v0.1|0.800|0.565|0.562|0.657|0.763|0.492|0.515|
### **TriviaQA**
| Rouge-L >= 0.7 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.875|0.559|0.590|0.767|0.869|0.505|0.513|
|Llama-2-70b-hf|0.874|0.572|0.587|0.728|0.871|0.514|0.524|
|Meta-Llama-3-8B|0.856|0.517|0.628|0.730|0.827|0.432|0.417|
|Meta-Llama-3-70B|0.838|0.552|0.624|0.696|0.821|0.458|0.461|
|Mistral-7B-v0.1|0.867|0.568|0.567|0.769|0.761|0.474|0.470|
|Mixtral-8x7B-v0.1|0.858|0.575|0.528|0.726|0.826|0.516|0.517|
|Mixtral-8x22B-v0.1|0.855|0.584|0.564|0.693|0.846|0.531|0.556|
### **SciQ**
| Rouge-L >= 0.7 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.872|0.577|0.507|0.786|0.860|0.422|0.507|
|Llama-2-70b-hf|0.864|0.671|0.586|0.732|0.835|0.481|0.548|
|Meta-Llama-3-8B|0.865|0.602|0.553|0.735|0.855|0.484|0.652|
|Meta-Llama-3-70B|0.860|0.641|0.561|0.721|0.857|0.474|0.463|
|Mistral-7B-v0.1|0.848|0.578|0.564|0.743|0.785|0.432|0.415|
|Mixtral-8x7B-v0.1|0.843|0.610|0.567|0.724|0.832|0.508|0.608|
|Mixtral-8x22B-v0.1|0.844|0.614|0.597|0.713|0.843|0.508|0.600|
### **NQ**
| Rouge-L >= 0.7 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.804|0.494|0.638|0.746|0.662|0.378|0.518|
|Llama-2-70b-hf|0.806|0.489|0.491|0.758|0.662|0.432|0.488|
|Meta-Llama-3-8B|0.789|0.454|0.437|0.715|0.742|0.423|0.584|
|Meta-Llama-3-70B|0.851|0.557|0.703|0.775|0.784|0.471|0.532|
|Mistral-7B-v0.1|0.792|0.527|0.410|0.737|0.694|0.441|0.544|
|Mixtral-8x7B-v0.1|0.825|0.542|0.538|0.765|0.704|0.483|0.550|
|Mixtral-8x22B-v0.1|0.829|0.530|0.407|0.764|0.722|0.501|0.640|

## Results for Rouge-L threshold of 0.9:
### **CoQA**
| Rouge-L >= 0.9 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.802|0.553|0.525|0.639|0.797|0.441|0.477|
|Llama-2-70b-hf|0.803|0.553|0.481|0.618|0.800|0.436|0.470|
|Meta-Llama-3-8B|0.766|0.505|0.537|0.735|0.684|0.432|0.514|
|Meta-Llama-3-70B|0.799|0.568|0.654|0.621|0.786|0.401|0.471|
|Mistral-7B-v0.1|0.811|0.566|0.567|0.654|0.784|0.426|0.466|
|Mixtral-8x7B-v0.1|0.809|0.568|0.501|0.636|0.798|0.432|0.477|
|Mixtral-8x22B-v0.1|0.807|0.544|0.496|0.617|0.786|0.433|0.478|
### **TriviaQA**
| Rouge-L >= 0.9 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.892|0.547|0.553|0.756|0.913|0.468|0.508|
|Llama-2-70b-hf|0.894|0.559|0.563|0.709|0.902|0.468|0.491|
|Meta-Llama-3-8B|0.855|0.508|0.609|0.714|0.838|0.414|0.417|
|Meta-Llama-3-70B|0.843|0.541|0.604|0.679|0.828|0.426|0.447|
|Mistral-7B-v0.1|0.868|0.559|0.544|0.753|0.772|0.449|0.447|
|Mixtral-8x7B-v0.1|0.867|0.569|0.498|0.709|0.844|0.489|0.503|
|Mixtral-8x22B-v0.1|0.865|0.571|0.535|0.674|0.860|0.497|0.547|
### **SciQ**
| Rouge-L >= 0.9 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.890|0.574|0.490|0.788|0.880|0.407|0.503|
|Llama-2-70b-hf|0.888|0.676|0.577|0.732|0.861|0.453|0.526|
|Meta-Llama-3-8B|0.881|0.602|0.544|0.734|0.881|0.464|0.652|
|Meta-Llama-3-70B|0.876|0.645|0.556|0.716|0.883|0.456|0.450|
|Mistral-7B-v0.1|0.861|0.574|0.556|0.737|0.806|0.412|0.397|
|Mixtral-8x7B-v0.1|0.855|0.608|0.554|0.721|0.851|0.494|0.604|
|Mixtral-8x22B-v0.1|0.857|0.620|0.588|0.708|0.863|0.490|0.591|
### **NQ**
| Rouge-L >= 0.9 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.845|0.487|0.662|0.760|0.734|0.351|0.500|
|Llama-2-70b-hf|0.852|0.495|0.471|0.776|0.722|0.420|0.481|
|Meta-Llama-3-8B|0.812|0.431|0.420|0.702|0.818|0.410|0.595|
|Meta-Llama-3-70B|0.882|0.569|0.738|0.790|0.838|0.468|0.523|
|Mistral-7B-v0.1|0.815|0.502|0.393|0.732|0.737|0.427|0.530|
|Mixtral-8x7B-v0.1|0.846|0.535|0.531|0.769|0.730|0.473|0.547|
|Mixtral-8x22B-v0.1|0.858|0.527|0.391|0.771|0.773|0.504|0.655|
Below are the AUROC scores when using a Rouge-L threshold of 0.1 and 0.5 (0.3 has already been shown in the original manuscript):

## Results for Rouge-L threshold of 0.1:

### **CoQA**
| Rouge-L >= 0.1 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.766|0.623|0.606|0.728|0.702|0.556|0.508|
|Llama-2-70b-hf|0.765|0.622|0.610|0.715|0.719|0.572|0.538|
|Meta-Llama-3-8B|0.727|0.601|0.617|0.790|0.632|0.566|0.511|
|Meta-Llama-3-70B|0.777|0.606|0.669|0.732|0.712|0.542|0.489|
|Mistral-7B-v0.1|0.776|0.616|0.667|0.737|0.705|0.553|0.508|
|Mixtral-8x7B-v0.1|0.766|0.624|0.623|0.726|0.707|0.555|0.520|
|Mixtral-8x22B-v0.1|0.777|0.617|0.646|0.729|0.710|0.560|0.525|
### **TriviaQA**
| Rouge-L >= 0.1 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.844|0.632|0.583|0.829|0.640|0.505|0.490|
|Llama-2-70b-hf|0.824|0.664|0.611|0.798|0.696|0.516|0.472|
|Meta-Llama-3-8B|0.862|0.572|0.652|0.793|0.798|0.480|0.425|
|Meta-Llama-3-70B|0.825|0.624|0.656|0.762|0.804|0.520|0.491|
|Mistral-7B-v0.1|0.855|0.620|0.589|0.822|0.693|0.487|0.476|
|Mixtral-8x7B-v0.1|0.830|0.654|0.569|0.789|0.742|0.541|0.511|
|Mixtral-8x22B-v0.1|0.818|0.660|0.618|0.758|0.779|0.577|0.535|
### **Sciq**
| Rouge-L >= 0.1 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.720|0.570|0.520|0.698|0.575|0.458|0.500|
|Llama-2-70b-hf|0.716|0.639|0.593|0.704|0.576|0.476|0.507|
|Meta-Llama-3-8B|0.755|0.583|0.582|0.716|0.633|0.468|0.526|
|Meta-Llama-3-70B|0.743|0.588|0.542|0.692|0.694|0.490|0.495|
|Mistral-7B-v0.1|0.761|0.588|0.578|0.729|0.635|0.462|0.468|
|Mixtral-8x7B-v0.1|0.751|0.598|0.602|0.707|0.675|0.523|0.566|
|Mixtral-8x22B-v0.1|0.756|0.600|0.613|0.712|0.677|0.500|0.525|
### **NQ**
| Rouge-L >= 0.1 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.697|0.589|0.576|0.704|0.555|0.521|0.503|
|Llama-2-70b-hf|0.691|0.553|0.590|0.701|0.555|0.506|0.488|
|Meta-Llama-3-8B|0.707|0.536|0.595|0.707|0.553|0.466|0.444|
|Meta-Llama-3-70B|0.701|0.573|0.574|0.711|0.556|0.503|0.490|
|Mistral-7B-v0.1|0.698|0.574|0.598|0.699|0.558|0.493|0.454|
|Mixtral-8x7B-v0.1|0.716|0.576|0.609|0.716|0.588|0.503|0.484|
|Mixtral-8x22B-v0.1|0.703|0.579|0.584|0.711|0.575|0.518|0.517|

## Results for Rouge-L threshold of 0.5:

### **CoQA**
| Rouge-L >= 0.5 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.791|0.598|0.582|0.716|0.701|0.503|0.515|
|Llama-2-70b-hf|0.791|0.599|0.559|0.697|0.712|0.506|0.527|
|Meta-Llama-3-8B|0.754|0.558|0.575|0.789|0.633|0.500|0.523|
|Meta-Llama-3-70B|0.790|0.585|0.669|0.702|0.694|0.470|0.494|
|Mistral-7B-v0.1|0.796|0.599|0.645|0.724|0.695|0.495|0.514|
|Mixtral-8x7B-v0.1|0.791|0.598|0.568|0.709|0.700|0.503|0.530|
|Mixtral-8x22B-v0.1|0.796|0.593|0.586|0.705|0.699|0.505|0.538|
### **TriviaQA**
| Rouge-L >= 0.5 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.854|0.593|0.588|0.807|0.689|0.479|0.518|
|Llama-2-70b-hf|0.839|0.620|0.548|0.773|0.710|0.491|0.519|
|Meta-Llama-3-8B|0.869|0.562|0.645|0.794|0.799|0.464|0.428|
|Meta-Llama-3-70B|0.832|0.616|0.648|0.761|0.801|0.504|0.489|
|Mistral-7B-v0.1|0.873|0.606|0.580|0.827|0.712|0.476|0.472|
|Mixtral-8x7B-v0.1|0.855|0.623|0.544|0.788|0.767|0.546|0.549|
|Mixtral-8x22B-v0.1|0.838|0.641|0.585|0.755|0.780|0.558|0.569|
### **SciQ**
| Rouge-L >= 0.5 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.796|0.557|0.550|0.746|0.704|0.418|0.511|
|Llama-2-70b-hf|0.779|0.644|0.569|0.712|0.649|0.460|0.548|
|Meta-Llama-3-8B|0.809|0.575|0.554|0.732|0.693|0.449|0.613|
|Meta-Llama-3-70B|0.800|0.601|0.567|0.716|0.716|0.452|0.454|
|Mistral-7B-v0.1|0.790|0.578|0.563|0.735|0.669|0.433|0.443|
|Mixtral-8x7B-v0.1|0.798|0.599|0.576|0.721|0.732|0.507|0.618|
|Mixtral-8x22B-v0.1|0.798|0.603|0.597|0.717|0.719|0.487|0.584|
### **NQ**
| Rouge-L >= 0.5 | SD | SE | P(True) | Deg | NL | NE | PE |
|----------------|----|----|---------|-----|----|----|----|
|Llama-2-13b-hf|0.723|0.494|0.591|0.706|0.528|0.392|0.496|
|Llama-2-70b-hf|0.719|0.487|0.503|0.717|0.534|0.438|0.486|
|Meta-Llama-3-8B|0.736|0.487|0.475|0.718|0.589|0.419|0.535|
|Meta-Llama-3-70B|0.774|0.545|0.666|0.747|0.641|0.455|0.524|
|Mistral-7B-v0.1|0.719|0.518|0.468|0.703|0.583|0.441|0.511|
|Mixtral-8x7B-v0.1|0.761|0.526|0.552|0.737|0.611|0.452|0.510|
|Mixtral-8x22B-v0.1|0.755|0.526|0.455|0.729|0.607|0.473|0.584|
Thanks for spending time reading our rebuttals and providing further comments! We did not include the detailed results in the initial rebuttal because this year Neurips only allows 6,000 characters in the initial rebuttal to each reviewer, urging the authors to be more concise. Moreover, only one page of PDF can be attached in the general rebuttal to include necessary tables/figures, and no external URL is allowed. Given this restriction, we have to conclude the additional experimental results with a very brief text summary in the initial rebuttal. However, we understand that providing the detailed results would be more informative to you, so we will attach all the detailed results in follow-up comments per your request. Due to the format limitation, we will use markdown tables to display all the results. Please check them in the comments following this one. We also provide some further discussions below:


The most interesting results are indeed the Rouge-L threshold study that you suggested as one of your main comments. In additional to the observation that the proposed semantic density still provides overall best performance across all the tested Rouge-L thresholds, we also noticed that the absolute AUROC scores show an increasing trend for semantic density when the Rouge-L threshold is increased. This indicates that the strictness of correctness check is indeed affecting the absolute performance of the evaluated methods. The initial Rouge-L threshold of 0.3 used in the experiments was following the same setup in the original semantic entropy paper [3], aiming to make the comparisons fair. However, we agree that it is indeed more informative to include results that are under different Rouge-L thresholds. We will include all the results in the revision, and add a discussion regarding this aspect. Thanks for this constructive comment!


[3] Lorenz Kuhn and Yarin Gal and Sebastian Farquhar, “Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation”, The Eleventh International Conference on Learning Representations (ICLR), 2023
Thank you for providing the additional results on the summarization dataset. This leads me to the following question: Your method employs an NLI model (i.e. Deberta-large-mnli) to compute the expectation of $|| v_* - v_i ||$ (Eq. 9). Given that this NLI model is trained on single-sentence pairs from the Multi-Genre Natural Language Inference corpus, how do you apply this model to multiple-sentence generations $v$?
Thank you for the rebuttal. However, I cannot locate any of the additional results you claim to have conducted for the rebuttal:

> As suggested, we have added experimental results using Rouge-L with different thresholds ranging from 0.1 to 1.0.

> We have added an experiment that investigates the performance change of semantic entropy when decreasing the number of reference responses. 

> As suggested, we have added an ablation study following the setup in reference [1], which tests different choices of diversity penalty.

Could you please provide these results for further review?