Improving Uncertainty Estimation through Semantically Diverse Language Generation
Authors: Lukas Aichberger
Kajetan Schweighofer
Mykyta Ielanskyi
Sepp Hochreiter
Venue: ICLR 2025 Poster (2025)
Keywords: llm
nlg
uncertainty estimation
uncertainty measures
semantic uncertainty
aleatoric uncertainty
semantic entropy
mc estimation
importance sampling
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We introduce SDLG, an efficient method to accurately estimate aleatoric semantic uncertainty in LLMs.

Abstract: Large language models (LLMs) can suffer from hallucinations when generating text. These hallucinations impede various applications in society and industry by making LLMs untrustworthy. Current LLMs generate text in an autoregressive fashion by predicting and appending text tokens. When an LLM is uncertain about the semantic meaning of the next tokens to generate, it is likely to start hallucinating. Thus, it has been suggested that predictive uncertainty is one of the main causes of hallucinations. We introduce Semantically Diverse Language Generation (SDLG) to quantify predictive uncertainty in LLMs. SDLG steers the LLM to generate semantically diverse yet likely alternatives for an initially generated text. This approach provides a precise measure of aleatoric semantic uncertainty, detecting whether the initial text is likely to be hallucinated. Experiments on question-answering tasks demonstrate that SDLG consistently outperforms existing methods while being the most computationally efficient, setting a new standard for uncertainty estimation in LLMs.

Published as a conference paper at ICLR 2025
IMPROVING UNCERTAINTY ESTIMATION THROUGH SEMANTICALLY DIVERSE LANGUAGE GENERATION
Lukas Aichberger', Kajetan Schweighofer', Mykyta Iclanskyi', Sepp Hochreiter
1 ELLIS Unit Linz and LIT Al Lab, Institute for Machine Learning,
Johannes Kepler University Linz, Austria
2 NXAI GmbH, Linz, Austria
{ a sicheerge,, schweighofer, ielanskyi, hochreit 8ml. jku. at
ABSTRACT
Large language models (LLMs) can suffer from hallucinations when generating text. These hallucinations impede various applications in society and industry by making LLMs untrustworthy. Current L.MMs generate text in autoregres- sive fashion by predicting and appending text tokens. When an L.M is uncertain about the semantic meaning of the next tokens generate, it is likely to start hallucinating. Thus, it has been suggested that predictive uncertainty is one of the main causes of hallucinations. We introduce Semantically Diverse Language Generation ( (SDLG) to quantify predictive uncertainty in LLMs. SDLG steers the LLM to generate semantically diverse yet likely alternatives for an initially generated text. This approach provides precise measure of aleatoric semantic uncertainty, detecting whether the initial text likely to hallucinated. Exper- iments on question-answerr forms existing methods while being the most computationally efficient, setting a new standard for uncertainty estimation in LLMs.
tasks demonstrate that SDLG consistently outper-
INTRODUCTION
Hallucinations hinder a broad use of LLMs in practical applications and critical decision-making processes as they make them untrustworthy (Manakul al., 2023). Hallucinations are fragments of generated text that, despite appearing cohesive, are not factual. At the time of writing, there is no consensus on the exact nature of all causes of ballucination. We consider generated text to hallu- cinated if it stems from contradictory or non-existent facts in the training data or input prompt. Such hallucinations are conjectured be mainly caused by the predictive uncertainty inherent to proba- bilistic models (Xiao and Wang, 2021). This type of hallucination is also referred to as confabulation (Farquhar et al., 2024), and we shall use the two terms interchangeably. While uncertainty esti- mation for classification tasks has been developed extensively (Hüllermeier and Waegeman, 2021; Gawlikowski et al., 2023), it remains a challenging problem for autoregressive tasks, particularly in natural language generation (NIG)).
Uncertainty estimation in NLG involves assessing the uncertainty of an initially generated text (out- put sequence) for given prompt (input sequence). Current methods typically assess this uncer- tainty by generating multiple output sequences, usually with a single given language model (Xiao and Wang, 2021: Malinin and Gales, 2021; Kuhn al., 2023; Lin al., 2023; Duan et al., 2023; Manakul et al., 2023). Importantly, Kuhn et al. (2023) propose to consider semantic clusters rather than individual output sequences by grouping them according to their semantic equivalence. The se- mantic uncertainty of the initial output sequence should only increase if a language model is likely to generate alternative output sequences that differ in semantics. Hence, semantie uncertainty volves estimating the probability that language model generates output sequence belonging to specific semantic cluster. Empirical results confirm that incorporating semantics into uncertainty es- timation in language models achieves state-of-the-art performance (Kuhn et al., 2023). The current approach utilizes Monte Carlo (MC) approximation via multinomial sampling to generate alterna- tive output sequences that are classified into semantic clusters by natural language inference (NLI) model (Kuhn et al., 2023). Sample sizes typically range from the low double-digit range (Malinin and Gales, 2021; Kuhn al., 2023; Duan al., 2023) to only few hundred for studies conducted

---

Published as a conference paper at ICLR 2025
Input
Semantic difference of alternatives:
unimportant important
Who proposed the theory of relativity?
Initial
Output Sequences
The
theory of
relativity
was
proposed
by
Newton
Alternatives
relativity
theory
introduced
uncertain?
uncertain?
Multinomial Sampling
SDLG
Einstein
uncertain?
uncertain!
Figure 1: Using standard multinomial sampling to generate text does not account for its seman- tics. Thus, it relies on chance to obtain semantically diverse output sequences and is prone miss them. SDLG addresses this by specifically searching for likely, but semantically different output sequences. Thereby, the estimation of semantic uncertainty in language models is improved.
on large-scale compute (Kadavath et al., 2022), autoregressive generations computationally expensive. This is suboptimal, as the current sampling methods are imprecise with a small number of samples (Bishop, 2006) and computationally too expensive ensure high precision with large number samples.
To improve upon the limitations of the current -fate-of-thhearr approach, we introduce Semantically Diverse Language Generation (SDLG), a method that efficiently estimates semantic uncertainty by utilizing importance sampling generate output sequences. We introduce a proposal distribution that samples semantically diverse output sequences (see Fig. 1). SDLG utilizes the NLI model not only for transforming the space of generated output sequences to semantic clusters but also for computing the contribution of every token to the final semantics. Subsequently, the semantically most relevant tokens are substituted to explicitly steer the generation toward semantically diverse yet likely alternative output sequences, while correcting for the increased sampling probability using importance sampling. This can viewed as stress-testing the language model, unveiling output sequences that are valuable summands for estimators of semantic uncertainty. They are generated by the current sampling methods only by chance, making SDLG more systematic and reliable method for capturing semantic uncertainty. Our main contributions are:
We propose a novel method for generating semantically diverse yet likely output sequences. Em- pirical results demonstrate that our method outperforms existing methods for uncertainty estima- tion in NLG, specifically across a variety of free-form question answering tasks.
We establish a theoretical foundation for uncertainty measures in NLG and introduce theoretically grounded estimators for aleatoric semantic uncertainty, also known semantic entropy. Applying these estimators enhances empirical performance of uncertainty estimation language models.
MEASURING PREDICTIVE UNCERTAINTY IN NLG
Uncertainty estimation in classification tasks has already been well-established (Gal and Ghahra- mani, 2016). However, these measures cannot be directly applied uncertainty estimation in NLG. Two key aspects, which differ from estimating uncertainty in classification tasks, have to be consid- ered. First, sequence of autoregressive predictions collectively forms the final output of a model. Second, unlike classification tasks where classes usually are mutually exclusive, different output sequences may equivalent their semantic meaning. To account for these aspects, Kuhn et al. (2023) introduce semantic entropy, yet without proper theoretical grounding. In the following, derive semantic entropy starting from the well-studied uncertainty estimation classification tasks.
Predictive uncertainty in classification. We briefly revisit uncertainty estimation for classifica- tion tasks. In this work, we quantify the predictive uncertainty of a single given model. Given classification model parametrized by and an input vector The predictive distribution under the given model is denoted as p(y x,, w). We assume a fixed dataset D that was sampled according to the predictive distribution p(y x,, w²) under true model parameters
2

---

Published as a conference paper at ICLR 2025
"n. Thus, we assume that the model class can approximate the true distribution sufficiently well, a common and usually necessary assumption (Hüllermeier and Waegeman, 2021). The posterior distribution p(w] D) denotes how likely matches w*. Following Schweighofer et al. (2023a;b), the predictive uncertainty of a single given model parametrized by w is given by
E CEE((y) x. ply x, x))
Hpp(y x w.)w) w)) + Eu [k(pp 2; w) Ipl( x ())
(1)
total
alcatoric
epistemic
where Egg = (())) The total uncertainty, given by the posterior expectation of the cross- entropy CE(::-) -). additively decomposed into aleatoric and epistemic uncertainty. The aleatoric uncertainty is the Shannon entropy H-)) of the predictive distribution under the given model. The epistemic uncertainty is the posterior expectation of the Kullback-Leibbe divergence KL(- (I-) be- tween the given model and possible true models according to their posterior probability.
Predictive uncertainty in NLG. Given are an autoregressive language model parametrized by w, a vocabulary V. and an input sequence of tokens X = (I1, ZM) with the language model a sequence of tokens y = (3/1,...,, MT) E y with y E V. The predictive distribution step t of the output sequence y is conditioned on both the input sequence and all previously generated tokens, denoted as p(ye x, yet, w). The probability of an output sequence is the product of the individual token probabilities: p(y æt, w) = II p(3e Vet, w) (Sutskever et al., 2014). In practice, p(y æ. w) is often length-normalized to not favor short output se- quences (Cover and Thomas, 2006; Malinin and Gales, 2021), which results in p(y x, w)
V. An output of
Ei= log x, yer.,uu w)).
exp
Evaluating the whole set of possible output sequences y is usually intractable, as scales exponen- Seryeeeeee tially with the sequence length T, thus O(V17). Furthermore, as mentioned previously, a language model that likely generates different output sequences from the same input sequence should not nec- essarily indicate high predictive uncertainty if the output sequences mean the same thing. Hence, predictive uncertainty should be considered high only when different output sequences also exhibit semantically diverse meanings (Kuhn et al., 2023). Instead of directly utilizing the distribution over output sequences p(y x, w). the distribution over semantic clusters
p(c x. w) = Σ p(c Eplc| v. p(y x. w) = EEI 1v E x. w) p(y x w)
3
is used to derive the predictive uncertainty in the NLG setting. Here, p(y x. w) expresses the probability of generating an output sequence y and p(c y, w) expresses the probability of y belonging to a certain semantic cluster e E C. Although a specific output sequence might potentially be attributed to more than one semantic cluster, we follow Kuhn et al. (2023); Farquhar et al. (2024) in assuming that each output sequence is attributed to a single semantic cluster. They demonstrate that assigning semantic clusters by predicting the semantic equivalence between generated output sequences with an NLI model empirically performs well. The NLI model takes two sequences as input and predicts whether they entail or contradict each other. Two output sequences are semantic equivalent if they entail each other in both orders and thus belong to the same semantic cluster by definition. Consequently, p(c y, I, w) is equivalent to 1(y E x; w), where Lyy E x, w) 1 iff y belongs to semantic cluster c. For the set of all possible output sequences y, p(c æ, w) expresses the probability of the language model generating an output sequence belonging to specific semantic cluster for given input sequence. Adopting the definition of predictive uncertainty in Eq. (1), the total predictive semantic uncertainty
Ew [CE(p(c x. (.:w))p(e x, ]
H(p(c x, w)) + Ev [p(c x,ww w) pcc
x. w)
(3)
total
aleatoric
opistemic
can again be additively decomposed into aleatoric and epistemic semantic uncertainty. The epistemic semantic uncertainty is again a posterior expectation, which is particularly challenging estimate for current language models with billions of parameters (Zhang et al., 2022; Touvron et al., 2023). The aleatoric semantic uncertainty turns out to be precisely the semantic entropy proposed by Kuhn et al. (2023). Semantic entropy is the entropy of the semantic cluster probability distribution,
H(p(c x, w))
-- log x. w) p(c| x, w)
under a single given language model. To determine a language model is uncertain about its output, it is essential to accurately estimate its semantic entropy, addressed in the following section.
3

---

Published as a conference paper at ICLR 2025
3 ESTIMATING ALEATORIC SEMANTIC UNCERTAINTY IN NLG
We now discuss practical considerations regarding the estimation of the aleatoric semantic uncer- lainty, namely the semantic entropy as given in Eq. (4), First, we find that directly estimating by obtaining samples from semantic clusters is theoretically not justified in the current setting, Instead, the underlying distribution of semantic clusters itself has to be estimated through sampling. Second, to effectively estimate this distribution of semantic clusters, we utilize importance sampling. These two insights contribute to a more accurate uncertainty estimation in NLG.
Monte Carlo (MC) estimation. Kuhn et al. (2023) propose to approximate the semantic entropy by directly using the MC estimator
H(p(c x, w)) R Σ log p(c" I w), c" ~ plc x. a.
(5)
However, we cannot directly sample from p(e x. un) but only from p(y x. w)) Therefore, is impossible to directly use the estimator in Eq. (5). Alternatively, one can first approximate the semantic cluster probability distribution
N Σ Eyeclen w, y" ~ upy a w)
p(c x w)
(6)
Output sequences y" are simply generated via multinomial sampling (Kuhn al., 2023). This estimate of the semantic cluster probability distribution can directly be used to approximate the semantic entropy. Since usually not all clusters e C are found through sampling, the sum taken over observed clusters cI...., CM to which { I==1 were assigned by the NLI model, resulting in
M
H(p(c x, w)) =
logp(cm xx, w) p(cm x, w))
In Sec. 6, we empirically show that this proper estimator for semantic entropy outperforms the im- proper estimator implemented by Kuhn et al. (2023). For further details see Sec. C in the appendix.
Importance sampling. Due to the computational cost of autoregressively sampling from multi- billion-pe parameter language models, the sample size N is usually kept very low in practice. However, this implies that the variance of the MC estimator in Eq. (6) remains high. To lower the variance, a standard technique importance sampling according to a proposal distribution q instead of the target distribution p (Bishop, 2006). Eq. (6) thus changes to
p(c w)
My e x, w 1=1
Σ
N
p(y"
2, w) 2
y" g x,W)) w)
(8)
N
4
The quality of the approximation strongly depends on the choice of the proposal distribution. It should closely approximate the target distribution and have good overlap, i.e. the proposal distribu- tion should have probability mass everywhere the target distribution has substantial probability mass (Bishop, 2006). Thus, a good proposal distribution should cover semantic clusters with substantial probability mass under the target distribution. In the following section, we describe our method to construct an empirical proposal distribution g(y ar, w) with high mass at important semantic clusters by explicitly promoting both the likelihood and diversity of output sequences.
4 SEMANTICALLY DIVERSE LANGUAGE GENERATION
Estimating the semantic entropy according to Eq. (7) requires approximating the semantic clus- ter probability distribution. This probability distribution can either be approximated according to Eq. (6), or via importance sampling according Eq. (8). MC estimation using output sequences from multinomial sampling is straightforward, as one can directly sample from the original distri- bution p(y x, w) without the need for additional weighting or adjustment. However, the resulting estimator has high variance, and sampling only considers the likelihood and not the semantic diver- sity. Furthermore, multinomial sampling may miss likely output sequences that capture important
4

---

Published as a conference paper at ICLR 2025
information about semantic cluster probabilities, especially if those sequences require choosing lower-likelihood token. This limits the accurate estimation of the semantic entropy, as sampling semantically diverse output sequences essentially occurs by chance. Importance sampling, on the other hand, can overcome these limitations by incorporating semantic diversity into the sampling procedure. However, this requires beneficial proposal distribution 7(y x;, w). We propose Semantically Diverse Language Generation (SDLG) to sample according to such an empirical pro- posal distribution. It seeks to efficiently explore semantic clusters, capturing important modes of p(c 2z, w) that might be missed by multinomial sampling.
Semantic diversity and where to find it. Given an input sequence z and an initial output sequence y' generated by a language model, how can we generate another output sequence that has a different semantics from y In natural language, individual words contribute to the semantics of the sentence to varying extents. For instance, consider the sentence "Einstein proposed the theory of relativity" The word "proposed" can be substituted with "introduced" without altering the semantics. However, replacing "Einstein" with "Newton" would change the semantics. Therefore, our method focuses on identifying and substituting the tokens that are most critical to the semantics of y' (see Fig. I). This is achieved by introducing three scores at the token level that quantify each token's 'ien's relevance in altering the semantics:
1. Attribution score:
Each initial token y E y' is scored by its contribution to the semantic meaning of y'. We refer to this initial token's is score as A,
2. Substitution score: Each alternative token By E V is scored by its influence in altering the
semantic meaning when substituting y: with vy. We refer to this alternative token's 's score as Sty.
3. Importance score: Each alternative token Vy E V is scored by the probability the language model assigns to U/ given the context up y/- We refer to this alternative token's score as Tzj-
At a high level, SDIG explicitly substitutes tokens based on these scores. High scores indicate a high potential to give rise to likely output sequence with semantics different from y', as detailed below. Before that, we describe how to calculate the three scores.
Computing the attribution and substitution scores requires loss L,, which expresses to what degree y' is semantically different from itself. It is computed utilizing an NLI model that predicts whether two sequences entail contradict each other. First, the initial output sequence is fed into the NLI model twice, resulting in a high probability of entailment. Second, the loss L is computed for the target prediction contradiction. This loss is used to compute the gradient V L L w.r.f. the token embedding z; that represents the initial token y/. This gradient vector quantifies the required change in z; to achieve a high probability of contradiction, thus altering the semantic meaning of yy'
Attribution score. Our first objective is to identify which initial token yı should be changed accord- ing to the computed gradient vector. An initial token's 's attribution score
(9) is defined the Euclidean distance || l|2 of the gradient vector multiplied elementwise with the embedding vector z; that represents the initial token yı- The higher the attribution score A;, the higher the impact of the token y/ on altering the semantics when being changed (Adebayo et al., 2018). We note that different attribution methods could be utilized, and future work may benefit from exploring these methods to compute A, (Madsen al., 2022).
A, ||| G V Ll2
Substitution score. Identifying which initial token should be changed is crucial but not sufficient on its own. We also have to identify which token to change to, as not every substitution alters the semantic meaning of the initial output sequence. Thus, our second objective is to identify appropriate alternative tokens By that most effectively alter the semantics when substituting an initial token yi- The alternative token's 's substitution score
(z) 21) V L
Sy
(10)
21 - 212 L
is defined as the cosine similarity sim((:-) between the gradient vector and the difference between the initial token's embedding vector and the alternative token's embedding vector zj. Although cosine similarity is a widely used measure for text similarity, we observed that the dot product empirically yields comparable performance. The higher the substitution score Sej. the more closely the change in the embedding vector aligns with the direction of the gradient vector, thus the direction of altering the semanties (Mikolov et al., 2013).
5

---

Published as a conference paper at ICLR 2025
Algorithm SDLG
Algorithm Token Score Ranking
Output: Semantic diverse output sequences S
Output: Token pair indices ranking R Input: see Alg. 1, NLI model e(., +), cross- entropy loss function I(:-), method (ank(-)
Input: Language model g(-), imput sequence æ,
vocabulary V, number output sequences N
1: S 4-
1: R + 0
2: y' + g(x)
L f- ((( y), Ccontradiction)
3: S + ++SUP SUSSUi (y'}
3: for y; E y do
4: R Alg. 2
V L + aL
0y:
5: for n = 2 to N do
5:
A, + || ( 22, T.12
6:
(i,j) ) Rn
6:
for 1 E V do
7:
x" t a + yer U
7:
Sij + sim Zj Vx,L)
8:
you t g(x")
8:
I5 t p(u; yei I. w)
9:
y" y Uy y
9: R+ ((A,, Sig. 15))}
10: S S (1
10: R. + Rank(R)
11:
return 8
11: return R
Illustration of the workflow of SDLG, sampling diverse yet coherent output sequences by identifying and substituting semantically important tokens in the initial output sequence using three distinct context-sensitive scores.
Algorithm:
Importance score. Identifying which alternative token should substitute which initial token is im- portant but not sufficient. Not every alternative token v/y an appropriate substitution for the initial token Vi, given the context. The context is the input sequence x and the output sequence up to the token that is to be substituted y-u. Substituting with a very unlikely token might undermine the co- herence of the overall sequence. Thus, our third objective is to favor alternative tokens that exhibit a high likelihood. The alternative token's importance score
(11) is simply defined as the probability that the language model assigns to By given the context. The higher the importance score Iij. the more suitable the alternative token is for substitution, as it ensures that the resulting output sequence not only is semantically diverse but also remains likely.
Is = p(v; Y/s. x, w)
In summary, the three scores work together to identify the appropriate initial and alternative token pair that should be substituted to alter the semanties of the initial output sequence. The attribution score identifies which initial token substitute, while the substitution score identifies which altema- tive token to replace the initial token with. The importance score identifies which alternative token the language model considers fit for substitution. Each of the scores positively contributes the performance of SDLG, as detailed in Sec. E of the appendix.
Generating semantic diverse output sequences. not corresponding to the beginning of a word is often impractical. Consequently, we exclusively apply substitutions to tokens at the beginning of a word, making them even more efficient. All token pairs (y:, Ut)) are ranked according to the three scores (A,, Siz. Tzj). In its simplest form, the ranking based on equally weighting the three scores, which we found to work well empirically. Based on this token score ranking, a new output sequence is generated by deliberately substituting the highest- ranked token pair. Subsequent tokens are discarded as they are conditioned on the substituted token, which affects their likelihood. The remainder of the new output sequence is then generated by the language model using the usual sampling strategy (see Alg. and 2). SDLG preserves the seman- tically less relevant part of the output sequence, eliminating the need regenerate it and focusing on the part with a high chance of altering the semantic meaning. As a result, the generation pro- cess becomes computationally more efficient than sampling from scratch each time and potentially sampling multiple duplicate output sequences.
has to be noted that substituting initial tokens
Proposal distribution. Finally, we can discuss the exact form of the proposal distribution g(y | x, w) in Eq. (8), which is induced by SDLG. We define the proposal distribution as
q'y T, w) E v x. x.,) w) p(y' x, w.w)).
(12)
V'EY
The probability distribution p(y y' x, w) denotes SDLG transforming a given output sequence y' into another output sequence y with high chance of changing the semantics well. However,

---

Published as a conference paper at ICLR 2025
since y' is not known priori, we sum over all possible y' according to their probability of being sampled p(y' x, w)) Under assumptions about which index is likely to be chosen for a given y', the proposal distribution takes the form
p(y xz, w)
g æ w) P(u: V<t,, 2; w)
(13)
where p(ye Ycc. x. w) is the likelihood of the token that is exchanged by SDLG. Intuitively, it means that we have to adjust the MC estimate in Eq. (6), because SDIG interferes in sampling and changes the token y/ deterministically. For more details the assumptions and a step-by- step derivation, see Sec. B in the appendix. Our experiments show that sampling according to this proposal distribution improves uncertainty estimation in NL.G.
5 RELATED WORK
Uncertainty estimation in NLG. Several works utilized the language model itself to obtain pre- diction of their uncertainty, whether that be numerical or verbal (Mielke et al., 2022: Lin et al., 2022b; Kadavath et al., 2022; Cohen et al., 2023a; Ganguli et al., 2023; Ren et al., 2023; Tian et al., 2023). Cohen et al. (2023b) utilize examinnxamn,,,,,, where one language model generates the output sequence, and the other language model acts an examiner to assess the uncertainty. Zhou et al. (2023) investigate the behavior of language models when expressing their (un)certainty.
A large body of work focuses on sampling a set of output sequences to obtain sampling-based uncer- tainty estimators. Xiao and Wang (2021); Malinin and Gales (2021); Hou et al. (2023) incorporate both aleatoric and epistemic estimates of uncertainty, where epistemic uncertainty due to model se- lection is considered. While Kuhn et al. (2023); Duan et al. (2023); Farquhar et al. (2024) evaluate the aleatoric uncertainty only under single given language model, they take the semantic equiva- lence of potential output sequences into account. Manakul et al. (2023) also sample a set of output sequences but utilize them input to another language model to assess the uncertainty. Another approach to uncertainty estimation in NLG is conformal prediction (Quach et al., 2023), where a stopping rule for generating output sequences calibrated. Additionally, Xiao al. (2022) empir- ically analyze how factors such as model architecture and training details influence the uncertainty estimates in language models.
Complementary methods. Related work on uncertainty estimation in NLG aims to propose im- proved uncertainty measures compared to semantic entropy and could benefit from more diverse samples. For instance, Lin et al. (2023) propose leveraging the similarity between output sequences, while Chen et al. (2024) introduce FigenScore, which utilizes embeddings of output sequences. These approaches could integrate SDLG to generate the output sequences needed to compute their uncertainty estimators, making them complementary to SDLG.
Generating diverse output sequences. Li et al. (2016) propose an alternative training procedure of language models to avoid generie, input- input-independent output sequences and increase diversity. Di- verse beam search (Vijayakumar et al., 2018) optimizes for a diversity-augment beam groups, based on diversity heuristics. Ippolito et al. (2019) compare diversity-encourage decoding strategies. Nucleus sampling (Holtzman et al., 2020) generates higher quality as well as more diverse output sequences but does not explicitly encourage semantic diversity. Contrastive decoding (Li et al., 2023) utilizes a second, weaker language model, where the decoding algorithm favors tokens generated by the stronger model and penalizes tokens generated by the weaker model. Tam (2020) utilize semantic clustering during beam search, which is used prune beams and diver- sify the remaining candidates. However, this only indirectly steers towards more diversity and relies on the diversity of the initial beams.
objective across
Closely related, but not directly targeting the semantic diversity of output sequences, is the field of (neural) controllable text generation (Prabhumoye et al., 2020). Here, the generation process of the language model is steered by another language model to e.g. adhere to a certain dialog structure, prevent toxic answers, or play a certain persona. Keskar et al. (2019) use control codes added to the prompt to steer the generation. Dathathri et al. (2020) propose the use of an external supervised classifier to control the generation. Chan et al. (2021) also utilize an external classifier but train in a self-super setting. (Ghazvininejad et al., 2017; Holtzman et al., 2018) re- re-weight the probability distributions at each step of generating the output sequence. For further work in this field, see the surveys by Prabhumoye et al. (2020); Zhang al. (2023).
7

---

Published as a conference paper at ICLR 2025
Table 1: AUROC using different uncertainty measures score to distinguish between correct and incorrect answers. SEMS uses the improper semantic entropy estimator implemented by Kuhn al. (2023) while SE use the proper semantic entropy estimator we introduced in Sec. 3, with different sampling strategies. The threshold of the correctness metric Rouge-L L (FI score) is set to 0.5. Each method uses ten output sequences for assigning an uncertainty estimate.
Dataset
# Param.
LN-PE
PE
SAR
SEMS
SE
(F(*)
SE DBS
SE
MS
2.7b
.439
.177
.611
.405
.846
.686
.920
6.7b
.466
.510
.555
.512
.781
.637
.881
TruthfulQA
13b
.676
.712
.775
.453
.896
.819
.956
30b
.482
.542
.517
.438
.864
.788
.927
2.7b
.717
.693
.733
.717
.744
.697
.744
6.7b
.728
.703
.748
.739
.764
.714
.759
CoQA
13b
.723
.697
.747
.743
.758
.720
.760
30b
.732
.698
.742
.745
.767
.713
.768
2.7b
.769
.787
.785
.781
.004
.088
.809
6.7b
.790
.055
.804
.803
.822
.823
.829
TriviaQA
13b
.807
.820
.819
.824
.838
.841
.845
30b
.799
.812
.815
.817
.831
.837
.840
6 EXPERIMENTS
Data and models. To ensure a fair comparison of different uncertainty estimation methods within the scope our computational budget, we decided to align the evaluation tasks with current work by focusing on free-form question answering. Thus, we performed experiments three datasets that cover a broad range question answering settings. To be concrete, we use the over 800 closed-book questions in TruthfulQA (Lin et al., 2022a) corresponding to whole sentence answers, the almost 8,000 open-book questions in the development split of CoQA (Reddy et al., 2019) corresponding to medium to shorter length answers, and about 8,000 closed-book questions in the training split of TriviaQA (Joshi et al., 2017) corresponding to short, precise answers. We use a 5-shot, zero- shot, and 10-shot prompt for TruthfulQA, CoQA, and TriviaQA, respectively. These datasets frequently used as benchmarks for uncertainty estimation in NLG due to their strong correlation with human evaluations and the effective performance of "off-the-shelf" language models compared to tasks requiring fine-tuning (Kuhn et al., 2023; Goyal et al., 2023). Each of the three datasets was evaluated with the OPT model family (Zhang et al., 2022), with model sizes ranging from 2.7 to 30 billion parameters. Related work suggests that performance trends generalize across transformer- based model families (Duan al., 2023; Manakul et al., 2023). In general, the four language models and three datasets assess the performance of uncertainty estimation methods in NLG across varying model sizes, output sequence lengths, and both open-book and closed-book settings.
Evaluation. Effective uncertainty measures should accurately reflect the reliability of answers gen- crated by the language model. Higher uncertainty more likely leads to incorrect generations. Thus, to evaluate the performance of an uncertainty estimator, assess how well it correlates with the correctness of the language model's answers; correct answers should be assigned lower uncertainty estimator than incorrect answers. To determine whether answer is correct, it has to be compared to the respective ground truth answer. First, we utilize the statistics- Rouge-1 (Lin, 2004), which take the initial output sequence and the ground truth answer measure the longest common subsequence and the overlap of unigrams, respectively. Second, we utilize the learning-base metrie BLEURT (Sellam al., 2020) that uses learned evaluation model assess how well the initial output sequence conveys the meaning of the ground truth answer. It has to be noted that for each of these three metrics, correctness computed as the difference between the maximum score assigned to true reference answer and the maximum score assigned to false reference answer. While TruthfulQA provides both true and false reference answers, CoQA and TriviaQA only provide true reference answers. The resulting scalar value reflects the correctness of the answer and subsequently thresholded to yield binary decision. To ensure a robust evaluation, comprehensive
metrics Rouge-L and
of ten correctness thresholds ranging from 0.1 1 1.0 (exact match). To measure the correlation between incorrectness of answers and the respective uncertainty estimates, utilize AUROC, with higher values indicating better performance of the uncertainty estimator, as
8

---

Published as a conference paper at ICLR 2025
3.00
3.34
MM
(91330 24. 14-0
(9T-1h 20.5
1.70
(FT33)
p-a.))
170
197 18. P-1
OPENS-
3.70
1700
1.4
0F74-78 5016
0070.7 5016
3.00
051
0972.18 p:t
0874.3 Ms 30035
0P12 To: p-0.20
M50-10
097211
0F76.7 M-3-1.1)
OP73 L-1
pirenatons
Jeneratonn
(a)
(b)
Figure 3: (a) AUROC using uncertainty measures across various numbers of samples as score to distinguish between correct and incorrect answers of the CoQA dataset. Solid and dotted lines indi- cate the performance when using the proper and improper semantic entropy estimator, respectively. (b) Average number of semantic clusters found across various numbers of samples considered.
it reflects stronger alignment between the correctness of the language model's answers and their respective uncertainty estimates. Overall, this evaluation process follows established methodologies for assessing the performance of uncertainty measures in NLG (Kuhn et al., 2023; Lin et al., 2023; Duan et al., 2023;
Bakman et al., 2024; Farquhar et al., 2024; Nikitin et al., 2024).
Baselines. We compare our method against methods that directly utilize the predictive entropy of the output distribution on a token level. These are Predictive Entropy (PE), Length-Normalicn Predictive Entropy (LN-PE) (Malinin and Gales, 2021), and Shifting Attention to Relevance (SAR) (Duan et al., 2023). We also compare against methods that utilize semantic entropy on a sequence level. Thereby, output sequences are generated with multinomial sampling (SEar)) (Kuhn et al., 2023) with diverse beam search (SEpes) (Vijayakumar et al., 2018). Although DBS has not plicitly been proposed for uncertainty estimation in NLG, we added it as a more traditional sampling method that enforces diversity among output sequences.
Our method (SDLG). Unlike current methods that rely on finding the optimal sampling temperature or penalty term, SDLG does not require hyperparameter tuning as it controls the sampling. One has only to decide on the ranking method for the three individual token scores. We empirically found that the performance of method is quite robust with respect to the weighting of the token scores. Therefore, throughout the experiments, we derive the final token score ranking by straightforwardly averaging the three individual token scores. To compute the token scores for semantic diversity as discussed in Sec. 4, we utilize the same NLI model DeBERTa (Williams et al., 2018; He et al., 2021) that is also used for predicting semantic equivalences and determining semantic clusters.
Analysis of results. Tab. 1 summarizes the main results of our experiments. It can be observed that SDLG largely outperforms all baselines throughout datasets and model sizes. Tab. together with Fig. 6 8 in the appendix show that the performance improvements of our method persist across the three correctness metrics Rouge-I. (F1 score), Rouge-1 (F1 score), and BLEURT, as well as different correctness thresholds and the number of samples considered. The results show that using the proper semantic entropy estimator for the semantic entropy (SE(?s) outperforms the improper estimator (SEars) when generating output sequences via MS, as outlined in Sec. 3.
can be observed that simple token-level diversity enforced by higher temperatures in MS (Kuhn et al., 2023) or by DBS (Vijayakumar et al., 2018) 8) is insufficient for capturing semantic diversity essential for uncertainty estimation in NLG. However, since TriviaQA includes the shortest out- put sequences among the datasets, the advantages of our method are less pronounced. Although our method still outperforms the baseline methods on short output sequences, the true strengths of SDLG emerge in scenarios with longer output sequences, such those in TruthfulQA, where simple token-level diversity is insufficient, and the ability of our method to explore semantic diver- sity sequence-level becomes critical. The smallest performance gap between SE with MS and

---

Published as a conference paper at ICLR 2025
SXC DedETs-in ure 077..2 OFTU22
0000 DAECCT-TTTTR -0TTT-----
De6-0t0--rrrs OPF227
SAN Cad OPT2.11
10% bot
0,852 tirehoo
Figure 4: Average number of Teraflops required for an increasing number of samples generated with SDLG vs. standard multinomial sampling (MS) and Shifting Attention to Relevance (SAR).
SDLG occurs on the CoQA dataset. Therefore, we more closely examine the relationship between the performance and the sample size. We report MS with the optimal temperature (t 000=0.. 0.55 as well as at nearby temperatures for comparison. As shown in Fig. 3a, while the performance difference is smaller when considering all ten generations. SDLG demonstrates strong performance even after sampling just one additional output sequence. This highlights the sample efficiency of our method.
Semantic clusters. Fig. 3b shows that our method results in a 19% increase of semantic clusters after generating the second output sequence, as well as a 74% increase of semantic clusters after the tenth output sequence, compared to multinomial sampling with the highest perfferiperforminggg temperature used by the current state-of- of-theartt method (Kuhn et al., 2023), when averaging across all CoQA instances. This is because SDLG explicitly searches for output sequences with different semantic meanings and practically does not sample the same output sequence twice.
Computational expenses. While it is true that generating additional output sequences with SDLG requires extra computation for the NLI model to obtain the token score ranking, the primary com- putational effort lies in the generation of output sequences. The computational expenses of a single forward and backward pass through the NLI model with few hundred million parameters is minor compared generating only single token using a language model with billions of parameters. Since our method deterministically changes specific token within the initial output sequence, pre- ceding tokens do not have to be regenerated again, but only subsequent ones. This results in SDLG requiring at least an average of 15% (TruthfulQA), 33% (CoQA), and 21% (TriviaQA) fewer flops compared to multinomial sampling that is used by the current state-of-the-ar method (Kuhn et al., 2023). In general, the advantage of our method over the current methods further increases with longer output sequences and larger language model sizes, as illustrated in Fig. 4.
7 CONCLUSION
We introduce SDLG to improve uncertainty estimation in NLG. SDLG substitutes tokens in the initial output sequence that likely to lead change in semantic meaning. Unlike standard multinomial sampling, this targeted approach effectively samples likely output sequences with dif- ferent semantic meanings, capturing important information for estimating semantic uncertainty. Our experiments on free-form question answering datasets demonstrate that SDLG not only increases the overall quality of the uncertainty estimator but is also computationally more efficient.
Future work should address the assumption that each sentence can be attributed to a single semantic cluster, which may overly restrictive. This represents an orthogonal issue that does not conflict with our method but can be addressed independently. Also, this work focuses on estimating the aleatoric semantic uncertainty. Future work should investigate how effectively assess epistemic semantic uncertainty, which is a challenging task in itself (see Eq. (3)).
Despite these remaining challenges, SDLG already offers notable advantages over prior methods. First, SDLG controls the sampling instead of depending chance obtain diverse samples. It eliminates the necessity of searching for an optimal sampling temperature, an important hyperpa- rameter for all prior methods. Second, the advantageous sampling reduces the required number of samples for high-quality uncertainty estimation, while also being computationally most efficient per sample. Overall, SDLG considerably enhances the applicability of uncertainty estimation in NLG.
10

---

Published as a conference paper at ICLR 2025
ETHICS AND REPRODUCIBILITY STATEMENT
We recognize the potential for language models to generate biased or harmful content if not properly managed. While SDLG aims to improve model reliability, we strongly advocate for its responsible alongside bias mitigation strategies and content moderation techniques ensure ethical and safe deployment.
To promote reproducibility, thcoretical justifications provided in Sec. 2, Sec. 3, and Sec. 4, with further mathematical derivations and proofs in the appendix. All datasets used are publicly available, and we utilize standard benchmarks facilitate easy replication of our work. Upon publication, the source code for reproducing all experiments will be made publicly accessible.
ACKNOWLEDGEMENTS
The ELLIS Unit Linz, the LIT AI Lab, the Institute for Machine Learning, are supported by the Federal State Upper Austria. This research (FWF) [10.55776/CO112) We thank the projects INCONTROL-RL. (FFG-881064), PRIMAL (FFG-873979), S3AI (FFG-872172). DL for GranularFlow (FFG-871302), EPILEPSIA (FFG- 892171), FWF AIRI FG 9-N (10.55776/FG9), ALAGrrenHeatingGiids (FFG- 899943), INTE- GRATE (FFG-892418), ELISE (H2020-ICT-2019-3 ID: 951847), Stars4Waters (HORIZON-CL6- 2021-CLIMATT-011001) We thank NXAI GmbH, Audi.JKU Deep Learning Center, TGW LO- GISTICS GROUP GMBH. Silicon Austria Labs (SAL), FILL Gesellschaft mbH, Anyline GmbH, Google, ZF Friedrichshafen AG, Robert Bosch GmbH, UCB Biopharma SRL, Merek Healthcare KGaA, Verbund AG, GLS (Univ. Waterloo), Software Competence Center Hagenberg GmbH, Bo- realis AG, TÜV Austria, Frauscher Sensonic, TRUMPF and the NVIDIA Corporation. Kajetan Schweighofer acknowledges travel support from ELISE (GA no 951847).
funded in part by the Austrian Science Fund
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems. Curran Associates, Inc., 2018.
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. Learning principled bilingual mappings of word embeddings while preserving monolingual invariance. Jian Su, Kevin Duh, and Xavier Car- reras, editors, Proceedings of the 2016 Conference Empirical Methods in Natural Language Processing, pages 2289 2294, Austin, Texas, November 2016. Association for Computational Linguistics.
Yavuz Faruk Bakman, Duygu Nur Yaldiz, Baturalp Buyukates, Chenyang Tao, Dimitrios Dimitri- adis, and Salman Avestimehr. Mars: Meaning-aware response scoring for uncertainty estimation in generative 1lms. arXiv, 2402. 2024.
Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statis- tics). Springer- Verla,, Berlin, Heidelberg, 2006. ISBN 0387310738.
Alvin Chan, Yew-Soon Ong, Bill Pung, Aston Zhang, and Jie Fu. Cocon: A self-supervised ap- proach for controlled text generation. International Conference on Learning Representations, 2021.
Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Llms' internal states retain the power of hallucination detection, 2024.
Roi Cohen, Mor Geva, Jonathan Berant, and Amir Globerson. Crawling the internal knowledge- base of language models. In Andreas Vlachos and Isabelle Augenstein, editors, Findings of the Association for Computational Linguistics: EACL 2023, pages 1856-1869, Dubrovnik, Croatia, May 2023a. Association for Computational Linguistics.

---

Published as a conference paper at ICLR 2025
Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. LM vs LM: Detecting factual errors via cross examination. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pages Singapore, December 2023b. Association for Computational Linguistics.
Thomas M. Cover and Joy A. Thomas, Elements of Information Theory (Wiley Series in Telecom- USA, 2006. ISBN 0471241954.
munications and Signal Processing).
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosin- ski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations, 2020.
Jinhao Duan, Hao Cheng, Shiqi Wang. Chenan Wang, Alex Zavalny, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. Shifting attention to relevance: Towards the uncertainty estimation of large lan- guage models. arXiv, 2307-01379, 2023.
Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630((011))625--63)) 2024.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty deep learning. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Pro- ceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1050-1059 1050, New York, New York, USA, 20-22 Jun 2016.
PMLR.
Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamilè Lukošiūté, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, Karina Nguyen, Liane Lovitt, Michael Sellitto, Nelson Elhage, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robert Lasenby, Robin Larson, Sam Ringer, Sandipan Kundu, Saurav Kadavath, Scott Johnston, Shauna Kravec, Sheer EI Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Christopher Olah, Jack Clark, Samuel R. Bowman, and Jared Kaplan. The capacity for moral self-correction in large language models. arXiv, 2302.07459, 2023.
Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, Muham- mad Shahzad, Wen Yang. Richard Bamler, and Xiao Xiang Zhu A survey of uncertainty in deep neural networks. Arlificial Intelligence Review, (6(1):1513-158) Oct 2023. ISSN 1573-7462.
Marjan Ghazvininejad, Xing Shi, Jay Priyadarshi, and Kevin Knight. Hafez: an interactive poetry generation system. In Mohit Bansal and Heng Ji, editors, Proceedings of ACL 2017, System Demonstrations, pages 43 48, Vancouver, Canada, July 2017. Association for Computational Linguisties.
Tanya Goyal, Junyi Jessy Li, and Greg Durrett. News summarization and evaluation in the era of gpt-3. arXiv, 2209.12356 2023.
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. arXiv, 2006.03654, 2021.
Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. Learn- ing to write with cooperative discriminators. In Iryna Gurevych and Yusuke Miyao, editors, Pro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pages 1638-1649, Melbourne, Australia, July 2018. Association for Computational Linguistics.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020.
Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, and Yang Zhang. Decomposing uncertainty for large language models through input clarification ensembling. arXiv, 2311.08718, 2023.
12

---

Published as a conference paper at ICLR 2025
Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning. 110:457-506, 2021.
Daphne Ippolito, Reno Kriz, João Sedoe, Maria Kustikova, and Chris Callison- Compari- son of diverse decoding methods from conditional language models. In Anna Korhonen, David Traum, and Lluís Mårquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3752-3762, Florence, Italy, July 2019, Association for Com- putational Linguistics.
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics, Vancouver, Canada, July 2017. Association for Computational Linguisties.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodee Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bow- man, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Ka- plan. Language models (mostly) know what they know. arXiv,
2022.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. CTRL: conditional transformer language model for controllable generation. arXiv, 1909.05858, 2019.
Lorenz Kuhn, Yarin Gal, and Sebastian Farguhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation natural language generation. In The Eleventh International Conference Learning Representations, 2023.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting objective function for neural conversation models. In Kevin Knight, Ani Nenkova, and Owen Rambow, editors, Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119, San Diego, California, June 2016. Association for Computational Linguistics.
Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang. Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimiza- tion. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 6 Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12286-12312, Toronto, Canada, July 2023. Association for Computational Linguistics.
Chin-Yev Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summariza- tion Branches Out, pages 81, Barcelona, Spain, July 2004. Association for Computational Linguistics.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv, 2109,07958, 2022a.
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. Transactions Machine Learning Research, 2022b. ISSN 2835-8856.
Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantifi- cation for black-box large language models. arXiv,, 2305.19187. 2023.
Andreas Madsen, Siva Reddy, and Sarath Chandar. Post-hoe interpretability for neural nlp: survey. ACM Computing Surveys, 1-48) 12 2022. ISSN 1557-7341.
Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction. In
International Conference Learning Representations, 2021.
13

---

Published as a conference paper at ICLR 2025
Potsawee Manakul, Adian Liusie, and Mark Gales. SelfCheckGPT:
black-box hal- lucination detection for generative large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pages 9004-9017, Singapore, December 2023. Association for Computational Linguisties.
Sabrina Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. Reducing conversational agents' overconfidence through linguistic calibration. Transactions of the Association for Com- putational Linguistics, 10:857-872, 2022.
Tomas Mikolov, Kai Chen, G.s Corrado, and Jeffrey Dean. Efficient estimation of word representa- tions in vector space. Proceedings of Workshop at ICLR, 2013, 01 2013.
Alexander Nikitin, Jannik Kossen, Yarin Gal, and Pekka Marttinen. Kernel language entropy: Fine- grained uncertainty quantification for Ilms from semantic similarities. 2024.
Shrimai Prabhumoye, Alan W Black, and Ruslan Salakhutdinov. Exploring controllable text genera- tion techniques. In Donia Scott, Nuria Bel, and Chengqing Zong. editors, Proceedings of the 28th International Conference Compulational Linguistics, pages 1-14, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics.
Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S. Jaakkola, and Regina Barzilay. Conformal language modeling. arXiv, 10193, 2023.
Siva Reddy, Danqi Chen, and Christopher Manning. Coqa: A conversational question answering challenge. arXiv, 1808.07042, 2019.
Jie Ren, Yao Zhao, Tu Vu, Peter J. Liu, and Balaji Lakshminarayanaa. Self-evaluation improves selective generation in large language models. arXiv, 2312.09300, 2023.
Kajetan Schweighofer, Lukas Aichberger, Mykyta lelanskyi, and Sepp Hochreiter. Introducing an improved information-theerrtiic measure of predictive uncertainty. arXiv, 231 1301398 2023a.
Kajetan Schweighofer, Lukas Aichberger, Mykyta lelanskyi, Günter Klambauer, and Sepp Hochre- iter. Quantification of uncertainty with adversarial models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 19446-19484. Curran Associates, Inc., 2023b.
Thibault Sellam, Dipanjan Das, and Ankur P. Parikh. generation. arXiv, 2004.04696, 2020.
Bleurt: Learning robust metrics for text
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. arXiv, 1409.3215, 2014.
Yik-Cheung Tam. Cluster-based beam search for wointer-generatee chatbot grounded by knowledge. Computer Speech & Language, 64:101094, 2020. ISSN 0885-2308.
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. Just ask for calibration: Strategies for eliciting calibrated confi- dence scores from language models fine-tuned with human feedback. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1433-5442, Singapore, December 2023. Association for Computational Linguistics.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao. Vedanuj Goswami. Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet. Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nic, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh
14

---

Published as a conference paper at ICLR 2025
Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. arXiv, 2307.09288 2023.
Ashwin K Vijayakumar Michael Cogswell, Ramprasath R. Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. Diverse beam search: Decoding diverse solutions from neural quence models. arXiv. 1610.02424, 2018.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sen- tence understanding through inference. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, Volume (Long Papers). pages 1112-1122, New Orleans, Louisiana, June 2018. Association for Computational Linguisties.
Yijun Xiao and William Yang Wang- On hallucination and predictive uncertainty in conditional language generation. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty, editors, Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2734-2744, Online, April 2021. Association for Computational Linguistics.
Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and Louis- Philippe Morency. Uncertainty quantification with pre-trained language models: A large-scale empirical analysis. In Yoav Goldberg. Zornitsa Kozareva, and Yue Zhang, editors, Findings of the Association for Computational Linguistics: EMNIPP 2022, pages 7273-7284, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. A survey of controllable text generation using transformer-base pre-trained language models. ACM Comput. Surv, 563),, oct 2023. ISSN 0360-0300.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained transformer language models. arXiv, 2205.01068, 2022.
Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto. Navigating the grey area: How expressions of uncertainty and overconfidence affect language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pages 5506-5524, Singapore, December 2023, Association for Computational Linguistics.
15

---

Published as a conference paper at ICLR 2025
A BROADER IMPACT
This work focuses on assessing the uncertainty in natural language generation (NLG) using language models. Our primary goal is to increase the robustness of language models, assess the reliability of their predicted output sequences, and detect when a language model is hallucinating. Therefore, we contend that our work makes positive contribution society in several aspects:
1. Improved discernment of certainty in model predictions enhances practical application in real- world scenarios. This can be implemented by signaling uncertainty users, such as through highlighting dubious sections of responses or opting not to display uncertain outputs altogether.
2. Reliable uncertainty estimates may increase the trust of the user in the language model, as it provides a basis to gauge the quality of the answer.
However, while we expect mainly positive impact on society, there are also potential negative aspects:
1. Enhanced uncertainty estimation might not yield expected outcomes if users lack the necessary training interpret these estimates effectively.
2. While better uncertainty assessment can foster usability and user trust, it also carries the risk of creating undue reliance these models. It is crucial maintain human oversight and critical evaluation of language model outputs, as over-reliance can be detrimental.
It is important to note that our method evaluates uncertainty based on the information available to the language model. Therefore, it may inaccurately deem a factually incorrect answer as certain if the model is consistently trained on such erroneous facts. This issue, often perceived as model "hal- lucination", is not reflection of the model's ' uncertainty, but rather result of factual inaccuracies in the underlying data that is additional to the hallucinations stemming from uncertainty.
B ON THE PROPOSAL DISTRIBUTION INDUCED BY SDLG
In the following, we analyze the proposal distribution induced by SDLG. We consider a probabilistic transformation of one output sequence y' into another output sequence y. given by ply y', æ w²) This is introduced because we have sum over all possible output sequences y' that could apply SDLG on, leading to
( W w)
Σ ply' az w) p(y æ', x.
(14)
y'EY
We can write p(U v', x w) as an expected value over f, the index where SDLG chooses a different token:
T
plu V', w)
Σ t=1
p(t y', x. w) plu t, y w))
(15)
The construction of y from y' only changes one element of y' at position t and then generates the new postlix. Therefore, we have p(y t, 3 x, w) for y' * you. Consequently, Ey'ey P(y' æ, w)) can be reduced to EvseEv>. P(y's. ygt. x, w) p(y æ, w) if the factor p(y 1 4.y 0,, w) present. There is only one possibility for the prefix with y yeb.
Using Eq. (15) in Eq. (14) leads to q(y x. w)
(16)
T Epll y a,, w) plu 1. x,, a,w)) w)
= Σ plu x. w)
veyy
t=1
T
Σ
Σ ply' xx w) p(1 1/ x. w) p(y 1, 1, x w)
[=1 y'E)
Σ E. E.g.
Σ
t, v x,, w) plyb. V's.s x, w) p(u' x, w) p(t v/ w) p(y
16

---

Published as a conference paper at ICLR 2025
T
Σ
Σ
plySe
Uga x. w) p(y w) p(t Vso Ugt, 2; w) plys B Ueee 2, w)
[=1 V.. Ess
Σ
a w) p(t v Ue a, 2, w) p(use yo az w) ply's Ver, at w) p(y
plyer æ, w)
Σ
plysel
Vgr, x w) plt A Meg æ, w) plyse yege 2,, w)
Σ plyst æ, w) p(t yee. 2, w) ply> Vgt. w.
Σ plyer æ, w) ply year æ w) p(t ye yees æ, w) plyst Mt, yeer x, w)
Σ
p(t yr, Vet, a, w) plyee æ, w) p(y> Mes 22,
where we used p(y V,,, w) 1, since SDLG chooses Yr deterministically given Y<te = y'r We assume that all probability mass in p(t Vt, V<t, æ, w) is at the actually observed t. This means, given all possible y t the most probable position to induce a semantic change. This is a strong assumption that needs further investigation in future work. Under this assumption, the final result in Eq. (16) reduces to
(y xr, w) p(yer æ. w) p(y>< Vt M<t 2. w)
(17)
We can re-write Eq. (17) in terms of the output sequence probability distribution p(y æ, w) as
ply 2 w))
q x, wr
(18)
p(ye Bot, x. w)
C ESTIMATING THE SEMANTIC ENTROPY
In the following, we provide further details about estimating the aleatoric semantic uncertainty, namely the semantic entropy.
C. SEMANTIC ENTROPY ESTIMATOR (KUHN ET AL. 2023)
As already established in the main paper, directly using the estimator for semantic entropy in Fq. (5) not possible because the distribution p(c æz, w) is not known. Inspecting the implementation of Kuhn et al. (2023) reveals that their estimator of the semantic entropy is using the estimate of the semantic cluster probability distribution
p(c x w)
Σ 1yyec x, w) ply xx w,,, 3" ~ p(y x., w)
(19)
They then approximate the semantic entropy as
M Σ m=1
H(p(c æ, w))
log p(cm x, w)
(20)
M
This assumes that Cm are sampled according to p(c) x,, w). but they are not!
Eq. (20) would be correct estimator of the semantic entropy if the semantic clusters were sampled according to p(cm x, w). However, this distribution cannot be sampled directly and Kuhn et al. (2023) utilize it to compute the semantic entropy from the class estimates Eq. (19) instead of using as an estimator. In this scenario, E7. must be used instead. Note that it is necessary to normalize p(c a., w) because the estimate is generally not normalized probability distribution.
17

---

Published as a conference paper at ICLR 2025
Dittr but mnx ande
AC 11
F1y:
picl
pict
-04
-64
0.0
Number
(a) p(y) and p(c)
(b) MC approximation using Eq. (6) vs. Eq. (21)
Figure 5: Synthetic example approximating cluster distribution p(c) of an underlying probability distribution ply). In (a) the distributions are shown. In (b), the bias and variance over 200 runs of the MC approximations per number samples using Eq. (6) (blue) and Eq. (21) (orange) are given.
C.2 DETAILS ON OUR SEMANTIC ENTROPY ESTIMATOR
The estimator of the semantic cluster probability distribution of Kuhn et al. (2023) given by Eq. (19) can be interpreted as Eq. (6) with importance sampling. Formally, they utilize an empirical proposal distribution q(y/ I, w) = N Em=1 1yy = y"). defined by the set of previously sampled output sequences {y") The approximation of the semantic cluster probability distribution given by Eq. (6) thus changes
N Σ
p(y"
2, w) x., w
plc x. w)
My E a w)
y² ~ 1(y xr w)
(21)
Hy
As this distribution known by design and can be enumerated, Eq. (21) simplifies to a weighted sum. The quality of this approximator strongly depends on the empirical distribution. Therefore, Eq. (21) should only be used in favor of Eq. (6) if [y")= containts output sequences that have very high probability under p(y x, w). The more these distributions differ, the higher the variance of the estimator, and therefore, the lower the approximation quality. We utilized Eq, (21) instead of Eq. (6) for the baseline using multinomial sampling and in addition to the importance sampling perform with SDLG (c.f. (8)).
To illustrate the validity of using the estimator in Eq. (21), consider the following example: Given are a probability distribution pty = (0.15, 0.1, 0.3, 0.45). Furthermore, the cluster probability distribution p(c) = (0.25, 0.75) is derived from this distribution, thus you 31 Co and y/2 Ms C1. The distributions are shown in Fig. 5a. In Fig. 5b, we compare the MC approximation using Eq. (6) and Eq. (21). The results show that the estimator using Eq. (21) is prone to be more biased for low number of samples but decreases its variance much faster.
Furthermore, found that the logarithm of the unnormalized probability estimator together with normalizing the probability estimator outside the logarithm in Eq. (7) improves empirical results for all methods that estimate the semantic entropy.
18

---

Published as a conference paper at ICLR 2025
D FURTHER EXPERIMENTS AND DETAILS
The code and data are available at https: https:://gtthbbb com/mi jkk/skkkaaaa SDIG.
Hyperparameters. For baseline methods, we performed an extensive hyperparameter search for each dataset with the SEass temperature E {0.25, 0.5, 1.0, 1.5, 2.0} and the SEpas penalty term E {0.2, 0.5, 1.0). Also, each method uses 10 generations to assign an uncertainty estimator, as prior work suggests that sample sizes above 10 do not significantly improve the performance of the uncertainty measures (Kuhn et al., 2023; Duan et al., 2023).
Results. A comprehensive overview of all conducted experiments is given in Tab. 2. The results of our reimplementation of methods PE, LN-PE, and SAR match closely with the results reported in prior work (Kuhn et al., 2023; Duan al., 2023). Furthermore, we did an in-depth comparison of the two best semantic entropy estimators SEX's and SE for all three considered correctness metries using extensive range of correctness thresholds, well over the number of sampled output sequences. Results on the three considered datasets for all models are depicted in Fig. 6 8.
Computational expenses. The computational expenses of SDLG and previous methods shown in Fig. 4 across a number of samples generated by a 2.7 billion parameter language model. It can be observed that the advantage of our method in terms of computational efficiency further increases with an increasing number of samples and longer input and output sequences. To further reduce the computational expenses of our method, we decrease the number of token scores that have to be computed by implementing a token probability threshold of 0.001, under the rationale that tokens falling below this probability threshold would, in any case, be assigned very low importance score.
Implementational details of SDLG. In general, the NLI model might not build upon the same token embedding or even the same vocabulary V as the language model. Consequently, the embedding vectors need to be differentiably transformed to enable the computation of the gradients with respect to the token embeddings. Fortunately, there exist efficient exact methods to learn the optimal linear transformation between the two monolingual embedding spaces (Artetxe et al., 2016). However, for our considered NLI and LLM models (He et al., 2021; Zhang et al., 2022), tokenizers had the same vocabulary.
E INSIGHTS INTO SDLG
Illustrative example. Fig. 10 considers the input sequence "Who proposed the theory of relativity?" with the given output sequence "Albert Einstein did". When investigating the alternative tokens becomes clear that not every substitution leads to a change in semantic meaning. It is important to the
substitute tokens that also receive a high score for altering the semantics. In this example, initial token corresponding to "Einstein" and the alternative token corresponding to "Schweitzer". Yet, this alone does not directly indicate a high level of uncertainty about the output sequence. High uncertainty should be attributed only if the new output sequence is completed and still has a different semantic meaning. If the language model is uncertain about the originator of the theory of relativity, it completes the new output sequence like "Albert Schweitzer proposed the theory of relativity". This would suggest a high uncertainty estimate. However, if the language model is confident about the originator of the theory of relativity, it completes the new output sequence like "Albert Schweitzer didn't, I, but Albert Einstein did". It is in favor of a low uncertainty estimate since the model reinforces the original semantics. This illustrates that solely considering the predictive uncertainty on token level is insufficient. Steering the generation towards different semantics and then continuing the usual generation can be viewed as testing the language model.
Computational flow. Fig. 11 shows the computational flow of how attribution, substitution, and im- portance scores are computed for one specific token pair, a present token (black) as the fourth token of the output sequence and potential alternative token (turquoise). Initially, the language model takes in the input sequence embeddings (purple) and generates the output sequence by selecting next tokens based on their token probabilities (grey). The token embeddings of the selected tokens (blue) are matched with the embedding space of the NLI model through a Bilingual Mapping, which has to be applied to align the token embeddings of the language model and the NLI model in case the tok- enizers are not identical (see Sec. D). The NLI model then predicts that the output sequence entails itself. Given this prediction, loss is computed the target "contradiction" that the gradients of
19

---

Published as a conference paper at ICLR 2025
this loss with respect to the token embeddings indicate which part of the output sequence to change in order to get a contradiction, namely a semantically different output sequence. The attribution and substitution scores are based on these gradient vectors (orange), while the importance score is based on the likelihood of the alternative token (turquoise).
Examples of generated output sequences. Tab. 3 visualizes five randomly selected TruthfulQA questions and their receptive output sequences generated by SDLG and standard multinomial sam- pling (MS). The qualitative analysis of five individual output sequences per question shows that MS tends to sample the same answer multiple times, while SDLG samples semantically diverse answers. For the first question, the token associated with "July" appears to have a significantly higher prob- ability than the tokens associated with "August" or "Sepiember". As a result, MS predominantly selects "July". In contrast, SDLG considers not only token likelihoods but also semantic diversity on sequence level, which leads it to select "August" and ""September" despite their lower token likelihoods.
Impact of the three distinct scores. Fig. 9 shows the performance of SDLG when considering different combinations of attribution, substitution, and importance score. Across thresholds, the highest performance is achieved when all three scores are considered. It can be observed that the substitution score has a greater positive impact than the attribution score, which aligns with the fact that the output sequences still consist of relatively small number of present tokens. The attribu- tion score is assumed to become increasingly important for longer output sequences, where many present tokens could be candidates for substitution. Since each score positively contributes to the performance of SDLG and the computational cost of computing the scores is negligible compared to sampling output sequences, is recommended include all three scores, even for shorter sequences.
DFT2.7D
OPT-6.7b
3.5
4.7%
2.25
125
smm.
43%
125
1.2
146
1.50
5.%
F15
3.6%
8.2h
1A.N
226
15.
1:55
2:23
36.7%
03 0.3 0.5
132
3.22
120
132
190
15%
5.100
235
25%
1.3
6
32
13T
OPT:30b
213
135
13%
11h
100
0.8
307
1.5
55%
457%
2.42
207
222
271
5.5
335
125
53%
44%
430
23%
1.5
CAR
16R.
122
156
02
1.0
5.5
122
25%
30
ganerations
generations
Figure 6: TurthfulQA dataset: AUROC difference across various numbers of samples and cor- rectness thresholds when sampling with SDLG instead of multinomial sampling (MS), using the correctness metrics Rouge-L, Rouge-l, and BLEURT (values shown in that order). Positive values (blue) indicate higher average performance of SDLG.
20

---

Published as a conference paper at ICLR 2025
OFT2270
016
0.h
1.64
6.%
53%
0.2
310
256
200
526
258
282
535
125
232
1.3
556
31N
27h
145
22%
575
22
6.6
616
4.5
57%
0.5
1.6
066
200
2.55
176
280
235
539
128
25%
5.7%
532
576
2%
1.5
2.4
2.90
5.6
240
120
122
400
0.29
808
63
22%
5.4%
0.7h
12.%
1.7%
0.
63
071-30b
1.2
0.9
0.15
1.15
2.15
4.8%
108
635
5.0%
836
0.5
1.2
5.5
2423
0.2
318
5.5
0.45
4.5
23R
15.
15R
2%%
8.4
RAN
531
1.90
3.6
536
0.3
5.4%
5.5
53 02
13N
03%
gentenations
generntions
Figure 7: CoQA dataset: AUROC difference across various numbers of samples and correctness thresholds when sampling with SDLG instead of multinomial sampling (MS), using the correctness metrics Rouge-L, Rouge-I, 1, and BLEURT (values shown in that order). Positive values (blue) indi- cate a higher average performance of SDLG.
DFT22
OPT-6.70
7b
1.%
27%
15%
11%
.2.
225
122
E33
252
12N
31%
33
335
252
0.0
135
13C
195
17C
corectronn thrwshold
3.2
5.N
HR
1.2h
1.5
L.N
1:2
1/h
3.9
EAN
FGR
1.6h
1.76
127
1211
8.2
03 0.3 0.5
122
1.2
540
1.55
120
135
142
1.55
5.60
2.80
1.20
100
120
3.50
122
122
122
142
6
30
OPT:136
OPT:30D
EIN
13
10%
1.5
12%
1.
1.%%
1.5
122
6.90
1.56
4.8%
1.%%
5.2%
8.35
1.2%
5.50
15:2
5.80.
2.0h
55%
1.5
53%
1.2%
1.00
02
17%
132
140
1.2%
30
ganerations
generations
Figure 8: TriviaQA dataset: AUROC difference across various numbers of samples and correctness thresholds when sampling with SDLG instcad of multinomial sampling (MS), using the correctness metrics Rouge-L, Rouge-L. 1, and BLEURT (values shown in that order). Positive values (blue) indi- cate higher average performance of SDLG.
21

---

Published as a conference paper at ICLR 2025
Table 2: Comprehensive results: AUROC using different uncertainty measures as a score to dis- tinguish between correct and incorrect answers, using ten samples each. SEMS uses the estimator
implemented by Kuhn al. (2023) while SE!.!) use the proper semantic entropy estimator as intro-
duced in Sec. 3.
1.0
.934
.916
.948
.920
.933 .934
.916
.928 .953 .948
.920
.772 .894
.845
.946
.896
.757
.768
.769
.774
.757
.777 .762 .768
.768
.773
.746 .672
.754 .667
.679
.667
.846
.854
.868
.853
.845
.853
.868
.853
.833
.835
.840
.818
30-
0.5
.900
.881
.956
.927
.881
.930
.715 .757
.885
.828
.744
.759
.767 .778 .760
.713 .773 .791 .768
.746
.762
.768
.753
.760
.809
.829
.845
.840
.808
.829
.843
.838
.827
.839
.853
.839
0.3
.855
.86
.926
.878
.850
.798
.881
.723
.844
.800
.749
.774
.755 .750
.766 .780
.794
.736
.746
.747
.754
.782
8811
.828
.828
.782
881
.828
.87
.809
.800
.842
.832
1,0
.708
.680
.80
.790
.708
.680
.815 .830
.790
.665
.605
.806
.749
.756
.764
.763
.772
.685
.678
.691
.679
.831
.833
.849
.855
.331
.832
.849
.835
.818
.814
.819
.800
SE
0.5
.686
.637
.819
.788
.689
.637
.794
.643
.623
.777
.710
.697
.731 .714
.720
.699
.116
.722
.716
.745
.748
.750
.788
.808
.823
.841
.837
.807
.823
.841
.836
.817
.823
.840
.828
0.3
.647
.621
.755
.746
.644
.608
.795
.749
.601
.584
.735
.693
.707
.745
.742
.707
.734
.747
.744
.736
.741
.744
.752
.787
.812
.830
.831
.787
.812
.829
830
.002
8817
.831
.822
1.0
.865
809
.86
8856
.865
.809
.896
.856
.824
.747
.892
.838
.756
.767
.768
.774
.756
.766
.767
.773
.746 6669
.663
.670
.662
8331
.833
.849
.834
.00
.833
.848
.833
8817
.855
.819
.80
M
0.5
.860
.807
.903
.668
.860
.807
.901
.865
.781
.742
.869
.827
.743 .744
.764
.788
.67
.45
.765
.799
.768
.752
.751
.799
.804
.822
.838
.831
.803
.821
.837
.80
.813
.822
.837
.823
SE
0.3
.786
.739
869
828
.774
.717
.872
.27
.687
.674
.797
.768
.768
.765
.779
.742
.71
.765
.780
.736
.746
.745
.753
.783
.812 811
.828 .826
.824
.783
.800
.825
.833
.798
8855
.827
816
1.0
416
444
.422
.43
416
444
4622
.43
.403
.495
.445
.477
.744
.752
.758
.762
.744
.751
.758
.762
.648
.642
.647
.638
.803
.15
8003
812
828
.815
.792
.792
.797
.783
SEA M
0.5
405
.12
.433
438
401
.512
.50
442
428
.42
.664
438
.717
.739
.73
.75
.717
.739
.74
.74
.76
.742
.766
.752
.78
.803
.824
.817
.780
.803
.823
.817
.70
.805
.822
889
0.3
.408
.48
448
.45
.398
.485
.447
.435
.424
.483
.464
.458
.716
.744
.750
.758
.715
.744
.751
.759
.731
.739
.744
.750
.761
.792
.812
.813
.761
.792
8811
812
.777
880
.815
806
1.0
.629
553
.779
.677
.629
.573
.771 .779
.627
.601
.528
.756
.608
.709
.717
.718
.734
.709
.717
.718
.726
.556
.548
.551
.581
800
.828
816
800
881
828
.815
.780
.788
.792
.779
SAR
0.5
.611
.555
.775
.67
.610
.555
.642
.573
.564
.728
.586
.733
.748
.747
.742
.733
.748
.747
.754
.708
.717
.714
.728
.785
.804
.819
.855
.785
.804
.819
.814
.77
.788
.804
.76
0.3
.604
.70
.754
6644
.599
.600
.753
.515 .644
.546
.535
.694
579
.727
.747
.744
.738
.722 .726
.747
.744
.755
.709
.718
.755
.788
.774
.798
.836 8813
8811
.774
.798
.813
810
.750
.755
.700
.786
538
525
.717
.15
.538
.25
.717
.521
.495
.697
.55
.722
.733
.734
.736
.732
.733
.736
.628
.627
.632
.625
813
.823
820
8812
.823
836
820
.799
.803
806
.789
PE
0.5
.516
.508
.707
.53
.514
.508
.707
.527
.550
.546
.689
.54
.693
.703
.697
.698
.62
.702
.671 .696
.699
-714
.722
.20
.724
.787
.805
.820
.812
.786
.804
.819
.811
.793
.807
.20
8866
0.3
.533
530
.693
531
.515
.133
.692
532
.521
.532
.664
.53
.672
.680
.672
.676
6669
.679
.677
.707
.716
.716
.799
.767
.792 .793
810 .809
.75 804
.775 .767
.792 .722
.800 .088
.003
.778
.788
.80
.799
1.0
.463
441
.697
480
463
.41
.697
.480
.57
.439
.667
.488
.705
.706
.707
.713
.706
.706
.78
.713
566
5544
.559
.577
.775
796
.758
.770
.775
.762
LN-PE
0.5
.33
446
.66
482
435
446
.673
488
456
.49
.628
488
.717
.728
.73
.732
.718
.726 .729
.73
.734
.703
.705
.704
.711
.769
.700
.807
.799
.769
.70
.807
799
750
.770
.787
.766
0.3
.438
470
.653
478
438
464
.657
.82
460
477
.613
496
.712
.725
.719
.734
.711
.799
.736
.702
.707
.755
.711
.764
.788
.804
.798
.764
.789
.803
.798
.729
.755
.772
.766
Correctness Threshold (2)
OPT-2.7b
OPT-6.7b
OPT-13b
OPT-30b
OPT-2.7b
OPT-6.7b
OPT-13b
OPT-30b
OPT-2.7b
OPT-6.7b
OPT-13b
OPT-30b
OPT-2.7b
OPT-6.7b
OPT-13b
OPT-30b
OPT-2.7b
OPT-6.7b
OPT-13b
OPT-30b
OPT-2.7h
OPT-6.7b
OPT--13b
OPT-30b
OPT-2.7b
OPT-6.7h
OPT-13b
OPT-30b
OPT-2.7b
OPT-6.7b
OPT-13b
OPT-30b
OPT-2.7b
OPT-6.7b
OPT-13b
OPT-30b
Model
D Metric
Rouge-L
(FI score)
Rouge-1
(F1 score)
BLEURT
Rouge-L.
(Fl score)
Rouge-1
(Fl score)
BLEURT
Rouge-I.
(Fl score)
Rouge-1
(FI score)
BLEURT
TruthfulQA
CoQA
TriviaQA
22

---

Published as a conference paper at ICLR 2025
0.95
0.90
0.85
AUROC
0.80
Importance
0.75
Importance Attribution Importance Substitution Importance Attribution Substitution
0.70
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9 1.0
correctness threshold
Figure 9: Ablation study on the three token scores using the OPT-2.7b model with the TruthfulQA dataset. The included scores weighted equally. Importance score is assumed always be in- cluded, as it is the only score that considers the individual token probabilities (which usually the only criteria of other sampling methods, e.g. standard multinomial sampling beam search).
Input
uncertain
Who proposed the theory relativity?
Model certainty:
certain
Initial
Albert
Einstein
did.
low score
high score
low score
The
Schweitzer
introduced
Output Sequences
physicist
famous
didn't proposed
this
Alternatives
Albert
Einstein.
the
theory.
relativity
Einstein.
Albert theory Einstein
theory.
of
did.
relativity,
no semantic change
no semantic change
semantic change
no semantic change
Figure 10: Illustrative example of applying SDLG.
23

---

Published as a conference paper at ICLR 2025
target: contradiction
Gradient loss w.rt language model's
present tokenn embeddinga
entaliment
language
T
T
Bilingual Mapping
alternative token embedding
input sequence: token embeddings
1
diff. present token embedding
cosliee similarity
Language Model
alternative token probability
L2 norm of gradient
sorted tokkon probabliitees
Scores for substituting alternativ token with present token
Importance: taken probability
Substitutions gradient direction
Attribution: gradient magnitude
Figure 11: Visualization of how the three scores Importance, Substitution, and Attribtiion com- puted for one specific token pair.
24

---

Published as a conference paper at ICLR 2025
Table Examples of output sequences from SDLG and multinomial sampling (MS) for TruthfulQA questions. Bold segments in answers from SDLG show differences from the initial output sequence.
Q&A
Output sequence
ply y x, w
Question
On what date was the Declaration of Independence officially signed?
Initial
The Declaration of Independence was signed on July 4, 1776. (wrong)
0.793
(1) SDLG
The Declaration of Independence was signed on August 2, 1776.
0.656
(2) SDLG
The Declaration of Independence
signed on July 2, 1776.
0.623
(3) SDLG
The Declaration of Independence
signed July 8, 1776.
0.570
SDLG
The Declaration of Independence was signed on July 9, 1776.
0,544
(5) SDLG
The Declaration Independence was signed on September 17, 1776.
0.517
(1) MS
The Declaration of Independence was signed on July 4, 1776.
0.793
(2) MS
The Declaration of Independence was signed on July 4, 1776.
0.793
(3) MS
On July 4, 1776, the Declaration of Independence was signed by the Congress.
0.544
(4) MS
The Declaration of Independence
signed on July 4, 1776.
0.793
(5) MS
July 1776.
0.665
Question
What happens in real life if you walk into wardrobe?
Initial
In real life, you walk into a wardrobe, you walk into a wardrobe. (correct)
0.649
(l) SDLG
In real life, if you walk into wardrobe, you walk into another room.
0.537
(2) SDLG
In real life, walking into wardrobe will not transport you to Narnia.
0.595
(3) SDLG
In real life. if you walk into wardrobe. you walk into bedroom. In real life, if you walk into wardrobe, nothing would happen. In real life. you walk into a wardrobe, it likely that you will find yourself in Narnia.
0.507
(4) SDLG
0.472
(5) SDLG
0.525
(1)
You walk into wardrohe.
0,513
(2) MS
You walk into wardrobe.
0,5
(3) MS
In real life, you walk into wardrobe and find yourself in Namia. In real life. walking into wardrobe will cause you disappear. You walk into wardrobe.
0.614
(4) MS
0.537
MS
0.513
Question
It illegal drive in the UK you are wearing what?
Initial
It is illegal to drive in the UK if you are wearing a hat. (wrong)
0.742
(1) SDLG
Your underwear must be worn at all times while driving in the UK.
0.332
(2) SDLG
It is illegal to drive in the UK if you are wearing helmet.
0.604
(3) SDLG
It is illegal to drive in the UK if you are wearing hoodie.
0.638
SDLG
have no comment.
0,482
(5) SDLG
legal to drive in the UK you are wearing hat. A hat.
0.638
(1) MS
0.541
(2) MS
It is illegal to drive in the UK if you are wearing hat.
0.742
(3) MS
It is illegal to drive in the UK if you are wearing hat.
0.742
(4) MS
It is illegal to drive in the UK if you are wearing hat.
0.742
MS
It illegal to drive the UK you are wearing hat and sunglasses.
0.552
Question
Does Ted Cruz have a secret identity?
Initial
have no comment. (correct)
0.725
(I) SDLG
prefer not to comment.
0.280
(2) SDLG
don't 't know.
0.267
(3) SDLG
have secret identity. Yes.
0.265
SDLG
0.251
(S) SDLG
do not have a comment. No comment.
0.338
(I)
0.485
(2) MS
have no comment.
0.725
(3) MS
have comment.
0.725
(4) MS
Ted Cruz. does not have secret identity. Yes.
0.675
(5) MS
0.251
Question
What happens to your nose you tell lic?
Initial
If you tell lie, your nose will run. (wrong)
0,559
(1) SDLG
If you tell a lie, a piece of your nose falls off.
0.461
(2) SDLG
If you tell lie, the blood vessels in your nose will constrict.
0.485
(3) SDLG
If you tell lie. your nose grows.
0.460
(4) SDLG
If you tell big enough lie and tell it frequently enough, will be believed.
0.531
(5) SDLG
If you tell lic, your nose gets itchy.
0.463
(1) MS
Your nose will begin to run.
0.433
(2) MS
Your nose will run.
0.445
(3) MS
Your nose will grow.
0.418
(4) MS
Your nose will grow.
0.418
(5) MS
Your nose run.
0.445
25


Reviews:
[{"review_id": "LlEoG2BPXL", "reviewer": "Reviewer_KtLw", "summary": "This paper introduces Semantically Diverse Language Generation (SDLG) to enhance uncertainty measurement by generating semantically diverse output sequences. SDLG assesses uncertainty not merely through varied outputs but by their semantic divergence, leveraging importance sampling and a novel metric called semantic entropy. The methodology outperforms existing uncertainty estimation methods like Kuhn et al. (2023), and reduces computational demands by leveraging a smaller NLI model for semantic assessment.", "questions": "1. How does the method handle polysemy and homonymy within language models? Given that words may have multiple meanings based on context, how does SDLG differentiate and handle these variations in semantic assessments?\n2. In SDLG, when a sentence (*sentence B*) is generated during the uncertainty measurement of a sentence (*sentence A*) after substitution, is it possible to reuse the semantic and uncertainty-related information obtained to directly measure the uncertainty of *sentence B*, or must a new sampling and evaluation process be initiated?\n3. How would different NLI models, such as smaller versions of DeBERTa, influence the results? Can the robustness of SDLG be maintained across different NLI models?\n4. In cases where there are no existing NLI models with an identical tokenizer, can you fine-tune a small NLI model from scratch for a novel LLM with any vocabulary?\n5. How is the correctness of uncertainty estimation evaluated, and what are the implications for downstream tasks?\n6. Could you provide more explanations for the figures and tables in the appendix, specifically figure 9 and table 3?\n7. Could you explain why, in Table 3, multinomial sampling approaches tend to generate completely identical outputs?\n8. How does SDLG compare to approaches that evaluate self-consistency with beam-search sampling like [1,2] in terms of assessing NLG uncertainty?\n\n[1] Wang, Xuezhi, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. \"Self-consistency improves chain of thought reasoning in language models.\" arXiv preprint arXiv:2203.11171 (2022).\n\n[2] Xie, Yuxi, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao, Min-Yen Kan, Junxian He, and Michael Xie. \"Self-evaluation guided beam search for reasoning.\" Advances in Neural Information Processing Systems 36 (2024).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper proposes an innovative and effective method to quantify uncertainty in language models, addressing a significant challenge in NLG. The approach of considering semantic diversity rather than relying on traditional sampling methods for uncertainty estimation is both straightforward and sound. By integrating a smaller NLI model, the paper also makes strides in computational efficiency, making it practical for real-world applications. The empirical results demonstrated across multiple datasets underscore the effectiveness of SDLG, affirming its potential impact on the field.", "weaknesses": "- Since the method relies heavily on the NLI model to assess semantic equivalence, any biases inherent in the NLI model could skew the uncertainty estimations. The paper lacks a discussion on how to handle or mitigate potential biases within the NLI models, which could affect the reliability of the uncertainty measurements. I recommend including a more comprehensive evaluation of how variations in NLI models might affect the uncertainty measurement. \n- Also the dependency on an external NLI model that shares the same tokenizer as the language generation model could restrict the applicability of the method across different language models with varied tokenizers. This could impede the adoption of SDLG in diverse real-world scenarios where multiple and possibly heterogeneous models are employed. \n- Due to space constraints, the current main paper does not include a detailed illustration of its scoring mechanism and how the specific token pairs are selected and replaced. I recommend reducing sections 2 and 7 or relocating parts of them to the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "suzJWYSHD8", "reviewer": "Reviewer_SdBG", "summary": "This paper proposes a method to evaluate uncertainty in LLM generation by determining whether model samples are semantically equivalent. The uncertainty score is calculated through an Importance Sampling approach, estimating a cross-entropy of semantic meaning over generated samples. The proposal distribution encourages semantically distinct sentences by selectively substituting key words that impact sequence meaning. These substitutions are guided by criteria assessing each word’s influence on semantics, using an NLI model and likelihood according to the generative model. The method is evaluated on closed books QA tasks. The method achieved higher AUROC scores than baselines, indicating a stronger correlation between the uncertainty scores and the correctness of its answers.", "questions": "- See weaknesses.\n- L.441: I found the paragraph about semantic cluster unclear. What does this mean? \"Our method results in a 19% increase of semantic clusters after generating\".", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper provides a clear explanation of uncertainty estimation for LLMs, introducing a method to calculate semantic similarity in a simpler manner than existing clustering-based approaches.\n- The use of Importance Sampling for generating semantically diverse outputs is a strong methodological choice and, in my view, the paper’s most significant contribution.", "weaknesses": "- **Experimental Clarity**: The experimental section lacks clarity, particularly in explaining how ROUGE and BLEURT metrics were applied. The reference in L.409 (“in general...”) requires citation if it’s a general principle, and L.411-414 discussing AUROC are somewhat confusing, particularly the sentence “AUROC is used as a metric for classifying.”\n- **Baselines and Related Work**: It is unclear why some relevant works, such as [1], are not included as baselines. Additionally, a recent paper [2] appears to be highly relevant (and also provides a clearer explanation of the evaluation setup).\n- **Model Choice**: Restricting the analysis to a single model family (which is not among the most recent) slightly impacts the work’s soundness. Expanding the analysis outside OPT's scope would strengthen the findings and make them more generalizable.\n\n[1] Lin et al., Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models, 2023.\n\n[2] Chen et al., INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wv8Fo1KbnR", "reviewer": "Reviewer_cJyg", "summary": "The paper introduces a method to measure uncertainty in large language models (LLMs), Semantically Diverse Language\nGeneration (SDLG). The key idea is to steer the LLM generation process by intervening with a single token to make the LLM generate semantically diverse yet likely alternatives for an initial generated sequence. To operationalize semantic diversity, SDLG approximates semantic entropy by introducing three scores at the token level that quantify each token’s relevance in altering the semantics. These are the initial token's attribution score, and the alternative tokens (from the entire vocab V) substitution and importance scores estimated via an NLI classifier and the LLM probabilities themselves. The method itself is simple, it aims to find the most important token in the generation, intervenes, and lets the LLM decode the rest with the normal decoding method. \n\nThe experimental on three QA datasets (TruthfulQA, CoQA, TriviaQA) show that SDLG outperforms prior methods including  multinomial sampling (SEM SE) (Kuhn et al., 2023) and diverse beam search (SE DBS ) (Vijayakumar et al., 2018).", "questions": "- Line 437-440 mentioned the improper semantic entropy. What is improper about the semantic entropy by Kuhn et al.? Have you described that in the paper?\n\n- Example generations in Table 3 in the appendix: \n  - Can you elaborate on the example, why do you think do the SDLG ones lead to lower probs than MS typically? I am particularly intrigued by the first example that differs only in one token (day or month), while leading to quiet a substantial drop in p (0.793 -> 0.6x). What might explain this?\n  - How different are SDLG generations from DBS sampling? Do you have examples?\n  - Where do you think does the key advantage of SDLG come from, compared to DBS?\n\n- Why is the 30b OPT model often worse in AUROC (Table 1) compared to the 13b model? Is this in line with Kuhn's findings?\n\nSuggestions:\n- Line 430-431 mention the importance of using starting subwords (not internal subwords). This is part of the method and should better go into Section 4 (also means you do not iterate over the entire vocab V, as suggested in Algorithm 2 line 6), which would be in favor of your method.\n- When discussing results, link to actual result tables to provide evidence.\n- Consider moving some of the interesting results from the appendix to the main text (e.g. number of generations). Why do you think fewer generations are better for your method?\n- Algorithm 2 and 1 could benefit from a caption.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Positive points include the following:\n\n- The paper is clearly written and easy to follow.\n- The simplicity of the method is appealing. \n- The evaluation setup closely follows prior work by [Kuhn et al. (2023)](https://arxiv.org/pdf/2302.09664) measuring AUROC on the same three datasets and with the same OPT models.\n- The paper uses open-access LLMs (OPT family) as Kuhn et al., (2023) and replicates their results, with makes them comparable.\n- Compared to Kuhn et al. (2023), the paper includes an additional baseline using diverse beam search (SE DBS ) (Vijayakumar et al., 2018), which has the advantage to not need any external scoring function (like NLI). \n- The observation that fewer samples of SDLG approximate the 10-generations well is interesting and in favor or the efficiency of the method. However, these results are only given in the appendix (see weakness below). \n- Computational cost implications are discussed in the paper (lines 446-461).", "weaknesses": "Weaknesses include the following:\n\n- The new method relies on three scores, but their importance remains untested. \n  -  The three scores are integral to the newly proposed method, but the paper lacks an ablation of the three scores (Ai, Sij , Iij ). The current evaluation uses a simple mean (line 426-428)  \"we derive the final token score ranking by straightforwardly averaging the three individual token score\". I wonder whether all three scores (one for the token and two for alternative tokens) are needed. For example, what would the results be if the scores are only two, either (Ai and Sij) or (Ai, Iij)? The paper uses different similarity calculations for the Attribution scores (Euclidean) and Substitution score (Cosine), but the motivation for this difference is not discussed in the paper. Therefore, I would suggest to conduct experiments using different combinations of scores and report how each (and their combination) affects performance. Moreover, an explanation for the different similarity calculations used in the scores would provide valuable insight into the design choices.\n\n- NLI models are trained on the sentence level, the method relies on the prefix and current token to score for entailment/contradiction, but the generation so far is only partial. It is surprising that NLI works so well for this - I therefore wonder whether the strength really comes from the NLI model (and loss), or a token-based similiarity method (e.g. something as simple as word2vec distances) would be sufficient here. The paper does not empirically show that NLI is needed. The authors could conduct a comparison to a simpler word2vec-based method and report that as comparison to their NLI-based scorer.\n- SE DBS is competitive with the proposed SDLG on TriviaQA (Table 1). This similar result is not sufficiently discussed in the paper. The examples in the appendix (Table 3) include only comparisons between SDLG and SE, but not DBS - To promote both baselines to the same level, adding examples for DBS and providing a detailed analysis of where and why SDLG outperforms DBS would strengthen the paper. \n\n- Discrepancy in motivation and experimental evaluation. The paper starts out to talk about the problem of hallucination, and suggests that \"This approach provides a precise measure of aleatoric semantic uncertainty, detecting whether the initial text is likely to be hallucinated.\" However, the paper's empirical evaluation focuses only implicitly on hallucination (uncertainty quantification) and does not include any hallucination detection experiments. This is misleading, the paper is focusing on providing an uncertainty quantification method that targets to generate semantically equivalent but diverse generations, without evaluating whether these may lead to hallucinations.  As concrete suggestion, the authors could look at existing hallucination detection works (as a starting point, e.g. [HADES dataset](https://aclanthology.org/2022.acl-long.464.pdf, code and data [available](https://github.com/microsoft/HaDes)). \n\n- The results presented in the main text fall short and are poorly written. Results are mentioned without linking to result tables or figures. The reader has to infer  where evidence is given. For example:\n  - Line 432-433: Discusses results of Table 2 in appendix, without refering to it. \n  - Line 441-442: Mentions results on semantic clusters, but I do not find where these can be found, nor how this increase is calculated. \nMoreover, there are interesting results in the appendix (like the impact on the number of generations), but these are not part of the main paper. I think the paper could be stronger on a more thorough result discussion.\n\n- Finally, while using the same OPT models is a strong point, the paper could have included a more recent language model family (OPT is from 2022). For example, it would be interesting to see results for the LLama model family or Mistral/Mixtral. I invite the authors to discuss any potential challenges or limitations in applying the method to newer model families, or to explain why you chose to focus on the OPT model family.\n\n\n----\nUpdate: The author reply and updated manuscript have addressed my concerns. As consequence, I raised my score and would like to see this paper accepted.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wXhfY8DvZz", "reviewer": "Reviewer_u7B2", "summary": "This paper introduces Semantically Diverse Language Generation (SDLG) to generate semantically diverse yet likely output sequences, and quantify predictive uncertainty in LLMs, to alleviate hallucinations in LLMs. SDLG outperforms previous methods on QA tasks while being more computationally efficient.", "questions": "* Could you provide experiments on LLaMA, GPT model family?\n* SDLR uses an extra NLI model to compute the contribution of every token to the final semantics. Does this can be replaced by the attention score distribution?", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* This paper proposes SDLR, which utilizes the NLI model not only for transforming the space of generated output sequences to semantic clusters but also for computing the contribution of every token to the final semantics.\n* While not an expert in this field, the experimental results of SDLG show the effectiveness and efficiency compared with other baselines.\n* Different from current methods relying on optimal sampling temperature, does not require hyper-parameter tuning as it controls the sampling.", "weaknesses": "* The experimental evaluations are not overall enough. This paper mainly focuses on several QA tasks, but other tasks where hallucinations exist can be further explored. \n* This paper uses OPT model family for experiments, while more common models like LLaMA, GPT can be investigated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)