Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
Authors: Lorenz Kuhn
Yarin Gal
Sebastian Farquhar
Venue: ICLR 2023 notable top 25% (2023)
Keywords: uncertainty estimation
natural language generation

Abstract: We introduce a method to measure uncertainty in large language models. For tasks like question answering, it is essential to know when we can trust the natural language outputs of foundation models. We show that measuring uncertainty in natural language is challenging because of "semantic equivalence"—different sentences can mean the same thing. To overcome these challenges we introduce semantic entropy—an entropy which incorporates linguistic invariances created by shared meanings. Our method is unsupervised, uses only a single model, and requires no modifications to off-the-shelf language models. In comprehensive ablation studies we show that the semantic entropy is more predictive of model accuracy on question answering data sets than comparable baselines. 

Published as a conference paper at ICLR 2023
SEMANTIC UNCERTAINTY: LINGUISTIC INVARIANCES FOR UNCERTAINTY ESTIMATION IN NATURAL LANGUAGE GENERATION
Lorenz Kuhn, Yarin Gal, Sebastian Farquhar OATML Group, Department of Computer Science, University of Oxford lorenz. kuhn@cs. ox ex.ac.uk
ABSTRACT
We introduce method measure uncertainty in large language models. For tasks like question answering, it is essential to know when we can trust the natu- language outputs of foundation models. We show that measuring uncertainty in natural language is challenging because of 'semantic equivalence' different sentences can mean the same thing. To overcome these challenges we introduce semantic entropy-an entropy which incorporates linguistic invariances created by shared meanings. Our method unsupervised, uses only a single model, and requires no modifications to "off-the-shell"" language models. In comprehensive ablation studies we show that the semantic entropy is more predictive of model accuracy on question answering data sets than comparable baselines.
INTRODUCTION
Despite progress in natural language generation (NLG) tasks like question answering or abstractive summarisation (Brown et al., 2020: Hoffmann et al. 2022, Chowdhery et al. 2022), there is little understanding of uncertainty in foundation models. Without measures of uncertainty transformer- based systems it is hard to use generated language as reliable source of information. Reliable measures uncertainty have been identified as a key problem building safer AI systems (Amodei et al. 2016 Hendrycks et al. 2022).
Unfortunately, uncertainty in free-form NLG faces unique challenges. This limits how much we can learn from uncertainty estimation techniques in other applications of deep learning (Gal al. 2016 Lakshminarayanan et al. 2017 Ovadia et al. 2019) which focuses especially on image classification
(Kendall & Gal (017) regression in dimensiiii data spaces (Kuleshov et al. 2018).
The key challenges come from the importance in language of meanings and form. This corresponds to what linguists and philosophers call the semantic content of sentence and its syntactic or lexical form. Foundation models output
lexical confidence. But for almost all applications we care about meanings! For example, a model which is uncertain about whether to generate "France's 's capital is Paris" or "Paris is France's capital" is not uncertain in any important sense. Yet, at a token-level the model is uncertain between two forms of the same meaning. Existing unsupervised methods (e.g., Malinin & Gales (2020)) ignore this distinction.
To address semantic equivalence, estimate semantic lieeihhods-probbbliiies attached mean- ings of text rather than standard sequence sequence-likelihoods. We introduce an algorithm for clustering sequences that mean the same thing based on the principle that two sentences mean the same thing if you can infer each from the other. We then use these semantic-likelhhoods to estimate seman- tic incertainty-uncertaiin
over different meanings. In particular, we compute the entropy of the probability distribution over meanings. Adjusting for semantic equivalence in this way offers bet- ter uncertainty estimation than standard entropy and also greatly improves over methods for model self-evaluation (Kadavath al. 2022). In addition, semantic entropy scales better with model size and makes better use of increasing numbers of samples than baselines.
We further analyse major challenges for measuring uncertainty in NLG. We show empirically how sampling a set of model answers to estimate entropies in NLG must balance sample accuracy and diversity, which significantly strengthens the baselines we compare against relative prior imple-

---

Published as a conference paper at ICLR 2023
0.85
0.82
0.80
0.80
AUROC
AUROC
0.75
0.78
0.76
Semantic entropy (ours)
0.70
Normalised entropy
0.74
Predictive entropy
0.65
Semantic
entropy
Normalised
entropy
Entropy
Lexical
similarity
pfTrue)
0.72
Lexical similarity
(ours)
2:78
6:78
138
30B
Number of parameters of model
(a)
b
Figure (a) Our semantic entropy (blue) predicts model accuracy better than baselines on the free-form question answering data set TriviaQA (30B parameter OPT model). Normalised entropy reimplements single-model variant of Malinin Gales (220)), lexical similarity measures the av- erage Rouge-L in sampled set of answers for given question analogously to /Fomicheva et al. (2020), entropy and p ((True) reimplement Kadavath et al. (2022). (b) Our method's outperformance increases with model size while also doing well for smaller models.
mentations. We also examine the situational heuristic of length-normalising predictive entropies. Our main contributions are thus follows:
We explain why uncertainty in free-form NLG is different from other settings (Section
We introduce semantic entropy- a novel entropy-based uncertainty measure which uses our algorithm for marginalising over semanticall--cquivalleee samples (Section 4 and show that it outperforms comparable baselines in extensive ablations with both open- and closed- book free-form question answering using TriviaQA and CoQA (Section ().
Through hyperparameter ablations suggest how balance the trade-off' between sam- pling diverse and accurate generations for our method as well as baselines (Section 6.2 and show that far fewer samples are needed for effective uncertainty than prior work presumes.
We focus on free-form question answering (QA) because it is difficult and important use of NLG with high-stakes applications. At the same time, it is easier to establish a ground truth without expensive human evaluation than more nebulous tasks like summarisation.
Ultimately, we show that semantic entropy is an effective unsupervised way to estimate uncertainty in NLG. As an unsupervised method, requires no further training or aattaggthering, unlike super- vised methods including Lin et al. (2022a); Kadavath et al. (2022) Semantic entropy is designed to work with existing foundation and large language models with no modifications 'out-of-the-bon Our experiments use OPT (Zhang et al. 2022) but semantic entropy works with any similar model.
BACKGROUND ON UNCERTAINTY ESTIMATION
Our method draws inspiration from probabilistic tools for uncertainty estimation, which have been extensively employed in settings like deep image classification (Gal al.) (0066). Although these methods are often used in Bayesian models, we emphasise that our method does not require any special training or architectural modifications and is not limited to Bayesian settings.
The total uncertainty of prediction can be understood as the predictive entropy of the output dis- tribution. This measures the information one has about the output given the input. This entropy is highest when the output is minimally outcomes. The predictive entropy for point x is the conditional entropy of the output random variable Y with realisation given x:
preeiaiiiepeiiiiiinggg.g the same probability for all possible
PE(x) H(Y x)) x)== - - f ply 2x) In x)
One can further distinguish aleatoric uncertainty- uncertainty the underlying data distribution- and epistemic uncertainty-resule from missing information (Kendall & Gall 2017). Epistemic
2

---

Published as a conference paper at ICLR 2023
uncertainty, measured using mutual information, can be useful but is hard to estimate, especially for very large models, requiring special methods and computational expense. Instead of estimating the epistemic uncertainty based on the model variance, the epistemic uncertainty
also be pre-
dicted directly using a second model (see e.g. Jain et al. () We do not use mutual information in this work, because our focus is on existing foundation models 'off-the-shelff Similarly, while, e.g., Malinin & Gales (2220) use ensembles of models to estimate the integral in Eq. (T) we use
samples from single model's 's output distribution. Prior networks (Malinin & Gales 2018 Malinin et al. 2020) estimate model uncertainty by emulating an ensemble with single model. This could be important for NLG because of large model sizes.
For sequence sequence-prediction tasks like NLG, the probability of the entire sequence, s, is the product of the conditional probabilities of new tokens given past tokens, whose resulting prg-rrbbability is log z) E, logp(s; sę;), where is the i'th output token and Sci denotes the set of previous tokens. Sometimes, instead of the entropy of these probabilities, the geometric mean token- probability is used instead (Malinin & Gales 2020) becoming arithmetic mean log-probability
N Ex" log p(>; s.z)). Despite empirical success
Murray & Chiang (2018), so far this has little
theoretical justification.
Direct application of language models to uncertainty. In contrast to approach using prob- abilistic methods, recent work has sought to use the generating language model itself to estimate its own uncertainty. For example, [Lin al. (2022a) finetune language models to verbally describe their confidence. Meanwhile, Kadavath et al. (2022) sample multiple generations and return the completion to an NLG prompt asking if proposed answer true (further detail in Appendix|B.3 Both Lin et al. (2022a) and Kadavath et al. (022) also propose ways to finetune predictors on the embeddings of generating models to predict models uncertainty. While promising, these approaches need task- labels, additional training, and seem to be unreliable out-of-distributin (as shown in Figures 13 and 14 in Kadavath et al. (2022)).
CHALLENGES IN UNCERTAINTY ESTIMATION FOR NLG
Approaches to NLG uncertainty might treat the language model black-box (e.g., asking it if its answer is correct) or alternatively focus on the probabilistic model without accounting for the special characteristics of language (e.g., measuring predictive entropy).
Our unsupervised approach instead uses the powerful tools of probabilistic modelling, but also recognises the unique challenges posed by free-form NLG. In this section, we critically analyse the probabilistic interpretation of language models in order to ground both our method and future exploration of the field.
3.1 SEMANTIC EQUIVALENCE IN LANGUAGE OUTPUTS
Most machine learning problems have mutually exclusive outputs. An image in class 17 is not class 29 as well: regression output of 23.1 is not anything else; an RL agent going left does not go right. In contrast, for free-form text generation an output usually means the same thing as many other outputs. For example, "The capital of France is Paris" means the same thing as "France's capital is Paris". Linguists and philosophers distinguish text's meaning-its semantic content-from its syntactic and lexical form. The syntax the grammatical structure while its lexical form is the specific words used. Lexical equivalence entails the other two, but not the reverse.
We almost always care about the semantic content of a sentence. For Docision--robllemt relying on NLG, meaning is usually an invariance in output-space which is not present in the model spec- ification. This is true for question answering, summarisation, artificial assistants. Mcanings are especially important for trustworthiness: system can be reliable even with many different ways to say the same thing but answering with inconsistent meanings shows poor reliability.
We can formalize semantic equivalence mathematically. Let the space of tokens in a language be T. The space of all possible sequences of tokens of length is then SN = N For some sentence 8 E Sx, sequence of tokens E T there is an associated meaning
Let us introduce placeholder semantic equivalence relation, E(.,)) ), which holds of any two sen- tences that mean the same thing-we operationalise this in Section Recall that equivalence
'Thcories of meaning are contested (Speaks) 2021). However, for specific models and deployment contexts many considerations can he set aside. Care should be taken comparing very different models and contexts.
3

---

Published as a conference paper at ICLR 2023
Table I: Answers to the question "What is the capital of France?" (a) When all generations from the model mean different things, semantic clustering has no effect-the entropy and semantic entropy identical. (b) When some of the answers semantically equivalent ("Paris" and "It's Paris") the semantic entropy does better job of capturing the actually low uncertainty.
(a) Scenario 1: No semantic equivalence
(b) Scenario 2: Some semantic equivalence
Answer
Likelihood p(s x)
Semantic likelihood
Answer
Likelihood p(s x)
Semantic likelihood
Esco p(s ())
Enco p(s x))
Paris
0.5
0.5
Paris
0.5
0.9
Rome
0.4
0,4
It's Paris
0.4
London
0.1
0.1
London
0.1
0.1
Entropy 0.31
0.31
Entropy
0.31
0.16
relation is any reflexive, symmetric, and transitive relation, and that any equivalence relation set corresponds to a set equivalence classes. Each semantic equivalence class corresponds to one possible meaning that text can have. That is, for the space of semantic equivalence classes C the sentences in the set с E C all share a meaning such that Vs, s' e E(s, s').
Ordinarily, large language models produce conditional distributions over tokens and their resulting sequences. That is, the probability of the sequence conditioned on the context comes from condi- tional token probabilities p(s x)) II. p(s; s<i; x). Instead, we focus the probability of the model generating any sequence that shares some meaning. This can be written as
p(c ())
Σ p(s p(s|
[0ll))>[[> p(s; sc..z).
(2)
sEc
8Ec
Formally, this treats the output random variable whose event-space is C, a sub-o-algebra of the standard event-space
3.2 SAMPLING THE EXTREMELY HIGH-DIMENSIONAL LANGUAGE-SPACE
Recall from Eq. [ that estimating predictive entropy requires taking an expectation in output-space. However, the output-space of natural language has ((T) M) dimensions. Moreover, while we can sample from our autoregressive token- -odel, we lack a normalized probability density function over sentences. The expectation must be approximated by Monte Carlo integration pegeraatio-smmplline sampling a finite set of sentences from the output distribution and averaging their likelihoods to compute the entropy. For entropies the average dominated by low-probability sentences (whose logs are large and negative) making Monte Carlo integration difficult (Mackay 2003).
3.3 VARIABLE LENGTH GENERATIONS
Sentences of natural language have different lengths. As widely noted (Murray & Chiang 2018) and especially in the context of NLG uncertainty by Malinin & Gales (020), in expectation longer sequences have lower joint likelihoods because of the conditional independence of the token prob- abilities. The joint likelihood of sequence of length N shrinks exponentially in N. Its negative log-probability therefore grows linearly in N, so longer sentences tend to contribute more to entropy.
We therefore interpret norgaliiim the log- probabbiliisss when estimating the entropy serting that the expected uncertainty of generations is independent of sentence length. Sometimes, this is approximately valid. Other times, longer sentences may well be usually more uncertain (e.g.. when the goal is to exactly match a typically short reference answer, such as for TriviaQA). In these cases, the advantages of length- normalisation become less clear-cut, as we show empirically in Sec- tion 6. This offers some guidance priori on cases when cength-normalisation is appropriate.
SEMANTIC UNCERTAINTY
We have introduced the idea that uncertainty over meanings is more important for most situations than uncertainty over the exact tokens used to express those meanings. Our method examines un- certainty in meaning-space spacce--spee--eee the entropy of the random variable representing the output distribution in the semantic event-space. This is in contrast entropy in the usual token event-space. To do this we introduce a novel algorithm for estimating the semantic equivalence relation as well as a novel uncertainty estimation algorithm for semantic entropy. At high level this involves three steps:
4

---

Published as a conference paper at ICLR 2023
1. Generation: Sample M sequences {s( ".."..................... (((....))())) from the predictive distribution of large language model given a context 2.
2. Clustering: Cluster the sequences which mean the same thing using our bi-directional entailment algorithm.
3. Entropy estimation:
Approximate semantic entropy by summing probabilities that share a meaning following Eq. (2) and compute resulting entropy. This is illustrated in Table [
Step 1: Generating a set of answers from the model
First we sample M sequences (s(1)....,.
which we will use later to estimate the uncertainty.
These sequences must be sampled according to the distribution p(s x)). In this paper, we sample these sequences only from single model using either multinomial sampling multinomial beam sampling. We show in Section 6.2. that the choice of sampling temperature and sampling method can have a significant impact Malinin & Gales (2020), we do not use an ensemble of models. Ensembling would probably improve performance, but the cost of training multiple independent foundation models is often prohibitive.
the performance of both our method and the baselines. Unlike
Step 2: Clustering by semantic equivalence
In Section 3.1 we formalised semantic equivalence by introducing the semantic equivalence rela- tion, E(.,), J. which induces semantic equivalence classes which are sets of sequences that share a meaning. Recall that the equivalence class is set of sequences s such that Vs, s' c:: E(s,s'), We operationalise -):--) using the idea of bi-directional entailment. A sequence, s, means the same thing as second sequence, s if and only if they entail (i.e. logically imply) each other. E.g., "The capital of France is Paris." entails "Paris is the capital of France." because they mean the same thing.
Importantly, we require that the sequences mean the same thing with respect to the context-key meaning is sometimes contained within the context. For example, "Paris." does not entail "The capital of France is Paris." because "Paris." " not a declarative sentence without context. But within the context of the question, the one-word answer does entail the fuller answer.
In general, any natural language inference classification system (NLI) can be used for our bidirec- tional entailment clustering algorithm. In our case, we use a Deberta-large model (He et al. 2020a) that is fine-tuned on the NLI data set MNLI (Williams al. 2017). For each pair of sequences in our set of samples, s and s', we detect whether it is possible to infer the concatenation of the context and s from the concatenation of the context and s' and vice versa. To do this we concatenate each of the two question/answer pairs, and then concatenate them both together separated by special neutral,
token. The Deberta model then classifies this sequence into one of: entailment contradic coi... We compute both directions, and the algorithm returns equivalent if and only if both directions were enta i ment. Algorithm pseudocode is provided in Appendix A.2
Because this component is novel, confirm in Appendix B.2 that the bidirectional entailment classifier works by manually labelling 300 generations for semantic equivalence, finding an accuracy of 92.7% on TriviaQA and 95.5% on CoQA.
Computational cost. The bidirectional equivalence algorithm is combinatorially complex in M, it requires ) comparisons in the worst-case. In practice, however, the computational cost is small compared to the cost of generating sequences. First, as show in Section 6.2] M < 20 is often sufficient for good uncertainty. Second, because the Deberta-large model is so much smaller than the main language model (1.5B parameters), each pair comparison is much faster than generat- ing even one token from the main model. Third, because semantic equivalence is transitive we only need to compare one member of each equivalence class remaining sequences (see Algorithm Additionally, the number of semantic clusters in our tasks is empirically quite low, see Table 2
Step 3: Computing the semantic entropy
Having determined the clusters of generated sequences that mean the same thing, we add their like- lihoods following Eq. (2) as way of determining the likelihood of each meaning, rather than each sequence. We then compute the semantic entropy (SE) as the entropy over the meaning-distributio
SE(x) = Eplc|
----et p(c x) logple| x) = -
((E
(Σ p(s x)) log Σ (a>ss)))) 10)) SEC
(3)
sEc
5

---

Published as a conference paper at ICLR 2023
We do not have access to every possible meaning- classs Instead, we can only sample from the Sequuecce-geneatting distribution induced by the model. To handle this, we estimate the expectation in Eq. (3) using Monte Carlo integration over the semantic equivalence classes C' from Algorithm
SE(x) ~ -CI log p(C, z).
(4)
This is an unbiased estimator of the entropy in Eq. In addition, in some cases we use length- normalisation as described in Section 3.3
4.1 HOw THE SEMANTIC ENTROPY ADDRESSES THE CHALLENGES OF NLG
The main inspiration of semantic entropy is to address the semantic invariance of natural language head-on by converting the problem of uncertainty estimation into meaning space.. In addition, se- mantic entropy goes some way towards addressing unequal token importance. Generations whose meanings are the same but differ on unimportant tokens will be added together, which we expect will reduce the effect of the likelihoods of unimportant tokens although we do not demonstrate this empirically. However, this challenge is only partially addressed: semantic entropy will still pay too much attention to non-keyword likelihoods. This is one area where supervised language-modke -oogel-- based uncertainty tools (Lin et al., 2022a Kadavath et al., 2022) might enable future improvements that handle this challenge better. We address the challenges of sampling and variable-length gener- ation using specific details of our sampling procedure in Section
RELATED WORK
Prior work on uncertainty in foundation models for NLP has largely focused the calibration of classifiers (Jiang et al. 2021; Desai Durrett 2020) and text regressors (Glushkova et al. 2021 Wang et al. 2002). These settings, analogous to classification
regression settings in other modalities like vision, and conventional uncertainty measures like MC dropout or Deep Ensembles can be applied without modification (see Section f discussion of uncertainty in deep learning in general). As we argue in Section generative natural language poses important further challenges. Jiang al. (202T) do examine calibration in generative question answering and find only a weak correlation between the log-likelihood models assign to their answer and the answer's correctness. In Section we explain however why semantic equivalence in natural language makes calibration problematic evaluation for generative language models. Reliable uncertainty can be useful on downstream tasks such as graph semantic parsing (Lin al. 20226).
Some research has addressed uncertainty or calibration in NLG either by prompting the models to evaluate their own generations or by fine-tuning the generating model to predict its uncertainty (Mielke et al. 2020 Lin et al.) 2022a Kadavath et al. (022). These methods need further training and supervision. Because they need additional training and supervision, they are hard to reproduce, expensive to create, and have been shown to be sensitive to distribution shift. For example, unable to implement one proposal by Kadavath et al. (022)) to train a language model to directly predict confidence due to hardware limitations. Our unsupervised method which uses models 'off- the-shelf' avoids these limitations.
were
Many of the issues that make probabilistic uncertainty estimation in NLG difficult also make tomatic evaluation of NL. difficult. Ott et al.] (2018), for instance, study how the performance of machine translation models suffers because one sentence be translated in multiple ways. Sim- ilarly, Sai et al. (2022) discuss how paraphrase detection can be used to evaluate NLG and other related methods might transfer to uncertainty estimation.
Automatic paraphrase identification can be based on comparing lexical features of two given se- quences (Fernando & Stevenson 2008 Issa al. 2018) or on measuring the similarity between the embeddings of the two sequences (Yu et al. 2014]] Socher et al. 2011). Recently however, SotA paraphrase identification approaches have primarily used BERTTbasse models to classify pairs of sequences into the classes paraphrase. and not pa paraphrases (He et al.) 2020b Tay et al.)
202T]. The idea of formalising semantic equivalence via textual entailment has a long history in
linguistics (Culicover 1968) and NLP (Padó et al. 2009 Androutsopoulos & Malakasiotis 2010).
Transformer- based paraphrase detection models such as EFL (Wang et al.) 2021) achieve SotA per-
formance on paraphrase detection benchmarks such as Quora Question Pairs Wang et al. (2017)

---

Published as a conference paper at ICLR 2023
Semantic entropy
Predictive entropy p(True)
Normalised entropy
Lexical similarity
0.80
0.75
AUROC
AUROC
0.70
0.70
0.60
2.78
6.78
138
308
2.78
6.78
138
308
Number of parameters of model
Number of parameters of model
(a) CoQA
(h) TriviaQA
Figure 2: (a) On CoQA open-book question answering semantic entropy demonstrates better uncer- tainty than ordinary predictive entropy with and without normalisation at larger model sizes. It also performs significantly better than p(Tre)). (b) TriviaQA shows similar results. Identical to Fig. lb with the addition of p(True), which was previously omitted to avoid stretching the scale.
EMPIRICAL EVALUATION
We demonstrate that semantic entropy is effective way to quantify the uncertainty of NLG on free-form QA tasks. Effective uncertainty measures should offer information about how reliable the model's answers are-that is, very uncertain generations should be less likely be correct.
Performance evaluation. Following prior work (e.g. Filos et al. (2019)), we evaluate uncertainty by treating uncertainty estimation as the problem of predicting whether to rely on a model generation for a given context-whether to trust an answer to a question. The area under the receiver operator characteristic curve (AUROC) metric is equivalent to the probability that a randomly chosen correct answer has higher uncertainty score than randomly chosen incorrect answer. Higher scores are better, with perfect uncertainty scoring 1 while a random uncertainty measure would score 0.5.
The AUROC is a better measure of uncertainty for free-form question answering and NLG than calibration measures like the Brier score, which are often used classification or for multiple choice QA. This is because the language model outputs a likelihood for given token-sequence, but not for an entire meaning.
order to estimate the Brier score, we would need to estimate the entire probability mass assigned to any possible way of saying the correct answer. This is intractable for free form text where we do not have access to probabilities about meanings. contrast, we can estimate the entropy because it is structured as expected information, which makes Monte Carlo integration suitable.
Model. We the GPT-like OPT models (Zhang et al.) 2022). We vary the size of the model between 2.7B, 6.7B, 13B and 30B parameters, while our headline results are all reported using the largest computationally feasible model, with 30B parameters. In all cases we use only a single un- modified model. There is no ensembling and no stochastic or Bayesian modification. We chose this because in cases cutting-edge foundation models are not available ensembles and large to efficiently perform approximate Bayesian inference with. We do not fine-tune these models on TriviaQA or CoQA but use them in their pre-trained form.
Datasets. We use CoQA Reddy et al. (2019) as an open-book conversational question answering problem (in which the model answers a question using a supporting paragraph). We use the devel- opment split (~~8000 questions). We also use TriviaQA (Joshi et al. (207)) as a closed-book QA problem (in which the model must answer a question without access to supporting paragraph). We use subset of 8000 questions of the training split to match the size of CoQA.
We evaluate correctness of our model's generations on the underlying dataset using the fuzzy matching criterion: L(s, s() 1 Ltoo That is, we consider answer to be correct if its Rouge-L (Lin Och (204)) - a measure the longest common subsequence - with regards to the reference answer larger than 0.3. In Appendix B.3 we study other objective functions such as exact matching and Rouge-1 and find our method to be robust to these choices.
Baselines.
We compare our method against predictive entropy, length- normalieedd predictive
tropy (Malinin & Gales 2020), p( p(True) (Kadavath et al. 2022). and lexical similarity (similar to (Fomicheva et al. 2020)) Predictive entropy is widely used measure of uncertainty in other
7

---

Published as a conference paper at ICLR 2023
Table 2:
Incorrectly answered questions have more semantically distinct answers than correct ones. On its own, this count is a reasonable uncertainty measure, though semantic entropy is better.
Dataset
Average of semantically distinct answers
AUROC
Correctly answered Incorrectly answered
Semantic entropy
distinct answers
CoQA TriviaQA 1.89
1.27
1.77
0.77
0.66 0.79
3.89
0.83
domains, and has been used as a baseline without length- noggalioomaiiiiionnnn in, e.g., Kadavath et al. Here, the score just the predictive entropy the output distribution as described in Eq. ( Length-normalised predictive entropy divides the joint log-probability of each sequence by the
(0022).
length of the sequence, as proposed by Malinin & Gales (2020) in the case of NLG uncertainty and
further discussed in Section 3.3] Note that unlike Malinin & Gales (2020), we use only a single
model, not an ensemble, and use multinomial sampling as we do for all other methods. p p(True) proposed by (Kadavath al. 2022)
way estimate the probability that a model's 's generation is correct by "asking" the model if its answer is correct. They propose sampling M answers and constructing a new natural language question using these possible answers as context before ask- ing whether the proposed answer is correct and measuring the probability of the completion being True. An example of the format is provided in Appendix [ Note that our implementation of this uses OPT models with up to 30B parameters, while Kadavath et al. (2022) use a proprietary 52B parameter model. We are also limited computationally to 10-shot prompting while the original paper uses 20-shot prompting. Lexical similarity uses the average similarity of the answers in the answer set A: غ E! Σ/ sim (5,,))) where C' |A (A) - 1)/2. and sim is Rouge-L. Additionally, evaluate
baseline (Lin et al. 2022b Appendix [.6] and study why it is not very predictive of model accuracy in this setting. All code and data used in our experiments are available at httpt:: htttps://githb... com wwwtm/ooeenzun///// semantic_uncertaale
6.1 SEMANTIC ENTROPY UNCERTAINTY
For both TriviaQA and CoQA, semantic entropy improves over baselines in predicting whether a model's ss answer to a question is correct. For TriviaQA. using the largest model we show in Fig. lalwe show that semantic entropy has significantly higher AUROC than entropy in sequence- proocccceprbabblltt- space with and without gength-normalisationn as well as the lexical similarity baseline. time, it performs dramatically better than p pTTrue). Similarly, we find in Fig. 1b that our method out- performs more for larger model sizes and continues to stcadily improve as the model size increases, with the performance of the p (Tuee) baseline added in Fig. nnot (not shown in the opening figure for visual clarity). For COQA, in Fig. 2a we show that semantic entropy predicts model correctness significantly better than the baselines at larger model sizes.
the same
The ground truth answers for TriviaQA are generally single words or very short phrases, while CoQA contains both longer and shorter ground truth answers. This is why performing length- normalisation has a large effect for CoQA but no effect for TriviaQA (compare Fig. and Fig. 2h). TriviaQA is also more challenging dataset: accuracy of 50.6% against 82.3% for CoQA.
We can better understand the mechanism of action for semantic entropy by examining the differ- ence between incorrect and correct answers generated by the model. In Table 2 we show that the average number of semantically distinct clusters of answers ((C)) from the 30B parameter model significantly greater for incorrectly answered questions than correctly answered ones. This fits our predictions, which is that the model is more likely to generate incorrect answers when it is un- certain about the most likely generation. There are 10 answers generated per question, so there substantial overlap in meaning. We also show that simply using the number of semantically distinct answers as an uncertainty measure on its own performs reasonably well. Semantic entropy has a higher AUROC than the number of distinct answers, especially for CoQA whose difficulty causes greater spread in predicted probabilities between possible answers.
Finally, we can see that much of the performance gain comes from making better use of more samples. In Fig. Ba we show that for both CoQA (top) and TriviaQA (bottom) the gap between semantic entropy and
entropy widens as the number of samples increases.
6.2 HYPERPARAMETER FOR EFFECTIVE SAMPLING
Adjusting the temperature used for multinomial sampling has two competing effects on the gener- ated sequences produced by the model. Increasing the temperature increases the diversity of samples
8

---

Published as a conference paper at ICLR 2023
0.76
0,800
AUROC
0.72
AUROC
0.775
0.70
0.750
Semantic entropy (ours)
0.68
0.725
Length-nommalised entropy
0.66
10
0.25
0.50
1.50
0.4
0.800
0.9
AUROC
0.775
Accuracy
Diveesitt
0.750
0.7
Semantic entropy (ours)
0.725
Length- normalised entropy Entropy
Average correctness of sampled answers
0.6
Diversity of sampled answers
0.700
5 #samples used to estimate entropy
10
0.25
0.50
1.00
1.50
Temperature
(a) (top) CoQA, (bottom) TriviaQA
(b)
Figure 3: (a) Semantic entropy makes better use of additional samples because it handles duplication better, the performance gap therefore continues to improve. (b) (bottom) Higher temperatures result in more diversity but less accurate generations. (top) The best performing uncertainty comes from an intermediate temperature that balances these two forces. Results on TriviaQA.
(Fig. 3b bottom figure, solid line). One would expect more diverse generations to cover the space of possible meanings more fully. Here we measure the diversity using the average overlap of the longest sub-sequence among sampled answers (1 - Rouge-L(s, s')), At the same time, reducing the temperature improves the average correctness of the answer (Fig. 3b bottom figure, dashed line). Typically, more accurate models are also better at estimating uncertainty.
In fact, we find that these two effects compete and the highest AUROC for semantic entropy and length-normalised entropy is optimised by an intermediate temperature of 0.5 (Fig. top figure). A lower temperature would improve accuracy, while higher temperature would improve diversity. A similar figure for CoQA can be found in Appendix [ Note that prior work using predictive entropy baseline uses temperature of 1.0 (adavvtth al. 2022), which our evaluation suggests would significantly weaken the baseline relative to our implementation.
7 DISCUSSION
Many natural language problems display a crucial invariance: sequences of distinct tokens mean the same thing. Addressing this directly, we introduce semantic entropy-the entropy of the distribu- tion over meanings rather than sequences- and show that this is more predictive of model accuracy on QA than strong baselines. Our unsupervised approach using 'out-of-the-box models improves reproducibility and is easier to deploy. Unsupervised uncertainty may also help address the obser- vation raised in prior work that supervised uncertainty measures struggle with distribution shift.
For semantic entropy, we introduce a novel bidirectional entailment clustering algorithm which uses smaller natural language inference model. Our method therefore represents middle ground be- tween fully probabilistic methods and methods that use language models to exploit aspects of natural language that not transparently present in the model activations. We believe that this sort joint approach is more promising than relying on either perspective on its own, especially as language models continue improve. This will become more important in cases where language models are capable of deception, something which our method does not protect against, rather than merely being uncertain between many possible meaningful options. By combining model internals with model prediction can hope to stay a step ahead of model capabilities.
We focus on question answering because this a particularly important free-form NLG problem with relatively clear ground truths. In the future, however, we hope our work on semantic equiva- lence can pave the way towards progress settings like summarisation where correctness requires more human evaluation although additional progress paraphrase identification in these settings likely required first. Semantic likelihoods could also extended to other tools for probabilistic uncertainty like mutual information, potentially offering new strategies for NLG uncertainty.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
We are grateful to Geoffrey Irving, Kuba Perlin, Laura Rimell, and Miles Turpin for their advice and feedback on earlier drafts of this paper. We are also grateful to the members of the OATML group for helpful discussions about this project.
ETHICS STATEMENT
Our aim is to work towards safer AI systems by enabling users to understand the confidence and re- liability of language model generations. In principle, this could help mitigate many of the potential harms of NLG from foundation models such as generating false and harmful information in response to genuine questions about important topics like medical questions. However, this potential benefit comes with the risk that systematic mistakes in the assessment of uncertainty or its communication could cause unfounded and misplaced confidence. While this paper represents research progress in identifying new considcrations and methods for uncertainty quantification in NLG, before deploy- ment we advise that practitioners conduct extensive evaluations specific to the deployment context to make sure that uncertainty is communicated in way that empowers users and is not misleading or confusing.
REPRODUCIBILITY STATEMENT
Because of the computational cost of experimentation with foundation models, most of the relatively small amount of existing research into NL.G uncertainty relies on proprietary models, finetuning of expensive models, and human evaluation. These factors put this kind of rescarch out of reach for many academic groups. Our work takes advantage of the recently released, publicly available OPT models, and builds on this to provide an uncertainty quantification pipeline for NLG that uses entirely open source tools. Meanwhile our method requires no finetuming or training of foundation models and can work with "off-the-shelf" existing models. We hope that this can facilitate more research these important topics in the academic community as well as making our methods easier to replicate. We make all of our code, as well as the hand-labelled semantic equivalence dataset drawn from TriviaQA and CoQA, available under an MIT license.
REFERENCES
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Con- crete Problems in AI Safety. arXiv, 2016.
Ion Androutsopoulos and Prodromos Malakasiotis. A survey of paraphrasing and textual entailment methods. Journal of Artificial Intelligence Research, 38: 38:135-1877 2010.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020 !I
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.0233 2022.
[
Peter Culicover. Paraphrase generation and information retrieval from stored text. Mech. Transl. Comput. Linguistics, 11(3-4):78-83 1968. 6
Shrey Desai and Greg Durrett. Calibration of pre-trained transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 295- 302, 2020. 6
Samuel Fernando and Mark Stevenson. semantic similarity approach to paraphrase detection. In Proceedings of the !1th annual research colloquium of the UK special interest group for compu- tational linguistics, pp. 45-52. Citeseer, 2008 6
Angelos Filos, Sebastian Farquhar, Aidan N Gomez, Tim G Rudner, Zachary Kenton, Lewis Smith, Milad Alizadeh, Arnoud de Kroon, and Yarin Gal. Benchmarking Bayesian Deep Learning with Diabetic Retinopathy Diagnosis. Bayesian Deep Learning Workshop at NewrlPS, 2019. 7
10

---

Published as a conference paper at ICLR 2023
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics, 8: 539-555, 2020.
Yarin Gal et al. Uncertainty in deep learning. PhD thesis, 2016.
2
Taisiya Glushkova, Chrysoula Zerva, Ricardo Rei, and André FT Martins. Uncertainty- ma- chine translation evaluation. arXiv preprint arXiv:2109.06333 2021.
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Recoding-enhancee bert with disentangled attention. arXiv preprint arXiv:2006. 03654, 2020a.
Ruining He, Anirudh Ravula, Bhargav Kanagal, and Joshua Ainslie. Realformer: Transformer likes
residual attention. arXiv preprini arXiv:20/2. 2020b. 6
Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved Problems ML
Safety. arXiv, 2022. 1
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, al. Train- ing compute-optimal large language models. arXiv preprint arXiv:2203. 15556, 2022. |
Fuad Issa, Marco Damonte, Shay B Cohen, Xiaohui Yan, and Yi Chang. Abstract meaning repre- sentation for paraphrase detection. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol- ume I (Long Papers), pp. 442-452, 2018. 6
Moksh Jain, Salem Lahlou. Hadi Nekoci, Victor Butoi, Paul Bertin, Jarrid Rector-Brooks Maksym Korablyov, and Yoshua Bengio. Deup: Direct epistemic uncertainty prediction. arXiv preprint
arXiv:2102.08500 2021. 3
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962-977 977, 2021.
Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv: arXiv:/705.08:: 1705.0355,, 2017.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language mod- els (mostly) know what they know. arXiv preprint (xXi::2277.0522/1 2022.
Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017. 2
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncertainties for deep learning using calibrated regression. In International conference on machine learning, pp. 2796-2804.
PMLR, 2018. I
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Chin-Yer Lin and Franz Josef Och. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pp. 605-612, 2004. 7
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205. 14334, 2022a.

---

Published as a conference paper at ICLR 2023
Zi Lin, Jeremiah Zhe Liu, and Jingbo Shang. Towards collaborative neural-symbolic graph semantic parsing via uncertainty. In Findings of the Association for Computational Linguistics: ACL 2022,
pp. 4160-4173, 2022b.
19
David Mackay. Information Theory, Inference and Learning Algorithms. Cambridge University Press, 2003.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018. 3
Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction.
arXiv preprint arXiv:2002.07650, 2020.
7
Andrey Malinin, Sergey Chervontsev, Ivan Provilkov, and Mark Gales, Regression prior networks. arXiv preprint arXi::2006.11590 2020.
Sabrina J Mielke, Arthur Szlam, Y-Lan Boureau, and Emily Dinan. Linguistic calibration through
metacognition: aligning dialogue agent responses with expected correctness. arXiv preprint
arXiv:2012. 14983, 149883 2020. 6
Kenton Murray and David Chiang. Correcting length bias in neural machine translation. arXiv preprint arXiv: 10006, 2018. 3
Myle Out, Michael Auli, David Grangier, and Mare' Aurelio Ranzato. Analyzing uncertainty in neural machine translation. In International Conference Machine Learning, pp. 3956-3965.
PMLR, 2018. 6
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan. and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019. [
Sebastian Pad6, Daniel Cer, Michel Galley, Dan Jurafsky, and Christopher D Manning. Measur- ing machine translation quality as semantic equivalence: A metric based on entailment features. Machine Translation, 23(2):181-193, 2009. 6
Siva Reddy, Danqi Chen, and Christopher D Manning. Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249-266, 2019. 7
Ananya B Sai, Akash Kumar Mohankumar, and Mitesh M Khapra. A survey of evaluation metrics used for nig systems. ACM Computing Surveys (CSUR), 55(2):1-39, 2022. 6
Richard Socher, Eric Huang, Jeffrey Pennin, Christopher D Manning, and Andrew Ng. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. Advances in neural infor- mation processing systems, 24, 2011.
Jeff Speaks. Theories of Meaning. In Edward N. Zalta (ed.), The Stanford Encyclopedia of Philos- ophy. Metaphysics Research Lab, Stanford University, 2021.
Yi Tay, Vinh Q Tran, Sebastian Ruder. Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Si- mon Baumgartner, Cong Yu, and Donald Metzler. Charformer: Fast character transformers via gradient-based subword tokenization. arXiv preprint arXiv:2106. 12672 2021.
Sinong Wang, Han Fang, Madian Khabsa, Hanzi Mao, and Hao Ma. Entailment as few-shot learner.
arXiv preprini arXiv:2104: 102.1044444440 /4690, 2021. 6
Yuxia Wang, Daniel Beck, Timothy Baldwin, and Karin Verspoor. Uncertainty estimation and reduc-
tion of pre-trained models for text regression. Transactions of the Association for Computational Linguistics, 10:680-696, 2022.
Zhiguo Wang. Wael Hamza, and Radu Florian. Bilateral multi-perspective matching for natural language sentences. arXiv preprint arXiv: / 1702.03814 2017. 6
Adina Williams, Nikita Nangia, and Samuel R Bowman. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv: arXiv: 1704.0542, 2017.
12

---

Published as a conference paper at ICLR 2023
Lei Yu, Karl Moritz Hermann, Phil Blunsom, and Stephen Pulman. Deep learning for answer sentence selection. arXiv preprint Sed.::I412.12222 2014.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo-
pher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, al. Opt: Open pre-trained transformer
language models. arXiv preprint arXiv:205. 00066,, 2022.
13

---

Published as a conference paper at ICLR 2023
Table 3: Illustration of semantic, syntactic, and lexical equivalence. Work with foundation mod- els implicitly focuses on lexical equivalence, which entails the others, but we usually care about semantic equivalence.
Equivalence
Sentence A
Sentence B
Lexical
Syntactic
Semantic
Paris is the capital of France.
Paris is the capital of France.
Berlin the capital of France.
V
France's 's capital is Paris.
A FURTHER DETAILS ON SEMANTIC ENTROPY
A.I FURTHER DISCUSSION OF SEMANTIC EQUIVALENCE
We illustrate the distinction between different kinds of equivalence in Table Lexically equivalent sequences use exactly the same symbols. They are always also semantically and syntactically equiv- alent (in given context). Syntactically equivalent sentences have the same grammatical form. But they can have different meanings (not semantically equivalent) and can use different symbols (not lexically equivalent), Semantically equivalent sentences mean the same thing, but they might have different grammatical form (not syntactically equivalent) or symbols (not lexically equivalent). Two sentences can also be both syntactically and semantically equivalent but not lexically equivalent they match up to synonym.
Soft equivalence and transitivity. Formally, semantic equivalence is transitive. That is, if E(s, s') and E(s', s")) then it follows that s"s. However, the implementation of our bidirectional equiv- alence algorithm permits some classification errors and slightly 'soft'-it will sometimes return equiva va 1 ent for pairs that are not quite equivalent. As a result, is not strictly true that our equiv- alence relation transitive, and therefore not strictly true that there is unique set of equivalence classes. For example, the clusters might depend on the order in which the comparisons made. inspecting the
In practice, however, we find that this does not pose a noticeable outputs shows that the equivalence appears clear cut. However, we acknowledge this potential issue as an area for improvement in future clustering algorithms.
Unequal token importance. From the perspective of meaning, others-key words. Naive methods like predictive entropy do distinguish between key words or unimportant tokens. Supervised uncertainty methods that make of language models in the un- certainty evaluation can potentially take this into account better. In addition, our semantic entropy approach partly adjusts for this, as discussed in Section
tokens matter more than
A.2 FURTHER ALGORITHMIC DETAILS
In addition to the description of the method provided in the main body, in Algorithm I we provide the pseudocode for our bi-directional entailment algorithm.
A.3 IMPACT OF SAMPLING METHOD ON QUALITY OF UNCERTAINTY ESTIMATE
the performance of the uncertainty measures. Here, we show variant of Fig. 3b for the CoQA dataset showing an almost identical pattern. Like TriviaQA, the optimal temperature 0.5 despite a significantly harder prob- lem with lower accuracy, suggesting that this choice hyperparameter may generalize well. Unlike TriviaQA, normalised entropy outperforms semantic entropy high temperatures.
In Section study the impact of the temperature hyper-parameter
Beyond the temperature, there are number of other design choices to be made when sampling: the sampling method and hyper-parameters such as top P and top-k. Our contribution this paper is to show the importance of these choices for uncertainty estimation which has been over- looked previously, and study the temperature in particular. While we leave the detailed study of these hyperparameters to future work, do compare our default multinomial sampling method, to multinomial beam search sampling which focuses more on high-likelhhood regions of the output space.
14

---

Published as a conference paper at ICLR 2023
Algorithm Bidirrctional Entailment Clustering
Require: context z, set seqs. {s(?), ,...,, s(M) NLI classifier M, set of meanings = {{s())} for 2 S m < M do
for C C
Compare to already-processee prryeessoceeed meanings.
s(() + co
Use first sequence for each semantic-class.
left +- M "<g/>", )(m)))
Does old sequence entail new one?
right + M(cat(z,, "<g/>" , s(
Vice versa?
if frfet entai ieeett and right is centailment then c c cU(mm end if end for C Clfsmm) end for return C
Put into existing class.
Semantically distinct, gets own class.
0.76
0.74
0.72
0.70
0.68
Semantic entropy (ours)
0.66
Length normalized entropy
0.25
0.50
1.00
1.50
0.6
0.8
0.5
0.7
0.0
0.3
Averagee sampled
..gg Diversity of sampled answers
0.25
0.50
1.00
1.50
Temperature
Figure 4: CoQA temperature ablation. (bottom) Similar to TriviaQA, higher temperatures mean higher diversity and lower accuracy. (top) The best performance for both methods comes temperature of 0.5. Unlike TriviaQA, normalised entropy outperforms semantic entropy at high temperatures.
In Table we show that multinomial beam search sampling yields uncertainty measures that are less predictive of model accuracy than multinomial sampling. Beam search also generates much less diverse samples. We conjecture that multinomial beam search sampling focuses too much on the most likely sequences. The diversity of this beam search corresponds to the lowest temperature result in Fig. 4 As in the main body of the paper, measure diversity the average lexical overlap the answers in the answer set. Additionally, we investigate, why the semantic entropy underperforms the length-normaliss classifications of our semantic equivalence method at T=1.5, and we find that at these temperatures, many the generated model answers are nonsensical combinations of words from the context that provided for the question. While the likelihood of these sequences still seems somewhat predictive of the model's accuracy, semantic clustering becomes very difficult and an unreliable signal for uncertainty estimation. At this temperature, the accuracy of the semantic equivalence methods is only 61 %, whereas it over 92% at lower temperatures (see Appendix B.2
entropy at high temperatures. To that end, we manually inspect and label 100
Note, that cow-temperatures, where does gets plausible and well-formed model generations, semantic entropy does clearly outperform the baselines. This finding further underlines the im- portance of choosing appropriate sampling hyper-parameters when using entropy-based uncertainty measures in NL.G.
15

---

Published as a conference paper at ICLR 2023
Table Multinomial beam search sampling produces sampled answers that are less diverse and thus less useful for uncertainty estimation than multinomial sampling.
Sampling method
Semantic Entropy AUROC
Diversity of answers
Multinomial sampling
0.758
0.490
Multinomial beam search sampling
0.735
0.258
B EXPERIMENTAL. DETAILS AND ABLATIONS
We use both the OPT model and the Deberta-large mode f via the HuggingFace transformers library which can be easily adopted for reproducibility. All of our code is open-source and relies on no proprictary models.
We use the following functions of the HuggingFace API to sample the most likely answers, and the of answers:
To obtain the answer which is compared to the reference answer, which determines whether the question is correctly answered, we use beam search using the generate () function with num.beams
5 and do-sample
True.
To obtain the answer set for uncertainty estimation, by default we use multinomial sam-
pling, that is generate () using do do_sample True and num_beams 1 indicated explicitly, we use beam multinomial sampling, that is generate () using
num_beams and do.sample True.
We all of experiments 80GB NVIDIA A100s.
Testing to 20 samples per answer on the 2.7B, 6.7B and 13B CoQA experiments, we conclude that using more than 10 samples does not significantly improve the performance of the uncertainty measure, we use 10 sampled answers per question in the remaining experiments on TriviaQA. Note, that in Table we compare the 30B model on CoQA and TriviaQA where both settings we use answer sets of size 10.
We use the following prompts on CoQA and TriviaQA. We find that on CoQA, we obtain accurate model results with zero-shot prompting. While we have to use few-shot prompting to obtain accurate answers on closed-book TriviaQA. We use the following prompts for each of the settings:
CoQA:
[The provided context paragraph] [additional question-answer pairs] Q: (Provided question]
A:
where additional question-arsee answer pairs are preceding turns of the conversation about the paragraph consisting of questions and reference answers.
TriviaQA:
For TriviaQA, use a 10-shot prompt of the format:
Q: Which Oscar-nominatee film had You Sexy Thing as its theme song? A: The Full Monty Q: Which Joan's career revived in Whatever Happened to Baby Jane? A: Crawford Q: Which much-loved actor won the Best Actor Oscar for The Philadelphia Story? A: James Stewart (...) Q: In which river is the Boulder Dam? A:
To account for generations where the model continues the .:...:: pattern after providing an answer to the given question, we trim all generations by pattern matching for a selection of stop- words that observe in the generations: Q: Questions QUESTION: and quest ons::
questions:.
hhttps//huuggingffac.c. htt.:/gggg...gta.oo//////e/mmmmn doc/opt
https//ttngiigaac..oooo/ssssaass coldaosstrannffomrrr//mdad doc/opt
16

---

Published as a conference paper at ICLR 2023
Table Automatic evaluation of question answering highly accurate compared to human evaluation. We evaluate how accurate the automatic evaluation metric. The predictions, in this settings are the automatically determined accuracy labels on our question answering task, and the ground truth are human labels for the accuracy of the provided model generation given the reference answer
Data set CoQA TriviaQA
Accuracy of automatic evaluation
0.89
0.96
Table 6: TriviaQA: the exact choice of accuracy metric for the free-form QA task has little effeet on the assessment of the quality the uncertainty measure.
Metric
AUROC
Accuracy
Semantic entropy
Normalised entropy
Rouge-L(y,))) y') > 0.3
0.828
0.802
0.506
Pouge-L(y,.0 0.5
0,835
0.810
0,456
Rouge- 1y.,8/) > 0.5
0.835
0.810
0.457
Exact matching
0.828
0.808
0.394
B.1 RELIABILITY OF ACCURACY METRIC AS COMPARED TO HUMAN EVALUATION
In experiments, evaluate how well our uncertainty measures predict the model's accuracy when answering a given question. The choice of accuracy metric is thus a crucial component of experimental setup. Generally, it has been shown to be difficult to develop automatic metrics for free-form generation that correlate well with human evaluations. We thus verify our choice of accuracy criterion: Rouge-L(y, y') > 0.3, for given reference answer y and model generation y'. We manually evaluate the accuracy of 200 answers of the 30B parameter model on both COQA and on TriviaQA, and evaluate how closely the human evaluation matches the automatic evaluation. We find that on both data sets, the accuracy of the automatic labels as compared to the human labels as ground truth is high, see T
B.2 TESTING THE BI-DIRECTIONAL ENTAILMENT CLASSIFIER
To the best of our knowledge, this paper is the first application of the bi-directional entailment approach to identifying answers with the same meaning in question answering. Since this a core component of approach, we verify how accurately this approach identifies model answers with the same meaning. To this end, we manually label 300 samples for each of TriviaQA and CoQA produced by the 13B parameter model to provide ground truth as whether or they mean the same thing, We find that the model achieves an accuracy of 92.7% and 95.3% respectively.
B.3 SENSITIVITY OF RESULTS TO ACCURACY METRIC
In principle, the choice metric to decide whether or not an answer is 'correct' might have large effect on the assessment of our method and baselines. However, we find empirically that our results are relatively insensitive to the choice of accuracy metric.
In Table 6 we show that for TriviaQA the choice of accuracy metric for the question answering has almost no effect on the measured AUROC of the uncertainty estimation, despite making the measured accuracy of the model's generation significantly different. In particular, the exact matching requirement reduces the accuracy significantly but has little effect on the AUROCs.
For CoQA, which is open-bocde book QA task with greater answer variability and longer answers the results are broadly similar (see Table 7 except for the exact matching accuracy criterion which is too demanding because of the much larger variety of possible answers for this task.
17

---

Published as a conference paper at ICLR 2023
Table 7: CoQA: the exact choice of the accuracy metric for the free-form open-book QA task has little effect on the assessment of the quality of the uncertainty measure except for the use of exact matching. For CoQA, getting an exact match is significantly harder.
Metric
AUROC
Accuracy
Semantic entropy
Normalised entropy
Kouge-L(yyyV > 0.3
0.7672
0.7533
0.8239
Rouge-L(y,y)) 0.5
0.7379
0.7290
0.7657
Rouge-1 (y, 0.3
0.7672
0.7533
0.8239
Rouge- (y. (v.3/) >> 0.5
0.7397
0.7309
0.7677
Exact matching
0.6749
0.6727
0.6459
0.8
0.80
0.75
0.4
0.70
Accuracy on QA task
Accuracy on QA task
AUROC of Semantic entropy
AUROC of Semantic entropy
0.2
1
78
138
308
136
308
6.7
Number parameters of model
Number parameters model
(a) CoQA
(b) TriviaQA
Figure 5: Accuracy improves with model size, does semantic entropy's uncertainty performance. At the smallest model size, both accuracy and uncertainty diminish.
B.4 ACCURACY ABLATIONS WITH MODEL SIZE
We confirm that increasing the model size improves the accuracy of the generations on both QA datasets (see Fig. 5a and Fig. 5b). Semantic entropy's uncertainty performance is also shown for context.
B.5 EXAMPLE P(TRUE) FORMAT
The format of the prompt, reproduced here for convenient reference from the original source Kada- vath et al. (2022). is:
Question: Who was the third president of the United States?
Here are some brainstormed ideas: James Monroe
Thomas Jefferson John Adams
Thomas Jefferson
George Washington
Possible Answer: James Monroe
the possible answer:
(A) True
(B) False
The possible answer is:
where the "brainstormed answers" are from the set of sampled answers A and P(True), iec. the likelihood of the next token being True is taken the uncertainty measure. The authors note that doing the above needs to be done in a few-shot manner and does not work well as in a zero-shot format. In experiments,
few-shot prompt with 10 examples.
18

---

Published as a conference paper at ICLR 2023
Semantic entropy
Predictive entropy
Normalised entropy
p(True)
Lexical similarity
Margin probability
0.80
AUROC
0.70
AUROC
0.70
0.60
0.60
2.78
6.7B
13B
30B
2.7B
6.7B
13B
30B
Number of parameters of model
Number of parameters of model
CoQA
(b) TriviaQA
Figure 6: The margin probability, i.e. the difference between the likelihood of the most likely answer and the likelihood of the second most likely answer, not very predictive of models' accuracy on CoQA open- open-book question answering (a) nor on TriviaQA (b). Identical to Fig. 2 with the addition of Margin probability which was previously omitted to avoid stretching the scale.
B.6
BASELINE
We additionally compare our method to the margin probability method used for neural-symbolic parsing in Lin et al. (002b))
Hmargin (ply x D)) = =p (2(" x. D) - (2) x D)
(3(()))
where y() is the top-1 beam search result and (2) is the top-2 beam search result.
Initially, running the method as proposed [in et al. (2022b) using 13B parameter model on CoQA, we find that Hmargin is not very predictive of the model's accuracy answering questions CoQA achieving an AUROC of 0.54.
We hypothesise that two factors contribute to this poor performance. First, since this measure only looks at the difference of likelihoods, the information about the magnitude of the likelihood of given answer is lost. Sccond-analogoussyy to the predictive entropy would be important to take semantic uncertainty into account when computing Harrinn Manually inspecting model answers on CoQA, and the corresponding Haarginn alent answers and two semantically distinct answers is often similar. That is, this measure does not distinguish between uncertainty between paraphrases of the same meaning (in which case the model might actually be confident about meaning of the answer), and the model's uncertainty about which semantically distinct meaning is correct.
that the margin between two semantically equiv-
We find that if instead of obtaining (2) and (2) by multinomial sampling (as in our other periments) instead of by beam search, this second problem becomes less pronounced and Hmarrin performs better while still being clearly outperformed by the other methods study. We report our full results in Fig. 6
Table 8: Example of challenges for Hmargin- Hmargin does not distinguish between lexical and semantic uncertainty and thus can not distinguish cases where the model is certain about the cor- rect answer (but uncertain about the precise formulation) as in row 1, and cases where the model uncertain about the correct answer as in row The semantic entropy correctly indicates low uncertainty in the first case and high uncertainty in the second case.
(1)
y
H Hmargin
Semantic entropy
Thomas Edison.
Edison.
0.90
0.10
Thomas.
George.
0.36
0.87
19


Reviews:
[{"review_id": "TtgscDlKqJ4", "reviewer": "Reviewer_hFPa", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an approach for uncertainty estimation in free-form text generation called \"semantic entropy\". The method proposed in this paper is unsupervised and can directly be applied to off-the-shelf language models. Semantic entropy provides a better uncertainty estimation than standard entropy, scales better with number of samples and performs better for model self-evaluation. This paper also provides an in depth explanation of uncertainty in free-form text generation. The method proposed first clusters sequences that mean the same thing based on bi-directional NLI. Then the average likelihoods is used to estimate uncertainty over different meanings. ", "summary_of_the_review": "The paper proposes a novel method to perform uncertainty estimation in model generated free-form text by introducing a concept of semantic entropy. The paper is very well written with clear justification for design choices and adequately supported claims and has only minor questions and clarifications.", "strength_and_weaknesses": "Strengths:\n* This paper is very well written and has very clearly defined motivation.\n* The claims made in this paper are well supported with experiments and analysis.\n* The design choices have been properly justified. \n* The approach is \"decently obvious\" and simple and I'm surprised it has not been done before.\n\nWeaknesses and Questions:\n* The last line of section 2 should have a proper citation (\"While promising, these approaches need task-specific labels, additional training, and \"seem to be unreliable out-of-distribution\"). I don't think this work performed experiments to verify / claim this.\n* The theory that semantic equivalence can be operationalized using bi-directional entailment should be grounded in linguistic theory. Please add more details about relevant linguistic work that talks about this to motivate your usage of bi-directional entailment to mean semantic equivalence more.\n* In section 4.2 it is mentioned that \"A sequence is part of a semantic equivalence class if it shows bidirectional equivalence with any other member of that class\". Q1) Why any other? Why not a more stricter definition of all or more than half matched? However, in the discussion on computation cost just below and the pseudocode, it seems that only one sequence is used to judge if the given sequence lies in the same semantic class. Q2) Wont you need to compare with all? If you are testing if any one of them matches? Can you clarify this a bit?\n* 3rd last sentence in \"Computation cost\" section under 4.2 is incomplete. Consequently, what is \"this\" in the last line (last word in 4.2)?\n* Why do you use multinomial sampling? Did you try any other approach to sample?\n* It is claimed in 4.4 that this method \"goes some way towards addressing unequal token importance\". Are there any experiments to support that (I might have missed)?\n* Any insight into why (Figure 4) normalized entropy performs better than the proposed semantic entropy approach at high temperatures?\n\nGrammar and Other suggestions:\n* Be consistent in your usage of UK vs US English. Some instances in the paper have \"summarization\" while others have \"summarisation\".\n* Please add links in references of tables and figures. Capitalize \"Table\". etc.\n* Table 1 caption should be at the top of the table.\n* Para 3 of 4.2 - 3rd line - Use citep instead of citet.\n* Figure 2 legend \"P(True) -> p(True)\"\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HjZoJbpV3G", "reviewer": "Reviewer_wSB5", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a method to calculate uncertainty of language models over generated text that is conditioned on some input context. They propose to estimate \"semantic entropy\" i.e., entropy over clusters of semantically close groups of sampled text. The motivation behind this approach is that entropy-based metrics of uncertainty for natural language generation tasks typically consider uncertainty over tokens/token sequences without taking semantic equivalence of these into account. The authors propose to use bidirectional entailment as a way to confirm whether two sampled sentences are semantically equivalent and use this information to generate classes of semantically equivalent samples. They then calculate the entropy over these samples. They apply this method on two question answering datasets and demonstrate competitive performance compared to other methods.", "summary_of_the_review": "Well motivated and interesting work, that requires a bit more thorough analysis of related work. I still have some questions regarding applicability to other tasks where longer/more complex sentences are expected.", "strength_and_weaknesses": "The motivation for this approach is clear and well established, and the authors show that the semantic entropy can help achieve better uncertainty estimates for question answering datasets. The perform ablation tests regarding the sample size and language model size and justify well their modelling choices for the problem at hand. I enjoyed reading the paper however, I am missing a few things:\n\n1. A better discussion of related work. The authors seem to disregard a considerable amount of work on uncertainty, unless there is a reason the focus was moved away from this work I would expect some more discussion and comparisons. As it is it is a bit harder to fully appreciate the impact of this method and place it in the space of uncertainty work. See my more detailed comment about related work at the end.\n2. Some more discussion and ablation tests on the impact of generated sentence length on the computational cost and performance of the method would be nice. The datasets chosen require short sentences or single word answers, but the authors mention summarisation as a potential next step but it is unclear how the proposed semantic equivalence solution as a metric of semantic relatedness can be applied to text spans that are more complex than subject-verb—object sentences. The manual evaluation of the entailment identification method showed good performance for the datasets in question but it would be nice to see how efficient it is on longer/more complex text. As it is, I am worried that the entailment method would be significantly less accurate if applied in different scenarios.\n\nOn related work and comparisons with other work:\nThe mention of work in the area of uncertainty for regression is a bit outdated, especially in terms of mentioning work on \"regression in low-dimensional data spaces\". There are more recent works that address high-dimensional regression problems (focussing on text) such as: \n-Glushkova, T., Zerva, C., Rei, R., & Martins, A. F. (2021). Uncertainty-aware machine translation evaluation. arXiv preprint arXiv:2109.06352.\n-Wang, Y., Beck, D., Baldwin, T., & Verspoor, K. (2022). Uncertainty estimation and reduction of pre-trained models for text regression. Transactions of the Association for Computational Linguistics, 10, 680-696.\n-Malinin, A., Chervontsev, S., Provilkov, I., & Gales, M. (2020). Regression prior networks. arXiv preprint arXiv:2006.11590\n\nIt would be important to see how the proposed method compares to these approaches instead.\n\nAdditionally, work in direct epistemic uncertainty prediction and perhaps older work in evidential deep learning should also be discussed: \n- Jain, Moksh, Salem Lahlou, Hadi Nekoei, Victor Butoi, Paul Bertin, Jarrid Rector-Brooks, Maksym Korablyov, and Yoshua Bengio. \"Deup: Direct epistemic uncertainty prediction.\" arXiv preprint arXiv:2102.08501 (2021).\n\nOn another note, I am wondering if the authors considered the lexical variability estimate that has been proposed as an uncertainty “glass-box” feature for machine translation models. The implementation is different, but I think the underlying principle is similar: sample a few sentences conditioning on the same context (in MT this is the source sentence, and they use MC dropout to sample different sentences) and then estimate the semantic diversity. The way the authors of the reviewed paper estimate semantic diversity and conceptualise eating is better established for the problem at hand, but I think it is worth comparing the two approaches and I am wondering whether the simpler approach of calculating similarity between sampled sentences yields good results as well.\nReference: Fomicheva, M., Sun, S., Yankovskaya, L., Blain, F., Guzmán, F., Fishel, M., ... & Specia, L. (2020). Unsupervised Quality Estimation for Neural Machine Translation. Transactions of the Association for Computational Linguistics, 8, 539-555.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HhneJuAbVn", "reviewer": "Reviewer_E8hD", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper is trying to produce a better measure of entropy for NLG tasks where sentences with same semantics can have really high entropy in the conventional methods as they rely on superficial measures from the exact token or sequence of tokens. The same sentence can be paraphrased in many ways and hence be equivalent to the \"target\". \n\nThe paper proposes a model based entropy measurement where output with same semantics would have low entropy. And this technique doesn't need any retraining.\n\nThe crux of the idea is \n1. generate all the sequences based on the distribution p(s/x) ; s- sequence, x- input\n2. After N sequences are computed they are clustered using an off the shelf Langauge models to produce NLI classes. Using that measure we can now cluster the sequences into C classes\n3. Compute entropy SE(x) across all classes, where each class's likelihood is Sum(p(s_i/x)) Such that s_i belongs to c\n\n\nThey use AUROC to measure the performance of the uncertainty measurement scheme. ", "summary_of_the_review": "The paper proposes a well grounded idea to compute uncertainty in the space of NLG where the modality (natural language) of the predicted sequence allows more than one way of representing the right meaning. In language there are many ways to convey the same semantics and the NLG models would tend to distribute their output across sequences where semantics match. \n\nSo in this paper the authors propose to use entropy at the level of semantic-class rather individual queries as a measure of uncertainty. The method proves itself to be better that others. \n\nThere are some concerns around the scalability / compute this would need. And concerns around how this measure depends on the performance of the NLI model itself. And the experiments were run on just 2 datasets restricted to QA. \n\nNonetheless the technique seems to produce good results and paves way for future work that could address these issues.", "strength_and_weaknesses": "Strengths\n- The high level idea is reasonable and beats other methods on 2 benchmark datasets\n- With increase in number of samples the semantic entropy method seems to provide better auroc\n- Sampling of sequence is a critical component and the paper addresses those concerns in detail.\n\nWeakness\n- The performance is shown only on two datasets. \n- The author claim it is not expensive in the context of 1.5B / 30B param models. But uncertainty prediction task which adjacent to the main task of NLG, can be quite expensive if we consider smaller, production ready nlg models.\n- The performance of the uncertainty prediction depends on the performance on the NLI task. And given that the clusters are formed using equivalence measure using an imperfect NLI model it would lead to clusters that are much larger indicating ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1Qy4boargJh", "reviewer": "Reviewer_tFub", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes semantic uncertainty, a new uncertainty estimation metric that operates on the meaning space of natural language sentences. Semantic uncertainty accounts for the invariance of the meaning of sentences against surface syntactic or linguistic styles. Formally, semantic uncertainty of an input context is defined as the conditional entropy of the distribution over clusters of meanings conditioned on the context, and can be estimated using a model’s samples. To cluster samples into meaning groups, a bidirectional entailment model is used. Experiments are conducted on two datasets, CoQA and TrivialQA, which shows that the proposed uncertainty estimator has better correlation with the model error rate compared to some existing approaches.\n", "summary_of_the_review": "Interesting idea on measuring semantic uncertainty.", "strength_and_weaknesses": "Strength:\n* Uncertainty estimation for generation results from large language models is an important research avenue towards improving the trustworthiness of those models.\n* The motivation of measuring NLG outputs in the meaning space is quite reasonable\nExperimental results showed that the proposed semantic uncertainty metric outperforms other uncertain estimation methods that operate on the surface form space of sentences instead of the meaning space, although I am not sure how to interpret the gain versus baselines using the AUC metric.\n\nWeaknesses:\n* **[Significance of Results]** Judging from AUC numbers on Figure 2 it is difficult to intuitively understand how much the proposed method is improved over the baseline, especially given the fact that results from the second-best methods on Figure 2 (a) and (b) are quite close to the proposed method. The authors also stated in Table 2 that a simple approach of counting the number of semantically distinct clusters performs quite competitively already in terms of AUROC. Does this suggest that AUROC is not a good evaluation metric since it lacks enough “resolution” to distinguish between different methods? Also, why isn’t the counting method included in Figure 2?\n\nCould the authors consider more direct approaches to evaluate uncertainty metrics? For example, showing end-to-end answer accuracy of a decision tree: answer is from OPT if the model is certain on the input $x$ ($SE(x)$ is low), otherwise, use an oracle answer.\n\n* **[Correctness Metric]** From Section 6, the authors used $1_{rougeL(s, s’) > 0.3}$ (typo in the original equation that uses $<0.3$?) as the metric that evaluates if a prediction \n$s’$ is correct under a reference $s$. Rouge-L is a very brittle metric and doesn’t perform well when measuring semantic equivalence of short phrases. Also, the threshold of $0.3$ is chosen without further justification. Does Rouge-L and this particular threshold correlates well with human judgment of answer correctness? \n\n* **[Models for Sentence Clustering]** The authors used bidirectional entailment as the metric to determine semantic equivalence between pairs of sentences. This task, to my knowledge, is often referred to as paraphrase identification in NLP. There is no mention of this relevant topic in section 4.2, and SoTA models for paraphrase identification literature are not used for sentence clustering in this work.\n\n* **[Related Work]** Could the authors comment on the relationship of their work with existing research on improving generations of neural sequence-to-sequence models based on models’ uncertainty estimation (e.g., Lin et al., 2022b)? This paper is not cited.  In addition, the proposed uncertainty estimation metric in Lin et al., 2022b is not used as baseline in this paper. \n\n`Lin et al., 2022b`: Lin, Zi, Jeremiah Zhe Liu, and Jingbo Shang. \"Towards Collaborative Neural-Symbolic Graph Semantic Parsing via Uncertainty.\" In Findings of the Association for Computational Linguistics: ACL 2022, pp. 4160-4173. 2022.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%