Selectively Answering Ambiguous Questions
Authors: Jeremy R. Cole
Michael JQ Zhang
Daniel Gillick
Julian Martin Eisenschlos
Bhuwan Dhingra
Jacob Eisenstein
Venue: EMNLP 2023 Main (2023)
Keywords: question answering
calibration
ambiguity
TL;DR: We consider multiple sources of uncertainty in question answering models and find repeated sampling is a useful tool for calibration.

Abstract: Trustworthy language models should abstain from answering questions when they do not know the answer. 
However, the answer to a question can be unknown for a variety of reasons. 
Prior research has focused on the case in which the question is clear and the answer is unambiguous but possibly unknown.
However, the answer to a question can also be unclear due to uncertainty of the questioner’s intent or context.
We investigate question answering from this perspective, focusing on answering a subset of questions with a high degree of accuracy, from a set of questions in which many are inherently ambiguous.
In this setting, we find that the most reliable approach to calibration involves quantifying repetition within a set of sampled model outputs, rather than the model's likelihood or self-verification as used in prior work. 
We find this to be the case across different types of uncertainty, varying model scales and both with or without instruction tuning.
Our results suggest that sampling-based confidence scores help calibrate answers to relatively unambiguous questions, with more dramatic improvements on ambiguous questions.

Selectively Answering Ambiguous Questions
Jeremy R. Cole and Michael JQ Zhang and Daniel Gillick'
Julian Martin Eisenschlos'
and Bhuwan Dhingra and Jacob Eisenstein'
1. Google DeepMind
2. University of Texas
3. Duke University
Abstract
We describe a range of experiments to help dis- entangle uncertainty about the world from uncer- tainty about the question. While we focus on the question answering capabilities of large pretrained language models, these two forms of uncertainty can be more clearly distinguished in an idealized scenario in which a symbolic question-answering system first converts the question into formal denotation (such an SQL query or lambda ex- pression), which then applied to knowledge base (e.g.. Zelle and Mooney, 1996). The system may be uncertain about the meaning of the ques- tion - demotational uncertainty - if the question is underspecified because of assumed background knowledge or context. For any given denotation, the system may also be uncertain about the correct answer epistemic uncertainty if the knowl- edge base is incomplete'. In Figure 1, denotational uncertainty is shown the upper fork and epis- temic uncertainty is shown in the lower set of forks.
Trustworthy language models should abstain from answering questions when they not know the answer. However, the answer to question can be unknown for a variety of rea- sons. Prior research has focused the in which the question is clear and the answer is unambiguous but possibly unknown. But the answer to a question can also be unclear due to uncertainty of the questioner's intent or con- text. We investigate question answering from this perspective, focusing answering a sub- set of questions with a high degree of accu- racy, from a of questions in which many are inherently ambiguous. In this setting, we find that the most reliable approach to decide when abstain involves quantifying repetition within sampled model outputs, rather than the model's likelihood or self-verification as used in prior work. We find this to be the case across different types of uncertainty and model scales, and with or without instruction tuning. Our re- sults suggest that sampling-based confidence scores help calibrate answers to relatively un- ambiguous questions, with more dramatic im- provements on ambiguous questions.
The most competitive systems today do not con- struct explicit denotations of questions. As a re- sult, it can be difficult to clearly separate these two forms of uncertainty. Nonetheless, approach is attempt to first approximately dis- ambiguate the question (in natural language) and then selectively answer user questions with suffi- ciently high confidence. This allows the model to more effectively represent ambiguous user in- puts. This scheme is represented in Figure Fur- ther, we argue that repeated sampling within our disambiguate caaambiiguattthee-aaswwee thnnnnsser framework provides liable confidence estimates of the model. We summarize our contributions as follows:
general
1 Introduction
Any practical Question Answering (QA) system must be able to assess its own confidence so that it (1) avoid making up incorrect, incomplete, or misleading answers, and (2) request clarification to resolve ambiguity arising from unclear questions or missing context. recognition of this desirable property, recent work has addressed confidence esti- mation of Large Language Models (LLMs) by eval- uating QA tasks in unambiguous settings (Jiang et al., 2021; Kadavath et al., 2022). In these ex- periments, questions are posed to the model with multiple-choice answers the probability of each answer can be computed. Here we extend the study of confidence to practical scenarios, requiring free- form text answers to arbitrary questions which may be underspecified or ambiguous.
We reframe the discussion of model confi- dence with respect to denotational and epis- temic uncertainty and address each in turn.
'Gruber et al. (2023) present an overview of uncertainty in Machine Learning, making a related distinction between epis- temic and aleatoric uncertainty. We do not address alcatoric uncertainty in this work.

---

User
Who the gold medal Olympics fencing?
Uncerisinty pleatrotttons Cenotational uncertuiity
0.5
0.3
Who won gold medal the 2020 Olympics in the men's foil?
Who won the gold medal in the 2020
Who won the gold medal in the 2016 Olympics fencing?
Olympics in men's
0.6
0.4
0.6
0.3
0.3
0.4
Cheung Ka Long
>Unknown>
Áron Szilägyi
<Unknown>
Emese Szász
Inna Deriglazova
<Unknown>
Unceriality over answers given interortation Epistemic uncartanty
Madill output
Interpretation: Who won the gold medal in the 2020 Olympics in the men's foil? Cheung Long
Figure 1: Uncertainty in Question Answering systems may arise in various ways. We propose a scheme called disambiguate then answer where the model first attempts pose an unambiguous interpretation of the user question (yellow), then selectively produces answer to this question, alternatively abstaining (" ("Unknown") (green). The log likelihood of the model shown above each generation. In addition, we find that sampling multiple times from the model generally allows for more robust confidence estimates.
Our experiments suggest that the answer prob- abilities given by current LLMs are somewhat helpful for assessing epistemic uncertainty but cannot detect denotational uncertainty.
with 2 representing abstention. Given a probabilis- tic model P(a q). a natural choice for the confi- dence score s(g, a) is the conditional probability of the answer. This and other scoring functions are discussed in Section 3.
2.
We present two simple and effective ap- proaches for measuring confidence under all types of uncertainty, based answer counts within repeated samples from the LLM. These approaches are better calibrated than the model's answer probabilities and a self-verification prompt, and are especially effective for ambiguous questions and for instruction-tuned model variants.
For information-seeking queries such as those found in Natural Questions (Kwiatkowski et al., 2019), the askers have no answer in mind, which makes it more likely that the questions are acci- dentally ambiguous. The task of answering ambiguous question requires solving at least two subtasks: determining the meaning of the question (its denotation), and then identifying the answer. This pipeline is shown in Figure tic model, this can be represented mathematically as P(a 9) - P(a Pdd q). with d indicating the denotation. While prior work has investigated the calibration of answer probabilities given by pretrained language models (e.g., Kada- vath al., 2022), it is not clear whether the rela- tively positive findings of these studies extend questions with significant denotational ambiguity, as represented by datasets such as AmbigQA (Min et al., 2020) and SituatedQA (Zhang and Choi, 2021). We address this question in Section 6.4.
probabilis-
2 Calibration for Question Answering
A predictor said to be well-calibrated if its pre- dictions are accompanied by confidence scores and those scores are informative of the likelihood that the prediction is correct. Such confidence scores can be used to support applications such selec- tive prediction, in which the model can abstain from predicting when its confidence low (Chow, 1957; EI- Yaniv et al., 2010). In selective question answering (e.g., Kamath et al. 2020; Zhang et al. 2021b), we assign each question question-answer answer pair (q, a) confidence score s(g, a). The system's output is then parameterized by threshold
3 Confidence Scores
Before proposing confidence scores for question answering. it will be helpful to review how these answers are generated in practical settings. A lan- guage model over finite vocabulary Σ defines a probability distribution X over sequences of to-
arg max, s(g, max, s(g. >T Ø
1r (g)
else,
(1)

---

kens in Σr, by modeling the marginal probability of sequence p(o1.-.. .n) using the conditional probabilities of individual tokens via the chain rule [[=pp(oa/o--1-- , on). For QA, answers are obtained by conditioning on the question and first sampling from X (or a version of X adjusted with modifications such temperature) and then applying normalization or extraction process to it in order to map it answer space A. This function f Σn A can be as simple as punc- tuation removal and lower-casing, or as complex as code exccution or extracting the answer from a chain-of -f-thught (Wei et al., 2022).
many possible IQVs, we have chosen two fairly different approaches for our experiments. each case, we generate 10 sampled outputs (at tempera- ture of 0.5) and use exact match (after lowercasing and removing puncutation) for comparison among outputs. Naturally, there are many other ways to compare outputs, including token-level overlap or an answer-span equivalence model, but we leave these alternatives for future work. See also Kuhn et al. (2023) and Lin et al. (2023) for concurrent work investigating alternative IQVs.
Sampling Repetition Our first approach is based on Wilcox's classic Variation Ratio which mea- sures deviation from the mode. Instead, we simply compute the fraction of times that the sampled out- puts match the greedy output (Temperature = 0.0), The more samples match the greedy output, the more confident we are in the answer. We count the number of samples which exactly match the greedy answer. We experimented briefly with more sophisticated approaches based the BERT based answer equivalence model from Bulian et al. (2022) as well as the summed Fl score across sampled an- swers and the greedy answers, but these approaches did not meaningfully improve the calibration.
Using this background, we can describe various methods for estimating the scores of predictions from language model (Figure 2).
Likelihood The most basic method we can use is likelihood-based calibration by computing p(alg) IT= p( .n, Q) for se- quence sampled from X. This may be used to rank the answers from most confident to least con- fident based on the model's produced likelihood of that answer. However, in practical settings this method may not accurately reflect the probability of observing an answer for several reasons:
(1) Language models typically incorporate sev- eral inference-time biases improve the quality of the outputs, such as nucleus sampling (Holtz- man et al., 2020), top-k sampling (Fan et al., 2018), length penalties, length truncation, and temperature. These techniques affect the output distribution in ways that are hard or impossible to capture in the likelihood score (Zhang et al., 2021a).
Sampling Diversity Our second approach is based on the diversity of the samples, computed by num uom_uuique Here, our confidence is inversely numn samples' proportional to the number of distinct samples and is estimated as zero if all samples are different. Note that while this works relatively well in prac- tice, it is heavily dependent on the number of sam- ples, we do not expect the number of unique answers scale linearly with the number of sam- ples.
(2) Even for decoders that do not incorporate inference-time biases, the likelihood function as de- fined by the auto-regressive decomposition might leak probability mass infinite sequences (see Meister et al., 2023, Proposition 2.4). This implies that the model that we sample from might not be a valid probability distribution.
We note that some recent work (Meister et al., 2021; Arora et al., 2022) has also described esti- mating statistics of random variables induced by language models. This involved sampling without replacement and using importance weighting, but they seem to surpass Monte-Carlo estimates only with many samples and peaky distributions. We leave further exploration of these methods on our evaluations as well as other IQVs for future work.
(3) Finally, the likelihood fails to account for the extraction step f. which requires an intractable marginalization step over the model outputs.
In view of these limitations, it is preferable to use sampling to estimate properties of the resulting marginal distribution. Formally, we can use an In- dex of Qualitative Variation (IQV) (Wilcox, 1973) to measure the statistical dispersion of discrete sam- ples produced by the model. Practically, there are a
Self- Verification Finally, verification, in which the language model assesses its own confidence (Kadavath al., 2022). This is a two-step process, where first the model is prompted to provide a list of possible answers given a ques- tion. Then, this list is fed back into the model,
consider self-

---

Prompts
Question
Samples
Answer 1: Theresa May
Quastiin 1: Who is the prim miniier the UK 20167
Who the director of Scarface?
Sprwwdy)
Brian de Palma
P2 Howard Hawkss
Question 2 What family cces pans beiong to romeo jullet?
Answor 2: Capulit
LM
Pr Brian de Palma
Question N: (question) Answor
P, Brian de Palma
Unembiguous Questions
P; Brian de Palma
Qowalian Who in the prins minidler of angland 2018? Diaambguation None
is this (question) (anowed correc?
Queatton Who Sansa manyy Thrones? Airambiguation 3 Game Thrones? lynonn Lannister
mary
LM
Likelihood: p
Sampling repetition: 4/5
Pross
Sampling diversity: 1 2/5
Questoon (question) Atammbguatton
Self-verification: PsuM
Anbiguous Questions
Figure 2: Methods for estimating the confidence of answers from an LM. Sampling repetition counts the number times the greedy answer appears among samples from the LM. Sampling diversity counts the number of unique answers among samples from the LMM Self-verification, proposed by Kadavath et al. (2022), prompts the LM again with one of the sampled answers to measure the token-level probability of True. The prompts used for unambiguous and ambiguous questions are shown on the left-for the latter, we additionally prompt model for disambiguations (omitted in the outputs shown on the right for brevity).
where the task to determine if each answer correct or incorrect. In practice, we re-use the sam- pled outputs from the previous run as the list of possible answers, and the greedy answer as the pro- posed answer. Then, the model scores the token "True", which we use to obtain confidence score. In practice, these possible answers come from sam- pling with replacement, similar to the Sampling Repetition and Diversity methods, so the list of possible answers may contain duplicates. While these duplicates may affect the usefulness of the brainstorming exercise, the model is not choosing among them; instead, it is only scoring whether the greedy answer is correct, using the list of possible answers context.
These criteria are conflict because any system can achieve zero error rate by always abstaining from answering. When the confidence score is a probability, we can also ask whether this probabil- ity is well-calibrated, in the sense that confidence score of s(q, a) implies is the probability of being the correct answer to q.
Expected Calibration Error (ECE) Predic- tions are grouped into ten equally sized bins, ranked by the evaluated system's assigned confidence scores. We compute the mean absolute distance between the average confidence score and the accu- racy of predictions in each bin, averaging across all bins. If we interpret confidence score to represent probability, this corresponds to the difference in the predicted probability of correctness from the actual probability of correctness.
4 Evaluation setup
Our evaluations test the utility of the confidence scoring methods described in Section 3. We focus mainly on the pretrained language model PaLM, which obtains high accuracy on several question answering datasets using few-shot in-context learn- ing (Chowdhery et al., 2022).
ROC-AUC Area under the receiver operating characteristic curve evaluates the uncertainty es- timate's diagnostic ability as a binary classifier for correct and incorrect predictions by integrat- ing over the tradeoff curve between the rates of true and false positives.
4.1 Metrics
Coverage Acc While ECE and ROC-AUC sess absolute and relative calibration respectively, we want metric closely aligned with practical use case: selective answering above a confidence threshold. To this end, we measure the fraction
There are many metrics for evaluating selective question answering systems, but fundamentally we want systems that (1) frequently return correct answers, and (2) rarely return incorrect answers.

---

of questions the system can answer correctly if needs to maintain certain accuracy. Specifically, C@ Acc the maximum coverage such that the accuracy on the C% of most-confident predictions is at least Acc %. For example, C@ = 20, then the system achieves at least 80% accuracy on its 20% most-confident predictions (and lower than 80% accuracy on its X% most-confident predic- tions for any X 20). With user-facing systems in mind, we an accuracy of 80% (C@80).
for both datasets. We discuss investigation of other prompts Appendix B.
5.3 Calibration Results
Results can be found Table and Figure 3. First, we observe that the verification method used prior work is inferior across nearly all calibration metrics. For unambiguous Natural Questions, de- pending on the metric used, we see that likelihood and sample repetition both work relatively well calibration metries. For TriviaQA, the sample repetition method works notably better: this is potentially due to the presence of ambiguous questions in TriviaQA, which we do not attempt to remove. This hypothesis strengthened by experi- ments in the next section that show that sampling methods tend to improve calibration more in the presence of more ambiguity.
that
5 Unambiguous Questions
To start, we explore the case of unambiguous ques- tions with a single answer. In this scenario, all uncertainty should be of the epistemic form; the denotation of the questions should be clear. Pre- vious work with this scenario has focused on the domain shift setting (Kamath et al., 2020) where the goal is to avoid answering questions from other datasets. Other work has looked at reading com- prehension with relatively small models (Zhang et al., 202 bb; Jiang et al., 2021). Open-domain question answering is substantially more difficult than reading comprehension, which likewise makes calibration more challenging.
O Likelihood
Diversity
0.8
Repetition
Verification
accuracy
0.6
0.4
5.1 Datasets
0.2
We explore this setting using datasets: TriviaQA (Joshi et al., 2017) and NQ- Open (Kwiatkowski et al., 2019; Lee et al., 2019) because they are widely used for few-shot question answering. However, while it might be assumed that each question in these datasets has only one possible interpretation, this is often not the case. In fact, AmbigQA (Min et al., 2020) finds that over 50% of NQ-Open questions are underspecified. As such, we limit ourselves to unambiguous questions as annotated in AmbigQA. We will discuss the extended setting that includes underspecified queries in Section 6.
0
0
0.2
0.4
0.6
0.8
confidence score
Figure Plot of calibration error by comparing buck- eted accuracy to bucketed confidence scores across methods. Plotted on the unambiguous portion of Nat- ural Questions.
Setting
M
EM
ROC-AUC
ECE
C@80
L
0.843
0.141
44.2
D
0.814
0.152
44.8
NQ
51.7
5.2 Experiment Setup
R
0.830
0.103
45.4
0.712
0.242
9.0
Our investigation focuses on few-shot in-context learning with large language models (LLMs).. Our primary prompt is composed of four question and answer pairs from the training sets of Natural Ques- tions and TriviaQA, respectively (see Figure 2). To reduce variance across experiments, the example QA pairs are selected randomly so that each input to the model has different version the prompt. We compute metrics over the entire evaluation set
L
0.826
0.205
87.6
0.829
88.1
TriviaQA
72.5
0.844
0.740
Table I: Non-Ambiguous QA Calibration on Natu- ral Questions and TriviaQA. The column M refers the calibration methods, abbreviated with (L.jikelhhood. (D)iversity, (R)epetition, and (V)riification.

---

6 Ambiguous Questions
additional time ranges and their corresponding an- swers are crowdsourced. The geographic questions are largely created by removing references to loca- tion and then crowdsourcing locations and corre- sponding answers. Note that SituatedQA does not provide metrics assessing the precision or possi- ble recall of the disanbiguations (though the recall would be particularly difficult to measure).
Questions can be ambiguous for a variety of rea- sons. Language is highly context-sensitive and meaning can be lost when context is missing. Of- ten, interlocutors fail to clearly translate their intent into unambiguous language. Alternatively, ambi- guity occurs due to lack of alignment between interlocutors at a higher level of abstraction. Re- gardless of its origin, ambiguous questions give rise to denotational uncertainty, which we can concep- tualize as a latent distribution over interprctations. Realistic scenarios such as search and chat fre- quently contain ambiguous questions, which makes calibration more difficult because uncertainty may be epistemic, denotational, or both. Our approach will again involve sampling: this time over both interpretations and answers. Note that in this case, the approach has some similarity to self- consistency (Wang et al., 2022), as we compute sampling-based confidence scores over the final answers regardless of the exact interpretation.
6.2 Experiment Setup
As above, we use prompts of question-answer pairs, with each example having six exemplars drawn from the training data. For AmbigQA, use only the examples that have been further annotated by Stelmakh al. (2022). For SituatedQA, we eval- uate the Geographic and Temporal sets separately. For each example, three of the exemplars will be ambiguous; in those cases, the prompt will first include an interpretation of the question which is one of the provided disambiguations. In practice, use the first disambiguation for each example.
6.1 Datasets
6.3 Ambiguity Prediction
We evaluate using two datasets of underspecified queries that are derived from NQ-Open: AmbigQA and SituatedQA.
Before trying to assess calibration, we first seek to explore whether our method can predict whether question is ambiguous according to AmbigQA. SituatedQA is not suitable for this task because its questions tend to follow templated patterns where the ambiguity arises from missing time or place reference, while AmbigQA has more diverse ex- pressions of ambiguity. We are not aware of prior work that evaluates ambiguity classification on Am- bigQA or other datasets.
AmbigQA AmbigQA created from a subset of Natural Questions (Kwiatkowski et al., 2019) that raters assess as ambiguous. After a rater deemed a question ambiguous, various unambiguous in- terpretations of the question and their respective answers are written. Thus, AmbigQA measures model's ability disambiiuutte-an--ansva am- biguous questions. For instance, given a query like "Where does the new fallout game take place?", the model should be able to produce the disambiguated question "Where does the fallout 76 game take place?" and answer with "Appalachia". Models are evaluated based on the similarity between the gen- erated question and the closest reference question well as by producing the correct answer for the generated question.
We use a first transformations of Section 3 predict whether the model predicts the question is ambiguous. We discuss this in more detail in Appendix A. In short, none of the methods are particularly capablc, with the best method achiev- ing 58% accuracy when 53% of the questions are labeled as ambiguous.
Why is question ambiguity so difficult to pre- dict? One possible explanation is that all queries are somewhat ambiguous. Queries are typically not intentionally underspecified: the questioner believes they have provided enough information to answer. In conversation, this may fail and require developing common ground; in an infor- mation retrieval system, this may take the form of query refinement. All of the questions in the Nat- ural Questions dataset were asked of the Google search engine and are answerable using Wikipedia
SituatedQA SituatedQA is likewise created from a subset of existing questions and uses rela- tively similar process, but focuses more narrowly on temporal and geographic ambiguity. As the type of ambiguity and the process used create them is different, we generally evaluate them separately. The temporal questions are annotated with the time range that a question's 's answer is valid. Afterward,

---

(Kwiatkowski et al., 2019), yet many are labeled as ambiguous by AmbigQA.
7 Instuction Tuning and Scaling
Pretrained language models are typically fine-tuned before being used in applications. However, fine- tuning and other alignment techniques may disturb the model's calibration by emphasizing high per- formance on specific sets of tasks rather than the full pretraining distribution." 3 Table shows that for Flan-PaLM (Chung et al., 2022), instruction tuning dramatically improves exact match accu- racy while simultaneously worsening performance on manfidence-based ranking (ROC-AUC) and cal- ibration (ECE), when using model likelihood as the confidence score. However, this miscalibration can be mitigated by using sample-based confidence scores, which dramatically improves the calibra- tion on ambiguous questions (AmbigQA). For the gelective-predicttin metric C@ 80, the instruction- tuned model with sampling is far ahead of any other method investigated. Note that we investigate Nat- ural Questions and AmbigQA here as they are not in the instruction tuning training data.
For instance, "Where is the world cup going to be?" might typically refer to the host country, but fans from the host country that year might instead refer to the cities the games will be played. Even bi- nary choice questions like "Who won more games between the Packers and the Bears?" may appear unambiguous, but this question has different an- swers depending on the date it is asked.
In this sense, we propose that ambiguity a function of both the interpreter and the query. How- ever, measuring whether a model (or indeed, any interlocutor) understood a question is challenging. Often in dialogue, interlocutors may assume their counterpart understood if they receive a response similar to what they expected; alternatively, they may ask explicitly if they were understood.
This leads us to our disambiguate-then--nne paradigm: if there is no fundamental difference be- tween ambiguous and unambiguous queries when taken out of the original context, then the model should first establish precisely what question is attempting to answer.
We examine the effect of model scale on cali- bration, finding that in general, accuracy declines substantially in closed book question answering, but calibration stays roughly constant. See Ap- pendix D for full results.
6.4 Calibration Results
As sampling-bass based approaches require a linear increase in compute for the number of samples, we also examine how calibration scales with the number. In particular, we test three, five, and eight samples and compare that to the original results containing ten samples. Results can be found in Ap- pendix E. Unsurprisingly, more samples seems to improve calibration, though it seems the unam- biguous Natural Questions slice, sampling diversity with a small number of samples works relatively well for the cost.
We evaluate calibration on ambiguous questions using the same methods described in Section 5 with results in Table 2 and Figure 4. We exclude the self-verificatio setup here, because its calibra- tion is much worse the unambiguous questions, and it is unclear how to translate the prompt when there are multiple interpretations of the question. Note that we consider the answer to be correct if matches the answer of any of the disambiguations. We also present results for a "strict" matching setup, where we only consider the answer to be correct if it matches the answer of the closest disambigua- tion to the provided interpretation, measured by token overlap.
Related Work
This paper investigates calibration, selective ques- tion answering, and ambiguity within single model. It builds on work across these topics. While AmbigQA (Min et al., 2020), ASQA (Stel- makh et al., 2022) and SituatedQA (Zhang and Choi, 2021) introduce new annotations for ques- tion answering datasets, neither these papers nor
Overall, the ambiguous questions are much more difficult than the unambiguous ones. In general, likelihood seems to become a worse method of measuring model uncertainty when the questions are ambiguous and sample repetition appears im- prove calibration significantly." 2 The striet match- ing setup does not affect the ordering of the results, though numbers are lower overall.
"The GPT-4 technical report (https: /cdn. openai. com/ papers/pp////ee saspers/gpa /gpptgpt//// 4.. .p.f)) shows that calibration is made significantly worse by the combination of instruction-tuningg and reinforeement learning, but does not distinguish hetween these two factors.
"Note that the interpretations sometimes consist of longer sequences, thus confounding the usability of likelihood

---

Loose
Strict
Dataset
M
EM
ROC-AUC
ECE
C@80
EM
ROC-AUC
ECE
C@80
L
0.731
0.316
17.4
0.724
0.287
12.4
AmbigQA
D
44.8
0.767
0.114
16.1
41.9
0.763
0.085
13.0
R
0.821
0.120
26.2
0,812
0.147
20.6
0.757
0,223
0.751
0.157
3.0
SQA-Temp
35.7
0.772
29.1
0.757
0.048
2.7
0.797
0.784
0.167
2.4
L
0.759
0.221
10.4
0.757
0.156
3.3
SQA-Geo 35.6
0.743
0.085
8.9
29.0
0.723
0.056
4.5
0.800
0.120
13.7
0.789
0.176
4.6
Table 2: Ambiguous Calibration. Column refers to the calibration methods, abbreviated with (L)ikelihood. (D)iversity, and (R)epetition. Loose matching counts the answer correct if it matches the answer from any interpretation; strict matching only counts as correct one of the closest interpretations has the same answer.
Dataset
N
ROC-AUC
ECE
C@80
Setting
M
EM
ROC-AUC
ECE
C@80
0.749
0.132
40.8
L
0.833
0.098
70.7
0.791
0.086
44.2
NQ
D
65.8
0.807
0.142
74.7
D
0.806
0.125
43.9
R
0.797
0.137
71.2
10
0.814
0.152
44.8
NQ
0.698
0,356
27.3
0.789
0.155
43.3
AmbigQA
D
57.6
0.718
0.148
28.1
0,811
0.116
45.0
R
0.794
0.137
52.9
0,826
0.099
45.3
10
0,830
0.103
45.4
Table 4: Calibration for instruction tuned Flan-PaLM models. The column labeled M refers the calibration methods, abbreviated with (L)ikelihood, (D)iversity. and (R)epetition
0.650
0.297
12.1
0.719
0.218
12.5
D
0.755
0.148
16.8
10
0.767
0.114
16.1
AmbigQA
0.769
16.7
0.801
0.130
24.3
R
0,815
26.9
any follow-up work investigate how the uncertainty in interpreting a question interacts with uncertainty about the answer.
10
0,821
26.2
Table 3: Calibration by number samples (N), EM is excluded, because is solely based on the greedy answer and does not depend on the number of samples.
8.1 Calibration and Selective QA
Calibration of modern machine learning ap- proaches difficult due to their non-linearity, scale, and architecture details (Guo et al., 2017). Generation-like tasks are even more difficult, con- taining sequence of tokens with individual con- fidence estimates and special end of sentence marker that may be poorly calibrated (Kuleshov and Liang, 2015; Kumar and Sarawagi, 2019; Ja- gannatha and Yu, 2020). common approach for various natural language processing tasks is to train a separate classifier to estimate the model's con- fidence (Kumar and Sarawagi, 2019; Jiang et al., 2021; Zhang et al., 2021b: Kamath et al., 2020; Desai and Durrett, 2020; Dong et al., 2018) using various featurizations of the model's input, output, and likelihoods. Alternatively, Ren al. (2023) use embedding distances and Oshand et al. (2021) use Bayesian -nnpiieed approaches that try to predict the full distribution over labels.
1
0.8
accuracy
0.6
0.4
Likelihand
0.2
Divensiyy
Repetition
0.2
0.4
0.6
0.8
confidence score
Figure 4: Plot of calibration error by comparing buck- eted accuracy to bucketed confidence scores across methods. Plotted the combined version of Am- bigQA, containing ambiguous and unambiguous ques- tions.

---

Within the specific setting of question answer- ing, Kamath al. (2020) and Zhang et al. (2021b) both address a similar selective question answering framework to ours, but they do not explore ambigu- ous questions. Varshney et al. (2022) investigate selective prediction for various tasks, and Varsh- ney and Baral (2022) aim to improve coverage by re-answering questions that the model originally abstained on. Kuhn al. (2023) and Lin et al. (2023) use similar sampling-bast based methods over free-form question answering, using slightly differ- ent formulations of confidence scores, but they do not investigate ambiguous questions. Kuhn et al. (2022) examine synthetically-created ambiguous questions, but focus on multi-turn interactions.
with calibration, noting that additional expressions of uncertainty improve calibration without hurting accuracy. This an interesting direction for future research into its interaction with ambiguity.
Conclusion
We investigate the calibration of large language models, extending prior work by distinguishing uncertainty about the answer (epistemic uncer- tainty) from uncertainty about the meaning of the question (denotational uncertainty). We propose disambiguate- cattmmbiguat--an--nae -nd-anndde paradigm, where the model first attempts rephrase the question be- fore providing its answer. This paradigm enables straightforward techniques for quantifying model confidence by counting the frequency of answers within repeated samples from the language model. These sample-based confidence metries are par- ticularly effective for ambiguous questions, and are significantly better calibrated than traditional likelihood- measures and self-verification. For ambiguous questions, the proposed method is similar to calibration through self-consistency.
Question ambiguity can be formalized as un- certainty about the denotation of the question, re- lating prior work on uncertainty in semantic parsing (Dong et al., 2018). However, rather the expressing candidate denotations in a formal se- mantic representation, we express them informally in natural language, as a waypoint toward quantify- ing the uncertainty of candidate answers.
8.2 Self-Calibrating Language Models
10 Limitations
A number of papers propose to quantify confi- dence and detect hallucinations by (a) directly ask- ing the model to rate its own confidence or cor- rectness (e.g.. Kadavath et al., 2022; Lin et al., 2022), (b) drawing repeated samples (Wang et al., 2022; Manakul et al., 2023), or (c) requesting addi- tional supporting information about proposed out- puts (Agrawal et al., 2023). As discussed in Sec- tion 5 and Appendix C, we found that neither self- verification nor offering the option of "unknown" were effective strategies for selective question swering. Wang et al. (2022) focus accuracy rather than calibration and Manakul et al. (2023) focus on fact-checking long-fom answers.
In this work, we explore selectively answering ambiguous questions and calibrating such mod- els, but we do so only within the context of a sin- gle model: PaLM. We do not explore alternative paradigms, such as supervised fine-tuning, or other large language models for in-context learning with which to replicate our results. Furthermore, we only explore calibration in closed-book setting, despite that open-book question answering (i.c.. augmented with retrieval) generally has higher per- formance and may pose additional calibration chal- lenges. Moreover, our exploration of various con- fidence scores was relatively brief, and while we explored some additional prompts, we did not it- erate heavily on prompt tuning. Lastly, while the proposed method has some advantages, especially for instruction-tuned models on ambiguous ques- tions, it also increases the compute linearly with the number of samples.
8.3 Prompting Strategies
in-context learning. the choice of prompt and the order of exemplars can affect downstream perfor- mance (Lu et al., 2022). Other work suggests that giving more long-fom form answers and explanations in the prompt may encourage the model to do the same, increasing its likelihood of arriving at the cor- rect answer (Wei et al., 2022). In our work, we find the choice of prompt plays little role and that longer answers or explanations do not seem to improve calibration (see Appendix B). Zhou et al. (2023) in- vestigate how adding words of uncertainty interacts
Acknowledgements
We thank William W. Cohen, Urvashi Khandelwal and Clara Meister for their valuable comments and feedback while discussing this work.

---

References
Sources of uncertainty in machine learning-a a statis- ticians' view. arXiv e-prints. pages atXiv-2305.
Ayush Agrawal, Lester Mackey, and Adam Tauman Kalai. 2023, Do language models know when they're rr hallucinating references? arXiv preprint arXiv:2305. 18248.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Wein- berger. 2017. On calibration of modern neural net- works. In International conference on machine learning, pages
PMLR.
Aryaman Arora, Clara Meister, and Ryan Cotterell. 2022. Estimating the entropy of linguistic distribu- tions. In Pruceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Vol- ume 2: Short Papers), pages 175-195, Dublin, Ire- land. Association for Computational Linguistics.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text de- generation. In International Conference on Learn- ing Representations.
Abhyuday Jagannatha and Hong Yu. 2020. Calibrat- ing structured output predictors for natural language processing. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 2078-2092 Online. Association for Computa- tional Linguisties.
Jannis Bulian, Christian Buck, Wojciech Gajewski, Benjamin Börschinger, and Tal Schuster. 2022. Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation. In Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing. pages 291-305, Abu Dhabi, United Arab Emirates. Asso- ciation for Computational Linguistics.
Zhengbao Jiang. Jun Araki, Haibo Ding, and Graham Neubig. 2021. How can we know when language models know? on the calibration of language mod- els for question answering. Trunsactions of the Asso- ciation for Computational Linguistics, 9:962-977.
Chi-Keung Chow. 1957. An optimum character recog- nition system using decision functions. IRE Trans- actions on Electronic Computers, EC-6(4))247-254.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale dis- tantly supervised challenge dataset for reading com- prehension. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601-1611, I, Van- couver, Canada. Association for Computational Lin- guistics.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint 204.2.200..03311.
Hyung Won Chung. Le Hou, Shayne Longpre, Bar- ret Zoph, Yi Tay. William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. -inettunn language mod- els. arXiv preprint arXiv:2210.1 11416.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer. Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221.
2022. Scaling
Shrey Desai and Greg Durrett. 2020. Calibration of pre-trained transformers. In Proceedings of the 2020 Conference on Empirical Methods Natural Lan- guage Processing (EMNLP), pages 295-302, Online.
Amita Kamath, Robin Jia, and Percy Liang. 2020. Se- lective question answering under domain shift. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5684- 5696. Online. Association for Computational Lin- guistics.
Association for Computational Linguistics.
Li Dong, Chris Quirk, and Mirella Lapata. 2018. Confi- dence modeling for neural semantic parsing. In Pro- ceedings of the 56th Annual Meeting of the Associa- tion for Computational Linguistics (Volume I: Long Pupers), pages 743-753, Melbourne, Australia. As- sociation for Computational Linguistics.
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2022. Clam: Selective clarification for ambiguous ques- tions with large language models. arXiv preprint arXiv:2212.07769.
Ran El- -aniy et al. 2010. On the foundations of noise- free selective classification. Journal of Machine Learning Research, 11(5).
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for un- certainty estimation in natural language generation. arXiv preprint arXiv:2302.0966.
Angela Fan, Mike Lewis, and Yann Dauphin. 2018 Hi- erarchical neural story generation. In Proceedings gs of the 56th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 889-898, Melbourne, Australia. Association for Computational Linguistics.
Volodymyr Kuleshov and Percy S Liang, 2015. Cali- brated structured prediction. In Advances in Newral Information Processing Systems, volume 28, Curran Associates, Inc.
Aviral Kumar and Sunita Sarawagi. 2019. Calibration of encoder decoder models for neural machine trans- lation, arXiv preprint arXiv: 00802..
Cornelia Gruber, Patrick Oliver Schenk, Malte Schier- holz, Frauke Kreuter, and Göran Kauermann. 2023.

---

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Al- berti, Danielle Epstein, Illia Polosukhin, Jacob De- vlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Waaee Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question an- swering research. Transactions of the Association for Compulational Linguistics, 7:452 466.
Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mo- hammad Saleh, Balaji Lakshminarayanan, and Pe- ter J Liu. 2023, Out-of-distribution detection and se- lective generation for conditional language models. In The Eleventh International Conference on Learn- ing Representations.
Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming- Wei Chang. 2022. ASQA: Factoid questions meet long-form answers. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 8273-8288, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Kenton Lee, Ming- Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Anmual Meeting of the Association for Com- putational Linguistics, pages 6086 6096, Florence, Italy, Association for Computational Linguistics.
Neeraj Varshney and Chitta Baral. 2022. Model cas- cading: Towards jointly improving efficiency and accuracy of NLP systems. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Procexsing, pages 11007-11021, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words, arXiv preprint arXiv:2205.143: 14334.
Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2023. Generating with confidence: Uncertainty quantifica- tion for black-box large language models. arXiv e- prints, pages ar3iv
Neeraj Varshney, Swaroop Mishra, and Chitta Baral. 2022. Investigating selective prediction approaches across several tasks in IID, OOD, and adversarial settings. In Findings of the Association for Compu- tational Linguistics: ACL 2022. pages 1995-2002, Dublin, Ireland. Association for Computational Lin- guisties.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. Fantastically or- dered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume I: Long Papers). pages 8086-8098, Dublin, Ireland. Association for Computational Linguistics.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171
Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Lan- guage Models. arXiv preprini arXiv:2303. 08896.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoe Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. In Ad- vances in Neural Information Processing Systems.
Clara Meister, Afra Amini, Tim Vieira, and Ryan Cot- terell. 2021. Conditional Poisson stochastic beams. In Proceedings of the 2021 Conference on Empiri- cal Methods in Natural Language Processing, pages 664-681, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Allen R. Wilcox. 1973. Indices of qualitative variation and political measurement. The Western Political Quarterly, 26(2):325-33 3433.
John M Zelle and Raymond J Mooney. 1996. Leam- ing to parse database queries using inductive logic programming. In Proceedings of the National Con- ference on Artificial Intelligence, pages 1050-1055.
Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023. Locally typical sampling. Transac- tions of the Association for Computational Linguis- tics, 1 102-112.
Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan. 2021a. Trading off diversity and quality in natural language generation. In Pro- ceedings of the Workshop on Human Evaluation of NLP Systems (HumEval), pages 25 33, Online. As- sociation for Computational Linguistics.
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. AmbigQA: Answering am- biguous open domain questions. In Proceedings of the 2020 Conference Empirical Methods in Nat- ural Language Processing (EMNLP), pages 5783-
5797, Online. Association for Computational Lin- guistics.
Michael Zhang and Eunsol Choi. 2021. SituatedQA: Incorporating extra- ling-sing contexts into QA. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7371-7387, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Tan Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, and Benjamin Van Roy. 2021. Epistemic neural networks. arXiv preprini arXiv:2107.0892.

---

Shujian Zhang. Chengyue Gong, and Eunsol Choi. 2021b. Knowing more about questions can help: Im- proving calibration in question answering. In Find- ings of the Association for Computational Lingwis- lics: ACL-IJCNLP 2021, pages 1958-1970, Online. Association for Computational Linguistics.
0.95
Voting Disambig.
0.9
Num Answers
0.85
-....
Num Disagreements
Greedy Disambig.
precision
0.8
Voting Direct
0.75
Greedy Direct
Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto. 2023. Navigating the grey area: Expressions of overconfidence and uncertainty in language models. arXiv preprint 1334499.0...333.
0.7
0.65
0.6
0.55
0.5
A Ambiguity Prediction
0
0.1
0.2
2.3
0.4 0.5 0.6 0.7 recall
0.8 0.9
A full description of how we transformed our method into ambiguity prediction below.
Figure 5: Precision vs recall for ambiguity predic- tion. For the sampling- based methods, each point cor- responds to a classification threshold correspouding to counts over the ten sampled outputs. The greedy pre- ing it. Thus, if the model produces an interpretation, it predicts that it is more like the ambiguous ques- tions in the prompt than the unambiguous ones. We can thus use this as a prediction of umbiguity. We use both the greedy output (Greedy Disambig,) and the sampled output as voting mechanism (Voting Disambig.).
Disambiguate and Answer In the prompting setup described, the model should produce an inter- pretation of an ambiguous question before answer-
dictions are plotted as single points. None these sys- tems improve precision the baseline rate of 53%.
first produce the long answer and then the short an- swer, or vice versa. These long answers are taken from ASQA for AmbigQA or the provided long an- swers for Natural Questions, and we refer to these conditions as 'Long Short" and 'Short Long', depending whether the short answer is placed before or after the long answer. From Table 6, we can see this has small and mostly negative effects on Natural Questions: the strongest effect comes from the decline in accuracy when the long answer is produced first, because the model sometimes hits its max decode length. For ambiguous questions, found in Table 7, there is small gain in accuracy at the cost of every calibration metric.
Disagreements and Unique Answers These methods are simply the inverse of Sampling Rep- etition and Sampling Diversity described Sec- tion 3. Num Disagreements refers to the fraction of sampled answers that are not the same as the greedy answer; Num Answers refers to the fraction of unique answers produced in the sampling proce- dure. As previously discussed, more diverse set of answers could reflect either question ambiguity (denotational uncertainty) or epistemic uncertainty.
Direct Prediction We also experiment with a dif- ferent few-shot prompt where instead of predicting an answer, the task is to predict either the string Ambiguous or Unambiguous. We can then use the greedy output binary label (Greedy Direct) use sampling as voting mechanism (Voting Direct).
In the direction of chain of thought, does the choice of exemplars in the prompts materr? Here, we select the questions by hand and write clear long answers and use these as the exemplars the prompt. This also did not have much impact on performance, so we only do this investigation for the unambiguous questions, and call it 'Static Short + Long', 'Static Long + Short', and 'Static Short Only' (for the original case but with the hand selected questions). We find that the prompt largely does not matter, achieving relatively similar results regardless of prompt, though many better prompts may exist. Results can be found in Table 6 for Natural Questions.
As stated in the main text, none of the methods are very effective. An exploration of the preci- sion/recall tradeoff can be found in Figure 5.
B Chain of Thought
Chain-of-though prompts can improve perfor- mance by providing examples of the reasoning pat- terns that yield valid answers (Wei et al., 2022). As a simple chain-of thahough, we have the model

---

Natural Questions
AmbigQA
M
EM
ROC-AUC
ECE
C@88
EM
ROC-AUC
ECE
C@80
L
51.7
0.843
0.141
44.2
44.8
0.731
0.316
17.4
540B
D
0.814
0.152
44.8
0.767
0.114
16.1
R
0,830
0.103
45.4
0.821
0.120
26.2
L
38.8
0.823
0.111
18.8
34.6
0.690
0.235
3.4
62B
D
0.826
0.203
24.2
0.757
0.045
0.8
R
0,836
0.151
24.5
0.805
0.150
3.4
L
16.0
0.829
0.038
0.8
14.8
0.699
0.121
8B
D
0.813
0.228
0.8
0.747
0.080
0.0
R
0.800
0.162
0.8
0.814
0.148
0.4
Table 5: Sealing results on the unambiguous and mixed version Natural Questions AmbigQA. Note that the 540B model is the same model reported carlier in Section 5 and Section 6.4, reshown here for comparison. We use L, D, and R to mean Likelihood, Diversity, and Repetition for brevity.
Natural Questions
AmbigQA
Prompt
M
EM
ROC-AUC
ECE
C@ 80
Prompt
M
EM
ROC-AUC
ECE
C@
Static
50.5
0.756
0.474
8.0
Long
L
36.3
0.663
0.152
1.2
Long
0.820
0.117
36.3
D
0.758
0.041
6.0
Short
0.838
0.084
42.8
Short
R
0.859
0.061
21.8
Static
48.7
0.844
0.196
39.2
Short
L
0.716
0.351
12.4
Short
0.817
0.194
38.0
D
0.742
0.127
Long
0.824
0.177
37.0
Long
R
0.795
0.123
21.8
Long
42.5
0.656
0.42
4.2
0.793
0.147
22.7
Table 7: Experiments on AmbigQA using different prompt formats. We randomly chose exemplars from the ASQA annotations over Natural Questions. Note that the ramdomly selected exemplars are sometimes quite long, causing the model sometimes decoding space in that setting. Note that we do not hand-write prompts here due to the difficulty of writing intentionally ambiguous questions well.
Short
0.848
0.079
31.0
Short 1.
49.6
0.835
0.154
37.3
D
0.820
0,168
38.0
Long
R
0.832
0.133
42.5
of
Table 6: Experiments Natural Questions using dif- ferent prompt formats. In the first, we hand-wrote the prompt. the second set, we randomly chose exem- plars from Natural Questions. Note that the randomly selected exemplars are sometimes quite long, causing the model sometimes run out of decoding space in that setting.
higher accuracy. The sampling repetition method is able to answer 86. % of the questions at 80% accuracy, which is substantially better.
Text as Calibration
Can language models simply tell us when they don't know the answer? Here, we use a prompt where two of the questions are given the answer "Unknown". instead of the given answer. The method follows the above prompting strategy, ex- cept two of the answers are randomly replaced with Unknown, and the unambiguous and ambiguous both use six exemplars. Results can be found in Table Overall, on the unambiguous portion of Natural Questions, the model chooses to abstain a relatively large fraction the time for only slightly
The ambiguous results are more interesting, where the model very rarely outputs unknown even though the questions are supposedly more chal- lenging. The model is able to improve its accuracy somewhat by not answering small number of questions: however, it cannot reliably abstain when is wrong, thus leading to overall low percent- age of accuracy. While not computed above, the standard model's C@45 would be approximately 99.6%.

---

Method
Acc
Cov
Natural Questions
Long + Short
49.8
65.3
Short + Long
62.1
65.4
Short Only
60.4
73.0
Static L+S
61.9
53.7
Static S+L
63.9
53.7
Static SO
63.9
63.7
AmbigQA
Long Short
38.8
86.4
Short + Long
46.6
91.3
Short Only
45.7
95.2
Table 8: Answer Unknown Results. Note that the Ace is the EM rate only on the answered questions, so they are most comparable to the Cov@ Ace num- bers. While they somewaht difficult compare to the above numbers as neither Cov or Acc is fixed, this method is generally uniformly worse than the other methods.
D Scaling
See Table for results. Note that calibration as defined by error is roughly constant, the proportion of the questions that can be answered with reason- able accuracy declines dramatically, approximating zero on the smallest model sizes.
E Sampling
See ?? for results.


Reviews:
[{"review_id": "BpjfBIiBZK", "reviewer": "Reviewer_RjGt", "summary": "", "questions": "A. Results show that sample repetition method improves calibration significantly from likelihood method for ambiguous questions, but results for self-verification method is missing from Table 2. How does the self-verification method hold for calibration of ambiguous questions?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review. As mentioned, this can be thought of as a form of self-consistency; however, unlike self-consistency, we are using the greedy (temperature=0) answer rather than the majority-vote answer and focus on ambiguous questions and calibration. \n\nWe did not run self-verification on the ambiguous questions because it performed poorly on the unambiguous questions and seemed like an unnecessary use of resources. Beyond that, it is not completely clear how to convert the self-verification prompt to handle multiple question interpretations. However, if the reviewer thinks this experiment would improve the final version, then we can run it.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- While works that consider ambiguous questions in QA and calibration exist, few work study them together and this paper presents a novel insight into the difficulty of calibration of AQ in terms of the two categories of AQ.\n- The proposed sampling repetition calibration method is simple to implement and well motivated.", "reasons_to_reject": "- The repetiton sampling mechanism itself is similar to self-consistency decoding method [1] , and thus limited in novelty. However, its application to calibration in this paper still seems novel and reasonable. \n\n[1] Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models, ICLR 2023.", "questions_to_authors": "A. Results show that sample repetition method improves calibration significantly from likelihood method for ambiguous questions, but results for self-verification method is missing from Table 2. How does the self-verification method hold for calibration of ambiguous questions?", "paper_topic_and_main_contributions": "This paper tackles the task of calibration in QA, specifically considering ambiguous questions that may have multiple interpretations. The contribution of the paper are as follows:\n- it frames and analyzes the calibration of ambiguous questions (AQ) in terms of two different categories based on the source of uncertainty: denotational and epistemic uncertainty. \n- it presents two approaches for measuring confidence for both types of uncertainty that is better than the simple likelihood-based calibration and self-verification prompt of previous work. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BxLMS9yK0l", "reviewer": "Reviewer_x5aR", "summary": "", "questions": "1.\tThe paper mentions that for ambiguous questions, “the prompt will first include an interpretation of the question which is one of the provided disambiguations” (lines 412-414). What is the idea behind doing this? If the prompt already includes the disambiguation of the question, is the question even ambiguous anymore?\n2.\tThe paper discusses that the sampling diversity method is “heavily dependent on the number of samples, as we do not expect the number of unique answers to scale linearly with the number of samples” (line 217). Is there a way to normalize for the number of samples to avoid this effect? How would this affect selecting a threshold for practical purposes? \n3.\tThe paper mentions “This leads us to our disambiguate-then-answer paradigm: if there is no fundamental difference between ambiguous and unambiguous queries when taken out of the original context, …” (lines 453-456). What does this even mean? Ambiguous queries will be inherently ambiguous taken out of context, as opposed to the unambiguous queries. Can you please clarify what the point here is?\n4.\tFor the self-verification baseline, the model is first prompted to provide a list of possible answers to a given question, followed by a second task to determine if each answer is correct or incorrect.  The paper mentions that the candidate set of answer may contain duplicates. Wouldn’t this affect the evaluation? Wouldn’t repetition lead to probabilities being split between multiple answer? Why not consider the unique set of candidate answer before calculating confidence?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review. As the reviewer mentions, the focus of our paper is indeed calibration over ambiguous questions, which we tried to make clear in the title and abstract. However, the reviewer mentions that discussions of the various types of uncertainty are present in other work. However, do note that the focus on ambiguous questions meant that denotational uncertainty is more likely to be present. We can edit the summary of contributions to re-emphasize that handling ambiguous questions was the main contribution. Please also note that the work mentioned is extremely contemporaneous with our work, with Lin et al (2023) being released after the EMNLP anonymity period.\n\nFor answering the ambiguous questions, the prompt's exemplars contained demonstrations of disambiguating the question before answering it, which the model was expected to do as well. For the question the model was to answer, it did not see the interpretation of the question, it instead had to produce it.\n\nPreviously, we only gave the model credit if it's produced interpretation was closest to the interpretation matching the answer -- however, this made little difference in practice, and there were many ways to measure if the interpretation matched the answer. In general, those results lowered the accuracy but kept calibration roughly the same:\n\n\n--|Likelihood | Diversity | Repetition \n-----|-----|-----|-----\n| ROC/AUC | 0.724 | 0.762 | 0.814  |\n| ECE |        0.284 | 0.082 | 0.150 |\n| Cov@80 |     12.4 | 13 | 21.2 |\n\nComparing to the original numbers, nothing is directionally different. We thought this was an aside and not worth centering in the discussion, but can try to clarify this with the additional space provided.\n\n## Re: Number of Samples\nOn selecting for the number of samples, there are some suggestions (e.g., Menhinick's index suggests dividing by the square root), but probably using a dev set to choose the correct value will always be useful in practice. While time/resource limitations prevent us from checking more samples for now, we did check 3, 5, and 8 (by subsampling from our original 10) for NQ (Table 1) and AmbigQA (Table 2). For the Sample Repetition confidence metric, the ROC-AUC and C@80 decreased by less than 2% when the number of samples was reduced from 10 to 5. For the Sample Diversity confidence metric, the impact of reducing the number of samples was somewhat larger, especially on the AmbigQA dataset. Both metrics were considerably worse when using only 3 samples. Indeed, as the reviewer notes, more samples increases the cost linearly, so it is important to get the number correct. \n\nFor NQ:\n\n|–|n|roc_auc|Cov@80|ece|\n|----|----|----|----|----|\n|D|3|0.749|40.8|0.132|\n|D|5|0.791|44.2|0.086|\n|D|8|0.806|43.9|0.125|\n|D|10|0.814|44.8|0.152|\n—\n|R|3|0.789|43.3|0.155|\n|R|5|0.811|45.0|0.116|\n|R|8|0.826|45.3|0.099|\n|R|10|0.830|45.4|0.103|\n\n\nFor AmbigQA:\n|–|n|roc_auc|Cov@80|ece|\n|----|----|----|----|----|\n|D|3|0.650|12.1|0.297|\n|D|5|0.719|12.5|0.218|\n|D|8|0.755|16.8|0.148|\n|D|10|0.767|16.1|0.114|\n—\n|R|3|0.769|16.7|0.152|\n|R|5|0.801|24.3|0.130|\n|R|8|0.815|26.9|0.120|\n|R|10|0.821|26.2|0.120|\n\n\n## Re: \" Ambiguous queries will be inherently ambiguous taken out of context, as opposed to the unambiguous queries.\"\nThe goal of the method was not to abstain when there was denotational uncertainty in the question. In fact, our position was that there is always some amount of denotational uncertainty in the question (usually by accident). Instead, we wanted the model to only answer if it was confident it 'knew what the user meant'. We will try to clarify the wording of this in the final version. The point is that among natural questions, many queries are unintentionally ambiguous because of a lack of shared context, and queries that seemingly have only one interpretation to one reader may still depend on time and place in interesting ways. Thus, ambiguity is a function of the reader (or model, in this case), not just the question, and re-stating an interpretation of the question is one way to determine if the reader/model understood correctly. This is part of our argument for why a model might disambiguate and answer instead of abstain-when-ambiguous.\n\n## Re: Self-verification duplicates\nThe model is a decoder-only model and the answer spans do not compete with each other for probability. Instead, the model sees the list of candidate answers as well as the proposed greedy answer (similar to the prompt described in the cited paper). It then scores the token TRUE. Thus, the only competition for probability is between TRUE and all of the other tokens in the vocabulary, and nothing in the prompt is scored at all. We will clarify this method in the final version.\n\n## Re: Other baselines\nMany previous baselines look at multiple choice question answering, reading comprehension question answering reframed as multiple choice question answering  (Zhang et al 2021b)., or sentence classification (Varshney et al, 2022). There are methods in e.g., Lin et al (2023) could potentially be used; however, they unfortunately do not release numbers that we can compare against directly. As that paper was contemporaneous with us, we were unable to re-implement their methods for this submission.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe problem of studying uncertainty calibration in the case of ambiguous questions is quite interesting.  However, I think the proposed experiments are severely lacking.", "reasons_to_reject": "1.\tThe main contributions of the paper seem exaggerated. The paper claims its first main contribution as “we reframe the discussion of model confidence with respect to denotational and epistemic uncertainty”. Lin et al. (2023)---one of the papers cited in this submission---already brings out this distinction as “uncertainty, which depends only on the input, and confidence, which additionally depends on the generated response.” The paper includes no discussion or reference to this overlap in the key idea. The second main contribution of the paper is “we present two simple and effective approaches for measuring confidence under all types of uncertainties”. Again, the paper later reveals that the resampling based confidence estimates are previously also proposed in Kuhn et al. (2023) and Lin et al. (2023). I feel that this misrepresents the contribution of the paper initially, where eventually the only emerging distinguishing factor from existing literature is additional experiment on ambiguous question.\n2.\tThe experimental setup and evaluation design does not align well with the motivated problem. For evaluating calibration, the paper considers the confidence calibration metrics that are standard in the literature, such as Expected Calibration Error, ROC-AUC, etc. However, for this evaluation, the paper “consider(s) the answer as correct if it matches the answer of any of the disambiguation” (line 463-464). This defeats the whole purpose of model being able to abstain when there is denotational uncertainty in the question if the model is rewarded for answer to any possible interpretation of the question. What is the objective of this evaluation and how is this evaluation serving its intended purpose?\n3.\tThe paper mentions various works in uncertainty calibration literature, such as Zhang et al. (2021b), Varshney et al. (2022), and Lin et al. (2023), which propose host of different methods beyond likelihood or self-verification for confidence calibration. However, the paper does not include most or all of the proposed methods in these works as baselines. Based on the discussion of the prior literature in the paper, it is not clear if the baselines discussed in the paper are the best possible baselines and good representative of the different approaches in the existing literature.   \n4.\tThe likelihood-based baseline and resampling based methods vary in compute cost as the latter require multiple queries before processing a single query (to decide whether to answer or abstain). How does that factor into the differences in the baseline and the “proposed method”. Further, how do confidence estimates depend on the number of samples and what is a good number of samples to be able to obtain reliable confidence estimates? These are some key discussions that are missing from the paper that are important to address.", "questions_to_authors": "1.\tThe paper mentions that for ambiguous questions, “the prompt will first include an interpretation of the question which is one of the provided disambiguations” (lines 412-414). What is the idea behind doing this? If the prompt already includes the disambiguation of the question, is the question even ambiguous anymore?\n2.\tThe paper discusses that the sampling diversity method is “heavily dependent on the number of samples, as we do not expect the number of unique answers to scale linearly with the number of samples” (line 217). Is there a way to normalize for the number of samples to avoid this effect? How would this affect selecting a threshold for practical purposes? \n3.\tThe paper mentions “This leads us to our disambiguate-then-answer paradigm: if there is no fundamental difference between ambiguous and unambiguous queries when taken out of the original context, …” (lines 453-456). What does this even mean? Ambiguous queries will be inherently ambiguous taken out of context, as opposed to the unambiguous queries. Can you please clarify what the point here is?\n4.\tFor the self-verification baseline, the model is first prompted to provide a list of possible answers to a given question, followed by a second task to determine if each answer is correct or incorrect.  The paper mentions that the candidate set of answer may contain duplicates. Wouldn’t this affect the evaluation? Wouldn’t repetition lead to probabilities being split between multiple answer? Why not consider the unique set of candidate answer before calculating confidence?", "paper_topic_and_main_contributions": "The paper discusses uncertainty in question answering setup from two perspectives—the uncertainty in the meaning of the question (denotational uncertainty) and uncertainty in the answer (epistemic uncertainty). The paper argues that the existing work in confidence calibration is only done with unambiguous questions and extends this to ambiguous questions leading to denotational uncertainty. The paper proposes two resampling based confidence estimation strategy (1) # times the sampled output matches the greedy output and (2) the ratio of number of unique samples over total samples). The paper argues that these confidence estimates capture uncertainty more accurately both for ambiguous and unambiguous questions.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1.\tThe paper oscillates between abstaining from answering questions vs disambiguating before answering the question, especially in the Section 6. I would recommend clarifying this dual objective upfront in the introduction to better motivate what is expected out of the setup and evaluation."}}, {"review_id": "8J8f3I7FmE", "reviewer": "Reviewer_DXG6", "summary": "", "questions": "1. Since the sampling-based results can yield different outputs on multiple runs, can we obtain some statistics like standard deviations on these results? \n\n\n2. Why was the self-verification baseline not included in Table 2? \n\n\n3. For section 7, were any of these datasets used for experiments directly used in the FLAN Palm fine-tuning? This could be a confounder in the improvement in performance/calibration. ", "limitations": "", "rating": 4, "confidence": 1, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful review.\n\n## Re: Standard deviation of performance\nThis is a very interesting request. Unfortunately, computing even a relatively inaccurate standard deviation based on three samples would require running all of our experimental conditions multiple times which is computationally expensive and not practical in the limited time available for the author response. We ran our methods twice on Natural Questions, and were somewhat satisfied that nothing shifted directionally and there were only small deviations.\n\n## Re: Self-verification baseline\nWe did not run self-verification on the ambiguous questions because it performed poorly on the unambiguous questions and seemed like an unnecessary use of resources. Beyond that, the method to convert the prompt to handle multiple interpretations of a question was somewhat unclear. However, if the reviewer thinks this experiment would improve the final version, then we can run it.\n\n## Re: NQ in the instruction tuned set\nNeither Natural Questions nor AmbigQA are used to train FLAN. TriviaQA was used in the FLAN fine-tuning data, which is one reason we excluded this dataset from section 7.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is written and structured very well making it easy to follow from the general breakdown of uncertainty motivating the experiments and results.\n\n\nThe results show clear results of trends across various calibration-based metrics that the sampling-based confidence scores, while simple and intuitive, are more effective than the two baseline methods. \n\n\nThe main paper is self-contained with the experiments to evaluate the basic research questions and the appendix has a number of interesting related results that I had as questions, including scaling trends and variations in prompt format.", "reasons_to_reject": "Hard to state it as a 'weakness' per se, but given that the paper proposes a general formulation it would be very helpful to see if these trends follow in other (open-source) models as well. The paper reports multiple variants and do list this as a limitation as well but I think it's worth mentioning since many readers won't be able to access the Palm model. ", "questions_to_authors": "1. Since the sampling-based results can yield different outputs on multiple runs, can we obtain some statistics like standard deviations on these results? \n\n\n2. Why was the self-verification baseline not included in Table 2? \n\n\n3. For section 7, were any of these datasets used for experiments directly used in the FLAN Palm fine-tuning? This could be a confounder in the improvement in performance/calibration. ", "paper_topic_and_main_contributions": "This paper breaks down the uncertainty in answering a (potentially ambiguous) question into the uncertainty associated with understanding the question and the uncertainty in giving the right answer due to a lack of knowledge.  The authors then present multiple candidate formulations of model confidence and evaluate these when asked to answer ambiguous and unambiguous questions in a few-shot manner. The candidates presented by the authors are based on sampling diversity and repetition which are calculated on a set of 10 generated outputs for each question. These confidence measures are compared to two from existing work - using the model likelihood and using model self-verification via prompting. \n\nThe sampling repetition confidence score obtains good performance on a suite of calibration metrics across both kinds of questions. While model likelihood is a strong candidate on unambiguous questions, its performance drops considerably on ambiguous ones highlighting the connection to the breakdown of uncertainty. Finally, the authors also present experiments that show that the sampling-based confidence scores do not degrade when fine-tuning the model whereas likelihood gets worse.", "excitement": 4, "reproducibility": 1, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies calibration of question answering systems when presented ambiguous questions. It first decomposes uncertainty in this setting into uncertainty about interpreting the question vs. uncertainty in the right answer (due to lack of knowledge). The paper also proposes different ways to measure model confidence; the best method samples 10 outputs and measures their similarity. The reviewers were generally supportive of this paper, finding that it proposed an important setting and a method that performs well. The paper also includes valuable analysis of how things like model scale, prompt format, and fine-tuning affect results.

One reviewer was concerned about the evaluation setting used, as the paper currently considers a model to be correct if it outputs any correct answer to any valid interpretation of an ambiguous question, even if that interpretation differs from the model's predicted interpretation. During the rebuttal, the authors presented additional results where the model is only considered correct if its answer matches its chosen interpretation. These results show the same patterns as before. I agree with the reviewer who raised this concern that this issue merits further discussion in the camera-ready version, and that both sets of results should be presented (either in the main text or appendix, depending on space considerations).
Decision: Accept-Main