RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Calibration for Question Answering
A predictor is said to be well-calibrated if its predictions are accompanied by confidence scores and those scores are informative of the likelihood that the prediction is correct.
Such confidence scores can be used to support applications such as selective prediction, in which the model can abstain from predicting when its confidence is low (Chow, 1957 ; El-Yaniv et al., 2010 ) . In selective question answering (e.g., Kamath et al. 2020 ; Zhang et al. 2021b ), we assign each question-answer pair (q,a) a confidence score s(q,a) . The system’s output is then parameterized by a threshold \tau ,

\hat{y}_{\tau}(q)=\begin{cases}arg\max_{a}s(q,a),&\max_{a}s(q,a)>\tau\ \varnothing,&else,\end{cases} (1)

with \varnothing representing abstention.
Given a probabilistic model P(a\mid q) , a natural choice for the confidence score s(q,a) is the conditional probability of the answer. This and other scoring functions are discussed in Section 3 .

For information-seeking queries such as those found in Natural Questions (Kwiatkowski et al., 2019 ) , the askers have no answer in mind, which makes it more likely that the questions are accidentally ambiguous.
The task of answering an ambiguous question requires solving at least two subtasks: determining the meaning of the question (its denotation ), and then identifying the answer. This pipeline is shown in Figure 1 ; in a probabilistic model, this can be represented mathematically as P(a\mid q)=\sum_{d}P(a\mid d)P(d\mid q) , with d indicating the denotation.
While prior work has investigated the calibration of answer probabilities given by pretrained language models (e.g., Kadavath et al., 2022 ) , it is not clear whether the relatively positive findings of these studies extend to questions with significant denotational ambiguity, as represented by datasets such as AmbigQA (Min et al., 2020 ) and SituatedQA (Zhang and Choi, 2021 ) . We address this question in Section 6.4 .
Confidence Scores
Before proposing confidence scores for question answering, it will be helpful to review how these answers are generated in practical settings. A language model over a finite vocabulary \Sigma defines a probability distribution X over sequences of tokens in \Sigma^{*} , by modeling the marginal probability of a sequence p(\sigma_{1},\cdots,\sigma_{n}) using the conditional probabilities of individual tokens via the chain rule \prod_{i=1}^{n}p(\sigma_{i}|\sigma_{i-1},\cdots,\sigma_{n}) .
For QA, answers are obtained by conditioning on the
question q and first sampling from X (or a version of X adjusted with modifications such as temperature) and then applying a normalization or extraction process to it in order to map it to an answer space \mathcal{A} .
This function f:\Sigma^{*}\to\mathcal{A} can be as simple as punctuation removal and lower-casing, or as complex as code execution or extracting the answer from a chain-of-thought Wei et al. ( 2022 ) .

Using this background, we can describe various methods for estimating the scores of predictions from
a language model ( Figure 2 ).
Likelihood
The most basic method we can use is likelihood-based calibration by computing p(\sigma|q)=\prod_{i=1}^{n}p(\sigma_{i}|\sigma_{i-1},\cdots,\sigma_{n},q) for a sequence sampled from X . This may be used to rank the answers from most confident to least confident based on the model’s produced likelihood of that answer. However, in practical settings this method may not accurately reflect the probability of observing an answer for several reasons:
(1)
Language models typically incorporate several inference-time biases to improve the quality of the outputs, such as nucleus sampling Holtzman et al. ( 2020 ) , top-k sampling Fan et al. ( 2018 ) , length penalties, length truncation, and temperature.
These techniques affect the output distribution in ways that are hard or impossible to capture in the likelihood score Zhang et al. ( 2021a ) .
(2)
Even for decoders that do not incorporate inference-time biases, the likelihood function as defined by the auto-regressive decomposition might leak probability mass to infinite sequences (see Meister et al., 2023 , Proposition 2.4) .
This implies that the model that we sample from might not be a valid probability distribution.
(3)
Finally, the likelihood fails to account for the extraction step f , which requires an intractable marginalization step over the model outputs.

In view of these limitations, it is preferable to use sampling to estimate properties of the resulting marginal distribution.
Formally, we can use an Index of Qualitative Variation (IQV) Wilcox ( 1973 ) to measure the statistical dispersion of discrete samples produced by the model. Practically, there are a many possible IQVs,
so we have chosen two fairly different approaches for our experiments. In each case, we generate 10 sampled outputs (at a temperature of 0.5 ) and use exact match (after lowercasing and removing puncutation) for comparison among outputs. Naturally, there are many other ways to compare outputs, including token-level overlap or an answer-span equivalence model, but we leave these alternatives for future work. See also Kuhn et al. ( 2023 ) and Lin et al. ( 2023 ) for concurrent work investigating alternative IQVs.
Sampling Repetition
Our first approach is based on Wilcox’s classic Variation Ratio which measures deviation from the mode. Instead, we simply compute the fraction of times that the sampled outputs match the greedy output (Temperature =0.0 ). The more samples match the greedy output, the more confident we are in the answer.
We count the number of samples which exactly match the greedy answer. We experimented briefly with more
sophisticated approaches based on the BERT based answer equivalence model from Bulian et al. ( 2022 ) as well as the summed F1 score across sampled answers and the greedy answers, but these approaches did not meaningfully improve the calibration.
Sampling Diversity
Our second approach is based on the diversity of the samples, computed by 1-\frac{\mathrm{num\_unique}}{\mathrm{num\_samples}} .
Here, our confidence is inversely proportional to the number of distinct samples and is estimated as zero if all samples are different.
Note that while this works relatively well in practice, it is heavily dependent on the number of samples, as we do not expect the number of unique answers to scale linearly with the number of samples.

We note that some recent work Meister et al. ( 2021 ); Arora et al. ( 2022 ) has also described estimating statistics of random variables induced by language models. This involved sampling without replacement and using importance weighting, but they seem to surpass Monte-Carlo estimates only with many samples and peaky distributions. We leave further exploration of these methods on our evaluations as well as other IQVs for future work.
Self-Verification
Finally, we consider self-verification, in which the language model assesses its own confidence (Kadavath et al., 2022 ) . This is a two-step process, where first the model is prompted to provide a list of possible answers given a question. Then, this list is fed back into the model, where the task is to determine if each answer is correct or incorrect. In practice, we re-use the sampled outputs from the previous run as the list of possible answers, and the greedy answer as the proposed answer. Then, the model scores the token “True”, which we use to obtain a confidence score. In practice, these possible answers come from sampling with replacement, similar to the Sampling Repetition and Diversity methods, so the list of possible answers may contain duplicates. While these duplicates may affect the usefulness of the brainstorming exercise, the model is not choosing among them; instead, it is only scoring whether the greedy answer is correct, using the list of possible answers as context.
Evaluation setup
Our evaluations test the utility of the confidence scoring methods described in Section 3 .
We focus mainly on the pretrained language model PaLM, which obtains high accuracy on several question answering datasets using few-shot in-context learning (Chowdhery et al., 2022 ) .
Metrics
There are many metrics for evaluating selective question answering systems, but fundamentally we want systems that (1) frequently return correct answers, and (2) rarely return incorrect answers.
These criteria are in conflict because any system can achieve zero error rate by always abstaining from answering.
When the confidence score is a probability, we can also ask whether this probability is well-calibrated , in the sense that a confidence score of s(q,a)=\alpha implies \alpha is the probability of a being the correct answer to q .
Expected Calibration Error (ECE)
Predictions are grouped into ten equally sized bins, ranked by the evaluated system’s assigned confidence scores. We compute the mean absolute distance between the average confidence score and the accuracy of predictions in each bin, averaging across all bins. If we interpret a confidence score to represent a probability, this corresponds to the difference in the predicted probability of correctness from the actual probability of correctness.
ROC-AUC
Area under the receiver operating characteristic curve evaluates the uncertainty estimate’s diagnostic ability as a binary classifier for correct and incorrect predictions by integrating over the tradeoff curve between the rates of true and false positives.
Coverage@Acc
While ECE and ROC-AUC assess absolute and relative calibration respectively, we want a metric closely aligned with a practical use case: selective answering above a confidence threshold.
To this end, we measure the fraction of questions the system can answer correctly if it needs to maintain a certain accuracy. Specifically, \text{C@}\operatorname{Acc} is the maximum coverage such that the accuracy on the C% of most-confident predictions is at least \operatorname{Acc}\% . For example, if C@ 80=20 , then the system achieves at least 80\% accuracy on its 20% most-confident predictions (and lower than 80\% accuracy on its X\% most-confident predictions for any X>20 ). With user-facing systems in mind, we set an accuracy of 80% (C@80).
Unambiguous Questions
To start, we explore the case of unambiguous questions with a single answer. In this scenario, all uncertainty should be of the epistemic form; the denotation of the questions should be clear.
Previous work with this scenario has focused on the domain shift setting (Kamath et al., 2020 ) where the goal is to avoid answering questions from other datasets.
Other work has looked at reading comprehension with relatively small models (Zhang et al., 2021b ; Jiang et al., 2021 ) .
Open-domain question answering is substantially more difficult than reading comprehension, which likewise makes calibration more challenging.
Datasets
We explore this setting using two datasets: TriviaQA Joshi et al. ( 2017 ) and NQ-Open Kwiatkowski et al. ( 2019 ); Lee et al. ( 2019 ) because they are widely used for few-shot question answering. However, while it might be assumed that each question in these datasets has only one possible interpretation, this is often not the case. In fact, AmbigQA Min et al. ( 2020 ) finds that over 50% of NQ-Open questions are underspecified. As such, we limit ourselves to unambiguous questions as annotated in AmbigQA. We will discuss the extended setting that includes underspecified queries in Section 6 .
Experiment Setup
Our investigation focuses on few-shot in-context learning with large language models (LLMs). Our primary prompt is composed of four question and answer pairs from the training sets of Natural Questions and TriviaQA, respectively (see Figure 2 ). To reduce variance across experiments, the example QA pairs are selected randomly so that each input to the model has a different version of the prompt.
We compute metrics over the entire evaluation set for both datasets. We discuss investigation of other prompts in Appendix B .
Calibration Results
Results can be found in Table 1 and Figure 3 .
First, we observe that the verification method used in prior work is inferior across nearly all calibration metrics.
For unambiguous Natural Questions, depending on the metric used, we see that likelihood and sample repetition both work relatively well as calibration metrics.
For TriviaQA, we see that the sample repetition method works notably better: this is potentially due to the presence of ambiguous questions in TriviaQA, which we do not attempt to remove.
This hypothesis is strengthened by experiments in the next section that show that sampling methods tend to improve calibration more in the presence of more ambiguity.
Ambiguous Questions
Questions can be ambiguous for a variety of reasons. Language is highly context-sensitive and meaning can be lost when context is missing. Often, interlocutors fail to clearly translate their intent into unambiguous language. Alternatively, ambiguity occurs due to a lack of alignment between interlocutors at a higher level of abstraction.
Regardless of its origin, ambiguous questions give rise to denotational uncertainty, which we can conceptualize as a latent distribution over interpretations.

Realistic scenarios such as search and chat frequently contain ambiguous questions, which makes calibration more difficult because uncertainty may be epistemic, denotational, or both.
Our approach will again involve sampling: this time over both interpretations and answers. Note that in this case, the approach has some similarity to self-consistency (Wang et al., 2022 ) , as we compute sampling-based confidence scores over the final answers regardless of the exact interpretation.
Datasets
We evaluate using two datasets of underspecified queries that are derived from NQ-Open: AmbigQA and SituatedQA.
AmbigQA
AmbigQA is created from a subset of Natural Questions (Kwiatkowski et al., 2019 ) that raters assess as ambiguous. After a rater deemed a question ambiguous, various unambiguous interpretations of the question and their respective answers are written. Thus, AmbigQA measures a model’s ability to disambiguate-and-answer ambiguous questions. For instance, given a query like “Where does the new fallout game take place?”, the model should be able to produce the disambiguated question “Where does the fallout 76 game take place?” and answer with “Appalachia”. Models are evaluated based on the similarity between the generated question and the closest reference question as well as by producing the correct answer for the generated question.
SituatedQA
SituatedQA is likewise created from a subset of existing questions and uses a relatively similar process, but focuses more narrowly on temporal and geographic ambiguity. As the type of ambiguity and the process used to create them is different, we generally evaluate them separately.
The temporal questions are annotated with the time range that a question’s answer is valid. Afterward, additional time ranges and their corresponding answers are crowdsourced.
The geographic questions are largely created by removing references to location and then crowdsourcing locations and corresponding answers.
Note that SituatedQA does not provide metrics assessing the precision or possible recall of the disambiguations (though the recall would be particularly difficult to measure).
Experiment Setup
As above, we use prompts of question-answer pairs, with each example having six exemplars drawn from the training data.
For AmbigQA, we use only the examples that have been further annotated by Stelmakh et al. ( 2022 ) .
For SituatedQA, we evaluate the Geographic and Temporal sets separately.
For each example, three of the exemplars will be ambiguous; in those cases, the prompt will first include an interpretation of the question which is one of the provided disambiguations. In practice, we use the first disambiguation for each example.
Ambiguity Prediction
Before trying to assess calibration, we first seek to explore whether our method can predict whether a question is ambiguous according to AmbigQA.
SituatedQA is not suitable for this task because its questions tend to follow templated patterns where the ambiguity arises from a missing time or place reference, while AmbigQA has more diverse expressions of ambiguity.
We are not aware of prior work that evaluates ambiguity classification on AmbigQA or other datasets.

We use a first transformations of Section 3 to predict whether the model predicts the question is ambiguous. We discuss this in more detail in Appendix A . In short, none of the methods are particularly capable, with the best method achieving 58% accuracy when 53% of the questions are labeled as ambiguous.

Why is question ambiguity so difficult to predict? One possible explanation is that all queries are somewhat ambiguous. Queries are typically not intentionally underspecified: the questioner believes they have provided enough information to answer.
In a conversation, this may fail and require developing common ground; in an information retrieval system, this may take the form of query refinement.
All of the questions in the Natural Questions dataset were asked of the Google search engine and are answerable using Wikipedia (Kwiatkowski et al., 2019 ) , yet many are labeled as ambiguous by AmbigQA.

For instance, “Where is the world cup going to be?” might typically refer to the host country, but fans from the host country that year might instead refer to the cities the games will be played.
Even binary choice questions like “Who won more games between the Packers and the Bears?” may appear unambiguous, but this question has different answers depending on the date it is asked.

In this sense, we propose that ambiguity is a function of both the interpreter and the query. However, measuring whether a model (or indeed, any interlocutor) understood a question is challenging. Often in dialogue, interlocutors may assume their counterpart understood if they receive a response similar to what they expected; alternatively, they may ask explicitly if they were understood.

This leads us to our disambiguate-then-answer paradigm: if there is no fundamental difference between ambiguous and unambiguous queries when taken out of the original context, then the model should first establish precisely what question it is attempting to answer.
Calibration Results
We evaluate calibration on ambiguous questions using the same methods described in Section 5 with results in Table 2 and Figure 4 . We exclude the self-verification setup here, because its calibration is much worse on the unambiguous questions, and it is unclear how to translate the prompt when there are multiple interpretations of the question.
Note that we consider the answer to be correct if it matches the answer of any of the disambiguations. We also present results for a “strict” matching setup, where we only consider the answer to be correct if it matches the answer of the closest disambiguation to the provided interpretation, as measured by token overlap.

Overall, the ambiguous questions are much more difficult than the unambiguous ones. In general, likelihood seems to become a worse method of measuring model uncertainty when the questions are ambiguous and sample repetition appears to improve calibration significantly. 2 2 2 Note that the interpretations sometimes consist of longer sequences, thus confounding the usability of likelihood The strict matching setup does not affect the ordering of the results, though numbers are lower overall.
Instuction Tuning and Scaling
Pretrained language models are typically fine-tuned before being used in applications. However, fine-tuning and other alignment techniques may disturb the model’s calibration by emphasizing high performance on specific sets of tasks rather than the full pretraining distribution. 3 3 3 The GPT-4 technical report ( https://cdn.openai.com/papers/gpt-4.pdf ) shows that calibration is made significantly worse by the combination of instruction-tuning and reinforcement learning, but does not distinguish between these two factors. Table 4 shows that for Flan-PaLM Chung et al. ( 2022 ) , instruction tuning dramatically improves exact match accuracy while simultaneously worsening performance on confidence-based ranking (ROC-AUC) and calibration (ECE), when using model likelihood as the confidence score.
However, this miscalibration can be mitigated by using sample-based confidence scores, which dramatically improves the calibration on ambiguous questions (AmbigQA). For the selective-prediction metric C@80, the instruction-tuned model with sampling is far ahead of any other method investigated. Note that we investigate Natural Questions and AmbigQA here as they are not in the instruction tuning training data.

We examine the effect of model scale on calibration, finding that in general, accuracy declines substantially in closed book question answering, but calibration stays roughly constant. See Appendix D for full results.

As sampling-based approaches require a linear increase in compute for the number of samples, we also examine how calibration scales with the number. In particular, we test three, five, and eight samples and compare that to the original results containing ten samples. Results can be found in Table 3 . Unsurprisingly, more samples seems to improve calibration, though it seems on the unambiguous Natural Questions slice, sampling diversity with a small number of samples works relatively well for the cost.
Related Work
This paper investigates calibration, selective question answering, and ambiguity within a single model. It builds on work across these topics. While AmbigQA (Min et al., 2020 ) , ASQA (Stelmakh et al., 2022 ) and SituatedQA (Zhang and Choi, 2021 ) introduce new annotations for question answering datasets, neither these papers nor any follow-up work investigate how the uncertainty in interpreting a question interacts with uncertainty about the answer.
Calibration and Selective QA
Calibration of modern machine learning approaches is difficult due to their non-linearity, scale, and architecture details (Guo et al., 2017 ) . Generation-like tasks are even more difficult, containing a sequence of tokens with individual confidence estimates and a special end of sentence marker that may be poorly calibrated (Kuleshov and Liang, 2015 ; Kumar and Sarawagi, 2019 ; Jagannatha and Yu, 2020 ) . A common approach for various natural language processing tasks is to train a separate classifier to estimate the model’s confidence (Kumar and Sarawagi, 2019 ; Jiang et al., 2021 ; Zhang et al., 2021b ; Kamath et al., 2020 ; Desai and Durrett, 2020 ; Dong et al., 2018 ) using various featurizations of the model’s input, output, and likelihoods. Alternatively, Ren et al. ( 2023 ) use embedding distances and Osband et al. ( 2021 ) use Bayesian-inspired approaches that try to predict the full distribution over labels.

Within the specific setting of question answering, Kamath et al. ( 2020 ) and Zhang et al. ( 2021b ) both address a similar selective question answering framework to ours, but they do not explore ambiguous questions. Varshney et al. ( 2022 ) investigate selective prediction for various tasks, and Varshney and Baral ( 2022 ) aim to improve coverage by re-answering questions that the model originally abstained on. Kuhn et al. ( 2023 ) and Lin et al. ( 2023 ) use similar sampling-based methods over free-form question answering, using slightly different formulations of confidence scores, but they do not investigate ambiguous questions. Kuhn et al. ( 2022 ) examine synthetically-created ambiguous questions, but focus on multi-turn interactions.

Question ambiguity can be formalized as uncertainty about the denotation of the question, relating to prior work on uncertainty in semantic parsing (Dong et al., 2018 ) . However, rather the expressing candidate denotations in a formal semantic representation, we express them informally in natural language, as a waypoint toward quantifying the uncertainty of candidate answers.