RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Self-Calibrating Language Models
A number of papers propose to quantify confidence and detect hallucinations by (a) directly asking the model to rate its own confidence or correctness (e.g., Kadavath et al., 2022 ; Lin et al., 2022 ) , (b) drawing repeated samples (Wang et al., 2022 ; Manakul et al., 2023 ) , or (c) requesting additional supporting information about proposed outputs (Agrawal et al., 2023 ) . As discussed in Section 5 and Appendix C , we found that neither self-verification nor offering the option of “unknown” were effective strategies for selective question answering. Wang et al. ( 2022 ) focus on accuracy rather than calibration and Manakul et al. ( 2023 ) focus on fact-checking long-form answers.
Prompting Strategies
In in-context learning, the choice of prompt and the order of exemplars can affect downstream performance (Lu et al., 2022 ) . Other work suggests that giving more long-form answers and explanations in the prompt may encourage the model to do the same, increasing its likelihood of arriving at the correct answer (Wei et al., 2022 ) . In our work, we find the choice of prompt plays little role and that longer answers or explanations do not seem to improve calibration (see Appendix B ). Zhou et al. ( 2023 ) investigate how adding words of uncertainty interacts with calibration, noting that additional expressions of uncertainty improve calibration without hurting accuracy. This is an interesting direction for future research into its interaction with ambiguity.
Conclusion
We investigate the calibration of large language models, extending prior work by distinguishing uncertainty about the answer (epistemic uncertainty) from uncertainty about the meaning of the question (denotational uncertainty). We propose a disambiguate-and-answer paradigm, where the model first attempts to rephrase the question before providing its answer. This paradigm enables straightforward techniques for quantifying model confidence by counting the frequency of answers within repeated samples from the language model. These sample-based confidence metrics are particularly effective for ambiguous questions, and are significantly better calibrated than traditional likelihood-based measures and self-verification. For ambiguous questions, the proposed method is similar to calibration through self-consistency.
Limitations
In this work, we explore selectively answering ambiguous questions and calibrating such models, but we do so only within the context of a single model: PaLM. We do not explore alternative paradigms, such as supervised fine-tuning, or other large language models for in-context learning with which to replicate our results. Furthermore, we only explore calibration in a closed-book setting, despite that open-book question answering (i.e., augmented with retrieval) generally has higher performance and may pose additional calibration challenges. Moreover, our exploration of various confidence scores was relatively brief, and while we explored some additional prompts, we did not iterate heavily on prompt tuning. Lastly, while the proposed method has some advantages, especially for instruction-tuned models on ambiguous questions, it also increases the compute linearly with the number of samples.
Acknowledgements
We thank William W. Cohen, Urvashi Khandelwal and Clara Meister for their valuable comments and feedback while discussing this work.
Disambiguate and Answer
In the prompting setup described, the model should produce an interpretation of an ambiguous question before answering it. Thus, if the model produces an interpretation, it predicts that it is more like the ambiguous questions in the prompt than the unambiguous ones. We can thus use this as a prediction of ambiguity. We use both the greedy output ( Greedy Disambig.) and the sampled output as a voting mechanism ( Voting Disambig.) .
Disagreements and Unique Answers
These methods are simply the inverse of Sampling Repetition and Sampling Diversity described in Section 3 . Num Disagreements refers to the fraction of sampled answers that are not the same as the greedy answer; Num Answers refers to the fraction of unique answers produced in the sampling procedure. As previously discussed, a more diverse set of answers could reflect either question ambiguity (denotational uncertainty) or epistemic uncertainty.
Direct Prediction
We also experiment with a different few-shot prompt where instead of predicting an answer, the task is to predict either the string Ambiguous or Unambiguous . We can then use the greedy output as a binary label ( Greedy Direct ) or use sampling as a voting mechanism ( Voting Direct ).

As stated in the main text, none of the methods are very effective. An exploration of the precision/recall tradeoff can be found in Figure 5 .
Figure 1: Uncertainty in Question Answering systems may arise in various ways. We propose a scheme called disambiguate then answer where the model first attempts to pose an unambiguous interpretation of the user question (yellow), then selectively produces an answer to this question, alternatively abstaining ("Unknown") (green).
The log likelihood of the model is shown above each generation.
In addition, we find that sampling multiple times from the model generally allows for more robust confidence estimates.; Uncertainty in Question Answering systems may arise in various ways. We propose a scheme called disambiguate then answer where the model first attempts to pose an unambiguous interpretation of the user question (yellow), then selectively produces an answer to this question, alternatively abstaining ("Unknown") (green).
The log likelihood of the model is shown above each generation.
In addition, we find that sampling multiple times from the model generally allows for more robust confidence estimates.
Figure 2: Methods for estimating the confidence of answers from an LM. Sampling repetition counts the number of times the greedy answer appears among samples from the LM. Sampling diversity counts the number of unique answers among samples from the LM. Self-verification , proposed by Kadavath et al. ( 2022 ) , prompts the LM again with one of the sampled answers to measure the token-level probability of True.
The prompts used for unambiguous and ambiguous questions are shown on the left—for the latter, we additionally
prompt the model for disambiguations (omitted in the outputs shown on the right for brevity).; Methods for estimating the confidence of answers from an LM. Sampling repetition counts the number of times the greedy answer appears among samples from the LM. Sampling diversity counts the number of unique answers among samples from the LM. Self-verification , proposed by Kadavath et al. ( 2022 ) , prompts the LM again with one of the sampled answers to measure the token-level probability of True.
The prompts used for unambiguous and ambiguous questions are shown on the left—for the latter, we additionally
prompt the model for disambiguations (omitted in the outputs shown on the right for brevity).
Figure 3: Plot of calibration error by comparing bucketed accuracy to bucketed confidence scores across methods. Plotted on the unambiguous portion of Natural Questions.; Plot of calibration error by comparing bucketed accuracy to bucketed confidence scores across methods. Plotted on the unambiguous portion of Natural Questions.
Figure 4: Plot of calibration error by comparing bucketed accuracy to bucketed confidence scores across methods. Plotted on the combined version of AmbigQA, containing ambiguous and unambiguous questions.; Plot of calibration error by comparing bucketed accuracy to bucketed confidence scores across methods. Plotted on the combined version of AmbigQA, containing ambiguous and unambiguous questions.
Figure 5: Precision vs recall for ambiguity prediction. For the sampling-based methods, each point corresponds to a classification threshold corresponding to counts over the ten sampled outputs.
The greedy predictions are plotted as single points.
None of these systems improve precision over the baseline rate of 53%.; Precision vs recall for ambiguity prediction. For the sampling-based methods, each point corresponds to a classification threshold corresponding to counts over the ten sampled outputs.
The greedy predictions are plotted as single points.
None of these systems improve precision over the baseline rate of 53%.
Non-Ambiguous QA Calibration on Natural Questions and TriviaQA. The column M refers to the calibration methods, abbreviated with (L)ikelihood, (D)iversity, (R)epetition, and (V)erification.
Ambiguous Calibration. Column M refers to the calibration methods, abbreviated with (L)ikelihood, (D)iversity, and (R)epetition. Loose matching counts the answer as correct if it matches the answer from any interpretation; strict matching only counts it as correct if one of the closest interpretations has the same answer.
Calibration by number of samples (N). EM is excluded, because it is solely based on the greedy answer and does not depend on the number of samples.
Calibration for instruction tuned Flan-PaLM models. The column labeled M refers to the calibration methods, abbreviated with (L)ikelihood, (D)iversity, and (R)epetition.
Scaling results on the unambiguous and mixed version of Natural Questions / AmbigQA. Note that the 540B model is the same model as reported earlier in Section 5 and Section 6.4 , reshown here for comparison. We use L, D, and R to mean Likelihood, Diversity, and Repetition for brevity.
Experiments on Natural Questions using different prompt formats. In the first, we hand-wrote the prompt. In the second set, we randomly chose exemplars from Natural Questions. Note that the randomly selected exemplars are sometimes quite long, causing the model sometimes to run out of decoding space in that setting.
Experiments on AmbigQA using different prompt formats. We randomly chose exemplars from the ASQA annotations over Natural Questions. Note that the randomly selected exemplars are sometimes quite long, causing the model sometimes to run out of decoding space in that setting. Note that we do not hand-write prompts here due to the difficulty of writing intentionally ambiguous questions well.
Answer or Unknown Results. Note that the Acc is the EM rate only on the answered questions, so they are most comparable to the Cov@Acc numbers. While they are somewaht difficult to compare to the above numbers as neither Cov or Acc is fixed, this method is generally uniformly worse than the other methods.