Calibrating Expressions of Certainty

Authors: Peiqi Wang, Barbara D. Lam, Yingcheng Liu, Ameneh Asgari-Targhi, Rameswar Panda, William M Wells, Tina Kapur, Polina Golland
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: calibration, uncertainty, optimal transport, language models

Abstract:
We present a novel approach to calibrating linguistic expressions of certainty, e.g., "Maybe" and "Likely". Unlike prior work that assigns a single score to each certainty phrase, we model uncertainty as distributions over the simplex to capture their semantics more accurately. To accommodate this new representation of certainty, we generalize existing measures of miscalibration and introduce a novel post-hoc calibration method. Leveraging these tools, we analyze the calibration of both humans (e.g., radiologists) and computational models (e.g., language models) and provide interpretable suggestions to improve their calibration.

Decision: accept (poster)
Reviewer scores: 5, 6, 8, 6

Reviews and discussion:
<thread>
creation date:
1729327033168

thread_type:
official_review

<thread_object>
creator: reviewer_ityd

<content>
summary:
This paper describes a novel approach to calibrate certainty phrases directly instead of first mapping these phrases to an explicitly confidence score and next calibrating these scores. The method uses probability distributions from a data set with expressions and confidence scores to find a “translation” mapping. The model is tested on radiology reports with human descriptions and on Q&A tasks for LLMs. Their approach performs similar to the traditional methods that use indirect mappings. The main advantage is that their approach uses a direct mapping from certainty phrase to certainty phrase representing global value ranges.

strengths:
- calibration method that does not require an explicit mapping from phrase to score
- intuitively models peoples use of uncertainty expressions to calibrate for populations
- strong mathematical modeling (as far as I can judge as a non-expert on this)

weaknesses:
- the model does not outperform standard SOTA approaches
- the model seems to remove outlier expressions and calibrates to the middle area
- the mathematical modeling is dense and difficult to understand for non-experts
- misses a deeper analysis and clarifying examples about the semantics and use of uncertainty phrases
- introduction promises that it can help radiologists to report more consistently but cannot find and evidence of this in the paper.

questions:
- the model does not outperform standard SOTA approaches
- the model seems to remove outlier expressions and calibrates to the middle area
- the mathematical modeling is dense and difficult to understand for non-experts
- misses a deeper analysis and clarifying examples about the semantics and use of uncertainty phrases
- introduction promises that it can help radiologists to report more consistently but cannot find and evidence of this in the paper.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730446579624

thread_type:
official_review

<thread_object>
creator: reviewer_qs5l

<content>
summary:
The paper measures and calibrate expressions of certainty made in natural language (e.g., likely, unlikely). By treating certainty phrases
as distributions over the probability simplex, the authors generalize existing estimators for miscalibration metrics, such as the expected calibration error. The work has real-world implications in fields like radiology.

strengths:
- A good understanding and presentation of the existing literature on calibration.
- Using confidence distributions makes the proposed estimator more robust to increasing the number of bins.
- Propose a novel framing of calibration as a composition of source confidence distributions, an optimal transport map to target confidence distributions, and an indexing function.
- Provides potential real-world use cases, such as calibrating human expressions of uncertainty in medicine and LLM expressions of uncertainty.

weaknesses:
- The paper focuses on binary classification, which could make its impact limited.
- Conversely, the relative complexity of the approach to non-statisticians could limit its adoption.
- The candidate confidence distributions could in practice be very large, which could make some of the optimization problems very slow to solve.

questions:
In practice, if humans are already mentally correcting for the under/over-estimation of expressed uncertainty (either by a doctor or LLM), couldn't such calibration lead to worse outcomes, since their mental models no longer apply while the calibrated expressions are still not perfect? This is more of a human problem than a technical one, but just a thought.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730482022867

thread_type:
official_review

<thread_object>
creator: reviewer_v58g

<content>
summary:
This paper introduces a novel miscalibration measure that incorporates a flexible, user-defined distribution for each bin, enabling a supervised "soft binning" approach. The authors propose this generalization to address the ambiguity in verbalized uncertainty levels, such as "unlikely" or "likely," which may not correspond to specific probability values. Additionally, the paper presents a post-hoc calibration method to adjust the use of uncertainty phrases based on the true distribution associated with each phrase. Experimental results on CT reports and outputs from language models demonstrate that the proposed miscalibration metric offers additional information of both human and language model usage of uncertainty phrases, and that the post-hoc calibration technique effectively refines the usage of these uncertainty phrases.

strengths:
- Interesting proposal that uncertainty phrases could be associated with a distribution instead of a single confidence score.
- Very clear presentation of the method and insightful comparison against existing work.

weaknesses:
- The link to uncertainty phrases seems superficial, as they serve merely as names for distributions without strong justification for this treatment.
- The motivation to represent uncertainty phrases as distributions is unclear; individual users often interpret these phrases in consistent orders, making scalar values (or simple binning) potentially sufficient. At a population level, it’s unclear why uncertainty phrases should align with complex empirical distributions rather than a straightforward rubric. In fact, it seems that using scalar values or distinct, non-overlapping probability regions might be a simpler and more operational approach.
- Limited Application: Only works for fixed uncertainty phrases in isolation, does not work with free-form generation with flexible uncertainty quantifiers.
- From table 1, the method does not seem to perform better than simpler post-hoc calibration methods that do not require confidence distribution.
- **NOTE** formula 6 is well outside the margin limit, please fix.

questions:
- In the captions for Figure 4 and Table 1, the authors highlight the interpretability and informativeness of their calibration methods. However, the proposed post-hoc calibration method seems to make the calibration map less intuitive (e.g., Figure 3). Could the authors clarify how their methods enhance human interpretability, given the challenges of conceptualizing uncertainty as a distribution?
- It’s unclear how the ground truth uncertainty distribution is determined. The authors experimented with various methods to derive density functions for each confidence distribution, but it remains unclear which is best. Table 3 and Figure 5 suggest that the choice of ground truth density function impacts evaluation results. How can these ground truth density functions be reliably collected?
- In the "Calibrating Radiologists" experiments, confidence labels are extracted using Llama 3 (line 320 - 321), rather than from the radiologists' own report-level confidence labels. If this is correct, the analysis in Section 4.2 attributing uncertainty phrases to radiologists seems problematic. Could the authors clarify how this experiment was conducted?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730665961431

thread_type:
official_review

<thread_object>
creator: reviewer_88fm

<content>
summary:
This paper proposed a method to calibrate expressions of uncertainty in LLM generations. Instead of treating each uncertainty phrase as a probabilistic score between $[0, 1]$, this paper proposed that they be mapped to a distribution, expressed as a probability simplex. Under this formulation, calibration is reduced to a problem of learning an optimal transport. The authors conducted experiments over a curated dataset of radiologists, finding that their proposed method is comparable to baseline methods.

strengths:
- By considering confidence as a probabilistic simplex, the formulation of calibration as an instance of optimal transport is interesting.

weaknesses:
- 
The experiments are performed over a self-curated dataset whose details are not aptly described. Maybe more details on the dataset is preferred. Additionally, there are public datasets that could be used for these experiments (e.g. https://nlp.jhu.edu/unli/, for natural language inference). As the authors proposed a general calibration method, I believe that datasets with diverse settings should be considered.

 - Some experimental setup are unclear. See below.
 - Table 1: The proposed method's improvement is marginal, or even non-existent as compared to baseline calibration methods such as Platt scaling and histogram binning. This does not justify the usefulness of your proposed method.

questions:
- L265: $\triangle_{K-1}$: The general notation for the probabilistic simplex should be $\Delta^{K-1}$.
 - L420: I am dubious of LLMs' ability to generate a sequence like `Beta(2, 3)` that accurately reflects its internal belief. How many exemplars are used in the in-context learning? This is not spelled out clearly in the Appendix. (It seems that only 1 example is used?)
</content>
</thread_object>

</thread>
<thread>
creation date:
1732029865286

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
shared response

comment:
We sincerely thank all reviewers for their time and helpful comments, which have helped us strengthen our submission. We are happy they found our paper (1) offers clear and rigorous presentation of method and insightful comparison against existing work, (2) provides an interesting proposal to treat certainty phrases as distributions and formulate calibration as optimal transport, and (3) demonstrates strong real-world applicability (e.g., in medicine).

In response to reviewers’ comments, we have updated the draft as summarized in the table below.

| Changes                                                                                                                                                                                 | Reviewer         |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- |
| Provide code to reproduce LM experiments in the supplementary material.                                                                                                                 | 88Fm,qs5L       |
| Clarify interpretation of results that compares our approach with baselines (Table 1).                                                                                                  | 88Fm,V58g,itYd   |
| Add future work to conduct clinical study on whether radiologists can adapt their use of certainty phrases to improve calibration (Section 5).                                          | V58g,qs5L,itYd |
| Add more details on the self-curated radiology report dataset (Appendix B.1).                                                                                                           | 88Fm             |
| Add qualitative examples on how radiologists and LMs use certainty phrases (Appendix B.2).                                                                                              | itYd             |
| Clarify why we believe our smooth calibration curves are more informative than previously used methods (Appendix C.3) and the interpretability of our calibration method (Section 4.2). | V58g             |
| Add detailed answers on why we treat certainty phrases as distributions, and our methods' use cases (Appendix D).                                                                       | V58g             |
| Fix typos, formatting issues, & unclear sentences in experimental setups.                                                                                                               | 88Fm,V58g       |

Beyond these changes, we have addressed all comments raised by the reviewers (see individual responses below). Again we deeply appreciate the feedback and are happy to address any remaining questions during the discussion phase.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper explores expressions of certainty ("maybe" and "likely") and how to calibrate LLM certainty with these expressions.  Rather than pegging these to a particular likelihood value, this paper seeks to understand both individual- and population-level variation in the use of these phrases. Instead of representing confidence as a score, this paper represents it as a distribution over [0,1]. The paper generalizes notions of ECE to this setting and connects this to kernel density estimation. With everything in a continuous space, calibration can be viewed as optimal transport.  The paper then evaluates how calibrated radiologists are on their predictions, as well as calibration of LLMs with probabilistic and verbal calibration. Post-hoc calibration based on optimal transport performs similarly to baselines, although this is based on treating each certainty phrase as the mean of its distribution.

This paper presents an interesting new way of thinking about expressions of certainty. It is well-written and develops a new mathematical framework for thinking about this. In my view, this could significantly impact the ongoing work of researchers in this area, although more work and more data is needed to really deliver on this view of calibration.

The main issues here are the lack of a slam-dunk experimental result here. The gains reported in Table 1 are marginal, and the authors backpedal and say that improving things here is not their intent. I see this claim and agree that it is hard to show why this framework is worth using in an empirical way, but also feel that this paper should show something stronger experimentally.  There are some other weaknesses such as the self-curated nature of the dataset and the overall framing of the work.

While the weaknesses here are significant, I think the significant recent body of work on linguistic calibration would benefit from this treatment. This paper breaks new ground in how to think about and model this task which other researchers can substantially benefit from. There are rough edges in the paper, but it is thought provoking and likely to be more impactful than papers that marginally advance SOTA in some area.