Uncertainty in Language Models: Assessment through Rank-Calibration (2024)

Abstract: Language Models (LMs) have shown promising performance in natural language generation. However, as LMs often generate incorrect or hallucinated responses, it is crucial to correctly quantify their uncertainty in responding to given inputs. In addition to verbalized confidence elicited via prompting, many uncertainty measures (e.g., semantic entropy and affinity-graph-based measures) have been proposed. However, these measures can differ greatly, and it is unclear how to compare them, partly because they take values over different ranges (e.g., [0,\infty) or [0,1] ). In this work, we address this issue by developing a novel and practical framework, termed *Rank-Calibration*, to assess uncertainty and confidence measures for LMs. Our key tenet is that higher uncertainty (or lower confidence) should imply lower generation quality, on average. Rank-calibration quantifies deviations from this ideal relationship in a principled manner, without requiring ad hoc binary thresholding of the correctness score (e.g., ROUGE or METEOR). The broad applicability and the granular interpretability of our methods are demonstrated empirically.

## introduction language models (lms), especially large language models (llms), have shown promising performance in natural language generation (nlg). these models, fitted on huge text corpora, can produce responses resembling those of humans @xcite . however, since lms often generate wrong or hallucinated responses @xcite @xcite , it is crucial to correctly quantify their level of uncertainty in responding to particular inputs. uncertainty quantification is well-explored in supervised learning, specifically in classification (e.g., @xcite @xcite . in classification, a confidence measure is an estimate of the probability that the predicted class y matches the true class label y @xcite . a confidence measure c is considered calibrated if it reflects the probability of correct prediction, i.e., p( @xmath0, for all values in c's range. the expected calibration error (ece) measures the miscalibration of a confidence measure @xcite : in classification, confidence measures are predominantly built on model logits @xcite . however, these methods are less suitable for nlg tasks. first, the label space is of-ten too large to assess correctness via @xmath1, since lms produce potentially long textual responses y for any given input. second, for lms, logits encode the likelihood of selecting the next token and do not necessarily capture linguistic sense @xcite . third, even hand-crafted prompts intended to make lms express confidence explicitly may not lead to reliable confidence values because elicitation is heavily tied to prompt formats @xcite . recent works have studied uncertainty measures as an alternative to confidence measures. these capture the "dispersion" of an lms' potential outputs for a fixed input. @xcite introduce semantic entropy, which incorporates linguistic invariances arising from the shared meaning of generated responses. @xcite extend semantic entropy by leveraging the affinity matrices induced by entailment scores of generated outputs. further, @xcite characterize differential entropy in the embedding space with eigenscore, via the covariance of embeddings of potential responses. uncertainty measures are more general and arguably more principled than confidence measures for lms, but they lack a universal assessment metric such as ece. a key issue is that uncertainty measures are not necessarily commensurate. for instance, the semantic entropy @xcite can take arbitrarily large positive values, whereas the eigv measure of @xcite depends on the number of responses generated. this makes it difficult to understand, evaluate, and compare uncertainty measures via a unified lens. this paper develops a principled framework to assess the quality of uncertainty and confidence measures for lms. we provide a novel and practical framework, termed rank-calibration. specifically, our contributions are as follows. let v be the token vocabulary of an lm and v ⋆ := ∪ ℓ≥1 v ℓ the space of sequences of arbitrary length. given a query @xmath2, an lm m can generate output y ≜ ( y ℓ ) ℓ≥1 ∈v ⋆ by sequentially sampling from the distribution p( y |x) := ℓ≥1 p( y ℓ | x, @xmath3. here, y ℓ ∈ v is the ℓ-th generated token and p ≜ p m is the generative distribution of m. we work with a deterministic correctness function a:v ⋆ × v ⋆ →r mapping each pair (x; y) to a correctness value a(x; y). in practice, correctness is often not a binary variable in nlg tasks and can be assessed in at least two different ways. for the reader's convenience, the concepts and notations used in the paper are summarized in an uncertainty measure is a (possibly random) function u m :v ⋆ × v ⋆ → r, (x; y) → u m (x; y) associated with the lm that maps any pair (x; y) to an uncertainty value. 1 we will omit m and write u (x; y), p(• | x) when the choice of the lm is clear. some examples are reviewed below, while additional examples and details are in appendix b. ## limitations of existing assessments this section illustrates some limitations of existing assessments for lm uncertainty measures via a case study applying the gpt-3.5-turbo @xcite model on the triviaqa benchmark @xcite . we use the validation set of triviaqa, which contains 11, 322 questionanswer pairs (after deduplication). we use the same prompt template as that in @xcite . the template is shown in appendix e.2. the uncertainty measures examined here include the negative log-likelihood u nll , the semantic entropy u se @xcite , the affinity-graphbased measures u eigv , u ecc , and u deg @xcite , with the affinity determined by the nli model @xcite , and the verbalized confidence c verb @xcite ; see the definitions in appendix b. these include both white box and grey box measures, foot_3 as well as a diversity of prompt strategies. we use the rouge-l score as the correctness function a. we follow a common assessment pipeline @xcite @xcite , as depicted in fig. 2 . the assessment metrics are detailed in appendix c. however, thresholding can lead to inconsistencies. taking auroc as an example, we plot the assessed results of uncertainty/confidence measures under varying thresholds in fig. 3 [top]. the relative auroc results of distinct measures vary drastically with the choice of τ . for example, u nll appears inferior to other methods if τ < 0.2, but it becomes the best measure if τ > 0.8. this is especially concerning given that there seems to be no principled way to set this threshold. the same limitation also affects other metrics (e.g., auprc, auarc) and configurations; see appendix e.4. ## rank-calibration in this section, we introduce a novel assessment framework satisfying the criteria outlined in sec. 3. ## experiments we provide more comprehensive experiments and justify the advantages of our assessment. ## conclusion this paper investigates the limitations of common assessments for lm uncertainty/confidence measures. we develop an alternate framework, termed rank-calibration, to assess their quality. our approach does not require binarizing correctness at ad hoc thresholds and is compatible with uncertainty measures taking values in any output range. we experimentally show the broad applicability and the granular interpretability of our method, and provide a comprehensive robustness analysis. future directions include developing uncertainty measures with guaranteed rank-calibration and enhancing generative pipelines of lms (e.g., the retrieval-augmented generation) with rank-calibrated measures.