Title: Beyond Confidence: Reliable Models Should Also Consider Atypicality
Venue: NeurIPS 2023
Accepted: yes
Keywords: trustworthy machine learning

reliable machine learning

uncertainty
Abstract: While most machine learning models can provide confidence in their predictions, confidence is insufficient to understand a prediction's reliability. For instance, the model may have a low confidence prediction if the input is not well-represented in the training dataset or if the input is inherently ambiguous. In this work, we investigate the relationship between how atypical~(rare) a sample or a class is and the reliability of a model's predictions. We first demonstrate that atypicality is strongly related to miscalibration and accuracy. In particular, we empirically show that predictions for atypical inputs or atypical classes are more overconfident and have lower accuracy. Using these insights, we show incorporating atypicality improves uncertainty quantification and model performance for discriminative neural networks and large language models. In a case study, we show that using atypicality improves the performance of a skin lesion classifier across different skin tone groups without having access to the group attributes. Overall, we propose that models should use not only confidence but also atypicality to improve uncertainty quantification and performance. Our results demonstrate that simple post-hoc atypicality estimators can provide significant value.

Beyond Confidence: Reliable Models Should Also Consider Atypicality

Abstract

While most machine learning models can provide confidence in their predictions, confidence is insufficient to understand a prediction's reliability. For instance, the model may have a low confidence prediction if the input is not well-represented in the training dataset or if the input is inherently ambiguous. In this work, we investigate the relationship between how atypical (rare) a sample or a class is and the reliability of a model's predictions. We first demonstrate that atypicality is strongly related to miscalibration and accuracy. In particular, we empirically show that predictions for atypical inputs or atypical classes are more overconfident and have lower accuracy. Using these insights, we show incorporating atypicality improves uncertainty quantification and model performance for discriminative neural networks and large language models. In a case study, we show that using atypicality improves the performance of a skin lesion classifier across different skin tone groups without having access to the group attributes. Overall, we propose that models should use not only confidence but also atypicality to improve uncertainty quantification and performance . Our results demonstrate that simple post-hoc atypicality estimators can provide significant value. 1

1 Introduction

Typicality is an item's resemblance to other category members [RM75]. For example, while a dove and a sparrow are typical birds, a penguin is an atypical bird. Many works from cognitive science (e.g., [Rip89, RSS73, MP80]) suggest that typicality plays a crucial role in category understanding. For instance, humans have been shown to learn, remember, and refer to typical items faster [Mur04]. Similarly, the representativeness heuristic is the tendency of humans to use the typicality of an event as a basis for decisions [TK74]. This cognitive bias is effective for making swift decisions, but it can lead to poor judgments of uncertainty. For instance, the likelihood of typical events can be overestimated [TK74] or uncertainty judgments can be inferior for atypical events [TK92].

While it is hard to quantify the uncertainty of human judgments, machine learning models provide confidence in their predictions. However, confidence alone can be insufficient to understand the reliability of a prediction. For instance, a low-confidence prediction could arise from an ambiguity that is easily communicated, or due to the sample being underrepresented in the training distribution.

Typical

Atypical

Extrapolation

ES

Reliable

Y:Golden retriever

Y:Goldenretriever

Y: King penguin

:King penguin

Y:Golden retriever

Y:Golden retriever

Y:Hog

：Hog

：Piggy bank

Y:Hog

Y:Ashcan

Y:Hog

+

+

+

GUVEGAM!LAPTALISTPTC

+

Untrustworthy

Ambiguous

：Book jacket

Y:Hog

Y:Comic book

Y:Hog

Y:Golden retriever

Y:Irish setter

Y:Goldenretriever

Y: Jigsaw puzzle

Y.King penguin

Y:Web site

Arabian camel

Y:Hog

Similarly, a high-confidence prediction could be reliable or miscalibrated. Our main proposal is that models should quantify not only the confidence but also the atypicality to understand the reliability of predictions or the coverage of the training distribution. However, many machine learning applications rely on pretrained models that solely provide confidence levels, devoid of any measure of atypicality.

Contributions: To support our position, we use a simple formalization of atypicality estimation. With the following studies, we show that by using simple atypicality estimators, we can:

Understand Prediction Quality: Calibration is a measure that assesses the alignment between predicted probabilities of a model and the true likelihoods of outcomes [GR07]. Neural networks [GPSW17] or even logistic regression [BMWX21a] can be miscalibrated out-of-the-box. Here, we argue that using atypicality can give insights into when a model's confidence is reliable. Through theoretical analysis and extensive experimentation, we demonstrate that atypicality results in lower-quality predictions. Specifically, we show that predictions for atypical inputs and samples from atypical classes are more overconfident and have lower accuracy .

Improve Calibration and Accuracy: Recalibration methods offer some mitigation to miscalibration [GPSW17] by adjusting a probabilistic model. We show that models need different adjustments according to the atypicality of inputs and classes, and atypicality is a key factor in recalibration. In light of these findings, we propose a simple method: Atypicality-Aware Recalibration . Our recalibration algorithm takes into account the atypicality of the inputs and classes and is simple to implement. We show that complementing recalibration methods with atypicality improves uncertainty quantification and the accuracy of predictors. Further, in a case study for skin lesion classification, we show that atypicality awareness can improve performance across different skin-tone subgroups without access to group annotations.

Improve Prediction sets: An alternative approach to quantify uncertainty is to provide prediction sets that contain the label with high probability [ABMJ20]. Here, we investigate existing methods with atypicality and show that prediction sets could underperform for atypical or low-confidence samples. By using atypicality, we demonstrate the potential for improving prediction sets.

Overall, we propose that models should also consider atypicality, and we show simple- and easy-to-implement atypicality estimators can provide significant value .

2 Interpreting Uncertainty with Atypicality

Motivation: In many machine learning applications, we have access to a model's confidence, which aims to quantify the likelihood that a prediction will be accurate. In classification, model output is a probability distribution over classes and confidence is the predicted probability of the top class, i.e. max y ˆ P ( Y = y | X = x ) . In practical scenarios, confidence is the primary tool used to evaluate the reliability of a prediction where higher confidence is associated with better predictions. However, the uncertainty in confidence can stem from different sources that require different treatment [MKvA + 21].

Here, we call a prediction reliable if it is high-confidence and well-calibrated. High confidence could be reliable or miscalibrated, and low confidence could be due to ambiguity or rare inputs. We propose that atypicality provides a natural way to understand reliability when combined with confidence. A sample is called typical if it is well-represented in the previously observed samples, e.g., an image of a dog that is similar to other dogs in the training data. However, if the image is unlike any other seen during training, it is atypical. We argue that atypicality can help us interpret a prediction's reliability. Below we categorize samples and predictions according to atypicality and confidence in four quadrants (Figure 1).

High-confidence and representative: Reliable predictions often fall within the Reliable Quadrant , which includes typical, high-confidence samples. These samples are well-represented in the training dataset (typical), thus we expect the high-confidence prediction to be reliable. For instance, the first image on the top left (Figure 1) is a typical golden retriever and the model makes a reliable prediction.

High-confidence yet far from the support: Having high-confidence does not always indicate reliability. If the sample does not have support in the training distribution, the confidence could be miscalibrated. Such samples lie in the Extrapolation Quadrant which contains atypical, highconfidence samples. For instance, the second image in the top right of Figure 1 is a toy hog and the model has not seen similar ones during training.

Low confidence due to ambiguity: In contrast, low confidence could also be reliable when it correctly reflects an ambiguity. Such samples are in the Ambiguous Quadrant that contains typical, low-confidence samples. These are typical since they may represent multiple classes; yet, due to ambiguity, the model's confidence is low. For instance, the second image in the bottom left of Figure 1 can both be a hog and a comic book.

Low confidence and rare: For samples that are not well-represented in training data, we expect to have low-quality predictions. Untrustworthy Quadrant comprises atypical, low-confidence samples that can include extremely rare subgroups, for which we expect miscalibration and lower accuracy. For example, the image in Figure 1 bottom right is an origami hog that was not seen in training.

These examples suggest that relying solely on confidence does not provide a complete understanding of the reliability of the predictions, and we can use atypicality to interpret and improve reliability.

Formalizing Atypicality: Atypicality here is defined with respect to the training distribution. Informally, an input or a class is atypical if it is not well-represented in the training distribution. For instance, if there are no or limited similar examples to an input, it can be called atypical. Note that this notion is not restricted to being 'out-of-distribution' [HG16a], since in-distribution groups could also be atypical or rare, and our goal is to perform reliably for the entire spectrum.

Formally, let X ∈ R d be the random variable denoting features and Y ∈ Y = { 1 , 2 , ..., C } denote the class, where we focus on classification.

Definition 2.1 (Input Atypicality) . We define the atypicality of the input x as 2

We use the logarithm of the class-conditional densities due to high dimensionality and density values being close to zero. Intuitively, for a dog image x , if P ( X = x | Y = dog ) has a low value, we call x an atypical dog image. Overall, if a ( x ) is high, then we call x an atypical input. Specifically, if an input is not typical for any class, then it is atypical with respect to the training distribution. Similarly, we can also use marginal density, P ( X = x ) , or distance 3 to quantify atypicality.

Similarly, the notion of atypical (rare) classes is prevalent in imbalanced classification [CWG + 19, ZCLJ21]. Ensuring reliable performance for atypical classes can be safety-critical, e.g., for a rare presence of dangerous melanoma [DVN + 22]. We define class atypicality in the following:

Definition 2.2 (Class Atypicality) . For a class y , atypicality of a class is defined as

Estimating Atypicality for Discriminative Models: Quantifying input atypicality requires access to the class-conditional / marginal distributions. In practice, for neural networks trained for classification, these distributions are unavailable and we need to perform the estimation. This estimation can be challenging if the dimensionality is large, or the data is unstructured, requiring assumptions about the distributions. Prior works [MKvA + 21, LLLS18] showed that Gaussian Mixture Models (GMMs) in the embedding space of neural networks can be used to model these distributions.

In experiments, we use Gaussians with shared covariance, i.e. ˆ P ( X = x | Y = c ) ∼ N (ˆ µ c , ˆ Σ) , to estimate input atypicality. We perform the estimation in the penultimate layer of neural networks used to make predictions, using maximum-likelihood estimation with samples from the training data. We explore other metrics, such as k -Nearest Neighbors distance. We give implementation details and results with different metrics in Appendix A.1. With these estimators, atypicality estimation is cheap and can run on a CPU. Our goal is to show that simple estimators can already reap large benefits. Our framework is flexible and exploring more sophisticated estimators is a topic for future work.

Atypicality for LLMs: LLMs are increasingly used for classification [BMR + 20]. Modern LLMs are autoregressive models that compute a marginal distribution, ˆ P LLM ( X ) . We compute the negative loglikelihood of a prompt or a label and use this as an atypicality metric, i.e. a X ( x ) = -log ˆ P LLM ( x ) , a Y ( y ) = -log ˆ P LLM ( y ) . Similar to the discriminative setting, atypicality here aims to quantify whether a prompt is well-represented in the training data. A larger value for a ( X ) would imply that a prompt is more atypical. Below, we present typical and atypical prompts for the AGNews dataset:

Classify the news articles into the categories of World, Sports, Business, and Technology. Article: Safin tallest obstacle to host #39;s patriotic games hope AS tennis fans go, Houston #39;s Jim #39;Mattress Mack #39; McIngvale is very rich, extremely forthright, exceedingly patriotic and unflinchingly Republican. Answer: Atypicality: 353 . 45 , Percentile: %94 . 5 Classify the news articles into the categories of World, Sports, Business, and Technology. Article: Delta Air Lines Prepares Chapter 11 Filing Delta Air Lines Inc. could file for Chapter 11 bankruptcy protection as soon as next week, a source familiar with the matter said yesterday. Answer: Atypicality: 171 . 50 . Percentile: %0 . 9

3 Understanding the Prediction Quality with Atypicality

In this section, we show how our framework can be applied to understand the quality of predictions. Experimental Setup: We investigate three classification settings across a range of datasets:

Balanced Supervised Classification: We use ResNet18-50-152 [HZRS16], WideResNet28 [ZK16], RoBERTa [LOG + 19] trained on ImageNet [DDS + 09], CIFAR10,100 [Kri09], MNLI [WNB18] respectively.

Imbalanced Supervised Classification: We use ResNet18, ResNext50, ResNet152 trained on CIFAR-LT, ImageNet-LT and Places365-LT where models and data are mostly from [ZCLJ21, MKS + 20]. Note that all of the test and validation sets have balanced class distributions.

Classification with LLMs: We use open-source Alpaca7B [TGZ + 23] on IMDB [MDP + 11], TREC [LR02], and AG News [ZZL15] datasets with the prompts from [ZWF + 21].

Details on datasets, models, and prompts are in Appendix B. Our experiments were run on a single NVIDIA A100-80GB GPU. We report error bars over 10 random calibration/test splits.

3.1 Atypicality is Correlated with Miscalibration

We first explore the importance of atypicality to understand model calibration. Calibration quantifies the quality of a probabilistic model [GR07]. Informally, a model is considered perfectly calibrated if all events that are predicted to occur P % of the time occur P % of the time for any P ∈ [0 , 100] .

For the sake of simplicity, consider a binary classification problem where the predictor is ˆ P : X → [0 , 1] . We quantify miscalibration with Calibration Error (CE):

It is computationally infeasible to calculate the above expectation with the conditional probability P ( Y | ˆ P ( X ) = p ) . In practice, we use a binned version of this quantity, Expected Calibration Error (ECE) [NCH15, GPSW17], to estimate CE. See Appendix C.1 for a formal definition.

1

2

3

4

5

Input Atypicality Quantile

0.02-0.22

0.22-0.41

0.41-0.66

0.66-0.94

0.94-1.00

Confidence

0.07

0.08

0.08

0.08

0.09

0.08

0.12

0.14

0.15

0.17

0.05

0.12

0.19

0.22

0.27

0.04

0.12

0.20

0.26

0.35

0.03

0.06

0.09

0.12

0.15

ResNet18 on ImageNet + ImageNet-R

Confidence(

Bm

)-Accuracy(

Bm

)

(a)

0.5

1.0

0.03

0.05

0.08

ECE

ResNet50 - ImageNet

0.5

1.0

Input Atypicality Quantile

0.75

0.80

0.85

(b)

0.5

1.0

0.10

0.20

0.30

ECE

ResNext50 - ImageNet-LT

0.5

1.0

Class Atypicality Quantile

0.25

0.50

(c)

0.5

1.0

0.03

0.05

ECE

0.5

1.0

Input Atypicality Quantile

0.75

0.80

0.85

(d)

Here, we aim to examine the relationship between model calibration and atypicality. Given any K > 1 , we consider the quantiles of a ( X ) , a 1 , a 2 , . . . , a K +1 such that P ( a ( X ) ∈ ( a k , a k +1 ]) = 1 /K for k ∈ [ K ] . For imbalanced classification problems, we compute the quantiles using the class atypicality. Specifically, we investigate the atypicality-conditional calibration error ECE [ ˆ P | a ( X ) ∈ ( a k , a k +1 ]] , i.e., the expected calibration error of an input that falls within the atypicality quantile k .

Atypical Examples are Poorly Calibrated: In Figure 2a, we show the distribution of miscalibration where each bin within the grid contains the intersection of the corresponding confidence and atypicality quantiles. We observe that within the same confidence range, predictions for atypical points have lower accuracies and are more overconfident. In other words, predictions in the Extrapolation or Untrustworthy regions are more miscalibrated than the ones in the typical regions.

In Figure 2b, we split inputs into quantiles according to atypicality and compute the ECE and Accuracy for each group. Results show a monotonic relationship between atypicality and ECE or Accuracy across the three settings. Specifically, we see that predictions for atypical inputs or samples from rare classes are more miscalibrated and have lower accuracy. For samples from rare classes, the model overpredicts the probabilities of the typical class, hence we have overconfidence and low accuracy. Appendix C.3, and §4 present figures and tables for all model and dataset pairs.

3.2 Theoretical Analysis: Characterizing Calibration Error with Atypicality

We characterize how calibration error varies with atypicality in a tractable model that is commonly used in machine learning theory [BMWX21a, BMWX21b, ZDKZ22, CLKZ22]. Our theoretical analysis further supports our empirical findings.

Data Generative Model: We consider the well-specified logistic model for binary classification with Gaussian data, where Y ∈ {-1 , 1 } and the P ( Y = 1 | X ) is defined by the sigmoid function:

Where I d denotes the d -dimensional identity matrix, β ∗ is the ground truth coefficient vector, σ ( x ) = 1 / (1 + e -x ) , and we have i.i.d. observations { ( x i , y i ) } n i =1 sampled from the above distribution.

The Estimator: We focus on studying the solution produced by minimizing the logistic loss

For k ∈ {-1 , 1 } , ˆ P k ( x ) is an estimator of P ( y = k | x ) , with the form ˆ P k ( x ) = 1 e -k · ˆ β ⊤ x +1 .

Calibration: We consider all x where P 1 ( x ) > 1 / 2 , as P 1 ( x ) ≤ 1 / 2 can be analyzed similarly by symmetry (see Appendix G). For u ∈ (1 / 2 , 1) , the signed calibration error at a confidence level u is

We want to show that when X is atypical, i.e., when a ( X ) := ∥ X ∥ 2 / 2 is larger 5 , the accuracy P ( Y = 1 | ˆ P 1 ( X ) = u ) would be generally smaller than the confidence u (over-confidence).

Theorem 3.1. Consider the data generative model and the learning setting above. For any K > 1 , suppose we consider the quantiles of a ( X ) , a 1 , a 2 , ..., a K , a K +1 such that P ( a ( X ) ∈ ( a k , a k +1 ]) = 1 /K for k ∈ [ K ] . We assume ∥ β ∗ ∥ ≤ c 0 , and d/n = κ , for some sufficiently small c 0 . Then, for sufficiently large n , for k = 2 , . . . , K , we have

That is, the resulting classifier is over-confident, and the level of over-confidence becomes larger when the data is more atypical (with larger a ( X ) ). Further, the gap becomes larger for smaller sample sizes n . The proof of the theorem is in Appendix G.2 and builds on the results from [BMWX21a, SC19].

4 Using Atypicality to Improve Recalibration

Here, we show how atypicality can complement and improve post-hoc calibration. In §2, we observed that predictions for atypical inputs and samples from atypical classes are more overconfident with lower accuracy. We next show that taking input and class atypicality into account improves calibration.

4.1 Parametric Recalibration: Different Groups Need Different Temperatures

Temperature scaling (TS), a single parameter variant of Platt Scaling [P + 99], is a simple recalibration method that calibrates the model using a single parameter. The predictor is of the form

where ˆ P ( Y | X ) is the model that takes an input and outputs scores/logits, and τ is the temperature parameter. In practice, τ is optimized using a calibration set to minimize a proper scoring rule [GR07, BW19] such as the cross-entropy loss.

To understand the behavior of TS with respect to atypicality, we separately perform TS on points grouped according to the atypicality quantiles. Let us denote the temperature fitted to the quantile covering a ( X ) ∈ ( a k -1 , a k ] by τ a k . In Appendix Figure 10 we observe an increasing relationship between a k and τ a k . Different atypicality groups need different adjustments, and more atypical groups need larger temperatures. This suggests that being atypicality-aware can improve calibration. While a single temperature value improves average calibration, it may hurt certain groups.

4.2 Atypicality-Aware Recalibration

We showed that predictions are more reliable when the input is typical. However, predictions are less reliable for atypical inputs, and we may need further revision. An analogy can be drawn to decisionmaking literature where opinions of individuals are combined with geometric averaging weighted by their expertise [FP98, AR89]. Analogously, we propose Atypicality-Aware Recalibration (AAR) a method designed to address the reliability issues identified in dealing with atypical inputs:

where ψ ( a ( X )) is a function of input atypicality, S Y is a tunable score for class Y , Z ( X ) is the normalization term. Intuitively, when the input is typical, we trust the model confidence; otherwise, we use a score for the given class estimated from the calibration set. Note that this form simplifies to

(a)

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.02

0.04

ECE

ResNet18 - ImageNet

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.03

0.05

0.08

ECE

ResNet50 - ImageNet

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.03

0.05

ECE

RoBERTa - MNLI

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.00

0.10

ECE

WideResNet28 - CIFAR10

Uncalibrated

TS

AdaTS

AAR(Ours)

(b)

0.25

0.50

0.75

1.00

Class Atypicality Quantile

0.00

0.20

ECE

0.0

0.5

0.44

0.44

0.44

0.50

0.25

0.50

0.75

1.00

Class Atypicality Quantile

0.10

0.20

ECE

0.0

0.5

0.47

0.47

0.47

0.48

0.25

0.50

0.75

1.00

Class Atypicality Quantile

0.20

0.40

ECE

0.0

0.2

0.30

0.30

0.30

0.38

0.25

0.50

0.75

1.00

Class Atypicality Quantile

0.20

0.40

ECE

0.0

0.5

0.45

0.45

0.45

0.50

ResNext50 - ImageNet-LT

ResNext50(+P.B.) - ImageNet-LT

ResNet152 - Places365-LT

ResNet18 - CIFAR100-LT

Uncalibrated

TS

AdaTS

AAR(Ours)

(c)

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.20

0.40

ECE

0.0

0.5

0.65

0.65

0.54

0.75

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.25

0.50

ECE

0.0

0.5

0.64

0.64

0.60

0.80

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.05

0.10

0.15

ECE

0.0

0.5

0.82

0.82

0.82

0.91

Alpaca7B - AG News

Alpaca7B - TREC

Uncalibrated

TS

CF

AAR(Ours)

where we subsume (1 -ψ ( a ( X )) into ϕ ( a ( X )) . We give a simple interpretation of this form: the multiplicative term is an atypicality-dependent temperature, and the additive term is a class-dependent correction where exp( S Y ) can be considered to induce a correction distribution over classes estimated from the calibration set.

Intuitively, when ψ ( a ( X )) = 0 , the output reduces to a fixed distribution over classes that was estimated using the calibration set. This distribution can be seen to induce a prior probability over classes, and ψ controls the tradeoff between this prior and the model's predictive distribution. As the point becomes more typical, this distribution is closer to the model's predictive distribution. In Appendix Figure 11, we show how these values behave with class atypicality. We find that rare classes require larger positive corrections with larger S Y .

Implementation Details: Following TS, we minimize the cross-entropy loss on a calibration set. With the temperature-atypicality relationship observed in Figure 10 we choose to instantiate the multiplicative factor as a quadratic function, where ϕ ( a ( X )) = c 2 a ( X ) 2 + c 1 a ( X ) + c 0 and in total we have |{ S 1 , .., S |Y| , c 0 , c 1 , c 2 }| = |Y| + 3 interpretable parameters. Once the embeddings and logits are computed, AAR runs on a CPU in under 1 minute for all experimented settings.

Similar to our adaptive interpretation, a concurrent work, Adaptive Temperature Scaling (AdaTS) [JPL + 23], uses temperature scaling where the temperature is parameterized by a Variational Autoencoder(VAE) [KW13] and a multi-layer perceptron on top of the VAE embeddings. In the below experiments, we give results with AdaTS as a baseline when applicable.

Unknown 1&2

3&4

5&6

Skin T one

0.0

0.1

ECE

ResNet18

Unknown 1&2

3&4

5&6

Skin T one

0.50

0.75

ResNet18

Unknown 1&2

3&4

5&6

Skin T one

0.0

0.1

ECE

ResNet34

Unknown 1&2

3&4

5&6

Skin T one

0.50

0.75

ResNet34

TS

Skin T one Conditional

AAR(Ours)

For Balanced Supervised Classification , in Figure 3a, we observe that being atypicality aware improves recalibration across all groups. We perform comparably to AdaTS, where the temperature function has in the order of millions of parameters, whereas AAR has only |Y| +3 parameters.

In Imbalanced Supervised Classification (Figure 3b), our algorithm not only provides better calibration rates across all classes but also improves overall accuracy. Note that only our method can change accuracy (due to the additive term), and it performs better than other baselines in terms of ECE across all classes. Further, the second column shows using Progressive Balancing [ZWF + 21] in training, showing that our post-hoc method can complement methods that modify training procedures.

For Classification with LLMs , we add an LLM calibration baseline Content-Free Calibration (CF) [ZWF + 21]. We cannot use AdaTS as the embeddings are not fixed in size. In Figure 3c, we see AAR has better calibration and accuracy across the three datasets. Namely, by adjusting the LLM output using the LLM atypicality, we can adjust the probabilities to increase the prediction quality.

4.3 Case Study: Fairness through Atypicality-Awareness

Machine learning models reportedly have performance disparity across subgroups [BHN17] due to factors such as varying sample size or noise levels [CJS18]. For instance, skin lesion classifiers can exhibit performance disparity across different skin tones [DVN + 22]. Fitzpatrick17k [GHS + 21] is a dataset of clinical images with Fitzpatrick skin tone annotations between 1-to-6, where a larger number means darker skin tones, and when annotators do not agree, it is labeled as 'Unknown'. We explore the classification problem with 9 classes indicating the malignancy and the type of skin condition, using a ResNet18/34 pretrained on ImageNet and finetuned on this task (See Appendix F).

When the goal is to improve performance across groups, one can use group annotations and optimize performance within each group [HJKRR18, KGZ19]. Here, we investigate how complementing recalibration with atypicality can improve prediction quality across all groups without group annotations . For comparison, we perform 3 recalibration methods: TS, AAR, and Skin-Tone Conditional TS which calibrates the model individually for each skin-tone group with TS. Since the skin-tone conditional calibration uses group attributes, ideally it should act as an oracle.

In Figure 4, we give the Accuracy and ECE analyses where AAR improves performance across all groups. For instance, the worst-group Accuracy (0.69) or ECE (0.072) with AAR is close to the best-group Accuracy (0.63) or ECE (0.062) with the other two methods. Overall, our findings suggest that Atypicality-Awareness can complement fairness-enforcing methods, and improve performance even when the group annotations are unavailable. We hypothesize that with AAR, we can perform better than using supervised group attributes since groups may not have sufficient sample size in the calibration set (131, 1950, 1509, 555 samples for Unknown, 1&2, 3&4, and 5&6 respectively), and we can leverage atypicality to offer some mitigation. Further investigating how to leverage atypicality to improve fairness and factors affecting performance disparities is a promising direction for future work [CJS18].

5 Improving Prediction Sets with Atypicality

Conformal Prediction [SV08, AB21] is a framework that assigns a calibrated prediction set to each instance. The goal is to find a function C : X → 2 Y that returns a subset of the label space such that Y ∈ C ( X ) with high probability. The framework aims to guarantee marginal coverage , i.e., P ( Y ∈ C ( X )) ≥ 1 -α , for a choice of α . We investigate two conformal calibration methods,

Adaptive Prediction Sets (APS) [RSC20] and Regularized APS (RAPS) [ABMJ20]. Let π ( X ) be the permutation of the label set that sorts ˆ P ( Y = c | X ) , i.e. the predicted probabilities for each class c after TS. The prediction sets are produced by the function C ( x ) = { y : s ( x, y ) ≤ ˆ q } , and these methods fit the threshold ˆ q for a choice of the scoring function. APS uses the cumulative sum of the predicted probabilities s ( x, y ) = ∑ c j =1 ˆ P ( Y = j | X ) , where y = π c ( X ) . Intuitively, if the model was perfectly calibrated, we would have expected to have ˆ q = 1 -α . Similarly, RAPS builds on the idea that tail probabilities are noisy and regularizes the number of samples in the prediction set.

Building on our ideas in the previous sections we implement Atypicality-Aware prediction sets, namely AA-APS and AA-RAPS in the following way: We first group points according to their confidence and atypicality quantiles. A group G here is defined using 4 thresholds, namely G = x : ( l ( G ) a < q a ( x ) ≤ h ( G ) a ) ∧ ( l ( G ) c < q c ( x ) ≤ h ( G ) c ) where q a ( x ) and q c ( x ) denote the atypicality and confidence quantiles for the sample x , l ( G ) a and h ( G ) a denote the atypicality lower and upper bounds for group G , and l ( G ) c and h ( G ) c denote the confidence lower and upper bounds for group G . Using a calibration set, these bounds are simply determined by the quantiles of confidence and atypicality statistics. Then, we fit separate thresholds ˆ q G for each group's prediction sets with APS or RAPS as subroutines. This allows us to have an adaptive threshold depending only on the atypicality and confidence of predictions.

1

2

3

4

5

6

Input Atypicality Quantile

0.00-0.45

0.45-0.69

0.69-0.89

0.89-0.97

0.97-1.00

1.00-1.00

Confidence

0.98

0.99

0.98

0.97

0.96

0.95

0.96

0.98

0.97

0.96

0.96

0.96

0.95

0.96

0.95

0.98

0.98

0.96

0.96

0.96

0.95

0.99

0.99

0.99

0.99

1.00

0.99

0.94

0.94

0.94

0.94

0.92

0.93

0.94

0.92

0.90

APS(0.95) Coverage

1

2

3

4

5

6

0.96

0.96

0.95

0.95

0.95

0.95

0.95

0.96

0.95

0.95

0.95

0.95

0.95

0.95

0.95

0.95

0.96

0.95

0.96

0.95

0.95

0.97

0.98

0.98

0.96

0.96

0.97

0.97

0.99

0.99

0.99

0.99

1.00

0.99

0.94

0.94

AA-APS(0.95) Coverage

1

2

3

4

5

6

79.6

115.5

136.2

126.5

111.9

87.0

16.3

24.9

27.1

28.8

30.3

23.1

9.4

9.5

12.7

10.1

11.5

8.7

1.7

1.9

2.1

2.3

2.1

1.9

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

APS(0.95) Set Size

1

2

3

4

5

6

48.3

48.3

61.1

76.6

89.1

91.2

12.4

19.8

18.3

26.6

34.0

32.6

7.6

9.5

10.3

9.9

14.3

13.3

1.7

2.2

4.8

4.5

9.3

15.6

1.0

1.0

1.1

1.3

1.9

3.5

1.0

1.0

1.0

1.0

1.0

1.0

AA-APS(0.95) Set Size

1

2

3

4

5

6

0.00-0.45

0.45-0.69

0.69-0.89

0.89-0.97

0.97-1.00

1.00-1.00

0.97

0.97

0.96

0.95

0.99

0.98

0.97

0.98

0.96

0.99

0.99

0.99

0.98

0.98

0.97

1.00

0.99

0.99

0.99

0.98

0.98

0.99

1.00

0.99

0.99

1.00

0.99

0.93

0.89

0.82

0.82

0.76

0.79

0.94

0.92

0.94

RAPS(0.95) Coverage

1

2

3

4

5

6

0.96

0.95

0.97

0.95

0.96

0.96

0.96

0.96

0.95

0.95

0.97

0.96

0.95

0.96

0.95

0.96

0.95

0.96

0.98

0.98

0.97

0.96

0.97

0.97

0.99

0.99

0.99

0.99

1.00

0.99

0.94

0.94

0.94

0.94

0.94

0.94

AA-RAPS(0.95) Coverage

1

2

3

4

5

6

8.7

9.0

9.3

9.3

9.2

9.1

7.4

7.7

7.7

7.8

7.9

7.7

7.1

7.2

7.4

7.6

7.5

7.0

6.7

6.7

6.7

6.7

6.8

6.5

4.9

4.8

4.9

4.8

4.8

4.4

1.2

1.1

1.1

1.1

1.1

1.1

RAPS(0.95) Set Size

1

2

3

4

5

6

17.9

24.9

32.6

49.6

64.5

73.5

4.5

6.4

10.3

12.8

17.2

19.1

3.3

4.4

4.8

6.3

7.0

8.9

1.5

1.9

3.9

2.8

4.0

5.5

1.0

1.0

1.2

1.1

1.2

2.6

1.0

1.0

1.0

1.0

1.0

1.0

AA-RAPS(0.95) Set Size

In Figure 5, we provide the coverage plots for APS and RAPS in the first and third columns. Even though marginal coverage is satisfied, models do not satisfy conditional coverage for atypical inputs or low-confidence predictions. We observe that being Atypicality-Aware improves coverage across otherwise underperforming groups. Further, AA-APS has lower set sizes on average than APS ( 15 . 6 vs 21 . 3 ). While RAPS has a lower average set size than AA-RAPS ( 4 . 2 vs 9 . 1 ) AA-RAPS has smaller set sizes for high-confidence samples, whereas a larger set size for low-confidence samples where the coverage is not met for RAPS. In Appendix D.3, we provide the same analysis for ResNet18,50,152 at different coverage levels along with analyzing the performance in the Confidence and Atypicality dimensions individually. For instance in Figure 8, we observe that RAPS and APS do not satisfy coverage for high atypicality regions, even when averaged across different confidence levels.

6 Additional Related Work

Uncertainty and Atypicality: [MKvA + 21, PBC + 20] use density estimation to disentangle epistemic and aleatoric uncertainty. Following this, they show improvements in active learning and OOD detection [LLLS18]. We note that our goal is not this disentanglement (e.g. Untrustworthy quadrant can have both aleatoric or epistemic uncertainty), or Ambiguity could be due to a lack of features or noise. [LLP + 20] propose the related notion of distance awareness, and that it leads to better uncertainty quantification. They offer architecture and training modifications whereas we analyze existing models using our framework including imbalanced and LLM settings, and propose simple and post-hoc approaches. 'OOD' [HG16b] or 'anomaly' [HMD18] notions are tied to atypicality, yet our goal is not to make a binary distinction between 'in' or 'out'. We argue that in-distribution samples could also be atypical (e.g. rare groups), and the goal is to perform reliably in the entire spectrum. Other works with an atypicality notion include bounding calibration of groups by the

excess risk [LSH19], miscalibration under distribution shifts [OFR + 19], uncertainty in Gaussian Processes [Ras04], forgetting time for rare examples [MGLK22], the poor performance of groups with lower sample sizes [CJS18], energy-based models improving calibration [GWJ + 20], relating perplexity to zero-shot classification performance for LLMs [GIB + 22], grouping loss and local definitions of miscalibration [PLMV23], the relationship between active learning and atypicality [HDW22], sample size as a factor for subgroup performance disparity [CJS18]. [PBZ21] provide insightful discussion around the nature of softmax confidence, and here we show that its reliability depends on the atypicality of the input. Our new findings include showing that predictions for atypical samples are more miscalibrated and overconfident, and atypicality awareness improves prediction quality. Overall, while there are other relevant notions in the literature, our distinct goal is to show that post-hoc atypicality estimation and recalibration is a simple yet useful framework to understand and improve uncertainty quantification that complements existing methods.

Recalibration and Conformal Prediction: There is a rich literature on recalibration methods and prediction sets: TS [GPSW17], Platt Scaling [P + 99], conformal calibration [SV08, ABMJ20] among many. [LLC + 22, RBSC19, BYR + 21, BGJ + 22] make a relevant observation, showing that the coverage of conformal prediction is not equal across all groups. They propose group conformal calibration, which requires group labels whereas our proposal is unsupervised and does not depend on any attribute information. Concurrent work [JPL + 23] explores AdaTS, where they train a separate VAE and MLP to produce an adaptive temperature. However, our parameterization of temperature has 3 parameters and is interpretable.

7 Conclusion

Atypicality offers a simple yet flexible framework to better understand and improve model reliability and uncertainty. We propose that pretrained models should be released not only with confidence but also with an atypicality estimator. While there are other relevant notions in the literature, our main goal is to show that atypicality can provide a unifying perspective to discuss uncertainty, understand individual data points, and improve fairness. Here we focus on classification problems; it would be interesting to extend atypicality to regression and generation settings. Furthermore, we would like to extend the theoretical analysis to more general settings, as our empirical results demonstrate that the observed phenomena hold more broadly.

Acknowledgments

We would like to thank Adarsh Jeewajee, Bryan He, Edward Chen, Federico Bianchi, Kyle Swanson, Natalie Dullerud, Ransalu Senanayake, Sabri Eyuboglu, Shirley Wu, Weixin Liang, Xuechen Li, Yongchan Kwon, Yu Sun, and Zach Izzo for their comments and suggestions on the manuscript. Linjun Zhang's research is partially supported by NSF DMS-2015378. Carlos Ernesto Guestrin is a Chan Zuckerberg Biohub - San Francisco Investigator.

References

[AB21] Anastasios N Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511 , 2021. 8, 23

[ABMJ20] Anastasios Angelopoulos, Stephen Bates, Jitendra Malik, and Michael I Jordan. Uncertainty sets for image classifiers using conformal prediction. arXiv preprint arXiv:2009.14193 , 2020. 2, 9, 10, 23, 24

[AR89] J´ anos Acz´ el and Fred S Roberts. On the possible merging functions. Mathematical Social Sciences , 17(3):205-243, 1989. 6

[BGJ + 22] Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, and Aaron Roth. Practical adversarial multivalid conformal prediction. arXiv preprint arXiv:2206.01067 , 2022. 10

[BHN17] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness in machine learning. Nips tutorial , 1:2, 2017. 8

[BMR + 20] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020. 4

[BMWX21a] Yu Bai, Song Mei, Huan Wang, and Caiming Xiong. Don't just blame overparametrization for over-confidence: Theoretical analysis of calibration in binary classification. In International Conference on Machine Learning , pages 566-576. PMLR, 2021. 2, 5, 6, 27, 33

[BMWX21b] Yu Bai, Song Mei, Huan Wang, and Caiming Xiong. Understanding the undercoverage bias in uncertainty estimation. Advances in Neural Information Processing Systems , 34:18307-18319, 2021. 5

[BW19] David Bolin and Jonas Wallin. Local scale invariance and robustness of proper scoring rules. arXiv preprint arXiv:1912.05642 , 2019. 6

[BYR + 21] Noam Barda, Gal Yona, Guy N Rothblum, Philip Greenland, Morton Leibowitz, Ran Balicer, Eitan Bachmat, and Noa Dagan. Addressing bias in prediction models by improving subpopulation calibration. Journal of the American Medical Informatics Association , 28(3):549-558, 2021. 10

[CJS18] Irene Chen, Fredrik D Johansson, and David Sontag. Why is my classifier discriminatory? Advances in neural information processing systems , 31, 2018. 8, 10

[CLKZ22] Lucas Clart´ e, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborov´ a. Theoretical characterization of uncertainty in high-dimensional linear classification. arXiv preprint arXiv:2202.03295 , 2022. 5

[CWG + 19] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distribution-aware margin loss. Advances in neural information processing systems , 32, 2019. 3

[DCLT19] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pretraining of deep bidirectional transformers for language understanding. ArXiv , abs/1810.04805, 2019. 17

[DDS + 09] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009. 4, 17

[DVN + 22] Roxana Daneshjou, Kailas Vodrahalli, Roberto A Novoa, Melissa Jenkins, Weixin Liang, Veronica Rotemberg, Justin Ko, Susan M Swetter, Elizabeth E Bailey, Olivier Gevaert, et al. Disparities in dermatology ai performance on a diverse, curated clinical image set. Science advances , 8(31):eabq6147, 2022. 3, 8

[FP98] Ernest Forman and Kirti Peniwati. Aggregating individual judgments and priorities with the analytic hierarchy process. European journal of operational research , 108(1):165-169, 1998. 6

[GHS + 21] Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri. Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1820-1828, 2021. 8, 26

[GIB + 22] Hila Gonen, Srini Iyer, Terra Blevins, Noah A Smith, and Luke Zettlemoyer. Demystifying prompts in language models via perplexity estimation. arXiv preprint arXiv:2212.04037 , 2022. 10

[GPSW17] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning , pages 1321-1330. PMLR, 2017. 2, 4, 10, 23

[GR07] Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association , 102(477):359-378, 2007. 2, 4, 6

[GWJ + 20] Will Grathwohl, Kuan-Chieh Wang, Joern-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. In International Conference on Learning Representations , 2020. 10

[HDW22] Guy Hacohen, Avihu Dekel, and Daphna Weinshall. Active learning on a budget: Opposite strategies suit high and low budgets. arXiv preprint arXiv:2202.02794 , 2022. 10

[HG16a] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and outof-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 , 2016. 3

[HG16b] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-ofdistribution examples in neural networks. ArXiv , abs/1610.02136, 2016. 9

[HJKRR18] Ursula H´ ebert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the (computationally-identifiable) masses. In International Conference on Machine Learning , pages 1939-1948. PMLR, 2018. 8, 33

[HMD18] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv:1812.04606 , 2018. 9, 19

[HZRS16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016. 4

[JKGG18] Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust a classifier. Advances in neural information processing systems , 31, 2018. 3

[JPL + 23] Tom Joy, Francesco Pinto, Ser-Nam Lim, Philip HS Torr, and Puneet K Dokania. Sample-dependent adaptive temperature scaling for improved calibration. AAAI , 2023. 7, 10, 23

[KGZ19] Michael P Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box postprocessing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society , pages 247-254, 2019. 8, 33

[Kri09] Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009. 4, 17

[Kum22] Sawan Kumar. Answer-level calibration for free-form multiple choice question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 665-679, 2022. 19

[KW13] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 , 2013. 7

[LLC + 22] Charles Lu, Andr´ eanne Lemay, Ken Chang, Katharina H¨ obel, and Jayashree KalpathyCramer. Fair conformal predictors for applications in medical imaging. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 12008-12016, 2022. 10

[LLLS18] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems , 31, 2018. 4, 9

[LLP + 20] Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshminarayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in Neural Information Processing Systems , 33:7498-7512, 2020. 9

[LN89] Dong C Liu and Jorge Nocedal. On the limited memory bfgs method for large scale optimization. Mathematical programming , 45(1):503-528, 1989. 23

[LOG + 19] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019. 4

[LR02] Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics , 2002. 4

[LSH19] Lydia T Liu, Max Simchowitz, and Moritz Hardt. The implicit fairness criterion of unconstrained learning. In International Conference on Machine Learning , pages 4051-4060. PMLR, 2019. 10

[LVdMJ + 21] Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario ˇ Saˇ sko, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Cl´ ement Delangue, Th´ eo Matussi` ere, Lysandre Debut, Stas Bekman, Pierric Cistac, Thibault Goehringer, Victor Mustar, Franc ¸ois Lagunas, Alexander Rush, and Thomas Wolf. Datasets: A community library for natural language processing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . Association for Computational Linguistics, 2021. 17

[MDP + 11] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , pages 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics. 4

[MGLK22] Pratyush Maini, Saurabh Garg, Zachary C Lipton, and J Zico Kolter. Characterizing datapoints via second-split forgetting. arXiv preprint arXiv:2210.15031 , 2022. 10

[MKS + 20] Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip HS Torr, and Puneet K Dokania. Calibrating deep neural networks using focal loss. 2020. 4, 17

[MKvA + 21] Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deep deterministic uncertainty: A simple baseline. arXiv e-prints , pages arXiv-2102, 2021. 2, 4, 9, 16

[MP80] Carolyn B Mervis and John R Pani. Acquisition of basic object categories. Cognitive Psychology , 12(4):496-522, 1980. 1

[MR10] S´ ebastien Marcel and Yann Rodriguez. Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM international conference on Multimedia , pages 1485-1488, 2010. 17

[Mur04] Gregory Murphy. The big book of concepts . MIT press, 2004. 1

[NCH15] Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Twenty-Ninth AAAI Conference on Artificial Intelligence , 2015. 4

[OFR + 19] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems , 32, 2019. 10

[P + 99] John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers , 10(3):61-74, 1999. 6, 10

[PBC + 20] Janis Postels, Hermann Blum, Cesar Cadena, Roland Siegwart, Luc Van Gool, and Federico Tombari. Quantifying aleatoric and epistemic uncertainty using density estimation in latent space. arXiv preprint arXiv:2012.03082 , 2020. 9

[PBZ21] Tim Pearce, Alexandra Brintrup, and Jun Zhu. Understanding softmax confidence and uncertainty. arXiv preprint arXiv:2106.04972 , 2021. 10

[PLMV23] Alexandre Perez-Lebel, Marine Le Morvan, and Gael Varoquaux. Beyond calibration: estimating the grouping loss of modern neural networks. In The Eleventh International Conference on Learning Representations , 2023. 10

[Ras04] Carl Edward Rasmussen. Gaussian processes in machine learning. In Summer school on machine learning , pages 63-71. Springer, 2004. 10

[RBSC19] Yaniv Romano, Rina Foygel Barber, Chiara Sabatti, and Emmanuel J Cand` es. With malice towards none: Assessing uncertainty via equalized coverage. arXiv preprint arXiv:1908.05428 , 2019. 10

[Rip89] Lance J . Rips. Similarity, typicality, and categorization , page 21-59. Cambridge University Press, 1989. 1

[RM75] Eleanor Rosch and Carolyn B Mervis. Family resemblances: Studies in the internal structure of categories. Cognitive psychology , 7(4):573-605, 1975. 1

[RSC20] Yaniv Romano, Matteo Sesia, and Emmanuel Candes. Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems , 33:35813591, 2020. 9

[RSS73] Lance J Rips, Edward J Shoben, and Edward E Smith. Semantic distance and the verification of semantic relations. Journal of verbal learning and verbal behavior , 12(1):1-20, 1973. 1

[SC19] Pragya Sur and Emmanuel J Cand` es. A modern maximum-likelihood theory for highdimensional logistic regression. Proceedings of the National Academy of Sciences , 116(29):14516-14525, 2019. 6, 27, 33

[SV08] Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research , 9(3), 2008. 8, 10

[TGZ + 23] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instructionfollowing llama model. https://github.com/tatsu-lab/stanford_alpaca , 2023. 4, 18

[TJ06] MTCAJ Thomas and A Thomas Joy. Elements of information theory . WileyInterscience, 2006. 3

[TK74] Amos Tversky and Daniel Kahneman. Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. science , 185(4157):1124-1131, 1974. 1

[TK92] Amos Tversky and Daniel Kahneman. Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and uncertainty , 5(4):297-323, 1992. 1

[VGS05] Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world . Springer Science & Business Media, 2005. 23

[WDS + 20] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R´ emi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . Association for Computational Linguistics, 2020. 17

[WNB18] Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) . Association for Computational Linguistics, 2018. 4, 17

[ZCLJ21] Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia. Improving calibration for long-tailed recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16489-16498, 2021. 3, 4, 18

[ZDKZ22] Linjun Zhang, Zhun Deng, Kenji Kawaguchi, and James Zou. When and how mixup improves calibration. In International Conference on Machine Learning , pages 2613526160. PMLR, 2022. 5

[ZK16] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146 , 2016. 4

[ZKL + 16] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Antonio Torralba, and Aude Oliva. Places: An image database for deep scene understanding. arXiv preprint arXiv:1610.02055 , 2016. 18

[ZWF + 21] Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning , pages 12697-12706. PMLR, 2021. 4, 8, 18, 19

[ZZL15] Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS , 2015. 4

A Atypicality Estimation

A.1 Input Atypicality Estimation

To estimate input atypicality, we use two ways to estimate the likelihood of a point under the training distribution. First, we give methods for the discriminative models.

Fitting individual Gaussians to class conditionals Here, we follow a similar approach to [MKvA + 21]. Namely, we model the class conditionals with a Gaussian distribution, where the covariance matrix is tied across classes:

We fit the parameters µ y and Σ with maximum likelihood estimation. The reason to tie the covariance matrix is due to the number of samples required to fit the density. Namely, for a d -dimensional problem, the total number of parameters to fit individual matrices becomes O ( yd 2 ) , which results in low-quality estimates. Then, the atypicality becomes

Computing distance with k-Nearest Neighbors k-Nearest Neighbors: Similarly, we can use the nearest neighbor distance. Concretely, we use the nearest neighbor distance, a X ( x ) = d min ( x, D train ) = min x ′ ∈D train | x ′ -x | , as the atypicality metric. Alternatively, we can use different notions such as the average of k-nearest neighbors, or the distance to the kth neighbor. Below, we report the results by using the average distance to 5-nearest neighbors.

Fitting the estimators For all of the atypicality estimators, we fit the estimators using samples from the training sets and make inferences for the calibration and test sets. For instance, we use the training split of ImageNet to fit the corresponding density estimator and compute the atypicality for the samples from the validation/test split of ImageNet. All of our results using atypicality are reported for the test splits of the below datasets.

Atypicality Estimation with LLMs: For language models we simply compute the negative loglikelihood of each prompt as the atypicality metric: a X ( x ) = -log ˆ P LLM ( x ) . To define confidence, we use the logits of the language model, conditioned on the prompt. We use the logit of the first token of each class label and compute the predicted probabilities by applying softmax to the logits of each class.

A.2 Class Atypicality

To estimate class atypicality, we simply count the fraction of examples from a particular label in the training dataset. Let us have a training dataset D train = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , . . . , ( x N , y N ) } . Then, we estimate class atypicality with

A.3 Atypicality and Confidence

Are atypicality and confidence equally informative? Beyond the data perspective given in Figure 1, here we provide quantitative results to demonstrate the difference. In Figure 6, we give a grid plot where the x-axis indicates the typicality quantile of a point, and the y-axis indicates the confidence of a point. The coloring on the left indicates the accuracy within a bin split according to accuracy, and the right has the difference between average confidence and accuracy. Observe that for a specific confidence interval, larger values of typicality mean better quality probabilistic estimates and larger atypicality means more miscalibration.

ResNetl8onImageNet+ImageNet-R

ResNetl8onImageNet+ImageNet-R

Accuracy(Bm)

Confidence(Bm)-Accuracy(Bm)

[1.00-1.00]

0.99

0.99

0.99

0.99

0.98

0.98

0.98

0.97

0.97

0.98

0.96

0.96

[1.00-1.00]

0.01

0.01

0.01

0.01

0.02

0.02

0.01

0.03

0.03

0.02

0.04

0.04

0.35

[0.98-1.00)

0.97

0.96

0.95

0.96

0.94

0.94

0.94

0.92

0.92

0.91

0.88

0.88

[0.98-1.00)

0.02

0.03

0.04

0.04

0.05

0.05

0.06

0.07

0.07

0.08

0.11

0.11

0.30

[0.94-0.98)

0.93

0.94

0.92

0.91

0.90

0.88

0.88

0.83

0.82

0.84

0.80

0.78

0.8

[0.94-0.98)

0.03

0.02

0.04

0.05

0.06

0.08

0.08

0.13

0.14

0.12

0.17

0.18

[0.85-0.94)

0.87

0.87

0.84

0.85

0.82

0.76

0.76

0.73

0.74

0.72

0.68

0.55

[0.85-0.94)

0.03

0.02

0.06

0.05

0.08

0.14

0.14

0.17

0.16

0.18

0.21

0.35

0.25

dence

[0.73-0.85)

0.74

0.79

0.79

0.72

0.69

0.67

0.64

0.64

0.61

0.56

0.58

0.43

0.6

dence

@[0.73-0.85)

0.06

0.01

0.01

0.08

0.11

0.13

0.15

0.16

0.18

0.23

0.21

0.36

0.20

[0.61-0.73)

0.64

0.70

0.59

0.61

0.58

0.59

0.49

0.54

0.49

0.44

0.43

0.35

[0.61-0.73)

0.04

-0.02

0.08

0.06

0.09

0.09

0.18

0.13

0.18

0.23

0.25

0.32

Confide

[0.50-0.61)

0.56

0.54

0.48

0.45

0.47

0.46

0.44

0.39

0.36

0.34

0.33

0.33

Confic

[0.50-0.61)

-0.00

0.02

0.08

0.10

0.09

0.09

0.11

0.17

0.19

0.22

0.23

0.23

0.15

C

[0.40-0.50)

0.45

0.43

0.42

0.37

0.40

0.37

0.31

0.26

0.31

0.27

0.25

0.20

0.4O

[0.40-0.50]

0.01

0.02

0.04

0.08

0.05

0.08

0.14

0.19

0.14

0.18

0.21

0.25

[0.31-0.40)

0.33

0.34

0.29

0.31

0.27

0.24

0.24

0.23

0.25

0.19

0.19

0.17

[0.31-0.40)

0.03

0.01

0.07

0.04

0.09

0.12

0.11

0.12

0.10

0.17

0.17

0.18

0.10

[0.23-0.31)

0.28

0.23

0.22

0.17

0.19

0.18

0.19

0.16

0.14

0.15

0.16

0.12

[0.23-0.31)

-0.01

0.05

0.05

0.10

0.08

0.09

0.08

0.11

0.13

0.11

0.11

0.15

0.05

0.2

[0.15-0.23)

0.19

0.21

0.13

0.12

0.10

0.12

0.10

0.09

0.10

0.09

0.09

0.07

[0.15-0.23)

0.01

-0.02

0.06

0.07

0.09

0.07

0.09

0.10

0.09

0.10

0.10

0.12

[0.02-0.15)

0.07

0.06

0.06

0.06

0.05

0.06

0.07

0.06

0.07

0.05

0.05

0.06

[0.02-0.15)

0.02

0.03

0.04

0.05

0.05

0.05

0.04

0.05

0.04

0.06

0.06

0.05

0.00

1

2

3

4

5

6

7

8

9

10

11

12

1

2

3

4

5

6

7

8

6

10

11

12

AtypicalityQuantile

AtypicalityQuantile

B Experimental Details

B.1 Balanced Supervised Classification

B.1.1 Datasets

Below is a full list of datasets for balanced classification:

ImageNet [DDS + 09] from Torchvision [MR10] is an object recognition dataset with 1000 classes. We use the ImageNet-1k version.

CIFAR10/100 [Kri09] from Torchvision [MR10] are object recognition datasets with 10/100 classes.

MNLI [WNB18] from Huggingface Datasets [LVdMJ + 21] is a natural language inference dataset with 3 classes, indicating entailment, neutral, and contradiction outcomes.

B.1.2 Models

Most of the models are public models, e.g., obtained from the Transformers Library [WDS + 20] or Torchvision [MR10]. Below we give the full model details and how one can access them:

RoBERTa ( HuggingFace roberta-large-mnli ) trained on the MNLI dataset. One can use the id given here on HuggingFace to download the model.

ResNet18, ResNet50, ResNet152 from ( Torchvision [MR10]) trained on ImageNet.

WideResNet28 trained on CIFAR10,100 obtained from [MKS + 20].

For all BERT [DCLT19] style models we use the [CLS] token embeddings in the final layer to perform classification. For all vision models, we use the penultimate layer embeddings to fit the density estimators and perform the analyses. In the experiments, we randomly split the test sets into two equal halves to have a calibration split and a test split, and repeat the experiments over 10 random seeds.

B.2 Imbalanced Supervised Classification

B.2.1 Datasets

All of our imbalanced classification datasets are previous benchmarks obtained from the GitHub repository 6 of [ZCLJ21] with corresponding training, validation, and test splits. All of these datasets have an exponential class imbalance.

ImageNet-LT is the long-tailed variant of ImageNet with 1000 classes.

CIFAR10/100-LT is the long-tailed variant of CIFAR10/100 with 10/100 classes.

Places365-LT is the long-tailed variant of Places365 [ZKL + 16] with 365 classes.

B.2.2 Models

Similarly, most of these models are obtained from [ZCLJ21].

ResNeXt50 trained on ImageNet-LT with and without Progressive Balancing, which is a strategy to address class imbalance during training.

ResNet152 trained on Places365-LT

ResNet18 trained on CIFAR100-LT trained by us. This model is pretrained on ImageNet and finetuned on CIFAR100-LT.

We use the validation splits of these datasets as the calibration set, and report the results on the test set.

B.3 Classification with LLMs

B.3.1 Model

We use Alpaca-7B [TGZ + 23] in a zero-shot setting, where we simply prompt the model with the classification question. We use the prompting strategy from Content-Free Calibration [ZWF + 21].

Below, we show examples of each dataset and prompt.

B.3.2 Datasets

IMDB is a binary classification dataset of movie reviews, where the goal is to classify the sentiment in a review. The example prompt has the form 'The following review was written for a movie: [Review]. \ n What is the sentiment, Positive or Negative? Answer: '. Below is an example:

The following review was written for a movie: I and a friend rented this movie. We both found the movie soundtrack and production techniques to be lagging. The movie's plot appeared to drag on throughout with little surprise in the ending. We both agreed that the movie could have been compressed into roughly an hour giving it more suspense and moving plot.

What is the sentiment, Positive or Negative? Answer:

where the correct answer should be 'Negative'. We filter out the examples that exceed the context length limit of Alpaca7B ( 512 ). We noticed that the 'validation' split of IMDB leads to significantly worse calibration compared to splitting the test set. Thus, for all experiments, we use the test split of IMDB and split it into 2 sets (instead of using the validation split as a calibration set as in the other two datasets).

TREC is a 6-class question classification dataset where the goal is to predict whether a question will have an answer that is an 'Abbreviation', 'Entity', 'Description', 'Human', 'Location', or a 'Number'. We format the prompts with 'Classify the questions based on their Answer Type. Potential Answer Types are: Number, Location, Person, Description, Entity, or Abbreviation. \ n \ nQuestion: [question] \ n \ nAnswer Type: '. Below is an example prompt:

Classify the questions based on their Answer Type. Potential Answer Types are: Number, Location, Person, Description, Entity, or Abbreviation.

Question: What county is Modesto , California in ? Answer Type:

where the correct answer should be 'Location'.

AG News is a news classification dataset. The goal is to classify a given news into 4 potential classes: 'World', 'Sports', 'Business', or 'Science and Technology'. We format the prompts with 'Classify the news articles into the categories of World, Sports, Business, and Technology. \ n \ n Article: [article] \ n \ nAnswer: '. Below is an example prompt:

Classify the news articles into the categories of World, Sports, Business, and Technology.

Article: Wall St. Bears Claw Back Into the Black (Reuters) Reuters - Short-sellers, Wall Street's dwindling band of ultra-cynics, are seeing green again.

Answer:

where the correct answer should be 'Business'.

Furthermore, we also use [ZWF + 21] as another calibration baseline. Following their paper, we use N/A , [MASK] , and the empty string as content-free. Concretely, we follow their paper to first obtain the average predicted probabilities for each label token for the content-free input, denoted by p cf . We then let

When making test-time predictions, we compute Softmax ( W T p ) as the new predicted probabilities. In our experiments, we observe that it does not perform as well in this setting, as was previously suggested by [Kum22].

C Calibration

We run all our experiments with 10 different random seeds, where the seeds are { 0 , 1 , 2 , . . . , 9 } . Randomness is over fitting the atypicality estimators, and calibration-test splits (we use the same splits with the recalibration experiments for the sake of consistency).

C.1 Expected Calibration Error

To compute ECE, we generate B = { B 1 , B 2 , ..., B M } , M equally-spaced bins where samples are sorted and grouped according to their confidence. B m here denotes the set of the data indices where the confidence of the prediction for the sample falls into the interval ( m -1 M , m M ] . We compute ECEby

where acc ( B m ) = 1 | B m | ∑ | B m | i =1 1 [ˆ y i = y i ] is the accuracy for the bin m , and conf ( B m ) = 1 | B m | ∑ | B m | i =1 ˆ P ( Y = ˆ y i | X = x i ) gives the average confidence within the bin. | B m | is the size of the bin m , N is the total number of samples, and 1 [ · ] is the indicator function.

Throughout our experiments, we let the number of bins | B | = 10 by default when computing ECE.

Similarly, below we report results with RMSCE (Root Mean Squared Error) [HMD18] as another calibration metric, which is formulated as the following:

C.2 ECE and Atypicality Results with Different Atypicality Metrics

We further experiment with different atypicality metrics, such as the average distance to the 5-nearest neighbors (Figure 7). We broadly observe that while there are slight differences in the quantitative results between different atypicality metrics, the qualitative phenomena remain intact. In Tables 1, 2, and 3 we give all the results in the tabular form.

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.02

0.04

ECE

ResNet18 - ImageNet

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.03

0.05

0.08

ECE

ResNet50 - ImageNet

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.03

0.05

ECE

RoBERTa - MNLI

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.00

0.10

ECE

WideResNet28 - CIFAR10

Uncalibrated

TS

AdaTS

AAR(Ours)

C.3 Results in the Tabular Format

Uncalibrated

TS

AdaTS

Atypicality-Aware

Atypicality

ECE

RMSE

ECE

RMSE

ECE

RMSE

ECE

RMSE

GMM

ResNet152

ImageNet

0.2

0

.

029

±

0

.

001

0

.

077

±

0

.

001

0

.

014

±

0

.

001

0

.

066

±

0

.

002

0

.

014

±

0

.

001

0

.

064

±

0

.

002

0

.

016

±

0

.

001

0

.

065

±

0

.

001

0.4

0

.

039

±

0

.

001

0

.

086

±

0

.

001

0

.

020

±

0

.

001

0

.

074

±

0

.

001

0

.

020

±

0

.

002

0

.

075

±

0

.

002

0

.

019

±

0

.

001

0

.

070

±

0

.

001

0.6

0

.

050

±

0

.

001

0

.

097

±

0

.

001

0

.

026

±

0

.

001

0

.

079

±

0

.

002

0

.

025

±

0

.

002

0

.

081

±

0

.

002

0

.

026

±

0

.

000

0

.

078

±

0

.

001

0.8

0

.

062

±

0

.

001

0

.

106

±

0

.

001

0

.

024

±

0

.

001

0

.

076

±

0

.

001

0

.

023

±

0

.

002

0

.

078

±

0

.

002

0

.

024

±

0

.

001

0

.

077

±

0

.

001

1.0

0

.

084

±

0

.

001

0

.

123

±

0

.

001

0

.

037

±

0

.

001

0

.

088

±

0

.

001

0

.

033

±

0

.

004

0

.

085

±

0

.

003

0

.

026

±

0

.

001

0

.

080

±

0

.

001

ResNet18

ImageNet

0.2

0

.

017

±

0

.

001

0

.

074

±

0

.

001

0

.

031

±

0

.

001

0

.

084

±

0

.

001

0

.

028

±

0

.

004

0

.

081

±

0

.

004

0

.

016

±

0

.

001

0

.

069

±

0

.

001

0.4

0

.

016

±

0

.

001

0

.

074

±

0

.

001

0

.

020

±

0

.

001

0

.

079

±

0

.

001

0

.

023

±

0

.

003

0

.

079

±

0

.

003

0

.

015

±

0

.

001

0

.

072

±

0

.

002

0.6

0

.

025

±

0

.

001

0

.

080

±

0

.

001

0

.

023

±

0

.

001

0

.

077

±

0

.

001

0

.

024

±

0

.

003

0

.

080

±

0

.

002

0

.

019

±

0

.

001

0

.

074

±

0

.

001

0.8

0

.

040

±

0

.

001

0

.

092

±

0

.

001

0

.

024

±

0

.

001

0

.

078

±

0

.

001

0

.

029

±

0

.

002

0

.

082

±

0

.

003

0

.

018

±

0

.

001

0

.

074

±

0

.

001

1.0

0

.

054

±

0

.

001

0

.

100

±

0

.

001

0

.

033

±

0

.

001

0

.

085

±

0

.

001

0

.

032

±

0

.

003

0

.

083

±

0

.

003

0

.

017

±

0

.

002

0

.

073

±

0

.

002

ResNet50

ImageNet

0.2

0

.

021

±

0

.

001

0

.

069

±

0

.

002

0

.

018

±

0

.

001

0

.

071

±

0

.

001

0

.

017

±

0

.

001

0

.

068

±

0

.

002

0

.

018

±

0

.

001

0

.

067

±

0

.

002

0.4

0

.

029

±

0

.

001

0

.

079

±

0

.

001

0

.

023

±

0

.

001

0

.

077

±

0

.

001

0

.

020

±

0

.

002

0

.

074

±

0

.

001

0

.

022

±

0

.

001

0

.

074

±

0

.

001

0.6

0

.

041

±

0

.

001

0

.

091

±

0

.

001

0

.

023

±

0

.

001

0

.

078

±

0

.

001

0

.

026

±

0

.

002

0

.

079

±

0

.

002

0

.

026

±

0

.

001

0

.

078

±

0

.

001

0.8

0

.

042

±

0

.

001

0

.

092

±

0

.

001

0

.

024

±

0

.

001

0

.

078

±

0

.

001

0

.

024

±

0

.

001

0

.

080

±

0

.

002

0

.

020

±

0

.

001

0

.

075

±

0

.

001

1.0

0

.

078

±

0

.

000

0

.

118

±

0

.

000

0

.

049

±

0

.

001

0

.

095

±

0

.

001

0

.

047

±

0

.

004

0

.

096

±

0

.

003

0

.

031

±

0

.

001

0

.

082

±

0

.

001

RoBERTa

MNLI

0.2

0

.

005

±

0

.

001

0

.

062

±

0

.

004

0

.

013

±

0

.

001

0

.

077

±

0

.

002

0

.

006

±

0

.

002

0

.

063

±

0

.

005

0

.

006

±

0

.

001

0

.

063

±

0

.

002

0.4

0

.

016

±

0

.

001

0

.

091

±

0

.

004

0

.

013

±

0

.

001

0

.

083

±

0

.

002

0

.

010

±

0

.

002

0

.

075

±

0

.

005

0

.

008

±

0

.

001

0

.

072

±

0

.

002

0.6

0

.

034

±

0

.

002

0

.

116

±

0

.

002

0

.

015

±

0

.

001

0

.

092

±

0

.

004

0

.

016

±

0

.

001

0

.

093

±

0

.

005

0

.

017

±

0

.

002

0

.

090

±

0

.

004

0.8

0

.

061

±

0

.

002

0

.

153

±

0

.

003

0

.

031

±

0

.

002

0

.

113

±

0

.

004

0

.

024

±

0

.

002

0

.

104

±

0

.

004

0

.

028

±

0

.

002

0

.

114

±

0

.

004

1.0

0

.

065

±

0

.

002

0

.

156

±

0

.

002

0

.

028

±

0

.

002

0

.

112

±

0

.

003

0

.

021

±

0

.

002

0

.

106

±

0

.

004

0

.

023

±

0

.

003

0

.

107

±

0

.

004

WideResNet28

CIFAR10

0.2

0

.

003

±

0

.

000

0

.

041

±

0

.

001

0

.

009

±

0

.

000

0

.

057

±

0

.

000

0

.

003

±

0

.

000

0

.

051

±

0

.

002

0

.

001

±

0

.

000

0

.

041

±

0

.

001

0.4

0

.

005

±

0

.

001

0

.

051

±

0

.

002

0

.

007

±

0

.

001

0

.

055

±

0

.

001

0

.

002

±

0

.

001

0

.

047

±

0

.

002

0

.

002

±

0

.

000

0

.

048

±

0

.

002

0.6

0

.

004

±

0

.

001

0

.

050

±

0

.

003

0

.

008

±

0

.

001

0

.

058

±

0

.

001

0

.

004

±

0

.

001

0

.

049

±

0

.

004

0

.

002

±

0

.

000

0

.

044

±

0

.

002

0.8

0

.

009

±

0

.

001

0

.

066

±

0

.

003

0

.

004

±

0

.

001

0

.

056

±

0

.

002

0

.

003

±

0

.

001

0

.

058

±

0

.

003

0

.

002

±

0

.

001

0

.

055

±

0

.

002

1.0

0

.

142

±

0

.

002

0

.

240

±

0

.

002

0

.

073

±

0

.

002

0

.

168

±

0

.

003

0

.

046

±

0

.

002

0

.

129

±

0

.

004

0

.

041

±

0

.

002

0

.

123

±

0

.

002

WideResNet28

CIFAR100

0.2

0

.

007

±

0

.

001

0

.

059

±

0

.

003

0

.

018

±

0

.

001

0

.

084

±

0

.

001

0

.

021

±

0

.

003

0

.

089

±

0

.

005

0

.

021

±

0

.

001

0

.

094

±

0

.

002

0.4

0

.

029

±

0

.

001

0

.

110

±

0

.

002

0

.

005

±

0

.

001

0

.

070

±

0

.

003

0

.

008

±

0

.

002

0

.

073

±

0

.

003

0

.

005

±

0

.

001

0

.

078

±

0

.

003

0.6

0

.

101

±

0

.

002

0

.

201

±

0

.

002

0

.

052

±

0

.

001

0

.

136

±

0

.

002

0

.

044

±

0

.

004

0

.

124

±

0

.

007

0

.

049

±

0

.

001

0

.

132

±

0

.

003

0.8

0

.

257

±

0

.

004

0

.

297

±

0

.

002

0

.

106

±

0

.

003

0

.

191

±

0

.

003

0

.

101

±

0

.

003

0

.

187

±

0

.

002

0

.

105

±

0

.

003

0

.

190

±

0

.

003

1.0

0

.

371

±

0

.

004

0

.

351

±

0

.

002

0

.

105

±

0

.

004

0

.

200

±

0

.

003

0

.

101

±

0

.

004

0

.

199

±

0

.

003

0

.

109

±

0

.

005

0

.

207

±

0

.

005

KNN

ResNet152

ImageNet

0.2

0

.

035

±

0

.

001

0

.

085

±

0

.

001

0

.

014

±

0

.

001

0

.

062

±

0

.

002

0

.

016

±

0

.

002

0

.

068

±

0

.

001

0

.

011

±

0

.

001

0

.

061

±

0

.

001

0.4

0

.

039

±

0

.

001

0

.

085

±

0

.

001

0

.

015

±

0

.

001

0

.

067

±

0

.

001

0

.

018

±

0

.

001

0

.

069

±

0

.

002

0

.

017

±

0

.

001

0

.

071

±

0

.

001

0.6

0

.

035

±

0

.

001

0

.

083

±

0

.

001

0

.

021

±

0

.

001

0

.

076

±

0

.

002

0

.

024

±

0

.

003

0

.

079

±

0

.

004

0

.

019

±

0

.

001

0

.

072

±

0

.

001

0.8

0

.

057

±

0

.

002

0

.

102

±

0

.

001

0

.

031

±

0

.

001

0

.

084

±

0

.

001

0

.

033

±

0

.

003

0

.

085

±

0

.

003

0

.

032

±

0

.

001

0

.

082

±

0

.

001

1.0

0

.

099

±

0

.

001

0

.

129

±

0

.

001

0

.

036

±

0

.

001

0

.

090

±

0

.

001

0

.

042

±

0

.

006

0

.

093

±

0

.

004

0

.

037

±

0

.

001

0

.

091

±

0

.

001

ResNet18

ImageNet

0.2

0

.

025

±

0

.

001

0

.

077

±

0

.

002

0

.

020

±

0

.

001

0

.

074

±

0

.

001

0

.

037

±

0

.

021

0

.

115

±

0

.

042

0

.

019

±

0

.

001

0

.

072

±

0

.

001

0.4

0

.

017

±

0

.

001

0

.

072

±

0

.

001

0

.

021

±

0

.

001

0

.

076

±

0

.

001

0

.

045

±

0

.

025

0

.

120

±

0

.

046

0

.

019

±

0

.

001

0

.

074

±

0

.

001

0.6

0

.

021

±

0

.

001

0

.

074

±

0

.

001

0

.

027

±

0

.

001

0

.

083

±

0

.

001

0

.

052

±

0

.

029

0

.

130

±

0

.

050

0

.

020

±

0

.

001

0

.

074

±

0

.

002

0.8

0

.

033

±

0

.

002

0

.

085

±

0

.

001

0

.

022

±

0

.

001

0

.

079

±

0

.

001

0

.

061

±

0

.

038

0

.

136

±

0

.

058

0

.

021

±

0

.

001

0

.

077

±

0

.

001

1.0

0

.

054

±

0

.

001

0

.

102

±

0

.

001

0

.

030

±

0

.

001

0

.

084

±

0

.

001

0

.

079

±

0

.

045

0

.

152

±

0

.

064

0

.

027

±

0

.

001

0

.

081

±

0

.

002

ResNet50

ImageNet

0.2

0

.

029

±

0

.

001

0

.

078

±

0

.

001

0

.

016

±

0

.

001

0

.

066

±

0

.

001

0

.

015

±

0

.

002

0

.

068

±

0

.

002

0

.

016

±

0

.

001

0

.

067

±

0

.

001

0.4

0

.

029

±

0

.

001

0

.

078

±

0

.

001

0

.

018

±

0

.

001

0

.

072

±

0

.

001

0

.

019

±

0

.

001

0

.

074

±

0

.

002

0

.

017

±

0

.

001

0

.

070

±

0

.

002

0.6

0

.

029

±

0

.

001

0

.

078

±

0

.

001

0

.

021

±

0

.

001

0

.

077

±

0

.

001

0

.

020

±

0

.

002

0

.

077

±

0

.

002

0

.

020

±

0

.

001

0

.

075

±

0

.

001

0.8

0

.

046

±

0

.

001

0

.

096

±

0

.

001

0

.

028

±

0

.

001

0

.

079

±

0

.

001

0

.

026

±

0

.

001

0

.

079

±

0

.

001

0

.

031

±

0

.

001

0

.

082

±

0

.

001

1.0

0

.

076

±

0

.

001

0

.

116

±

0

.

001

0

.

039

±

0

.

002

0

.

092

±

0

.

001

0

.

040

±

0

.

004

0

.

091

±

0

.

003

0

.

038

±

0

.

002

0

.

090

±

0

.

001

RoBERTa

MNLI

0.2

0

.

003

±

0

.

000

0

.

056

±

0

.

003

0

.

011

±

0

.

001

0

.

079

±

0

.

002

0

.

011

±

0

.

002

0

.

067

±

0

.

005

0

.

003

±

0

.

001

0

.

061

±

0

.

003

0.4

0

.

014

±

0

.

001

0

.

080

±

0

.

003

0

.

015

±

0

.

002

0

.

091

±

0

.

003

0

.

016

±

0

.

004

0

.

092

±

0

.

007

0

.

008

±

0

.

001

0

.

074

±

0

.

002

0.6

0

.

029

±

0

.

002

0

.

111

±

0

.

003

0

.

014

±

0

.

001

0

.

077

±

0

.

002

0

.

023

±

0

.

005

0

.

095

±

0

.

008

0

.

015

±

0

.

001

0

.

081

±

0

.

005

0.8

0

.

067

±

0

.

002

0

.

156

±

0

.

003

0

.

034

±

0

.

002

0

.

108

±

0

.

004

0

.

030

±

0

.

003

0

.

107

±

0

.

004

0

.

026

±

0

.

003

0

.

101

±

0

.

005

1.0

0

.

064

±

0

.

002

0

.

153

±

0

.

003

0

.

027

±

0

.

002

0

.

110

±

0

.

003

0

.

033

±

0

.

007

0

.

119

±

0

.

009

0

.

026

±

0

.

003

0

.

107

±

0

.

005

WideResNet28

CIFAR10

0.2

0

.

001

±

0

.

000

0

.

031

±

0

.

003

0

.

011

±

0

.

000

0

.

061

±

0

.

001

0

.

005

±

0

.

001

0

.

049

±

0

.

004

0

.

003

±

0

.

000

0

.

037

±

0

.

001

0.4

0

.

000

±

0

.

000

0

.

010

±

0

.

005

0

.

012

±

0

.

000

0

.

062

±

0

.

001

0

.

007

±

0

.

001

0

.

054

±

0

.

004

0

.

004

±

0

.

000

0

.

037

±

0

.

001

0.6

0

.

005

±

0

.

000

0

.

049

±

0

.

002

0

.

007

±

0

.

001

0

.

055

±

0

.

002

0

.

003

±

0

.

001

0

.

046

±

0

.

004

0

.

001

±

0

.

000

0

.

048

±

0

.

002

0.8

0

.

009

±

0

.

001

0

.

061

±

0

.

001

0

.

004

±

0

.

001

0

.

058

±

0

.

001

0

.

004

±

0

.

001

0

.

062

±

0

.

003

0

.

003

±

0

.

000

0

.

052

±

0

.

002

1.0

0

.

148

±

0

.

002

0

.

241

±

0

.

002

0

.

079

±

0

.

003

0

.

170

±

0

.

003

0

.

053

±

0

.

004

0

.

136

±

0

.

005

0

.

039

±

0

.

003

0

.

121

±

0

.

003

WideResNet28

CIFAR100

0.2

0

.

006

±

0

.

000

0

.

054

±

0

.

002

0

.

020

±

0

.

001

0

.

089

±

0

.

001

0

.

021

±

0

.

002

0

.

089

±

0

.

003

0

.

014

±

0

.

001

0

.

083

±

0

.

001

0.4

0

.

022

±

0

.

001

0

.

092

±

0

.

002

0

.

004

±

0

.

001

0

.

070

±

0

.

002

0

.

006

±

0

.

001

0

.

067

±

0

.

004

0

.

005

±

0

.

001

0

.

076

±

0

.

002

0.6

0

.

090

±

0

.

001

0

.

183

±

0

.

002

0

.

054

±

0

.

001

0

.

140

±

0

.

002

0

.

049

±

0

.

003

0

.

132

±

0

.

004

0

.

041

±

0

.

002

0

.

120

±

0

.

002

0.8

0

.

266

±

0

.

005

0

.

295

±

0

.

003

0

.

124

±

0

.

004

0

.

205

±

0

.

003

0

.

118

±

0

.

004

0

.

202

±

0

.

003

0

.

091

±

0

.

003

0

.

185

±

0

.

003

1.0

0

.

380

±

0

.

003

0

.

356

±

0

.

002

0

.

086

±

0

.

003

0

.

187

±

0

.

004

0

.

087

±

0

.

004

0

.

185

±

0

.

005

0

.

117

±

0

.

005

0

.

216

±

0

.

004

Uncalibrated

TS

AdaTS

Atypicality-Aware

Atypicality

ECE

RMSE

ECE

RMSE

ECE

RMSE

ECE

RMSE

GMM

ResNet152

Places365-LT

0.2

0

.

159

±

0

.

000

0

.

144

±

0

.

000

0

.

492

±

0

.

000

0

.

064

±

0

.

000

0

.

099

±

0

.

000

0

.

492

±

0

.

000

0

.

044

±

0

.

004

0

.

088

±

0

.

002

0

.

492

±

0

.

000

0

.

065

±

0

.

000

0

.

101

±

0

.

000

0

.

412

±

0

.

000

0.4

0

.

208

±

0

.

000

0

.

166

±

0

.

000

0

.

387

±

0

.

000

0

.

061

±

0

.

000

0

.

095

±

0

.

000

0

.

387

±

0

.

000

0

.

053

±

0

.

001

0

.

099

±

0

.

001

0

.

387

±

0

.

000

0

.

042

±

0

.

000

0

.

090

±

0

.

000

0

.

415

±

0

.

000

0.6

0

.

270

±

0

.

000

0

.

186

±

0

.

000

0

.

317

±

0

.

000

0

.

073

±

0

.

000

0

.

111

±

0

.

000

0

.

317

±

0

.

000

0

.

076

±

0

.

005

0

.

111

±

0

.

003

0

.

317

±

0

.

000

0

.

054

±

0

.

000

0

.

097

±

0

.

000

0

.

398

±

0

.

000

0.8

0

.

334

±

0

.

000

0

.

214

±

0

.

000

0

.

218

±

0

.

000

0

.

120

±

0

.

000

0

.

150

±

0

.

000

0

.

218

±

0

.

000

0

.

127

±

0

.

006

0

.

152

±

0

.

002

0

.

218

±

0

.

000

0

.

058

±

0

.

000

0

.

103

±

0

.

000

0

.

371

±

0

.

000

1.0

0

.

437

±

0

.

000

0

.

243

±

0

.

000

0

.

081

±

0

.

000

0

.

211

±

0

.

000

0

.

185

±

0

.

000

0

.

081

±

0

.

000

0

.

212

±

0

.

007

0

.

187

±

0

.

002

0

.

081

±

0

.

000

0

.

103

±

0

.

000

0

.

137

±

0

.

000

0

.

283

±

0

.

000

ResNet18

CIFAR10-LT

0.2

0

.

017

±

0

.

000

0

.

077

±

0

.

000

0

.

927

±

0

.

000

0

.

060

±

0

.

000

0

.

164

±

0

.

000

0

.

927

±

0

.

000

0

.

084

±

0

.

008

0

.

190

±

0

.

008

0

.

927

±

0

.

000

0

.

030

±

0

.

000

0

.

104

±

0

.

000

0

.

874

±

0

.

000

0.4

0

.

054

±

0

.

000

0

.

145

±

0

.

000

0

.

825

±

0

.

000

0

.

082

±

0

.

000

0

.

182

±

0

.

000

0

.

825

±

0

.

000

0

.

102

±

0

.

008

0

.

201

±

0

.

006

0

.

825

±

0

.

000

0

.

027

±

0

.

000

0

.

104

±

0

.

000

0

.

741

±

0

.

000

0.6

0

.

152

±

0

.

000

0

.

236

±

0

.

000

0

.

672

±

0

.

000

0

.

049

±

0

.

000

0

.

133

±

0

.

000

0

.

672

±

0

.

000

0

.

069

±

0

.

005

0

.

174

±

0

.

007

0

.

672

±

0

.

000

0

.

044

±

0

.

000

0

.

127

±

0

.

000

0

.

770

±

0

.

000

0.8

0

.

098

±

0

.

000

0

.

188

±

0

.

000

0

.

779

±

0

.

000

0

.

022

±

0

.

000

0

.

107

±

0

.

000

0

.

779

±

0

.

000

0

.

032

±

0

.

005

0

.

113

±

0

.

007

0

.

779

±

0

.

000

0

.

034

±

0

.

000

0

.

104

±

0

.

000

0

.

810

±

0

.

000

1.0

0

.

244

±

0

.

000

0

.

286

±

0

.

000

0

.

584

±

0

.

000

0

.

128

±

0

.

000

0

.

206

±

0

.

000

0

.

584

±

0

.

000

0

.

157

±

0

.

011

0

.

248

±

0

.

009

0

.

584

±

0

.

000

0

.

034

±

0

.

000

0

.

109

±

0

.

000

0

.

829

±

0

.

000

CIFAR100-LT

0.2

0

.

138

±

0

.

000

0

.

231

±

0

.

000

0

.

660

±

0

.

000

0

.

106

±

0

.

000

0

.

203

±

0

.

000

0

.

660

±

0

.

000

0

.

105

±

0

.

012

0

.

198

±

0

.

010

0

.

660

±

0

.

000

0

.

031

±

0

.

000

0

.

112

±

0

.

000

0

.

594

±

0

.

000

0.4

0

.

235

±

0

.

000

0

.

291

±

0

.

000

0

.

523

±

0

.

000

0

.

056

±

0

.

000

0

.

137

±

0

.

000

0

.

523

±

0

.

000

0

.

058

±

0

.

006

0

.

145

±

0

.

007

0

.

523

±

0

.

000

0

.

040

±

0

.

000

0

.

109

±

0

.

000

0

.

556

±

0

.

000

0.6

0

.

295

±

0

.

000

0

.

320

±

0

.

000

0

.

431

±

0

.

000

0

.

063

±

0

.

000

0

.

166

±

0

.

000

0

.

431

±

0

.

000

0

.

067

±

0

.

009

0

.

160

±

0

.

010

0

.

431

±

0

.

000

0

.

043

±

0

.

000

0

.

142

±

0

.

000

0

.

508

±

0

.

000

0.8

0

.

381

±

0

.

000

0

.

360

±

0

.

000

0

.

344

±

0

.

000

0

.

121

±

0

.

000

0

.

221

±

0

.

000

0

.

344

±

0

.

000

0

.

100

±

0

.

016

0

.

193

±

0

.

013

0

.

344

±

0

.

000

0

.

080

±

0

.

000

0

.

179

±

0

.

000

0

.

432

±

0

.

000

1.0

0

.

403

±

0

.

000

0

.

382

±

0

.

000

0

.

269

±

0

.

000

0

.

116

±

0

.

000

0

.

234

±

0

.

000

0

.

269

±

0

.

000

0

.

129

±

0

.

015

0

.

233

±

0

.

010

0

.

269

±

0

.

000

0

.

051

±

0

.

000

0

.

165

±

0

.

000

0

.

423

±

0

.

000

ResNext50

ImageNet-LT

0.2

0

.

019

±

0

.

000

0

.

061

±

0

.

000

0

.

716

±

0

.

000

0

.

073

±

0

.

000

0

.

097

±

0

.

000

0

.

716

±

0

.

000

0

.

069

±

0

.

006

0

.

095

±

0

.

003

0

.

716

±

0

.

000

0

.

013

±

0

.

000

0

.

062

±

0

.

000

0

.

627

±

0

.

000

0.4

0

.

032

±

0

.

000

0

.

075

±

0

.

000

0

.

592

±

0

.

000

0

.

059

±

0

.

000

0

.

089

±

0

.

000

0

.

592

±

0

.

000

0

.

056

±

0

.

006

0

.

087

±

0

.

003

0

.

592

±

0

.

000

0

.

022

±

0

.

000

0

.

068

±

0

.

000

0

.

576

±

0

.

000

0.6

0

.

081

±

0

.

000

0

.

103

±

0

.

000

0

.

481

±

0

.

000

0

.

045

±

0

.

000

0

.

079

±

0

.

000

0

.

481

±

0

.

000

0

.

046

±

0

.

002

0

.

081

±

0

.

001

0

.

481

±

0

.

000

0

.

020

±

0

.

000

0

.

063

±

0

.

000

0

.

525

±

0

.

000

0.8

0

.

171

±

0

.

000

0

.

144

±

0

.

000

0

.

309

±

0

.

000

0

.

079

±

0

.

000

0

.

115

±

0

.

000

0

.

309

±

0

.

000

0

.

086

±

0

.

007

0

.

116

±

0

.

002

0

.

309

±

0

.

000

0

.

032

±

0

.

000

0

.

080

±

0

.

000

0

.

446

±

0

.

000

1.0

0

.

324

±

0

.

000

0

.

198

±

0

.

000

0

.

113

±

0

.

000

0

.

230

±

0

.

000

0

.

174

±

0

.

000

0

.

113

±

0

.

000

0

.

235

±

0

.

008

0

.

175

±

0

.

002

0

.

113

±

0

.

000

0

.

082

±

0

.

000

0

.

113

±

0

.

000

0

.

318

±

0

.

000

ResNext50(+P.B.)

ImageNet-LT

0.2

0

.

017

±

0

.

000

0

.

064

±

0

.

000

0

.

653

±

0

.

000

0

.

081

±

0

.

000

0

.

100

±

0

.

000

0

.

653

±

0

.

000

0

.

084

±

0

.

004

0

.

101

±

0

.

002

0

.

653

±

0

.

000

0

.

020

±

0

.

000

0

.

070

±

0

.

000

0

.

579

±

0

.

000

0.4

0

.

023

±

0

.

000

0

.

068

±

0

.

000

0

.

579

±

0

.

000

0

.

062

±

0

.

000

0

.

090

±

0

.

000

0

.

579

±

0

.

000

0

.

063

±

0

.

004

0

.

089

±

0

.

002

0

.

579

±

0

.

000

0

.

013

±

0

.

000

0

.

064

±

0

.

000

0

.

535

±

0

.

000

0.6

0

.

065

±

0

.

000

0

.

090

±

0

.

000

0

.

508

±

0

.

000

0

.

027

±

0

.

000

0

.

074

±

0

.

000

0

.

508

±

0

.

000

0

.

027

±

0

.

003

0

.

070

±

0

.

002

0

.

508

±

0

.

000

0

.

018

±

0

.

000

0

.

067

±

0

.

000

0

.

508

±

0

.

000

0.8

0

.

129

±

0

.

000

0

.

121

±

0

.

000

0

.

388

±

0

.

000

0

.

042

±

0

.

000

0

.

082

±

0

.

000

0

.

388

±

0

.

000

0

.

043

±

0

.

003

0

.

083

±

0

.

002

0

.

388

±

0

.

000

0

.

039

±

0

.

000

0

.

076

±

0

.

000

0

.

444

±

0

.

000

1.0

0

.

236

±

0

.

000

0

.

164

±

0

.

000

0

.

224

±

0

.

000

0

.

147

±

0

.

000

0

.

134

±

0

.

000

0

.

224

±

0

.

000

0

.

147

±

0

.

004

0

.

133

±

0

.

002

0

.

224

±

0

.

000

0

.

078

±

0

.

000

0

.

100

±

0

.

000

0

.

359

±

0

.

000

KNN

ResNet152

Places365-LT

0.2

0

.

159

±

0

.

000

0

.

144

±

0

.

000

0

.

492

±

0

.

000

0

.

064

±

0

.

000

0

.

099

±

0

.

000

0

.

492

±

0

.

000

0

.

047

±

0

.

004

0

.

089

±

0

.

002

0

.

492

±

0

.

000

0

.

091

±

0

.

000

0

.

116

±

0

.

000

0

.

387

±

0

.

000

0.4

0

.

208

±

0

.

000

0

.

166

±

0

.

000

0

.

387

±

0

.

000

0

.

061

±

0

.

000

0

.

095

±

0

.

000

0

.

387

±

0

.

000

0

.

054

±

0

.

001

0

.

097

±

0

.

001

0

.

387

±

0

.

000

0

.

048

±

0

.

000

0

.

090

±

0

.

000

0

.

448

±

0

.

000

0.6

0

.

270

±

0

.

000

0

.

186

±

0

.

000

0

.

317

±

0

.

000

0

.

073

±

0

.

000

0

.

111

±

0

.

000

0

.

317

±

0

.

000

0

.

068

±

0

.

004

0

.

107

±

0

.

002

0

.

317

±

0

.

000

0

.

057

±

0

.

000

0

.

098

±

0

.

000

0

.

416

±

0

.

000

0.8

0

.

334

±

0

.

000

0

.

214

±

0

.

000

0

.

218

±

0

.

000

0

.

120

±

0

.

000

0

.

150

±

0

.

000

0

.

218

±

0

.

000

0

.

117

±

0

.

004

0

.

149

±

0

.

002

0

.

218

±

0

.

000

0

.

065

±

0

.

000

0

.

106

±

0

.

000

0

.

367

±

0

.

000

1.0

0

.

437

±

0

.

000

0

.

243

±

0

.

000

0

.

081

±

0

.

000

0

.

211

±

0

.

000

0

.

185

±

0

.

000

0

.

081

±

0

.

000

0

.

202

±

0

.

004

0

.

184

±

0

.

001

0

.

081

±

0

.

000

0

.

111

±

0

.

000

0

.

152

±

0

.

000

0

.

207

±

0

.

000

ResNet18

CIFAR10-LT

0.2

0

.

017

±

0

.

000

0

.

077

±

0

.

000

0

.

927

±

0

.

000

0

.

060

±

0

.

000

0

.

164

±

0

.

000

0

.

927

±

0

.

000

0

.

098

±

0

.

009

0

.

204

±

0

.

008

0

.

927

±

0

.

000

0

.

025

±

0

.

000

0

.

085

±

0

.

000

0

.

873

±

0

.

000

0.4

0

.

054

±

0

.

000

0

.

145

±

0

.

000

0

.

825

±

0

.

000

0

.

082

±

0

.

000

0

.

182

±

0

.

000

0

.

825

±

0

.

000

0

.

127

±

0

.

008

0

.

220

±

0

.

005

0

.

825

±

0

.

000

0

.

024

±

0

.

000

0

.

106

±

0

.

000

0

.

742

±

0

.

000

0.6

0

.

152

±

0

.

000

0

.

236

±

0

.

000

0

.

672

±

0

.

000

0

.

049

±

0

.

000

0

.

133

±

0

.

000

0

.

672

±

0

.

000

0

.

054

±

0

.

005

0

.

157

±

0

.

005

0

.

672

±

0

.

000

0

.

039

±

0

.

000

0

.

124

±

0

.

000

0

.

768

±

0

.

000

0.8

0

.

098

±

0

.

000

0

.

188

±

0

.

000

0

.

779

±

0

.

000

0

.

022

±

0

.

000

0

.

107

±

0

.

000

0

.

779

±

0

.

000

0

.

028

±

0

.

002

0

.

106

±

0

.

005

0

.

779

±

0

.

000

0

.

029

±

0

.

000

0

.

102

±

0

.

000

0

.

810

±

0

.

000

1.0

0

.

244

±

0

.

000

0

.

286

±

0

.

000

0

.

584

±

0

.

000

0

.

128

±

0

.

000

0

.

206

±

0

.

000

0

.

584

±

0

.

000

0

.

153

±

0

.

009

0

.

248

±

0

.

007

0

.

584

±

0

.

000

0

.

037

±

0

.

000

0

.

123

±

0

.

000

0

.

832

±

0

.

000

CIFAR100-LT

0.2

0

.

138

±

0

.

000

0

.

231

±

0

.

000

0

.

660

±

0

.

000

0

.

106

±

0

.

000

0

.

203

±

0

.

000

0

.

660

±

0

.

000

0

.

088

±

0

.

008

0

.

184

±

0

.

008

0

.

660

±

0

.

000

0

.

029

±

0

.

000

0

.

108

±

0

.

000

0

.

595

±

0

.

000

0.4

0

.

235

±

0

.

000

0

.

291

±

0

.

000

0

.

523

±

0

.

000

0

.

056

±

0

.

000

0

.

137

±

0

.

000

0

.

523

±

0

.

000

0

.

049

±

0

.

003

0

.

135

±

0

.

004

0

.

523

±

0

.

000

0

.

049

±

0

.

000

0

.

123

±

0

.

000

0

.

554

±

0

.

000

0.6

0

.

295

±

0

.

000

0

.

320

±

0

.

000

0

.

431

±

0

.

000

0

.

063

±

0

.

000

0

.

166

±

0

.

000

0

.

431

±

0

.

000

0

.

073

±

0

.

008

0

.

171

±

0

.

008

0

.

431

±

0

.

000

0

.

030

±

0

.

000

0

.

113

±

0

.

000

0

.

509

±

0

.

000

0.8

0

.

381

±

0

.

000

0

.

360

±

0

.

000

0

.

344

±

0

.

000

0

.

121

±

0

.

000

0

.

221

±

0

.

000

0

.

344

±

0

.

000

0

.

119

±

0

.

012

0

.

210

±

0

.

009

0

.

344

±

0

.

000

0

.

078

±

0

.

000

0

.

175

±

0

.

000

0

.

435

±

0

.

000

1.0

0

.

403

±

0

.

000

0

.

382

±

0

.

000

0

.

269

±

0

.

000

0

.

116

±

0

.

000

0

.

234

±

0

.

000

0

.

269

±

0

.

000

0

.

145

±

0

.

011

0

.

244

±

0

.

008

0

.

269

±

0

.

000

0

.

059

±

0

.

000

0

.

163

±

0

.

000

0

.

424

±

0

.

000

ResNext50

ImageNet-LT

0.2

0

.

019

±

0

.

000

0

.

061

±

0

.

000

0

.

716

±

0

.

000

0

.

073

±

0

.

000

0

.

097

±

0

.

000

0

.

716

±

0

.

000

0

.

067

±

0

.

004

0

.

094

±

0

.

002

0

.

716

±

0

.

000

0

.

014

±

0

.

000

0

.

065

±

0

.

000

0

.

626

±

0

.

000

0.4

0

.

032

±

0

.

000

0

.

075

±

0

.

000

0

.

592

±

0

.

000

0

.

059

±

0

.

000

0

.

089

±

0

.

000

0

.

592

±

0

.

000

0

.

052

±

0

.

004

0

.

086

±

0

.

002

0

.

592

±

0

.

000

0

.

023

±

0

.

000

0

.

068

±

0

.

000

0

.

575

±

0

.

000

0.6

0

.

081

±

0

.

000

0

.

103

±

0

.

000

0

.

481

±

0

.

000

0

.

045

±

0

.

000

0

.

079

±

0

.

000

0

.

481

±

0

.

000

0

.

042

±

0

.

001

0

.

080

±

0

.

001

0

.

481

±

0

.

000

0

.

023

±

0

.

000

0

.

068

±

0

.

000

0

.

524

±

0

.

000

0.8

0

.

171

±

0

.

000

0

.

144

±

0

.

000

0

.

309

±

0

.

000

0

.

079

±

0

.

000

0

.

115

±

0

.

000

0

.

309

±

0

.

000

0

.

088

±

0

.

005

0

.

116

±

0

.

002

0

.

309

±

0

.

000

0

.

037

±

0

.

000

0

.

079

±

0

.

000

0

.

447

±

0

.

000

1.0

0

.

324

±

0

.

000

0

.

198

±

0

.

000

0

.

113

±

0

.

000

0

.

230

±

0

.

000

0

.

174

±

0

.

000

0

.

113

±

0

.

000

0

.

238

±

0

.

005

0

.

176

±

0

.

001

0

.

113

±

0

.

000

0

.

086

±

0

.

000

0

.

116

±

0

.

000

0

.

318

±

0

.

000

ResNext50(+P.B.)

ImageNet-LT

0.2

0

.

017

±

0

.

000

0

.

064

±

0

.

000

0

.

653

±

0

.

000

0

.

081

±

0

.

000

0

.

100

±

0

.

000

0

.

653

±

0

.

000

0

.

083

±

0

.

005

0

.

100

±

0

.

003

0

.

653

±

0

.

000

0

.

018

±

0

.

000

0

.

069

±

0

.

000

0

.

578

±

0

.

000

0.4

0

.

023

±

0

.

000

0

.

068

±

0

.

000

0

.

579

±

0

.

000

0

.

062

±

0

.

000

0

.

090

±

0

.

000

0

.

579

±

0

.

000

0

.

062

±

0

.

005

0

.

089

±

0

.

003

0

.

579

±

0

.

000

0

.

015

±

0

.

000

0

.

062

±

0

.

000

0

.

535

±

0

.

000

0.6

0

.

065

±

0

.

000

0

.

090

±

0

.

000

0

.

508

±

0

.

000

0

.

027

±

0

.

000

0

.

074

±

0

.

000

0

.

508

±

0

.

000

0

.

028

±

0

.

003

0

.

073

±

0

.

001

0

.

508

±

0

.

000

0

.

024

±

0

.

000

0

.

069

±

0

.

000

0

.

507

±

0

.

000

0.8

0

.

129

±

0

.

000

0

.

121

±

0

.

000

0

.

388

±

0

.

000

0

.

042

±

0

.

000

0

.

082

±

0

.

000

0

.

388

±

0

.

000

0

.

043

±

0

.

005

0

.

083

±

0

.

002

0

.

388

±

0

.

000

0

.

038

±

0

.

000

0

.

080

±

0

.

000

0

.

444

±

0

.

000

1.0

0

.

236

±

0

.

000

0

.

164

±

0

.

000

0

.

224

±

0

.

000

0

.

147

±

0

.

000

0

.

134

±

0

.

000

0

.

224

±

0

.

000

0

.

148

±

0

.

005

0

.

134

±

0

.

002

0

.

224

±

0

.

000

0

.

076

±

0

.

000

0

.

101

±

0

.

000

0

.

358

±

0

.

000

Uncalibrated

Content-Free

Atypicality-Aware

ECE

RMSE

ECE

RMSE

ECE

RMSE

LLM

AG News

0.25

0

.

180

±

0

.

000

0

.

219

±

0

.

000

0

.

681

±

0

.

000

0

.

452

±

0

.

000

0

.

411

±

0

.

000

0

.

527

±

0

.

000

0

.

070

±

0

.

000

0

.

142

±

0

.

000

0

.

760

±

0

.

000

0.50

0

.

165

±

0

.

000

0

.

202

±

0

.

000

0

.

671

±

0

.

000

0

.

413

±

0

.

000

0

.

370

±

0

.

000

0

.

571

±

0

.

000

0

.

027

±

0

.

000

0

.

101

±

0

.

000

0

.

775

±

0

.

000

0.75

0

.

169

±

0

.

000

0

.

204

±

0

.

000

0

.

657

±

0

.

000

0

.

429

±

0

.

000

0

.

364

±

0

.

000

0

.

549

±

0

.

000

0

.

030

±

0

.

000

0

.

099

±

0

.

000

0

.

752

±

0

.

000

1.00

0

.

202

±

0

.

000

0

.

222

±

0

.

000

0

.

580

±

0

.

000

0

.

448

±

0

.

000

0

.

350

±

0

.

000

0

.

524

±

0

.

000

0

.

028

±

0

.

000

0

.

110

±

0

.

000

0

.

715

±

0

.

000

IMDB

0.25

0

.

023

±

0

.

001

0

.

087

±

0

.

002

0

.

887

±

0

.

001

0

.

141

±

0

.

001

0

.

194

±

0

.

001

0

.

883

±

0

.

001

0

.

011

±

0

.

001

0

.

068

±

0

.

002

0

.

927

±

0

.

001

0.50

0

.

024

±

0

.

001

0

.

095

±

0

.

002

0

.

851

±

0

.

001

0

.

117

±

0

.

002

0

.

185

±

0

.

002

0

.

838

±

0

.

002

0

.

014

±

0

.

001

0

.

078

±

0

.

002

0

.

920

±

0

.

001

0.75

0

.

051

±

0

.

002

0

.

124

±

0

.

002

0

.

795

±

0

.

002

0

.

110

±

0

.

001

0

.

181

±

0

.

002

0

.

823

±

0

.

002

0

.

009

±

0

.

001

0

.

069

±

0

.

001

0

.

895

±

0

.

001

1.00

0

.

063

±

0

.

001

0

.

136

±

0

.

001

0

.

756

±

0

.

001

0

.

122

±

0

.

001

0

.

201

±

0

.

001

0

.

752

±

0

.

001

0

.

021

±

0

.

001

0

.

088

±

0

.

002

0

.

883

±

0

.

001

TREC

0.25

0

.

139

±

0

.

000

0

.

355

±

0

.

000

0

.

744

±

0

.

000

0

.

286

±

0

.

000

0

.

265

±

0

.

000

0

.

776

±

0

.

000

0

.

109

±

0

.

000

0

.

266

±

0

.

000

0

.

896

±

0

.

000

0.50

0

.

217

±

0

.

000

0

.

444

±

0

.

000

0

.

680

±

0

.

000

0

.

314

±

0

.

000

0

.

503

±

0

.

000

0

.

672

±

0

.

000

0

.

068

±

0

.

000

0

.

148

±

0

.

000

0

.

816

±

0

.

000

0.75

0

.

238

±

0

.

000

0

.

482

±

0

.

000

0

.

592

±

0

.

000

0

.

464

±

0

.

000

0

.

622

±

0

.

000

0

.

520

±

0

.

000

0

.

067

±

0

.

000

0

.

198

±

0

.

000

0

.

784

±

0

.

000

1.00

0

.

207

±

0

.

000

0

.

451

±

0

.

000

0

.

544

±

0

.

000

0

.

528

±

0

.

000

0

.

685

±

0

.

000

0

.

432

±

0

.

000

0

.

150

±

0

.

000

0

.

199

±

0

.

000

0

.

696

±

0

.

000

D Recalibration

Through all our recalibration results, we first split the test set into two equally sized calibration and test splits. Then, we fit the recalibration method using the calibration split and compute the performance on the test split. We run all our experiments with 10 different random seeds.

D.1 Temperature Scaling

To perform temperature scaling [GPSW17], we use the calibration set to fit the temperature parameter. To perform the optimization, we use the LBFGS [LN89] algorithm from PyTorch with strong Wolfe line search, following [GPSW17]. Namely, we optimize the parameter τ with

and then use it during inference to rescale the logits produced by f . We use 0 . 1 learning rate and 3000 maximum iterations across all experiments and initialize the temperature value as 1 , although find that TS is pretty robust to the choice of hyperparameters.

D.2 Atypicality-Aware Recalibration

Here we describe the implementation details For Atypicality-Aware Recalibration (AAR). We formulate AAR with:

In total, this gives us |Y| +3 parameters. Using exactly the same setting as TS, we use LBFGS with a strong Wolfe search to optimize the three parameters, with the same splits as temperature scaling. We normalize the atypicality values (subtract the mean and divide by the standard deviation of the calibration set) for numerical stability. We use the same hyperparameters as TS (with 0 . 1 learning rate and 3000 maximum iterations) without any modification across all experiments, initialize c 0 , c 1 , c 2 as 0 and S Y parameters as 1 . We run the recalibration procedure on a CPU with precomputed logits.

D.2.1 Adaptive Temperature Scaling

For AdaTS [JPL + 23] we use the implementation provided with the paper 7 . We identically use the hyperparameters and the architecture provided in the paper and their repository. They use an encoder and decoder architecture with [1024 , 512 , 512] hidden units each, and a temperature predictor network with [128 , 128] hidden units. They use an Adam Optimizer with a learning rate of 5 e -4 with 128 batch size.

D.3 Conformal Prediction

We follow the presentation in [AB21, ABMJ20]. Let π ( X ) be the permutation of Y = { 1 , . . . , C } that sorts ˆ P ( Y = c | X ) , i.e. the predicted probabilities for each class c . We define a score function

This means greedily including classes until the set contains the true label, and using the cumulative sum of the probabilities as the score function. We compute all of the scores for the calibration set, S calib = { s ( x 1 , y 1 ) , ..., s ( x N , y N ) } , we the ⌈ ( N +1)(1 -α ) ⌉ N th quantile of the scores, ˆ q . Then, the prediction set is defined as

We can further add randomization to the procedure where we have the prediction set function to be C ( x, u ) : X × [0 , 1] for randomization purposes to satisfy exact coverage. We refer to [VGS05, ABMJ20, AB21] for a more thorough presentation.

RAPS is a variant of APS that regularizes the set sizes. They modify the scoring function to add a regularization term. This is controlled by the test size offset k reg that controls the value beyond which the regularization is applied, and the λ reg gives the strength of the regularization. To fit the k reg , λ reg parameters in RAPS, we follow the procedure in [ABMJ20] to fit both parameters. Namely, we fit k reg by Algorithm 4 in their paper that leverages the set sizes in the calibration set, and we fit λ reg by the largest regularization parameter that achieves the smallest set sizes, searched over a grid of { 0 . 001 , 0 . 01 , 0 . 1 , 0 . 2 , 0 . 5 } following their presentation.

D.4 Atypicality-Aware Conformal Prediction

We have a simple discrete grouping scheme to make conformal prediction atypicality aware. Namely, we group points using their atypicality and confidence percentiles and fit individual thresholds. Concretely, we construct a dataset of D AA = { ( c i , c i +1 ] , ( a j , a j +1 ] , ˆ q i,j } i,j ∈ [ N ] using the calibration set where ( c i , c i +1 ] denotes the confidence range for quantile i , ( a j , a j +1 ] denotes the atypicality range for quantile j and ˆ q i,j denotes the threshold fitted to the group specified by these intervals. We let N = 6 as the number of groups, and in total, we end up with 36 thresholds. At test time, we check the quantile of the confidence and atypicality of a point and use the corresponding temperature. For AA-RAPS, we use the same k reg and λ reg values as was found with the RAPS procedure. For practical purposes, we do not allow zero sets (prediction sets at least include the top prediction).

We would like to make the remark that sometimes the marginal coverage can exceed the desired value (e.g. Figure 9). This is often because the underlying model is already very confident for a majority of data points (e.g. More than half of the data points have 92% confidence). The gains we provide are often for points with lower confidence regions, as the coverage is not satisfied in those regions.

1

2

3

4

5

6

Input Atypicality Quantile

0.00-0.34

0.34-0.55

0.55-0.77

0.77-0.93

0.93-0.99

0.99-1.00

Confidence

0.98

0.98

0.97

0.97

0.96

0.97

0.95

0.96

0.96

0.96

0.96

0.95

0.95

0.96

0.96

0.95

0.95

0.95

0.96

0.95

0.99

0.99

0.99

0.99

0.99

0.99

0.94

0.94

0.93

0.92

0.93

0.94

0.91

0.90

0.93

0.94

APS(0.95) Coverage

1

2

3

4

5

6

0.96

0.95

0.95

0.96

0.95

0.95

0.96

0.96

0.95

0.95

0.96

0.96

0.96

0.97

0.96

0.96

0.96

0.95

0.95

0.96

0.95

0.95

0.96

0.96

0.97

0.97

0.99

0.99

0.99

0.99

0.99

0.99

0.94

0.94

0.93

0.94

AA-APS(0.95) Coverage

1

2

3

4

5

6

164.5

153.7

142.0

124.2

106.3

78.5

35.0

43.1

41.2

39.6

35.2

27.4

16.7

15.3

18.1

18.7

17.9

12.3

5.3

6.0

5.4

6.1

5.6

4.9

1.2

1.2

1.2

1.2

1.2

1.2

1.0

1.0

1.0

1.0

1.0

1.0

APS(0.95) Set Size

1

2

3

4

5

6

85.2

88.3

92.8

113.8

99.0

101.4

25.4

28.9

38.7

42.2

45.0

40.3

14.5

14.4

16.4

23.1

21.8

21.7

4.5

4.8

6.7

11.0

8.6

11.6

1.2

1.3

1.4

3.0

2.5

4.1

1.0

1.0

1.0

1.0

1.0

1.0

AA-APS(0.95) Set Size

1

2

3

4

5

6

0.00-0.34

0.34-0.55

0.55-0.77

0.77-0.93

0.93-0.99

0.99-1.00

0.98

0.98

0.98

0.98

0.98

0.96

0.96

1.00

0.99

0.99

0.98

0.99

0.98

0.99

0.99

1.00

0.99

0.99

0.99

1.00

1.00

1.00

1.00

1.00

1.00

0.84

0.83

0.80

0.81

0.80

0.80

0.94

0.94

0.93

0.93

0.92

RAPS(0.95) Coverage

1

2

3

4

5

6

0.96

0.96

0.95

0.96

0.96

0.96

0.95

0.96

0.96

0.96

0.95

0.95

0.97

0.97

0.96

0.96

0.96

0.95

0.96

0.97

0.95

0.95

0.96

0.96

0.96

0.95

0.97

0.99

0.99

0.99

0.99

0.99

0.99

0.93

0.93

0.93

AA-RAPS(0.95) Coverage

1

2

3

4

5

6

17.2

17.3

17.2

17.1

16.9

16.5

15.3

15.5

15.5

15.5

15.5

15.3

15.0

15.0

15.1

15.1

15.1

15.0

14.8

14.9

14.9

14.9

14.9

14.8

14.8

14.8

14.8

14.8

14.8

14.8

14.7

14.7

14.8

14.7

14.8

14.7

RAPS(0.95) Set Size

1

2

3

4

5

6

74.0

89.6

96.3

115.3

117.2

103.0

9.4

12.8

25.1

32.1

34.7

23.8

6.2

7.5

9.0

12.4

12.5

12.1

3.9

4.6

5.1

6.4

6.0

7.1

1.2

1.4

1.4

2.3

2.6

3.9

1.0

1.0

1.0

1.0

1.0

1.0

AA-RAPS(0.95) Set Size

1

2

3

4

5

6

Confidence Quantile

0.85

0.90

0.95

1.00

Coverage

Method

RAPS

APS

AA-RAPS

AA-APS

1

2

3

4

5

6

Input Atypicality Quantile

0.94

0.95

0.96

0.97

Coverage

1

2

3

4

5

6

Confidence Quantile

0

25

50

75

100

125

SetSize

1

2

3

4

5

6

Input Atypicality Quantile

10

20

30

40

SetSize

1

2

3

4

5

6

Input Atypicality Quantile

0.00-0.50

0.50-0.74

0.74-0.92

0.92-0.98

0.98-1.00

1.00-1.00

Confidence

0.99

0.99

0.98

0.98

0.97

0.96

0.97

0.96

0.96

0.95

0.96

0.96

0.95

0.96

0.96

0.99

0.98

0.97

0.97

0.96

0.98

0.99

0.99

0.99

1.00

0.99

0.99

0.94

0.93

0.93

0.94

0.93

0.93

0.93

0.93

0.89

APS(0.95) Coverage

1

2

3

4

5

6

0.96

0.97

0.95

0.95

0.95

0.96

0.96

0.97

0.95

0.95

0.97

0.95

0.95

0.95

0.96

0.95

0.97

0.99

0.98

0.97

0.97

0.96

0.99

0.99

0.99

0.99

1.00

0.99

0.99

0.93

0.94

0.94

0.94

0.94

0.94

0.94

AA-APS(0.95) Coverage

1

2

3

4

5

6

48.2

81.8

111.1

118.1

110.7

87.9

16.5

18.3

22.1

23.0

24.3

21.8

6.8

6.6

7.4

8.2

8.7

7.8

1.4

1.4

1.5

1.5

1.4

1.4

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

APS(0.95) Set Size

1

2

3

4

5

6

28.1

47.8

45.5

58.9

65.6

75.4

12.1

13.9

16.2

21.7

29.6

29.8

5.1

5.7

10.4

11.3

10.9

12.3

1.2

1.5

2.6

2.5

4.6

9.5

1.0

1.0

1.0

1.0

1.0

2.0

1.0

1.0

1.0

1.0

1.0

1.0

AA-APS(0.95) Set Size

1

2

3

4

5

6

0.00-0.50

0.50-0.74

0.74-0.92

0.92-0.98

0.98-1.00

1.00-1.00

0.95

0.97

0.98

0.97

0.96

0.98

0.98

0.97

0.96

0.96

0.98

0.98

0.96

0.97

0.96

0.96

0.99

0.98

0.97

0.98

0.96

0.99

0.99

0.99

0.99

1.00

0.99

0.99

0.94

0.89

0.85

0.81

0.77

0.94

0.92

0.94

RAPS(0.95) Coverage

1

2

3

4

5

6

0.98

0.97

0.96

0.95

0.95

0.97

0.96

0.96

0.96

0.95

0.95

0.96

0.96

0.96

0.96

0.96

0.95

0.95

0.95

0.97

0.99

0.98

0.97

0.97

0.96

0.98

0.99

0.99

0.99

1.00

0.99

0.99

0.94

0.94

0.94

0.94

AA-RAPS(0.95) Coverage

1

2

3

4

5

6

7.3

8.1

8.8

9.1

9.2

8.9

5.1

5.8

6.0

6.3

6.5

6.6

4.7

5.3

5.4

5.6

5.7

5.8

3.5

3.7

3.6

3.7

4.0

3.6

1.1

1.1

1.1

1.1

1.1

1.1

1.0

1.0

1.0

1.0

1.0

1.0

RAPS(0.95) Set Size

1

2

3

4

5

6

24.2

18.0

29.2

33.9

43.4

55.0

3.6

4.7

5.1

6.5

9.4

14.0

3.0

3.5

4.4

4.8

5.6

12.0

1.5

1.3

1.9

2.2

3.0

4.6

1.0

1.0

1.0

1.0

1.0

1.6

1.0

1.0

1.0

1.0

1.0

1.0

AA-RAPS(0.95) Set Size

1

2

3

4

5

6

Confidence Quantile

0.85

0.90

0.95

Coverage

Method

RAPS

APS

AA-RAPS

AA-APS

1

2

3

4

5

6

Input Atypicality Quantile

0.92

0.94

0.96

0.98

Coverage

1

2

3

4

5

6

Confidence Quantile

0

20

40

60

80

100

SetSize

1

2

3

4

5

6

Input Atypicality Quantile

10

20

30

SetSize

0.00

0.25

0.50

0.75

Atypicality Percentile

1.0

1.2

Temperature

τk

ImageNet

0.00

0.25

0.50

0.75

Atypicality Percentile

1.25

1.50

CIFAR100

E Tables for Results

Here, we present the table version of the results in Figure 3. Tables 1,2 contain the ECE analysis.

5

6

7

8

9

10

Class Atypicality

a

(

Y

)

4

2

0

2

4

ˆ

SY

Additive Correction vs Class Atypicality ResNext50 on ImageNet-LT

1

2

3

4

5

6

Input Atypicality Quantile

0.00-0.45

0.45-0.69

0.69-0.89

0.89-0.97

0.97-1.00

1.00-1.00

Confidence

0.98

0.99

0.98

0.97

0.96

0.95

0.96

0.98

0.96

0.96

0.95

0.95

0.95

0.96

0.95

0.95

0.96

0.95

0.98

0.98

0.96

0.96

0.96

0.95

0.99

0.99

0.99

0.99

1.00

0.99

0.94

0.92

0.93

0.93

0.93

0.90

APS(0.95) Coverage

1

2

3

4

5

6

0.96

0.96

0.95

0.95

0.95

0.96

0.95

0.96

0.95

0.96

0.95

0.96

0.96

0.96

0.95

0.95

0.95

0.95

0.97

0.98

0.98

0.97

0.97

0.97

0.97

0.99

0.99

0.99

0.99

1.00

0.99

0.94

0.94

0.94

0.94

0.93

AA-APS(0.95) Coverage

1

2

3

4

5

6

79.5

115.4

136.1

126.5

111.8

86.9

16.3

24.9

27.1

28.7

30.3

23.1

9.3

9.5

12.7

10.1

11.5

8.7

1.7

1.9

2.1

2.3

2.1

1.9

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

1.0

APS(0.95) Set Size

1

2

3

4

5

6

53.6

50.9

62.2

77.7

88.4

90.4

14.2

26.0

20.5

29.4

32.8

29.6

6.7

9.6

11.2

11.1

14.0

12.5

1.7

2.0

4.1

4.9

10.4

14.0

1.0

1.0

1.2

1.3

2.0

2.7

1.0

1.0

1.0

1.0

1.0

1.0

AA-APS(0.95) Set Size

1

2

3

4

5

6

0.00-0.45

0.45-0.69

0.69-0.89

0.89-0.97

0.97-1.00

1.00-1.00

0.97

0.97

0.96

0.95

0.99

0.98

0.97

0.98

0.96

0.95

0.99

0.99

0.99

0.98

0.98

0.96

0.99

0.99

0.98

0.98

0.98

0.97

0.99

0.99

0.99

0.99

1.00

0.99

0.93

0.89

0.82

0.82

0.76

0.79

0.94

0.92

RAPS(0.95) Coverage

1

2

3

4

5

6

0.97

0.95

0.95

0.97

0.95

0.96

0.95

0.96

0.96

0.95

0.95

0.96

0.96

0.95

0.96

0.96

0.97

0.95

0.95

0.96

0.98

0.98

0.97

0.96

0.97

0.97

0.99

0.99

0.99

0.99

1.00

0.99

0.94

0.94

0.93

0.94

AA-RAPS(0.95) Coverage

1

2

3

4

5

6

9.1

9.3

9.5

9.5

9.4

9.4

7.7

8.1

8.1

8.3

8.4

8.1

7.3

7.5

7.8

8.0

7.9

7.4

6.6

6.7

6.8

6.8

6.9

6.5

3.7

3.7

3.8

3.6

3.7

3.4

1.0

1.0

1.0

1.0

1.0

1.0

RAPS(0.95) Set Size

1

2

3

4

5

6

18.5

25.1

33.2

49.7

64.9

73.8

4.7

6.5

10.3

13.0

17.3

18.9

3.5

4.7

4.8

6.4

7.0

9.4

1.7

2.0

3.9

3.1

4.5

5.8

1.0

1.1

1.2

1.2

1.4

2.5

1.0

1.0

1.0

1.0

1.0

1.0

AA-RAPS(0.95) Set Size

1

2

3

4

5

6

Confidence Quantile

0.85

0.90

0.95

Coverage

Method

RAPS

APS

AA-RAPS

AA-APS

1

2

3

4

5

6

Input Atypicality Quantile

0.94

0.96

0.98

Coverage

1

2

3

4

5

6

Confidence Quantile

0

25

50

75

100

SetSize

1

2

3

4

5

6

Input Atypicality Quantile

10

20

30

SetSize

F Fitzpatrick17k and Skin Lesion Classification

We use the training script from [GHS + 21] to finetune models on the Fitzpatrick17k dataset. We train the models for 50 epochs, fixing the backbone and training only the probe on top of the penultimate layer. The probe consists of 2 layers, one layer of 256 units followed by ReLU and Dropout with probability 0 . 4 , followed by the classifier layer with an output dimensionality of 9 . We use an Adam optimizer with a 0 . 0001 learning rate.

The entire dataset consists of 16 , 577 images, where the potential labels are: 10 , 886 inflammatory, 1 , 352 malignant epidermal, 1 , 194 genodermatoses, 1 , 067 benign dermal, 931 benign epidermal, 573 malignant melanoma, 236 benign melanocyte, 182 malignant cutaneous lymphoma, and 156 malignant dermal. We split the dataset into 3 sets (Training ( 0 . 5 ), Validation ( 0 . 25 ), and Test ( 0 . 25 )). We use the validation set as the calibration set and perform the experiments with 10 random splits.

G Proofs

G.1 Detailed derivation of the claim on Page 5

When P 1 ( X ) ≤ 1 / 2 , the signed calibration error at level u ∈ (1 / 2 , 1) becomes u -P ( Y = -1 | ˆ P -1 ( X ) = u ) = u -P ( Y = -1 | ˆ P 1 ( -X ) = u ) = u -P ( Y = 1 | ˆ P 1 ( X ) = u ) .

The last inequality is due to symmetry. More specifically, we claim ( X,Y ) d = ( -X, -Y ) , where the notation d = denotes equal in distribution. In fact, as X d = -X , it suffices to show that for any y ∈ {-1 , 1 } , and x ∈ R d , we have

When y = -1 , the right hand side

Similarly, when y = 1 ,

We complete the proof.

G.2 Proof of Theorem 3.1

Theorem G.1 (Restatement of Theorem 3.1) . Consider the data generative model with the algorithm described in Section 3.2. For any K > 1 , suppose we consider the quantiles of a ( X ) , a 1 , a 2 , ..., a K , a K +1 such that P ( a ( X ) ∈ ( a k , a k +1 ]) = 1 /K for k ∈ [ K ] . In addition, we assume ∥ β ∗ ∥ ≤ c 0 , and d/n = κ for some sufficiently small c 0 , κ > 0 . Then for sufficiently large n , we have

for k = 2 , .., K .

Proof. Following [BMWX21a], we have

where cos ˆ θ = ˆ β ⊤ β ∗ ∥ ˆ β ∥·∥ β ∗ ∥ and Z ∼ N (0 , 1) .

According to the results in Section 2.2 of [SC19], we have ∥ ˆ β ∥ → R ∗ = R ∗ ( κ, β ∗ ) and cos ˆ θ → c ∗ = c ∗ ( κ, β ∗ ) , for two quantities R ∗ and c ∗ that depend on κ and β ∗ . We then have

Using the proof of Theorem 3 in [BMWX21a], we have that

where for two positive constants c 1 , c 2 .

As a result, we have

In addition, since when z ∈ [ -1 , 1] , z · σ ′ ( z ) and -σ ′′ ( z ) are both increasing, we then have C κ ( u ) increasing for ˆ β ⊤ x = σ -1 ( u ) ∈ ( -1 , 1) .

Proving the result for { k = 2 , . . . , K -1 } In addition, by our model assumption x ∼ N (0 , I d ) , we have that ∥ x ∥ and x ∥ x ∥ are independent, and x ∥ x ∥ ∼ S where S is a uniform distribution on the sphere in the d -dimensional space. As the monotonic transformations will not change the events defined by quantiles, and exp( -∥ x ∥ 2 / 2) is a monotonic function in ∥ x ∥ , for the simplicity of presentation we use a ( X ) = ∥ X ∥ in the rest of this proof. As a result, given ∥ x ∥ = a , we have

where S 1 is the first coordinate of S .

Consequently, if we further condition on the event where ˆ β ⊤ x > 0 (as we assume u > 0 throughout Section 3.2), we have

where Q ∼ χ 2 p -1 , Z 1 ∼ N (0 , 1) and they are independent.

Due to the monotonicity of C κ ( u ) on u , we have that for any a 1 > a 2 ,

where the notation d > denotes stochastic dominance.

Consequently, we have

Proving the result for k = K To complete the proof, it suffices to show that the inequality is also true for K th quantile:

E u [ u -P ( Y = 1 | ˆ P 1 ( X ) = u ) | a ( X ) ∈ [ a K -1 , a K ]] < E u [ u -P ( Y = 1 | ˆ P 1 ( X ) = u ) | a ( X ) ∈ ( a k , a k +1 ]] , which is equivalent to

In the above inequality, the right hand side can be decomposed into

Denote the α quantile of χ 2 p by χ 2 α,p . We then have a k = χ 2 k K +1 ,p . We further decompose the equation into

In the following, we proceed to prove

We now use the approximation of the chi-square quantile: when p →∞ , we have

where z denotes the α -quantile of a standard normal random variable.

α Then

Using the fact that z 1 -1 K = √ 2 log K + o (1) for K →∞ , then we have

In addition, for any a ∈ [ χ 2 K + δ K +1 ,p , 2 p ] and a ′ ∈ [ a K -1 , a K ] , we have

for some universal constant C .

Therefore

Then

The last equality uses the fact that P ( a ( X ) ∈ [ a K -1 , a K ]) = 1 /K, and therefore

Then use the fact that | E u [( u -P ( Y = 1 | ˆ P 1 ( X ) = u )) | a ( X ) ∈ [ a K -1 , a K ]] | = O (1) and we choose δ = o (1 / log K ) so

Consequently,

which implies

Combining with equation 13, we prove equation 14 and complete the proof.

G.3 Theoretical Justification of the calibration improvement using the atypicality score

In this section, we provide the theoretical justification to understand why incorporating the atypicality score will improve calibration. In particular, we consider the binary classification problem with prediction f : X ∈ [0 , 1] indicating the predicted probability of Y = 1 given X = x .

For a predictor f , let us denote its conditional calibration error at an atypicality level γ by CE γ ( f ) = E [( f ( X ) -E [ Y | f ( X )]) 2 | a ( X ) = γ ] .

Theorem G.2. Consider the same setting as Theorem 3.1. Suppose the temperature function ˆ τ ( a ( X )) = arg min τ E [ l ( Y, Softmax ( f ( X ) /τ ( a ( X ))))] with l being the cross entropy loss, and let ˆ P AAR ( X ) = Softmax ( f ( X ) / ˆ τ ( a ( X ))) . Then

Proof: For a prediction function f , we first define the conditional mean squared error of f at an atypicality level γ by MSE γ ( f ) = E [( f ( X ) -Y ) 2 | a ( X ) = γ ] , then we have

Since we have

Now that ˆ P AAR ( f ( x ) , a ( x )) is monotonic on the ˆ P ( x ) , we have

implying

Similarly, we have

In the following, we will show that

First, as we consider the binary classification setting, with l being the cross-entropy loss, we have l ( Y, Softmax ( f ( X ) /τ ( a ( X )))) = Y log( σ ( f ( X ) /τ ( a ( X )))+(1 -Y ) log(1 -σ ( f ( X ) /τ ( a ( X ))) , where σ ( x ) = 1 / (1 + e x ) .

Then, by the definition of ˆ τ ( a ( X )) , we have that

Taking the derivative on the last line and setting it to zero, we have

implying

This makes the derivative of E [( Y -σ ( f ( X ) /τ ( a ( X )))) 2 | a ( X )] zero and therefore ˆ τ ( a ( X )) is also a minimizer of E [( Y -σ ( f ( X ) /τ ( a ( X )))) 2 | a ( X )] :

ˆ

τ

(

a

(

X

)) = arg min

τ

and therefore

As a result,

Similarly, we have MSE γ ( ˆ P AAR ) ≤ MSE γ ( ˆ P TS ) , and therefore equation 18 holds. Combining with equation 16 and equation 17, we have

E

[

Y

log(

σ

(

f

(

X

)

/τ

(

a

(

X

)))+(1

-

Y

) log(1

-

σ

(

f

(

X

)

/τ

(

a

(

X

)))] = arg min

τ

E

[(

Y

-

σ

(

f

(

X

)

/τ

(

a

(

X

2

))))

Model

Dataset

Input Atypicality Group

Confidence Group

APS

AA-APS

RAPS

AA-RAPS

Coverage

SetSize

Coverage

SetSize

Coverage

SetSize

Coverage

SetSize

ResNet152

ImageNet

1

1

0

.

982

±

0

.

002

51

.

192

±

1

.

570

0

.

970

±

0

.

003

32

.

694

±

3

.

517

0

.

951

±

0

.

004

7

.

168

±

0

.

127

0

.

963

±

0

.

004

14

.

408

±

2

.

275

2

0

.

984

±

0

.

001

80

.

425

±

2

.

506

0

.

961

±

0

.

002

39

.

752

±

1

.

670

0

.

940

±

0

.

003

7

.

771

±

0

.

180

0

.

966

±

0

.

004

15

.

713

±

0

.

977

3

0

.

980

±

0

.

001

114

.

526

±

1

.

293

0

.

955

±

0

.

004

55

.

887

±

2

.

830

0

.

885

±

0

.

005

8

.

485

±

0

.

217

0

.

952

±

0

.

004

25

.

191

±

1

.

349

4

0

.

979

±

0

.

001

118

.

819

±

1

.

787

0

.

949

±

0

.

002

60

.

998

±

1

.

293

0

.

833

±

0

.

005

8

.

677

±

0

.

235

0

.

949

±

0

.

002

33

.

256

±

0

.

812

5

0

.

975

±

0

.

001

111

.

696

±

1

.

090

0

.

953

±

0

.

002

66

.

884

±

1

.

347

0

.

807

±

0

.

006

8

.

801

±

0

.

242

0

.

948

±

0

.

002

42

.

109

±

0

.

829

6

0

.

958

±

0

.

002

90

.

283

±

1

.

460

0

.

949

±

0

.

002

77

.

404

±

1

.

682

0

.

777

±

0

.

005

8

.

565

±

0

.

229

0

.

946

±

0

.

003

52

.

214

±

1

.

401

2

1

0

.

961

±

0

.

004

16

.

767

±

0

.

389

0

.

953

±

0

.

004

14

.

533

±

1

.

036

0

.

977

±

0

.

002

5

.

279

±

0

.

035

0

.

954

±

0

.

005

3

.

888

±

0

.

102

2

0

.

964

±

0

.

002

18

.

359

±

0

.

530

0

.

951

±

0

.

003

12

.

902

±

0

.

814

0

.

974

±

0

.

001

5

.

801

±

0

.

046

0

.

950

±

0

.

003

4

.

484

±

0

.

079

3

0

.

961

±

0

.

001

21

.

409

±

0

.

413

0

.

948

±

0

.

002

14

.

636

±

0

.

318

0

.

966

±

0

.

002

5

.

964

±

0

.

051

0

.

952

±

0

.

003

5

.

172

±

0

.

112

4

0

.

964

±

0

.

002

23

.

162

±

0

.

321

0

.

956

±

0

.

003

20

.

246

±

0

.

634

0

.

966

±

0

.

002

6

.

253

±

0

.

081

0

.

962

±

0

.

004

6

.

485

±

0

.

112

5

0

.

953

±

0

.

002

24

.

833

±

0

.

486

0

.

952

±

0

.

003

25

.

470

±

0

.

829

0

.

938

±

0

.

002

6

.

331

±

0

.

092

0

.

953

±

0

.

003

9

.

446

±

0

.

332

6

0

.

929

±

0

.

003

21

.

124

±

0

.

290

0

.

943

±

0

.

004

32

.

341

±

1

.

343

0

.

909

±

0

.

003

6

.

409

±

0

.

106

0

.

952

±

0

.

002

16

.

970

±

0

.

828

3

1

0

.

957

±

0

.

002

6

.

843

±

0

.

249

0

.

954

±

0

.

004

5

.

860

±

0

.

354

0

.

984

±

0

.

001

4

.

839

±

0

.

046

0

.

951

±

0

.

004

3

.

015

±

0

.

098

2

0

.

963

±

0

.

003

6

.

654

±

0

.

202

0

.

965

±

0

.

003

6

.

689

±

0

.

406

0

.

986

±

0

.

001

5

.

274

±

0

.

044

0

.

966

±

0

.

003

3

.

494

±

0

.

071

3

0

.

951

±

0

.

002

7

.

609

±

0

.

195

0

.

955

±

0

.

003

8

.

542

±

0

.

472

0

.

968

±

0

.

001

5

.

435

±

0

.

042

0

.

951

±

0

.

003

4

.

052

±

0

.

097

4

0

.

938

±

0

.

002

8

.

415

±

0

.

205

0

.

955

±

0

.

002

12

.

476

±

0

.

349

0

.

961

±

0

.

002

5

.

728

±

0

.

048

0

.

951

±

0

.

002

5

.

085

±

0

.

085

5

0

.

945

±

0

.

002

8

.

500

±

0

.

204

0

.

955

±

0

.

003

12

.

539

±

0

.

815

0

.

964

±

0

.

002

5

.

724

±

0

.

050

0

.

956

±

0

.

003

5

.

531

±

0

.

154

6

0

.

932

±

0

.

004

8

.

009

±

0

.

199

0

.

953

±

0

.

004

13

.

942

±

0

.

657

0

.

944

±

0

.

003

5

.

744

±

0

.

048

0

.

953

±

0

.

003

7

.

718

±

0

.

745

4

1

0

.

958

±

0

.

001

1

.

480

±

0

.

034

0

.

962

±

0

.

003

1

.

828

±

0

.

131

0

.

985

±

0

.

001

3

.

810

±

0

.

132

0

.

962

±

0

.

003

1

.

613

±

0

.

078

2

0

.

953

±

0

.

002

1

.

388

±

0

.

021

0

.

958

±

0

.

002

1

.

913

±

0

.

118

0

.

982

±

0

.

001

3

.

816

±

0

.

141

0

.

957

±

0

.

002

1

.

692

±

0

.

070

3

0

.

931

±

0

.

002

1

.

451

±

0

.

028

0

.

947

±

0

.

003

2

.

699

±

0

.

146

0

.

967

±

0

.

002

3

.

920

±

0

.

148

0

.

951

±

0

.

003

2

.

237

±

0

.

071

4

0

.

935

±

0

.

001

1

.

450

±

0

.

020

0

.

958

±

0

.

003

3

.

488

±

0

.

224

0

.

976

±

0

.

002

3

.

880

±

0

.

128

0

.

957

±

0

.

003

2

.

577

±

0

.

117

5

0

.

928

±

0

.

002

1

.

424

±

0

.

018

0

.

951

±

0

.

003

5

.

173

±

0

.

391

0

.

963

±

0

.

002

4

.

160

±

0

.

127

0

.

952

±

0

.

002

3

.

185

±

0

.

077

6

0

.

902

±

0

.

003

1

.

476

±

0

.

028

0

.

958

±

0

.

005

8

.

900

±

0

.

726

0

.

955

±

0

.

004

3

.

925

±

0

.

130

0

.

959

±

0

.

004

4

.

235

±

0

.

163

5

1

0

.

988

±

0

.

001

1

.

000

±

0

.

000

0

.

988

±

0

.

001

1

.

000

±

0

.

000

0

.

989

±

0

.

001

1

.

344

±

0

.

116

0

.

988

±

0

.

001

1

.

000

±

0

.

000

2

0

.

982

±

0

.

001

1

.

000

±

0

.

000

0

.

982

±

0

.

001

1

.

000

±

0

.

000

0

.

984

±

0

.

001

1

.

363

±

0

.

122

0

.

982

±

0

.

001

1

.

000

±

0

.

000

3

0

.

976

±

0

.

001

1

.

000

±

0

.

000

0

.

976

±

0

.

001

1

.

001

±

0

.

001

0

.

980

±

0

.

002

1

.

351

±

0

.

122

0

.

976

±

0

.

001

1

.

001

±

0

.

001

4

0

.

977

±

0

.

001

1

.

000

±

0

.

000

0

.

977

±

0

.

001

1

.

020

±

0

.

009

0

.

980

±

0

.

001

1

.

363

±

0

.

109

0

.

977

±

0

.

001

1

.

004

±

0

.

003

5

0

.

965

±

0

.

001

1

.

000

±

0

.

000

0

.

965

±

0

.

001

1

.

137

±

0

.

055

0

.

968

±

0

.

002

1

.

342

±

0

.

115

0

.

965

±

0

.

001

1

.

063

±

0

.

028

6

0

.

970

±

0

.

002

1

.

000

±

0

.

000

0

.

972

±

0

.

003

1

.

607

±

0

.

125

0

.

973

±

0

.

003

1

.

339

±

0

.

107

0

.

973

±

0

.

003

1

.

539

±

0

.

080

6

1

0

.

992

±

0

.

000

1

.

000

±

0

.

000

0

.

992

±

0

.

000

1

.

000

±

0

.

000

0

.

992

±

0

.

000

1

.

000

±

0

.

000

0

.

992

±

0

.

000

1

.

000

±

0

.

000

2

0

.

989

±

0

.

001

1

.

000

±

0

.

000

0

.

989

±

0

.

001

1

.

000

±

0

.

000

0

.

989

±

0

.

001

1

.

000

±

0

.

000

0

.

989

±

0

.

001

1

.

000

±

0

.

000

3

0

.

992

±

0

.

000

1

.

000

±

0

.

000

0

.

992

±

0

.

000

1

.

000

±

0

.

000

0

.

992

±

0

.

000

1

.

000

±

0

.

000

0

.

992

±

0

.

000

1

.

000

±

0

.

000

4

0

.

995

±

0

.

001

1

.

000

±

0

.

000

0

.

995

±

0

.

001

1

.

000

±

0

.

000

0

.

995

±

0

.

001

1

.

000

±

0

.

000

0

.

995

±

0

.

001

1

.

000

±

0

.

000

5

0

.

989

±

0

.

001

1

.

000

±

0

.

000

0

.

989

±

0

.

001

1

.

000

±

0

.

000

0

.

989

±

0

.

001

1

.

000

±

0

.

000

0

.

989

±

0

.

001

1

.

000

±

0

.

000

6

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

ResNet18

ImageNet

1

1

0

.

982

±

0

.

001

167

.

265

±

2

.

058

0

.

952

±

0

.

004

79

.

307

±

3

.

180

0

.

876

±

0

.

008

20

.

623

±

1

.

724

0

.

952

±

0

.

003

67

.

925

±

2

.

552

2

0

.

986

±

0

.

001

157

.

710

±

1

.

132

0

.

956

±

0

.

002

85

.

303

±

1

.

015

0

.

848

±

0

.

009

20

.

740

±

1

.

745

0

.

958

±

0

.

003

80

.

333

±

3

.

054

3

0

.

970

±

0

.

001

147

.

730

±

1

.

267

0

.

943

±

0

.

003

95

.

805

±

2

.

283

0

.

810

±

0

.

009

20

.

689

±

1

.

725

0

.

938

±

0

.

003

91

.

786

±

1

.

213

4

0

.

963

±

0

.

002

129

.

188

±

1

.

035

0

.

954

±

0

.

003

109

.

238

±

1

.

837

0

.

815

±

0

.

009

20

.

328

±

1

.

592

0

.

953

±

0

.

002

102

.

926

±

2

.

651

5

0

.

957

±

0

.

002

111

.

271

±

1

.

097

0

.

952

±

0

.

002

103

.

457

±

2

.

047

0

.

809

±

0

.

011

19

.

932

±

1

.

441

0

.

942

±

0

.

004

91

.

695

±

3

.

655

6

0

.

941

±

0

.

004

82

.

127

±

1

.

167

0

.

951

±

0

.

005

96

.

879

±

4

.

165

0

.

828

±

0

.

008

19

.

108

±

1

.

138

0

.

950

±

0

.

004

87

.

490

±

3

.

277

2

1

0

.

972

±

0

.

001

37

.

918

±

1

.

031

0

.

958

±

0

.

001

25

.

001

±

0

.

687

0

.

978

±

0

.

002

15

.

096

±

0

.

135

0

.

959

±

0

.

003

10

.

650

±

0

.

322

2

0

.

959

±

0

.

002

42

.

117

±

0

.

743

0

.

951

±

0

.

004

31

.

357

±

1

.

294

0

.

954

±

0

.

002

15

.

879

±

0

.

162

0

.

951

±

0

.

004

16

.

748

±

0

.

617

3

0

.

964

±

0

.

003

43

.

312

±

0

.

868

0

.

957

±

0

.

003

39

.

329

±

1

.

425

0

.

951

±

0

.

003

16

.

138

±

0

.

225

0

.

960

±

0

.

002

22

.

619

±

1

.

125

4

0

.

951

±

0

.

003

41

.

803

±

0

.

701

0

.

955

±

0

.

003

45

.

617

±

1

.

420

0

.

930

±

0

.

002

16

.

357

±

0

.

295

0

.

953

±

0

.

003

31

.

513

±

1

.

447

5

0

.

945

±

0

.

002

36

.

697

±

0

.

482

0

.

957

±

0

.

002

45

.

379

±

1

.

101

0

.

935

±

0

.

002

16

.

094

±

0

.

249

0

.

961

±

0

.

001

32

.

569

±

0

.

974

6

0

.

930

±

0

.

003

27

.

148

±

0

.

321

0

.

949

±

0

.

005

43

.

307

±

2

.

384

0

.

924

±

0

.

003

15

.

730

±

0

.

144

0

.

948

±

0

.

005

28

.

152

±

1

.

657

3

1

0

.

964

±

0

.

001

15

.

809

±

0

.

414

0

.

954

±

0

.

002

13

.

080

±

0

.

586

0

.

985

±

0

.

002

13

.

208

±

0

.

471

0

.

955

±

0

.

002

6

.

864

±

0

.

146

2

0

.

960

±

0

.

002

18

.

034

±

0

.

515

0

.

956

±

0

.

003

16

.

515

±

0

.

618

0

.

978

±

0

.

001

13

.

659

±

0

.

394

0

.

955

±

0

.

002

8

.

275

±

0

.

221

3

0

.

964

±

0

.

002

19

.

231

±

0

.

344

0

.

957

±

0

.

003

17

.

463

±

0

.

722

0

.

979

±

0

.

001

13

.

901

±

0

.

310

0

.

957

±

0

.

003

8

.

794

±

0

.

163

4

0

.

950

±

0

.

001

19

.

479

±

0

.

207

0

.

952

±

0

.

003

21

.

470

±

0

.

744

0

.

974

±

0

.

001

13

.

899

±

0

.

304

0

.

955

±

0

.

003

11

.

390

±

0

.

313

5

0

.

940

±

0

.

002

17

.

801

±

0

.

160

0

.

953

±

0

.

003

22

.

066

±

0

.

545

0

.

957

±

0

.

002

14

.

036

±

0

.

296

0

.

951

±

0

.

003

12

.

743

±

0

.

271

6

0

.

931

±

0

.

002

13

.

096

±

0

.

153

0

.

949

±

0

.

004

21

.

352

±

1

.

080

0

.

956

±

0

.

002

13

.

169

±

0

.

394

0

.

949

±

0

.

005

12

.

924

±

0

.

663

4

1

0

.

964

±

0

.

002

5

.

377

±

0

.

082

0

.

957

±

0

.

003

4

.

595

±

0

.

226

0

.

990

±

0

.

002

11

.

670

±

0

.

870

0

.

957

±

0

.

004

3

.

823

±

0

.

144

2

0

.

958

±

0

.

002

6

.

371

±

0

.

130

0

.

954

±

0

.

003

5

.

887

±

0

.

275

0

.

986

±

0

.

003

11

.

954

±

0

.

767

0

.

953

±

0

.

003

4

.

648

±

0

.

106

3

0

.

946

±

0

.

002

5

.

810

±

0

.

109

0

.

953

±

0

.

002

7

.

575

±

0

.

321

0

.

984

±

0

.

003

12

.

145

±

0

.

751

0

.

954

±

0

.

003

5

.

364

±

0

.

157

4

0

.

940

±

0

.

001

6

.

698

±

0

.

159

0

.

950

±

0

.

003

9

.

025

±

0

.

498

0

.

980

±

0

.

002

12

.

432

±

0

.

700

0

.

950

±

0

.

004

6

.

110

±

0

.

225

5

0

.

926

±

0

.

003

5

.

817

±

0

.

102

0

.

949

±

0

.

004

10

.

234

±

0

.

682

0

.

977

±

0

.

003

12

.

325

±

0

.

750

0

.

950

±

0

.

004

6

.

925

±

0

.

198

6

0

.

915

±

0

.

002

4

.

948

±

0

.

098

0

.

949

±

0

.

003

11

.

054

±

0

.

601

0

.

968

±

0

.

004

11

.

478

±

0

.

813

0

.

948

±

0

.

003

7

.

368

±

0

.

183

5

1

0

.

963

±

0

.

002

1

.

193

±

0

.

008

0

.

968

±

0

.

002

1

.

497

±

0

.

052

0

.

989

±

0

.

002

8

.

559

±

1

.

391

0

.

967

±

0

.

002

1

.

413

±

0

.

041

2

0

.

959

±

0

.

002

1

.

232

±

0

.

014

0

.

963

±

0

.

002

1

.

456

±

0

.

063

0

.

984

±

0

.

003

8

.

755

±

1

.

352

0

.

962

±

0

.

002

1

.

485

±

0

.

057

3

0

.

957

±

0

.

001

1

.

243

±

0

.

012

0

.

961

±

0

.

002

1

.

693

±

0

.

085

0

.

986

±

0

.

003

8

.

947

±

1

.

344

0

.

961

±

0

.

002

1

.

637

±

0

.

071

4

0

.

945

±

0

.

002

1

.

264

±

0

.

013

0

.

959

±

0

.

002

2

.

492

±

0

.

131

0

.

985

±

0

.

003

8

.

933

±

1

.

325

0

.

959

±

0

.

002

2

.

309

±

0

.

082

5

0

.

943

±

0

.

002

1

.

234

±

0

.

013

0

.

953

±

0

.

002

2

.

770

±

0

.

345

0

.

976

±

0

.

004

8

.

913

±

1

.

365

0

.

954

±

0

.

002

2

.

540

±

0

.

195

6

0

.

932

±

0

.

002

1

.

214

±

0

.

010

0

.

961

±

0

.

002

3

.

875

±

0

.

249

0

.

976

±

0

.

004

8

.

459

±

1

.

351

0

.

963

±

0

.

003

3

.

635

±

0

.

192

6

1

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

993

±

0

.

001

4

.

540

±

1

.

729

0

.

991

±

0

.

001

1

.

000

±

0

.

000

2

0

.

986

±

0

.

001

1

.

000

±

0

.

000

0

.

986

±

0

.

001

1

.

000

±

0

.

000

0

.

990

±

0

.

002

4

.

395

±

1

.

738

0

.

986

±

0

.

001

1

.

000

±

0

.

000

3

0

.

988

±

0

.

001

1

.

000

±

0

.

000

0

.

988

±

0

.

001

1

.

000

±

0

.

000

0

.

992

±

0

.

001

4

.

449

±

1

.

739

0

.

988

±

0

.

001

1

.

000

±

0

.

000

4

0

.

988

±

0

.

001

1

.

000

±

0

.

000

0

.

988

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

002

4

.

481

±

1

.

729

0

.

988

±

0

.

001

1

.

000

±

0

.

000

5

0

.

986

±

0

.

001

1

.

000

±

0

.

000

0

.

986

±

0

.

001

1

.

000

±

0

.

000

0

.

990

±

0

.

002

4

.

390

±

1

.

741

0

.

986

±

0

.

001

1

.

000

±

0

.

000

6

0

.

981

±

0

.

001

1

.

000

±

0

.

000

0

.

981

±

0

.

001

1

.

003

±

0

.

003

0

.

987

±

0

.

002

4

.

146

±

1

.

767

0

.

981

±

0

.

001

1

.

009

±

0

.

007

ResNet50

ImageNet

1

1

0

.

980

±

0

.

002

80

.

190

±

1

.

594

0

.

955

±

0

.

004

50

.

203

±

2

.

778

0

.

945

±

0

.

002

9

.

874

±

0

.

169

0

.

964

±

0

.

002

16

.

099

±

0

.

593

2

0

.

986

±

0

.

002

118

.

839

±

1

.

737

0

.

955

±

0

.

003

52

.

066

±

2

.

094

0

.

896

±

0

.

003

10

.

681

±

0

.

209

0

.

950

±

0

.

003

25

.

545

±

0

.

845

3

0

.

979

±

0

.

001

134

.

702

±

0

.

697

0

.

951

±

0

.

003

60

.

743

±

1

.

012

0

.

848

±

0

.

004

11

.

056

±

0

.

228

0

.

940

±

0

.

002

35

.

518

±

0

.

870

4

0

.

976

±

0

.

001

126

.

892

±

0

.

939

0

.

952

±

0

.

003

79

.

627

±

2

.

106

0

.

835

±

0

.

004

11

.

079

±

0

.

230

0

.

953

±

0

.

004

51

.

339

±

1

.

445

5

0

.

964

±

0

.

001

114

.

559

±

0

.

975

0

.

954

±

0

.

003

93

.

609

±

3

.

277

0

.

797

±

0

.

005

11

.

011

±

0

.

232

0

.

942

±

0

.

003

63

.

441

±

1

.

422

6

0

.

949

±

0

.

002

88

.

034

±

0

.

799

0

.

950

±

0

.

003

88

.

418

±

1

.

898

0

.

806

±

0

.

005

10

.

744

±

0

.

221

0

.

957

±

0

.

002

62

.

049

±

1

.

862

2

1

0

.

961

±

0

.

002

17

.

461

±

0

.

533

0

.

952

±

0

.

004

14

.

757

±

0

.

785

0

.

976

±

0

.

002

7

.

080

±

0

.

116

0

.

956

±

0

.

004

5

.

205

±

0

.

127

2

0

.

967

±

0

.

002

24

.

896

±

0

.

732

0

.

958

±

0

.

002

20

.

892

±

1

.

441

0

.

965

±

0

.

002

8

.

013

±

0

.

106

0

.

958

±

0

.

002

6

.

879

±

0

.

191

3

0

.

961

±

0

.

001

29

.

101

±

0

.

759

0

.

947

±

0

.

003

21

.

754

±

1

.

123

0

.

963

±

0

.

001

8

.

134

±

0

.

126

0

.

956

±

0

.

003

8

.

773

±

0

.

321

4

0

.

958

±

0

.

002

30

.

078

±

0

.

379

0

.

956

±

0

.

003

28

.

199

±

0

.

951

0

.

946

±

0

.

002

8

.

398

±

0

.

129

0

.

951

±

0

.

003

11

.

264

±

0

.

347

5

0

.

944

±

0

.

001

29

.

401

±

0

.

357

0

.

946

±

0

.

002

32

.

362

±

1

.

180

0

.

937

±

0

.

003

8

.

559

±

0

.

143

0

.

952

±

0

.

002

15

.

040

±

0

.

547

6

0

.

939

±

0

.

002

22

.

100

±

0

.

488

0

.

954

±

0

.

004

33

.

527

±

1

.

955

0

.

922

±

0

.

003

8

.

269

±

0

.

135

0

.

955

±

0

.

001

18

.

645

±

0

.

283

3

1

0

.

958

±

0

.

002

9

.

385

±

0

.

202

0

.

953

±

0

.

002

7

.

633

±

0

.

346

0

.

988

±

0

.

001

6

.

471

±

0

.

129

0

.

955

±

0

.

002

3

.

820

±

0

.

076

2

0

.

951

±

0

.

003

9

.

619

±

0

.

194

0

.

949

±

0

.

005

8

.

739

±

0

.

516

0

.

977

±

0

.

002

6

.

919

±

0

.

118

0

.

950

±

0

.

004

4

.

648

±

0

.

082

3

0

.

957

±

0

.

002

11

.

449

±

0

.

274

0

.

957

±

0

.

003

10

.

853

±

0

.

446

0

.

975

±

0

.

001

7

.

163

±

0

.

112

0

.

954

±

0

.

004

5

.

130

±

0

.

102

4

0

.

953

±

0

.

001

10

.

809

±

0

.

223

0

.

951

±

0

.

002

10

.

546

±

0

.

396

0

.

972

±

0

.

001

7

.

600

±

0

.

113

0

.

952

±

0

.

003

5

.

872

±

0

.

235

5

0

.

947

±

0

.

002

11

.

507

±

0

.

228

0

.

951

±

0

.

003

14

.

754

±

0

.

650

0

.

966

±

0

.

003

7

.

414

±

0

.

114

0

.

953

±

0

.

004

6

.

874

±

0

.

157

6

0

.

920

±

0

.

003

9

.

150

±

0

.

188

0

.

951

±

0

.

005

20

.

041

±

1

.

624

0

.

944

±

0

.

002

7

.

143

±

0

.

101

0

.

950

±

0

.

002

9

.

505

±

0

.

576

4

1

0

.

955

±

0

.

002

1

.

739

±

0

.

025

0

.

953

±

0

.

002

1

.

667

±

0

.

070

0

.

985

±

0

.

002

5

.

088

±

0

.

196

0

.

954

±

0

.

002

1

.

574

±

0

.

038

2

0

.

948

±

0

.

002

1

.

958

±

0

.

016

0

.

950

±

0

.

002

1

.

964

±

0

.

083

0

.

983

±

0

.

001

5

.

285

±

0

.

189

0

.

951

±

0

.

003

1

.

845

±

0

.

061

3

0

.

936

±

0

.

003

2

.

082

±

0

.

034

0

.

955

±

0

.

002

4

.

255

±

0

.

202

0

.

976

±

0

.

002

5

.

495

±

0

.

164

0

.

957

±

0

.

003

3

.

423

±

0

.

123

4

0

.

936

±

0

.

003

2

.

173

±

0

.

045

0

.

955

±

0

.

003

4

.

508

±

0

.

303

0

.

973

±

0

.

002

5

.

573

±

0

.

170

0

.

952

±

0

.

002

3

.

013

±

0

.

110

5

0

.

928

±

0

.

002

2

.

071

±

0

.

041

0

.

953

±

0

.

003

8

.

483

±

0

.

789

0

.

968

±

0

.

002

5

.

720

±

0

.

167

0

.

951

±

0

.

002

4

.

338

±

0

.

138

6

0

.

895

±

0

.

004

1

.

932

±

0

.

026

0

.

950

±

0

.

005

10

.

274

±

1

.

147

0

.

950

±

0

.

003

5

.

379

±

0

.

151

0

.

948

±

0

.

004

5

.

270

±

0

.

220

5

1

0

.

979

±

0

.

001

1

.

000

±

0

.

000

0

.

979

±

0

.

001

1

.

001

±

0

.

001

0

.

985

±

0

.

001

1

.

918

±

0

.

222

0

.

979

±

0

.

001

1

.

000

±

0

.

000

2

0

.

982

±

0

.

001

1

.

000

±

0

.

000

0

.

982

±

0

.

001

1

.

015

±

0

.

008

0

.

985

±

0

.

001

1

.

880

±

0

.

221

0

.

982

±

0

.

001

1

.

019

±

0

.

007

3

0

.

964

±

0

.

002

1

.

000

±

0

.

000

0

.

964

±

0

.

002

1

.

126

±

0

.

028

0

.

971

±

0

.

002

1

.

977

±

0

.

222

0

.

965

±

0

.

002

1

.

178

±

0

.

045

4

0

.

972

±

0

.

002

1

.

000

±

0

.

000

0

.

973

±

0

.

002

1

.

186

±

0

.

047

0

.

978

±

0

.

002

1

.

958

±

0

.

205

0

.

973

±

0

.

002

1

.

156

±

0

.

050

5

0

.

968

±

0

.

002

1

.

000

±

0

.

000

0

.

971

±

0

.

002

1

.

417

±

0

.

102

0

.

974

±

0

.

002

1

.

911

±

0

.

222

0

.

971

±

0

.

002

1

.

293

±

0

.

067

6

0

.

946

±

0

.

002

1

.

000

±

0

.

000

0

.

957

±

0

.

002

1

.

842

±

0

.

143

0

.

956

±

0

.

003

1

.

833

±

0

.

193

0

.

957

±

0

.

002

1

.

763

±

0

.

112

6

1

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

2

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

0

.

991

±

0

.

001

1

.

000

±

0

.

000

3

0

.

994

±

0

.

001

1

.

000

±

0

.

000

0

.

994

±

0

.

001

1

.

000

±

0

.

000

0

.

994

±

0

.

001

1

.

000

±

0

.

000

0

.

994

±

0

.

001

1

.

000

±

0

.

000

4

0

.

992

±

0

.

001

1

.

000

±

0

.

000

0

.

992

±

0

.

001

1

.

000

±

0

.

000

0

.

992

±

0

.

001

1

.

000

±

0

.

000

0

.

992

±

0

.

001

1

.

000

±

0

.

000

5

0

.

992

±

0

.

001

1

.

000

±

0

.

000

0

.

992

±

0

.

001

1

.

000

±

0

.

000

0

.

992

±

0

.

001

1

.

000

±

0

.

000

0

.

992

±

0

.

001

1

.

000

±

0

.

000

6

0

.

987

±

0

.

001

1

.

000

±

0

.

000

0

.

987

±

0

.

001

1

.

000

±

0

.

000

0

.

987

±

0

.

001

1

.

000

±

0

.

000

0

.

987

±

0

.

001

1

.

000

±

0

.

000

(a)

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.02

0.04

ECE

ResNet18 - ImageNet

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.03

0.05

0.08

ECE

ResNet50 - ImageNet

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.03

0.05

ECE

RoBERTa - MNLI

0.25

0.50

0.75

1.00

Input Atypicality Quantile

0.00

0.10

ECE

WideResNet28 - CIFAR10

Uncalibrated

TS

AdaTS

AAR(Ours)

(b)

0.25

0.50

0.75

1.00

Class Atypicality Quantile

0.00

0.20

ECE

0.0

0.5

0.44

0.44

0.44

0.50

0.25

0.50

0.75

1.00

Class Atypicality Quantile

0.10

0.20

ECE

0.0

0.5

0.47

0.47

0.47

0.48

0.25

0.50

0.75

1.00

Class Atypicality Quantile

0.20

0.40

ECE

0.0

0.2

0.30

0.30

0.30

0.37

0.25

0.50

0.75

1.00

Class Atypicality Quantile

0.20

0.40

ECE

0.0

0.5

0.45

0.45

0.45

0.50

ResNext50 - ImageNet-LT

ResNext50(+P.B.) - ImageNet-LT

ResNet152 - Places365-LT

ResNet18 - CIFAR100-LT

Uncalibrated

TS

AdaTS

AAR(Ours)

H Limitations

H.1 Quantifying Atypicality

Since we do not have access to the true distribution of P ( X ) , we estimate it through the model, e.g. using the embeddings. This means we are capturing the atypicality not solely with respect to the training distribution but also the model. It is possible that a model that does not fit the data well and produces low-quality atypicality estimates. We would like to stress that our goal here is to show that even simple estimators can demonstrate significant benefits. In general, we observe that our findings hold for large datasets and widely used models, and atypicality gives a semantically meaningful way to group data points qualitatively. Our findings suggest that we can unify the understanding and improve uncertainty quantification and recalibration methods with atypicality, however, practitioners should be careful about incorporating atypicality, as poor atypicality estimates can lead to worse performance.

H.2 Subgroup Fairness Experiments

While the literature on algorithms to satisfy group fairness is rich [KGZ19, HJKRR18], here we wanted to give a case study with skin-lesion classification. Our goal was to provide further evidence that atypicality awareness could improve fairness algorithms. In this domain, there is more verification to do to better characterize how and when atypicality helps, thus, to better understand which subgroups could benefit more from atypicality-aware algorithms and whether these findings apply generally in the literature.

H.3 Theoretical Analysis

Following the earlier work [BMWX21a, SC19], we analyzed the calibration behavior of wellspecified logistic regression. However, our empirical findings suggest that the phenomena are much more broadly applicable. We suggest that future work can analyze the behavior in more general settings to better understand the dynamics.

Distribution of Input Atypicality

250

500

750

1000

1250

0

200

400

600

800

1000

Count

ResNet18 on ImageNet (GMM)

10

15

20

25

0

200

400

600

800

Count

ResNet18 on ImageNet (KNN)

1

2

3

4

5

Label Atypicality Quantile

0.03-0.26

0.26-0.44

0.44-0.66

0.66-0.91

0.91-1.00

Confidence

0.21

0.17

0.16

0.12

0.06

0.38

0.34

0.30

0.24

0.11

0.55

0.51

0.46

0.37

0.14

0.77

0.75

0.65

0.45

0.18

0.97

0.93

0.86

0.62

0.16

ResNext50 on ImageNet-LT

Bm

)

1

2

3

4

5

Label Atypicality Quantile

0.03-0.26

0.26-0.44

0.44-0.66

0.66-0.91

0.91-1.00

Confidence

-0.03

0.01

0.01

0.06

0.12

-0.03

0.01

0.05

0.11

0.24

-0.00

0.04

0.08

0.17

0.39

0.03

0.05

0.14

0.33

0.59

0.01

0.04

0.10

0.34

0.79

ResNext50 on ImageNet-LT

Confidence(

Bm

)-Accuracy(

Bm

)

1

2

3

4

5

Input Atypicality Quantile

0.06-0.63

0.63-0.90

0.90-0.99

0.99-1.00

1.00-1.00

Confidence

0.44

0.44

0.38

0.35

0.32

0.74

0.71

0.71

0.67

0.62

0.91

0.91

0.88

0.86

0.85

0.98

0.96

0.96

0.96

0.96

0.99

0.99

0.99

0.99

0.99

ResNet152 on ImageNet

Bm

)

1

2

3

4

5

Input Atypicality Quantile

0.06-0.63

0.63-0.90

0.90-0.99

0.99-1.00

1.00-1.00

Confidence

0.06

0.03

0.05

0.07

0.11

0.04

0.08

0.08

0.11

0.16

0.04

0.05

0.08

0.09

0.10

0.02

0.03

0.04

0.04

0.04

0.01

0.01

0.01

0.01

0.01

ResNet152 on ImageNet

Confidence(

Bm

)-Accuracy(

Bm

)

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the question of uncertainty quantification in classification by introducing the atypicality during recalibration. This paper exhibits a highly explicit motivation, yet it suffers from certain deficiencies in definitions and errors therein, and I harbor reservations regarding the soundness of the approach posited in Sec. 4.2 Atypicality-Aware Recalibration.\n\nSTRENGTHS:\nThis paper evinces a highly explicit motivation and lucidly illustrates the concept of atypicality posited therein through the use of schematic diagrams.\n\nThe paper presents copious experimental results and supplements them with additional experiments in the appendix, which serves to facilitate the reader's comprehension of the experimental outcomes.\n\nWEAKNESSES:\nThe presence of errors and deficiencies in definitions within the article renders this work somewhat unreliable.\n\nAlthough the article contains numerous derivations, the critical section, namely Sec. 4.2, lacks theoretical analysis. The author fails to analyze the validity and necessity of Eq. 2, and whether there exist alternative methods to the approach posited therein, or whether it is indeed necessary. These issues require requisite analysis.\n\nQUESTIONS:\nQ1: In Fig.~1, it is unclear whether the example presented therein is a genuine instance from the experiment. If it is, then it is puzzling why specific values for atypicality and confidence are not provided, and instead, actual and predicted labels are employed.\n\nQ2: There are errors in definition of accuracy(Bm), what's worse, I can't find the formal defination of Bm.\n\nQ3: RMSE, which is used as a metric in the paper, but I can't find the definition of it. To my understanding, RMSE is a metric for regression rather than calibration.\n\nQ4: The necessity of Eq. 2, and whether there exist alternative methods to the approach posited therein, or whether it is indeed necessary. I could understand Sec. 4.1, and the natural selution is group-wise TS calibration, so more detailed analysis is required.\n\nQ5: Atypicality for LLMs. The defination in line 124 a(x) is a distribution, it seems different to the defination 2.1.\n\nQ6: How is the expected calibration error (ECE) computed on large-scale models? Is it obtained by taking the softmax of all possible values along the final dimension, or by normalizing the top-p values as confidence scores?\n\nNote: Use \"Line xx\" to describe the Eqs in the text is cumbersome and detracts from the readability of the communication.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-13"}

comments: SUMMARY:
This paper is addressing the problem of atypicality in data, and how this impact performance and confidence.

STRENGTHS:
Good perspective on the needs to consider atypicality in data
Good presentation
Good review on the links between atypicality in data and performance with confidence

WEAKNESSES:
N/A

QUESTIONS:
N/a

LIMITATIONS:
N/a
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-06-13

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a series of \"atypicality\" measures to be used in complement to the more popular uncertainty metrics.\nThe authors defined input and class atypicality for both classical classification tasks as well as NLG.\nSuch atypicality measures are then combined with temperature scaling to improve calibration quality, as well as prediction sets (RAPS).\n\nSTRENGTHS:\n1. The problem is definitely important, and this paper touches a lot of areas where the particular measures of \"atypicality\" could help.\n2. Linking atypicality to conformal prediction is good. (I'd suggest using \"prediction set\" as opposed to \"uncertainty set\" though). This seems to lead to good results in Figure 5 as well. \n3. AAR seems to improve ECE over the baselines significantly. \n4. I actually think this paper has a lot of interesting ideas, such as Theorem 3.1 (but somehow constrained by the space and many are not fully developed.)\n\nWEAKNESSES:\n1. I really think it is inappropriate to make the dichotomy of \"Atypicality\" vs \"Confidence/Uncertainty\". In fact, I would argue the various forms of atypicality in this paper are just a bunch of confidence measures. This paper actually also proposes several different variants of \"atypicality\", and in the uncertainty space, for example, monte-carlo dropout variance and entropy are both used to estimate uncertainty. Thus, I'm not sure what makes \"atypicality\" a separate concept. \n\n2. In my opinion, this paper is proposing another calibration method, but puts everything in very different languages. \nIt is also missing a lot of calibration baselines. \nIn fact, recent works rarely focus on confidence calibration anymore, but more on full calibrtion [[1]](https://proceedings.neurips.cc/paper/2019/hash/8ca01ea920679a0fe3728441494041b9-Abstract.html), [[2]](http://proceedings.mlr.press/v89/vaicenavicius19a/vaicenavicius19a.pdf), [[3]](https://proceedings.neurips.cc/paper/2019/file/1c336b8080f82bcc2cd2499b4c57261d-Paper.pdf), [[4]](https://openreview.net/forum?id=p_jIy5QFB7).\nIn fact, [4] also used density estimates to calibrate classifiers much like L115-L117 and Definition 2.1\nI also think the \"Recalibration\" section in Section 6 needs a re-write, as it's uncommon to put conformal prediction in this context: Yes, conformal prediction could be considered as calibrating the distribution/p-value, but when we use ECE to measure model calibration, conformal prediction is just out of context. \n\n3. Related to 2, I think the improvement of RAPS/APS would benefit mostly from full calibration as opposed to confidence calibration. The lack of such experiment is a limitation. \n\n4. While the improvement of LLM performance is good, I don't see the connection between the main point of the paper and such experiments. See Questions as well.\n\n5. Presentation is not clear, probably due to the attempt to jam in too much content. For example, L122-126 should be expanded as it is a very different thing in my opinion.\n\nQUESTIONS:\n1. Related to W4, why is atypicality defined so differently for NLP tasks? The intro keeps suggesting that the input is atypical wrt a class (e.g. Figure 1). In Definition 2.1, for example, it is also conditioned on $Y=y$. However, L124 uses a completely different definition. In fact, even L125 ($a_Y(y)$) has a completely different meaning than Definition 2.2. \n2. Intuitively, what's the rationale of assigning temperatures basing on atypicality? It seems like typical samples could share a temperature, but for atypical ones, it's not like atypical samples themselves are similar.\n3. Are the theoretical guarantees still maintained after the modifications to RAPS?\n4. Is Theorem 3.1 somewhat mechanical? That is, is this just due to large variance and the fact that $u-\\mathbb{P}(Y=1|\\hat{\\mathbb{P}}_1(X) = u)$ tends to mean reverse?\n\nLIMITATIONS:\nMostly discussed in weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper questions the reliance of probabilistic classifiers on the confidence score alone for measuring reliability. This is an important question that has not been asked that rigorously in the machine learning literature. In particular, there are two notions of uncertainty: aleatoric and epistemic. A widespread consensus is to consider the softmax applied output of a neural network as the confidence of the classifier. However, it is not clear what kind of uncertainty that score captures. Literature in proper scoring rule would suggest that it measures the posterior probability of the input falling into some class, which is inherently an aleatoric notion of uncertainty. However, practitioners and researchers in the community also hope that such a score would be measure of reliability of rare / out-of-distribution / atypical samples, which is more of an epistemic nature of uncertainty. Therefore, there is some sort of conflation is happening on the nature of this score. A sample could be low confidence because it is inherently ambiguous (aleatoric) or is not well represented in the training distribution (epistemic). The paper opens up with this question, and gives arguments to quantify *atypicality* of the sample as well for more reliability. With a simple measure to quantify atypicality, it shows that there is a a correspondence between how atypical a sample is and how mis-calibrated it is. It then studies atypicality aware recalibration. There is a sufficient empirical evaluation.\n\nSTRENGTHS:\n1. One of the major strengths of this paper is that it raises a very relevant question. Interpretation of the confidence outputted by the classifier has a tendency to suffer from serious conflation issues where it is not clear what it captures. The paper argues for a different notion that can settle this issue (or at least start interesting discussions in the community). I believe this is highly valuable.\n\n2. While the operationalisation of the proposed atypicality score is simple, the scoring criteria is already good enough to support the arguments made in the paper in a satisfactory way.\n\n3. Thorough empirical evaluation is done on different settings ranging from balanced supervised classification, imbalanced classification , Language modelling, recalibration, and conformal prediction uncertainty sets on a range of datasets.\n\nWEAKNESSES:\n1. There are no major glaring weaknesses of the paper. Except that maybe it underplays itself, as in my mind, it puts interesting and insightful commentary on the nature of softmax output (or sigmoid for Binary classification) of the classifier. There is a great debate on what that is, see Appendix B.2 in [1]. The paper could be even more impactful if these discussions are taken into consideration.\n\n2. Section 3.2: Although some theoretical results are presented that shows correspondence between atypicality and overconfidence in a simple model, it is not clear how these results apply to the other things in the paper. It would be good to draw other insights from it. Do these results provide any insight (or intuition) on why atypical examples would be mis-calibrated? Is this due to estimation errors?\n\nPersonally, I thought about this question myself. And to think of it,  it does make sense intuitively why mis-calibration might be an issue for atypical examples. Consider Binary classification, and assume that one is perfectly able to model $\\eta(x) = \\mathbb{P}(Y=1\\vert X=x)$ where the evaluation of $\\mathbb{P}$ happens with respect to the training data distribution $P$. Now for a sample $x_{new}$ that is not well-represented in $P$, it is not unsurprising to see that $\\mathbb{P}(Y=1 \\vert X=x_{new})$ would be low. However, there is no reason to expect that it would be calibrated. Could authors comment on or augment this intuition?\n\n\n\n\n[1] Tim Pearce et al. Understanding Softmax Confidence and Uncertainty. (2021)\n\nQUESTIONS:\nSome questions follow:\n\n1. (Section 4.2): I'm not sure I follow the intuition behind Equation 2. $\\psi$ is a function of input atypicality. What is the nature of $\\psi$? Is it monotonic, as in it preserves the atypicality of the input. If I assume so, then for a typical input, let's say the atypicality score is 0, in this case $\\hat{\\mathbb{P}}_{\\text{ARR}}(Y\\vert X) = \\frac{\\exp\\{S_Y\\}}{Z(X)}$, which is not intuitive. I'm sure this is my misunderstanding, so would appreciate the clarification. Specifically, the nature of $\\psi$ and $\\phi$. \n\n2. (Line 222): The paper shows that atypicality-aware recalibration improves the overall accuracy in imbalanced supervised classification. In my mind, this is problematic, no? Recalibration techniques should preserve the nature of classifier, is called the sharpness / refinement property of calibration maps. Furthermore, ECE roughly translates to difference in average confidence and average accuracy. And the high ECE in atypicality is the result of lower accuracy but higher confidence. One could improve on ECE by compensating for accuracy instead of the confidence, which in my opinion is not the goal of calibration. I would love to hear from the authors on this.\n\n3. (Section 4.3): I understand that atypicality aware recalibration is useful in fairness methods. Although it is an interesting case study and not the major focus of the paper, could authors comment (and compare) on the notion of multicalibration in the literature. I think atypicality aware recalibration has the same spirit as multicalibration in fairness literature.\n\n4. (Section 5, Line 266): For conformal prediction, the authors write \"group points according to their confidence and atypicality quantiles...\", do the authors use some function to club the confidence score and the atypicality score into a single score? Or some multiple testing procedure is used to accommodate both the score?\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}