Title: Proximity-Informed Calibration for Deep Neural Networks
Venue: NeurIPS 2023
Accepted: yes
Keywords: Calibration

Uncertainty Estimation

Trustworthiness

Fairness

Multicalibration
Abstract: Confidence calibration is central to providing accurate and interpretable uncertainty estimates, especially under safety-critical scenarios. However, we find that existing calibration algorithms often overlook the issue of proximity bias, a phenomenon where models tend to be more overconfident in low proximity data (i.e., data lying in the sparse region of the data distribution) compared to high proximity samples, and thus suffer from inconsistent miscalibration across different proximity samples. We examine the problem over $504$ pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling; 4) Models tend to overfit more heavily on low proximity samples than on high proximity samples. Motivated by the empirical findings, we propose ProCal, a plug-and-play algorithm with a theoretical guarantee to adjust sample confidence based on proximity. To further quantify the effectiveness of calibration algorithms in mitigating proximity bias, we introduce proximity-informed expected calibration error (PIECE) with theoretical analysis. We show that ProCal is effective in addressing proximity bias and improving calibration on balanced, long-tail, and distribution-shift settings under four metrics over various model architectures. We believe our findings on proximity bias will guide the development of fairer and better-calibrated} models, contributing to the broader pursuit of trustworthy AI.

Proximity-Informed Calibration for Deep Neural Networks

Abstract

Confidence calibration is central to providing accurate and interpretable uncertainty estimates, especially under safety-critical scenarios. However, we find that existing calibration algorithms often overlook the issue of proximity bias , a phenomenon where models tend to be more overconfident in low proximity data (i.e., data lying in the sparse region of the data distribution) compared to high proximity samples, and thus suffer from inconsistent miscalibration across different proximity samples. We examine the problem over 504 pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling; 4) Models tend to overfit more heavily on low proximity samples than on high proximity samples. Motivated by the empirical findings, we propose PROCAL, a plug-and-play algorithm with a theoretical guarantee to adjust sample confidence based on proximity. To further quantify the effectiveness of calibration algorithms in mitigating proximity bias, we introduce proximity-informed expected calibration error (PIECE) with theoretical analysis. We show that PROCAL is effective in addressing proximity bias and improving calibration on balanced, long-tail, and distribution-shift settings under four metrics over various model architectures. We believe our findings on proximity bias will guide the development of fairer and better-calibrated models, contributing to the broader pursuit of trustworthy AI.

1 Introduction

Machine learning systems are increasingly deployed in high-stakes applications such as medical diagnosis [30, 36, 9, 6], where incorrect decisions can have severe human health consequences. To ensure safe and reliable deployment, confidence calibration approaches [10, 23, 28] are employed to produce more accurate uncertainty estimates, which allow models to establish trust by communicating their level of uncertainty, and to defer to human decision-making when the models are uncertain.

In this paper, we present a calibration-related phenomenon termed proximity bias , which refers to the tendency of current deep classifiers to exhibit higher levels of overconfidence on samples of low proximity, i.e., samples in sparse areas within the data distribution (see Figure 1 for an illustrative example). In this study, we quantify the proximity of a sample (Eq. 1) using the average distance to its K (e.g. K = 10 ) nearest neighbor samples in the data distribution, and we observe that proximity bias holds for various choices of K . Importantly, the phenomenon persists even after applying existing popular calibration methods, leading to different levels of miscalibration across proximities.

The proximity bias issue raises safety concerns in real-world applications, particularly for underrepresented populations (i.e. low proximity samples) [30, 35]. A recent skin cancer analysis highlights

(a) All Samples:

Well-calibrated

(b)

Samples:

Underconfident

(c)

Samples:

Overconfident

1.0

1.0

1.0

Perfect Calibration

Perfect Calibration

Perfect Calibration

0.8

Output

0.8

0.8

Gap

Gap

Gap

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0.0

0.0

00

0.0

0.2

0.4

0.6

8'0

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

Confidence

Confidence

this concern by revealing that AI-powered models demonstrate high performance for light-skinned individuals but struggle with dark-skinned individuals due to their underrepresentation [11]. This issue can also manifest in the form of proximity bias: suppose a dark-skinned individual has a high risk of 40% of having the cancer. However, due to their underrepresentation within the data distribution, the model overconfidently assigns them 98% confidence of not having cancer. As a result, these low proximity individuals may be deprived of timely intervention.

To study the ubiquity of this problem, we examine 504 ImageNet pretrained models from the timm library [43] and make the following key observations: 1) Proximity bias exists generally across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms including temperature scaling; 4) Low proximity samples are more prone to model overfitting while high proximity samples are less susceptible to this issue.

Besides, we argue that proximity bias is overlooked by confidence calibration . Revisiting its definition, P ( Y = ˆ Y | ˆ P = p ) = p for all p ∈ [0 , 1] , we find that its primary goal is to match confidence with the accuracy of samples sharing the same confidence level. However, Figure 1a reveals that although the model seems well-calibrated within each confidence group, there still exists miscalibration errors among these groups (e.g. low and high proximity samples) due to proximity bias.

Motivated by this, we propose a debiased variant of the expected calibration error (ECE) metric, called proximity-informed expected calibration error (PIECE) to further capture the miscalibration error due to proximity bias. The effectiveness is supported by our theoretical analysis that PIECE is at least as large as ECE and this equality holds when there is no cancellation effect with respect to proximity bias.

To tackle proximity bias and further improve confidence calibration, we propose a plug-and-play method, PROCAL. Intuitively, PROCAL learns a joint distribution of proximity and confidence to adjust probability estimates. To fully leverage the characteristics of the input information, we develop two separate algorithms tailored for continuous and discrete inputs. We evaluate the algorithms on large-scale datasets: balanced datasets including ImageNet [7] and Yahoo-Topics [49], long-tail datasets iNaturalist 2021 [3] and ImageNet-LT [27] and distribution-shift datasets MultiNLI [44] and ImageNet-C [15]. The results show that our algorithm consistently improves the performance of existing algorithms under four metrics with 90% significance (p-value < 0.1).

Our main contributions can be summarized as follows:

Findings : We discover the proximity bias issue and show its prevalence over large-scale analysis ( 504 ImageNet pretrained models).

Metrics : To quantify the effectiveness of mitigating proximity bias, we introduce proximityinformed expected calibration error (PIECE) with theoretical analysis.

Method Effectiveness : We propose a plug-and-play method PROCAL with theoretical guarantee and verify its effectiveness on various image and text settings.

2 Related Work

Confidence Calibration Confidence calibration aims to yield uncertainty estimates via aligning a model's confidence with the accuracy of samples with the same confidence level [10, 25, 28].

To achieve this, Scaling-based methods, such as temperature scaling [10], adjust the predicted probabilities by learning a temperature scalar for all samples. Similarly, parameterized temperature scaling [39] offers improved expressiveness via input-dependent temperature parameterization, and Mix-n-Match [48] adopts ensemble and composition strategies to yield data-efficient and accuracypreserving estimates. Binning-based methods divide samples into multiple bins based on confidence and calibrate each bin. Popular methods include classic histogram binning [46], mutual-informationmaximization-based binning [32], and isotonic regression [47]. However, existing calibration methods overlook the proximity bias issue, which fundamentally limits the methods' capabilities in delivering reliable and interpretable uncertainty estimates.

Multicalibration Multicalibration algorithms [13, 21] aim to achieve a certain level of fairness by ensuring that a predictor is well-calibrated for the overall population as well as different computationally-identifiable subgroups. [33] proposes a grouping loss to evaluate subgroup calibration error while we propose a metric to integrate the group cancellation effect into existing calibration loss. [21] focuses on understanding the fundamental trade-offs between group calibration and other fairness criteria, and [13] proposes a conceptual iterative algorithm to learn a multi-calibrated predictor. In this regard, our proposed framework can be considered a specific implementation of the fairness objectives outlined in [13], with a particular focus on proximity-based subgroups. This approach offers easier interpretation and implementation compared to subgroups discussed in [13, 21].

3 What is Proximity Bias?

In this section, we study the following questions: What is proximity bias? When and why does proximity bias occur?

Background We consider a supervised multi-class classification problem, where input X ∈ X and its label Y ∈ Y = { 1 , 2 , · · · , C } follow a joint distribution π ( X,Y ) . Let f be a classifier with f ( X ) = ( ˆ Y , ˆ P ) , where ˆ Y represents the predicted label, and ˆ P is the model's confidence, i.e. the estimate of the probability of correctness [10]. For simplicity, we use ˆ P to denote both the model's confidence and the confidence calibrated using existing calibration algorithms.

Proximity We define proximity as a function of the average distance between a sample X and its K nearest neighbors N K ( X ) in the data distribution:

where dist( X,X i ) denotes the distance between sample X and its i -th nearest neighbor X i , estimated using Euclidean distance between the features of X and X i from the model's penultimate layer. K is a hyperparameter (we set K = 10 in this paper). We use the validation set as a proxy to estimate the data distribution. That is, we compute any point's proximity by finding its nearest neighbors in the held-out validation set. Although the training set can also be employed to compute proximity, we utilize the validation set because it is readily accessible during the calibration process.

The exponential function is used to normalize the distance measure from a range of [0 , inf] to [0 , 1] , making the approach more robust to the effects of distance scaling since the absolute distance in Euclidean distance can cause instability and difficulty in modeling. This definition allows us to capture the local density of a sample and its relationship to its neighborhood. For instance, a sample situated in a sparse region of the training distribution would receive a low proximity value, while a sample located in a dense region would receive a high proximity value. Samples with low proximity values represent underrepresented samples in the data distribution that merit attention, such as rare ('long-tail') diseases, minority populations, and samples with distribution shift.

Proximity Bias To investigate the relationship between proximity and model miscalibration, we define proximity bias as follows:

Definition 3.1. Given any confidence level p , the model suffers from proximity bias if the following condition does not hold:

̸

The intuition behind this definition is that, ideally, a sample with a confidence level of p should have a probability of being correct equal to p , regardless of proximity. However, if low proximity samples consistently display higher confidence than high proximity samples (as shown in Figure 1), it can lead to unreliable and unjust decision-making, particularly for underrepresented populations.

3.1 Main Empirical Findings

To showcase the ubiquity of this problem, we examine the proximity bias phenomenon on 504 ImageNet pretrained models from the timm library [43] and show the results in Figure 2 (see Appendix D for additional figures and analysis). We use statistical hypothesis testing to investigate the presence of proximity bias. The null hypothesis H 0 is that proximity bias does not exist, formally, for any confidence p and proximities d 1 > d 2 :

To test the above null hypothesis, we first split the samples into 5 equal-sized proximity groups and select the highest and lowest proximity groups. From the high proximity group, we randomly select 10,000 points and find corresponding points in the low proximity group that have similar confidence levels. Next, we reverse this process, randomly selecting 10,000 points from the low proximity group and find corresponding points in the high proximity group with matched confidence. We then merge all the points from the high proximity group into B H and those from the low proximity group into B L , with the B H and B L having similar average confidence. Finally, we apply the Wilcoxon rank-sum test [24] to evaluate whether there is a significant difference in the sample means (i.e. accuracy) of B H and B L . More implementation details can be found in Appendix C.

Inspired by the hypothesis testing, we define Bias Index as the accuracy drop between the confidencematched high proximity group B H and low proximity group B L to reflect the degree of bias:

Note that B H , B L are obtained from the hypothesis testing process and hence have the same mean confidence.

The hypothesis testing results indicate that over 80% of 504 models have a p-value less than 0 . 05 (72% after Bonferroni correction [4]), i.e., the null hypothesis is rejected with a confidence level of at least 95%, indicating that proximity bias plagues most of the models in timm .

We show the bias index of 504 models in Figure 2 and make the following findings:

1. Proximity bias exists generally across a wide variety of model architecture and sizes. Figure 2 shows that most models (80% of the models as supported by hypothesis testing) have a bias index larger than 0, indicating the existence of proximity bias.

2. Transformer-based methods are relatively more susceptible to proximity bias than CNNbased methods. In Figure 2, models with lower accuracy (primarily CNN-based models such as VGG, EfficientNet, MobileNet, and ResNet [12] variants) tend to have lower bias index. On the other hand, among models with higher accuracy, Transformer variants (e.g., DEiT, XCiT [1], CaiT [40], and SwinV2) demonstrate relatively higher bias compared to convolution-based networks (e.g., ResNet variants). This is concerning given the increasing popularity of Transformer-based models in recent years and highlights the need for further research to study and address this issue.

3. Popular calibration methods such as temperature scaling do not noticeably alleviate proximity bias. Figure 2 (upper right) shows that the proximity bias index remains large even after applying temperature scaling, indicating that this method does not noticeably alleviate the problem. In contrast, Figure 2c demonstrates that our proposed approach successfully shifts the models to a much closer distribution around the line y = 0 (indicating no proximity bias). The bias index figures for more existing calibration methods are provided in Appendix D.

4. Low proximity samples are more prone to model overfitting. Figure 4 in Appendix D shows that the model's accuracy difference between the training and validation set is more significant on low proximity samples (31.67%) compared to high proximity samples (0.6%). This indicates that the model generalizes well on samples of high proximity but tends to overfit on samples of low proximity. The overconfidence of low proximity samples can be a consequence of the overfitting tendency, as the overfitting gap also reflects the mismatch between the model's confidence and its actual accuracy.

4 Proximity-Informed ECE

As depicted in Figure 1, existing evaluation metrics underestimate the true miscalibration level , as proximity bias causes certain errors in the model to cancel out. As an example, consider a scenario:

Example 4.1. All samples are only drawn from two proximity groups of equal probability mass, d = 0 . 2 and d = 0 . 8 , with true probabilities of P ( Y = ˆ Y | X,f ) being 0 . 5 and 0 . 9 , respectively. The model outputs the confidence score p = 0 . 7 to all samples.

We consider the most commonly used metric, expected calibration error (ECE) that is defined as ECE = E ˆ P [∣ ∣ ∣ P ( ˆ Y = Y | ˆ P ) -ˆ P ∣ ∣ ∣ ] . In Example 4.1, the ECE is 0, suggesting that the model is perfectly calibrated in terms of confidence calibration. In fact, the model has significant miscalibration issues: it is heavily overconfident in one proximity group while heavily underconfident in the other, highlighting the limitations of existing calibration metrics. The miscalibration errors within the same confidence group are canceled out by samples with both high and low proximity, resulting in a phenomenon we term cancellation effect .

To further evaluate the miscalibration canceled out by proximity bias, we propose the proximityinformed expected calibration error (PIECE). PIECE is defined in an analogous fashion as ECE, yet it further examines information about the proximity of the input sample, D ( X ) , in the calibration evaluation:

Back to Example 4.1 where ECE = 0 , we have PIECE = 0 . 2 , revealing its miscalibration level in the subpopulations of different proximities, i.e., the calibration error regarding proximity bias. Additionally, we demonstrate in Theorem 4.2 that PIECE is always at least as large as ECE, with the equality holding only when there is no cancellation effect w.r.t proximity. The detailed proof is relegated to Appendix B.

Theorem 4.2 (PIECE captures cancellation effect.) . Given any joint distribution π ( X,Y ) and any classifier f that outputs model confidence ˆ P for sample X , we have the following inequality, where equality holds only when there is no cancellation effect with respect to proximity:

5 How to Mitigate Proximity Bias?

In this section, we propose PROCAL to achieve three goals: 1) mitigate proximity bias , i.e., ensure samples with the same confidence level have the same miscalibration gap across all proximity levels, 2) improve confidence calibration by reducing overconfidence and underconfidence, and 3) provide a plug-and-play method that can combine the strengths of existing approaches with our proximity-informed approach.

The high-level intuition is to explicitly incorporate proximity when estimating the underlying probability of the model prediction being correct. In addition, existing calibration algorithms can be classified into 2 types: 1) those producing continuous outputs , exemplified by scaling-based methods [10]; 2) those producing discrete outputs , such as binning-based methods, which group the samples into bins and assign the same scores to samples within the same bin. To fully leverage the distinct properties of the input information, we develop two separate algorithms tailored for continuous and discrete inputs. This differentiation is based on the observation that continuous outputs (e.g., those produced by scaling-based methods) contain rich distributional information suitable for density estimation. On the other hand, discrete inputs (e.g., those generated by binning-based methods) allow for robust binning-based adjustments. By treating these inputs separately, we can effectively harness the characteristics of each type.

In summary, Density-Ratio Calibration (§5.1) estimates continuous density functions, and aligns well with type 1) methods that produce continuous confidence scores. In contrast, Bin-Mean-Shift (§5.2) does not rely on densities, making it more compatible with type 2) calibration methods that yield discrete outputs. Together, these two calibration techniques constitute a versatile plug-and-play framework PROCAL, applicable to confidence scores of both continuous and discrete types.

5.1 Continuous Confidence: Density-Ratio Calibration

The common interpretation of confidence is the likelihood of a model prediction ˆ Y being identical to the ground truth label Y for every sample X . Computing this probability directly with density estimation methods can be computationally demanding, particularly in high-dimensional spaces. To circumvent the curse of dimensionality and address proximity bias, we incorporate the model confidence ˆ P and proximity information D ( X ) to estimate the posterior probability of correctness, i.e., P ( ˆ Y = Y | ˆ P,D ) . This approach is data-efficient since it conducts density estimation in a two-dimensional space only, rather than in the higher dimensional feature or prediction simplex space.

Consider a test sample X with proximity D = D ( X ) and uncalibrated confidence score ˆ P , which can be the standard Maximum Softmax Probability (MSP), or the output of any calibration method. P ( ˆ Y = Y | ˆ P,D ) can be computed via Bayes' rule:

where ˆ Y is the model prediction and Y is the ground truth label. This can be re-expressed as follows by using the law of total probability:

̸

̸

̸

To compute this calibrated score, we need to estimate the distributions P ( ˆ P,D | ˆ Y = Y ) and P ( ˆ P,D | ˆ Y = Y ) , and the class ratio P ( ˆ Y = Y ) P ( ˆ Y = Y ) .

̸

̸

̸

To estimate the probability density functions P ( ˆ P,D | ˆ Y = Y ) and P ( ˆ P,D | ˆ Y = Y ) , various density estimation methods can be used, such as parametric methods like Gaussian mixture models or non-parametric methods like kernel density estimation (KDE)[31]. We choose KDE because it is flexible and robust, making no assumptions about the underlying distribution (see Appendix C for specific implementation details). Specifically, we split samples into two groups based on whether they are correctly classified and then use KDE to estimate the two densities. To obtain the class ratio P ( ˆ Y = Y ) P ( ˆ Y = Y ) , we simply use the ratio of the number of correctly classified samples and the number of misclassified samples in the validation set. The pseudocode for inference and training can be found in Appendix 2 and 1.

5.2 Discrete Confidence: Bin Mean-Shift

The Bin-Mean-Shift approach aims to first use 2-dimensional binning to estimate the joint distribution of proximity D and input confidence ˆ P and then estimate P ( ˆ Y = Y | ˆ P,D ) . Considering test samples X with proximity D = D ( X ) and uncalibrated confidence score ˆ P , we first group samples into 2-dimensional equal-size bins based on their D and ˆ P (other binning schemes can also be used; we choose quantile for simplicity). Next, for each bin B mh , we calculate its accuracy A ( B mh ) and mean confidence F ( B mh ) . Then, the confidence scores of samples within the bin are adjusted as:

where the shrinkage coefficient λ ∈ (0 , 1] is a hyper-parameter, controlling the bias-variance trade-off. Ideally, setting λ = 1 would ideally achieve our goal. However, in practice, we often encounter bins with a smaller number of samples, whose estimate of A ( B mh ) -F ( B mh ) will have high variance and therefore inaccurate. To reduce variance in these scenarios, we can set a smaller λ . In practice, we choose λ = 0 . 5 as a reasonable default for all our experiments, which we find offers consistent performance across various settings.

Note that our approach (i.e. applying a mean-shift in each bin) differs from the typical histogram binning method (replacing the confidence scores with its mean accuracy in each bin). Rather than completely replacing the input confidence scores ˆ P (which are often reasonably well-calibrated), our approach better utilizes these scores by only adjusting them by the minimal mean-shift needed to correct for proximity bias in each of the 2-dimensional bins.

5.3 Theoretical Guarantee

Here we present that our method, Bin-Mean-Shift, can consistently achieve a smaller Brier Score given a sufficient amount of data in the context of binary classification. The Brier Score [5] is a strictly proper score function that measures both calibration and accuracy aspects [22], with a smaller value indicating better performance. As illustrated below, our algorithm's Brier Score is asymptotically bounded by the original Brier Score, augmented by a non-negative term.

Theorem 5.1 (Brier Score after Bin-Mean-Shift is asymptotically bounded by Brier Score before calibration) . Given a joint data distribution π ( X,Y ) and a binary classifier f , for any calibration algorithm h that outputs score h ( ˆ P ) based on model confidence ˆ P , we apply Bin-Mean-Shift to derive calibrated score ˜ h ( ˆ P ) as defined in Equation (5) . Let h c ( ˆ P ) = h ( ˆ P ) × ✶ { ˆ Y = 1 } +(1 -h ( ˆ P )) × ✶ { ˆ Y = 0 } denote the probability assigned to class 1 by h ( ˆ P ) , and define ˜ h c ( ˆ P ) similarly. Then, the Brier Score before calibration can be decomposed as follows:

where P ( B ) is determined by the binning mechanism used in Bin-Mean-Shift.

Remark. Note that when the calibration algorithm h is an identity mapping, it demonstrates that Bin-Mean-Shift achieves better model calibration performance than the original model confidence, given a sufficient amount of data. The detailed proof is relegated to Appendix A.

6 Experiments

In this section, we aim to answer the following questions:

Performance across different datasets and model architectures: How does PROCAL perform on datasets with balanced distribution, long-tail distribution, and distribution shift, as well as on different model architectures?

Inference efficiency: How efficient is our PROCAL? (see Appendix E.1)

Hyperparameter sensitivity : How sensitive is PROCAL to different hyperparameters, e.g. neighbor size K ? (See Appendix F.2)

Ablation study: What is the difference between Density-Ratio and Bin-Mean-shift on calibration? How should the choice between these techniques be determined? (See Appendix F.1)

6.1 Experiment Setup

Evaluation Metrics. Following [10], we adopt 3 commonly used metrics to evaluate the confidence calibration : Expected Calibration Error ( ECE ), Adaptive Calibration Error ( ACE [29]), Maximum Calibration Error ( MCE ) and our proposed PIECE to evaluate the bias mitigation performance. More detailed introduction of these metrics can be found in Appendix C.

Datasets. We evaluate the effectiveness of our approach across large-scale datasets of three types of data characteristics (balanced, long-tail and distribution-shifted) in image and text domains: (1) Dataset with balanced class distribution (i.e. each class has an equal size of samples) on vision dataset ImageNet [7] and two text datasets including Yahoo Answers Topics [49] and MultiNLI-match [44]; (2) Datasets with long-tail class distribution on two image datasets, including iNaturalist 2021 [3] and ImageNet-LT [27]; (3) Dataset with distribution-shift on three datasets, including ImageNet-C [15], MultiNLI-Mismatch [44] and ImageNet-Sketch [42].

Comparison methods. We compare our method to existing calibration algorithms: base confidence score ( Conf ) [16], scaling-based methods such as Temperature Scaling ( TS ) [10], Ensemble Temperature Scaling ( ETS ) [48], Parameterized Temperature Scaling ( PTS ) [39], Parameterized Temperature Scaling with K Nearest Neighbors ( PTSK ), and binning based methods such as Histogram Binning ( HB ), Isotonic Regression ( IR ) and Multi-Isotonic Regression ( MIR ) [48]. Throughout the experiment section, we apply Density-Ratio Calibration to Conf, TS, ETS, PTS, and PTSK and apply Bin-MeanShift to binning-based methods IR, HB, and MIR. HB and IR are removed from the long-tail setting due to its instability when the class sample size is very small.

More details on baseline algorithms, datasets, pretrained models, hyperparameters, and implementation details can be found in Appendix C.

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Validation Accuracy

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

Metric Value

ECE

Conf

TS

PTSK

ETS

MIR

Ours

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Validation Accuracy

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

Metric Value

MCE

Conf

TS

PTSK

ETS

MIR

Ours

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Validation Accuracy

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

Metric Value

ACE

Conf

TS

PTSK

ETS

MIR

Ours

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Validation Accuracy

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

Metric Value

PIECE

Conf

TS

PTSK

ETS

MIR

Ours

6.2 Effectiveness

Datasets with the balanced class distribution. The results on ImageNet of 504 models from timm [43] are depicted in Figure 3, where our method (red color markers) consistently appears at the bottom, achieving the lowest calibration error across all four evaluation metrics in general. This indicates that our method consistently outperforms other approaches in eliminating proximity bias and improving confidence calibration. We also select four popular models from these 504 models, specifically BeiT [2], MLP Mixer [38], ResNet50 [12] and ViT [8]. A summary of their results is presented in Table 4 of Appendix E. Additionally, Table 2a and Table 5 present the calibration results for the text classification task on Yahoo Answers Topics and the text understanding task on MultiNLI, where our method consistently improves the calibration of existing methods and model confidence. SeeAppendix E for more details.

Datasets with the long-tail class distribution. Table 1 shows the results on the long-tail image dataset iNaturalist 2021. Our method ('ours') consistently improves upon existing algorithms ('base') regarding reducing confidence calibration errors ( ECE , ACE , and MCE ) and mitigating proximity bias ( PIECE ). Note that even when used independently ('Conf+ours') without combining with existing algorithms, our method achieves the best performance across all metrics. This result suggests that our algorithm can make the model more calibrated in the long-tail setting by effectively mitigating the bias towards low proximity samples (i.e. tail classes), highlighting its practicality in real-world scenarios where data is often imbalanced and long-tailed. ImageNet-LT results in Table 6 of Appendix E.3 show similar improvement.

Datasets with distribution shift. Table 2b shows our method's calibration performance when trained on an in-distribution validation set (MultiNLI Match) and applied to a cross-domain test set (MultiNLI Mismatch). The results suggest that our method can improve upon most existing methods on ECE, ACE and MCE, and gain consistent improvement on PIECE, indicating its effectiveness in mitigating proximity bias. Moreover, empirical results on ImageNet-C (Figure 12) and ImageNetSketch (Table 7) also demonstrate consistent improvements of our method over baselines. Besides, compared to Bin-Mean-Shift, Density-Ratio exhibits more stable performance on enhancing the existing baselines. More analysis on their comparison can be found in Appendix F.1.

7 Conclusions and Discussion

In this paper, we focus on the problem of proximity bias in model calibration, a phenomenon wherein deep models tend to be more overconfident on data of low proximity (i.e. lying in the sparse region of data distribution) and thus suffer from miscalibration. We study this phenomenon on 504 public models across a wide variety of model architectures and sizes on ImageNet and find that the bias persists even after applying the existing calibration methods, which drives us to propose PROCAL for tackling proximity bias. To further evaluate the miscalibration due to proximity bias, we propose a proximity-informed expected calibration error (PIECE) with theoretical analysis. Extensive empirical studies on balanced, long-tail, and distribution-shifted datasets under four metrics support our findings and showcase the effectiveness of our method. 2

Potential Impact, Limitations and Future Work We uncover the proximity bias phenomenon and show its prevalence through large-scale analysis, highlighting its negative impact on the safe deployment of deep models, e.g. unfair decisions on minority populations and false diagnoses for underrepresented patients. We also provide PROCAL as a starting point to mitigate the proximity bias, which we believe has the potential to inspire more subsequent works, serve as a useful guidance in the literature and ultimately lead to improved and fairer decision-making in real-world applications , especially for underrepresented populations and safety-critical scenarios. However, our study also has several limitations. First, our PROCAL maintains a held-out validation set during inference for computing proximity. While we have shown that the cost can be marginal for large models (see Inference Efficiency in Appendix E.1), it may be challenging if applied to small devices where memory is limited. Future research can investigate the underlying mechanisms of proximity bias and explore various options to replace the existing approach of local density estimation. Additionally, we only focus on the closed-set multi-class classification problem; future work can generalize this to multi-label, open-set or generative settings.

Acknowledgments

The author extends gratitude to Yao Shu and Zhongxiang Dai for the insightful discussions during the review response period, and to Yifei Li for providing constructive feedback on the manuscript draft. This research is supported by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: [AISG2-TC-2021-002]).

References

Alaaeldin Ali, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, et al. Xcit: Crosscovariance image transformers. Advances in neural information processing systems , 34:2001420027, 2021.

Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers, 2022.

Sara Beery, Arushi Agarwal, Elijah Cole, and Vighnesh Birodkar. The iwildcam 2021 competition dataset. arXiv preprint arXiv:2105.03494 , 2021.

Carlo Bonferroni. Teoria statistica delle classi e calcolo delle probabilita. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commericiali di Firenze , 8:3-62, 1936.

Glenn W Brier et al. Verification of forecasts expressed in terms of probability. Monthly weather review , 78(1):1-3, 1950.

Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining , pages 1721-1730, 2015.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Omar Elfanagely, Yoshiko Toyoda, Sammy Othman, Joseph A Mellia, Marten Basta, Tony Liu, Konrad Kording, Lyle Ungar, and John P Fischer. Machine learning and surgical outcomes prediction: a systematic review. Journal of Surgical Research , 264:346-361, 2021.

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning , pages 1321-1330. PMLR, 2017.

Lisa N Guo, Michelle S Lee, Bina Kassamali, Carol Mita, and Vinod E Nambudiri. Bias in, bias out: underreporting and underrepresentation of diverse skin types in machine learning research for skin cancer detection-a scoping review. Journal of the American Academy of Dermatology , 87(1):157-159, 2022.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR , abs/1512.03385, 2015. URL http://arxiv.org/abs/1512.03385 .

Ursula Hebert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the (Computationally-identifiable) masses. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 1939-1948. PMLR, 10-15 Jul 2018.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 , 2019.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 , 2019.

Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 , 2016.

Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust a classifier. Advances in neural information processing systems , 31, 2018.

Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data , 7(3):535-547, 2019.

Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data , 7(3):535-547, 2019.

Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv:1910.09217 , 2019.

Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. 67, 09 2016.

Volodymyr Kuleshov and Shachi Deshpande. Calibrated and sharp uncertainties in deep learning via density estimation. In International Conference on Machine Learning , pages 11683-11693. PMLR, 2022.

Meelis Kull, Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

FC Lam and MT Longnecker. A modified wilcoxon rank sum test for paired data. Biometrika , 70(2):510-513, 1983.

Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Taking a step back with kcal: Multi-class kernel-based calibration for deep neural networks. arXiv preprint arXiv:2202.07679 , 2022.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019.

Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Largescale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2537-2546, 2019.

Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In Advances in Neural Information Processing Systems , volume 34, pages 15682-15694, 2021.

Jeremy Nixon, Michael W Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In CVPR Workshops , volume 2, 2019.

Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used to manage the health of populations. Science , 366(6464):447-453, 2019.

Emanuel Parzen. On estimation of a probability density function and mode. The annals of mathematical statistics , 33(3):1065-1076, 1962.

Kanil Patel, William Beluch, Bin Yang, Michael Pfeiffer, and Dan Zhang. Multi-class uncertainty calibration via mutual information maximization-based binning. arXiv preprint arXiv:2006.13092 , 2020.

Alexandre Perez-Lebel, Marine Le Morvan, and Gaël Varoquaux. Beyond calibration: estimating the grouping loss of modern neural networks. In ICLR , 2023.

MRafiee and M Abbasi. Pruned kd-tree: a memory-efficient algorithm for multi-field packet classification. SN Applied Sciences , 1(12):1537, 2019.

Alvin Rajkomar, Michaela Hardt, Michael D Howell, Greg Corrado, and Marshall H Chin. Ensuring fairness in machine learning to advance health equity. Annals of internal medicine , 169(12):866-872, 2018.

Mark Sendak, Madeleine Clare Elish, Michael Gao, Joseph Futoma, William Ratliff, Marshall Nichols, Armando Bedoya, Suresh Balu, and Cara O'Brien. The human body is a black box supporting clinical decision-making with deep learning. In Proceedings of the 2020 conference on fairness, accountability, and transparency , pages 99-109, 2020.

Jonathan Taylor. statsmodels: Statistical modeling and econometrics in Python, 2009-. URL https://www.statsmodels.org .

Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mlp-mixer: An all-mlp architecture for vision. arXiv preprint arXiv:2105.01601 , 2021.

Christian Tomani, Daniel Cremers, and Florian Buettner. Parameterized temperature scaling for boosting the expressive power in post-hoc uncertainty calibration. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XIII , pages 555-569. Springer, 2022.

Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers amp; distillation through attention. In International Conference on Machine Learning , volume 139, pages 10347-10357, July 2021.

Grant Van Horn, Elijah Cole, Sara Beery, Kimberly Wilber, Serge Belongie, and Oisin Mac Aodha. Benchmarking representation learning for natural world image collections. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12884-12893, 2021.

Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems , pages 10506-10518, 2019.

Ross Wightman. Pytorch image models. https://github.com/rwightman/ pytorch-image-models , 2019.

Adina Williams, Nikita Nangia, and Samuel R Bowman. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426 , 2017.

Miao Xiong, Shen Li, Wenjie Feng, Ailin Deng, Jihai Zhang, and Bryan Hooi. Birds of a feather trust together: Knowing when to trust a classifier via adaptive neighborhood aggregation. arXiv preprint arXiv:2211.16466 , 2022.

Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Icml , volume 1, pages 609-616, 2001.

Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining , pages 694-699, 2002.

Jize Zhang, Bhavya Kailkhura, and T Yong-Jin Han. Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning. In International conference on machine learning , pages 11117-11128. PMLR, 2020.

Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems , 28, 2015.

A Proof of Theorem 5.1

Notation. Let π ( X,Y ) denote the true underlying distribution of the sample X and its label Y . The empirical dataset used for Bin-Mean-Shift is denoted by D n = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , . . . , ( x n , y n ) } , where n is the number of samples in the dataset, and each datapoint ( x i , y i ) is independently and identically sampled from π ( X,Y ) .

A classifier f is defined as f ( X ) = ( ˆ Y , ˆ P ) , where ˆ Y represents the predicted label and ˆ P represents the model's confidence, i.e., the estimate of the probability of correctness [10].

We use h ( ˆ P ) to denote the output confidence score for class prediction ˆ Y calibrated by any calibration algorithm h . For example, in the simplest case, h can be the identity mapping, in which case h ( ˆ P ) represents the original model confidence.

To simplify notation, we use B = B ( X ) to denote the bucket to which X belongs. Given a bucket B , we compute the empirical accuracy ˆ A n ( B ) and mean confidence score ˆ F n ( B ) for samples in the bucket, based on the empirical dataset D n . Given the classifier f and calibrator h , they are computed as follows:

Additionally, we use A ( B ) and F ( B ) to denote the expected confidence and actual accuracy of samples from the underlying data distribution π ( X,Y ) and belonging to the bucket B :

With these definitions, our proposed Bin-Mean-Shift algorithm can be expressed as:

where B is the bucket that the input sample X belongs to, and ˜ h ( ˆ P ) is the score calibrated using our Bin-Mean-Shift algorithm.

First we revisit the definition of Brier Score. The Brier Score [5] is a strictly proper score function that measures both calibration and accuracy aspects [22], with a smaller value indicating better performance. The Brier Score is defined as mean square loss as follows for binary classification:

where h c ( ˆ P ) is the probability assigned to class 1 by the calibration algorithm h :

In short, this can be represented as h c ( ˆ P ) = h ( ˆ P ) × { ˆ Y = 1 } +(1 -h ( ˆ P )) × { ˆ Y = 0 } .

✶ ✶ Theorem A.1 (Brier Score after Bin-Mean-Shift is asymptotically bounded by Brier Score before calibration) . Given a joint data distribution π ( X,Y ) and a binary classifier f , for any calibration algorithm h that outputs a score h ( ˆ P ) based on model confidence ˆ P , we apply Bin-Mean-Shift to derive the calibrated score ˜ h ( ˆ P ) as defined in Equation (10) . Let h c ( ˆ P ) = h ( ˆ P ) × ✶ { ˆ Y = 1 } +(1 -h ( ˆ P )) × ✶ { ˆ Y = 0 } denote the probability assigned to class 1 by h ( ˆ P ) , and define ˜ h c ( ˆ P ) similarly. Then, the Brier Score before calibration can be decomposed as follows:

where P ( B ) is determined by the binning mechanism used in Bin-Mean-Shift.

Proof. First we prove the following equality which re-expresses the Brier score in an equivalent form:

where ˜ Y = ✶ { Y = ˆ Y } is a binary variable representing whether the model's prediction is correct:

Then we consider h c ( ˆ P ) -Y . By Eq. (12):

So we have ( h c ( ˆ P ) -Y ) 2 = ( h ( ˆ P ) -˜ Y ) 2 which completes the proof of the equality in Eq. (13). Using this, we rewrite the Brier score as follows:

Second, we decompose the Brier score:

First note that term ( a ) is the Brier score after calibration (recalling our earlier equivalent form for the Brier score). For term ( b ) we recall that ˜ h ( ˆ P ) = h ( ˆ P ) + ˆ A n ( B ) -ˆ F n ( B ) , which is our proposed Bin-Mean-Shift algorithm in Equation (10). Note that X can be sampled by first sampling the bin B and then sampling the point X from the corresponding bin. So term ( b ) can be expressed as:

where ˆ A n ( B ( X )) and ˆ F n ( B ( X )) remain the same for all samples following into the same bucket B .

For term ( c ) we show that:

where in the last step ( ˆ A n ( B ) -ˆ F n ( B )) is a function of B and thus can be moved out of the inner expectation. For term ( d ) we have:

For term ( e ) we have:

E ( X,Y ) ∼ P ( X,Y | B ) [ ˜ Y ] = E ( X,Y ) ∼ P ( X,Y | B ) [ ✶ { Y = ˆ Y } ] = A ( B ) . (30) By the Law of Large Numbers, the sample means converges to their expectations as follows:

This further leads to the following statement for term ( c ) :

Then finally we have the statement:

Discussion This theorem shows that Bin-Mean-Shift (BMS) preserves the calibration properties of the input scores ˆ P ; i.e. if h ( ˆ P ) was already well-calibrated, the BMS-calibrated scores ˜ h ( ˆ P ) will continue to be well-calibrated (up to an o (1) term). At the same time, BMS helps to correct for miscalibrations with respect to any choice of buckets (represented by the ( ˆ A n ( B ) -ˆ F n ( B )) 2 term).

Interestingly, this theorem implies that we can have theoretical guarantees for a pipeline of different calibration methods: for example, consider a pipeline consisting of any calibration method h ( ˆ P ) , followed by one or more applications of BMS with different choices of binning schemes. Then this theorem shows that the Brier score will decrease with each application of BMS (setting aside the o (1) term). Thus, in contrast to the theoretical guarantees of existing calibration methods (such as histogram binning), which only apply to a single calibration method, our approach points to the theoretical benefits of such pipelines of calibration methods.

B Proof of PIECE Guarantee

Recall that f is a classifier (e.g. neural network) with f ( X ) = ( ˆ Y , ˆ P ) , where ˆ Y represents the predicted label, and ˆ P is the model's confidence, i.e. the estimate of the probability of correctness [10]. Theorem B.1 (PIECE captures cancellation effect.) . Given any joint distribution π ( X,Y ) and any classifier f that outputs model confidence ˆ P for sample X , we have the following inequality, where equality holds only when there is no cancellation effect with respect to proximity:

Proof. Note that ECE [10] is defined as:

We first consider the formula within the expectation:

where ( a ) is derived using the Law of Total Expectation and ( b ) is derived using Jensen's Inequality.

The equality holds if and only if ∣ ∣ ∣ P ( ˆ Y = Y | ˆ P,D ) -ˆ P ∣ ∣ ∣ is a linear function in terms of D . This condition is satisfied only when there is no cancellation effect with respect to proximity D , i.e. either P ( ˆ Y = Y | ˆ P,D ) ≥ ˆ P or P ( ˆ Y = Y | ˆ P,D ) ≤ ˆ P hold for all choices of D .

Then applying this formula to Equation 33, we have:

C Experimental Setup

Evaluation Metrics. Following [10, 29], we adopt three commonly used metrics to evaluate the confidence calibration : Expected Calibration Error ( ECE ), Adaptive Calibration Error ( ACE [29]), Maximum Calibration Error ( MCE ) and our proposed PIECE to evaluate the bias mitigation performance. To compute these metrics, we first divide samples into M = 15 bins and compute every bin's average confidence and accuracy. Then we compute the absolute difference between each bin's average confidence and its corresponding accuracy. The final calibration error is measured using the weighted difference (the fraction of samples in each bin as the weight). The key distinction between ECE and ACE lies in the binning scheme: ECE divides bins with equal-confidence intervals while ACE uses an adaptive scheme that spaces the bin intervals to contain an equal number of samples in each bin. In addition, PIECE splits the bin both based on confidence and proximity. It splits samples into M = 15 equal-number bins based confidence and then split every confidence group into H = 10 equal-number bins based on proximity. MCE chooses the bin with the largest difference between average confidence and accuracy and output their absolute difference as the final error.

Datasets. We evaluate the effectiveness of our approach across large-scale datasets of three types of data characteristics (balanced, long-tail and distribution-shifted) in image and text domains: (1) Dataset with balanced class distribution (i.e. each class has an equal size of samples) on vision dataset ImageNet [7] and two text datasets including Yahoo Answers Topics [49] and MultiNLI-match [44]; (2) Datasets with long-tail class distribution on two image datasets, including iNaturalist 2021 [3] and ImageNet-LT [27]; (3) Dataset with distribution-shift on three datasets, including ImageNet-C [15], MultiNLI-Mismatch [44] and ImageNet-Sketch [42].

Comparison methods. We compare our method to existing calibration algorithms: base confidence score ( Conf ) [16], scaling-based methods such as Temperature Scaling ( TS ) [10], Ensemble Temperature Scaling ( ETS ) [48], Parameterized Temperature Scaling ( PTS ) [39], Parameterized Temperature Scaling with K Nearest Neighbors ( PTSK ), and binning based methods such as Histogram Binning ( HB ), Isotonic Regression ( IR ) and Multi-Isotonic Regression ( MIR ) [48]. Throughout the experiment section, we apply Density-Ratio Calibration to Conf, TS, ETS, PTS, and PTSK and apply Bin-MeanShift to binning-based methods IR, HB, and MIR. HB and IR are removed from the long-tail setting due to its instability when the class sample size is very small.

Pretrained Models. For the proximity bias analysis in section 3 and the balanced ImageNet evaluation, we use 504 pretrained models from timm [43] (the list of models are shown in the code repository). For ImageNet-LT evaluation, we use the model ResNext50 pretrained by Liu et al. [27] using classifier re-training technique. For iNaturalist 2021, we directly use the ImageNet pretrained backbone ResNet [12] which we follow this paper [41] and download from the repo 3 .. For Yahoo Answers Topics and MultiNLI datasets, we use pretrained RoBERTa from HuggingFace API and fine-tuned on the corresponding datasets with 3 epochs.

Hyperparamerters. Regarding nearest neighbor computation, we use indexFlatL2 from faiss [18]. Except the Hyperparamerter sensitivity experiments, we use K = 10 for the proximity computation. Regarding our method, for Density-Ratio, the kernel density estimation for two variables are implemented using statsmodel library [37]. For the Bin Mean-Shift method, we set the regularization parameter λ = 0 . 5 . For the calibration setup, we adopt a standard calibration setup [10] with a fixed-size calibration set (i.e. validation set) and evaluation test datasets. Specifically, we randomly split the hold-out dataset into calibration set and evaluation set n c = n e = 25000 for ImageNet, n c = n e = 50000 for iNaturalist 2021 and n c = n e = 5000 for ImageNet-LT. For Yahoo and MultiNLI-Match dataset, we sample 20% data from the training dataset as calibration set and use the original test dataset as the test dataset. For the evaluation, we use random seed 2020, 2021, 2022, 2023, 2024 and compute the mean (this does not apply to NLP dataset for its fixed test set).

Details on Statistical Hypothesis Testing. The statistical hypothesis testing requires that for two samples with the same confidence but different proximity, their probability of being correct is the same. To achieve this, we aim for every pair of high and low proximity samples to have the same confidence levels. However, there is indeed an observable difference in the average confidence of B H and B L , with high proximity samples B H having higher average confidence. This leads to certain samples that do not have a counterpart with equivalent confidence in the other proximity group. To address this issue, we employ a nearest neighbor search combined with a rejection policy. Specifically, pairs with a confidence difference greater than 0.05 are discarded. This ensures that we are only pairing samples from different proximity groups that have the closest possible confidence levels to one another. When implementing this algorithm, users can also visualize the confidence distribution to verify whether the confidence of two proximity groups have evident overlapping. If their confidence levels have no overlap, we suggest reducing the number of splits from 5 to 3 to ensure low/high proximity groups have similar confidence but different proximity.

Details on KDE For its simplicity and effectiveness, we use the KDEMultivariate function from the statsmodel library for density estimation. This function employs a Gaussian Kernel and applies the normal reference rule of thumb (i.e. bw= 1 . 06ˆ σn -1 5 ) based on the the standard deviation ˆ σ and sample size n to select an appropriate bandwidth. While it is possible to use other density estimation kernels such as Exponential Kernel in Scikit Learn , we found that the Gaussian kernel coupled with

1.0

0.8

0.6

0.4

0.2

train

val

0.0

0

123

456

7

8

9

Proximity

the normal reference rule for bandwidth selection generally yields better performance across various models and datasets.

D Additional Empirical Findings

D.1 Low proximity samples are more prone to model overfitting

To study the behavior of low proximity samples and high proximity samples, we compute their accuracy difference between training dataset and validation set. Figure 4 reveals a clear tendency: the model's accuracy difference between the training and validation set is more significant on low proximity samples (31.67%) compared to high proximity samples (0.6%). This indicates that the model generalizes well on samples of high proximity but tends to overfit on samples of low proximity. The overconfidence of low proximity samples can be a consequence of the overfitting tendency, as the overfitting gap also reflects the mismatch between the model's confidence and its actual accuracy.

D.2 Proximity Bias Across A Variety of Models

Here we present the bias index of several popular calibration algorithms on 504 models on ImageNet. As depicted in Figure 5, most models have a bias index larger than 0, indicating the existence of proximity bias across a variety of models. Notably, even after applying temperature scaling, as demonstrated in Figure 6, the calibrated model confidences still exhibit proximity bias. This tendency is further corroborated by Figure 7 (after multi isotonic regression calibration) and Figure 8 (after ensemble temperature scaling). In stark contrast, Figure 9 reveals the effectiveness of our proposed approach in mitigating proximity bias.

D.3 Confidence and Accuracy Are Positively Correlated With Proximity

Our initial investigation delves into the relationship between sample proximity, confidence, and accuracy across a variety of deep neural network models. We observe a clear trend wherein the

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Model Acc

0.2

0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Model Confidence

well-calibrated

Various

DEiT

XCiT

CaiT

SwinV2

ViT

MLP Mixer

EfficientNet

MobileNet

VGG

ResMLP

ResNet

ResNext

RegNet

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Model Acc

0.2

0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Model Confidence + Tempearture Scaling Calibration

well-calibrated

Various

DEiT

XCiT

CaiT

SwinV2

ViT

MLP Mixer

EfficientNet

MobileNet

VGG

ResNext

ResMLP

ResNet

RegNet

model's confidence and accuracy exhibit an upward trajectory from low proximity samples to high proximity samples. This trend is illustrated in Figure 10.

This tendency suggests that the model is less confident with samples from low proximity regions, where training samples are sparse. From the perspective of distribution, as samples move towards sparse regions, they are stepping out of the main mass of the training distribution and are considered as out-of-distribution (OoD) samples. This aligns with the general expectation that out-of-distribution data points should have high uncertainties and thus, low confidence estimates.

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Model Acc

0.2

0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Bias Index of Multi Isotonic Regression

well-calibrated

Various

DEiT

XCiT

CaiT

SwinV2

ViT

MLP Mixer

EfficientNet

MobileNet

VGG

ResMLP

ResNet

ResNext

RegNet

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Model Acc

0.2

0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Bias Index of Ensemble Temperature Scaling

well-calibrated

Various

DEiT

XCiT

CaiT

SwinV2

ViT

MLP Mixer

EfficientNet

MobileNet

VGG

ResMLP

ResNet

ResNext

RegNet

However, Figure 10 also shows that the slopes of the accuracy and confidence change are not the same, resulting in a larger miscalibration gap between low proximity and high proximity samples. Furthermore, commonly used calibration techniques such as temperature scaling and histogram binning seems not alleviate this issue. This motivated us to study how proximity information relates to calibration.

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Model Acc

0.2

0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Model Confidence + Our Calibration

well-calibrated

Various

DEiT

XCiT

CaiT

SwinV2

ViT

MLP Mixer

EfficientNet

MobileNet

VGG

ResNext

ResMLP

ResNet

RegNet

0.95

conf

temperature_scaling

acc

beit_large_patch16_224

1.0 -

0.9

mixer_b16_224

1.00

0.95

Conf

temperature_scaling

acc

vit_large_patch16_224

1.00

0.95

conf

acc

temperature_scaling

resnet50

0.90

0.85

0.70

0.65

0.80

0.75

0.8

0.7

0.6

0.5

0.4

temperature_scaling

acc

conf

0.80

0.90

0.85

0.75

0.70

0.65

0.90

0.85

0.75

0.70

0.80

0.65

0.60

0.4

0.5

proximity

0.6

0.7

0.8

0.4

0.5

preximity

0.6

0.7

0.35

0.40

0.45

proximity

0.50

0.55

0.60

0.65

0.40

0.45

0.50

0.55

preximity

0.60

0.65

0.70

0.75

E Additional Experimental Results

E.1 Inference Efficiency

Runtime Efficiency To verify the time efficiency of our method, we compare the inference time with baseline methods. The result is reported in Table 3. Compared to the confidence baseline, our method, Bin-Mean-Shift, exhibits a slight increase of 1.17% in runtime, while Density-Ratio introduces a modest overhead of 12.3%. These results demonstrate that our method incurs minimal computational overhead while achieving comparable runtime efficiency to the other baseline methods. In addition, it is worth noting that the cost of computing proximity has been reduced due to the recent advancement in neighborhood search algorithms. In our implementation, we employ indexFlatL2 from the Meta open-sourced GPU-accelerated Faiss library [18] to calculate each sample's nearest neighbor. This algorithm enables us to reduce the time for nearest neighbor search to approximately 0.04 ms per sample (shown in Table 3). The computation overhead beyond the neighbor search is actually quite similar to isotonic regression (IR) and histogram binning (HB), which leads to the total time being roughly twice that of isotonic regression ( 0 . 04 + 0 . 05 ≈ 0 . 1 s ).

Memory Efficiency Similar to K-nearest-neighbor-based methods [17, 45], our method requires maintaining a held-out neighbor set for proximity computation during inference. To achieve memory efficiency, we employ these three techniques: 1) Reduce the size of the held-out neighbor set since it is unnecessary to utilize the entire raw neighbor-set, such as the entire training dataset, particularly when dealing with large-scale data in practical scenarios. For instance, our experimental results on ImageNet [7] are obtained using a neighbor-set comprising 25,000 randomly sampled images

from the validation set. 2) Leverage pre-computed feature embeddings instead of full images, such as ResNext101 embeddings ( d = 1024 ), which consumed a mere 229MB in our experiments. 3) Leverage memory-efficient neighbor search algorithms to further enhance memory efficiency [34, 19].

E.2 Effectiveness on Datasets with Balanced Class Distribution.

First, we present high-resolution figures of Figure 3 illustrating the results of 504 models from timm [43] on ImageNet. In Figure 11, our method (indicated by red color markers) consistently demonstrates the lowest calibration error across all four evaluation metrics, maintaining a consistently superior performance.

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Validation Accuracy

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

Metric Value

ECE

Conf

TS

PTSK

ETS

MIR

ours

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Validation Accuracy

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

Metric Value

ACE

Conf

TS

PTSK

ETS

MIR

ours

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Validation Accuracy

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

Metric Value

MCE

Conf

TS

PTSK

ETS

MIR

ours

0.60

0.65

0.70

0.75

0.80

0.85

0.90

Validation Accuracy

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

Metric Value

PIECE

Conf

TS

PTSK

ETS

MIR

ours

Second, we select four widely used ImageNet pre-trained models from the pool of 504 models depicted in Figure 11. We compare the performance of our proposed approach with baseline calibration algorithms, namely BeiT , MLP Mixer , ResNet50 and ViT . The detailed results are presented in Table 4, showcasing the effectiveness of our methods in mitigating proximity bias and enhancing calibration performance compared to existing calibration techniques. Additionally, even when applied to the most successful base calibration algorithm, our method achieves a notable reduction in calibration errors. This consistent improvement is particularly remarkable, especially in scenarios where the original calibration methods fail to enhance or even worsen performance.

Third, we present the outcomes of our approach on the Natural Language Understanding task, specifically on the MultiNLI Match dataset, as displayed in Table 5. The results demonstrate that our method can improve confidence calibration performance in balanced datasets and achieve comparable performance in addressing proximity bias.

E.3 Effectiveness on Datasets with the Long-tail Class Distribution.

To evaluate our method in large-scale long-tail datasets, we conduct experiments on long-tail datasets ImageNet-LT , and iNaturalist 2021 . Table 6 shows our method's performance on ImageNet-LT, showing that our algorithm improves upon the original calibration algorithms in most cases under all four evaluation metrics, particularly on ECE, ACE, and PIECE. This suggests that our algorithm

can effectively mitigate the bias towards low proximity samples (i.e. tail classes), highlighting its practicality in real-world scenarios where data is often imbalanced.

E.4 Effectiveness on Datasets with Distribution Shift

To further evaluate the effectiveness of our proposed method in handling distribution shifts, we conduct experiments on the ImageNet Corruption dataset [14]. The base models are trained on the ImageNet dataset without any distribution shifts and are subsequently tested on the corrupted dataset. The results, shown in Figure 12, indicate that our method is able to consistently improve the performance of the original model even in the presence of distribution shifts. This is particularly notable in the case of data with distribution shifts that tend to fall in the low proximity region, where current algorithms tend to be poorly calibrated, but our method effectively addresses this problem.

OurframeworkcanfurtherreducePlECEwhenapplied toexistingcalibrationalgorithms.

0.08

TS

PTS

PTSK

Error

0.06

ETS

Conf

Reduced

0.04

IR

HB

MIR

0.02

0.00

Brightness

Elastic Transform

Frost

Pixelate

Snow

To explore the issue of domain shift, where test data has a shifted distribution not seen in training, we conduct experiments using ImageNet (training set) and ImageNet-Sketch (test set). The datasets are chosen because all ImageNet images are real-world photos, while all ImageNet-Sketch images are sketches, collected using Google Search with class label keywords and "sketch", similar to the case of skin color example provided in the introduction. We employ a Vision Transformer backbone from TIMM, trained on the ImageNet. Then we train our ProCal using the validation set from ImageNet and tested it on the 50,000 images from ImageNet-Sketch. The result shown in Table 7 shows that ProCal effectively improves upon existing algorithm in many cases. While we observe a slight increase in ECE, ACE, and MCE when ProCal is paired with with PTS and PTSK, this is probably attributed to the original methods suffering from the cancellation effect , where positive and negative calibration errors within the same confidence bin cancel out each other (see section 6.1). Under the PIECE metric that captures the cancellation effect, our method consistently outperforms all methods by large margins and effectively mitigates their proximity bias.

F Ablation Study

F.1 Comparison between Density-Ratio and Bin-Mean-shift.

Our comparison of Density-Ratio and Bin-Mean-shift reveals that Density-Ratio performs better in scaling-based methods while Bin-Mean-Shift demonstrates general robustness and adaptability in both continuous and discrete settings. This can be attributed to the fact that Density-Ratio can be thought of as an infinite binning-based method, enjoying good expressiveness, but it relies on density estimation which may not be as accurate when dealing with discrete outputs. In contrast, Bin-Mean-shift does not make any assumptions about the output. Therefore, it is important to note that both techniques have their own strengths and weaknesses, and the choice of which one to use should be based on the specific task at hand.

F.2 Hyperparameter Sensitivity

In this section, we evaluate the sensitivity of our method to various hyperparameter choices. Specifically, we examine the impact of the choice of distance metric and the number of neighbors in the local neighborhood. To evaluate the performance of our method under different hyperparameter settings, we use ResNet50 [12] as our base model and compare its calibration performance of the integration of our method and existing popular claibration algorithms under different hyperparameters. This study aims to provide a guideline for selecting appropriate hyperparameters when using our method.

Effect of Neighbor Size K In this study, we examine the influence of the number of neighbors ( K ) on the performance of our proposed method. We assess the performance by varying the number of neighbors from K = 1 to K = 1000 and comparing the results. Figure 13 illustrates the impact of neighbor size K on performance. The results reveal a V-shaped relationship between the number of neighbors and performance. Initially, an increase in the number of neighbors from 1 yields increasing performance improvement. However, when the number of neighbors exceeds a certain threshold (e.g. K > 50 ), performance begins to deteriorate as the neighborhood becomes more global rather than local, eventually reaching a saturation point. Notably, a small neighborhood size of K = 10 is sufficient to capture the local neighborhood, and further increasing the neighborhood size does not yield additional benefits. This finding aligns with previous works [17, 45] that demonstrate the insensitivity of proximity to the choice of K . Based on these findings, we recommend employing a moderate range of neighbors, specifically between 10 and 50, to achieve optimal performance.

Effect of Distance Measure In this study, we explore the impact of different distance measures on the performance of our method. We compare the performance under four distance measures: L2 (Euclidean distance), cosine similarity, IVFFlat, and IVFPQ. L2 distance and cosine similarity are widely used measure in machine learning. IVFFlat and IVFPQ are approximate nearest-neighbor search methods implemented using the faiss library [18]. IVFFlat is memory-efficient and suitable for high-dimensional datasets, while IVFPQ is optimized for datasets with a large number of points and high-dimensional features. The results are depicted in Figure 14. Overall, we observe minimal performance differences across the various distance measures. Specifically, the use of cosine similarity

EffectofNeighborSize

Effect ofNeighborSize

0.06

Conf

0.08

Conf

ETS

ETS

0.05

HB

0.06

HB

IR

IR

ECE

0.04

MIR

ACE

MIR

PTS

0.04

PTS

0.03

PTSK

PTSK

0.02

TS

TS

0.02

0.01

0

200

400

600

800

1000

0

200

400

600

800

1000

# of neighbors under ECE

# of neighbors under ACE

EffectofNeighborSize

Effectof NeighborSize

0.025

Conf

0.08

Conf

ETS

ETS

0.020

HB

HB

IR

0.06

IR

MIR

PIECE

MIR

PTS

PTS

0.010

PTSK

0.04

PTSK

TS

TS

0.005

0.02

0

200

400

600

800

1000

0

200

400

600

800

1000

# of neighbors under MCE

# of neighbors under PIECE

and L2 distance yields comparable performance across the four calibration metrics. Additionally, employing IVFFlat results in slightly smaller calibration errors. However, the performance disparity is not significant. Different distance measures, including the approximation methods, offer similar performance improvements. This is because the density function for proximity values is smooth, and small measurement noise does not significantly impact the final density estimation. Considering efficiency, we recommend utilizing IVFFlat due to its favorable efficiency characteristics. However, it is important to note that the choice of the best distance measure depends on the specific problem and dataset, as each measure may exhibit varying performance in different scenarios.

G Pseudo-codes

We present the procedural steps of our approach in the form of a pseudocode. Algorithm 1 encompasses the general inference phase:

Algorithm 1 Inference procedure.

Algorithm 2 encapsulates the Density-Ratio Calibration algorithm.

EffectofDistanceMeasure:ECE

EffectofDistanceMeasure:ACE

0.08

0.06

Conf

Conf

ETS

ETS

0.05

HB

0.06

HB

IR

IR

0.04

MIR

MIR

PTS

0.04

PTS

0.03

PTSK

PTSK

TS

TS

0.02

0.02

0.01

IVFFlat

IVFPQ

L2

cosine

IVFFlat

IVFPQ

L2

cosine

Effect of Distance Measure: MCE

Effect of Distance Measure:PIECE

0.025

Conf

0.08

Conf

ETS

ETS

0.020

HB

HB

IR

IR

0.06

MIR

MIR

0.015

PTS

PTS

PTSK

PTSK

0.010

0.04

TS

1

TS

0.005

0.02

IVFFlat

IVFPQ

L2

cosine

IVFFlat

IVFPQ

L2

cosine

Algorithm 2 Density-Ratio Calibration.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper quantifies and proposes a mitigation for a phenomenon in DNN training, where more 'unusual' examples (here defined as having a higher average distance to its K=10 nearest neighbors) are generally more miscalibrated across a range of models and tasks. The authors propose a new proximity-aware calibration metric, PIECE, and demonstrate that it can capture calibration issues that are not necessarily captured by the standard ECE metric (and, in fact, always P>= ECE). Additionally, the authors propose a mitigation, ProCal. This mitigation is presented in two variations, one for continuous and one for discrete confidence. These work by adjusting the uncalibrated probability score based on the model's miscalibration on examples with that average distance.\n\nSTRENGTHS:\nThe paper is very clearly written, with a logical flow and good explanations of all steps taken. I particularly enjoyed the motivation via experiments on existing models leading to theory-driven proposals leading to further experimental verification. I also appreciated the through ablation study and the extra attention given to OOD examples.\n\nIn addition, the original miscalibration problem that the paper brings up seems to me to be important and relevant, and is well-motivated. As I am not an expert in miscalibration mitigation techniques I cannot fully comment on the novelty aspect of the work, though I have not seen this 'atypicality' concern brought up explicitly before; it also dovetails neatly with other research in atypical examples.\n\nThe proposed metric and mitigations, while very simple, are logical and effective, which I hope will lead to their adoption.\n\nWEAKNESSES:\nI felt that the 'atypical' (high $D(X)$) examples could have been better characterized. In particular, the authors relate these examples to underrepresented categories in datasets (eg, Black people in health datasets), but this is not necessarily so. It does not seem incredible to have a scenario where a data is divided into two clusters, where the smallest cluster is nevertheless very tight, and so has a smaller average $D(X)$.\n\nIt is not really clear what the ProCAL method refers to. It seems like it mostly splits into two methods each of which have their own name. Maybe it would be better to call them ProCAL-C and ProCAL-D  (for continuous and discrete)?\n\nThe analysis of ProCAL effectiveness seems a little sparse in claiming \"our method consistently improves the calibration\"... My interpretation of the data would be that ProCAL is very helpful in conjunction with raw confidence as well as some of the other methods with higher PIECE scores (TS variants), but not as effective for methods such as IR and MIR, where it seems to hurt as much as help. I don't think that this disqualifies the paper, but a more thorough explanation and analysis of this would be appropriate.\n\n(minor) in section 3.1, what is the sensitivity of two points having the 'same' confidence?\n(minor) It is not clear why the definition of D(X) contains an exponent, rather than the simple average of the ten smallest distances.\n\nQUESTIONS:\nPlease see the weakness section above. In addition:\n\n* For cases where the test distribution doesn't match the train distribution, how much are we losing by using the test data to estimate values of D (versus measuring the distance of a test sample point to the closest 10 training points)? It would seem that the training distribution is actually the important one for estimating atypicality.\n* How does the Bias Index changed if only $B_H$ is used for sampling points and $B_L$ is used for matching, and vice versa? What if the intermediate groups are used?\n* Is it possible to characterize the $B_H$ and $B_L$ groups? Is there a large difference of average confidence between the two, and how well do their confidences intersect?\n* Why is ProCAL less effective in conjunction with IR and MIR?\n\nLIMITATIONS:\nThe limitations are adequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the prevalence of proximity bias in calibration, i.e. the rate of miscalibration on samples that are far away from their nearest neighbors in the data (\"low proximity\"). The authors empirically show that this type of miscalibration is present across many models, and propose a new post-training calibration procedure for mitigating it. Their approach shows significant empirical improvements over standard post-training calibration approaches.\n\nSTRENGTHS:\n- **Originality:** The empirical investigation and proposed methodology in the paper is quite novel, as I am not aware of prior work that has studied this type of bias in calibration (although others have studied subgroup calibration). \n- **Quality:** The claims in the paper are technically sound, and the experiments exploring proximity bias are extensive (more than 500 pretrained models considered, and various calibration baselines).\n- **Clarity:** Overall, the paper is well-written and well-organized, with motivating experiments and intuitive definitions. However, I believe the presentation of the calibration algorithm in the paper could be improved (detailed further in weaknesses).\n- **Significance:** The idea of low proximity samples introduced and analyzed by the authors seems quite significant, as these samples can correspond to underrepresented populations in the data.\n\nWEAKNESSES:\n- **Algorithm Details:** The weakest part of the paper in my view is the lack of detail in Section 5.1. This part of the paper would be significantly improved by including something akin to the pseudo-code algorithm in the appendix. There are several questions that arise when reading this part: what does one do after estimating the posterior probability conditional on prediction and proximity (in the algorithm I can see that this is just the output on the test point)? Do we compute proximity only with respect to the test data? What type of KDE is used (i.e. kernel, bandwidth, etc.)? \n- **Implementation Details:** In addition to algorithm details, some parts of the experimental setup could also be made clearer. How are ECE and ACE computed (i.e. binning scheme)? Do you set aside calibration data for the scaling methods (TS, ETS, PTS, PTSK) in addition to the set aside data for ProCal?\n\nQUESTIONS:\nThe main questions I have are detailed in weaknesses above.\n\nLIMITATIONS:\nThe authors appropriately discuss limitations, but it might be helpful to include some bits from the appendix (particularly regarding efficiency) in the main body.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article focuses on the problem of uncertainty quantification in classification.\nCalibration provides some guarantees on the estimated class probabilities on average. However, subgroups can still be miscalibrated. The article first aims to characterize these subgroup miscalibrations through proximity levels of the samples. It claims that a classifier, even calibrated, tends to be underconfident on high-proximity samples and overconfident on low-proximity samples. To measure this effect, it defines a proximity-informed ECE. Then, it proposes a recalibration framework based on this proximity-informed measure. Finally, it benchmarks the proposed method on numerous datasets and models.\n\nSTRENGTHS:\n* The problem is well presented and motivated. Figure 1 is pedagogical and helps the comprehension of the problem.\n* The idea of characterizing subgroup miscalibrations through proximity is interesting, and refining uncertainty estimates is a good direction.\n* The proposed framework is versatile: it can work as a stand-alone or combined with standard calibration techniques. It provides two versions: binning-based and continuous.\n* The experimental study is substantial.\n  * Datasets are large-scale, numerous, and multimodal: ImageNet, Yahoo-Topics, iNaturalist, ImageNet-LT, MultiNLI, ImageNet-C.\n  * The article studies numerous models, e.g. 504 pre-trained models on ImageNet.\n  * It compares many standard calibration methods, both scaling-based and histogram-based: temperature scaling, ensemble temperature scaling, parameterized temperature scaling, histogram binning, isotonic regression, and multi-isotonic regression.\n* The experimental study provides substantial evidence.\n  * It reveals proximity bias in most of the 504 pre-trained networks on ImageNet (72% according to a Wilcoxon rank-sum test).\n  * The proposed method consistently improves over standard calibration methods.\n  * The time overhead of the method is small, with an increase of 1.17% in inference runtime.\n* Completeness of the study: It reveals the proximity bias, proposes a metric to measure it, a recalibration procedure to address it, and substantial experiments showing consistent improvements.\n\nWEAKNESSES:\nNo major weaknesses.\n\nQUESTIONS:\n* In equation (5) on the confidence score adjustments, why a weight hyperparameter $\\lambda \\in (0, 1]$ is needed? Don't we want to take $\\lambda=1$ to minimize PIECE? The paper talks quickly about regularization. What is the tradeoff? Are results wrong with $\\lambda$ too close to 1? Do you take $\\lambda=0.5$ everywhere in the experiments? How to choose it?\n* Concerning the time overhead introduced by the recalibration methods, isn't it weird that Bin-Mean-Shift has only twice the overhead of isotonic regression (+0.1s vs +0.05s)? Since isotonic regression just needs to apply the mapping function from [0, 1] to [0, 1] to the inferred sample, while Bin-Mean-Shift needs to compute the distance from the new sample to all samples from the calibration set. The same question applies to mapping-based recalibration methods such as histogram binning (+0.03s).\n* Could you develop when to use the histogram-based version (section 5.2) and when to use the continuous version (section 5.1) of your method?\n\n\n---\n\n\nTypos:\n* L148: it should be 80% instead of 85%. \n* L162: repeated word Appendix.\n\nLIMITATIONS:\nThe authors discuss the following limitations:\n* The proposed recalibration technique needs to maintain the calibration set during inference to compute the proximity of the new points.\n* Maintaining the calibration set for inference may challenge the method when memory is limited.\n* Focus limited to the closed-set multi-class classification problem.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work addresses the problem of proximity bias and confidence calibration by performing a comprehensive empirical study of various pretrained ImageNet models. The empirical findings provide insights on persistence of proximity bias even after performing calibration using existing post-hoc calibration algorithms. To mitiagte proximity bias and improve confidence calibration based on sample proximity, the paper proposes PROCAL algorithm that can be used as a plug-and-play method combined with existing calibration approaches. Further, proximity-informed expected calibration error metric is introduced to quantify the effectiveness of calibration algorithms in mitigating proximity bias.\n\nSTRENGTHS:\n- A comprehensive study of pretrained ImageNet models involving various neural network architectures on their model calibration and proximity bias evaluation. The empirical study is performed on image classification tasks (under balanced, long-tail, and distribution-shift settings) and text classification tasks. \n- Experimental evaluation if thorough using various calibration metrics. \n- The paper provides many interesting observations from the empirical study related to proximity bias and model calibration, which is an important area of study under long-tailed data distribution settings.\n\nWEAKNESSES:\n- The presentation of experimental results and writeup of Experiments Section 6 can be improvised, the findings for many of the questions are pointed to the Appendix without any brief details in the manuscript. I understand this is due to page limitation, but I would suggest the authors to focus on the results that are presented in main manuscript, or include the brief info in the manuscript at least.\n\nQUESTIONS:\n- Are the iNaturalist 2021 results in Table 1 based on transfer learning from pretrained ImageNet models, or directly evaluated on pretrained backbone?\n- What are the criteria to choose between density-ratio calibration versus Bin-Mean-shift approach?\n- typo in line#309 ImageNet-TL --> ImageNet-LT\n\nLIMITATIONS:\nThe authors have addressed the limitations of their work in the manuscript.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-26"}