Proximity-Informed Calibration for Deep Neural Networks
Authors: Miao Xiong
Ailin Deng
Pang Wei Koh
Jiaying Wu
Shen Li
Jianqing Xu
Bryan Hooi
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Calibration
Uncertainty Estimation
Trustworthiness
Fairness
Multicalibration
TL;DR: We investigate the proximity bias issue in model calibration and propose a framework to address this issue.

Abstract: Confidence calibration is central to providing accurate and interpretable uncertainty estimates, especially under safety-critical scenarios. However, we find that existing calibration algorithms often overlook the issue of proximity bias, a phenomenon where models tend to be more overconfident in low proximity data (i.e., data lying in the sparse region of the data distribution) compared to high proximity samples, and thus suffer from inconsistent miscalibration across different proximity samples. We examine the problem over $504$ pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling; 4) Models tend to overfit more heavily on low proximity samples than on high proximity samples. Motivated by the empirical findings, we propose ProCal, a plug-and-play algorithm with a theoretical guarantee to adjust sample confidence based on proximity. To further quantify the effectiveness of calibration algorithms in mitigating proximity bias, we introduce proximity-informed expected calibration error (PIECE) with theoretical analysis. We show that ProCal is effective in addressing proximity bias and improving calibration on balanced, long-tail, and distribution-shift settings under four metrics over various model architectures. We believe our findings on proximity bias will guide the development of fairer and better-calibrated} models, contributing to the broader pursuit of trustworthy AI.

Proximity-nnformed Calibration for Deep Neural Networks
Miao Xiong'
Ailin Deng' Pang Wei Koh Jiaying Wu' Shen Li
Jianqing Xu
Bryan Hooi
National University of Singapore University of Washington 3 Google
Abstract
Confidence calibration is central to providing accurate and interpretable uncertainty estimates, especially under safety-critical scenarios. However, we find that existing calibration algorithms often overlook the issue of proximity bias, phenomenon where models tend to be more overconfident in low proximity data (i.e., data lying in the sparse region of the data distribution) compared to high proximity samples, and thus suffer from inconsistent miscalibration across different proximity samples. We examine the problem over 504 pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling: 4) Models tend overfit more heavily on low proximity samples than on high proximity samples. Motivated by the empirical findings, we propose PROCAL, plug-and-play algorithm with a theoretical guarantee to adjust sample confidence based on proximity. To further quantify the effectiveness of calibration algorithms in mitigating proximity bias, we introduce proximity-informet expected calibration error (PIECE) with theoretical analysis. We show that PROCAL effective in addressing proximity bias and improving calibration balanced, long-tail, and distribution-shif settings under four metrics over various model architectures. We believe our findings on proximity bias will guide the development of fairer and better-calibratee models, contributing to the broader pursuit of trustworthy AL.
Introduction
Machine learning systems are increasingly deployed in high-stakes applications such medical diagnosis [30] 36 g () where incorrect decisions can have severe human health consequences. To employed to
ensure safe and reliable deployment, confidence calibration approaches [10 23 28
produce more accurate uncertainty estimates, which allow models to establish trust by communicating their level of uncertainty, and to defer to human decision-making when the models uncertain.
In this paper, we present a calibration-related phenomenon termed proximity bias, which refers to the tendency of current deep classifiers to exhibit higher levels of overconfidence samples of low proximity, i.e., samples in sparse areas within the data distribution (see Figure for an illustrative example). In this study, quantify the proximity of a sample (Eq. [ using the average distance to its K (c.g. K = 10) nearest neighbor samples in the data distribution, and we observe that proximity bias holds for various choices of K. Importantly, the phenomenon persists even after applying existing popular calibration methods, leading to different levels miscalibration across proximities.
The proximity bias issue raises safety concerns in real-world applications, particularly for underrepre- sented populations (i.e. low proximity samples) [30 35 A recent skin cancer analysis highlights
"Miao Xiong (miao.xiong. @ e.uu.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

(x) All Samples: Well-culibrated
[b] High Proximity Samples: Underconfident
[c) Low Proximity Samples: Overconfident
Ratal
0.5
00
61
Figure 1: Samples with lower (higher) proximity tend to be more overconfident (underconfident) The results conducted using XCIT, an Image Transformer, on the ImageNet validation set (All Samples). The sample's proximity is measured using the average distance to its nearest neighbors (K 10) in the validation set. We split samples into 10 equal-size bins based on proximity and choose the bin with the highest proximity (High Proximity Samples) and lowest proximity (Low Proximity Samples).
this concern by revealing that AT-powered models demonstrate high performance for light-skinned individuals but struggle with dark-skinned individuals due to their underrepresentation [1] This issue can also manifest in the form of proximity bias: suppose a dark-skinned individual has high risk of 40% of having the cancer. However, due to their underrepresentation within the data distribution, the model overconfidently assigns them 98% confidence of not having cancer. As a result, these low proximity individuals may be deprived of timely intervention.
To study the ubiquity of this problem, we examine 504 ImageNet pretrained models from the timm library [43] and make the following key observations: 1) Proximity bias exists generally across a wide variety of model architectures and sizes; 2) Transformer-basel models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms including temperature scaling; 4) Low proximity samples more prone to model overfitting while high proximity samples are less susceptible to this issue.
Besides, we argue that proximity bias is overlooked by confidence calibration. Revisiting its definition, F(Y - P p) p for all p [0, 1], we find that its primary goal is to match confidence with the accuracy of samples sharing the same confidence level. However, Figure Th reveals that although the model seems well-calibrated within each confidence group, there still exists miscalibration errors among these groups (e.g. low and high proximity samples) due to proximity bias.
Motivated by this, we propose debiased variant of the expected calibration error (ECE) metric, called proximity-informee
expected calibration error (PIECE) to further capture the miscalibration error due to proximity bias. The effectiveness is supported by our theoretical analysis that PIECE is at least as large as ECE and this equality holds when there is no cancellation effect with respect to proximity bias.
To tackle proximity bias and further improve confidence calibration, we propose plug-and-play method, PROCAL. Intuitively, PROCAL learns joint distribution of proximity and confidence to adjust probability estimates. To fully leverage the characteristics of the input information, we develop two separate algorithms tailored for continuous and discrete inputs. We evaluate the algo- rithms on large-scale datasets: balanced datasets including ImageNet 77] and Fahoo-Topice [49], long-tail datasets iNaturalist 2021 3 and Imagelett-LT [27 and distribution-shif datasets Mult MultiNLI [44 and ImageNet-CC [55) The results show that our algorithm consistently improves the performance of existing algorithms under four metrics with 90% significance (p- (p-value < 0.1).
Our main contributions can be summarized as follows:
Findings: We discover the proximity bias issue and show its prevalence over large-scale analysis (504 ImageNet pretrained models).
Metrics: To quantify the effectiveness of mitigating proximity bias, we introduce proximity-
informed expected calibration error (PIECE) with theoretical analysis.
Method Effectiveness: We propose a plug-and-play method PROCAL with theoretical guarantee and verify its effectiveness on various image and text settings.
2 Related Work
Confidence Calibration Confidence calibration aims to yield uncertainty estimates via aligning model's confidence with the accuracy of samples with the same confidence level [10 25| 28
2

---

To achieve this, Scaling-based methods, such as temperature scaling [TO], adjust the predicted probabilities by learning temperature scalar for all samples. Similarly, parameterized temperature scaling [39] offers improved expressiveness via input-dependent temperature parameterization, and Mix-n-Match [48] adopts ensemble and composition strategies to yield data-efficient and aceuracy- preserving estimates. Binning-basee based methods divide samples into multiple bins based on confidence and calibrate each bin. Popular methods include classic histogram binning [46], mutual-information maximization-basee binning [32), and isotonic regression [47]. However, existing calibration methods overlook the proximity bias issue, which fundamentally limits the methods' capabilities in delivering reliable and interpretable uncertainty estimates.
Multicalibration Multicalibration algorithms
21 aim to achieve a certain level of fair- 13
ness by ensuring that predictor is well-calibrated for the overall population well as different computationalll-idenii
subgroups. [33] proposes a grouping loss to evaluate subgroup calibra- tion error while we propose metric to integrate the group cancellation effect into existing calibration loss. [2I] focuses on understanding the fundamental trade-offs between group calibration and other fairness criteria, and [13] proposes a conceptual iterative algorithm learn multi-calibrated predic- tor. In this regard, our proposed framework can be considered a specific implementation of the faimess objectives outlined in (13), with particular focus on proximity-basee subgroups. This approach offers easier interpretation and implementation compared to subgroups discussed in [13 21.
3 What is Proximity Bias?
In this section, we study the following questions: What is proximity bias? When and why does proximity bias occur?
Background We consider a supervised multi-class classification problem, where input X X ,C} follow a joint distribution T ((X, Y). Let f be classifier with f(X) = (Ý, P), where Y represents the predicted label, and P is the model's confidence, i.e. the estimate of the probability of correctness [10]. For simplicity, we use P to denote both the model's confidence and the confidence calibrated using existing calibration algorithms.
and its label Y E y = (1. ..... 2..
Proximity We define proximity as a function of the average distance between a sample and its K nearest neighbors (X) in the data distribution:
D(X) = exp
1
Σ
dist(X,
(I)
K
(X)
where dist(X, Xx) denotes the distance between sample X and its i-th nearest neighbor X,, estimated using Euclidean distance between the features of X and X, from the model's penultimate layer. K is a hyperparameter (we set K = 10 in this paper). We use the validation set as a proxy to estimate the data distribution. That is, we compute any point's proximity by finding its nearest neighbors in the held-out validation set. Although the training set can also be employed to compute proximity, we utilize the validation because it is readily accessible during the calibration process.
The exponential function used normalize the distance measure from a range of [o, inf] to [o, 1], making the approach more robust to the effects of distance scaling since the absolute distance in Euclidean distance can cause instability and difficulty in modeling. This definition allows us to capture the local density of sample and its relationship to its neighborhood. For instance, sample situated in sparse region of the training distribution would receive a low proximity value, while a sample located in dense region would receive high proximity value. Samples with low proximity values represent underrepresented samples in the data distribution that merit attention, such as rare ("long-tail") diseases, minority populations, and samples with distribution shift.
Proximity Bias
To investigate the relationship between proximity and model miscalibration, we define proximity bias as follows:
Definition 3.1. Given any confidence level the model suffers from proximity bias if the following condition does not hold:
P Y D.D D= =d: = -P = p = p. = dz) Vdi, E 10, + dz-
3

---

Bias Index of Model Confidence
After Scaling Calbration
0.6
well-callbrated
CA-
0.5
INSOO
Swart
DEIT
0.4
XCIT
CalT
SwinV2
Blas Index
0.3
VIT
MLP Mixer
0.2
EfficientNet
Model
MobliaNet
ibration
0.1
VGG
0.0
ResNext
ResNet
Indiox
-0.1
RegNet
-02
0.60
0.65
0.70
0.75
0.80
0.85
0.90
-11
Model Acc
Model Arc
Figure 2: Proximity bias analysis on 504 public models. Each marker represents a model, where marker sizes indicate model parameter numbers and different colors/shapes represent different architectures. The bias index is computed using Equation (3) (0 indicates no proximity bias). Left: We observed the following: 1) Models with higher accuracy tend have larger bias index. 2) Proximity bias exists across wide range of model architectures. 3) Transformer variants (e.g. DEÎT, XCIT, CaiT, and V2in have a relatively larger bias compared to convolution-
networks (e.g.
VGG and ResNet variants). Right: Confidence calibrated by temperature scaling (Upper Right) similar to the original model confidence w.r.t proximity bias. Our PROCAL (Bottom Right) is effective in reducing proximity bias. Analysis of other existing calibration algorithms can be found in Appendix D
The intuition behind this definition is that, ideally, a sample with a confidence level of p should have probability of being correct equal to p. regardless of proximity. However, if low proximity samples consistently display higher confidence than high proximity samples (as shown in Figure ID. it can lead to unreliable and unjust decision-making, particularly for underrepresented populations.
3.1 Main Empirical Findings
To showcase the ubiquity of this problem, we examine the proximity bias phenomenon on 504 ImageNet pretrained models from the timm library [43 and show the results in Figure 2 (see Appendix [orr additional figures and analysis). We use statistical hypothesis testing investigate the presence of proximity bias. The null hypothesis Ho is that proximity bias does not exist, formally, for any confidence p and proximities d, > d2:
IP (x=)\P=pD=<===(<<>>>> P =
p. (ex1)==.D=di=PPP =
(2)
To test the above null hypothesis, we first split the samples into equal-sized proximity groups and select the highest and lowest proximity groups. From the high proximity group, we randomly select 10,000 points and find corresponding points the low proximity group that have similar confidence levels. Next, we reverse this process, randomly selecting 10,000 points from the low proximity group and find corresponding points in the high proximity group with matched confidence. We then merge all the points from the high proximity group into BB and those from the low proximity group into Bi, with the Bh and B2 having similar average confidence. Finally, we apply the Wilcoxon rank-sum test [24] to evaluate whether there is significant difference in the sample means (i.e. accuracy) of By and Bz. More implementation details can be found in Appendix [
Inspired by the hypothesis testing, we define Bias Index as the accuracy drop between the confidence- matched high proximity group B₁ and low proximity group Br, to reflect the degree of bias:
Bias Index E(x.yje
1( =
Ei
118 = YY = Acc(BH) - Acc((B). (3)
BHl
[B
Note that Bir Br, are obtained from the hypothesis testing process and hence have the same mean confidence.

---

The hypothesis testing results indicate that over 80% of 504 models have p-value less than 0.05 (72% after Bonferroni correction (4). i.e. the null hypothesis is rejected with a confidence level of at least 95%, indicating that proximity bias plagues most of the models in timm.
We show the bias index of 504 models in Figure and make the following findings:
Proximity bias exists generally across wide variety of model architecture and sizes. Figure 2 shows that most models (80% of the models supported by hypothesis testing) have bias index larger than 0, indicating the existence of proximity bias.
2. Transformer-basee methods are relatively more susceptible to proximity bias than CNN- based methods. In Figure 2 models with lower accuracy (primarily CNN-based models such as VGG, EfficientNet, MobileNet, and ResNet [12] variants) tend to have lower bias index. On the other hand, among models with higher accuracy, Transformer variants (e.g., DEÎT, XCIT D. CaiT 40, and V2inV2) demonstrate relatively higher bias compared to convolution-baser networks (e.g., ResNet variants). This is concerning given the increasing popularity of Transformer- based models recent years and highlights the need for further research to study and address this issue.
3. Popular calibration methods such as temperature scaling do not noticeably alleviate proximity bias. Figure 2 (upper right) shows that the proximity bias index remains large even after applying temperature scaling, indicating that this method does not noticeably alleviate the problem. In contrast, FigureZi demonstrates that our proposed approach successfully shifts the models to much closer distribution around the line y = 0 (indicating no proximity bias). The bias index figures for more existing calibration methods are provided in Appendix 1)
4. Low proximity samples more prone to model overfitting. Figure 4 in Appendix D shows that the model's accuracy difference between the training and validation set is more significant on low proximity samples (31.67%) compared high proximity samples (0.6%). This indicates that the model generalizes well on samples of high proximity but tends to overfit on samples of low proximity, The overconfidence of low proximity samples can be a consequence of the overfitting tendency, as the overfitting gap also reflects the mismatch between the model's confidence and its actual accuracy.
Proximity-Informed ECE
existing evaluation metrics underestimate the true miscalibration level, as proximity bias causes certain errors in the model to cancel out. As an example, consider scenario:
As depicted in Figure !
Example 4.1. All samples are only drawn from two proximity groups of equal probability mass, 0.2 and d = 0.8, with true probabilities of P(Y = f| being 0.5 and 0.9, respectively. The model outputs the confidence score 0.7 to all samples.
We consider the most commonly used metric, expected calibration error (ECE) that is defined as ECE = Ep [P(( Y P) p In Example 4.1 the ECE is O, suggesting that the model is perfectly calibrated in terms of confidence calibration. fact, the model has significant miscalibration issues: it is heavily overconfident in one proximity group while heavily underconfident in the other, highlighting the limitations of existing calibration metrics. The miscalibration errors within the same confidence group canceled out by samples with both high and low proximity, resulting in phenomenon we term cancellation effect.
To further evaluate the miscalibration canceled out by proximity bias, we propose the proximity- informed expected calibration error (PIECE). PIECE is defined in an analogous fashion ECE, yet it further examines information about the proximity of the input sample, D(X), in the calibration evaluation:
PIECE = Ep | [ =
(P,
where ECE O. we have PIECE 0.2, revealing its miscalibration level in the subpopulations of different proximities, i.e., the calibration error regarding proximity bias. Additionally, we demonstrate in Theorem 4.2 that PIECE always at least as large ECE, with the equality holding only when there is no cancellation effect w.r.t proximity. The detailed proof is relegated Appendix
Back Example 4.1
5

---

Theorem 4.2 (PIECE captures cancellation effect.). Given any joint distribution π (X, Y) and any classifier f that outputs model confidence P for sample X. we have the following inequality, where equality holds only when there is no cancellation effect with respect to proximity:
Ep
P( - - Y
< EppD
- Y -
ECE
PIECE
5 How to Mitigate Proximity Bias?
In this section, we propose PROCAL to achieve three goals: 1) mitigate proximity bias, i.c., ensure samples with the same confidence level have the same miscalibration gap across all proximity levels, 2) improve confidence calibration by reducing overconfidence and underconfidence, and 3) provide plug-and-play method that can combine the strengths of existing approaches with our proximity-informed approach.
The high-level intuition explicitly incorporate proximity when estimating the underlying probabil- ity of the model prediction being correct. In addition, existing calibration algorithms can be classified into types: 1) those producing continuous outpuls, exemplified by scaling-based methods [TO]; 2) those producing discrete outputs, such as binning-based methods, which group the samples into bins and assign the same scores to samples within the same bin. To fully leverage the distinct properties of the input information, we develop two separate algorithmss tailored for continuous and discrete inputs. This differentiation is based on the observation that continuous outputs (e.g.. those produced by scaling- baaedd methods) contain rich distributional information suitable for density estimation. On the other hand, discrete inputs (e.g., those generated by binning-bat based methods) allow for robust minning-basee adjustments. By treating these inputs separately, we can effectively harness the characteristics of each type.
In summary, Density-Ratin Calibration ($5.)) estimates continuous density functions, and aligns well with type 1) methods that produce continuous confidence scores. contrast, Bin-Mcan-Shiin ($5.2) does not rely on densities, making it more compatible with type 2) calibration methods that yield discrete outputs. Together, these two calibration techniques constitute versatile plug-ant aadg--ad-llay framework PROCAL, applicable to confidence scores of both continuous and discrete types.
5.1 Continuous Confidence: Censity-Ratio Calibration
The common interpretation of confidence is the likelihood of model prediction Ý being identical to the ground truth label Y for every sample X. Computing this probability directly with density estimation methods be computationally demanding, particularly in high-dimensional spaces. To circumvent the curse of dimensionality and address proximity bias, we incorporate the model confidence 1 and proximity information D(X) to estimate the posterior probability of correctness, i.e, P(-- = Y P, D). This approach is data-efficient since it conducts density estimation in two-dimensional space only, rather than in the higher dimensional feature or prediction simplex space.
Consider a test sample X with proximity D = D(X) and uncalibrated confidence score P, which can be the standard Maximum Softmax Probability (MSP), or the output of any calibration method. F( Y P, D) can be computed via Bayes' rule:
P (..DI&-V)P(Y---.
P (
P(P,D)
where Y is the model prediction and Y is the ground truth label. This can be re-expressed as follows by using the law of total probability:
P(P, D & (P.DIÝ=Y rr-
P P(P.D(8-Y) +P ( (P,DII+V) 7((=Y)
M(Px))

---

To compute this calibrated score, we need to estimate the distributions IP (P.D) 1 =X)
and
(P.D +V). and the class ratio
PY
P(Y P(==))"
To estimate the probability density functions P (P.DIÝ=Y) and (P.DIV various density estimation methods can be used, such parametric methods like Gaussian mixture models or non-parametric methods like kernel density estimation (KD We choose KDE because flexible and robust, making assumptions about the underlying distribution (see Appendix [ specific implementation details). Specifically, we split samples into two groups based on whether they are correctly classified and then use KDE estimate the two densities. To obtain the class ratio P(Ý=Y)
P(Y=Y)' we simply use the ratio of the number of correctly classified samples and the number of misclassified samples in the validation set. The pseudocode for inference and training can be found in Appendix and
5.2 Discrete Confidence: Bin Mean-Shift
The Bin-Mean- Shift approach aims to first use 2-dimensional binning estimate the joint distribution of proximity D and input confidence P and then estimate P (y = Y -.,.D)) Considering test
samples X with proximity = ((X) and uncalibrated confidence score P. we first group samples into 2-dimensional equal-size bins based on their D and İ (other binning schemes can also be used; choose quantile for simplicity). Next, for each bin Buh. calculate its accuracy A( (mnh) and mean confidence F F(Bmr). Then, the confidence scores of samples within the bin are adjusted as:
Pours P 1P+A X (A(Bmhh) F(Bmi))
(5)
where the shrinkage coefficient À E (0, 1] is a Ideally, setting À 1 would ideally achieve our goal. However, in practice, we often encounter bins with a smaller number of samples, whose estimate of A(Bm)) - F(Bmh) will have high variance and therefore inaccurate. To reduce variance in these scenarios, we can set smaller A. practice, we choose À = 0.5 as a reasonable default for all our experiments, which we find offers consistent performance across various settings.
controlling the bias-variance trade-off.
Note that our approach (i.e. applying a mean-shift in each bin) differs from the typical histogram binning method (replacing the confidence scores with its mean accuracy in each bin). Rather than completely replacing the input confidence scores P (which are often reasonably well-calibrated), our approach better utilizes these scores by only adjusting them by the minimal mean-shift needed to correct for proximity bias each of the 2-dimensional bins.
5.3
Theoretical Guarantee
Here we present that our method, Bin-Mean-Shiff, can consistently achieve a smaller Brier Score given sufficient amount of data in the context of binary classification. The Brier Score (5) is strietly proper score function that measures both calibration and accuracy aspects (22], with a smaller value indicating better performance. As illustrated below, our algorithm's Brier Score is asymptotically bounded by the original Brier Score, augmented by a non-negative term.
Theorem 5.1 (Brier Score after Bin-Mean-SS Shift asymptotically bounded by Brier Score before calibration). Given joint data distribution
and binary classifier f.ror any calibration algorithm that outputs score h(P) based on model confidence P. we apply Bin-Mean-Shine to derive calibrated score hi ()P) as defined in Equation 5p. Let he hp(P)) h((P) x 11 = 13 + (1
h h())) 1( = =0} denote the probability assigned to class I by h(PP), and define (P(P) similarly. Then, the Brier Score before calibration can be decomposed as follows:
Em(x,Y) (hc(P)) -1) = Em(X,Y) (nac(P) -x) + EBBP(E)
(A(B)- +o(1),
Brier Score before Calibration
Brier Score after Calibration
20
where IP PPB) is determined by the binning mechanism used in Bin-Mean-Shhn
7

---

Table Calibration performance of ImageNet pretrained ResNer50 on long-tail dataset iNaturalist
2021. denotes significant improvement (p- (p-value 0.1). 'Base' refers existing calibration methods,
"Ours' to our method applied to calibration. Note that 'Conf+Ours' shows the result of our method applied directly model confidence. Calibration enror is given by x x10-?
ECE
ACE
MCE 4
PIECE
Method
base
+ours
base
+ours
base
+ours
base
+ours
Conf
4.85
0.78*
4.86
0.76*
0.55
0.18*
4.91
1.51 *
TS
2.03
0.70*
2.02
0.78*
0.30
0.14*
2.34
1.43*
ETS
1.12
0.66
1.15
0.77
0.18
0.13*
1.79
1.38*
PTS
4.86
0.71
4.90
0.86*
2.96
0.12
7.04
1.44*
PTSK
2.97
0.65*
3,01
0.82*
0.56
0.11 *
4.66
1.41*
MIR
1.05
0.98
1.09
1.06
0.18
0.19
1.63
1.64
Remark. Note that when the calibration algorithm is an identity mapping. it demonstrates that Bin-Mean-Shift achieves better model calibration performance than the original model confidence, given a sufficient amount of data. The detailed proof is relegated Appendix
6 Experiments
In this section, we aim answer the following questions:
Performance across different datasets and model architectures: How does PROCAL perform on datasets with balanced distribution, long-tail distribution, and distribution shift, as well as on different model architectures?
Inference efficiency: How efficient is our PROCAL? (see Appendix E.1
Hyperparameter sensitivity: How sensitive is PROCAL to different hyperparameters, e.g.
neighbor size K? (See Appendix F.2)
Ablation study: What is the difference between Density-Ratio and Bin-Mean-shiff on calibration? How should the choice between these techniques be determined? (See Ap-
pendix F.1
6.1 Experiment Setup
we adopt 3 commonly used metrics to evaluate the confidence calibration: Expected Calibration Error (ECE), Adaptive Calibration Error (ACE (29), Maximum Calibration Error (MCE) and our proposed PIECE to evaluate the bias mitigation performance. More detailed introduction of these metrics can be found in Appendix[C
Evaluation Metrics. Following 10
Datasets. We evaluate the effectiveness of our approach across large-scale datasets of three types of data characteristics (balanced, long-tail and distribution-shiftee) in image and text domains: (1) Dataset with balanced class distribution (i.e. each class has an equal size of samples) on vision dataset ImageNet 7 and two text datasets including Yahoo Answers Topics (49] and MultiNLI-match [44]:
(2) Datasets with long-tail class distribution on two image datasets, including iNaturalist 2021 3 and
ImageNet-LT (27]: (3) Dataset with distribution-shift on three datasets, including ImageNet-C [15
MultiNLI-Mismatch [4] and ImageNet-Skedt 42)
Comparison methods. We compare our method to existing calibration algorithms: base confidence score (Conf) [16,, scaling- based methods such Temperature Scaling (TS) [10), Ensemble Temper- ature Scaling (ETS) [48), Parameterized Temperature Scaling (PTS) [3)]. Parameterized Temperature Scaling with K Nearest Neighbors (PTSK), and binning based methods such as Histogram Binning (HB), Isotonic Regressiion (IR) and
Regression (MIR) [48). Throughout the experiment section, we apply Density-Raue Ratio Calibration Conf, TS. ETS, PTS, and PTSK and apply Bin-Mean- Shift to binning-bass based methods IR, HB, and MIR. HB and IR are removed from the long-tail taigtail setting due to its instability when the class sample size is very small.
More details on baseline algorithms, datasets, pretrained models, hyperparameters, and implementa- tion details can be found in Appendix C
8

---

MCC
PTEC
Nov Valu
/wl.//ooonnoo Accuracy
Valdaiiin Acourary
Uniddtion Accuracy
Val dilidiiio Accurecy
Figure 3: Calibration errors on ImageNet across 504 timm models. Each point represents the calibration result of applying calibration method the model confidence. Marker colors indicate different calibration algorithms used. Among all calibration algorithms, our method consistently appears at the bottom of the plot. See Appendix Figure for high resolution figures.
Table 2: We use RoBERTa models [26] fine-tuned on Yahoo and MultiNLI Match, respectively, as their models. 'Base' refers to existing calibration methods and 'Ours' refers to our method applied to existing calibration methods. Calibration error is given by x 10
BCE J
ACE
MCE
PIECE
BCE
ACE
MCE
PIECE
Method
buse
base
basc
+ours
busc
Method
base
hase
base
Coaf
4:56
0.51
Conf
2.47
0.50
08.42
0.46
2.22
1.34
TS
1.70
1.22
1.76
1.35
0:41
0.60
3.03
2.78
ETS
0.62
0.43
0.59
0.45
0.09
0.07
2.22
1.37
ETS
1.58
1.24
1.56
1.26
0.66
0.39
3.04
2.77
PTS
0.55
0.42
0.52
0.42
0.10
0.09
2,03
1.41
PTS
7,53
2.42
7.50
2.46
4.21
0.93
7.78
3.56
0.47
PISK
10.14
4.26
10.14
4.40
7.40
2.83
10.33
5.15
1.50
IR
0.64
0.84
0.64
0.77
0.13
0.19
2,08
1.75
IR
0.88
1,43
1,07
1.23
0.36
0.34
2.46
2.61
MIR
0.63
0.72
0.54
0.61
11.11
0.18
2,08
1.65
MIR
0.71
1.03
1.07
1.36
0.26
0.40
2.58
2.35
(a) Yahoo Answer Topics
(b) MultiNLI Mismatch
6.2 Effectiveness
Datasets with the balanced class distribution. The results on ImageNet of 504 models from t timm 43 are depicted in Figure where our method (red color markers) consistently appears at the bottom, achieving the lowest calibration error across all four evaluation metrics in general. This indicates that our method consistently outperforms other approaches in eliminating proximity bias and improving confidence calibration. We also select four popular models from these 504 models. specifically BeiT [2), MLP Mixer (38), ResNet55 [12 and ViT [8] A summary of their results is presented Table 4 of Appendix E Additionally, Table 2a and Table present the calibration results for the text classification task on Yahoo Answers Topics and the text understanding task on MultiNLI, where our method consistently improves the calibration of existing methods and model confidence. SceAppendix for more details.
Datasets with the long-tail class distribution. Table shows the results on the long-tail image dataset iNaturalist 2021. Our method ('ours') consistently improves upon existing algorithms ("base") regarding reducing confidence calibration errors (ECE, ACE, and MCE) and mitigating proximity bias (PIECE). Note that when used independently ( ('Conftours') without combining with existing algorithms, our method achieves the best performance across all metrics. This result suggests that our algorithm make the model more calibrated in the long-tail tail setting by effectively mitigating the bias towards low proximity samples (i.e. tail classes), highlighting its practicality in real-world scenarios where data is often imbalanced and long-tailed. ImageNet-LT results in Table 6 of Appendix E.3 show similar improvement.
Datasets with distribution shift. Table 2b shows our method's calibration performance when trained on an in-distribution validation set (MultiLLI Match) and applied to a cross cross-domain test set (MultiNLI Mismatch). The results suggest that our method can improve upon most existing methods on ECE, ACE and MCE, and gain consistent improvement on PIECE, indicating its effectiveness in mitigating proximity bias. Moreover, empirical results on ImageNet-C (Figure [2] and ImageNet- Sketch (TTalle 77) also demonstrate consistent improvements of our method over baselines. Besides, compared to Bin-Mean-Shiin Density-Rand Ratio exhibits more stable performance on enhancing the existing baselines. More analysis their comparison can be found in Appendix F.I

---

7 Conclusions and Discussion
In this paper, we focus on the problem of proximity bias in model calibration, a phenomenon wherein deep models tend to be more overconfident data of low proximity (i.e. lying in the sparse region of data distribution) and thus suffer from miscalibration. We study this phenomenon on 504 public models across wide variety of model architectures and sizes ImageNet and find that the bias persists even after applying the existing calibration methods, which drives us to propose PROCAL for tackling proximity bias. To further evaluate the miscalibration due to proximity bias, we propose proximity-informed expected calibration emorr (PIECE) with theoretical analysis. Extensive empirical studies balanced, long-tail, and distribution-shiftes datasets under four metrics support findings and showcase the effectiveness our method. P
Potential Impact, Limitations and Future Work We uncover the proximity bias phenomenon and show its prevalence through large-scale analysis, highlighting its negative impact the safe deployment of deep models, e.g. unfair decisions on minority populations and false diagnoses for underrepresented patients. We also provide PROCAL as a starting point to mitigate the proximity bias, which we believe has the potential to inspire more subsequent works, serve as a useful guidance in the literature and ultimately lead to improved and fairer decision-making in real-world applications, especially for underrepresented populations and safety-critical scenarios. However, our study also has several limitations. First, our PROCAL maintains a held-out validation set during inference for computing proximity. While we have shown that the cost can be marginal for large models (see Inference Efficiency Appendix E.T), may be challenging if applied to small devices where memory is limited. Future research can investigate the underlying mechanisms of proximity bias and explore various options to replace the existing approach of local density estimation. Additionally, we only focus on the closed-set multi-class classification problem; future work can generalize this to multi-label, open-set or generative settings.
Acknowledgments
The author extends gratitude to Yao Shu and Zhongxiang Dai for the insightful discussions during the review response period, and Yifei Li for providing constructive feedback on the manuscript draft. This research is supported by the National Research Foundation Singapore under its Al Singapore Programme (Award Number: [AISG2-TC-2021-000]]
References
[1] Alaaeldin Ali, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, et al. Xeit: Cross- covariance image transformers. Advances in neural information processing systems, 34:20014 20027, 2021.
[2] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image trans- formers, 2022.
[3] Sara Beery, Arushi Agarwal, Elijah Cole, and Vighnesh Birodkar. The iwildcam 2021 competi- tion dataset. arXiv preprint arXiv:2105.03494 2021.
[4] Carlo Bonferroni. Teoria statistica delle classi calcolo delle probabilita. Pubblicazioni del Istituto Superiore di Scienze Economiche e Commericiali di Firenze, 8:3-62, 1936.
[5] Glenn Brier et al. Verification forecasts expressed terms of probability. Monthly weather review, 78(1):1-3, 1950.
[6] Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Mare Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 2 ACM SIGKDD international conference on knowledge discovery and data mining, pages 1721-1730, 2015.
Our codes are available at: http:///gttiiiaanakkkknuuuuuuuga Calibbatttaiiioo.ggi
10

---

[7] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. leee, 2009.
[8] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprini arXiv:2010.11922 2020.
[9] Omar Elfanagely, Yoshiko Toyoda, Sammy Othman, Joseph A Mellia, Marten Basta, Tony Liu, Konrad Kording, Lyle Ungar, and John P Fischer. Machine learning and surgical outcomes prediction:
systematic review. Journal of Surgical Research, 264:346-361 3613666661166 2021.
[10] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321-1330. PMLR, 2017.
[11] Lisa N Guo, Michelle Lee, Bina Kassamali, Carol Mita, and Vinod E Nambudiri. Bias in, bias out: underreporting and underrepresentation of diverse skin types in machine learning research for skin cancer detection-a scoping review. Journal of the American Academy of Dermalology, 87(1):157-159, 2022.
[12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, abs/ 0bbs/1512.0336d 2015. URL httt:://axiiiii httpaaawwwwa...aaaaaaoaaaaoooaaa abs///a2..111/ 03385
[13] Ursula Hebert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the cumpputatioallly-deniffiaa editors, Proceedings of the 35th International Conference on Machine Learning. volume 80 of Proceedings of Machine Learning Research, pages 1939-1948. PMLR, 10-15 Jul 2018.
masses. In Jennifer Dy and Andreas Krause,
[14] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv: / /2261, 2019.
[15] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv: 1903. 12261, 2019.
[16] Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv: 2016.
[17] Heinrich Jiang. Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust classifier. Advances in neural information processing systems, 31, 2018
[18] Jeff Johnson. Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):335-55) 547, 2019.
[19] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535-547, 2019.
[20] Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. arXiv preprint 191.0:1110.09217,, 2019.
[21] Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. 67, 09 2016.
[22] Volodymyr Kuleshov and Shachi Deshpande. Calibrated and sharp uncertainties deep learning via density estimation. In International Conference on Machine Learning, pages 11683-11693, PMLR, 2022.
[23] Meelis Kull, Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' AAlché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.

---

[24] FC Lam and MT Longnecker. A modified wilcoxon rank sum test for paired data. Biometrika, 70(2):510-513, 1983.
[25] Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Taking step back with kcal: Multi-class kernel-based calibration for deep neural networks. arXiv preprint arXiv:2202.07671 076797 2022.
[26] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach, 2019.
[27] Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang. Boqing Gong, and Stella X Yu. Large- scale long-tailed recognition in an open world. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 2537-2546, 2019.
[28] Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In Advances in Neural Information Processing Systems, volume 34, pages 15682-15694, 2021.
[29] Jeremy Nixon, Michael Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In CVPR Workshops, volume 2, 2019.
[30] Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in algorithm used to manage the health of populations. Science, 366(6464)):47-553, 2019.
[31] Emanuel Parzen. On estimation of probability density function and mode. The annals of mathematical statistics, 3367) 1962.
[32] Kanil Patel, William Beluch, Bin Yang, Michael Pfeiffer, and Dan Zhang. Multi-class un- certainty calibration via mutual information maximization-basee binning. arXiv preprint arXiv:2006. 13092, 2020.
[33] Alexandre Perez-Lebel, Marine Le Morvan, and Gaël Varoquaux. Beyond calibration: estimat- ing the grouping loss of modern neural networks. In /CLR, 2023.
[34] M Rafiee and M Abbasi. Pruned kd-tree: a memory-effiiiet algorithm for multi-field packet classification. SN Applied Sciences, 1(12):1537, 2019.
[35] Alvin Rajkomar, Michaela Hardt, Michael D Howell, Greg Corrado, and Marshall Chin. Ensuring fairness in machine learning advance health equity. Annals of internal medicine, 169(12):66--77, 2018.
[36] Mark Sendak, Madeleine Clare Elish, Michael Gao, Joseph Futoma, William Ratiff, Marshall Nichols, Armando Bedoya, Suresh Balu, and Cara O' Brien. The human body is a black box supporting clinical decision-making with deep learning. In Proceedings of the 2020 conference on fairness accountability, and transparency, pages 90-109, 2020.
[37] Jonathan Taylor. statsmodels: Statistical modeling and econometrics in Python, 2009-. URL https: http://wwwsstatmmdes...oo org
statsmodelss
[38] Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai Thomas Un- terthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy, Mlp-mixer: An all-mlp architecture for vision. arXiv preprint arXiv:2105.00 006000 2021.
[39] Christian Tomani, Daniel Cremers, and Florian Buettner. Parameterized temperature scaling for boosting the expressive power post-hoc uncertainty calibration. In Computer Esion-ECCCV 2022: 17th European Conference, Tel Aviy, Israel, October 23-27, 2022, Proceedings, Part XIII, pages 555-569. Springer, 2022.
[40] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers amp; distillation through attention. In International Conference Machine Learning, volume 139, pages 10347-10357, July 2021.
12

---

[41] Grant Van Horn, Elijah Cole, Sara Beery, Kimberly Wilber, Serge Belongie, and Oisin Mac Aodha. Benchmarking representation learning for natural world image collections. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 12884-12893, 2021.
[42] Haohan Wang, Songwei Ge, Zachary Lipton, and Eric Xing. Learning robust global represen- tations by penalizing local predictive power. In Advances in Neural Information Processing Systems, pages 10506-10518, 2019.
[43] Ross Wightman. Pytorch image models. 2019.
https://gtthabb.com////ggggagi
pytorch-image-mode
[44] Adina Williams, Nikita Nangia, and Samuel R Bowman. broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:/704.05426, 2017.
[45] Miao Xiong, Shen Li, Wenjie Feng, Ailin Deng, Jihai Zhang, and Bryan Hooi. Birds of a feather trust together: Knowing when trust classifier via adaptive neighborhood aggregation. arXiv 2022.
preprint
[46] Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. Ieml, volume pages 609-616, 2001.
[47] Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 694-699, 2002.
[48] Jize Zhang. Bhavya Kailkhura, and T Yong-Jin Jin Han. Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning. In International conference on machine learning, pages 1 1111-11112 1128. PMLR, 2020.
[49] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances neural information processing systems, 28, 2015.
13

---

A Proof of Theorem 5.1
Notation, (Xx, Y) denote the true underlying distribution of the sample X and its label Y. The empirical dataset used for Bin-Mean-Shine is denoted by D, = {(x1, U1); (X2, U2)..... (X,, yn } where is the number of samples in the dataset, and each datapoint (x; Mc) is independently and identically sampled from π (XX, Y).
classifier f is defined as f(x)= = (Ý. where Y represents the predicted label and P represents the model's confidence, i.e., the estimate of the probability of correctness (TO)
We use h( to denote the output confidence score for class prediction Y calibrated by any calibration algorithm h. For example, in the simplest case, h can be the identity mapping. in which case h( P) represents the original model confidence.
To simplify notation, use B (3) to denote the bucket to which X belongs. Given bucket B, compute the empirical accuracy A., (B) and mean confidence score Fx( for samples in the bucket, based on the empirical dataset D.. Given the classifier f and calibrator h, they are computed as follows:
An(B)
Σ (x.y)ED,
1( = 1) A (B(x) = B)}
(6)
B
(B)
Σ
h(P)
B
(x.))
Additionally, we use A(B) and F(B) to denote the expected confidence and actual accuracy of samples from the underlying data distribution TT (XX, Y) and belonging to the bucket B:
A(B) (xx)) [(Y 8} B(X) B
(8)
F(B) Ex [((P) ( B)
With these definitions, our proposed Bin-Mean-Siie -Siftt algorithm can be expressed as:
h(P) = h(P) + An (B) - (B),,,
(10)
where B is the bucket that the input sample X belongs to, and h((P) is the score calibrated using our Bin-Mean-Shift algorithm.
First revisit the definition of Brier Score. The Brier Score [5] is strietly proper score function that measures both calibration and accuracy aspects [22], with a smaller value indicating better performance. The Brier Score is defined mean square loss as follows for binary classification:
(mo(e))-))
Brier Score Exx()))
(11)
where ho(P) (P) is the probability assigned to class by the calibration algorithm h:
hc(P) = ((P)) h(P) when Y=
when ý= ==1
(12)
In short, this can be represented he((P) h(P) x 1( = 1) Theorem A.1 (Brier Score after Bin-Mean Mea----inn---tttt is asymptotically bounded by Brier Score before calibration). Given a joint data distribution =(X, Y) and a binary classifier foror any calibration algorithm that outputs a score h((P) based on model confidence P, we apply Bin-Mean-Shhe Mean-Shiff to derive the calibrated score T.(P)) as defined in Equation (10). Let h((P) = 1((P) x 118
=[.
1} + (1- 11--h h(()) x 1(Ý = =0 denote the probability assigned to class I by h((P), and define he (P)
similarly. Then, the Brier Score before calibration can be decomposed as follows:
ExXXY (Ma(P)-))) =
ExXx. (T.(P)) + +EB-P(B)
(A. F.(B)) +o(1),
Brier Score before Colibration
Brier Score after Calibration
20
where IPB) is determined by the binning mechanism used in Bin-Mean-Shiff.
14

---

Proof.
First we prove the following equality which re-expresses the Brier score in an equivalent
form:
(xx() (()(())>vr] (x))) (n)
2)
(13)
where = =1Y 87 is a binary variable representing whether the model's prediction is correct:
11Y = 0} = =1 YY when y = 0
11
(14)
11 13 =Y, when y 1
Then we consider hc(P) -Y. By Eq. 12
(1- h((P) Y)2 when
Y
(ho(P) -
(15)
(h(P)- -Y),
when
Y
( (P-((P))2 when y= 0 (h(P)- - Y)2, when y=
(16)
So we have (ho(P)- x)² = (h(P) - ))² which completes the proof of the equality in Eq.
13
Using this, we rewrite the Brier score follows:
Brier Score B (M(P))
(17)
Second, we decompose the Brier score:
Ba(x,r) [M1P)) - -}) = (X.Y)
(M(P) - -h(P) + hh(p) - -1)
(18)
(M(P) - - -F) }
Ex(x
(19)
(0)
(M(P) (h(P))h(17)
h(P))² -h(P)) 21
+ Bx(x,))
(20)
(6)
[(M(P]) - -M(i)) (8)
2 (xx
(21)
(e)
First note that term (a) is the Brier score after calibration (recalling our earlier equivalent form for the Brier score). For term (b) we recall that h( P) h(P)+ + (B) F,, (B). which is our proposed Bin-Mean-Shhe Shift algorithm in Equation (TO). Note that X can be sampled by first sampling the bin B
and then sampling the point from the corresponding bin. So term (b) can be expressed as:
Ex(x,r) [((B)) (o(P))((P)) h(P)))
(M(P)- h(P)) An(B))+ FA())) "
E
Em(x,r) (An((B) - FB,(1))) '
Ex -,((x)) |
BBBP(B) (Bn(
where (B(X)) and (B())) remain the same for all samples following into the same bucket B.
15

---

For term (c) we show that:
Bx(X,,) ((((P) (h(P)) (
(22)
= -Ba(x.)) ( ((11013))-111101)) (atl
(23)
- (F(B(x))) A(B(X)))
( -(((()))))
((())-))
(24)
(An(B)- -(X-I)((X)111 -
Y-h(()) h(P))
BB-P(B)
(25)
(d)
where in the last step (An - F.)) is function of B and thus can be moved out of the inner expectation. For term (d) we have:
B(XYI-((XYII) y [(P)
(26)
= -BXXYWRRXXY1B --h(P)-A,BB)++
(27)
(i AnB) + (Fn(B)-h(P))
= =B(X.))P(XXM11)
(28)
(E(X,))-XXXXI
Y -A,(B) + (F,((B)-F(1))
(29)
(e)
For term (e) we have:
B(X.I)-((XX11) Y = -ExX,))=XXXYII)
| [yp] A(B).
(30)
By the Law of Large Numbers, the sample means converges to their expectations as follows:
lim =B) = and lim An A(B)
(31)
n
This further leads to the following statement for term (c):
lim (X,x) (M(P)-((P) (b(P)- O.
-
(32)
Then finally we have the statement:
lim Bx(X. (M(P))>) - Bax ((.(P)-))
EB-P(B)
=0
Brier Score before Calibration
Brier after Calibration
20
Discussion
This theorem shows that Bin-Mean-SSe Shift (BMS) preserves the calibration properties of the input scores P: i.e. h(P) was already well-calibrated, the BMS-calibrated scores h(P) will continue to be well-calibrated (up to an o(1) term). At the same time, BMS helps to correct for miscalibrations with respect to any choice of buckets (represented by the (A, - (B))² term).
Interestingly, this theorem implies that we can have theoretical guarantees for pipeline of different calibration methods: for example, consider a pipeline consisting of any calibration method h( h(P). followed by one more applications of BMS with different choices of binning schemes. Then this theorem shows that the Brier score will decrease with each application of BMS (setting aside o(1) term). Thus, in contrast to the theoretical guarantees of existing calibration methods (such as histogram binning), which only apply to a single calibration method, our approach points to the theoretical benefits of such pipelines of calibration methods.
16

---

B Proof of PIECE Guarantee
Recall that f is classifier (c.g. neural network) with f(x) = (Y, where y represents the predicted label, and P the model's confidence, i.e. the estimate of the probability of correctness [10]. Theorem B.1 (PIECE captures cancellation effect.). Given any joint distribution x (X, Y) and any classifier f that outputs model confidence P for sample X. we have the following inequality, where equality holds only when there is cancellation effect with respect proximity:
Bp
P(
Y P) P|]
- EpD
| = I
P, - P|]
ECE
PIECE
Proof. Note that ECE
10 is defined as:
ECE = Ep [(( - PP]
(33)
We first consider the formula within the expectation:
TP = Y P)
P
Ex,y
IT = YY ||] P
118- = -Y} - أ1 P|
Exyy
ED
E Ex
[( = y) P P, D
(34)
(b)
ED
E Ex,Y
I[8 =Y} - 1
ED
Ex,y [T(` = YY |p. P
En
( |
~|]
where (a) is derived using the Law of Total Expectation and (b) is derived using Jensen's 's Inequality.
The equality holds if and only if IP = Y P D) - is a linear function in terms of D. This condition is satisfied only when there no cancellation effect with respect to proximity D, i.e. either F(Y- Y P, D) 2 P or P(Ý- Y P. D) s P hold for all choices of D.
Then applying this formula to Equation
33
we have:
ECE = Ep
P(Ý = Y P) P|]
En |P(Ý =Y | D) r|]
$ Ep
(35)
E. |P- = y P) 11]
PIECE
C Experimental Setup
Evaluation Metrics. Following 10 29 we adopt three commonly used metrics to evaluate the
confidence calibration: Expected Calibration Error (ECE), Adaptive Calibration Error (ACE (29). Maximum Calibration Hrror (MCE) and our proposed PIECE to evaluate the bias miligation perfor- mance. To compute these metrics, we first divide samples into M 15 bins and compute every bin's 's average confidence and accuracy. Then we compute the absolute difference between each bin 's average confidence and its corresponding accuracy. The final calibration error is measured using the weighted difference (the fraction of samples in each bin as the weight). The key distinction between ECE and ACE lies in the binning scheme: ECE divides bins with equal-confidence intervals while ACE uses an adaptive scheme that spaces the bin intervals to contain an equal number of samples in each bin. In addition, PIECE splits the bin both based on confidence and proximity. It splits samples into M = 15 nquber bins based confidence and then split every confidence group into H = 10 equal-number bins based on proximity. MCE chooses the bin with the largest difference between average confidence and accuracy and output their absolute difference the final error.
17

---

Datasets. We evaluate the effectiveness of our approach across large-scale datasets of three types of data characteristies (balanced, long-tail and ciriibbtton-ssitteee Dataset with balanced class distribution (i.e. each class has an equal size of samples) on vision dataset ImageNet 7 and two text datasets including Yahoo Answers Topies [49] and MultiNI.I-machh [44]
image and text domains: (1)
(2) Datasets with long-tail class distribution on two image datasets, including iNaturalist 2021 [ and
ImageNet-LT [27]: (3) Dataset with distribution-shif on three datasets, including ImageNet-C [5 MultiNLL-Mismatch [44] and ImageNet-Sketch [42
Comparison methods. We compare our method to existing calibration algorithms: base confidence score (Conf) [1)., scaling-based methods such as Temperature Scaling (TS) [10], Ensemble Temper- ature Scaling (ETS) 48. Parameterized Temperature Scaling (PTS) [3), Parameterized Temperature Scaling with K Nearest Neighbors (PTSK), and binning based methods such as Histogram Binning (HB), Isotonic Regression (IR) and Multi-Isotonic Regression (MIR) [8 Throughout the experiment section, we apply Density- Ratio Calibration to Conf, TS, ETS, PTS, and PTSK and apply Bin-Mean Shift to binning-basse methods IR, HB, and MIR. HB and IR are removed from the long-tai tail setting due to its instability when the class sample size is very small.
Pretrained Models. For the proximity bias analysis in section B and the balanced ImageNet evaluation, we use 504 pretrained models from timm 443 (the list of models are shown in the code repository). For ImageNet-I.T evaluation, we use the model ResNext50 pretrained by Liu et al. [27] using classifier re-training technique. For iNatralist 2021, we directly use the ImageNet pretrained backbone ResNet [12) which we follow this paper [4] and download from the repo For Yahoo Answers Topics and MultiNLI datasets, we use pretrained RoBERTa from HuggingFace API and fine-tuned on the corresponding datasets with 3 epochs.
Hyppepaaraeererrs Regarding nearest neighbor computation, we use indexFlatL2 from faiss [8] Except the Hyperparamerter sensitivity experiments, we use K 10 for the proximity computation. Regarding our method, for Pensity-Ratio, the kernel density estimation for two variables are im- plemented using statsmodel library (37. For the Bin Mean-Shift method, we set the regularization parameter À = 0.5. For the calibration setup, we adopt a standard calibration setup [10] with a fixed-size calibration set (i.e. validation set) and evaluation test datasets. Specifically, we randomly split the hold-out dataset into calibration set and evaluation set n. nr = 25000 for ImageNet, nh mr 50000 for iNaturalist 2021 and n. = ne 5000 for Image For Yahoo and MultiNI Mathh dataset, we sample 20% data from the training dataset as calibration set and use the original test dataset as the test dataset. For the evaluation, we use random seed 2020, 2021, 2022, 2023, 2024 and compute the mean (this does not apply to NLP dataset for its fixed test set).
Details Statistical Hy Hypothesis Testing. The statistical hypothesis testing requires that for two samples with the same confidence but different proximity, their probability of being correct is the same. To achieve this, aim for every pair of high and low proximity samples to have the same confidence levels. However, there is indeed an observable difference in the average confidence of Bи and BL, with high proximity samples Bu having higher average confidence. This leads to certain samples that do not have a counterpart with equivalent confidence in the other proximity group. To address this issue, we employ nearest neighbor search combined with a rejection policy. Specifically, pairs with a confidence difference greater than 0.05 are discarded. This ensures that we only pairing samples from different proximity groups that have the closest possible confidence levels to one another. When implementing this algorithm, users can also visualize the confidence distribution verify whether the confidence of two proximity groups have evident overlapping. If their confidence levels have no overlap, we suggest reducing the number of splits from 5 to 3 to ensure low/high proximity groups have similar confidence but different proximity.
Details on KDE For its simplicity and efffcttiveness, we use the KDEMultivariate function from the statsmodel library for density estimation. This function employs a Gaussian Kernel and applies the normal reference rule of thumb (i.e. bw=1.06őn 1) based on the the standard deviation ô and sample size n to select an appropriate bandwidth. While it possible to use other density estimation kernels such as Exponential Kernel in Scikit Learn, we found that the Gaussian kernel coupled with
18

---

1.0
0.8
Accuracy
0.6
0.4
0.2
train
val
0.0
0
1
2
3
4
5
8 9
Proximity
Figure 4: The model's accuracy difference between the training and validation set is more significant on low proximity samples (31.67%) compared to high proximity samples (0.6%). The discrepancy in accuracy between the training and validation sets increases as the samples approach low proximity regions, despite the training dataset and validation set have overlapping proximity distributions.
the normal reference rule for bandwidth selection generally yields better performance across various models and datasets.
D Additional Empirical Findings
D.1 Low proximity samples are more prone to model overfitting
To study the behavior of low proximity samples and high proximity samples, we compute their accuracy difference between training dataset and validation set. rreveals clear tendency: the model's accuracy difference between the training and validation set more significant on low proximity samples (31.67%) compared to high proximity samples (0.6%). This indicates that the model generalizes well on samples of high proximity but tends overfit on samples low proximity. The overconfidence of low proximity samples can be a consequence of the overfitting tendency, as the overfitting gap also reflects the mismatch between the model's confidence and its actual accuracy.
D.2 Proximity Bias Across A Variety of Models
Here we present the bias index several popular calibration algorithms on 504 models on ImageNet. As depicted in Figure most models have a bias index larger than 0, indicating the existence of proximity bias across a variety of models. Notably, even after applying temperature scaling, as demonstrated in Figure 6 the calibrated model confidences still exhibit proximity bias. This tendency further corroborated by Figure ( multi isotonic regression calibration) and Figure R (after ensemble temperature scaling). In stark contrast, Figure reveals the effectiveness of our proposed approach in mitigating proximity bias.
D.3
Confidence and Accuracy Are Positively Correlated With Proximity
Our initial investigation delves into the relationship between sample proximity, confidence, and accuracy across variety of deep neural network models. We observe clear trend wherein the
19

---

Model Confidence
0.6
well-calibrated
0.5
Varlous
DEIT
XCIT
0.4
CaiT
SwinV2
Bias Index
0.3
VIT
MLP Mixer
0.2
EfficientNet
MoblleNet
0.1
VGG
ResMLP
0.0
ResNet
ResNext
-0.1
RegNet
-.2
0.60
0.65
0.70
0.75
0.80
0.85
0.90
Model Acc
Figure 5: Proximity bias analysis of the model confidence on 504 public models. Each marker represents model, where marker sizes indicate model parameter numbers and different colors/shapes represent different architectures. The bias index is computed using Equation (3) (0 indicates no proximity bias).
Model Confidence + Tempearture Scaling Calibration
0.6
well-calibrated
0.5
Various
DEiT
XCIT
0.4
CalT
SwinV2
Bias Index
0.3
VIT
MLP Mixer
0.2
EfficlentNet
MobileNet
0.1
VGG
ResNext
0.0
ResMLP
ResNet
-0.1
RegNet
-022
0.60
0.65
0.70
0.75
0.80
0.85
0.90
Model Acc
Figure 6: Proximity bias analysis of the model confidence calibrated using temperature scaling on 504 public models. Each marker represents a model, where marker sizes indicate model parameter numbers and different colors/shapes represent different architectures. The bias index is computed using Equation 33 (0 indicates no proximity bias).
model's confidence and accuracy exhibit upward trajectory from low proximity samples to high proximity samples. This trend is illustrated in Figure 10
This tendency suggests that the model is less confident with samples from low proximity regions, where training samples are sparse. From perspective of distribution, as samples move towards sparse regions, they are stepping out of the main mass of the training distribution and are considered as out-of-distribution (OoD) samples. This aligns with the general expectation that out-of-distribution data points should have high uncertainties and thus, low confidence estimates.
20

---

Bias Index of Multi Isotonic Regression
0.6
well-calibrated
0.5
Varlous
DEIT
XCIT
0.4
CaiT
SwinV2
Bias Index
0.3
VIT
MLP Mixer
0.2
EfficientNet
MoblleNet
0.1
VGG
ResMLP
0.0
ResNet
ResNext
-0.1
RegNet
-0.2
0.60
0.65
0.70
0.75
0.80
0.85
0.90
Model Acc
Figure 7: Proximity bias analysis of the model confidence calibrated using multi isotontic regression on 504 public models. Each marker represents model, where marker sizes indicate model parameter numbers and different colors/shapes represent different architectures. The bias index computed using Equation 33 indicates proximity bias).
Bias Index of Ensemble Temperature Scaling
0.6
well-callbrated
0.5
Varlous
DEIT
XCIT
0.4
CalT
SwinVz
Bias Index
0.3
ViT
MLP Mixer
0.2
EfficientNet
MobileNet
0.1
VGG
ResMLP
0.0
ResNet
ResNext
-0.1
RegNet
-0.2
0.60
0.65
0.70
0.75
0.80
0.85
0.90
Model Acc
Figure 8: Proximity bias analysis of the model confidence calibrated using ensemble temperature scaling 504 public models. Each marker represents model, where marker sizes indicate model parameter numbers and different colors/shapes represent different architectures. The bias index is computed using Equation (3) (0 indicates no proximity bias).
However, Figure 10 also shows that the slopes of the accuracy and confidence change are not the same, resulting a larger miscalibration gap between low proximity and high proximity samples. Furthermore, commonly used calibration techniques such as temperature scaling and histogram binning seems not alleviate this issue. This motivated us to study how proximity information relates to calibration.
21

---

Model Confidence Our Calibration
0.6
well-calibrated
0.5
Varlous
DEIT
XCIT
0.4
CaiT
SwinV2
Bias Index
0.3
VIT
MLP Mixer
0.2
EfficientNet
MoblleNet
0.1
VGG
ResNext
0.0
ResMLP
ResNet
-0.1
RegNet
-0.2
0.60
0.65
0.70
0.75
0.80
0.85
0.90
Model Acc
Figure 9: Proximity bias analysis of the model confidence calibrated using our proposed PROCAL on 504 public models. Each marker represents model, where marker sizes indicate model parameter numbers and different colors/shapes represent different architectures. The bias index computed using Equation 3 indicates proximity bias).
Figure 10: Relations between proximity and its corresponding accuracy, confidence and calibrated confidences. The result shows that confidence and accuracy drop as the sample's proximity decreases.
E
Additional Experimental Results
E.1
Inference Efficiency
Runtime Efficiency To verify the time efficiency of our method, compare the inference time with baseline methods. The result is reported in Table 3 Compared to the confidence baseline, method, Sin-Mean-Shile exhibits slight increase of 1.17% in runtime, while Density- Ratio introduces modest overhead of 12.3%. These results demonstrate that our method incurs minimal computational overhead while achieving comparable runtime efficiency the other baseline methods. In addition, is worth noting that the cost computing proximity has been reduced due to the recent advancement in neighborhood search algorithms. In our implementation, we employ indexFlatL2 from the Meta open-sourced GPU-accelerated Faiss library [88 to calculate each sample's 's nearest neighbor. This algorithm enables us reduce the time for nearest neighbor search to approximately 0.04 ms per sample (shown in Table 3). The computation overhead beyond the neighbor search is actually quite similar isotonic regression (IR) and histogram binning (HB). which leads to the total time being roughly twice that of isotonic regression (0.04 + 0.05 x= 0.)s).
Memory Efficiency Similar to K-nearest- neaarst--eiihhoor- based methods [1 45), our method requires maintaining a held-out neighbor set for proximity computation during inference. To achieve memory efficiency, we employ these three techniques: 1) Reduce the size of the held-out neighbor set since is unnecessary to utilize the entire raw neighbor-set, such as the entire training dataset, particularly when dealing with large-scale data in practical scenarios. For instance, our experimental results on ImageNet [ are obtained using neighbor-set comprising 25,000 randomly sampled images
22

---

Table Average inference time(ms) per image on ImageNet on 10 runs using VTT/B-16@22444
model on a single Nvidia GTX 2080 Ti. * denotes our method (BIN#: Bin Mean-Shift; DEN*:
Density-Ratio Calibration). NS indicates the computational time consumed by the nearest neighbor search algorithm.
NS
Conf
TS
ETS
PTS
PTSK
HB
IR
MIR
BIN
DEN*
Time(ms)
0.04
5.60
5.60
5.64
5.61
5.66
5.63
5.65
5.84
5.70
6.29
from the validation set. 2) Leverage pre-computed feature embeddings instead of full images, such ResNext101 embeddings (d 1024), which consumed mere 229MB in experiments. 3)
Leverage memory-efficient neighbor search algorithms to further enhance memory efficiency [34 19
E.2 Effectiveness on Datasets with Balanced Class Distribution.
First, we present high-resolution figures of Figure illustrating the results of 504 models from timm 43 ImageNet. In Figure I1 our method (indicated by red color markers) consistently demonstrates the lowest calibration error across all four evaluation metrics, maintaining a consistently superior performance.
ECE
ACE
0.200
0.200
Conf
Conf
6.175
Metric Value
PTSK
Metric Value
0.125
0.125
ETS
0.100
0.100
0.075
0.075
8.053
0.000
0.60
0.65
6.70
0.75
0.80
0.85
0.90
0.60
0.65
0.70
0.75
0.80
0.85
0.50
Validation Accuracy
Validation Accuracy
MCE
PIECE
0.200
0.200
Conf
0.175
0.175
Metric Value
0.150
PTSK
PTSK
0.100
0.100
ours
ours
0.075
0.075
0.050
0.050
0.60
0.85
0.70
0.75
0,80
0.85
0.30
0.60
0.65
0.70
0.75
0.00
0.85
6.90
Validation Accuracy
Validation Accuracy
Figure 11: Calibration errors ImageNet across 504 timm models. Each dot represents the calibration results of applying a calibration method to the model confidence. Marker colors indicate different calibration algorithms used. Among all calibration algorithms, our method consistently appears at the bottom of the plot.
Second, we select four widely used ImageNet pre-trained models from the pool 504 models depicted in Figure [T We compare the performance of our proposed approach with baseline calibration algorithms, namely BeiT, MLP Mixer, ResNet50 and ViT. The detailed results are presented in Table 4 showcasing the effectiveness of our methods in mitigating proximity bias and enhancing calibration performance compared to existing calibration techniques. Additionally, even when applied to the most successful base calibration algorithm, our method achieves notable reduction calibration errors. This consistent improvement is particularly remarkable, especially in scenarios where the original calibration methods fail to enhance even worsen performance.
23

---

Table 4: Comparison calibration errors 10-2 between existing calibration methods ('base") and
proximity informeed framework ('ours"), ) ImageNet dataset. (+ means p == 0.01)
ECE
ACE
MCE L
PIECE
Model
Method
base
base
base
base
Conf
3.6137
0.8573*
3.5464
0.7205
1,5801
0.2866
4.2348
1,5379
ETS
2.1318
0.9930
2.1862
0.9023
1.2155
0,4333*
2.9592
1.5872
HB
4.8765
5.5631
6.1728
5.9747
1.833
4,1239
7.2174
6.2886
BeiT
MIR
0.4509
0.536
0.5376
0.5455
0.1065
0.1395
1.8039
1.2645t
PTS
1.2685
1.2744
0.8106
0.4858
0.4240
1.9782
1.6890
PTSK
1.8661
1,0288
1.9150
0.8582
0.7934
0.5022
2.7093
1.6168
TS
2.9894
1.3277
3.1132
1.2493
0.7388
0.7046
3,5366
1.9264
Conf
10.9366
2.8498
10.9337
2.7162
5.1519
1.0944*
11.0164
3.6485
ETS
1.9881
1.3586
2.1859
1.2210
0.3204
0.2986
4,1034
2.2010
9.1774
6.7207*
9.7965
7.4825*
3.5964
4,3997
12.9240
7.8672
Mixer
MIR
1.1128
0.9272
1.2190
0.9360
0.2628
0.1430
3.3912
2.2112
PTS
5.7741
2.2208
5.8027
2.1298
2.8498
0.6940
9.3215
3.0623
PTSK
6.6610
1.9466
6.6173
1.7905
3.1011
0.6131
8,3148
2.7503*
5.1937
1.6499*
5.0234
1.4455
2.0189
0.4255
5,8958
2.4809
Conf
8.7246
2.7752
8.6852
2.6344
4.6122
1.3005
8.9113
3,4224
2.7620
1.6548
3.6581
1.6677
0.6624
3,5750
2.4579
HB
7.6311
6.1812
9.3289
7.5380
2.6377
4.3484
10.1849
7.7372
ResNet50
MIR
1.0281
0,9533
0.9643
0.8436
0,2062
0.2095
1.9751
1.8776"
PTS
2.2196
1.0188
2.2098
10278
0.6319
0.2529
4.0331
2.0445
PTSK
4.4100
1.601
4.3761
1.5015
1.8375
0.4980
5.4413
2.5000
5.1181
1.7944
5.0864
1.7300
2.5503
0.6881
5,4640
2.5721
Conf
1,1815
0.9016
1.1839
0.7554
0.3489
0.2543
1.9984
1.7540*
ETS
1.1080
0.9074
1.1791
0.7732
0.2745
0.2684
1.9273
1.7533
HIB
4.6920
6.8776
7.1771
7.5824
2,4332
4,8925
7.3239
7.7552
VIT
MIR
0.8934
0.8638
0.8180
0.8537
0.1893
0.2182
1.7695
L6818
1.0609
0.7940
1.0257
0.7124
0.3937
0.2600
2.0929
1.7510
PTSK
1.6493
0.8966
1.5874
0.8139
0.6035
0.2957
2.5798
1.8211
TS
1.4905
0.9047
1.4465
0.7880
0,5069
0.2806
2.1462
1.7562
Third, we present the outcomes of our approach on the Natural Language Understanding task, specifically on the MultiNLI Match dataset, as displayed in Table The results demonstrate that our method can improve confidence calibration performance in balanced datasets and achieve comparable performance addressing proximity bias.
Table 5: Results of MultiNLI Match dataset on RoBERTa base Model that fine-tuned on MultiNLI Match. 'Base' refers existing calibration methods and 'Ours' refers method applied to existing calibration methods. Calibration error is given by x x10-2
ECE 1
ACE +
MCE
PIECE 1
Method
base
+ours
base
+ours
base
+ours
base
+ours
Conf
2.39
1.71
2.68
2.03
0.54
0.36
4.07
3.70
TS
1,68
1.53
1.96
1.86
0.49
0.40
4.19
3.45
ETS
1.88
1.57
2.08
1.76
0.70
0.44
3.97
3.61
PTS
9.28
3.48
9.25
3.49
5.86
1.13
9.44
5.29
PTSK
11.94
6.10
11.91
6.15
10.24
4.03
12.23
6,80
HB
2.07
1.80
2.10
2.10
1.03
0.67
4.63
3.67
IR 1.30
1.23
1.71
1.54
0.30
0.45
3.70
3.66
MIR
1.02
1.04
1.22
1.22
0.34
0.32
3.73
3.35
E.3
Effectiveness on Datasets with the Long-tail Class Distribution.
To evaluate our method in large-scale long-tail datasets, we conduct experiments on long-tail datasets ImageNet-LT, and iNaturalist 2021. showing that our algorithm improves upon the original calibration algorithms in most cases under all four evaluation metrics, particularly on ECE, ACE, and PIECE. This suggests that our algorithm
shows our method's performance on ImageNet-LTT

---

Table Performance of our proposed framework against base methods on ImageNet-LT using the pretrained ResNet50 model with classifier re- re-training techniques (20). The symbol * denotes that the method is significantly better than the other one with a confidence level least 90%.
ECE 4
MCE J
ACE
PIECE
Method base
+ours
base
+ours
base
+ours
base
+OUTS
conf
7.4933
1.8724*
0.8672
0.3380
7.4610
2.0168*
7.9229
3,9776
TS
2.1965
1.8541
0.3100
0.3956
2.0580
1.7472
3.8407
3,8478
ETS
2.2704
1.8642
0.3264
0.3926
2.0648
1.7622
4.0024
3.8520
PTS
5.0023
1.7300*
0.6512
0.3160
4.9927
1.7401
7.6502
3.9018
PTSK
9.1862
1.8558
1.7478
0.3475
9.2080
1.9875
11.4242
3.9175
НB
13.4644
12.9257
7.5672
7.5670
13.5931
13.4697
16.2915
15.3259#
IR
8.0541
8.5778
1.0642
1.0508
7.9947
8,5546
8.8631
9.3678
MIR
1.6207
1.5063
0.3205
0.3217
1.6562
1.5310
3.7481
3.6285
effectively mitigate the bias towards low proximity samples (i.e. tail classes), highlighting its practicality in real-world scenarios where data is often imbalanced.
E.4 Effectiveness on Datasets with Distribution Shift
To further evaluate the effectiveness of our proposed method in handling distribution shifts, conduct experiments on the ImageNet Corruption dataset [4]. The base models are trained on the ImageNet dataset without any distribution shifts and are subsequently tested the corrupted dataset. The results, shown Figure [2] indicate that our method is able to consistently improve the performance of the original model even in the presence of distribution shifts. This is particularly notable in the case of data with distribution shifts that tend to fall the low proximity region, where current algorithms tend be poorly calibrated, but our method effectively addresses this problem.
framework can futther reduce when apolied existigg calioration goritiim..
P15K
Enee
2.0
E15
Cant
1.00
Figure 12: The calibration еттог reduction in PIECE aachieved by integrating our method with existing calibration algorithms. Different colors indicate different base calibration algorithms. Each color represents a different base calibration algorithm. The bar indicates the difference in calibration error between the base algorithm and the one enhanced by our approach.
To explore the issue of domain shift, where test data has a shifted distribution not seen in training, we conduct experiments using ImageNet (training set) and ImageNet-Sketch (test set). The datasets chosen because all ImageNet images are real-world photos, while all ImageNet-Sketch images are sketches, collected using Google Search with class label keywords and "sketch". similar to the case of skin color example provided in the introduction. We employ a Vision Transformer backbone from TIMM, trained the ImageNet. Then train our ProCal using the validation from ImageNet and tested on the 50,000 images from ImageNet-Sken -Seechh The result shown in Table shows that ProCal effectively improves upon existing algorithm in many cases. While we observe a slight increase in ECE, ACE, and MCE when ProCal is paired with with PTS and PTSK, this is probably attributed to original methods suffering from cancellation effect, where positive and negative calibration errors within the same confidence bin cancel out each other (see section 6.1). Under the PIECE metric that captures the cancellation effect, our method consistently outperforms all methods by large margins and effectively mitigates their proximity bias.
25

---

Table 7: Calibration performance of ViT-Large (vit Bai__lage__achh__ooo (33)) from timm 4 on ImageNet-Sketch dataset. Calibrators trained using ImageNet validation set and tested on ImageNet-Sketciee 'base' refers to the methods, 'tours' shows the performance after integrating our method. Note that "Conf+Ours' shows the result of our method applied directly model confidence.
Calibration error is given by x x10-2.
ECE L
ACE +
MCE
+
PIECE +
Method
base
+ours
base
+ours
base
+ours
base
+ours
Conf
3.91
1.96
3.92
1.92
0.95
0.35
6.33
2.92
TS
7.60
2.38
7.57
2.38
1.42
0.57
7.95
3.27
ETS
3.16
2.07
3.22
2.03
0.37
0.39
4.92
2.97
PTS
1.62
1.60
1.65
1.66
0.34
0.25
3.89
2.74
PTSK
3.14
1.15
3.16
1.11
0.96
0.13
5.71
2.58
MIR
0.28
1.22
0.23
1.19
0.12
0.22
4.24
2.73
F
Ablation Study
F1
Comparison between Pensity-Ratio and Bin-Mean-shift.
Our comparison of Density-Ratio and Bin-Mean-shift reveals that Density-Ratin performs better in scaling-based methods while Bin-Mean-SHe Shiff demonstrates general robustness and adaptability in both continuous and discrete settings. This can be attributed to the fact that Density- -aatio can be thought of as an infinite binning-based method, enjoying good expressiveness. but it relies on density estimation which may not be as accurate when dealing with discrete outputs. In contrast, Bin-Mean-shift does not make any assumptions about the output. Therefore, it is important to note that both techniques have their own strengths and weaknesses, and the choice of which one to use should be based on the specific task at hand.
F.2 Hyperparameter Sensitivity
In this section, we evaluate the sensitivity of our method various hyperparameter choices. Specifi- cally, we examine the impact of the choice of distance metric and the number of neighbors in the local neighborhood. To evaluate the performance of method under different hyperparameter settings, we use ResNet50 [12] as our base model and compare its calibration performance of the integration of our method and existing popular claibration algorithms under different hyperparameters. This study aims to provide guideline for selecting appropriate hyperparameters when using our method.
Effect of Neighbor Size K In this study, examine the influence of the number of neighbors (K) on the performance of our proposed method. We assess the performance by varying the number of neighbors from K to K 1000 and comparing the results. Figure 13 illustraates the impact of neighbor size K on performance. The results reveal V-shaped relationship between the number of neighbors and performance. Initially, increase in the number of neighbors from yields increasing performance improvement. However, when the number of neighbors exceeds a certain threshold (e.g. K > 50), performance begins to deteriorate as the neighborhood becomes more global rather than local, eventually reaching saturation point. Notably, small neighborhood size of K = 10 sufficient to capture the local neighborhood, and further increasing the neighborhood size does not yield additional benefits. This finding aligns with previous works 17 45 that demonstrate the insensitivity of proximity to the choice of K. Based on these findings, we recommend employing a moderate range of neighbors, specifically between 10 and 50, to achieve optimal performance.
Effect of Distance Measure In this study, we explore the impact different distance measures on the performance of our method. We compare the performance under four distance measures: L2 (Euclidean distance), cosine similarity, IVFFlat, and IVFPQ. L2 distance and cosine similarity are widely used measure in machine learning. IVFFlat and IVFPQ are approximate nearest- neighor search methods implemented using the faiss library
[8] IVFFlat memory-efficien and suitable for high-dimensional datasets, while IVFPQ is optimized for datasets with large number of points and high-dimensional features. The results are depicted Figure Overall, we observe minimal performance differences across the various distance measures. Specifically, the use of cosine similarity
26

---

Effect Neighber Size
Effect of Neighber Size
Conf
0.08
Conf
ETS
0.05
0.06
1
0.04
PTS
0.04
PTSK
0.02
0.02
0.01
1000
400
1000
* neighboors under
of neighbors under ACE
Effect Neighbor Size
Effect of Neighber
0.025
Cont
0.08
Cont
0.020
HB
HB
0.06
MCE
0.015
PTS
PTS
T5
0.005
0.02
800
400
1000
of neighbers under MCE
of neighbors under PIECE
Figure 13: Hyperparameter sensitivity of the number of neighbors used in computing proximity. The results reveal a V-shaped relationship between the number of neighbors and performance. Initially, an increase in the number of neighbors from I yields increasing performance improvement. However, when the number of neighbors exceeds a certain threshold (e.g. K >> 50), performance begins to deteriorate.
and 1.2 distance yields comparable performance across the four calibration metrics. Additionally, employing IVFFlat results in slightly smaller calibration errors. However, the performance disparity is not significant. Different distance measures, including the approximation methods, offer similar performance improvements. This is because the density function for proximity values is smooth, and small measurement noise does not significantly impact the final density estimation. Considering efficiency, we recommend utilizing IVFFlat due to its favorable efficiency characteristics. However, it is important to note that the choice of the best distance measure depends the specific problem and dataset, as each measure may exhibit varying performance in different scenarios.
G Pseudo-codes
We present the procedural steps of approach in the form of pseudocode. Algorithm encom- passes the general inference phase:
Algorithm Inference procedure.
Require:
Test sample E RR", held-out embeddings E E Nxx, classifier f' (calibrated or un- calibrated), number of nearest neighbors K, nearest neighbor search algorithm S using Faiss library, PROCAI calibrator C
1: procedure INFERENCE(x)
2: Ew, - f(x)
>> get feature embedding, prediction and confidence
3: d + S.searchhi K)
4: dg f -
proximity the average distance to K nearest neighbors D return calibrated confidence
5: return Clp, dz)
6: end procedure
Algorithm Denaapsulates the Density-Ratio Calibration algorithm.
27

---

Effect of Distance Measure: ECE
Effect of Distance Measure: ACE
0.08
0.06
Conf
Conf
ETS
ETS
HB
HB
0.06
IR
0.04
MIR
MIR
PTS
0.04
PTS
0.03
PTSK
PTSK
T5
T5
0.02
0.02
IVFFlat
IVFPQ
L2
IVFFlat
IVFPQ
cosine
Effect of Distance Measure: MCE
Effect of Distance Measure: PIECE
0.025
Conf
0.08
Conf
ETS
ETS
HB
0.020
IR
0.06
MIR
MIR
0.015
PTS
PTSK
PTSK
0.010
0.04
TS
TS
0.005
0.02
IVFFlat
IVFPQ
cosine
IVFFlat
IVFPQ
cosine
Figure 14: Hyperparameter sensitivity of several proximity measure under four evaluation metrics.
Algorithm 2 Density-Rand Ratio Calibration.
Require: Pre-trained model M, validation set with pre-computed proximity Deal {X D), test set with pre-computed proximity Diest {Nestt Vrest: Drest).
1: procedure DENSITYRATIOCALIIR
2: D* vol = Dr
3: for i = 1, 1,..., X' do
4:
jh.Pi Pi <- (Tx), 24 X
get predicted class label and confidence split D based prediction correctness
5: if
y then
6:
D+ val - Dt val U {< Pi, pi, d4 >}
7:
else
8:
D + D Diat U {< Ps. de >
9:
end if
10:
end for
11: KDE +- (DE
2-dimension KDE given confidence and proximity
KDE +- KDE(D val)
D
13:
Y
D
14:
for
j = I,..., Nrest do
15:
Új. Pý M (x;), I, E Xiest KDE (D,,ss)
16:
8, KDE (XKEE (D,,ps)
> compute re-calibrated score for test sample
end for
18:
return 8,.j = 1,..., [eestl
end procedure
28


Reviews:
[{"review_id": "nqR9AWzh4i", "reviewer": "Reviewer_Wiui", "summary": "This work addresses the problem of proximity bias and confidence calibration by performing a comprehensive empirical study of various pretrained ImageNet models. The empirical findings provide insights on persistence of proximity bias even after performing calibration using existing post-hoc calibration algorithms. To mitiagte proximity bias and improve confidence calibration based on sample proximity, the paper proposes PROCAL algorithm that can be used as a plug-and-play method combined with existing calibration approaches. Further, proximity-informed expected calibration error metric is introduced to quantify the effectiveness of calibration algorithms in mitigating proximity bias. ", "questions": "- Are the iNaturalist 2021 results in Table 1 based on transfer learning from pretrained ImageNet models, or directly evaluated on pretrained backbone?\n- What are the criteria to choose between density-ratio calibration versus Bin-Mean-shift approach?\n- typo in line#309 ImageNet-TL --> ImageNet-LT", "limitations": "The authors have addressed the limitations of their work in the manuscript.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A comprehensive study of pretrained ImageNet models involving various neural network architectures on their model calibration and proximity bias evaluation. The empirical study is performed on image classification tasks (under balanced, long-tail, and distribution-shift settings) and text classification tasks. \n- Experimental evaluation if thorough using various calibration metrics. \n- The paper provides many interesting observations from the empirical study related to proximity bias and model calibration, which is an important area of study under long-tailed data distribution settings.", "weaknesses": "- The presentation of experimental results and writeup of Experiments Section 6 can be improvised, the findings for many of the questions are pointed to the Appendix without any brief details in the manuscript. I understand this is due to page limitation, but I would suggest the authors to focus on the results that are presented in main manuscript, or include the brief info in the manuscript at least.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewer Wiui for the constructive suggestions on the writing of the experiment part. We are also glad that the reviewer finds the phenomenon study comprehensive, the experiment evaluation thorough, the observation interesting, and the issue we identify to be important. Here we answer all the questions and hope they can address the concerns.\n\n**Q1: Are the iNaturalist 2021 results in Table 1 based on transfer learning from pretrained ImageNet models, or directly evaluated on pretrained backbone?**\n\nThe results are directly evaluated on the **pretrained backbone** which we follow this paper[2] and download from the repo: https://github.com/visipedia/newt/blob/main/benchmark/README.md\n\n[2] Van Horn G, Cole E, Beery S, Wilber K, Belongie S, Mac Aodha O. Benchmarking representation learning for natural world image collections. CVPR 2021.\n\n \n**Q2: What are the criteria to choose between density-ratio calibration versus Bin-Mean-shift approach?**\n\nThanks for your constructive question. First, we would like to clarify that existing calibration algorithms can be classified into 2 types: \n\n1) those producing *continuous outputs* such as softmax, temperature scaling (TS), and other methods (ETS, PTS, PTSK) which scale the input scores in a continuous way; and \n2) those producing *discrete outputs*, particularly binning or isotonic regression methods (HB, IR, MIR) which group the samples and output the exact same scores for samples in the same group. \n\nOur density-ratio calibration works by estimating continuous density functions, and is suited for the 1st case when the input confidence scores are continuous. In contrast, our bin-mean-shift method does not rely on densities and is more suited for utilizing outputs from the 2nd category of methods, i.e. discrete outputs. In combination, our 2 calibration approaches form a plug-and-play framework that is applicable to confidence scores of either type. We will revise the paper to clarify this.\n\n**W1: The presentation of experimental results and writeup of Experiments Section 6 can be improved, the findings for many of the questions are pointed to the Appendix without any brief details in the manuscript. I understand this is due to page limitation, but I would suggest the authors focus on the results that are presented in the main manuscript, or include the brief info in the manuscript at least.**\n\nWe appreciate your suggestions, and we will include a brief summary of the results when making reference to the appendix to improve the readability and flow of the paper in the future versions.\n"}, {"review_id": "BnFBSV2l3B", "reviewer": "Reviewer_1D75", "summary": "The article focuses on the problem of uncertainty quantification in classification.\nCalibration provides some guarantees on the estimated class probabilities on average. However, subgroups can still be miscalibrated. The article first aims to characterize these subgroup miscalibrations through proximity levels of the samples. It claims that a classifier, even calibrated, tends to be underconfident on high-proximity samples and overconfident on low-proximity samples. To measure this effect, it defines a proximity-informed ECE. Then, it proposes a recalibration framework based on this proximity-informed measure. Finally, it benchmarks the proposed method on numerous datasets and models.", "questions": "* In equation (5) on the confidence score adjustments, why a weight hyperparameter $\\lambda \\in (0, 1]$ is needed? Don't we want to take $\\lambda=1$ to minimize PIECE? The paper talks quickly about regularization. What is the tradeoff? Are results wrong with $\\lambda$ too close to 1? Do you take $\\lambda=0.5$ everywhere in the experiments? How to choose it?\n* Concerning the time overhead introduced by the recalibration methods, isn't it weird that Bin-Mean-Shift has only twice the overhead of isotonic regression (+0.1s vs +0.05s)? Since isotonic regression just needs to apply the mapping function from [0, 1] to [0, 1] to the inferred sample, while Bin-Mean-Shift needs to compute the distance from the new sample to all samples from the calibration set. The same question applies to mapping-based recalibration methods such as histogram binning (+0.03s).\n* Could you develop when to use the histogram-based version (section 5.2) and when to use the continuous version (section 5.1) of your method?\n\n\n---\n\n\nTypos:\n* L148: it should be 80% instead of 85%. \n* L162: repeated word Appendix.", "limitations": "The authors discuss the following limitations:\n* The proposed recalibration technique needs to maintain the calibration set during inference to compute the proximity of the new points.\n* Maintaining the calibration set for inference may challenge the method when memory is limited.\n* Focus limited to the closed-set multi-class classification problem.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "* The problem is well presented and motivated. Figure 1 is pedagogical and helps the comprehension of the problem.\n* The idea of characterizing subgroup miscalibrations through proximity is interesting, and refining uncertainty estimates is a good direction.\n* The proposed framework is versatile: it can work as a stand-alone or combined with standard calibration techniques. It provides two versions: binning-based and continuous.\n* The experimental study is substantial.\n  * Datasets are large-scale, numerous, and multimodal: ImageNet, Yahoo-Topics, iNaturalist, ImageNet-LT, MultiNLI, ImageNet-C.\n  * The article studies numerous models, e.g. 504 pre-trained models on ImageNet.\n  * It compares many standard calibration methods, both scaling-based and histogram-based: temperature scaling, ensemble temperature scaling, parameterized temperature scaling, histogram binning, isotonic regression, and multi-isotonic regression.\n* The experimental study provides substantial evidence.\n  * It reveals proximity bias in most of the 504 pre-trained networks on ImageNet (72% according to a Wilcoxon rank-sum test).\n  * The proposed method consistently improves over standard calibration methods.\n  * The time overhead of the method is small, with an increase of 1.17% in inference runtime.\n* Completeness of the study: It reveals the proximity bias, proposes a metric to measure it, a recalibration procedure to address it, and substantial experiments showing consistent improvements.", "weaknesses": "No major weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewer 1D75 for the constructive comments and we are glad that the reviewer finds the problem interesting, the solution a good direction, and the study complete. Here we answer all the questions and hope they can address the concerns.\n\n**Q1: In equation (5) on the confidence score adjustments, why a weight hyperparameter \\lambda is needed? Don't we want to take \\lambda=1 to minimize PIECE? What is the tradeoff? Are results wrong with  too close to 1? Do you take \\lambda=0.5 everywhere in the experiments? How to choose it?**\n\nTo start, we revisit equation (5): $\\hat{P}_{\\text {ours }}=\\hat{P}+\\lambda \\cdot\\left(\\mathcal{A}\\left(B\\_{m h}\\right)-\\mathcal{F}\\left(B\\_{m h}\\right)\\right)$.\n\nFirst, we agree with you that setting $\\lambda=1$ in Equation (5) would ideally calibrate the model and minimize PIECE more effectively. However, in practice, we often encounter bins with a smaller number of samples, whose estimate will have high variance. In such cases, the calculated $F(B_{mh}) -A(B_{mh})$ can be quite inaccurate as the estimate of the corresponding population quantity. To reduce variance in these scenarios, we introduce a **shrinkage coefficient, $\\lambda$**. By setting a smaller $\\lambda$, we can reduce the variance and make the final prediction more accurate at the expense of biased estimator. \n\nTherefore, the use of $\\lambda$ can be seen as a form of bias-variance trade-off. By adjusting the value of $\\lambda$, we are able to control the balance between bias and variance in the model's output. In practice, we choose $\\lambda=0.5$ as a reasonable default for all our experiments, which we find offers consistent performance across various settings.\n\n**Q2: Why Bin-Mean-Shift has only twice the overhead of isotonic regression (+0.1s vs +0.05s)? Isotonic regression just needs to apply the mapping function from [0, 1] to [0, 1] to the inferred sample, while Bin-Mean-Shift needs to compute the distance from the new sample to all samples from the calibration set. The same question applies to mapping-based recalibration methods such as histogram binning (+0.03s).**\n\nWe attribute our efficiency to the recent advances in neighborhood search algorithms. \nIn our paper, we employ `indexFlatL2` from the Meta open-sourced GPU-accelerated Faiss library[2] to calculate each sample's nearest neighbor. This algorithm enables us to **reduce the time for nearest neighbor search to approximately 0.04 ms per sample** (shown below). The computation overhead beyond the neighbor search is actually quite similar to isotonic regression, which leads to the total time being roughly twice that of isotonic regression **(0.04 + 0.05 ≈ 0.1s)**.\n\nFor comparison, here's the inference time comparison (ms per sample) for our method and baseline methods on ImageNet (using a ViT/B-16@224px model on a single Nvidia GTX 2080 Ti, with all settings following those in our paper, and averaged across 10 runs):\n|                  |  NS   |Conf  | TS | HB | IR | PTS | PTSK  |ETS| MIR| BIN* | DEN* | \n|:---------------------:|:-----:|:---------:|:---------------------------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n|  Time(ms) |0.04   |5.60|  5.60 | 5.63| 5.65 | 5.61 |5.66 |5.64|  5.84  | 5.70 | 6.29| \n\nNS represents the time consumption for **neighbor search**. BIN* and DEN* are our proposed methods, where BIN denotes Bin Mean-Shift for discrete output and DEN is Density-Ratio Calibration for continuous output. \n\n**Q3: Could you develop when to use the histogram-based version (section 5.2) and when to use the continuous version (section 5.1) of your method?**\n\nThanks for your constructive question. First, we would like to clarify that existing calibration algorithms can be classified into 2 types: \n1) those producing **continuous outputs** such as softmax, temperature scaling (TS), and other methods (ETS, PTS, PTSK) which scale the input scores in a continuous way; and \n2) those producing **discrete outputs**, particularly binning or isotonic regression methods (HB, IR, MIR) which group the samples and output the exact same scores for samples in the same group.\n\nOur density-ratio calibration works by estimating continuous density functions, and is suited for the 1st case when the input confidence scores are continuous. In contrast, our bin-mean-shift method does not rely on densities and is more suited for utilizing outputs from the 2nd category of methods, i.e. discrete outputs. In combination, our 2 calibration approaches form a plug-and-play framework that is applicable to confidence scores of either type. We will revise the paper to clarify this.\n"}, {"review_id": "C5EMX85gTb", "reviewer": "Reviewer_4gc8", "summary": "This paper studies the prevalence of proximity bias in calibration, i.e. the rate of miscalibration on samples that are far away from their nearest neighbors in the data (\"low proximity\"). The authors empirically show that this type of miscalibration is present across many models, and propose a new post-training calibration procedure for mitigating it. Their approach shows significant empirical improvements over standard post-training calibration approaches.", "questions": "The main questions I have are detailed in weaknesses above.", "limitations": "The authors appropriately discuss limitations, but it might be helpful to include some bits from the appendix (particularly regarding efficiency) in the main body.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- **Originality:** The empirical investigation and proposed methodology in the paper is quite novel, as I am not aware of prior work that has studied this type of bias in calibration (although others have studied subgroup calibration). \n- **Quality:** The claims in the paper are technically sound, and the experiments exploring proximity bias are extensive (more than 500 pretrained models considered, and various calibration baselines).\n- **Clarity:** Overall, the paper is well-written and well-organized, with motivating experiments and intuitive definitions. However, I believe the presentation of the calibration algorithm in the paper could be improved (detailed further in weaknesses).\n- **Significance:** The idea of low proximity samples introduced and analyzed by the authors seems quite significant, as these samples can correspond to underrepresented populations in the data.", "weaknesses": "- **Algorithm Details:** The weakest part of the paper in my view is the lack of detail in Section 5.1. This part of the paper would be significantly improved by including something akin to the pseudo-code algorithm in the appendix. There are several questions that arise when reading this part: what does one do after estimating the posterior probability conditional on prediction and proximity (in the algorithm I can see that this is just the output on the test point)? Do we compute proximity only with respect to the test data? What type of KDE is used (i.e. kernel, bandwidth, etc.)? \n- **Implementation Details:** In addition to algorithm details, some parts of the experimental setup could also be made clearer. How are ECE and ACE computed (i.e. binning scheme)? Do you set aside calibration data for the scaling methods (TS, ETS, PTS, PTSK) in addition to the set aside data for ProCal?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewer 4gc8 for the constructive feedback and we are glad that the reviewer finds the investigation and methodology novel, the claim technically sound, the experiment extensive, and the proximity bias issue significant. Here we answer all the questions and hope they can address the concerns.\n\n---\n\n\n**W1: The weakest part of the paper in my view is the lack of detail in Section 5.1. This paper would be significantly improved by including something akin to the pseudo-code algorithm in the appendix.**\n\nWe appreciate your constructive feedback. We will incorporate the pseudo-code algorithms in Section G into the main paper in our forthcoming revision.\n\n\n**Q1: what does one do after estimating the posterior probability conditional on prediction and proximity (in the algorithm I can see that this is just the output on the test point)?**\n\nThank you for the question. In fact, we directly treat the estimated posterior probability $\\mathbb{P}(\\hat{Y}=Y \\mid \\hat{P}, D)$ as the calibrated confidence. The rationale for this is: \n\n1) The common interpretation of confidence is the probability of the prediction being correct given the sample X, and our posterior probability serves as an estimate of this $\\mathbb{P}(\\hat{Y}=Y \\mid X)$ by using $\\hat{P}, D$ as the proxy of sample X (line 206-210) ; \n\n2) The commonly-used confidence is $\\hat{P}$, while the posterior can be regarded as an updated or modified version of the initial $\\hat{P}$. \n\n\n**Q2: Do we compute proximity only with respect to the test data?**\n\nIf we understand the question correctly, the reviewers is asking (please let us know if you meant the question in a different way): *given a test sample, if the proximity is computed based on the test set*. To address this, we clarify that the proximity for any test sample is computed using the **validation set**. Here are the details:\n\n1) During inference, when calibrating a test point, we only need to compute this point’s proximity by finding its $K$ nearest neighbors in the held-out **validation** set and no additional proximity computation is required.\n\n2) When training the calibration approach (using the validation set for training), we calculate every point's proximity to others within the validation set to estimate their proximity values, which are then used for training the density estimators for Density-Ratio Calibration and the binning parameters in Bin-Mean-Shift. Note that each point is excluded from its own neighbor search. \n\n\n**Q3: What type of KDE is used (i.e. kernel, bandwidth, etc.)?**\n\nFor its simplicity and effectiveness, we use the `KDEMultivariate` function from the `statsmodel` library for density estimation. This function employs a Gaussian Kernel and applies the normal reference rule of thumb (i.e. bw=$1.06\\hat{\\sigma} n^{-1 / 5}$) based on the the standard deviation $\\hat{\\sigma}$ and sample size $n$ to select an appropriate bandwidth. While it is possible to use other density estimation kernels such as `Exponential Kernel` in `Scikit Learn`, we found that the Gaussian kernel coupled with the normal reference rule for bandwidth selection generally yields better performance across various models and datasets.\n\n\n**Q4: How are ECE and ACE computed (i.e. binning scheme)?**\n\nWe follow [1] to implement Expected Calibration Error (ECE) and Adaptive Calibration Error (ACE): \n1) Firstly, we divide samples into 15 bins and compute every bin’s average confidence and accuracy. \n2) Next, we compute the absolute difference between each bin's average confidence and its corresponding accuracy. \n3) The final calibration error is measured using the weighted difference (the fraction of samples in each bin as the weight). \n\nThe key distinction between ECE and ACE lies in the binning scheme: ECE divides bins with **equal-confidence intervals** while ACE uses an adaptive scheme that spaces the bin intervals to contain an **equal number of samples** in each bin. \n\n[1]Nixon J, Dusenberry MW, Zhang L, Jerfel G, Tran D. Measuring Calibration in Deep Learning. InCVPR workshops 2019 Jun 16 (Vol. 2, No. 7).\n\n**Q5: Do you set aside calibration data for the scaling methods (TS, ETS, PTS, PTSK) in addition to the set aside data for ProCal?**\n\nIn our implementation, we **do not reserve additional data** specifically for ProCal. Instead, we employ the same validation set used to train other calibration methods (e.g. TS). We designed it this way considering the typical constraints of validation set data availability and the cost associated with acquiring extra data. However, if sufficient validation set points are available, we do recommend setting aside separate calibration data for ProCal in case that other calibration methods might overfit their validation sets.\n"}, {"review_id": "ItiswZ9HcA", "reviewer": "Reviewer_4HP9", "summary": "This paper quantifies and proposes a mitigation for a phenomenon in DNN training, where more 'unusual' examples (here defined as having a higher average distance to its K=10 nearest neighbors) are generally more miscalibrated across a range of models and tasks. The authors propose a new proximity-aware calibration metric, PIECE, and demonstrate that it can capture calibration issues that are not necessarily captured by the standard ECE metric (and, in fact, always P>= ECE). Additionally, the authors propose a mitigation, ProCal. This mitigation is presented in two variations, one for continuous and one for discrete confidence. These work by adjusting the uncalibrated probability score based on the model's miscalibration on examples with that average distance.", "questions": "Please see the weakness section above. In addition:\n\n* For cases where the test distribution doesn't match the train distribution, how much are we losing by using the test data to estimate values of D (versus measuring the distance of a test sample point to the closest 10 training points)? It would seem that the training distribution is actually the important one for estimating atypicality.\n* How does the Bias Index changed if only $B_H$ is used for sampling points and $B_L$ is used for matching, and vice versa? What if the intermediate groups are used?\n* Is it possible to characterize the $B_H$ and $B_L$ groups? Is there a large difference of average confidence between the two, and how well do their confidences intersect?\n* Why is ProCAL less effective in conjunction with IR and MIR?\n", "limitations": "The limitations are adequately addressed.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is very clearly written, with a logical flow and good explanations of all steps taken. I particularly enjoyed the motivation via experiments on existing models leading to theory-driven proposals leading to further experimental verification. I also appreciated the through ablation study and the extra attention given to OOD examples.\n\nIn addition, the original miscalibration problem that the paper brings up seems to me to be important and relevant, and is well-motivated. As I am not an expert in miscalibration mitigation techniques I cannot fully comment on the novelty aspect of the work, though I have not seen this 'atypicality' concern brought up explicitly before; it also dovetails neatly with other research in atypical examples.\n\nThe proposed metric and mitigations, while very simple, are logical and effective, which I hope will lead to their adoption.\n\n", "weaknesses": "I felt that the 'atypical' (high $D(X)$) examples could have been better characterized. In particular, the authors relate these examples to underrepresented categories in datasets (eg, Black people in health datasets), but this is not necessarily so. It does not seem incredible to have a scenario where a data is divided into two clusters, where the smallest cluster is nevertheless very tight, and so has a smaller average $D(X)$.\n\nIt is not really clear what the ProCAL method refers to. It seems like it mostly splits into two methods each of which have their own name. Maybe it would be better to call them ProCAL-C and ProCAL-D  (for continuous and discrete)?\n\nThe analysis of ProCAL effectiveness seems a little sparse in claiming \"our method consistently improves the calibration\"... My interpretation of the data would be that ProCAL is very helpful in conjunction with raw confidence as well as some of the other methods with higher PIECE scores (TS variants), but not as effective for methods such as IR and MIR, where it seems to hurt as much as help. I don't think that this disqualifies the paper, but a more thorough explanation and analysis of this would be appropriate.\n\n(minor) in section 3.1, what is the sensitivity of two points having the 'same' confidence?\n(minor) It is not clear why the definition of D(X) contains an exponent, rather than the simple average of the ten smallest distances.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate reviewer 4HP9's constructive feedback. We are glad the reviewer enjoyed the logic flow of this paper, finds the problem important and the solution simple, logical and effective.\n\n**Q1: I felt that the 'atypical' (low proximity) examples could have been better characterized. In particular, the authors relate these examples to underrepresented categories in datasets (eg, Black people in health datasets), but this is not necessarily so. It does not seem incredible to have a scenario where the data is divided into two clusters, where the smallest cluster is nevertheless very tight, and so has a smaller average.**\n\nThis is indeed a thoughtful question. First, we want to clarify that 'low proximity' in our definition refers to 'low density', but the context of 'underrepresented' here links more to 'low probability', which are correlated but not equivalent. \n\nSecond, we agree with the reviewer that our existing definition of proximity does not include samples with low probability but high density. \nIt is still an open question whether these samples underrepresented but with high proximity suffer from proximity bias (i.e., are low probability samples inherently more overconfident than high probability samples) and whether they need mitigation. We leave the investigation of these cases and a better characterization to the future work. \n\n**Q2: It is not really clear what the ProCAL method refers to. It seems like it mostly splits into two methods each of which have their own name. Maybe it would be better to call them ProCAL-C and ProCAL-D (for continuous and discrete)?**\n\nThank you for your suggestion. The intention behind the term 'ProCAL' is to represent the general strategy and a series of methods for mitigating proximity bias. Since we've tailored two specific methods for different output types, we've referred to these as Density-Ratio Calibration and Bin-Mean-Shift. We will clarify this in the paper.\n\n**Q3: why the definition of D(X) contains an exponent, rather than the simple average of the ten smallest distances.**\n\nThis is because the current definition normalizes the distance measure from a range of $[0, \\inf]$ to $(0,1]$, making the approach more robust to the effects of distance scaling since the absolute distance in Euclidean distance can cause instability and difficulty in modeling.\n\n**Q4: For cases where the test distribution doesn't match the train distribution, how much are we losing by using the test data to estimate values of D**.\n\nFirst, we want to clarify that we actually use a validation set rather than a test set for estimation atypicality (i.e., proximity), which we assume is iid drawn from the training distribution.  Besides, in most cases, including settings with distribution shift, we do not presume to have knowledge of the distribution-shifted data, and therefore we have no access to the test data.\n\nHowever, in case we can access the test dataset to estimate the proximity, how much we are losing is largely dependent on how far the test distribution deviates from the training distribution.\n\n**Q5: Why is ProCAL less effective in conjunction with IR and MIR?** My interpretation of the data would be that ProCAL is very helpful in conjunction with raw confidence as well as some of the other methods with higher PIECE scores (TS variants), but not as effective for methods such as IR and MIR, where it seems to hurt as much as help. I don't think that this disqualifies the paper, but a more thorough explanation and analysis of this would be appropriate.\n\nThere are two cases related to the question:\n1. **ProCal improves PIECE but negatively impacts ACE/ECE**:  we argue that the \"cancellation effect\" (see lines 170-181) is responsible for this. This effect emerges when positive calibration errors (i.e., acc - conf > 0) from high proximity samples and negative errors from low proximity samples offset each other within the same confidence group, causing ACE/ECE to potentially underestimate the calibration error. Binning-based methods are particularly prone to this because it shifts all samples in a bin to the same confidence, increasing the chances of the cancellation effect. Although ProCAL, which corrects each proximity bin independently, might increase ECE/ACE by disrupting this cancellation effect, a rise in ECE/ACE doesn't necessarily reflect overall calibration degradation. \n2. **ProCal worsens PIECE**: note that IR/MIR are binning-based and hence Bin-Mean-Shift is used. We speculate that the deterioration in Bin-Mean-Shift stems from the inaccuracies of each bin's estimates. When bins have few samples, their accuracy estimates can vary significantly and hence unreliable, leading to this observed performance decline.\n\n---\n\nFor questions pertaining to the computation of the bias index, we direct the reviewer to our **global response** above."}]

Author rebuttal:
**The following questions primarily pertain to the details of how to compute the bias index, and therefore, we have grouped them together for convenience and clarity.**

To provide further clarity, let's revisit the process of how we calculate the bias index: 
1. To do the hypothesis testing, we first split the samples into 5 equal-sized proximity groups and select the highest and lowest proximity groups.
2. From the high proximity group, we randomly select 10,000 points and find corresponding points in the low proximity group that have similar confidence levels. 
3. Next, we reverse this process to randomly select 10,000 points from the low proximity group and find corresponding points in the high proximity group with matched confidence.
4. We therefore merge all the points from the high proximity group into $B_H$ and those from the low proximity group into $B_L$, with the $B_H$ and $B_L$ having similar average confidence. 
5. We apply the Wilcoxon rank-sum test to evaluate whether the mean difference in their accuracy between $B_H$ and $B_L$ is significantly different from zero.

**Q4.1: what is the sensitivity of two points having the 'same' confidence?**

If we understand correctly, the reviewer is asking whether we can ensure that two points have the same confidence. Firstly, we would like to clarify that *the two samples have the **same confidence*** actually means they have the *closest confidence in the validation set with the maximum difference less than 0.05*. In practice, we apply nearest neighbor search to find points in the alternate group with approximately the same confidence (i.e. closest possible confidence) to our target. Pairs with significant confidence difference (> 0.05) are excluded, ensuring that $B_H$ and $B_L$ have comparable confidence levels.

**Q4.2:  Is there a large difference of average confidence between the two, and how well do their confidences intersect?**

If we understand correctly, 'the two' refers to the original groups with the highest and lowest proximities in step 1). 
1. In this case, there is indeed an observable difference in their average confidence, with high proximity samples having higher average confidence. Therefore, we apply nearest neighbor search and reject policy to ensure the sampled $B_H$ and $B_L$ in step 5 have almost the same confidence. 
2. As we only divide samples into 5 equally-sized groups, their confidences intersect well. When implementing this algorithm, users can also visualize the confidence distribution to verify whether overlaps occur. If their confidence levels have no overlap, we suggest reducing the number of splits from 5 to 3 to ensure low/high proximity groups have similar confidence but different proximities.

**Q4.3: How does the Bias Index changed if only B_H is used for sampling points and B_L is used for matching, and vice versa?** 

Thank you for your question. As suggested in step 2 and 3, we do not just use the $B_H$ for sampling and $B_L$ for matching; both groups are used for sampling and matching.
In addition, as we highlighted in Q4.1, we employ the nearest neighbor search and reject strategy to ensure that $B_H$ and $B_L$ have similar average confidences. This means that the samples drawn primarily fall within the range of intersecting confidences, resulting in minimal changes to the bias index even if we randomly choose one group for sampling. 

**Q4.4: What if the intermediate groups are used?**

We also conducted experiments in which we divided samples into 3 equal-sized proximity groups, which reduced the proximity difference between the high/low groups (can be seen as the proxy of intermediate groups). We found that the same observations hold true in this context. 



Meta-review: The authors examine the phenomenon of miscalibration in neural network classifiers.  They find empirically that typical ImageNet trained classifiers have significantly different levels of calibration across "low-proximity" and "high-proximity" subpopulations, where they define higher-proximity samples as those close to many other samples while low-proximity samples are far from other samples (where the distance is measured using the Euclidean distance of the features from the penultimate layer of the neural net).  They show that the standard approach for rectifying miscalibration, temperature scaling, does not remove the proximity bias of the classifiers.  On the other hand, they propose a new method, PIECE, which does remove this bias.  

The reviewers were unanimous in their support for the submission.  They were impressed by the empirical findings on the existence of proximity bias as well as the proposed method to address this bias.  I concur with the reviewers and I recommend this paper for acceptance with spotlight.
Decision: Accept (spotlight)