L OCAL C ALIBRATION : M ETRICS AND R ECALIBRATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Probabilistic classiﬁers output conﬁdence scores along with their predictions, and these conﬁdence scores should be calibrated, i.e., they should reﬂect the reliability of the prediction.
Conﬁdence scores that minimize standard metrics such as the expected calibration error (ECE) accurately measure the reliability on average across the entire population.
However, it is in general impossible to measure the reliability of an individual prediction.
In this work, we propose the local calibration error (LCE) to span the gap between average and individual reliability.
For each individual prediction, the LCE measures the average reliability of a set of similar predictions, where similarity is quantiﬁed by a kernel function on a pretrained feature space and by a binning scheme over predicted model conﬁdences.
We show theoretically that the LCE can be estimated sample-efﬁciently from data, and empirically ﬁnd that it reveals miscalibration modes that are more ﬁne-grained than the ECE can detect.
Our key result is a novel lo cal re calibration method LoRe, to improve conﬁdence scores for individual predictions and decrease the LCE.
Experimentally, we show that our recalibration method produces more accurate conﬁdence scores, which improves downstream fairness and decision making on classiﬁcation tasks with both image and tabular data.
I NTRODUCTION
Uncertainty estimation is extremely important in high stakes decision-making tasks.
For example, a patient wants to know the probability that a medical diagnosis is correct; an autonomous driving system wants to know the probability that a pedestrian is correctly identiﬁed.
Uncertainty estimates are usually achieved by predicting a probability along with each classiﬁcation.
Ideally, we want to achieve individual calibration, i.e., we want to predict the probability that each sample is misclassiﬁed.
However, each sample is observed only once for most datasets (e.g., image classiﬁcation datasets do not contain identical images), making it impossible to estimate, or even deﬁne, the probability of incorrect classiﬁcation for individual samples.
Because of this, commonly used metrics such as the expected calibration error (ECE) measure the gap between a classiﬁer’s conﬁdence and accuracy averaged across the entire dataset.
Consequently, ECE can be accurately estimated but does not measure the reliability of individual predictions.
In this work, we propose the local calibration error (LCE), a calibration metric that spans the gap between fully global (e.g., ECE) and fully individual calibration.
Motivated by the success of kernel-based locality in other ﬁelds such as fairness (where similar individuals should be treated similarly) (Dwork et al., 2012; Pleiss et al., 2017) and causal inference (where matching techniques are used to ﬁnd similar neighboring samples) (Stuart, 2010), we approximate the probability of misclassiﬁcation for an individual sample by computing the average classiﬁcation error over similar samples, where similarity is measured by a kernel function in a pre-trained feature space and a binning scheme over predicted conﬁdences.
Intuitively, two samples are similar if they are close in a pretrained feature space and have similar predicted conﬁdence scores.
By choosing the bandwidth of the kernel function, we can trade off estimation accuracy and individuality: when the bandwidth is very large, we recover existing global calibration metrics; when the bandwidth is small, we approximate individual calibration.
We choose an intermediate bandwidth, so our metric can be accurately estimated, and provides some measurement on the reliability of individual predictions.
Theoretically, we show that the LCE can be estimated with polynomially many samples if the kernel function is bounded.
Empirically, we also show that for intermediate values of the bandwidth, the
LCE can be accurately estimated and reveals modes of miscalibration that global metrics (such as
ECE) fail to uncover.
In addition, we introduce a non-parametric, post-hoc lo calized re calibration method (LoRe), for lowering the LCE.
Empirically, LoRe improves fairness by achieving low calibration error on all potentially sensitive subsets of the data, such as racial groups.
Notably, it can do so without any prior knowledge of those groups, and is more effective than global methods at this task.
In addition, our recalibration method improves decision making when there is a “safe” action that is selected whenever the predicted conﬁdence is low.
For example, an automated system which classiﬁes tissue samples as cancerous should request a human expert opinion whenever it is unsure about a classiﬁcation.
In a simulation on an image classiﬁcation dataset, we show that recalibrated prediction models more accurately choose whether to use the “safe” action, which improves the overall utility.
In summary, the contributions of our paper are as follows.
(1) We introduce a local calibration metric, the LCE, that is both easy to compute and can estimate the reliability of individual predictions.
We introduce a post-hoc localized recalibration method LoRe, that transforms a model’s conﬁdence predictions to improve the local calibration.
(3) We empirically evaluate LoRe on several downstream tasks and observe that LoRe improves fairness and decision-making more than existing baselines.
B ACKGROUND AND R ELATED W ORK 2. G LOBAL C ALIBRATION M ETRICS
Consider a classiﬁcation task that maps from some input domain (e.g., images)
X to a ﬁnite set of labels Y
= { 1 , · · · , m } .
A classiﬁer is a pair ( f, ˆ p ) where f : X → Y maps each input x ∈ X to a label y ∈ Y and ˆ p : X → [0 , 1] maps each input x to a conﬁdence value c .
Let Pr be a joint distribution on X × Y (e.g., from which training or test data pairs ( x, y ) are drawn).
The classiﬁer ( f, ˆ p ) is perfectly calibrated (Guo et al., 2017) with respect to Pr if for all c ∈ [0 , 1]
[Equation]
To numerically measure how well a classiﬁer is calibrated, the most commonly used metric is the expected calibration error (ECE) (Naeini et al., 2015; Guo et al., 2017), which measures the average absolute deviation from Eq.
1 over the domain.
In practice, given a ﬁnite dataset, the ECE is approximated by binning.
The predicted conﬁdences ˆ p are partitioned into bins B 1 , . . .
, B k , and then a weighted average is taken of the absolute difference between the average conﬁdence conf( B i ) and average accuracy acc
( B i ) for each bin B i :
[Equation]
Similarly, the maximum calibration error (MCE) (Naeini et al., 2015; Guo et al., 2017) measures the average deviation from Eq.
1 in the bin with the highest calibration error, and is deﬁned as
[Equation]
2. E XISTING G LOBAL R ECALIBRATION M ETHODS
Many existing methods apply a post-hoc adjustment that changes a model’s conﬁdence predictions to improve global calibration, including Platt scaling (Platt, 1999), temperature scaling (Guo et al., 2017), isotonic regression (Zadrozny & Elkan, 2002), and histogram binning (Zadrozny & Elkan, 2001).
These methods all learn a simple transformation from the original conﬁdence predictions to new conﬁdence predictions, and aim to decrease the expected calibration error (ECE).
Platt scaling ﬁts a logistic regression model; temperature scaling learns a single temperature parameter to rescale conﬁdence scores for all samples simultaneously; isotonic regression learns a piece-wise constant monotonic function; histogram binning partitions conﬁdence scores into bins { [0 , (cid:15) ) , [ (cid:15), 2 (cid:15) ) , · · · , [1 − (cid:15), 1] } and sorts each validation sample into a bin based on its conﬁdence ˆ p ( x ) ; it then resets the conﬁdence level of all samples in the bin to match the classiﬁcation accuracy of that bin.
2. L OCAL C ALIBRATION
Two notions of calibration that address some of the deﬁcits of global calibration are class-wise calibration and group-wise calibration.
Class-wise calibration groups samples by their true class label (Kull et al., 2019; Nixon et al., 2019) and measures the average class ECE, while group-wise calibration uses pre-speciﬁed groupings (e.g., race or gender)
(Kleinberg et al., 2016; Pleiss et al., 2017) and measures the average group-wise ECE or maximum group-wise MCE.
A few recalibration methods have been proposed for these notions of calibration as well.
Dirichlet calibration (Kull et al., 2019) achieves calibration for groups deﬁned by class labels, but does not generalize well to settings with many classes (Zhao et al., 2021).
Multicalibration (Hébert-Johnson et al., 2017) achieves calibration for any group that can be represented by a polynomial sized circuit, but lacks a tractable algorithm.
If the groups are known a priori, one can also apply global calibration methods within each group; however, this is impractical in many situations where the groups are not known for new examples at inference time.
At an even more local level, Zhao et al.
(2020) look at individual calibration in the regression setting and conclude that individual calibration is impossible to verify with a deterministic forecaster, and thus there is no general method to achieve individual calibration.
2. K ERNEL -B ASED C ALIBRATION M ETRICS
Kumar et al.
(2018) introduce the maximum mean calibration error (MMCE), a kernel-based quantity that replaces the hard binning of the standard ECE estimator with a kernel similarity k
(ˆ p ( x ) , ˆ p ( x (cid:48) )) between the conﬁdence of two examples.
They further propose to optimize the MMCE directly in order to achieve better model calibration globally.
Widmann et al.
(2019) extend their work and propose the more general kernel calibration error.
Zhang et al.
(2020) and Gupta et al.
(2020) also consider kernel-based calibration.
However, these methods only consider the similarity between model conﬁdences ˆ p ( x ) , ˆ p ( x (cid:48) ) , rather than the inputs x, x (cid:48) themselves.
T HE L OCAL C ALIBRATION E RROR
Recall that commonly used metrics for calibration, such as the ECE or the MCE, are global in nature and thus only measure an aggregate reliability over the entire dataset, making them insufﬁcient for many applications.
An ideal calibration metric would instead measure calibration at an individual level; however, doing so is impossible without making assumptions about the ground truth distribution (Zhao et al., 2020).
A localized calibration metric represents an adjustable balance between these two extremes.
Ideally, such a metric should measure calibration at a local level (where the extent of the local neighborhood can be chosen by the user) and group similar data points together.
In this section, we introduce the local calibration error (LCE), a kernel-based metric that allows us to measure the calibration locally around a prediction.
Our metric leverages learned features to automatically group similar samples into a soft neighborhood, and allows the neighborhood size to be set with a hyperparameter γ .
We also consider only points with a similar model conﬁdence as the prediction, so that similarity is deﬁned in terms of distance both in the feature space and in model conﬁdence.
Thus, the LCE effectively creates soft groupings that depend on the feature space; with a semantically meaningful feature space, these groupings correspond to useful subsets of the data.
We then mention a few design choices and visualize LCE maps over a 2D feature space to show that we can use our metric to diagnose regions of local miscalibration.
3. L OCAL C ALIBRATION E RROR M ETRIC
We propose a metric to measure calibration locally around a given prediction.
The calibration of similar samples should be similar, so we use a kernel similarity function k γ :
X × X
→ R + , which provides similarity scores, to deﬁne soft local neighborhoods.
k γ ( x, x (cid:48) ) has bandwidth γ > 0 , which determines the extent of the local neighborhood — as γ increases, the neighborhood grows.
Less similar (i.e., further away) samples x (cid:48) have less inﬂuence on the local calibration metric at x .
Also, as with the ECE and MCE (Eqs. 2 and 3), we use binning and consider only the points in the same conﬁdence bin as x .
Thus, the samples that inﬂuence the local calibration metric at x are similar to x in both features and model conﬁdences.
More formally, let φ : X → R d be a feature map that transforms an input to a feature vector, and let k γ be parameterized as k γ ( x, x (cid:48) )
=
g (( φ ( x ) − φ ( x (cid:48) ))
/γ )
for some Lipschitz function g : R d → R + .
Then given a data point x ∈ X and a classiﬁer ( f, ˆ p ) , the local calibration error (LCE) of the model at x is the expected difference between the model’s conﬁdence and accuracy on a randomly sampled data point x (cid:48) ∼ Pr , weighted by the kernel similarity k γ ( x, x (cid:48) ) . We say a probabilistic classiﬁer (ˆ p, f ) is perfectly locally calibrated with respect to k γ if [Equation]
Similar to perfect calibration, perfect local calibration is achieved by the Bayes-optimal classiﬁer.
In general, perfect local calibration is a much stricter notion than perfect calibration due to localizing to each indiviudal data point x , and reduces to perfect calibration if k γ ( x, x (cid:48) ) ≡ 1 is a trivial kernel. To deﬁne LCE on a ﬁnite dataset, we perform an additional binning on the conﬁdence to deal with the conditioning. Let D =
(( x 1 , y 1 ) , . .
, ( x N , y N )) be a dataset, and let β ( x )
= { i : ˆ p ( x i ) ∈
B (ˆ p ( x )) } be the set of indices of the points in D occupying the same conﬁdence bin as x .
Then we can compute the LCE by
[Equation]
Note that the quantity (ˆ p ( x i ) − 1 [ f ( x i ) =
y i ]) is simply the difference between the conﬁdence and the accuracy for sample x i , and the denominator is a normalization term.
We then deﬁne the maximum local calibration error (MLCE) as
[Equation]
Intuitively, the LCE considers a neighborhood about a sample x (as deﬁned by the kernel k γ and the conﬁdence bin B ), and computes the kernel-weighted average of the difference between the conﬁdence and accuracy for each sample in that neighborhood.
Note that by changing the bandwidth γ , we can interpolate the LCE between an individualized calibration metric (as γ → 0 ) and a global one (as γ → ∞ ).
Lemma 1 makes this more concrete under the assumption that lim γ →∞ k γ ( x, x (cid:48) ) =(proof in Appendix C).
For example, the Laplacian and Gaussian kernels satisfy this condition.
Lemma 1.
As γ → ∞ , the MLCE converges to the MCE.
Theorem 1 shows that under certain regularity conditions, the ﬁnite-sample estimator LCE( x ) converges uniformly and sample-efﬁciently to its true expected value LCE ∗ ( x ) :
Theorem 1.
(Informal) Let α ≤ inf x ∈X E [ k γ ( X, x )
[ˆ p ( X ) ∈ B (ˆ p ( x ))]] be a lower bound on the expectation of the kernel, and d be the dimension of the kernel’s feature space.
If the sample size is at least (cid:101) O
( d/α 4 (cid:15) 2 ) where (cid:15) > 0 is a target accuracy level, then with probability at least 1 − δ
we have
[Equation]
Here, (cid:101) O hides log factors of the form log(1 /αγδ(cid:15) ) .
In practice, α depends inversely on γ .
To summarize, the MLCE measures a worst-case individual calibration error as γ → 0 (i.e., the effective neighborhood is very small) and converges to the global MCE metric as γ → ∞ (i.e., the effective neighborhood is very large).
In practice, one must pick intermediate values of γ to balance a more local notion of calibration error with the sample efﬁciency of its estimation.
A more formal statement and full proof of Theorem 1 can be found in Appendix D.
3. C HOICE OF K ERNEL AND F EATURE M AP
In this work, we compute the LCE using 15 equal-width bins and use the Laplacian kernel
[Equation]
Because distances in a high-dimensional input space (e.g., image data) may not be meaningful on their own, we evaluate the kernel on a feature representation of x rather than on x itself.
Features learned from neural networks have proven useful for a wide range of tasks, and they have been shown to capture useful semantic features of their inputs (Huh et al., 2016;
Chen et al., 2020; Li et al., 2020).
The kernel similarity term k γ ( x, x (cid:48) ) in the LCE thus leverages learned features to automatically capture rich subgroups of the data.
For image data, we chose an Inception-v3 model as our feature map, since Inception features are widely accepted as useful and representative in many areas (e.g., for generative models (Salimans et al., 2016)), though other neural features can also be used (Appendix B).
For tabular data, we used the ﬁnal hidden layer of the neural network trained for classiﬁcation.
[Figure]
In general, we also use t-SNE or PCA to reduce the dimension of the feature space.
For example, the 2048-D Inception-v3 embedding is still very high-dimensional.
We report results using t-SNE to reduce the dimension to 2 or 3, as well as PCA to reduce the dimension to reduce the dimension to 50 for image data and 20 for tabular data.
Thus the overall representation function is φ ( x ) =
φ 2 ( φ 1 ( x )) , where φ 1 maps from the inputs to the neural features, and φreduces the feature space dimension.
Figure 1 plots the MLCE as a function of the kernel bandwidth for an ImageNet classiﬁcation task.
Note that when γ is small, the MLCE is 1 (a worst-case individual calibration error), and when γ is large, the
MLCE approaches the global MCE.
To obtain a single summary statistic describing the local calibration error, we can view this plot and pick a value of γ between the limiting behaviors.
We ﬁnd that γ = 0 . 2 and γ = 0 .
4 are good intermediate points for the 3-D t-SNE and 50-D PCA features, respectively (Figure 1).
3. L OCAL C ALIBRATION E RROR V ISUALIZATIONS
[Figure]
To provide more intuition for the LCE, we will now visualize some examples of the LCE metric over a 2-D feature embedding.
We consider a ResNet-50 model pre-trained on ImageNet as our classiﬁer
( f, ˆ p ) , and pre-trained Inception-v3 features as a feature map φ 1 : X → R 2048 .
φ 2 : R 2048 → Rthen reduces the 2048-D feature vectors with t-SNE to two dimensions for ease of visualization in the
LCE landscapes, so our overall representation function is φ = φ 2 ( φ 1 ( x )) .
Figure 2 visualizes the landscape of LCE 0 .
2 ( x ; f, ˆ p ) as a function of φ ( x ) for the entire ImageNet validation set, as well as the marginal CDF of LCE 0 .
2 ( x, f, ˆ p ) .
We show these visualizations for the two conﬁdence bins with the best and worst global calibration.
In the bin with the best global calibration, Figure 2 (top) shows that the landscape of the LCE is highly non-uniform, and the CDF of the LCE lies almost entirely to the right of the bin’s average calibration error.
Numerically, the bin’s average calibration error is 0 .
0022 , while its average LCE is 0 . 0259 .
This implies that the regions where the model is underconﬁdent and overconﬁdent are spatially clustered within the bin.
Because global calibration metrics solely consider the average accuracy and average conﬁdence within a bin, conﬁdence predictions that are too high and too low are averaged out to obtain a low overall error value; they fail to capture this localized miscalibration.
In the bin with the worst global calibration, Figure 2 (bottom) clearly shows that the LCE still has high variance, even though the average calibration error of the bin ( 0 . 0455 ) is much closer to its average LCE ( 0 . 0515 ).
The CDF plot provides more evidence that the landscape is not ﬂat — there is no sharp rise at the bin calibration error.
However, in this case the regions that are underconﬁdent and overconﬁdent are not clustered spatially.
LCE R ECALIBRATION
In this section, we introduce lo cal
re calibration (LoRe), a non-parametric recalibration method that adjusts a model’s output conﬁdences to achieve better local calibration.
Our method improves the LCE more than existing recalibration methods, and using our method improves performance on both downstream fairness tasks and downstream decision-making tasks.
Speciﬁcally, we can leverage the kernel similarity to achieve strong calibration for all sensitive subgroups of a population, without knowing those groups a priori.
As long as the feature space is semantically meaningful,
LoRe provides utility for downstream tasks without needing subgroup labels for the samples.
If the subgroups are known, one can recover standard group-wise recalibration methods (and metrics) by using the improper kernel k ( x, x (cid:48) )
=
[ x, x (cid:48) in same group ] .
The idea behind our method is simple: we can compute the kernel-weighted accuracy for each point x of all the points that are in the same conﬁdence bin as x , and then reset the conﬁdence of x to this kernel-weighted accuracy value.
Note that using the kernel function to compute this value is intuitively like taking a weighted average of the accuracy of the points in the local neighborhood of x .
Thus, LoRe can be considered a local analogue to histogram binning.
More formally, given a trained classiﬁer ( f, ˆ p ) , a recalibration dataset D =
(( x 1 , y 1 ) , . .
, ( x N , y N )) , and a ﬁxed point x ∈ X , let β ( x )
=
{ i : ˆ p ( x i ) ∈
B (ˆ p ( x )) } be the set of indices of the points in D occupying the same conﬁdence bin as x .
Then, we compute the recalibrated conﬁdence as
[Equation]
Equation 6 represents the kernel-weighted average accuracy of all points in the same conﬁdence bin as x .
In the limit as the kernel bandwidth γ → ∞ , ˆ p (cid:48) ( x ) → (cid:80) i ∈ β ( x ) 1 [ f ( x i )
=
y i ] / | β ( x ) | recovers histogram binning.
As γ → 0 , ˆ p (cid:48) ( x ) → 1 [ f ( x i ∗ ) =
y i ∗ ] , where i ∗ = arg min i
∈
β
( x ) k γ ( x, x i ) , thus recovering a nearest-neighbor method.
For intermediate γ , our method interpolates between the two extremes.
Throughout this work, we used used γ = 0 . 2 for LoRe with tSNE and γ = 0 . 4 for
LoRe with PCA throughout this work, since these represent intermediate points between the limiting behaviors of the LCE (e.g., see Fig. 1).
E XPERIMENTS
In this section, we show empirically that LoRe substantially improves LCE values, and that these lower LCE values lead to better performance on downstream fairness and decision-making tasks.
In particular, we evaluate the local calibration through the MLCE, because we are interested in
[Figure] [Figure]
understanding a model’s worst-case local miscalibration.
On each task we compare the performance of LoRe to no recalibration (‘Original’), temperature scaling (‘TS’) (Guo et al., 2017), histogram binning (‘HB’)
(Zadrozny & Elkan, 2001), isotonic regression (‘IR’) (Zadrozny & Elkan, 2002) and direct MMCE optimization (‘MMCE’) (Kumar et al., 2018)
all strong global recalibration methods.
We ﬁrst run extensive experiments on three datasets to demonstrate that LoRe outperforms all baselines and achieves the lowest MLCE over a wide range of γ values.
We then evaluate the performance of our method on a fairness task, where it is important that a model is well-calibrated for all sensitive subgroups of a given population and we demonstrate that it achieves the lowest group-wise MCE.
Notably, we ﬁnd that the MLCE is well-correlated with the group-wise MCE across all experimental settings and thus achieving low MLCE is a good indicator that a model has good group-wise calibration.
Finally, we compare the performance of our method against the baselines on a cost-sensitive decision-making task, where there is a low cost for a prediction of “unsure” but a high cost for an incorrect prediction, and show that our method achieves the lowest cost.
5. D ATASETS
ImageNet dataset (Deng et al., 2009):
A large-scale dataset of natural scene images withclasses; over 1.3 million images total.
The training/validation/test split is 1.3mil / 25,000 / 25,000.
UCI Communities and Crime dataset (Dua & Graff, 2017):
This tabular dataset contains a number of attributes about American neighborhoods (e.g., race, age, employment, housing, etc.).
The task is to predict the neighborhood’s violent crime rate.
The training/validation/test split is 1494 / 500 / 500.
We randomize the training/validation/test split over multiple trials.
CelebA dataset (Liu et al., 2015): A large-scale dataset of face images with 40 attribute annotations (e.g., glasses, hair color, etc.); 202,599 images total.
The training/validation/test split is 162,770 / 19,867 / 19,962.
5. R ECALIBRATION P ERFORMANCE
LoRe substantially improves the LCE values.
In Figure 3, we plot the MLCE as a function of γ .
We can see that our method outperforms all baselines (strong global calibration methods) across a wide range of γ values on ImageNet.
This is true despite the fact that we only implement LoRe for a single γ .
Appendix B provides similar results on the Communities & Crime, CelebA, CIFAR-10, and CIFAR-100 datasets.
Note that LoRe works well regardless of the feature map and the dimensionality reduction method (see Section 5.3 for results with both t-SNE and PCA).
{+Although the results shown in this section use Inception-v3 features, we show similar results in Appendix B with AlexNet (Krizhevsky, 2014), DenseNet
+}
(Huang et al., 2018), and ResNet101 (He et al., 2015) features.
Recall that as γ gets large, the MLCE recovers the MCE; because LoRe does well even at large γ , our method also works well at minimizing global calibration errors.
The fact that LoRe lowers the worst-case LCE suggests that it leads to lower LCE values across the entire dataset.
5. D OWNSTREAM F AIRNESS P ERFORMANCE
Experimental Setup
In many fairness-related applications, it is important to show that a model is well-calibrated for all sensitive subgroups of a given population.
For example, when predicting the crime rate of a neighborhood, a model should not be considered well-calibrated if it consistently underestimates the crime rate for neighborhoods of one demographic, while overestimating the crime rate for neighborhoods of a different demographic. Therefore, in this section
[Table]
we examine the worst-case group-wise miscalibration of a classiﬁer, as measured by the maximum group-wise MCE when evaluated only on sensitive sub-groups.
We consider the following experimental settings:
UCI Communities and Crime: Predict whether a neighborhood’s crime rate is higher than the median; group neighborhoods by their plurality race (White, Black, Asian, Indian, Hispanic).
60 random seeds for model training.
CelebA: Predict a person’s hair color (bald, black, blond, brown, gray, other); group people by hair type (bald, receding hairline, bangs, straight, wavy, other).
20 random seeds for model training.
CelebA: Predict a person’s hair type; group people by their hair color; inverse of Setting 2.
20 random seeds for model training.
For each task [Table] we train a classiﬁer (see Appendix A for full details) and recalibrate its output conﬁdences using each of the recalibration methods.
[Table]
Results Table 1 reports the maximum group-wise MCE for each of the recalibration methods on each of the three tasks.
LoRe outperforms the other baselines, achieving an average 49% reduction over no recalibration and an average 23% improvement over the next best global recalibration method.
(Note in Figures 5, 6, and 7 in Appendix B that LoRe is the most effective method of lowering the MLCE over a wide range of γ ).
Notably, LoRe is robust to the feature map used (tSNE vs. PCA). It even outperforms global methods applied to each individual group, implying that correcting local calibration errors is a robust way to improve group calibration that generalizes better than naive alternatives [Table]
[Table] Moreover, Table 2 shows that the maximum group-wise MCE is well-correlated with the
MLCE, and it is in fact much better correlated with MLCE than with global calibration metrics
Taken together, our results indicate that lowering the LCE has positive implications in fairness settings that cannot be achieved by simply lowering global metrics like the ECE.
For reference, we also include the performance of all recalibration methods on various global calibration metrics in Table 3, which shows that LoRe is able to improve worst-case group-wise calibration without meaningfully sacriﬁcing (and in some cases improving) average-case global calibration.
5. D OWNSTREAM D ECISION -M AKING
Experimental Setup Machine learning predictions are often used to make decisions, and in many situations an agent must select a best action in expectation.
As an example, suppose there is a low [Table] cost u associated with returning “unsure” and a high cost w associated with returning an incorrect classiﬁcation (e
[Table]
g., in situations such as autonomous driving, being unsure incurs only the small cost of calling a human operator, but making an incorrect classiﬁcation incurs a high cost).
An agent with good uncertainty quantiﬁcation can make a more optimal decision about whether to return a classiﬁcation or return “unsure”; for a calibrated model, it would be optimal for the agent to return “unsure” below the conﬁdence threshold of 1 − u/w , and return a prediction above this threshold. Following this policy — i
[Table] e., returning “unsure” when the conﬁdence is below this threshold and returning a prediction when the conﬁdence is above it [Table] we used a ResNet-50 model to make predictions on ImageNet
[Table]
and recalibrated the predictions with each of the recalibration methods. For each method, we then calculated the total reward attained under various reward ratios w/u [Table] as well as the prediction rejection area ratio (PRR) (Malinin et al., 2020), which measures the quality of the rejection curve when a model can choose to not provide any prediction.
Note that a PRR of 1.0 indicates optimal rejection, and a PRR of 0.0 indicates “random” rejection; a higher PRR is better.
[Table]
Results In Figure 4, we show the improvement in the total reward over the original classiﬁer (i.e., no recalibration) as a function of the reward ratio w/u (the ratio of the cost of an incorrect classiﬁcation to the cost of being unsure).
Across a wide range of reward ratios, LoRe achieves the highest reward.
The MLCE curves for this task are shown in Figure 3; note that
LoRe also achieves lower LCE values than the global recalibration methods.
Table 4 also reports the PRR achieved with each method under a cost function of u = − 1 , w = − 10 , as well as several global calibration metrics; LoRe generally outperforms the other baselines in terms of both PRR and global calibration metrics.
These results indicate that our recalibration method most effectively lowers LCE values without sacriﬁcing (and indeed often improving) average-case global calibration, and that these lower LCE values correspond to better performance on this decision-making task.
C ONCLUSION
In this paper, we introduce the local calibration error (LCE), a metric that measures calibration in a localized neighborhood around a prediction.
The LCE spans the gap between fully global and fully individualized calibration error, with an effective neighborhood size that can be set with a bandwidth parameter γ .
We also introduce LoRe, a recalibration method that greatly improves the local calibration.
Finally, we demonstrate that achieving lower LCE values leads to better performance on downstream fairness and decision-making tasks.
In future work, we hope to explore better feature spaces to deﬁne similarity, since the quality of our metric depends on the quality of the feature space.
A M ODEL A RCHITECTURE , T RAINING , AND O THER H YPERPARAMETERS
For ImageNet and CelebA, we compute the ECE, MCE, and LCE using 15 equal-width conﬁdence bins.
For the UCI communities and crime dataset, we use 5 equal-width bins because the dataset is much smaller (500 datapoints for recalibration).
These numbers of bins represent a good tradeoff between bias and variance in estimating the relevant calibration errors.
We also ran some initial experiments with equal-mass binning, but found that the results were very similar to those obtained with equal-width binning.
A. I MAGE N ET
For all experiments with the ImageNet dataset, we used the pre-trained ResNet-50 model from the PyTorch torchvision package as our classiﬁer.
To calculate the LCE and apply LoRe, we used pre-trained Inception-v3 features, applying either t-SNE to reduce their dimension to 3 or PCA to reduce their dimension to 50, as a feature representation for the kernel.
A. UCI C OMMUNITIES AND C RIME
For all experiments with the UCI communities and crime dataset, we used a 3-hidden-layer dense neural network as our base classiﬁer.
Each hidden layer had a width of 100 and was followed by a
Leaky ReLU activation.
We applied dropout with probability 0.4 after the ﬁnal hidden layer.
We trained the model using the Adam optimizer with a batch size of 64 and a learning rate of 3 × 10 −until the validation accuracy stopped improving.
All other hyperparameters were PyTorch defaults.
Training was done locally on a laptop CPU.
We trained 60 different models with different random seeds to perform the experiments described in Section 5.3 and Figure 5.
To calculate the LCE and apply LoRe, we used the ﬁnal hidden layer representation learned by our model, applying t-SNE to reduce the dimension to 2 or PCA to reduce their dimension to 20, as a feature representation for the kernel
. A. C ELEB A
For all experiments with the CelebA dataset, we trained a ResNet50 model and used it as our base classiﬁer.
We applied standard data augmentation to our training data (random crops & random horizontal ﬂips), and trained all models for 10 epochs using the Adam optimizer with a learning rate of 1 × 10 − 3 and a batch size of 256.
All other hyperparameters were PyTorch defaults.
Training was distributed over 4 GPUs, and training a single model took about 30 minutes.
For both Setting 2 and Setting 3 (described in Section 5.3), we trained 20 models with different random seeds to perform the experiments shown in Figures 6 and 7.
To calculate the LCE and apply LoRe, we used pre-trained Inception-v3 features, applying t-SNE to reduce their dimension to 2 or PCA to reduce their dimension to 50, as a feature representation for the kernel.
B A DDITIONAL E XPERIMENTAL R ESULTS
In Figures 5, 6, and 7, we visualize the MLCE achieved by all recalibration methods for the three experimental settings evaluated in Section 5.3.
Figure 3 in the main paper shows the same visualization for all methods on ImageNet.
In Figure 8, we plot the MLCE achieved by all recalibration methods for CIFAR-100, and in Figure 9, we do the same for CIFAR-10.
Across all settings and datasets, our method LoRe is the most effective at minimizing MLCE across a wide range of γ , even accounting for variations between runs.
[Figure]
[Figure]
[Figure]
In these ﬁgures, “Original” represents no recalibration, “TS” represents temperature scaling, “HB” represents histogram binning, “IR” represents isotonic regression, “MMCE” represents direct MMCE optimization, and “LoRe” is our method. Next, we examine the inﬂuence of the speciﬁc feature map used.
In Figures 10, 11, 12, and 13, we plot the MLCE achieved by all recalibration methods for ImageNet using Inception-v3, AlexNet, DenseNet121, and ResNet101 features.
In Figures 14 and 15, we plot the MLCE achieved by all recalibration methods for ImageNet when the features used to calculate the MLCE are different from the features used by LoRe. For completeness, in Figures 16, 17, 18, and 19, we also visualize the average LCE for all experimental settings.
All plots show similar results: LoRe performs best over a wide range of γ . [Figure] achieves the best for most [Figure]
[Figure]
[Figure]
[Figure]
C P ROOF OF L EMMA
We restate Lemma 1 below, and provide the proof:
Lemma 2.
Assume that lim γ →∞ k γ ( x, x (cid:48) )
= 1 for all x, x (cid:48) ∈ X .
Then, as γ → ∞ , the MLCE converges to the MCE.
Proof.
Since lim γ →∞ k γ ( x, x (cid:48) )
= 1 identically,
[Equation]
D F ORMAL S TATEMENT AND P ROOF OF T HEOREM
Let B 1 , . . .
, B N denote a set of bins that partition [0 , 1] , and B ( p ) denote the bin that a particular p ∈ [0 , 1] belongs to.
Let a f ( x, y ) =
1 [ f ( x ) =
y ] indicate the accuracy of a the classiﬁer ( f, ˆ p ) on an input x .
We consider the signed local calibration error (SLCE):
[Equation]
Y ))
k γ ( X, x ) | ˆ
p ( X ) ∈ B (ˆ p ( x ))]
E [ k γ ( X, x ) | ˆ p ( X ) ∈ B (ˆ p ( x ))]
=
E [(ˆ p ( X ) − a f ( X, Y ))
k γ ( X, x ) 1 [ˆ p ( X ) ∈ B (ˆ p ( x ))]]
E [ k γ ( X, x ) 1 [ˆ p ( X ) ∈ B (ˆ p ( x ))]] .
D. A SSUMPTIONS AND F ORMAL S TATEMENT OF T HEOREM
We make the following assumptions:
Assumption A (Lipschitz kernel) .
The kernel k γ takes the form
[Equation]
where φ : X → R d is a representation function, and g : R d → [0 , 1] is L -Lipschitz with respect to some norm (cid:107)·(cid:107) .
Note this deﬁnition may require an implicit rescaling (for example, we can take φ ( x ) ← φ feature ( x ) /d for a d -dimensional feature map φ feature and take g ( z ) =
exp( −(cid:107) z (cid:107) 1 ) , which corresponds to the
Laplacian kernel we used in Section 3.2).
Assumption B (Binning-aware covering number) .
For any (cid:15) > 0 , the range of the representation function φ ( X ) :
= { φ ( x ) : x ∈ X} has an (cid:15) -cover in the (cid:107)·(cid:107) -norm of size ( C/(cid:15) )
d for some absolute constant C > 0 : There exists a set N (cid:15) ∈
X with |N (cid:15) | ≤ ( C/(cid:15) )
d such that for any x ∈ X , there exists some x (cid:48) ∈ N (cid:15) such that (cid:107) φ ( x ) − φ ( x (cid:48) )
(cid:107) ≤ (cid:15) and B (ˆ p ( x ))
=
B (ˆ p ( x (cid:48) )) .
Assumption C (Lower bound on expectation of kernel within bin) .
We have
[Equation]
for some constant α ∈ (0 , 1) .
The constant α characterizes the hardness of estimating the SLCE from samples.
Intuitively, with a smaller α , the denominator in SLCE gets smaller and we desire a higher accuracy in estimating both the numerator and the denominator.
Also note that in practice the value of α typically depends on γ .
We analyze the following estimator of the SLCE using n samples:
[Equation]
Theorem 2.
Under Assumptions A, B, and C, Suppose the sample size n ≥ (cid:101) O
( d/α 4 (cid:15) 2 ) where (cid:15) >is a target accuracy level, then with probability at least 1 − δ
we have
[Equation]
where (cid:101) O hides log factors of the form log( L/γ(cid:15)δα ) .
Theorem 2 shows that (cid:101) O ( d/(cid:15) 2 α 4 ) samples is sufﬁcient to estimate the SLCE simultaneously for all x ∈ X .
When α = Ω(1) , this sample complexity only depends polynomially in terms of the representation dimension d and logarithmically in other constants (such as L, γ , and the failure probability δ ).
D. P ROOF OF T HEOREM
Step 1.
We ﬁrst study the estimation at ﬁnitely many x ’s. Let N ⊆
X be a ﬁnite set of x ’s with |N| =
N .
Since k γ ∈ [0 , 1] and | ˆ p ( x ) − a f ( x, y ) | ≤ 1 are bounded variables, by the Hoeffding inequality and a union bound, we have
[Equation]
Therefore, as long as n ≥ O (log( N/δ ) /(cid:15) 2 α 2 ) samples, the above probability is bounded by δ .
In other words, with probability at least 1 − δ , we have simultaneously
[Equation]
for all x ∈ N .
Similarly, when n ≥ O (log( N/δ ) /(cid:15) 2 α 4 ) , we also have (with probability at least 1 − δ ) [Equation] n
[Equation]
On these concentration events [Equation] we have for any x ∈ N that
[Equation]
≤
[Equation]
[Equation]
[Equation]
α
[Equation]
α(cid:15)/≤ (cid:15).
Step 2.
We now extend the bound to all x ∈ X using the covering argument.
By Assumption B, we can take an α 2 (cid:15)γ/ (10 L ) -covering of φ ( X ) with cardinality N ≤ (10 CL/α 2 (cid:15)γ ) d .
Let N ⊂ X denote the covering set (in the X space).
This means that for any x ∈ X , there exists x
(cid:48) ∈
N such that (cid:107) φ ( x ) − φ ( x (cid:48) )
(cid:107) ≤ α 2 (cid:15)γ/ (10 L )
amd B (ˆ p ( x ))
=
B (ˆ p ( x (cid:48) )) [Equation] which implies that for any (cid:101) x ∈ X we have
[Equation]
where we have used the Lipschitzness assumption of g (Assumption A).
This further implies
[Equation]
[Equation] [Equation]
[Equation]
] ≤ α 2 (cid:15)/ 10 .
Similarly, we have | A ( x ) − A ( x (cid:48) )
| ≤ α 2 (cid:15)/ 10 , | ˆ B ( x ) − ˆ B ( x (cid:48) ) | ≤ α 2 (cid:15)/ 10 , and | B ( x ) − B ( x (cid:48) )
| ≤ α 2 (cid:15)/ 10 .
This means that the estimation error at x is close to that at x (cid:48) ∈ N and consequently also bounded by (cid:15) :
[Equation]
ˆ A ( x ) ˆ B ( x ) − A ( x ) B ( x ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ≤ (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ˆ A ( x ) ˆ B ( x ) − ˆ A ( x (cid:48) )
ˆ B ( x (cid:48) )
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) + (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ˆ A ( x (cid:48) )
ˆ B ( x (cid:48) ) −
A ( x (cid:48) )
B ( x (cid:48) )
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) + (cid:12)(cid:12)(cid:12)(cid:12)
A ( x (cid:48) )
B ( x (cid:48) ) −
A ( x )
B ( x ) (cid:12)(cid:12)(cid:12)(cid:12) ≤(cid:20) 1 · α 2 (cid:15)/α ( α − α 2 (cid:15)/ 10) + α · α 2 (cid:15)/(cid:21) ≤ (cid:15).
Therefore, taking this N in step 1, we know that as long as the sample size
N ≥ O (cid:18) log( |N| /δ )
(cid:15) 2 α(cid:19)
=
O (cid:32) d (cid:2) log (cid:0) 10 CL/α 2 (cid:15)γ (cid:1)
+ log(1 /δ )
(cid:3) α 4 (cid:15)(cid:33) =
(cid:101) O (cid:0) d/α 4 (cid:15) 2 (cid:1) , we have with probability at least 1 − δ that
[Equation]
This is the desired result.