Uncertainty Calibration Error: A New Metric for Multi-Class Classification
Authors: Max-Heinrich Laves
Sontje Ihler
Karl-Philipp Kortmann
Tobias Ortmaier
Venue:  (2021)
Keywords: variational inference
uncertainty
calibration
classification

Abstract: Various metrics have recently been proposed to measure uncertainty calibration of deep models for classification. However, these metrics either fail to capture miscalibration correctly or lack interpretability. We propose to use the normalized entropy as a measure of uncertainty and derive the Uncertainty Calibration Error (UCE), a comprehensible calibration metric for multi-class classification. In our experiments, we focus on uncertainty from variational Bayesian inference methods and compare UCE to established calibration errors on the task of multi-class image classification. UCE avoids several pathologies of other metrics, but does not sacrifice interpretability. It can be used for regularization to improve calibration during training without penalizing predictions with justified high confidence.

Under review as a conference paper at ICLR 2021
UNCERTAINTY CALIBRATION ERROR: A NEW MET- RIC FOR MULTI-CLASS CLASSIFICATION
Anonymous authors Paper under double-blind review
ABSTRACT
Various metrics have recently been proposed to measure uncertainty calibration of deep models for classification. However, these metrics either fail to capture miscalibration correctly or lack interpretability. We propose to use the normalized entropy as a mcasure of uncertainty and derive the Uncertainty Calibration Error (UCE), a comprchensible calibration metric for multi-class classification. In our experiments, we focus on uncertainty from variational Bayesian inference methods and compare UCE established calibration errors on the task of multi-class image classification. UCE avoids several pathologies of other metrics, but does not sacrifice interpretability. It can be used for regularization to improve calibration during training without penalizing predictions with justified high confidence.
INTRODUCTION
Advances in deep learning have led to superior accuracy in classification tasks, making deep learning classifiers an attractive choice for safety-critical applications like autonomous driving (Chen et al.) 2015) or computer-aided diagnosis (Esteva et al. 2017). However, the high accuracy of recent deep learning models alone not sufficient for such applications. In cases where serious decisions are made upon model's 's predictions, it is essential to also consider the uncertainty of these predictions. We need to know if prediction is likely to be incorrect or invalid input data is presented to a deep model, e.g. data that is far away from the training domain obtained from defective sensor. The consequences of a false decision based on an uncertain prediction can be fatal.
A natural expectation is that the certainty of a prediction should be directly correlated with the quality of the prediction. In other words, predictions with high certainty are more likely to be accurate than uncertain predictions, which are more likely be incorrect. A common misconception the assumption that the estimated softmax likelihood can be directly used as confidence measure for the predicted class. This expectation dangerous in the context of critical decision decision-making. The estimated likelihood of models trained by minimizing the negative log- ligeliillhood (i.e. cross entropy) highly overconfident; that is, the estimated likelihood is considerably higher than the observed frequency of accurate predictions with that likelihood (Guo et al. 2017).
UNCERTAINTY ESTIMATION
In this work, we focus on uncertainty from approximately Bayesian methods. We assume gen- eral multi-class classification task with C' classes. Let input x E X be a random variable with corresponding label y E y = {1..... C). Let (w) be the output (logits) of a neural network with weight matrices w, and with model likelihood p(y (()()) for class which is sampled from a probability vector p= sM (fo((()))) obtained by passing the model output through the softmax function (-). From frequentist perspective, the softmax likelihood is often interpreted as confidence of prediction. Throughout this paper, we follow this definition.
The frequentist approach assumes single best point estimate of the parameters (or weights) of neural network. In frequentist inference, the weights of a deep model are obtained by maximum likelihood estimation (Bishop/ 2006), and the normalized output likelihood for unseen test input does not consider uncertainty in the weights (Kendall & Gal 2017). Weight uncertainty (also referred to as model or epistemic uncertainty) is considerable source of predictive uncertainty for models

---

Under review as a conference paper at ICLR 2021
trained on data sets of limited size (Bishop 2006 Kendall & Gal 2017). Bayesian neural networks
and recent advances in their approximation provide valuable mathematical tools for quantification
of model uncertainty (Gal & Ghahramani] 2016 Kingma & Welling 2014). Instead of assuming the existence a single best parameter set, we place distributions over the parameters and want to consider all possible parameter configurations, weighted by their posterior. More specifically, given training data set D and an unseen test sample æ with class label y. we are interested in evaluating the predictive distribution p(y|æ, D) = f p (zr, ())((w) ( dw This integral requires to evaluate the posterior p( w D), which involves the intractable marginal likelihood. A possible solution to this is to approximate the posterior with more simple, tractable distribution q(w) by optimization.
In this work, we incorporate the following approximately Bayesian methods which we use in our experiments obtaín weight uncertainty: Monte Carlo (MC) dropout (Gal Ghahramani 2016). Gaussian dropout (Wang& & Manning] 2013; Kingma et al. 2015), Bayes by Backprop (Blundell al. 2015), Gaussian (Maddox et al.) 2019), and (although not Bayesian) deep ensembles [Lakshminarayanan et al. 2017). A short review of each of the methods can be found in Appendix A.2
3 RELATED CALIBRATION METRICS
The expected calibration error (ECE) is one of the most popular calibration error metries and estimates model calibration by binning the predicted confidences ý = max, p(y = c x>) into M bins from equidistant intervals and comparing them to average
Expected Calibration Error
accuracies per bin (Naiini et 2015 Guo al. 2017):
M
Bmll
ECE
ace Bm) - conf (Bm)
(1)
with number of test samples n and acc and CB) denoting the accuracy and confidence of bin B, respectively. Several recent works have described severe pathologies of the ECE metric (Ashukha al., 20200 Nixon et al. 2019) Kumar al.) [019). Most notably, the ECE metric is minimized by a model constantly predicting the marginal distribution of the majority class which makes it impossible to directly optimize it (Kumar et al. 2018). Additionally, the ECE only considers the maximum class probability and ignores the remaining entries of the probability vector p(x)).
Adaptive Calibration Error Nixon et al. (2019) proposed the adaptive calibration error (ACE) to address the issue of fixed bin widths of ECE-like metrics. For models with high accuracy or overconfidence, most of the predictions fall into the rightmost bins, whereas only very few predictions fall into the rest of the bins. ACE spaces the bins such that an equal number of predictions contribute to each bin. The final ACE is computed by averaging over per-class ACE values to address the issue raised by Kull al. (019)) However, this makes the metric more sensitive the manually selected number of bins M as the number of bins effectively becomes M, with number of classes C. Using fixed bin widths, the numbers of samples in the sparsely populated bins is further reduced, which increases the variance each measurement per bin. Using adaptive bins, this results in the lower confidence bins spanning wide range of values, which increases the bias of the bin's measurement.
Negative Log-Likelihood Deep models for classification are usually trained by minimizing the average negative log-likelihood (NLL):
N NLL =
N Σ
loy = yi x..)) )
(2)
The NLL is also commonly used as metric for measuring the calibration of uncertainty. However, the NLL is minimized by increasing the confidence max, p(y = c æ)), which favors over-confident models and models with higher accuracy (Ashukha et al. 2020). This metric is therefore unable to compare the calibration of models with different accuracies and training a model by minimizing NLL does not necessarily lead good calibration.
2

---

Under review as a conference paper at ICLR 2021
Brier Score
The average Brier score is another popular metric for assessing the quality of predictive uncertainty and is defined as (Brier 1950:
Lakshminarayanan et al. 2017)
N E
BS
(1 = -
pl)=cc=,)))
p(u =
(3)
Similarly to the NLL, the Brier score favors high probabilities for correct predictions and low probabilities for incorrect predictions. Thus, models with higher accuracy tend to show better Brier score, which makes the metric unsuitable for comparing the quality of uncertainty for models with different accuracies.
Maximum Mean Calibration Error Common recalibration methods are applied post-hoc, e-g. temperature scaling on separate calibration set. Kumar et al. (2018) proposed the maximum mean calibration error (MMCE), a trainable calibration surrogate for the calibration error. It is defined as
(1 v.) pi) (1 w1) (D)) k(p.,
MMCE2(D) Σ
(4)
m²
I.ED
over batch C DD with batch size m, matrix-valued universal kernel and y = arg max, p(y = cl w)) Trainable calibration metrics are used in joint optimization with the negative log-likelihood
arg min Σ NLL(D,ww) w) + A AMMCE(D,w)
(5)
Kumar et al. (2018) claim have addressed the issue that the ECE is unsuitable for direct optimization due to its high discontinuity in w. However, MMCE is also minimized by model constantly predicting the marginal distribution of the classes. This leads subpar logit temperature when training with MMCE and temperature scaling can further reduce miscalibration (Kumar et al. 2018).
4 UNCERTAINTY CALIBRATION ERROR
To give an insight into our general approach to measuring the calibration of uncertainty, we will first revisit the definition of perfect calibration of confidence (Guo et al. 2017) and show how this concept can be extended to calibration of our definition uncertainty.
Let arg maxp p be the most likely class prediction of input with confidence P = max p and true label Then, following Guo et al. (2017), perfect calibration of confidence is defined as
p = a = a; Va E [0, 1].
That is. the probability of a correct prediction ú given the prediction confidence should exactly correspond to the prediction confidence. Instead of using only the probability of the predicted class, use the entropy of p to express prediction uncertainty:
H(p) Σ (c) log
Let
(kk) = (P(y 1| arg maxpp()) k), ... Ply Cl arg max p(x) k))
(8)
be a probability vector of true marginal class probabilities for all inputs æ predicted with class k. Consider the following example: Three i.i.d. inputs #1.3 binary classification task with ground truth labels {1, 1, 2} have all been predicted with arg max p(x)33)())) = 1. Then, q(1) = (2 1). With this, we define model be perfectly calibrated if
(()())) = H(p arg max p k) Vk E (1.
(9)
From this, we derive an error metric for calibration of uncertainty:
| H((p)]
(10)
However, this metric and the use of the entropy as measure of uncertainty lacks interpretability, as the entropy scales with the number of classes C. This does not allow compare the uncertainty or
3

---

Under review as a conference paper at ICLR 2021
the calibration of models trained different data sets. Therefore, propose to the normalized entropy to scale the values to a range between 0 and
H(p)
Σ logg' H E [0, E
(11)
log C
We further increase interpretability and argue, that the normalized entropy should correlate with the model error. From Eq. (6) and Eq. [T], we define perfect calibration of uncertainty as
P [o+ at = Va E [0, 1].
(12)
That is, in batch of inputs that all predicted with uncertainty of g. 0.2, top-1 error of 20 % is expected. The confidence is interpreted as the probability of belonging to particular class, which should naturally correlate with the model error of that class. This characteristic does not generally apply to entropy, and thus the question arises why entropy should correspond with the model error. Proposition 1.
The normalized entropy (uncertainty) H(p) approaches the top-1 error in the limit of number of classes C if the model p is well-calibrated.
Proof.
lim H(p) (1-
(13)
C-
The top-1 equals (1p) if the model is perfectly calibrated in the sense of Eq. (6) For detailed proof, see Appendix A.
Thus, the normalized entropy gives us an intuitive and interpretable measure of uncertainty. If model is perfectly calibrated, corresponds the top-1 error. We propose the following notion to quantify miscalibration of uncertainty:
[ # y H((p) = a] al]. Va E [0.
(14)
We refer to this as Expected Uncertainty Calibration Error (UCE) and approximate with
Bml (rr((m)) uncert(Bm.)
UCE
(15)
n
using the same binning scheme as in ECE estimation. The error per bin is defined as
err(Bm) = Bm
Σ 1(0: + y)
(16)
where 1() + y) and 1(): = 3) = Uncertainty per bin is defined as
uncert(B, (Bmert(Bm))) = 1m E..
Σ H(p.).
(17)
Properties of UCE The proposed UCE metric solves several problems of other metrics. First, the UCE is not zero for model constantly predicting the marginal class distribution. Estimators of metrics with this pathology (e.g. ECE, MMCE) suffer from varying bias and therefore do not allow comparing miscalibration of different models (Ashukha et al. 2020 Vaicenavicius et al. 2019). In contrast ACE, UCE is not highly sensitive the numbers of bins and provides a consistent ranking of different models for the same classification task (see Fig.D.. Additionally, UCE can be used as a trainable regularizer in similar manner to MMCE, During training, we compute the UCE over mini-batcher D D and add it to the NLL training objective
arg min Σ NLL(D, w) AUCE(D,w).
(18)
weighted by a factor A. UCE is zero for an optimal model and thus does not penalize high confident predictions for models with high accuracy, which is major disadvantage of plain entropy regular- ization (Pereyra et al.) 2017). Predictions with low uncertainty, but high top-1 error are penalized whereas predictions with high accuracy are encouraged have low uncertainty.
4

---

Under review as a conference paper at ICLR 2021
ACEA ACEa
ACE:
n
UCEP
Ensemble
MCDrop
calibration error in %
4.4
calibration error in
Gruss
BBB
20
SWAG
4.2
UCE:
4.0
Ensemble
MCDrop
3.8
GoussDopp
BBB
3.6
10
SWAG
20
40
60
80
25
50
75
number of bins
number of bins
values for ResNet models A and B CIFAR-10 test set. (Right) ACE and UCE values for different Bayesian methods on CIFAR-I00 test set. ECE and ACE very sensitive the number of bins used in the estimator, not yielding consistent ranking the models. UCE is less sensitive to the bin number and ranks models consistently, allowing comparison of different models.
Figure 1: (Left) Calibration
5 EXPERIMENTS
We evaluate the proposed uncertainty calibration enror mutli-class image classification CIFAR- 10 with ResNet-34 and on CIFAR-I00 with ResNet-50 (He et al. 2016 Krizhevsky Hinton 2009). The feature extractor of ResNet is used implemented in Py Torch 1.6 (Paszke al. 2019 and the last linear layer is implemented using the different Bayesian approximations from $2 All models were trained from random initialization. We employed early stopping highest validation set accuracy. More details on the training procedure and a link to our source code can be found in Appendix A..3
First, we compute the accuracies and all calibration error metrics from 3 and the UCE on the test of CIFAR-10/100 for all models. We investigate the effect of the number of bins in the estimators of the metrics involving binning and analyze the ranking of different models under varying softmax temperature т, where p = OSM(T-1 (z)). Finally, we train a ResNet on CIFAR- 10/100, SVHN, and Fashion-MNIST with added calibration error regularization as in Eq. and (8) We compare UCE regularization (A 10) regularization with MMCE (A 10) and confidence penalty arg min Ep NLL. w) + À HDD, w) with X = 0.1, which penalizes the entropy of the probability vector p of each prediction (Pereyra et al. 2017). We combine the regularization experiments with post-hoc calibration using temperature scaling (Guo al. 2017).
Additionally, we analyze the utility the normalized entropy as a measure of uncertainty and perform rejection and out-of-distribution (OoD) detection experiments using H. We define an uncertainty threshold Hmax and reject all predictions from the test set where H(p) > Hmax- A decrease in false predictions of the remaining test set is expected. To demonstrate the OoD detection ability, we provide images from CIFAR-100 to a deep model trained on CIFAR- 10 (note that both CIFAR data sets have mutual classes). In this experiment, from the test set of the training domain and stepwise replace images with oft-of-diitribution data. In practice, it is expected that models are applied step, we evaluate the mean batch uncertainty and expect, that the mean uncertainty monotonically increases as function of the fraction of OoD data.
compose batch of 100 random samples
mix of known and unknown classes. After each
5.1 RESULTS
In this section, the results the above mentioned experimental setups are presented and discussed.
Comparison of Calibration Error Metrics T test set accuracy and all calibration error results for all model/data set configurations. Without any post-hoc hoc calibration, such as temperature scaling, all metrics provide the same ranking of the models. The deep ensemble and SWAG perform
5

---

Under review as a conference paper at ICLR 2021
M(p)
0.30
A(p)
0.06
MCDrop
GaussDrop
GaussDrop
BBB
mean uncertainty
BB8
top-1 error
SWAG
0.20
SWAG
0.04
1 maxp
0.15
- maxy
MCDrop
MCDrop
GaussDrop
0.10
Gaussrop
BBB
BBB
SWAG
SWAG
0.05
0.00
100
uncertainty threshold
OoD data
Figure 2: (Left) Rejection results CIFAR-10 for decreasing uncertainty threshold comparing (p)
and max p as uncertainty metric. In both cases, the top-1 error decreases strictly monotonically with decreasing threshold. (Right) Out-of-distribttion detection for CIFAR-10 CIFAR- 100. The normalized entropy Hp)) measure of uncertainty can be used robustly detect OoD data.
best in terms of test set accuracy and calibration of uncertainty. Brier score and NI.L are both highly sensitive the model accuracy, which is especially apparent on CIFAR-10. For the first three models with similar accuracy, the Brier scores differ only marginally. Thus, both the Brier score and the NLL are unsuitable for comparing the calibration of different models. AAsuukha the calibrated NLL at optimal temperature for model comparison. However, Fig. p the metrics over varying softmax temperature and show, that the models with highest accuracy have lowest Brier and NLL., regardless of the temperature. From this we deduce that both Brier and NLL should not be used for comparison of multi-class calibration, even at optimal temperature. The remaining metrics show consistent ranking before and after the point of optimal temperature. The metrics ECE, UCE and MMCE have a narrow region in which the optimal temperature for all models can be found. This allows comparison of calibration of models if they all over- underconfident. However, all metrics fail at comparing underconfident models to overconfident models (see model ranking left and right of optimal temperature in Fig.
(2020) propose to use
Fig. shows the effect of the number of bins M in the estimators of ECE, ACE and UCE. Both ECE and ACE more sensitive to the number of bins and do not provide consistent ranking of models under varying bin count. This is due to the fact that fewer bins are populated using H as uncertainty (cf. Fig. IO in the appendix). This can be interpreted as possible downside of the UCE metric as the adaptive binning scheme of ACE explicitly addresses that. However, argue that consistent ranking due to robustness against bin count results in metric that is more useful in practice.
Uncertainty Regularization Tab shows results of ResNet-50 with SWAG trained on CIFAR- 100. All regularization methods considerably reduce miscalibration compared to unregularized models. Plain entropy regularization is surprisingly effective CIFAR- 100; however, CIFAR-10 (see Tab.T), increased miscalibration and is generally outperformed by MMCE and UCE regularization optimal temperature. Therefore, when performing post-hoc temperature scaling, MMCE and UCE regularization is preferable to entropy regularization. UCE regularization can be interpreted as entropy penalization for predictions with low accuracy. As UCE zero for an optimal model, encourages model to reach high accuracy.
Rejection & OoD Detection Fig. (left) shows the top-1 enror function of decreasing uncer- tainty threshold Hmax and (right) shows the mean batch uncertainty at increasing OoD data. Robust rejection of uncertain predictions and detection OoD data based on the normalized entropy 7 (p) possible and is generally more sensitive OoD data than the confidence max p.

---

Under review as a conference paper at ICLR 2021
6 CONCLUSION
We have proposed to measure uncertainty based on the normalized entropy. From this, we derived the uncertainty calibration emror; a new metrie that avoids several pathologies of existing calibration errors. In our experimental evaluation, we focused on uncertainty from approximate Bayesian methods and deep ensembles. The UCE does not only consider the class with the highest probability and is not minimized by a constant model predicting the marginal class distribution. In contrast to the Brier score and NLL., it allows comparison of models with different accuracy. is not sensitive to varying number of bins and provides a consistent ranking of models. However, we follow the suggestion of Assukhaa et al. (2000) and state that comparison of calibration for different models should only be done optimal softmax temperature. Regularization with UCE during training reduces miscalibration and does not penalize high accuracy and predictions with justified high confidence. UCE regularization with temperature scaling often performed best in experiments in terms of calibration. The normalized entropy itself is a useful measure of uncertainty and allows for robust rejection of uncertain predictions and detection of OoD data.
We hope to have provided a new useful metric for reliable evaluation of uncertainty estimation. UCE easy implement and interpretable as it expresses the discrepancy of the uncertainty from the model error, which increases the chance of being accepted by deep learning practitioners.
7 REBUTTAL
In this section, new or updated results during the rebuttal phase are presented. We identified the main question of the reviewers to why/where our calibration metric (UCE) beneficial compared to previous metrics (ECE, MMCE, ACE) and how normalized entropy is beneficial over vanilla entropy. We project the following main changes for the final manuscript:
illustrate the advantage of normalized entropy over vanilla entropy
add toy experiments show that UCE is able to capture miscalibration where ECE, ACE and MMCE fail (see results below)
add results from experiments additional data sets (SVHN, Fashion-MNIST)
To make room for the following content, we propose the removal/reduution (approx. -1 page) of
the review of Bayesian methods and refer to previous work instead, experiments regarding temperature (remove Table I and Figure 1)
Additionally, we will include all minor comments of the reviewers in the final manuscript.
7.1 UNCERTAINTY FROM NORMALIZED ENTROPY vs. ENTROPY
The advantages of normalized entropy over vanilla entropy our definition of UCE are twofold: First, the domain of the metric does not scale with the number of classes C, which helps comparing calibration of models trained on different data sets. Second, the value of the metric more interpretable: Rejecting test samples where H < 0.2 will result in a classification error < 0.2 for the remaining samples the model well-calibrated (see Fig. 2 We argue that using normalized entropy uncertainty measure is as interpretable max p, but avoids the pathologies of max when used in a calibration metric.
7.2
UCE vs. OTHER CALIBRATION METRICS
UCE more reliable than ECE and MMCE because it is based on normalized entropy and incorporates the predictions of all classes. It is more robust than ACF, as it is significantly less sensitive to binning.
Reliable Uncertainty Detection ECE and MMCE can be minimized by models with an uninforma- tive, constant output Ovadia et al. 2019). Given data set of two classes, with 60 % class and 40 % class 2, and a degenerated model that consistently predicts the marginal probabilities p = (0.6, 0.4). This leads to 60 % correctly classified samples for class 1 leading perfect calibration scores for
7

---

Under review as a conference paper at ICLR 2021
1.0
ECE 0.0
1.0
ACE = =5000
1.0
UCE 57.1
MMCE 0.0
accuracy
accuracy
error
0.5
0.5
0.5
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
confidence
confidence
uncertainty
Figure 3: Calibration diagrams for a toy experiment with a degenerated model constantly predicting
the marginal probabilities p (0.6, 0.4) in binary classification task. ECE and MMCE only consider max and fail at capturing the miscalibration of class with p(c = 2) = 0.4, but acc(c 2) = 0. The red bars show the measured miscalibration. Uncertainty is given as normalized
entropy. The left diagram is computed using ECE as MMCE does not involve binning.
ECE
ACE
UCE (ours)
calibration error in %
10
20
53
model A
model B
20
10
D
20
number of bins
number of bins
number of bins
Figure 4: Toy experiment with two random models A and in a binary classification task. UCE is less sensitive to number of bins used in the estimator and provides consistent ranking of the models. For results from multi-class experiments, see Fig.
ECE and MMCE, Class 2, however, is misclassified in 100% of the cases-and not 60 % expected. The miscalibration of class 2 is not reflected in ECE or MMCE. ACE is computed class wise and is able to capture this miscalibration. UCE based on the normalized entropy determine uncertainty and therefore incorporates the predictions of all classes. Fig. 3 visualizes these results.
Robustness Varying Number of Bins From reliable calibration metric, we would expect constant ranking of two models in comparison, independent of varying calibration parameters. However, ACE is highly sensitive to the number of bins and produces arbitrary rankings models. Fig. 4 highlights this by comparing two differently calibrated random models.
7.3 UCE REGULARIZATION
At optimal temperature (as suggested by Ashukha et al. (2020)). UCE and MMCE regularization considerably reduce miscalibration for employed calibration metric outperforming entropy regu- larization, with UCE achieving highest accuracy on (IFAR-100, SVHN and Fashion-MNIS" (see Tab. []. We want to stress out that UCE, in contrast to MMCE, was not specifically designed for the use as calibration regularizer (Kumar et al. 2018).
UCE regularization works best when computed classwise (in similar manner to ACE): UCE = Σ UCE(c), where UCE(c) is computed for predictions of class c. Consider the following binary classification example: batch with mainly samples from class 1 and few samples from class 2 are all predicted as class with high confidence. NLL further pushes the confidence of the predictions to 1.0, favoring overconfidence, whereas UCE is only reduced if the confidence of the overconfidently false predictions is reduced.
Why UCE Regularization Works
8

---

Under review as a conference paper at ICLR 2021
Table Revision of Table T Results from SWAG trained with entropy, MMCE and UCE regulariza- tion at optimal temperature (+T). We used the weighted MMCE (Kumar et al. 2018).
Regularization
Dataset
Acc.
ECE
ACE
UCE
MMCE
Brier
NLL
unregularized
CIFAR-10
94.3 %
3,8 %
3.8 %
3.6%
3.3%
0.10
0.28
Entropy+T
CIFAR-10
94.1 %
2.1
4.2 %
2.3%
1.1 %
0.10
0.25
MMCE+T
CIFAR-I0
92.0 %
0.4 %
1.6 %
0.8 %
0.1 %
0.12
0.24
UCE+T (ours)
C1
92.6 %
0.5 %
1.6 %
0.7 %
0.2 %
0.10
0.21
unregularized
CIFAR 100
68.3 %
21.8 %
22.0 %
25.7 %
18.3 %
0.52
2.26
Entropy+T
CIFAR-
68.1 %
2.9 %
12.3 %
3.7 %
2.1 %
0.44
1.41
MMCF+T
CIFAR-
67.7 %
1.3 %
11.0 %
2.1 %
0.5 %
0.43
1.20
UCE+T (ours)
CIFAR-100
70.9 %
2.4 %
10.4 %
1.1 %
1.2 %
0.40
1.10
unregularized
SVHN
96.8 %
2.10 %
2.16 %
1.89 %
1.82 %
0.05
0.19
+ntropy+T
SVHN
96.9 %
1.15 %
2.31 %
0.86 %
0.74 %
0.05
0.15
MMCE+T
SVHN
97.1
0.27 %
0.85 %
0.35
0.17 %
0.05
0.12
UCE+T (ours)
SVHN
97.1 %
0.38 %
0.92 %
0.38 %
0.14 %
0.05
0.12
unregularized
F-MNIST
94.7 %
3.97 %
3.96 %
3.85
3.60 %
0.09
0.29
Tntropy+T MMCE+T UCE+T (ours)
F-MNIST
94.7 %
1.86 %
4.28 %
2.13 %
0.96 %
0.09
0.24
F-NNIST
94.7 %
0.54 %
1.40 %
0.64 %
0.17 %
0.08
0.15
F-MNIST
94.8 %
0.52 %
1.75 %
0.63 %
0.11 %
0.08
0.16
ACKNOWLEDGMENTS
Acknowledgments withheld.
REFERENCES
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in-domain uncertainty estimation and ensembling in deep learning. In ICLR, 2020.
Christopher M Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In ICML, pp. 1613-1622, 2015.
Glenn Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1-3, 1950.
Chenyi Chen, Ari Seff, Alain Kornhauee,, and Jianxiong Xiao. Deepdriving: Learning affordance for direct perception in autonomous driving, In ICCV, 2015.
Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. Dermatologist-leve classification of skin cancer with deep neural networks. Nature, 542(7633): 2017.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In ICML, pp. 1050-1059, 2016.
Yarin Gal, Jiri Hron, and Alex Kendall, Concrete dropout. In NeurlPS, pp. 3581-3590, 2017. 2017.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. ICML, pp. 1321-1330, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, pp. 770-778, 2016.
Pavel Izmailov, Dmitrii Podoprikhín, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. Uncertainty Artificial Intelligence, 2018.
arXiv: 180.005407

---

Under review as a conference paper at ICLR 2021
Eric Jang, Shixiang Gu, and Ben Poole. Categorical Reparameterization with Gumbel-Softmaad In ICLR, 2017.
Alex Kendall and Yarin Gal. What uncertainties do we need bayesian deep learning for computer vision? In NeurIPS, pp. 5574-5584, 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Diederik Kingma and Max Welling, Auto-encoding variational bayes. In ICLR, 2014.
Durk Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameterizztioo trick. In NeurlPS, pp. 2575-2583, 2015.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images, 2009.
Meelis Kull, Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. In NeurlPS, pp. 12295-12305, 2019.
Ananya Kumar, Percy S Liang, and Tengyu Ma. Verified uncertainty calibration. In NeurlPS, pp. 3792-3803, 2019.
Aviral Kumar, Sunita Sarawagi, and Ujjwal Jain. Trainable calibration measures for neural networks from kernel mean embeddings. In ICML, pp. 2814, 2018.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. NeurIPS, pp. 6402-6413, 2017.
Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural networks. In ICML, pp. 2218-2227, 2017.
Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. In Bayesian Deep Learning Workshop, NeurlPS, 2016.
Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty deep learning. In NeurIPS, pp. 13132-13143, 2019.
Mahdi Pakdaman Naeini, Gregory F Cooper, and Milos Hauskrecht. Obtaining Well Calibrated Probabilities Using Bayesian Binning. In AAAI, pp. 2901-2907, 2015.
Jeremy Nixon, Michael W Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration deep learning. In CVPR Workshops, pp. 38 41, 2019.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanaa and Jasper Snock. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances Neural Information Processing Systems, volume 32, pp. 13991-14000 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Detito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In NeurlPS, pp. 8024-8035. 2019.
Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey Hinton. Regular- izing neural networks by penalizing confident output distributions. In ICLR Workshop, 2017. arXiv: 1ete1100106444
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 15:1929 1958, 2014.
Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, and Thomas Schön. Evaluating model calibration in classification. In AISTATS, volume 89, pp. 3459-3467, 2019.
Sida Wang and Christopher Manning. Fast dropout training. In ICMI, pp. 118-126 2013.
10

---

Lemma 1. Given a softmax oulput p with C entries and the most likely prediction i arg max p with likelihood p max p. Then, the remaining entries Piãgg are approximately uniformly dis- 1-6 Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 PROOFS
Proposition 1. The normalized entropy (uncertainty) H()) approaches the top-1 error in the limit of number of classes C if the model p is well-calibrated.
Proof. With Lemma I and mmxx p we rewrite the normalized entropy as
plog log C
(1 - p) log 1-A
H(p)
C-1
(19)
log
Now, in the limit of number of classes C
(1-1) log -e
lim H(p) lim C-+00
(20)
C+000
log C
logg( pp
log(C' - 1)
lim -(1 ())
(21)
C-000
log C log C
p)
(22)
The top-1 1 error equals (1 - p) if the model is perfectly calibrated in the sense of Eq.
(6
tributed with probability C-I-
Proof. This assumption is approximately correct (1) if P + 1 or o2 + co. = =
and q C-1
(1-6) Note that ğ and are not proper probability distributions ΣØ = Σの -1
(1) Consider KL[p as approaches 1:
C-1
lim 1-1
KL
lim
pg log
Pj qj
(23)
g-1
C-1
C-1
lim
py log P3
Σ j=l
ps log فj
(24)
j=1
(1 P)
lim
By log pJ (1 - p) log
(25)
C-1
(26)
(2) Let 2; be the logits of model trained with L2 regularization. The magnitude of the logits
xii| cannot become arbitrary large and due to the normalizing nature of softmax
exp
lim 1 C C
lim
(27)
== exp 2j
Alternatively, let z E A and x' K be two logit vectors with C < K. both models have been trained with L2 regularization, the magnitude of the logits |z4), |z/| cannot become arbitrary large. More specifically, A B C RR. Due to the normalizing nature of softmax, z' corresponds lower softmax temperature and as the temperature decreases with increasing number of classes, softmax approaches a uniform distribution (Jang et al. 2017).
11

---

Under review as a conference paper at ICLR 2021
A.2 BAYESIAN DEEP LEARNING METHODS
In the following, briefly describe common approximately Bayesian methods which we use in our experiments to obtain weight uncertainty.
Monte Carlo Dropout One practical approximation of the posterior is variational inference with
Monte Carlo (MC) dropout (Gal & Ghahramani (016). To determine model uncertainty, dropout variational inference is performed by training the model fw with dropout (Srivastava et al. 2014 4) and using dropout at test time to sample from the approximate posterior distribution by performing N stochastic forward passes per test sample (Gal Ghahramani] 2016) Kendall & Gal, (017). This is also referred to MC dropout. In MC dropout, the final probability vector of the predictive distribution is computed by MC integration:
p(x) = N Σ OSM (fw(()))
(28)
Gaussian Dropout Gaussian dropout was first proposed by Wang & Manning (013) and linked to variational inference by Kingma et al.] (2015). Dropout introduces Bernoulli noise during optimization and reduces overfitting of the training data. The resulting output y of layer with dropout is weighted sum of Bernoulli random variables. Then, the central limit theorem states, that is approximately normally distributed, Instead of sampling from the weights and computing the resulting output, we can directly sample from the implicit Gaussian distribution of dropout
y ~ ~N og)
(29)
with
By Elyy]] = Σ Wj
(30)
o Varlyn] =p/(1- -p) Σ w Wi
(31)
using the reparameterization trick (Kingma et al. 2015)
y My + ojey with 10.
(32)
Gaussian dropout is a continuous approximation to Bernoulli dropout, and in comparison it will better approximate the true posterior distribution and is expected to provide improved uncertainty estimates (Louizos& & Welling 2017). To obtain the final probability vector p(æ), we again use MC integration with N stochastic forward passes.
The dropout rate p is now learnable parameter and does not need to be chosen carefully by hand. In fact, p could be optimized w.r.t. uncertainty calibration, scaling the variance of the implicit Gaussian of dropout. A similar approach was presented by Gal et al. 2017) using the Concrete distribution (Maddison al., 2016, Jang al. 2017). However, we focus on metrics for measuring calibration and therefore fix in our subsequent experiments.
Bayes by Backprop Blundell et al. (0155) assume a Gaussian distribution with diagonal covariance matrix as variational posterior 7(w (),, parameterized by mean μ and standard deviation or, where {μ, }. A sample of the weights can obtained by sampling a multivariate unit Gaussian and shift it by y and scale it by Then, the network trained by minimizing
(()) (L[g( 1))lp))pp)))) - Eg log p( P(D|ww)]
(33)
In case a zero mean Gaussian prior, the first term can be implemented by weight decay. In contrast to Gaussian dropout, which operates the implicit distribution of the activations, Bayes by Backprop (BBB) directly operates on the weights. This doubles the number of trainable parameters in practice. MC integration is used to obtain the final probability vector p(æ).
12

---

Under review as a conference paper at ICLR 2021
SWA-Gaussian Stochastic weight averaging (SWA) uses stochastic gradient descent steps around local loss optimum of trained network and averages the weights WsSA + Σ w, of the model from each step (Izmailov et al. 20188) This explores the loss landscape and averaging helps find a better weight estimate than converging closely related to Bayes by Backprop (Maddox et al., 2019). It assumes Gaussian distribution with diagonal covariance matrix as approximate variational posterior. Instead of using backpropagation to directly optimize la and Ø, it fits Gaussian by using μ USSAA and
single local optimum. SWA-Gaussian (SWAG) is
ddng = diag( u² wŠŠAA,
W
1 T
Σ
(34)
This doubles the number of parameters at test time. The approximate Gaussian posterior results to N ((wswA, Mding) and MC integration with samples w; ~ N(wswA Eding) is used to compute final probability vector p(z).
Deep Ensembles Training multiple randomly initialized copies of a deep network by performing maximum posterior estimation and ensembling them to get multiple predictions for single input is not variational inference method. However, they have been reported to produce surprisingly useful uncertainty estimates in practice that are better calibrated (Lakshminarayanann Deep ensembles considerably increase the number of parameters at train and test time. We use deep ensembles as non Bayesian baseline for uncertainty estimation.
al. (017).
A.3 TRAINING SETTING
The base model implementations from PyTorch 1.6 (Paskke al. 2019) used and trained with following settings:
Adam optimizer with initial learn rate 3e-4 and 31 0.9, 82 = 0.999 and mini-batch size of 256 (Kingma & Ba 2015)
weight decay of lc-6
negative-log likelihood (cross entropy) loss
reduce-on-plateau learn rate scheduler (patience of 20 epochs) with factor of 0.1
additional validation set is randomly extracted from the training set (5, 000 samples)
ResNet-34 for CIFAR-10 and ResNet-50 for CIFAR-100 experiments
only the last linear layer is implemented in Bayesian manner for MC dropout, Gaussian dropout, Backroo and SWAG
the deep ensemble comprises fully individually trained networks
N 25 forward passes used Monte Carlo integration
in MC dropout and Gaussian dropout, dropout rate of 0.2 was used
in SWAG, learn rate of 3e-6 was used during weight averaging
Our code is available at: https//github.com/inkwwww
13

---

Under review as a conference paper at ICLR 2021
A.4
ADDITIONAL RESULTS
Bayes
Dataset
Accuracy
ECE
ACE
UCE
MMCE
Brier
NLL
MC Drop
CIFAR-10
93.6 %
4.3 %
4.3 %
4.0 %
3.8 %
0.11
0.31
Gauss Drop
CIFAR-10
93.2 %
4.4 %
4.4 %
4.1
3.8
0.11
0.31
BBB
CIFAR-10
93.3 %
4.6%
4.6%
4.4%
4.1 %
0.11
0.34
SWAG
CIFAR-10
94.4 %
3.7%
3.7%
3.5%
3.5 %
0.09
0.28
Ensemble
CIFAR-10
95.0% %
3.2%
3.2%
3.0%
2.8 %
0.08
0.22
MC Drop
CIFAR-100 100
66.9%
24.4 %
24.5 %
27.9%
20.6 %
0.55
2.55
Gauss Drop
CIFAR-100
66.5 %
24.5
24.7 %
28.2% %
20.7 %
0.56
2.64
BBB
CIFAR-100
65.1% %
24.9%
25.1 %
28.9% %
20.9 %
0.57
2.51
SWAG
CIFAR-100
68.3 %
21.8 %
22.1 %
25.7 %
18.3 %
0.52
2.26
Ensemble
CIFAR-100
72.5 %
19.2 %
19.4 %
22.5% %
16.1 %
0.45
1.82
Table Classification accuracy and calibration error results for different models on CIFAR-10/100. We used M = 15 bins where necessary. Here, all metrics provide the same ranking of models.
A.5
ADDITIONAL FIGURES
modrop
gouedlop
6
bib
ECE in
ACE in
UCE
5
1.0
1.5
2.0
2.5
3.5
4.0
1.5
2.0
2.5
3.0
3.5
1.0
2.0
2.5
3.5
4.0
temperature
temperature
temperature
0.11
MMCE in
Erier store
0.10
0.09
0.20
0.15
1.5
temperature
temperature
temperature
Figure 5: Calibration error vs. softmax temperature on CIFAR-10. All metrics provide inconsistent ranking of models over т. The metrics ECE, UCE and MMCE have narrow region in which the optimal temperature for all models can be found. They show more consistent ranking before and after the point of optimal temperature. This allows comparison of calibration of models if they are all over- or under-confident. However, metrics fail at comparing underconfident models to overconfident models. Even optimal temperature, Brier score and NLL fail at comparing calibration of models with different accuracy, as the metrics are always lower for models with better accuracy.
14

---

Under review as a conference paper at ICLR 2021
20
17.5
UCE in %
in
ACE in
15.0
12.5
10.0
0
1.5
2.0
2.5
3.0
3.5
40
4.5
1.5
2.0
2.5
3.0
3.5
4.0
4.5
1.5
2.0
2.5
3.0
3.5
4.0
4.5
temperature T
temperature 7
temperature
1.8
0.50
Brier score
1.6
MMCE in
10
0.45
1.2
0.10
1.0
1.5
2.0
3.0
3.5
1.5
2.5
3.5
4.5
1.5
2.0
3.0
3.5
4.6
temperature
temperature
temperature
Figure 6: Calibration error vs. softmax temperature on CIFAR-100.
20
20
15
15
ACE in %
ECE in
10
UCE
0.5
1.0
1.5
2.0
2.5
3.0
0.5
1.0
1.5
2.0
2.5
3.0
0.5
1.0
1.5
2.0
2.5
3.0
temperature
temperature 7
temperature
20
0.4
1.25
MMCE %
15
0.3
10
0.2
NLL
0.75
Brier
0.50
0.1
0.25
0.9
0.5
1.0
1.5
2.0
2.5
3.0
0.5
1.0
1.5
2.0
2.5
3.0
0.5
1.0
1.5
2.0
2.5
temperature
temperature
temperature
Figure 7: Calibration error vs. softmax temperature from SWAG trained with different regularization on CIFAR-10. Both MMCE and UCE regularization lead less overconfident models and reduce miscalibration (optimal temperature is closer to r = 1). Entropy regularization leads to underconfident models and is not effective as MMCE and UCE regularization on CIFAR- 10. MMCE and UCE regularization at optimal temperature outperform entropy regularization at optimal temperature for all metries except Brier score.
15

---

Under review as a conference paper at ICLR 2021
20
25
15
20
UCE in %
ECE in
ACE in
15
0..5
1.0
1.5
2.0
2.5
3.0
0.5
1.0
1.5
2.0
2.
3.0
0.5
1.0
1.5
2.0
2.5
3.0
temperature
temperature 7
temperature
20
0.8
15
Brter score
0.6
MMCE
10
NLL
0.4
5
0.2
0.5
1.0
1.5
2.0
2.5
3.0
0.5
1.0
2.0
2.5
3.0
0.5
1.0
1.5
2.0
2.5
temperature
temperature
temperature
Figure 8: Calibration error vs. softmax temperature from SWAG trained with different regularization CIFAR-I00. In this experiment, entropy regularization without temperature scaling (r 1) was surprisingly effective and outperforms MMCE and UCE regularization. However, at optimal temperature both MMCE and UCE regularization outperform entropy regularization for all metries.
H(p))
MCDrop
0.3
GaussDrop
BBB
top-1 error
SWAG
0.2
Ensemble
1 max p
MCDrop
0.1
GaussDrop
BBB
SWAG
Ensemble
0.0
1.0
0.8
0.6
0.4
0.2
0.0
uncertainty threshold
Figure 9: Rejection results on CIFAR-100 100 for decreasing uncertainty threshold comparing H((p) and max p as uncertainty metric. both cases, the top-1 error decreases monotonically with decreasing threshold.
16

---

Under review as a conference paper at ICLR 2021
CIFAR-I0/ReSNet-44
CIFAR-100/ResNett:0
modrop
modrap
10P
ECE
samples per in %
ECE
10
ACE
ACE
10"
10
10-2
10
12
15
10
15
bin
bin
goussdrop
goussdrop
10P
UCE
samples per in
107
UCE
10'
samples
10
bin
bin
bhh
bbh
10F
metnc
metri
10a
U.E.E
UCE
Samples per bin
10'
10
10
metri
metrc
%
ECE
Samples per bin in %
ECE
10'
ACE
ACE
10
10
1.1
12
33
14
15
10
14
15
ensemble
ensemble
samples per in N6
10'
ACE
I0"
10
10
14 15
bin
bin
Figure 10: Binning estimator sample distribution for ResNet-34 CIFAR-10 (left) and for ResNet- 50 on CIFAR-100 (right) with M = 15 bins. ECE and UCE use fixed bin widths and ACE uses an adaptive binning scheme. The number of samples per bin for ECE and UCE are similar on CIFAR-10. On CIFAR-ION 100, UCE favors fewer bins, which makes UCE more insensitive to the total number of bins. Due to the adaptive binning, ACE is highly sensitive to the bin count.
17


Reviews:
[{"review_id": "uuO5BygoZt-", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new calibration error measurement named UCE (Uncertainty Calibration Error) for deep classification models. It consists in doing a calibration in order to achieve \"perfect calibration\" (i.e., the uncertainty provided is equivalent to the classification error at all levels in [0, 1]), relying on normalized entropy for multiclass classification. This UCE is well justified for classification problems with several classes to process, where the entropy is demonstrated to be asymptotically equivalent to the classification (top-1) error. A point with this UCE metric is that is has some interpretability properties in terms of its value, and is said to be robust to the number of bins used.\n\nThe proposed metric is well explained, and justified, although I am wondering how well stands the assumption that the normalized entropy approaches the top-1 error for reasonable number of classes (e.g. C=10, as with CIFAR-10, or C=100, as with CIFAR-100). The properties presented are interesting.\n\nIf found the experiments to be well aligned to evaluate the approach, although limited in terms of dataset used (only CIFAR-10 and CIFAR-100), a greater variety of datasets would be more convincing in the of overall good performances of the approach, especially if datasets with a varied number of classes can be tested. Moreover, looking at the results in detail (Table 1), UCE does not appear to be particularly strong, having a worse calibration than ECE and ACE on CIFAR-10, but slightly better on CIFAR-100, assuming that we want it to be increased to reach the real error rate obtained. Moreover, the presentation of the results in Table 1 is messy: it gets difficult to match the calibration error with the accuracy, providing the classification error instead of accuracy would help to make a direct comparison with calibration error. Moreover, why the last two columns in Table 1 (Brier and NLL) are provided as floating-point values instead of percentages as with the other columns. That's unnecessary confusion that should be fixed.\n\nOverall, I found the paper to be correct, relatively well written. I think that more room should have been given to experimentations, like with other datasets and with more space for OoD rejection and detection. Conversely, I am not sure of the relevance of providing all the detailed information on Bayesian methods in the second part of Sec. 2. It can be presented in a more concise way, as it uses a lot of space to explain well-known approaches.\n\nIn terms of potential impact of that paper, I still need to be convinced. What can tell me that this is just not yet another calibration metric. I think that the paper can have been made stronger on that aspect.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "C7VCzjPHm2j", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The work addresses an important problem in the study of uncertainty estimation: how does one compare model uncertainty at differing accuracy levels? The work proposes a novel uncertainty metric, relates this to existing methods and provides robust evaluation of the various merits of this approach. The paper is easy to follow. \n\nI have the following concerns with the work:\n1. Regarding the use of UCE as a regularizer: how is the described behaviour of UCE different from the NLL loss? The NLL loss should penalize highly confident incorrect predictions and strives for confident predictions in high accuracy. What does the UCE regularizer add here?  Can table 2 include a non-regularized baseline as well to study this? In appendix A.3. it is said that UCE performs on par without regularization; then what is the point of proposing UCE as a regularizer?\n2. What is the point of proposing the use of the normalized entropy as a thresholding factor for OOD detection? It seems that vanilla entropy would behave exactly the same. Is this considered to be a novel contribution in this work? \n3. Why is figure 2 (right) completely flat for UCE? Are there values not shown here where calibration error does change? Perhaps this should be included in the plot. \n4. Am I correct to say that max-p-based metrics might be preferable in very large-class problems such as language models? The paper does not discuss the computational tradeoffs of the method, and I believe this should be included. \n5. It appears that the experiment section does not provide much evidence  that this metric is favourable in selecting the best model for a downstream task where uncertainty is needed. This could be evaluated by e.g. an active learning problem. I believe it makes sense to include such an experiment. Right now, Table 1 and the accompanying discussion does not convince me that UCE is somehow more beneficial.\n\nOverall, the work has merit and of interest to the community. However, the proposal of the use of the metric as a regulariser and a OOD scoring function seems unproductive and if so, distracts from the core contribution. This core contribution is understudied in the work. The work would benefit from more analysis into the computational tradeoffs, and evaluation of the signal that the proposed metric provides on model selection for downstream uncertainty tasks.\n\nNitpick:\n- Table 1 could benefit from a vertical bar between the two datasets to clarify that the numbers are not comparable. \n\nUpdate:\nAlthough the paper has improved, I still vote for rejection. The new insight of binary-classwise v/s multiclass UCE as a regularizer seems poorly explored in the paper and would benefit from closer study. This appears to be the basis of the improved results in table 1.", "extra_scores": {}, "extra_text": {}}, {"review_id": "8nTzC1CtA26", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Update: After reading the other reviews and responses, and in light of the authors' updates to the paper, I have increased my score to a 6.\n\nThis paper proposes a new metric for uncertainty calibration, based on comparing the entropy of the marginal class probabilities conditioned on predicted class with the entropy of the predicted probabilities. The metric avoids the failure mode of ECE, where predicting the relative frequencies of classes results in perfect calibration, and can be used as a regularizer in a loss function. The paper demonstrates that regularization with UCE yields better-calibrated uncertainty on CIFAR predictions without sacrificing accuracy.\n\nThe paper is well-written and well-motivated. I’m uncertain as to its novelty. In particular, entropy as a basis for uncertainty estimation is well-explored (and was used as a baseline in (Lakshminarayanan et al., 2017) as well as Jie Ren et al., “Likelihood ratios for out-of-distribution detection,” NEURIPS 2017). It’s unclear what the results in Figure 3 contribute in light of these baselines (besides the normalization by the constant C).\n\nWhich loss function was used to produce the results in Table 1? If it’s the loss in (25), it would also be useful to see calibration metrics for NLL loss alone.\n\nFigure 2 shows strong sensitivity of ACE to the number of bins. Quantile ECE (an ECE metric with bins defined by quantiles instead of fixed-width) often shows less sensitivity -- was this metric considered as well?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2nu6qeuQS_B", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Thanks for the interesting paper!\n\nSummary\n\nThe authors focus on the important problem of improved calibration measures as compared to the (now fairly standard) expected calibration error (ECE). More specifically, they define a new \"Uncertainty Calibration Error\" (UCE) metric based on the normalized entropy of the predictive distribution, rather than the max probability (as in ECE). The metric still uses fixed-width binning (as in ECE), and they motivate the interpretation w.r.t. perfect calibration based on a theoretical limit. They provide a set of experiments to show the differences in model ranking, sensitivity to number of bins, etc. between UCE, ECE, etc. on various models.\n\nStrengths\n\n- As noted in previous literature (and referenced in this paper), improved measures of calibration is an important research area.\n- The authors provide a great background section to place their research within the broader area of uncertainty research.\n- The experiments on sensitivity to the number of bins is informative and highly relevant in the context of previous literature (e.g., Nixon et al. (2019)) where it has been shown that ECE is particularly sensitive to this setting.\n\nWeaknesses\n\nAs I have noted in more detail down below, I believe this paper suffers from a few weaknesses. Overall, at the end of the paper as a reader, I'm still left with questions of whether UCE is truly a better calibration metric. As noted above, the insensitivity to number of bins is great and an improvement on ECE and ACE. However, I don't believe the remaining experiments make a strong case that the metric (1) provides a better measure of calibration, (2) yields consistently improved model performance when used as a regularizer (though it's interesting that it can be used as one!), or (3) allows for improved model selection. Additionally, I believe its interpretability is limited. As noted below, the experiments have mixed results or make comparison claims that detract from the overall message, which I find troubling from an experimental rigor standpoint. Furthermore, I think this paper would benefit from experiments that are set up such that they can directly measure and compare the ability of the metrics to measure calibration error.\n\nRecommendation\n\nGiven the above strengths and weaknesses, I'm currently inclined to suggest rejection of the paper in its current form. However, I think this could be a great paper and as a community I don't believe we have yet to devise the perfect calibration metric -- perhaps this could be it! I would highly recommend the authors push on the points above.\n\nAdditional comments\n\n- p. 3, 4: It could be informative to includes notes about NLL and Brier score being strictly proper scoring rules (Gneiting & Raftery, 2007; Parmigiani & Inoue, 2009) that theoretically should be maximized only when the forecaster emits the distribution they believe to be true, and thus should, in theory, be well-calibrated asymptotically. However, we indeed know from Guo et al. (2017) that empirically, models can still overfit, leading to poor calibration.\n- p. 4: The definition of perfect calibration can be traced back to Brier (1950), and, unlike ECE, is not limited to only the max predicted probability. Rather, for any predicted probability $p_k$ for class $k$, the probability that the true class is class $k$ should be equal to $p_k$ for all $p_k$ and all $k$. That is, $\\mathbb{P}(Y = k | P_k = p_k) = p_k, \\forall p_k \\in [0, 1], \\forall k \\in [1, K]$.\n- p. 5: UCE is based on an argument that normalized entropy approaches the top-1 error in the limit of number of class going to infinity. While this is interesting theoretically, this assumption seems too strong for empirical settings, and I think this affects the interpretability of the metric as claimed in the conclusion.\n- p. 6, 7, Section 5.1: This section (and Figures 4 & 5 in the appendix) make claims that NLL and Brier score \"fail at comparing calibration of models with different accuracy, as the metrics are always lower for models with better accuracy\". I find this argument both surprising and confusing in terms of motivation. As strictly proper scoring rules, they should indeed have lower values for better probabilistic models. Although accuracy is a non-proper scoring rule, it should still correlate well with those strictly proper rules, so it is expected that the better models with lower NLL / Brier score will (typically with some variance) have higher accuracy (variance being due to the non-proper nature of accuracy). All strictly proper scoring rules can be decomposed into calibration and refinement terms (Parmigiani & Inoue, 2009; DeGroot & Fienberg, 1983), but in the non-decomposed setting, it is not expected that these rules would directly measure calibration. Therefore, given the focus on calibration measures, I'm confused as to the motivation behind comparing to NLL & Brier score directly (beyond the overconfidence analysis from Guo et al. (2017)) as a means of motivating the usefulness of UCE.\n- p. 8, 12: In the regularization experiment, different regularization approaches are being compared in terms of calibration, but it's difficult to assess the results. In Table 2 (which needs an additional entry for the non-regularized result from Table 1), UCE regularization appears to improve accuracy, NLL, and Brier score over the non-regularized baseline. Interestingly though, NLL, Brier score, ECE, ACE, UCE, and MMCE (i.e, all metrics other than accuracy) point towards entropy-regularization being superior. It does result in a lower accuracy than UCE reg and the baseline, but by the other metrics (including the strictly proper scoring rules NLL and Brier score), it produces a better probabilistic model. For CIFAR-10 UCE reg is worse than the other regularization methods and the baseline.\n- p. 8: Rejection & OOD Detection: This has been studied previously for unnormalized entropy, which should yield the same results. See, e.g., Malinin & Gales (2018), Ren et al. (2019).\n\nMinor\n\n- p. 3: s/as non Bayesian/as a non-Bayesian/\n- p. 7: Figure 1 is too small.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work proposes a novel metric for measuring calibration error in classification models.

Pros:
* Novel calibration metric addressing limitations of previously used metrics such as ECE

Cons:
* Limited experimental validation on CIFAR-10/CIFAR-100 only
* Unclear impact beyond proposing a new calibration metric
* Unclear value of using the proposed UCE metric for regularization and OOD detection

All reviewers appreciate the aim of the work to produce a calibration metric that addresses shortcomings of commonly used existing metrics such as expected calibration error (ECE), which is known to be sensitive to discretization choices.  However, all reviewers remain in doubt whether the proposed metric (uncertainty calibration error, UCE) is truly a better metric of calibration than previous proposals.  This doubt comes from two sources: 1. limited experiments that do not convincingly show the usefulness of UCE; and 2. interpretability of UCE not being as intuitive to the reviewers.  The experiments also use UCE as regularizer but the benefit of doing so over simple entropy regularization is not clear.

Overall the work is well-motivated and written and the proposed UCE measure is interesting.  However, the reviewers remain unconvinced of the claimed benefits and the potential impact for measuring or improving calibration.
Decision: Reject