Improved Uncertainty Post-Calibration via Rank Preserving Transforms
Authors: Yu Bai
Tengyu Ma
Huan Wang
Caiming Xiong
Venue:  (2021)
Keywords: uncertainty quantification
calibration
temperature scaling

Abstract: Modern machine learning models with high accuracy often exhibit poor uncertainty calibration: the output probabilities of the model do not reflect its accuracy, and tend to be over-confident. Existing post-calibration methods such as temperature scaling recalibrate a trained model using rather simple calibrators with one or few parameters, which can have a rather limited capacity. In this paper, we propose Neural Rank Preserving Transforms (NRPT), a new post-calibration method that adjusts the output probabilities of a trained classifier using a calibrator of higher capacity, while maintaining its prediction accuracy. NRPT learns a calibrator that preserves the rank of the probabilities through general monotonic transforms, individualizes to the original input, and allows learning with any loss function that encourages calibration. We show experimentally that NRPT improves the expected calibration error (ECE) significantly over existing post-calibration methods such as (local) temperature scaling on large-scale image and text classification tasks. The performance of NRPT can further match ensemble methods such as deep ensembles, while being much more parameter-efficient. We further demonstrate the improved calibration ability of NRPT beyond the ECE metric, such as accuracy among top-confidence predictions, as well as optimizing the tradeoff between calibration and sharpness.

Under review as a conference paper at ICLR 2021
IMPROVED UNCERTAINTY POST-CALIBRATION VIA RANK PRESERVING TRANSFORMS
Anonymous authors Paper under double-blind review
ABSTRACT
Modern machine learning models with high accuracy often exhibit poor uncer- tainty calibration: the output probabilities of the model do not reflect its accuracy. and tend to be over-confident. Existing post-calibration methods such as tem- perature scaling recalibrate trained model using rather simple calibrators with one or few parameters, which can have a rather limited capacity. In this paper, we propose Neural Rank Preserving Transforms (NRPT), new post-calibration method that adjusts the output probabilities of a trained classifier using a calibra- tor of higher capacity, while maintaining its prediction accuracy. NRPT learns a calibrator that preserves the rank of the probabilities through general mono- tonic transforms, individualizes the original input, and allows learning with any loss function that encourages calibration. We show experimentally that NRPT improves the expected calibration error (ECE) significantly over existing post- calibration methods such as (local) temperature scaling on large-scale image and text classification tasks. The performance of NRPT can further match ensemble methods such as deep ensembles, while being much more parameter--fffiient. We further demonstrate the improved calibration ability of NRPT beyond the ECE metric, such as accuracy among top-confidence predictions, as well as optimizing the tradeoff between calibration and sharpness.
INTRODUCTION
Modern machine learning models such deep neural networks have achieved high performance on many challenging tasks, and have been put into production that impacts billions of people (LeCun al., 2015). is increasingly critical that the outputs of these models are comprehensible and safe to use in downstream applications. However, high-accuracy classification models often exhibit the failure mode of miscalibration: the output probabilities of these models do not reflect the true accuracies, and tend to be over-confident (Guo et al., 2017; Lakshminarayanan et al., 2017). As the output probabilities are typically comprehended as (an estimate of) true accuracies and used in downstream applications, miscalibration can negatively impact the decision making, and is espe- cially dangerous in risk-sensitive domains such as medical AI (Begoli et al., 2019; Jiang et al., 2012) self-driving cars (Michelmore al., 2018). It is important question how properly calibrate these models so as to make the output probabilities more trustworthy and safer to use.
Existing methods for uncertainty calibration can roughly be divided into types. Diversity-based methods such as ensembles (Lakshminarayanan et al., 2017: Wen et al., 2020) and Bayesian net- works (Gal & Ghahramani, 2016; Maddox et al., 2019; Dusenberry al., 2020) work by aggre- gating predicted probability over multiple models or multiple times on a randomized model. These methods are able to improve both the accuracy and the uncertainty calibration over single deter- ministic model (Ovadia et al., 2019). However, deploying these models requires either storing all the ensemble members and/or running multiple random variants of the same model, which makes On the other hand, post-calibration methods work by learning a calibrator on top of the output probabilities (or logits) of an existing well-trained model (Platt al., 1999; Zadrozny & Elkan, 2001; 2002; Guo et al., 2017; Ding et al., 2020). For a K -class classification model that outputs logits 2 = =2(x) E K post-calibration methods learn calibrator f R R using additional holdout data, so that f2) is better calibrated than the original 2. The architectures of such calibrators are typically simple: A prevalent example is the temperature scaling method which learns fr(z) z/T with single trainable parameter T. by
them memory-expensine and runtime-inefficiee

---

Under review as a conference paper at ICLR 2021
Train
Val NLL
Val
Val Accuracy
0,959
0.950
0.30
80.4
6.925
0.925
0.900
0.900
0.08
80.3
0.675
0.875
0.859
0.850
0.06
80.2
0.825
0.825
0.800
Temp Scaling
0.800
0.04
80.1
0.775
0.775
0.02
80.0
0.750
0.750
Figure 1: Post-calibration training curves on a WideResNet-28- on CIFAR-100 100. Temperature scaling minimizes the training and validation NLL reasonably well (and improves the ECE), but still underfits the NLL. Matrix scaling learns a higher-capacity matrix calibrator and minimizes the training NLL better, but does not improve the ECE since the calibrator does not maintain the accuracy and is encouraged improve the accuracy instcad of calibration. Our Neural Bank-Preserving Transforms (NRPT) learns a higher-capacity calibrator that preserves the accuracy, and improves both the training/validation NLL as well as the ECE.
minimizing the negative log- log-likelihood (NLL) loss on holdout data. Such simple calibrators add no overhead to the existing model, and is empirically shown to improve the calibration significantly on a variety of tasks and models (Guo et al., 2017).
Despite its empirical success, the design of post-calibration methods is not yet fully satisfactory: In practice, simple calibrators such as temperature scaling often underfit the calibration loss on its training data, whereas more complex calibrators can often overfit-see Figure for quantitative illustration of this effect. While the underfitting of simple calibrators perhaps due to their lim- ited expressive power, the overfitting of complex calibrators is also believed to be natural since the holdout dataset used for training the calibrators are typically small (c.g. a few thousands of examples). One concrete example is the matrix scaling method which learns a matrix calibrator fw.a(2)= = W= b involving O(K2) trainable parameters. When K is large, matrix scaling often tend to overfit and hurt calibration, despite being strict generalization of temperature scaling (Guo al., 2017). It is further observed that the overfitting cannot be easily fixed by applying common regularizations such as Lg on the calibrator (Kull et al., 2019). These empirical evidence seems to suggest that complex calibrators with a large amount of parameters are perhaps not recommended in designing post calibration methods.
In this paper, we show that in contrast to the prior belief, large calibrators do not necessarily overfit; it is rather lack of accuracy constraint on the calibrator that may have caused the overfitting. Observe that matrix scaling, unlike temperature scaling, is not guaranteed to maintain the accuracy of the model: it applies general affine transform + W + b on the logits and can modify their rank (and thus the predicted top label), whereas temperature scaling is guaranteed to preserve the rank. When trained with the NLL loss, calibrator that does not maintain the accuracy may attempt to improve the accuracy at the cost of hurting (or not improving) the calibration.
Motivated by this observation, this paper proposes Neural Rank- Preserving Transforms (NRPT), method for learning calibrators that maintain the accuracy of the model, yet are complex enough for yielding better calibration performance than simple calibrators such as temperature scaling. Our key idea is that a sufficient condition for the calibrator to maintain the accuracy is for it to preserve the rank of the logits: any mapping that preserves the rank of the logits will not change the predicted top label. We instantiate this idea by designing a family of calibrators that perform entrywise monotone transforms on each individual logit (or log-pre pogg-obbbbillt):: for a K-class classification problem, NRPT scales each logit as 2; + f(2,, r), where 2 E R is the i-th logit (1 < < K). x E IR is the original input features, and f: IR x Rdd > TR monotonically increasing in its first argument but otherwise arbitrary. As f is monotone, we have f() < f22, 2) if 21 S 22, and thus f preserves the rank of the logits. This method strictly generalizes temperature scaling (which uses f(2,, (T) = 2/TT) and local temperature scaling (which uses 2(),,) xx)) (Ding et al., 2020). The fact that ff can depend on x further helps improve the expressivity of f and allows great flexibility in the architecture design. We compare our instantiation of NRPT against temperature scaling and matrix scaling in Figure 1, in which we see that NRPT is indeed able to fit the training loss better than temperature scaling and does not suffer from overfitting.
2

---

Under review as a conference paper at ICLR 2021
Our contributions We propose Neural Rank-Preserving Transforms (NRPT), improved method for performing uncertainty post-calibration on a trained classifier while maintaining its ac- curacy (Section 3). NRPT learns calibrators that scale the logits using general monotone transforms, are individualized to the original input features, and allow learning with any calibration loss func- tion (not restricted those that correlates with the accuracy). We show experimentally that NRPT improves the expected calibration error (ECE) significantly over existing post-calibration methods on large-scale image and text classification tasks such as CIFAR-IO0, ImageNet, and MNLI (Sec- tion 4.1). NRPT can further match diversity- -aaee methods such as deep ensembles, while using much less number of additional parameters. We further demonstrate the strong calibration abil- ity of NRPT beyond the ECE, by showing that it improves on the accuracy among top-confidence predictions, as well as the tradeoff' between ECE and sharpness of prediction (Section 4.2).
Due to the space constraint, we defer the discussions of additional related work to Appendix A and additional experimental details and results to the later Appendices.
BACKGROUND ON UNCERTAINTY CALIBRATION
We consider K K-class classification problems where X E R is the input (features), Y [K] {1...., }} is the true label, and (X, Y) follows some underlying joint distribution. Let p RR - AK be prediction model (for example, neural network learned from data) that maps inputs to probabilities, where AK := {(I,.... .pk.......)))))) )x : Pi 2 0, 0, 13 is the set of all probability distri-
butions on [K]. We say is perfectly calibrated if
P(Y k ((X) p) Pk for all p AK, & [K].
In other words, a model is perfectly calibrated if when the model predicts p(X) = p. the conditional distribution of Y is exactly p. is difficult to evaluate perfect calibration from finite data, as for almost all p we do not receive samples that satisfy the exact conditioning p(x)) p. This motivates considering alternative scalar metrics for calibration that can be estimated from data.
The Expected Calibration Error (ECE) is commonly used metric that measures calibration through grouping examples according to the confidence (i.e. the top predicted probability) (Naeini et al., 2015; Guo et al., 2017). Let {( ((),,:)) y:) be the evaluation dataset on which we wish to evaluate the calibration of a model p. Define the intervals Im (m 37 where M > 0 is a (fixed) number of bins, and partitions the examples into M bins according to confidence: Bm {i: max; f(x:)s Im]. Define the accuracy and confidence within Bm
ECE
aco(Bm) = Bm B.
Σ 1f arg max p(z;)x = y and conf(Bm) = Bm iE....
Σ max p(z,))
k
The ECE is then defined as the (weighted) average difference between accuracy and confidence:
ECE(1) = [ml] |aco (m.) - conf(B,..)
The ECE is a sensible calibration metric since it is a binned approximation of the top-label cal- ibration error (TCE) that measures the difference between accuracy and confidence under exact conditioning:
max TCE()) == E p arg max x(X) = max ((X)s
max P( X)x
Recent work shows that the ECE has an inherent positive bias and proposes the Debiased ECE that approximately removes this bias using Gaussian bootstrapping (Kumar et al., 2019):
Debiased ECE
DECE(P) = ECE(P)- -
ER
Bmll n
conf(Bm) Rm - ECE(P)
where Rm, N(aco Bm,); acei Byn accc (Bn)) Kumar et (2019) showed that the debiased ECE B a typically more accurate estimator the TCE than ECE, especially when the TCE is relatively small. In our experiments, we use both the ECE and the debiased ECE for evaluating calibration.
3

---

Under review as a conference paper at ICLR 2021
The Negative Log-Lin Likelihkooo (NLL), typically used as loss function for training classifiers, is also a measure calibration:
NLL
(L - Σ log p(ri)
NLL is proper scoring rule (Lakshminarayanan et al., 2017) in the sense that the population min- imizer over all possible p is achieved at the ground truth conditional distribution p. (Hastie et al., 2009). In general, the NLL measures the distance between p and p., and is thus a joint metric of accuracy and calibration.
Predictive entropy (sharpness) While of a model, these two metrics alone do not fully guarantee a proper uncertainty quantification. For example, any high-accuracy model can be calibrated in "trivial" way such that the ECE becomes exactly 0, by mapping the confidence on all examples to be equal to the (overall) accuracy of the model, and rescaling the non-top probabilities accordingly. order to prevent such trivial calibra- tion, we additionally measure the sharpness of the predictions using the predictive entropy (Laksh- minarayanan et al., 2017):
mostly concered about the accuracy and calibration
n
p(x,)a log p(x.))))))
Lower predictive entropies indicate sharper predictions (i.e. predictions closer to delta distributions than the uniform distribution). In general, the predictive entropy is not necessarily related to the cal- ibration; however, for models that have the same accuracies, we observe that the predictive entropy is typically negatively correlated with calibration salibration--sarper predictions are usually less calibrated.
3 RANK PRESERVING TRANSFORMS
We now introduce our main algorithm Neural post-calibration on trained classifiers. Throughout this section, consider K-class classification problems (K" 2 2), and let : Rd + R K denote the input-to-logit mapping of a trained classi- fier. The predieted probabilities of the model is the softmax of the logits: p(x) OSM (2(x) (exp(2(<)))) ( exp(2(x);)]..
Transforms (NRPT) for performing
Temperature Scaling method for post-calibration. Temperature scaling recalibrates a model by scaling down the logits using a single temperature parameter T > 0:
We begin by reviewing temperature scaling, simple yet strong baseline
fr(3) = fr(E Ex) = (3/ 2x 2/T.
and using (E)) calibrated probabilities. The parameter T is typically learned by mini- mizing the NLL loss on a hold-out calibration dataset. Temperature scaling clearly preserves the rank of the logits, and is observed to improve both the NLL and the ECE test data by learning temperature parameter that is typically above one (Guo et al., 2017) on large, over-confiden models. However, as have seen in Figure 1, temperature scaling often does not minimize the (training) NLL on the calibration dataset well due to its limited model capacity.
Individualization The first building block of our algorithm to individualize (or localize) tem- perature scaling, an idea recently proposed in the Local Temperature Scaling (LTS) method Ding al. (2020): Each input rank of the logits, but can substantially increase the capacity of the calibrator as now the temperature can adapt to the raw input. Formally, we calibrate the model by scaling down the logits using an individualized temperature model To(x) 0:
Bd can have its own temperature T(x) 0. This still preserves the
fr. (2)
We require the temperature model (z) always output positive scalar, but can otherwise have arbitrary architectures. In our experiments, we find LTS with the right choice of architecture can consistently outperform temperature scaling.
4

---

Under review as a conference paper at ICLR 2021
Rank-preserviie transforms via general monotone calibrators We now introduce our key idea of performing general monotone calibration, which preserves the rank of the logits and can be even more flexible than local temperature scaling. Our key observation is that the fundamental property that allows (local) temperature scaling (3) and (4) to preserve the rank is their monotone property:
2, -
guarantees f(²), S f(2)s-
Further, this is satisfied for (3) and (4) since the calibrator applies an entrywise, monotonically increasing function to each logit. Motivated by this, we consider general monotone calibrators of the form
(o(
m)], g(2, x) is monotonically increasing in z for all ar.
(5)
Observe that both temperature scaling (g(2;: = 2: TT) and local temperature scaling (g(2;; .) 2// (zr) as special cases of (5), and under this perspective are still limited in capacity as the g used in both cases are linear 2 given any x. To let the calibrator be more expressive, we would rather like to learn an arbitrary y under the monotonicity constraint.
Instantiation via monotone two-layer networks We now explain how we design a function class (2;22)) that is monotone in for any æ and not too restricted in its expressivity. Existing techniques for building such monotone function classes include either classical non-parametric methods such as isotonic regression (Barlow & Brunk, 1972), or sophisticated tricks such as parametrizing the de [2; z) using non-negative neural network (Wehenkel & Louppe, 2019). However, for the purpose designing calibrators, we prefer a simpler parametric class that enables efficient gradient-based learning. We acheive this by using a class of two-layer neural networks in with coefficients depending on r:
derivative d
M go (0(3)))== = Σ agي
(z: ba, (z)))
To, (x)
(6)
1=1
where aj 2 0. To, (x) 0, and Φ is monotonically increasing nonlinearity.
It is straightforward see that (6) is guaranteed to be monotonically increasing in z; for any as desired. Further, by choosing a proper ø and using a large number of neurons M, (6) can express a fairly large class of monotonic functions in 2, for any fixed z. We also note that (6) recovers local temperature scaling (4) if we take (0t) = to be the identity mapping (as y becomes linear in z in that case), and therefore has a strictly higher expressivity.
Architectural choices For implementing the calibrator (5) and (6), in theory one is free to use any architecture as long as To, (x) > O is guaranteed. However, we observe experimentally that reusing the representation of the trained classifier and weight sharing can help improve the calibration performance. In all our experiments, we choose [To, (I)) be, (z)] to be a two-layer neural network of the last hidden layer (the pre-logit layer) of the trained classifier, with shared weights:
(Te, (x)) Texe (x)) Otemp (Acmpp o (WF (x)+++))) b)).
[be, (x),..., bese (x)) = Abas (W (x))++++ (b)+)))
where A : RRd - Rehis is the last hidden layer of the trained classifier, and W E N b E R.N. and Atemp; Abius IR MxN are the trainable parameters. We further a strictly positive nonlinearity Otemp to guarantee the temperatures are positive and not too small.
Flexible loss functions Our final observation is that our calibrator (5) and (6) can trained with not only the NLL loss, but any other loss function that encourages calibration. This is only possible for rank-preserving calibrators-calibraa using a loss that jointly encourages high accuracy and calibration (such as the NLL), otherwise the calibrator may hurt the accuracy for the sake of getting good calibration.
that do not preserve the accuracy have to be trained
We specifically propose to use the ECE (1) directly Notice that even though the ECE is non-smooth in the model outputs (due to the binning and the non-smoothness of the argmax prediction rule), it still has non-trivial gradient since the average
loss function for training the calibrator.
5

---

Under review as a conference paper at ICLR 2021
confidence conf(Bm) is differentiable with respect to the model outputs. We find that training with the ECE loss can often result in better calibration, at the cost of reducing the sharpness (see Section 4) for details). We remark that Kumar et al. (2018) has considered training the MMCE (maximum mean calibration error), a kernelized version of the ECE loss; however, we are not aware of prior work that has considered training the ECE directly to the best of our knowledge.
Summary of algorithm We summarize our NRPT algorithm follows: Build calibrator f0 using the rank aane-rreeggiig transform go x) defined in (5), (6), (7); Train the calibra- tor via minimizing any desired loss (e.g. NLL ECE) on holdout calibration dataset; Output the calibrated model p(x)= = (fo((((
4 EXPERIMENTS
4.1 CALIBRATION ON IMAGE AND TEXT CLASSIFICATION
Tasks and models
We perform calibration experiments on the three benchmark tasks:
CIFAR-100, splitted into 45K/5K/10K as train/calibrationn/aa data. We train a WideResNet-28- 10 (Zagoruyko & Komodakis, 2016) on the train split, achieving 80.36% accuracy.
ImageNet ILSVRC2012 (Deng al., 2009), splitted into 1.1M/100K/50K as train/calibration//all
data. We train a WideResNet-55022 on the train split, achieving 76.27% top-1 acccuracy.
MNLI, one of the largest text classification tasks in the GLUE benchmark (Wang et al., 2018), splitted into 350K/43K/00K train/calibration/val data. We finetune pretrained BERT-Base (De- vlin et al., 2018) model, achieving 83.36% accuracy on the matched (MNLI-m) data and 83.77% accuracy on the mismatched (MNLI-mm) data. Calibration performances are also evaluated on MNLI-m and MNLI-mm separately.
Methods and evaluation metries We implement our NRPT algorithm with a two-layer neural network calibrator on top of the last hidden representation H of the trained classifier (see (6) and (7).) In the case of BERT, И is the last encoder layer at the CLS token. For all three tasks, choose hidden dimension = 512, the number or neurons M E {5, 10). and ó to be the leaky-relu activation. We choose Stemp (t) 0.2 (tlus so that To, (x)) is guaranteed be within [0.2, 6.2].
We compare our NRPT against the original uncalibrated model, as well as two existing post- calibration methods: Temperature Scaling (TS, see (3)), strong baseline method for post- calibration (Guo et al., 2017), and Local Temperature Scaling (LTS, see (4)), a generalization of temperature scaling that is observed achieve state-of-the-aar performance on a variety of com- puter vision tasks (Ding et al., 2020)'. On CIFAR-100 and ImageNet we further compare with deep ensembles (Lakshminarayanai et al., 2017), a representative diversity-based method with strong calibration performance (Ovadia et al., 2019). We train the calibrators using either the NLL loss (2) or the ECE loss (1) with unregularized minibatch SGD on the calibration data. We use "+E" in (TS+E, LTS+E, NRPT+E] to indicate that a method is trained with the ECE loss. Additional archi- tectural and training details can be found in Appendix B.
We evaluate the calibration methods on three metrics: the NLL., the ECE and debiased ECE (DECE), as well as the predictive entropy (PEnt) for evaluating the sharpness of the calibrated predictions.
Results Table summarizes our main results. We observe that NRPT and NRPT+E consistently performs the best among each group in terms of both the NLL and the ECE metric. In particular, the best test likelihood on CIFAR 100 and ImageNet are achieved by NRPT and the best test ECE or debiased ECF are achieved by either NRPT or NRPT+E. This justifies our intuition that maximizing the expressivity of the calibrator while making sure it preserves the rank of the logits can indeed
We remark that method is partly motivated by the attempt improve over matrix scaling. However, we find matrix scaling consistently performs worse than temperature scaling, and thus we omit the results here. FFrr temperature scaling, since there is only one trainable parameter, we can in theory obtain the exact optimal solution on the entire dataset; however observed that the SGD solution with proper learning rate decay almost always coincides with the exact solution.

---

Under review as a conference paper at ICLR 2021
Table 1: Comparison between NRPT and existing post-calibration methods. Metrics are reported in terms of the mean and standard deviation over 4 random seeds.
Task
Metric
Uncal
TS
LTS
NRPT
TS+E
LTS+E
NRPT+E
NLL
0.90
0.84
0.82
10.00
0.78 10.00
0.84
0.84 #0.00
0.84 #0.00
CIFARI00
ECE
9.75
5.35
5.15
10.13
4.16 10.13
2.66
1.87
20.05
1.77 #0.14
DECE
9.37
5.34
4.99
10.12
4.08 40.09
2.33
1.69
10.18
1.39 10.15
PEnt
0.44
0.76
0.74
#0.0
0.71 +0.01
1.01
1.00
#0.00
1.02
#0.02
NLL
0.97
0.96
0.94
10000
0.93 10000
0.96
0.96
10.00
0.95 1:0,0
ImageNet
ECE
4.66
3.31
2.25
+0.00
1.62 #0.00
2.54
1.71
#0.04
1.60
=0.04
DECE
4.65
3.36
2.25
+0.03
1.45 +0.03
2.50
1.71
#0.10
1.50 +0.03
PEnt
0.78
0.92
0.93
40.01
0.91 +0.01
1.06
1.05
#0.01
1.01 +0.00
NLL
0.49
0.43
0.43
4:0.00
0.43 +00.00
0.43
0.47
+0.01
0.43 +0000
MNLI-m
ECE
7.52
1.43
1.09
4:0.13
0.88 +0.04
1.48
1.67
#0.11
1.13 +0018
DECE
7.53
1.37
0.63
40.05
0.35 10.14
1.40
1.29
10.17
0.79 10.15
PEnt
0.24
0.43
0.42
40.00
0.42 #0.00
0.42
0.39
#0.00
0.40 #0.01
NLL
0.47
0.42
0.42
1:000
0.42 10.00
0.42
0.45
10.01
0.42 1:000
MNLI-mm
ECE
7.40
1.68
0.86
10.16
0.63 +0.05
1.62
1.64 #0.13
1.20 10.10
DECE
7.39
1.89
0.61
t0.0s
0.38 +0.21
1.91
1.52
#0.23
0.10 #0.6
PEnt
0.24
0.42
0.41 10.00
0.41 10.00
0.42
0.39
10.00
0.40 10.01
improve the calibration performance and do not yield significant overfitting. As side note, we find that the choice of the loss function can substantially impact the behavior of the final calibrator: train- ing with the NLL loss typically minimizes the test NLL well and improves the ECE by a reasonable amount, whereas training on the ECE loss typically minimizes the test ECE better (at least on the image tasks) the cost of performing a little worse on the NLL and ECE.
We remark that the behaviors on MNLI are slightly different in that the best ECE is achieved by NRPT instead of NRPT+E, potentially due to the language task being different from the image tasks. However, we do observe that NRPT+E still performs best among those trained with ECE loss, again demonstrating the benefit of the improved expressivity the NRPT calibrator.
Comparison with deep ensembles; parameter efficiency We further compare NRPT against deep ensembles (Lakshminarayanan et al., 2017) in Table While deep ensembles in general can improve the accuracy and achieve much better NLL and sharpness (PEnt) due to the higher model capacity, we find that NRPT can consistently achieve a better ECE than an ensemble of models, and nearly match ensemble of 8 models on CIFAR100. NRPT further has much less memory overhead compared with the ensembles: the calibrators we used only has size 0.5% - 2% of the original model, whereas even an ensemble of 2 models requires doubling the model size. We also compare against Monte Carlo Dropout (Gal & Ghahramani, 2016) in Appendix C where we find Dropout cannot simultaneously maintain the accuracy and achieve calibration as well as NRPT.
4.2 CALIBRATION PERFORMANCE BEYOND THE ECE
Accuracy among top-confidence predictions We investigate the calibration ability of NRPT be- yond the ECE metric, by looking at how well improves the rank of the confidence among the individual examples. We measure the quality of the rank by visualizing the accuracy among top- confidence predictions: we take the subset of the test set for which the calibrated confidence ranks among the top x% (e.g. 20%, 10%), and evaluate the accuracy (or error rate) among these exam- ples. Roughly speaking, the error should become lower as we decrease the percentage x, but we can further compare this curve across different methods. In Figure 2a, we confirm that the NRPT has lower classification eтror than TS and LTS for the majority of the percentages, except at the very tail. This can be further quantitatively measured by the PRR (prediction rejection ratio) metric, where a higher PRR implies a better accuracy among top-confident examples (see Appendix for the details of PRR). In Table 3, we find that NRPT achieves better PRR than both TS and L.T..
7

---

Under review as a conference paper at ICLR 2021
Table 2: Comparison between NRPT and deep ensembles.
Task
Metric
Uncal
NRPT
NRPT+E
Ens-2
Ens-4
Ens-8
ACC
80.36
80.36
80.36
82.42
83.56
84.10
NLL
0.90
0.78
20.00
0.84
20.00
0.72
0.63
0.59
CIFARIO0
ECE
9.75
4.16
#0.13
1.77
20.14
4.77
2.63
1.71
DECE
9.37
4.08
10.09
1.39
10.15
4.75
2.53
1.23
PEnt
0.44
0.71
#0.01
1.02
=0.02
0.48
0.51
0.54
#params
1x
1.006x
1.006x
2x
4x
8x
ACC
76.27
76.27
76.27
77.29
77.76
NLL
0.97
0.93
40.00
0.95
#0.00
0.90
0.87
ImageNet
ECE
4.66
1.62
#0.06
1.60
#0.04
2.43
1.79
DECE
4.65
1.45
20.03
1.50
20.03
2.31
1.73
PEnt
0.78
0.91
#0.01
1.01
=0.00
0.83
0.88
#params
1x
1.015x
1.015x
2x
4x
Table 3: Prediction rejection ratio (PRR) metric (def in Appendix D). Higher the better.
Task
Uncal
TS
LTS
NRPT
CIFARI00
56. 49.49%
54.28%
10.00%
60.69%
10.22%
61.84% 10.37%
ImageNet
56.11%
55.69% +0.00%
58.47% +0.14%
58 28.65% +0.07%
Tradeoff between sharpness and ECE For post-calibration methods that maintain the accuracy of the model, the calibration (e.g. ECE) is typically negatively correlated to the sharpness of the prediction (e.g. predictive entropy). We investigate the ability of NRPT in optimizing the tradeoff when both metrics are desired. To test this, we train both {TS, LTS, NRPT) on a weighted com- bination of the NLL and ECE loss LLLL + C(1 a) LECE, where use multiple weight values and plot the resulting predictive entropy and ECE as trade-
{0, 0.1, .,, 0.7, 0.75, 0.8 . 1}. off curve. In Figure 2b, we see that NRPT achieves nearly universally better tradeoff curve than TS and LTS. This suggests that the improved expressivity in NRPT can be beneficial in practice when more than one metrics are desired and it is necessary to manage the tradeoff.
0.055
0.255
0.050
0.030
0.245
0.025
0.020
0.035
0.230
0.035
0.025
0.010
NSPT
0.005
Unco
(a) Error within confidniee predictions
(b) Tradeoff between ECE and sharpness
Figure 2: Comparison between TS, LTS, and NRPT in the calibration abilities. Each dot in (b) is obtained hy optimizing weighted combination of the NLL and ECE loss. Shaded area (a) and crosses in (b) indicate the standard deviation over 4 random seeds.
CONCLUSION
We proposed Neural Rank-Preserving Transforms (NRPT), an improved technique for uncertainty post-calibration, and showed that it outperforms existing post-calibration methods benchmark tasks. A number interesting research questions remain open: for example, can we have better understanding on the choice of the architecture for the monotonic transforms used in NRPT? Can build calibrators that combine the advantages in standard post-calibration methods and ensemble- like methods? We would like to leave these future work.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Richard E Barlow and Hugh D Brunk. The isotonic regression problem and its dual. Journal of the American Statistical Association, 67(337):140-144) 1972.
Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov. The need for uncertainty quantifi- cation in asshine-assisted medical decision making. Nature Machine Intelligence, 1(1):20-23, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Jacob Devlin, Ming-Woi Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Zhipeng Ding, Xu Han, Peirong Liu, and Mare Niethammer. Local temperature scaling for proba- bility calibration. arXiv preprint arXiv:2008.0 05 105, 2020.
Michael W Dusenberry, Ghassen Jerfel, Yeming Wen, Yi-an Ma, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanann and Dustin Tran. Efficient and scalable bayesian neural nets with rank- factors, arXiv preprint arXiv:2005.071880 2020.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059, 2016.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. arXiv preprint
2017.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning: data mining, inference, and prediction. Springer Science Business Media, 2009.
Xiaoqian Jiang, Melanie Osl, Jihoon Kim, and Lucila Ohno-Machar Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Asso- ciation, 19(2):263-22 274,, 2012.
Meelis Kull, Miquel Perello Nieto, Markus Kängsepp. Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. In Advances Neural Information Processing Systems, pp. 12316-12326, 2019.
Ananya Kumar, Percy S Liang, and Tengyu Ma. Verified uncertainty calibration. In Advances in Neural Information Processing Systems, pp. 3792-3803, 2019.
Aviral Kumar, Sunita Sarawagi, and Ujjwal Jain. Trainable calibration measures for neural networks from kernel mean embeddings. In International Conference on Machine Learning, pp. 2805- 2814, 2018.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predic- tive uncertainty estimation using deep ensembles. In Advances in neural information processing systems, pp. 6402-6413, 2017.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436-444 2015.
Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax- and Balaji Lakshmi- narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. arXiv preprint arXiv:2006. 10108, 2020.
David JC MacKay. Bayesian neural networks and density networks. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, (54(1):73 1995.

---

Under review as a conference paper at ICLR 2021
Wesley J Maddox. Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. In Advances in Neural Information Processing Systems, pp. 13153-13164 2019.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks, In Advances in Neural Information Processing Systems, pp. 7047-7058, 2018.
Andrey Malinin, Bruno Mlodozeniec, and Mark Gales. Ensemble distribution distillation. In In- ternational Conference on Learning Representations, 2020. URL https:: net wewoooouumnddmmyo SVt... 6vtvr.
Rhiannon Michelmore, Marta Kwiatkowska, and Yarin Gal. Evaluating uncertainty quantification in end-to-end autonomous driving control. arXiv preprint arXiv:181..06811 2018.
Mahdi Pakdaman Nacini, Gregory F Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the... AAAI Conference on Artificial Intel- ligence. AAAI Conference on Artificial Intelligence, volume 2015, pp. 2901. NIH Public Access, 2015.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan. and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances Neural Information Process- ing Systems, pp. 13991-14000 2019.
John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances large margin classifiers, 10(3):1177,, 1999.
Leslie N Smith. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Confer- on Applications of Computer Vision (WACV), pp.
IEEE, 2017.
Linh Tran, Bastiaan S Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V Dillon, Jasper Snoek, Stephan Mandt, Tim Salimans, Sebastian Nowozin, and Rodolphe Jenatton. Hydra: Preserving ensemble diversity for model distillation. arXiv preprint arXiv:2001.0400 2020.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arviv: 1804 07461, 2018
Antoine Wehenkel and Gilles Louppe. Unconstrained monotonic neural networks. In Advances in Neural Information Processing Systems, pp. 1545-1555, 2019.
Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: alternative approach to efficient ensemble and lifelong learning. arXiv preprint arXiv:2002.06715, 2020.
Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Icml, volume 1, pp. 609-616. Citeseer, 2001.
Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass proba- bility estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowl- edge discovery and data mining, pp. 694 699, 2002.
Sergey Zagoruyko and Nikos Komodakis. arXiv:1605.07146. 2016.
Wide residual networks.
arXiv preprint
A ADDITIONAL RELATED WORK
Post-calibration methods Post-calibration for binary classification problems has been studied ex- tensively in classical work, including non-parametric (bining type) methods such histogram bin- ning (Zadrozny & Elkan, 2001), isotonic regression (Zadrozny & Elkan, 2002), Bayesian binning into quantiles (Nacini al., 2015), well as parametric methods such Platt scaling (Platt al., 1999) which re-fits a one- one-oonenet -ie-emensiioal logistic regression model from the logit to the probability on holdout calibration dataset. Kumar al. (2019) combines the advantage of parammetitii and
10

---

Under review as a conference paper at ICLR 2021
parametric approachs in the scaling- bining calibrator, which first fits parametric function to the calibration dataset and then performs the binning.
While most binning type methods are defined for binary problems, they can be extended to multi- class classification (K 2 3 classes) by performing the calibration on all the 1-vs- - binary tasks, and re-normalizing the calibrated probabilities (Zadrozny & Elkan, 2002). The parametric can also be extended to the multi-class case with various degrees of freedom, including tempreature sealing, vector scaling, and matrix scaling. Guo al. (2017) tested the multi-class calibration methods on a variefy of tasks and found that temperature scaling performs the best across the board. Local Tempreature Scaling (Ding et al., 2020) proposes to use an individualized temperature for each example; in this paper we implement. Dirichlet calibration (Kull et al., 2019) improves the per-class calibration by using different Dirichlet distribution for each class as the calibrator. MMCE (Kumar et al., 2018 8) proposes to optimize a kernalized version of ECE for improving the calibration; however they do not consider optimizing the original ECE directly.
Diversity-based uncertainty quantification Diversity-based uncertainty quantification can be roughly divided into two types. Ensemble methods such deep ensembles (Lakshminarayanan et al., 2017) train an ensemble of models from different initializations (and with different SGD noise), and find that the aggregated (average) predicted probability exhibit better uncertainty cali- bration than a single deterministic model. As ensembles are memroy and runtime heavy, recent line of work proposes to make ensembles more efficient by either reducing the parameter count through smart reparametrizations (Wen et al., 2020) or single deterministic model that simulates the en- sembles (Liu et al., 2020). A related line of work proposes to distill an ensemble of models (Malinin & Gales, 2018; Malinin al., 2020; Tran et al., 2020). We remark that either the efficient ensem- bling approach or the distillation approach improves the uncertainty calibration through simulating an ensemble, and can be used jointly with post-calibration methods.
Bayesian neural networks (MacKay, 1995) are capable of producing uncertainty estimates by na- ture since it learns distribution of networks (that can be used aggregatedly) rather than a single network. Monte Carlo Dropout (Gal & Ghahramani, 2016) uses the randomized prediction capabil- ity of Dropout perform uncertainty calibration. SWAG (Maddox et al., 2019) performs uncertainty calibration via an approximate Bayesian model averaging using the SGD iterates. Bayesian rank- one factors (Dusenberry et al., 2020) is Bayesian version of Bntebenss that learns posterior over the rank-one parametrization of ensembles.
B ADDITIONAL EXPERIMENTAL DETAILS
B.1 MODELS
NRPT We choose in (7) to be the ReLU activation and & to be the leaky relu activation with negative slope tuned {0.5, 0.8, 1.5, 2.0}. The number of neurons M was tuned in {5, 10). We further initialize the To, (x) such that it has initial values (approximately) {0.5, 1.0, 1.,, 0.5MM by properly initializing the bias within these networks.
LTS For LTS (local temperature scaling) we use an architecture that is similar to NRPT:
(x() Ttemp (a (W + bb),
where H _dd + Iddi is the last hidden representation layer of the trained classifer, M e RRdiax and RV Similar as NRPT, we chose N 512 and Ttemp (t) = 0.2+relug We remark that our implementation is likely different from the implementation in (Ding et al., 2020) (and operates on different base models). Nevertheless we find that our implementation also strong calibrator that consistenly performs temperature scaling.
B.2 TRAINING AND EVALUATION
CIFAR-100 The base WideResNet-288 10 on CIFAR- 100 was trained with batchsize 128 for 200 epochs with cosine learning rate with initial learning rate 0.1.
11

---

Under review as a conference paper at ICLR 2021
ImageNet The base WideResNet-500 ImageNet was trained with batchsize 256 (parallelized onto 8 GPUs) for 100 epochs. The initial learning rate was 0.1, with fixed decay of factor 0. 0.1x at the {30, 60, 80)-th epochs.
MNLI The BERT-base on MNLI was finetuned from the pretrained model with batchsize 32 for epochs, with the Adam optimizer and learning rate 2 10-5
All post-calibrators are trained with one-cycle learning rate (Smith, 2017) and we tune the initial learning rate within {le-3, 3.1e-3, 1e-2, 3.1e-2]. All post-calibrators are trained with the same batchsize used in training the base model. The number of epochs for training the calibrators was 50 on CIFAR-100 FERAR-100,5 5 on ImageNet, and on MNLI.
ECE loss and evaluation metric We choose the number of bins be 15 for evaluating the ECE following the standard practice (Guo et al., 2017) (and the body recent work). However, evaluate the train-time ECE loss
train time, tune the number of bins within (5, 10), small minibatches, which could benefit from a smaller number of bins.
Hy prprrraareeeer tuning All the hyperparameter tuning were conducted by further splitting the calibration dataset into training and development set, where we train with grid of hyperparameters on the train set and select the best on the development set.
C COMPARISON BETWEEN NRPT AND MONTE CARLO DROPOUT
We compare the calibration performance of NRPT and Monte Carlo Dropout (Gal Ghahramani, 2016) CIFAR-100, where train a WideResNet-28-D1 model for each drop probability, and evaluate the calibration by aggregating over random predictions at test time (each with a different mask).
We observe a consistent trend on Dropout: increasing the drop probability improves the ECE at the cost of hurting the accuracy, which is expected since randomized predictions naturally get more calibrated but less accurate as we increase the level of randomization. Comparing NRPT+E with Dropout, we that NRPT+E achieves better ECE than Dropout up drop probability 0.7; for drop probability 0.8, the ECE is better than NRPT+E, however it comes at the cost of significantly lower accuracy. For the NLL and predictive entropy metric, NRPT performs slightly better than Drop 0.8 and slightly worse than Drop 0.7. However, Drop 0.7 also has worse accuracy than NRPT (which did not change the accuracy of the model). This suggests that NRPT may be preferred over Dropout if maintaining the accuracy is crucial; Dropout can only achieve better calibration by huring the accuracy.
Table 4: Comparison between NRPT and Monte Carlo Dropout. "Drop 0.4" indicates the dropout method with drop probability 0.4 and keep probability 0.6. All dropout methods are evaluated by aggregating the randomized predictions over masks.
Task
Metric
Uncal
NRPT
NRPT+E
Drop 0.4
Drop 0.6
Drop 0.7
Drop 0.8
ACC
80.36
80.36
80.36
80.91
80.21
78.92
77.61
NLL
0.90
0.78
40.00
0.84
10.00
0.75
0.74
0.76
0.81
CIFARI00
ECE
9.75
4.16
10.13
1.77
10.14
4.91
3.03
2.55
1.01
DECE
9.37
4.08
20.09
1.39 10.15
4.71
2.96
2.19
0.50
PEnt
0.44
0.71
40.01
1.02 +0.02
0.58
0.63
0.67
0.76
#params
1.006x
1.006x
1x
1x
D DETAILS ON THE PRR METRIC
The PRR (prediction rejection ratio) metric is commonly used for quantitatively summarizing the accuracy against cop-confidence predictions, equivalenty for evaluating the if the model can reli- ably reject to predict (Malinin et al., 2020). The PRR metric is based on the AUC (area under curve) on the accuracy among top-confidence curve in Figure 3. We compare the curve of method
12

---

Under review as a conference paper at ICLR 2021
against that of an "oracle" rank of confidence (which perfectly ranks all wrong predictions of lower confidence than all correct predictions), as well as a "random" rank confidence illustrated by straight line. The PRR metric is then defined as
(UC(naoom AUC(Mothod] shaded area in green AUC(Random) AUC(Oracle) shaded area in orange
PRR(MMethod) :
A higher PRR indicates that the method achieves a better (closer to the oracle) rank of confidence.
0.200
Method
Oracle
0.175
Random
0.150
Classification error
0.125
0.075
0.050
0.025
0.000
100%
80%
60%
40%
20%
0%
Top x% confidence
Figure 3: Illustration of the PRR metric.
E ACCCURACY AMONG MOST-CONFIDENT EXAMPLES ON IMAGENET
We plot the accuracy among most-confident examples on ImageNet in Figure 4. Similar as on CIFAR-100 (Figure 2a), we see that NRPT achieves lower error rate than LTS and TS most confidence levels, except for the very tail.
0.012
Classification error
0.008
0.006
0.004
0.002
TS
NRPT
Uncal
10%
8%
6%
4%
2%
0%
Top x% confidence
Figure 4: Accuracy among most-confident examples on ImageNet.
13


Reviews:
[{"review_id": "sLeBmX8CPT5", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Justification for Score**\nThe paper is missing a thorough literature review and there are many missing citations and comparisons. Overall, unfortunately there are multiple misleading (and sometimes false) claims throughout the paper. The evaluation performance is also a little unfair as they use ECE as an objective and then only show better performance on ECE. Overall, the paper is well written but the content does not have good enough quality as there seem to be many unsupported claims.\n\n## Review\n\n### Summary\nThis paper proposes a post-hoc calibration method which aims to preserve the accuracy of the classifier as well as improve its uncertainty calibration. The core of the method lies in formulating a general form of the commonly used scaling method Temperature Scaling (TS) and more recent variation of it, Local TS (Local TS). The proposed method Neural Rank-Preserving Transforms (NRPT) maintains the accuracy but also shows better calibration performance compared to TS and Local TS.\n\n### Strengths\n* It is important that a post-hoc calibration does not decrease the accuracy. I like the justification of the authors about this: \" a calibrator that does not maintain the accuracy may attempt to improve the accuracy at the cost of hurting (or not improving) the calibration\". Though, I have to point out that a method which can change the rank performance also has advantages compared rank-preserving methods (i.e. TS or NRPT): The literature has shown that accuracy and calibration performance can jointly be improved - the authors don't really acknowledge this point (see below for references and examples of such methods). \n\n* A rank preserving method has the advantage of optimizing loss functions which do not have to \"care\" about accuracy. In this case, the authors propose to use the ECE loss. A method which does not preserve the rank would completely decay the accuracy in an attempt to optimize the ECE. That being said, I still think that optimizing on the ECE and only showing improvements on the ECE metric is not surprising (see below for more discussion on this point). So it does have the advantage of optimizing ECE, but the paper does not really show the benefit here as it only shows that optimizing the ECE performs better on ECE (this is expected) but performs worse on the other metrics (see Tab. 1). Also, I would like to point out that I include DECE when I talk about ECE in this review.\n\n* Better calibration performance compared to the Baseline, TS and LTS. The authors also compared against deep ensembles which have been shown to improve calibration, though they only perform better on ECE for the case where NRPT is optimized using the ECE (NRPT-E). NRPT (without -E) performs significantly worse than ensembles larger than 4 (at least more than half better). \n\n* Even though NRPT does not have overall better performance compared to deep ensembles (i.e. only is better at ECE metric and only when ECE is specifically optimized during training), it does have the advantage of being computationally faster. \n\n\n### Weaknesses\n1. The major weakness of this paper is its view of the current post-hoc calibration literature seems to be outdated. A re-occurring theme throughout the paper is that post-hoc methods are very simple and always talks about Temperature Scaling (TS) as an example. The post-hoc literature has come far beyond simple methods such as TS. Even though it is true that many recent variants of TS have shown good improvements, there are many other non-\"simple\" methods which have much (1) larger capacity, (2) can improve accuracy (i.e. they do not have accuracy loss) and (3) greatly improve calibration performance. \n\n2. The related work paragraph also only talks about three post-hoc calibration methods. Even though, the literature grows very fast and it can be hard to always keep track of all new papers, there have definitely been much more than the three variants cited in this paper (see below for references). Later on, the paper does cite the work [1] and claims that they show that \"overfitting cannot be easily fixed by applying common regularizations such as L2 on the calibrator\". Despite, this statement not being entirely true (see below), it seems that the authors are aware of this paper. [1] has also presented a calibration method (i.e. Dirichlet calibration - mentioned in the title of [1]), so why has this method not been compared or cited for its calibration method? \n[1] has thoroughly compared against multiple other calibrators (in addition to TS and Matrix scaling), so why were none of these other calibrators cited, used and compared against?\n\n3. This is why I find that there are many misleading statements throughout the paper which imply that the (large) family of post-hoc calibration methods can be summarized as \"simple\" and limited to TS or Matrix Scaling. One example of such a statement can be found in the appendix: \"Existing post-calibration methods such as temperature scaling recalibrate a trained model using rather simple calibrators with one or few parameters, which can have a rather limited capacity.\" The reason I find them misleading is that for a reader/reviewer who is not familiar with post-hoc calibration, it might seem that TS (and its variants) are the only and best methods for post-hoc calibration. Recent years have shown multiple other solutions which are not limited to such simple methods.\n\n4. Here is a list of some recent and relevant approaches: GP [3], Histogram Binning [5], I-Max histogram binning [4], Beta Calibration [2], Dirichlet Calibration [1], Matrix Scaling with ODIR regularization [1], BTS [6], Isotonic Regression [9] and MnM [8]. These post-hoc calibration methods range from relatively old methods to much newer methods. Even though I certainly don't expect to see all of them cited and compared against, the authors should do a much more thorough search of the literature or not make statements which imply that the post-hoc literature is only limited to the methods which they compare against.\n\n5. Even though the authors compared against deep ensembles, the performance is only better in one case: the ECE metric and only when NRPT optimizes the ECE loss (i.e. NRPT-E). This is no surprise and cannot be used to claim that the method is better than deep ensembles. A method which optimizes the metric A directly will perform better at this metric A. Of course, there are some exceptions such as NLL training without regularization. But overall this is not really an impressive result, especially given that the ECE has lately been criticized (see above).\n\n6. State-of-the-art-performance: \"Local Temperature Scaling (LTS, see (4)), a generalization of temperature scaling that is observed to achieve state-of-the-art performance on a variety of computer vision tasks (Ding et al., 2020)\" This paper claims that LTS obtains state-of-the-art performance and by performing better than LTS implying that they have beaten state-of-the-art in post-hoc calibration. The authors of this current paper do not claim that they obtain state-of-the-art but they imply this by mentioning that they perform better than a method which is previously state-of-the-art. I looked into the LTS paper and they also did not make the claim of obtaining state-of-the-art performance. So where is this claim coming from? It is hard to claim that LTS obtains state-of-the-art performance. After a brief look in the LTS paper, it seems that they too ONLY compare against TS variants (please correct me if I am wrong). As I have listed above, there are multiple other methods which have shown to improve calibration and have a thorough comparison against multiple other post-hoc calibration methods (and not only limited to TS and some of its variants).  Again, I find this a little misleading that the authors of this paper seem to imply many statements to make their method seem good.\n\n7. Fig. 1 and Matrix Scaling: The authors seem to want to show the benefit of NRPT by showing superior performance compared to TS (\"simple\") and Matrix Scaling (\"complex calibrators can often overfit\"). Despite, the fact that these are poor baselines to compare against (see list of calibrators above), there has been a new proposal to fix the overfitting issues of Matrix Scaling. [1] proposed to use ODIR regularization (\"simple L2 regularization\") to improve the performance of Matrix Scaling. Firstly, the authors cite this work which means they are aware of this paper and this technique to reduce the overfitting. So then why has it not been used to in Fig. 1? And why was it chosen to compare against Matrix Scaling (without regularization) if there does exists a better solution for matrix scaling? Secondly, the authors instead make a misleading statement: \"It is further observed that the overfitting cannot be easily fixed by applying common regularizations such as L 2 on the calibrator (Kull et al., 2019)\" This statement is not true. This paper shows in Table 3 (of [1]) and Table 4 (of [1]) that Matrix Scaling with ODIR (i.e. L2 regularization) is sometimes the best performing method. Even for the rest of the cases where it does not perform best, it performs similarly. This shows that \"simple regularization\" can help address the over-fitting issue. This regularization has also been used in [4] with Matrix Scaling for the ImageNet classifiers and has shown to be the best performing method in terms of accuracy (showing that it helps with reducing the overfitting) Again, this is a mis-leading statement and the literature does not support this.\n\n8. Based on the previous (possibly) false claim, the next claim that \"This empirical evidence seems to suggest that complex calibrator with a a large number of parameters are perhaps not recommended in designing post-calibration methods\" is also not entirely true. As mentioned above, highly expressive methods can improve calibration.\n\n9. The authors mention that the small size of the validation set can cause highly expressive methods to overfit, thought [3,4] have both shown to greatly improve calibration performance with as little as 1000 samples for ImageNet classifier calibration and even shows better performance than methods using significantly more samples. Therefore, it is not true that a limited dataset will lead to overfitting and these results show that regularization and other techniques can handle a low data regime for calibration.\n\n10. The authors also claim that \"matrix scaling is not guaranteed to maintain the accuracy\". This claim is true, but the authors fail to mention that Matrix Scaling (and its regularized variants) can also improve the classification accuracy performance. The high capacity allows it to improve accuracy as well. After using L2 regularization the accuracy performance can be greatly improved. \nAlternatively, [5], also presented Vector Scaling (which has less capacity than Matrix Scaling but more than TS) and they also show that it too can change the accuracy performance and actually improve the accuracy in most cases. So even though I do see the benefit of having method which preserves the rank, it should clearly be pointed out that it has the disadvantage of not being able to improve the accuracy. For example, compared to [3] which improves the accuracy performance, NRPT has the disadvantage with respect to accuracy. So despite this above statement being true, the authors use this statement to justify why methods like TS which preserve the accuracy are better, though TS actually has a disadvantage compared to the many cases where these rank-changing methods improve the accuracy.\n\n11. ECE: The authors cite [10] to use their presented debiased estimator. Though, they completely ignored the main message of that paper. [10] has shown that continuous output scaling methods (e.g. TS) have trouble estimating the ECE and instead propose to use binning or quantized output methods instead. [1] also shows how the ECE is underestimated when using too few bins during evaluation. They show that the ECE of scaling methods are non-verifiable. So even though the authors are aware of this paper, they do not at all comment on the use of ECE for continuous output scaling methods such as the one presented. Other recent works [3,4] have also discussed this and proposed solutions such as increasing the number of bins when estimating the ECE of scaling methods. How many bins were used to estimate the ECE? As the authors are aware of this work, they should discuss this weakness of ECE in their paper and more importantly use approaches used in other recent works in the literature to address these issues to some extend. Again, it seems that the authors need to do a more thorough literature search on post-hoc calibration.\n\n12. The method also uses ECE during training: Even though this might be an advantage of rank preserving methods, it is not surprising that the method using ECE as a loss performs the best at ECE. Table 1 shows that \"NRPT+E\" is only better at ECE and no other metric. As discussed earlier, the ECE metric has major flaws and problems, so using this as a loss also comes with its problems. So I do not see this variant of NRPT as very useful. It would be interesting to how \"NRPT-E\" performs on other metrics and tasks which are not directly optimized during training, as using ECE as the objective and the evaluation is unfair.\n\n13. Tab.1 : Why are the uncalibrated ECEs so high? These are unusually high ECE even for uncalibrated networks? There is no surprise then that even simple methods such as TS can show great improvement when the networks are so miscalibrated. Many other works using similar networks report much lower uncalibrated ECE. Maybe the ECE was evaluated differently?\n\n14. ECE as objective: As you use ECE during the learning, you should be providing all details about ECE (e.g. the number of bins, the bin widths or bin edges ).\n\n### Minor comments\n* Some references as using arxiv versions of the paper (e.g. Guo2017 is a ICML paper from 2017, so shouldn't be using an arxiv reference)\n\n\n### References (all can be found on arxiv)\n\n[1] Meelis Kull, Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach.  \"Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration\"\n\n[2] Meelis Kull, Telmo Silva Filho, and Peter Flach. \"Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers\".\n\n[3] Jonathan Wenger, Hedvig Kjellström, and Rudolph Triebel. \"Non-parametric calibration for classification\"\n\n[4] Kanil Patel, William Beluch, Bin Yang, Michael Pfeiffer, Dan Zhang. \"Multi-Class Uncertainty Calibration via Mutual Information Maximization-based Binning\"\n\n[5] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. \"On calibration of modern neural networks\"\n\n[6] B. Ji, H. Jung, J. Yoon, K. Kim, and y. Shin. Bin-wise temperature scaling (BTS): \"Improvement in confidence calibration performance through simple scaling techniques\"\n\n[7] Jize Zhang, Bhavya Kailkhura, and T Han. \"Mix-n-Match: Ensemble and compositional methods for uncertainty calibration in deep learning.\"\n\n[8] Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. \"Obtaining well-calibrated probabilities using Bayesian binning.\"\n\n[9]  Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates.\n\n[10] Ananya Kumar, Percy S Liang, and Tengyu Ma. \"Verified uncertainty calibration\"", "extra_scores": {}, "extra_text": {}}, {"review_id": "o5FvYwMtfsy", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overview: \n\nThis paper proposes a post-calibration technique that is meant to be more powerful than temperature scaling without introducing overfitting. The authors argue that previous attempts to generalize temperature scaling have a tendency to overfit not because of the additional parameters, but rather because they are not rank preserving (and therefore can change model predictions / accuracy). The key idea is to use a higher-capacity (2-layer neural network) calibrator that is constrained to be rank-preserving so that model accuracy remains unchanged. The proposed method is compared against baselines on classification tasks from vision (CIFAR-100, ImageNet) and language (MNLI). \n\nStrengths:\n\nThe introduction is interesting reading, and does a great job motivating the importance of model calibration and putting the paper in context. Generally speaking, the paper is quite readable and the figures/tables are clear and informative. The proposed approach is simple, reasonable, and somewhat surprising - it would be easy to assume that higher capacity calibrators fail due to high parameter count, but the paper argues that rank-preservation is really the key. This is interesting and (as far as I know) novel. \n\nThe experiments are very thorough - all metrics are averaged over multiple runs and several complementary performance metrics are provided. The proposed method beats the baselines in most (but not all) of these cases. I think the authors make a convincing case for their approach, but they also successfully motivate further exploration into higher-capacity calibrators by showing that overfitting is not inevitable. \n\nWeaknesses:\n\nThe paper relies almost exclusively on summary metrics and does not conduct too much fine-grained introspection. For instance, the authors observe peculiar behavior for their proposed method on high-confidence examples but do not explain why. There could also be more effort invested in discovering why the proposed method outperforms the competition. For instance - how is calibration improvement distributed over examples? Do the performance gains come from reducing relatively few gross errors or from improving many moderate errors? Are there examples that are actually calibrated worse with the proposed method? While space constraints limit what can be included in the main paper, I think this kind of deeper analysis would improve the work.\n\nOverall: \n\nWhile I'd like to see some deeper analysis, I think this is paper makes an interesting and useful contribution to the calibration literature, backed up by solid empirical results.\n\nMinor comments:\n\nSection 1: \"a much less number of additional parameters\" should be replaced by something like \"a much smaller number of additional parameters\".\n\nSection 3: In the \"Instantiation via monotone two-layer networks\" section, it would be helpful for clarity if $a_j$, $\\theta_j$, and $b_{\\theta_j}$ are explicitly described. Similarly, in the \"Architectural choices\" section it is not entirely clear which parameters $\\theta_1, \\ldots \\theta_M$ are each supposed to represent. Also, what is $\\sigma$ (as opposed to $\\sigma_\\mathrm{temp}$ and $\\sigma_\\mathrm{SM}$)? Though this is described later, the reader may not want to jump around to hunt down these symbols.\n\nSection 4.1: It would have been nice to see the vision results on a more standard architecture (e.g. vanilla ResNet-50) which could have been used for both vision tasks.\n\nFigure 2: The caption should specify what dataset we're looking at.\n\nAppendix B.1: I think \"consistently performs temperature scaling\" should be \"consistently outperforms temperature scaling\" under the LTS heading.\n\n**UPDATE AFTER DISCUSSION:** R4's exceptionally thorough review raised a number of important concerns which I initially did not recognize. Moreover, the most important of these concerns went almost entirely unaddressed by the authors. Since the paper has not been appropriately revised, I have decided to lower my score from a 7 to a 4.", "extra_scores": {}, "extra_text": {}}, {"review_id": "lP4e7PGf4dN", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a new post-processing calibration method that can keep classification ranking, like the simple (local) temperature scaling methods, but has larger capacity, comparable to matrix scaling.  Essentially it tries to achieve the best of both approaches by avoiding the underfitting and overfitting issues of temperature and matrix scaling, respectively.  The method proposed essentially learns a temperature and a bias parameter that depends on the image, and two matrices that are global to all images.  During post-processing calibration training, the constraining of the matrices and the temperature to be strictly positive guarantees that the calibration is monotonically increasing, guaranteeing the preservation of classification ranking.  The paper further shows that this calibration model can be trained in a post-processing approach with NLL and ECE losses (the training with the ECE loss is claimed as another novelty).  Experiments are performed on CIFAR100, ImageNet, and MNLI, and results show that the proposed post-processing calibration stage trained with NLL improves NLL and Entropy for CIFAR100 and ImageNet, and the one trained with ECE improves the ECE related measures.  For MNLI, results seem to show that the calibration with NLL loss provides the best general result.  Additional results that show the value of the method for other calibration measures also display the value of the proposed approach.\n\nI find the paper very easy to read, with clear claims and clear explanations.  The method proposed is quite simple and extends local temperature scaling marginally from a technical viewpoint.  Nevertheless, I like the simplicity of the approach and the experiments that show the potential value of the approach.  In general, I think it is a good paper and would support its acceptance.\n\nMinor issues:\n1- The strange results of NRPT+E on MNLI-m and MNLI-mm for the ECE measures need to be better explained since they are quite unintuitive.\n2- The result of NRPT+E on MNLI-mm for DECE is 0.10, which is best for the row -- that should be the boldface number for that row.\n3- In section 4.2, the paper says \"In Figure 2a, we confirm that the NRPT has a lower classification error than TS and LTS for the majority of the percentages, except at the very tail.\"  This needs to be explained.", "extra_scores": {}, "extra_text": {}}, {"review_id": "NWkWwxDuHRl", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**\nThis work proposes a method for calibrating outputs of deep neural networks using higher capacity learning than previously possible. This is accomplished by a “Neural Rank Preserving Transform” (NRPT) which is applied post-logit to the network outputs. The learned NRPT transform is monotonic, it preserves the rank of the probabilities between logits. The authors also propose to train using a loss for the Expected Calibration Error (ECE), which is the metric that calibration models seek to minimize. This is a straightforward way to improve model calibration, and to my knowledge it has not been proposed in prior works.\n\n**Novelty**\nThe are reasonably solid novel contributions in this work, to my estimation. The insight as to why high-capacity calibration methods overfit and show poor ECE, which leads to the insight to preserve the accuracy of the model via enforcing monotonicity. As mentioned above, the use of ECE as a loss term also appears to be novel.\n\n**Impact**\nCalibration of neural networks is an important issue which is of concern to the ML community at large, especially in sensitive applications such as medical diagnostics and autonomous driving.\n\n\n**Clarity**\nThe paper is reasonably clear and well written. However there are some strange organizational choices, such as the lack of a clear related work section. This has led to the omission of several baseline methods that should have been cited and compared against. Some details of the experimental setup are not clear. For example, exactly how the train/calibration/val splits are used are not clear. One can assume, but it should not be necessary.\n\n**Evaluation**\nThe evaluation seems reasonable, appropriate metrics and datasets are chosen, however I am not sure how much insight about calibration we can really glean from NLL and PEnt. These, to me, are more indirect measures of calibration and more importance should be given to the ECE and DECE results. As the authors point out, it is curious that the +E models are outperformed by the vanilla models on the MNLI datasets. Since this happened on ½ of the datasets, I would have liked to see more datasets to verify that this is truly an anomaly. I’m not sure why other classic calibration techniques have not been considered such as isotonic regression, platt scaling, and Bayesian Binning.\n\n\n**Strengths (Reasons to accept)**\nThe novel contributions mentioned above are of interest to the community.\n\nExperiments are reasonably convincing although lacking in some respects.\n\n\n**Weaknesses (Reasons to reject)**\nMy big problem with this paper is the use of the word “uncertainty”. In my view, this paper is very clearly about calibration which is concerned with the problem of making sure the model outputs match true accuracies. Uncertainty estimation is concerned with determining the epistemic and aleatory uncertainty. Many uncertainty estimation methods can, as a side-effect, improve calibration. But this is typically not their aim. I do see some minor value in showing the results in Figure 3, but I take issue with saying the method performs “uncertainty post-calibration”. \n\nAnother criticism is that the authors do not acknowledge that other calibration methods such as temperature scaling have the desirable property of being monotonic not just in terms of the logits, but in terms of the rank of the confidences. That is, the order of the argmax classes will be preserved. To my understanding, the proposed method does not explicitly ensure this.\n\nFinally, as mentioned above, there are some weaknesses in the evaluation.\n\n\n=============== UPDATE ==============\n\nAfter reading the concerns of the other reviewers and the author response, it seems that many of the concerns remain unaddressed especially the top concerns. Accordingly, I have decided to lower my score.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: # Paper Summary

This paper considers calibrating the output of a multiclass classifier in such a way that the output probabilities approximately are approximately "correct". They observe that if such a method is able to re-order the logits, then it will change the accuracy of the classifier. Therefore, if they use a calibrator that is constrained to be monotonic in the input logits, then they can train it to optimize any metric they choose, without impacting the accuracy.

They propose minimizing ECE (expected calibration error, which is essentially a binned approximation to the L1 distance between the model's confidence in its top label, and the probability that the top label is correct). Borrowing an idea from local temperature scaling, they also allow their calibrator to see the input features, by also taking the top hidden layer (the layer before the logits) as an input.

Their experiments are, with one glaring exception, comprehensive: they have a good selection of datasets, a reasonable choice of metrics, and they dig pretty deep into what the results mean. Reviewer 2, however, believes that the baselines are far from state-of-the art, and two of the other reviewers (and I) agree.

# Pros

1. Well-organized and well-written (not exceptionally so, but above the bar)
1. Good insight overall. In particular, the observation that imposing a monotonicity constraint enables one to optimize any metric, including ECE, was considered both original and significant by the reviewers
1. Well-thought-out experiments. They were generally praised, aside from the (unfortunately crucial) question of whether the baselines are state-of-the-art

# Cons

1. The paper seems to mainly discuss related work coming from the temperature-scaling "tradition". The reviewers would like to see a more comprehensive discussion of other calibration approaches (Reviewer 2 provided a number of references)
1. There are some misstatements (e.g. that LTS is "state-of-the-art"), and incorrect implications (e.g. that temperature-scaling-like methods are dominant). Reviewer 2 listed several of these, all of which are fairly minor, but more care should be taken
1. The TS and LTS baselines are not state-of-the art. The reviewers were generally impressed with the experiments, but the lack of a strong baseline is a fatal flaw

# Conclusion

This was a paper that initially received mostly-positive reviews, but Reviewer 2 raised several concerns that were not adequately addressed in the author response, causing two other reviewers to lower their scores. Ultimately, three of the four reviewers recommended rejection.

The general consensus is that this is a well-written paper, with good insight, well thought-out experiments (except for the baselines), and that it overall makes a worthwhile contribution. The main issues, all of which were raised by Reviewer 2, are eminently fixable: (i) adding a more thorough discussion of related work, especially work unrelated to temperature scaling, (ii) being more careful to avoid misstatements, or to seem to imply incorrect statements (e.g. that temperature-scaling-like methods are dominant), and (iii) adding a couple of new state-of-the-art baselines to the experiments.
Decision: Reject