Distance-Based Learning from Errors for Confidence Calibration
Authors: Chen Xing
Sercan Arik
Zizhao Zhang
Tomas Pfister
Venue:  (2020)
Keywords: Confidence Calibration
Uncertainty Estimation
Prototypical Learning

Abstract: Deep neural networks (DNNs) are poorly calibrated when trained in conventional ways. To improve confidence calibration of DNNs, we propose a novel training method, distance-based learning from errors (DBLE). DBLE bases its confidence estimation on distances in the representation space. In DBLE, we first adapt prototypical learning to train classification models. It yields a representation space where the distance between a test sample and its ground truth class center can calibrate the model's classification performance. At inference, however, these distances are not available due to the lack of ground truth labels. To circumvent this by inferring the distance for every test sample, we propose to train a confidence model jointly with the classification model. We integrate this into training by merely learning from mis-classified training samples, which we show to be highly beneficial for effective learning. On multiple datasets and DNN architectures, we demonstrate that DBLE outperforms alternative single-model confidence calibration approaches. DBLE also achieves comparable performance with computationally-expensive ensemble approaches with lower computational cost and lower number of parameters.

Published as a conference paper at ICLR 2020
DISTANCE-BASED LEARNING FROM ERRORS FOR CONFIDENCE CALIBRATION
Chen Xing" College of Computer Science,
Sercan Ö. Arik Google Cloud Al Sunnyvale, CA
Zizhao Zhang
Tomas Pfister Google Cloud Al Sunnyvale, CA
Google Cloud Al Sunnyvale, CA
Nankai University Tianjin, China
ABSTRACT
Deep neural networks (DNNs) are poorly calibrated when trained in conventional ways. To improve confidence calibration of DNNs, we propose a novel training method, eestance-basee learning from enrors (DBLE). DBLE bases its confidence estimation on distances in the representation space. DBLE, we first adapt proto- typical learning train classification models. It yields representation space where the distance between a test sample and its ground truth class center can calibrate the model's classification performance. At inference, however, these distances are not available due to the lack of ground truth labels. To circumvent this by inferring the distance for every test sample, we propose to train confidence model jointly with the classification model. We integrate this into training by merely learning from mis-classified training samples, which we show to highly beneficial for effective learning. On multiple datasets and DNN architectures, we demonstrate that DBLE outperforms alternative single-model confidence calibration approaches. DBLE also achieves comparable performance with computationally-expeniis ensemble approaches with lower computational cost and lower number of parameters.
INTRODUCTION
Deep neural networks (DNNs) are being deployed in many important decision maasionnmmaing scenarios (Goodfellow et al., 2016). Making wrong decisions could be very costly in most them (Brundage al., 2018)- it could cost human lives in medical diagnosis and autonomous transportation, and it could cost significant business losses loan categorization and sales forecasting. To prevent these from happening, it is strongly desired for DNN to output confidence estimations on its decisions. In almost all of the aforementioned scenarios, detrimental consequences could be avoided by refraining from making decisions or consulting human experts, in the cases of decisions with insufficient confidence. In addition, by tracking the confidence in decisions, dataset shifts can be detected and developers can build insights towards improving the model performance.
However, confidence estimation (also referred as 'confidence calibration') is still challenging problem for DNNs. For a welllcallbbrated model, predictions with higher confidence should be more likely accurate. However, as studied in (Nguyen et al., 2014; Guo et al., 2017), for DNNs with conventional (also referred as 'vanilla') training to minimize the softmax cross-entropy loss, outputs do not contain sufficient information for well-calibrated confidence estimation. Posterior probability estimates (e.g. the softmax outputs) can be interpreted as confidence estimation, but calibrates the decision quality poorly (Gal & Ghahramani, 2016) the confidence tend to be large even when the classification is inaccurate. Therefore, it would be desirable if the training approach is redesigned to make its decision making more transparent, thus providing more information for accurate confidence calibration.
In this paper, propose novel training method, Distance-based Learning from Errors (DBLE), towards better-calibrated DNNs. DBLE learns a distance-based representation space through classifi- cation and exploits distances in the space yield well-calibrated classification. Our motivation is that a test sample's 's location in the representation space and its distance to training samples should
Work done during internship with Google Cloud AT.

---

Published as a conference paper at ICLR 2020
contain important information about the model's decision maaiing process, which is useful for guiding confidence estimation. However, in vanilla training, since both training and inference are not based on distances in the representation space, they not optimized to fulfill this goal. Therefore, in DBLE, we propose to adapt prototypical learning for training and inference, to learn a distance- representation space through classification. In this space, a test sample's ' distance to its ground-truth class center can calibrate the model's performance. However, this distance cannot be calculated at inference directly, since the ground truth label is unknown. To this end, we propose train separate confidence model in DBLE, jointly with the classification model, to estimate this distance at inference. To train the confidence model, we utilize the mis-classified training samples during the training of the classification model. We demonstrate that on multiple DNN models and datasets, DBLE achieves superior confidence calibration without increasing the computational cost as ensemble methods.
2 RELATED WORK
Many studies (Nguyen et al., 2014; Guo et al., 2017) have shown that the classification performance of DNNs are coorly-calibrated under vanilla training. One direction of work to improve calibration is adapting regularization methods for vanilla training. Such regularization methods are often originally proposed for other purposes. For example, Label Smoothing (Szegedy et al., 2016), which is proposed for improving generalization, has empirically shown to improve confidence calibration (Müller et al., 2019). Mixup (Zhang et al., 2017), which is designed mostly for adversarial robustness, also improves confidence calibration (Thulasidasan et al., 2019). The benefits from such approaches are typically limited, the modified objective functions do not represent the goal of confidence estimation. To directly minimize a confidence scoring objective, Temperature Scaling (Guo et al., 2017), modifies learning by minimizing Negative Log-Likelihood (Friedman training subset. It yields better-calibrated confidence by directly minimizing an evaluation metric of the calibration task. However, since it requires training set to be split for the two tasks, classification and calibration, it results in a trade-off if more training data is used for calibration, then the model's classification performance would drop because less training data are left for classification.
al., 2001) objective on small
Bayesian DNNs constitute another direction of related work. Bayesian DNNs aim to directly model the posterior distribution for test sample given the training set (Rohekar et al., 2019). In Bayesian DNNs, since getting the posterior of large amount of parameters is usually intractable, different approximations are applied, such as Markov chain Monte Carlo (MCMC) methods (Neal, 2012) and variational Bayesian methods (Blundell et al., 2015; Graves, 2011). Therefore, the calibration of Bayesian DNNs heavily depends on the degree of approximation and the pre-defined prior in variational methods. Moreover, in practice. Bayesian DNNs often yield prohibitively slow training due to the slow convergence of the posterior predictive estimation. (Lakshminarayanal et al., 2017). Recent work on Bayesian DNNs (Heek & Kalchbrenner, 2019) reports that the training time of their model is an order of magnitude higher than vanilla training. As Bayesian Carlo Dropout (Gal & Ghahramani, 2016) and Deep Ensembles (Lakshminarayanan et al., 2017) are two widely-used methods since they only require minor changes to the classic training method of DNNs and are relatively faster to train.
approaches, Monte
3 LEARNING A DISTANCE-BASHD SPACE FOR CONFIDENCE CALIBRATION
We first introduce how DBLE applies prototypical learning train DNNs for classification. Then, we explain that under this new training method, a distance-based representation space can be achieved to benefit confidence calibration.
3.1 PROTOTYPICAL LEARNING FOR CLASSIFICATION
DBLE bases the training of the classification model on prototypical learning (Snell et al., 2017). Prototypical learning is originally proposed to learn a distance- based representation space for few-shot learning (Ravi & Larochelle, 2016; Oreshkin et al., 2018; Xing et al., 2019). In prototypical learning, both training and prediction solely depend on the distance of the samples to their corresponding class 'prototypes' (referred as 'class centers' in the paper) in the representation space. It trains the model with the goal of minimizing the intra-class distances, whereas maximizing inter-class distances such that related samples are clustered together. We adapt this training method for classification training in DBLE. Subsequent subsections describe the two main components of prototypical training: episodic training and prototypical loss for classification. Later, we explain how to run inference at test time and why it leads representation space that benefits confidence calibration.
2

---

Published as a conference paper at ICLR 2020
3.1.1 EPISODIC TRAINING
Vanilla DNN training for classification based on variants of mini-batch gradient descent (Botton, 2010). Episodic training, on the other hand, samples K- N N-way episodes at every update (as opposed to sampling a batch of training data). An episode is created by first sampling N random classes out of M total classes', and then sampling two sets of training samples for these classes: (i) the support set S. = {(s}, 91) y=) X containing K examples for each of the N classes and (ii) the query set Qe = {xx, 1 containing different examples from the same N classes.
(()) = E Q..
log 0)
(1)
At episode e, the model updated map the query x; to its correct label V4. with the help of support set S.. The model is a function parameterized by 0 and the loss is negative log- log-likelihood of the ground-truth class label of each query sample given the support set according to Eq. 1.
Confidence Model
Proto-loss for calibration
Clasificatioo Model
Query Sample
Ungenndn Ruytuurs
XI
Z
f
Se
(C,, Cz....
3.1.2 PROTOTYPICAL LOSS
How does the support set S. help the classi- fication of query samples at episode e? We firstly employ the classification model, RT + Rd to encode inputs in representa- tion space, where 0 are trainable parameters. Prototypical training then uses the support set Se to compute the center for each class (in the sampled episode). The query samples classified based on their distance to each class center in the representation space. For ev- ery episode e, each center C₃ is computed by averaging the representations of the support samples from class k:
Proto-loss for classiiiction
Figure The training of DBLE. Taking support set Se and query image x; as input, fe learns repre- sentation space through the training of classification model. In this space, h, is the encoded query inputs and (c.,..., CN are the class centers. For classifi- cation, DBLE employs proto-loss to update fo. For calibration, DBLE employs the centers and the sam- pled representation z; for the query to update Jg- If the query x; is correctly classified, the components in the dotted rectangular are not be computed. Solid arrows represent the forward pass and dotted arrows represent the backward pass to update the network.
E
Ck
SE
where Sh C S. is the subset support samples belonging to class The prototypical loss calculates the predictive label distribution of query based on its distances to the N centers:
exp(-d(h,, cy.))
p(y X., S.;0) 0) = En exp(-d(h,,.p)) Cx²))
(3)
where h, is the representation of x, in the space:
h, fo(x)).
The model is trained by minimizing Eq. with p( Jz,, Se; f) Eq. 3. Through this training, in the representation space that we calculate h, and class centers, the inter-class distances are maximized and the intra-class distances are minimized. Therefore, training samples belonging to the same class are clustered together and clusters representing different classes are pushed apart.
3.1.3 INFERENCE AND THE TEST SAMPLE'S DISTANCE TO ITS GROUND-TRUTE CLASS CENTER
At inference, we calculate the center of every class e using the training set, by averaging the representations of all corresponding training samples:
Σ
fo(x:)
(5)
(T
Note that we do not require N to be equal to M because fitting the support samples of all M classes in batch processor memory can be challenging when M is very large.
3

---

Published as a conference paper at ICLR 2020
where To is the set of all training samples belonging to class k. Then, given test sample x, in the test set Drest {(x,, yt) )e) Jr (where y: is the ground-truth label), the distances of its representation he (Eq. 4) to all class centers are calculated. The prediction of the label of x; is based on these distances:
(6) where M consists of all classes in the training set. In other words, 2t is assigned to the class with the closest center in the representation space. Consequently, for a test sample (X,, 3r), if at inference h, is too far away from its ground-truth class center ejear, it is likely to be mis-classified. Therefore, in representation space, a test sample's 's distance to its ground- class center, is a
V/ argmin, {d(he, ck ( JKEM:
this distance- strong signal indicating how well the model would perform on this sample. In the next section, empirically show it under DBLE and compare with other methods.
3.2 DISTANCE TO THE GROUND-TRUTI CLASS CENTER CALIBRATES CLASSIFICATION PERFORMANCE IN DBLE
1.0
Vanilla training
Verilla training
0.8
0.8
DELE
Average Test
DBLE
0.6
0.6
0.4
0.4
0. Venilla traiking
Vanilaa
0.2
07 DELE
DBLE
0.0
10
15
20
6
8 10
Distance Aucending Order (a) CIFAR-100
Distance Din Aocendi my Order (b) ImageNNt
Figure 2: Average test accuracy as d, d' increases. de is the distance of test sample z; to its ground-truth class center and d; is its distance to the predicted class center. It shows that de in the space achieved by prototypical learning in DBLE can better estimate the model's performance Ft. since DBLE's accuracy curve is more monotonic and less oscillating as the distance increases.
We design an experiment to show that in the space learned by prototypical learning of DBLE, the model's 's performance given a test sample can be estimated by the test sample's 1.2-distance to its ground-truth class center. On the other hand, other intuitive measures such as L2-distance to the predicted class center in prototypical learning, L2-distance to the predicted class center in vanilla training or L2-distance to the ground-truth center in vanilla training, can't 't calibrate performance very well. Here, we describe the empirical observations on DBLE's classification model with prototypical learning, to motivate the confidence modeling of DBLE before we explain the method.
In the learned representation space (either the output space of a model from vanilla training or prototypical learning), for every sample (xt, y1) in the test set Drest {xx,, 36) 1 its L2-distance to its ground-trum class center ctest as:
Nest we calculate
d, dd=(((((x), stest) statt
where egest is given by Eq. 5. fo is the trained classification model and d(,) is 1.2-distance. We can also calculate x, 's distance to its predicted class center clest where s/ is the predicted label, in both DBLE and vanilla training as comparison. We denote xe distance to its predicted class center as di = d(fe(x;), c ) We then sort {de)) Noe.r or { Ne in ascending order and partition them in I equally-spacee spaced bins. For every bin, we calculate the model's classification accuracy of the test samples lying in the bin. Then, we plot the test accuracy curve as the distance increases. The curve shows the the relationship between the distance and the model's performance 2,, illustrating how well the calculated distance can act as confidence score, calibrating the model's prediction.
of
As shown in Fig, 2, de of DBLE, which is the distance of test sample x, to its ground-truth class center, calibrates the model's performance the best. The curve of d, under DBLE's prototypical learning, despite its slight oscillation near 0 in the end, decreases monotonically to almost zero. It indicates that farther away the sample is from its ground truth class center, more poorly the model performs it. These results suggest that d, can be used as gold confidence measure, calibrating
4

---

Published as a conference paper at ICLR 2020
the quality of the model's decision of for a classification model under prototypical learning. This benefit of DBLE's prototypical learning is due to the distance- training and inference if test sample is farther away from its ground-truth class center, it is more likely to be mis-classified as other classes.
Although we have observed this gold calibration in DBLE enabled by prototypical learning, it requires access to the ground-truth -ruth labels of test samples, d, At inference however, we do not have access to the ground-truth label of 2e inference and de cannot be directly calculated. Therefore, we propose to use a separate confidence model to estimate d, trained jointly with the classification training of DBLE. regresses d, by learning from the mis-classified training samples in DBLE's classification training. described next.
4 CONFIDENCE MODELING BY LEARNING FROM ERRORS
In the classification model learned by DBLE, test sample x, 's 1.2-distance de to its ground-truth class center (Eq. 7), is highly-calibrated with the model's performance on it, as described in Sec. 3. However, de cannot be directly computed without ground-truth labels, which is the case at inference. Therefore, we introduce a confidence model parameterized by o, to learn to estimate de jointly with the training of classification in DBLE.
To train @ straightforward option would be using the distances for all training samples, {(x, d 1==1' Through this way, & can be trained give an estimate of d, for the test sample x, at inference. How- ever, correctly-classife that state-of-the-arr art DNNs yield much lower training errors compared to test errors (Neyshabur et al., 2014). Therefore, if all data is used, training of 96 would be dominated by the small distances of the corsecty-classified samples, which would make it harder for Is to capture the larger de for the minor mis-classified samples. Moreover, given that choose Ie simple MLP with limited capacity for fast training, it becomes more challenging to capture larger de from the incorrectly sooreectl-ellsssfieed classified samples used (i.e. the confidence model would underfit mis-classified training data).
samples constitute the vast majority during training, especially considering
if all training samples Therefore, we propose to track all training samples that are mis-classified in episode during the training of the classification model. We save them in Me = ((x, Va), where y' + Ve for each episode e to train Jo- In our ablation studies in Sec. 5.4, we demonstrate the importance of learning from mis-classified training samples vs. learning from all.
Next, we introduce the training procedure of 96 with mis-classified samples for estimation of de at inference for test sample t. We train 96 by sampling from the isotropic Gaussian distribution (h,,, diag(a, O r))), where the standard deviation a, is Io 's output given mis-classified training sample x P6 is trained to output a larger o, for x, with a larger d,. Jø takes the representation h, (calculated with Eq. 4) of mis-classified training sample x, in M. {(x.. (6)3 as input, and outputs og as,
0, = 96 (h,)-
To train Pa. firstly sample another representation z, for x. from the isotropic Gaussian distribution parameterized by h, and as. 2₂ ~ Nh,, diag(o, @ a.)). Then, we optimize Jo based on the prototypical loss with 2,, using 2, 's predictive label distribution:
p(yc(xsi)) exp(-d((x. C.y.))
(9)
exp_dd(zz cx)))
When updating & with the prototypical loss given 2,, it encourages larger 0, when the d, of x, larger. This because when h, fixed for x, in the space, maximizing Eq. forces z, to be as close to its ground-truth class center as possible. Given 2, is sampled from N(h, diag(o, @o,)), )) if h, is farther away from its ground-truth center (which the case of mis-classified training samples), then it requires larger da for to go close to it. Fig. 3 visualizes how changes before vs. after updating ø. The training of DBLE is described in Fig. and in Algorithm in the Appendix A.I. Note that in order to make sampling of z, differentiable, we use the reparameterization trick (Kingma & Welling, 2013).
At inference, for every test sample xs, we make prediction 3/ with h, (see Eq. 6). While for confidence estimation of the predictions, we take advantage of or After getting h, and or we sample multiple 2 from ((h, diag(a, @ ox)) and average their predictive label distributions confidence
5

---

Published as a conference paper at ICLR 2020
(a) Before updating Φ
(b) After updating Φ
Figure 3: The Gaussian distribution within one standard deviation away from mean, shown before and after updating Φ. The dotted circles represent the a of the sample inside it. Red and blue dots represent training samples belonging two different classes in the representation space. The dotted line the decision boundary in the space. Let's take the two mis-classified training samples has h, and the correctly classified h, as examples. (a), before updating o, the as of both correctly and wrongly located samples are initialized with small value. (b), after updating @, a of mis-classified samples are much larger. Because the proto-loss for calibration will move sampled from.N(h, do)). to be as close to the correct class center as possible.
estimation:
1 U exp(-d(zi' c22))
Σ En exp(_d(z)) ck))
(10)
U is the total number of representation samples Ze. p(y/|x;;0) Φ) is used as the confidence score calibrating the prediction v/- Through this way, for test sample farther away from its ground-truth class center (which means they are more likely to be mis-classified), the model will add more randomness the representation sampling since its estimated variance from go is large. More randomness in the softmax input space leads to lower expected softmax output (Gal & Ghahramani, 2016), which is the confidence score p(y/ m):().
5 EXPERIMENTS
In this section, we first compare DBLE to recent baselines on confidence calibration of DNNs. We conduct experiments on variety of data sets and network architectures and evaluate DBLE's performance on two most commonly used metrics for confidence calibration: Expected Calibration Error (ECE) (Naeini et al., 2015) and Negative Log Likelihood (NLL) (Friedman al., 2001). Results show that DBLE outperforms single-moda modal baselines on confidence calibration in every scenario tested. We then conduct ablation study to verify the effectiveness of the two main components of DBLE. Implementation, training details, and the details of the baselines are described in Appendix.
5.1 EXPERIMENTAL SETUP
Baselines. We compare our method with 5 baselines that use a single DNN: vanilla training, MC- Dropout (Gal & Ghahramani, 2016), Temperature Scaling (Guo et al., 2017), Mixup (Thulasidasan et al., 2019), Label Smoothing (Szegedy et al., 2016) and TrustScore (Jiang et al., 2018). We also compare DBLE with Deep Ensemble (Lakshminarayanan et al., 2017) with 4 types of DNNs.
Datasets and Network Architectures. We conduct experiments on various combinations of datasets and architectures: MLP on MNIST (LeCun et al., 1998), VGG-11 (Simonyan & Zisserman, 2014) on CIFAR-I0 10 (Krizhevsky et al., 2009), ResNet-50 (He et al., 2016) on CIFAR- 100 (Krizhevsky et al., 2009) and ResNet-50 on Tmny-ImageNer (Deng et al., 2009).
Evaluation Metrics. We evaluate DBLE model calibration with Expected Calibration Error (ECE) and Negative Log Likelihood (NLL). ECE approximates the expectation of the difference between accuracy and confidence. It partitions the confidence estimations (the likelihood of the predicted label p(v/ (xx)) of all test samples into I. equally-spaced bins and calculates the average confidence and accuracy of test samples lying in each bin Ir:
ECE
Σ
Σ 1 z.E1
(11)
Dinael
1=1
where 2 is the predicted label of IG. NLL averages the negative log- likeeiildoood of all test samples:
NLL
Drest
Σ
-og(p((((xx)),
(12)

---

Published as a conference paper at ICLR 2020
Method
MNIST-MLP
CIFARIO-VGGII
Accuracy%
ECE%
NLL
Accuracy% %
ECE%
NLL
Vanilla Training
98.32
1.73
0.29
90.48
6.3
0.43
MC-Dropout
98.32
1.71
0.34
90.48
3.9
0,47
Temperature Scaling
95.14
1.32
0.17
89.83
3.1
0.33
Label Smoothing
98.77
1,68
0.30
90.71
2.7
0.38
Mixup
98.83
1.74
0.24
90.59
3.3
0.37
TrustScore
98.32
2.14
0.26
90.48
0.40
DBLE
98.69
0.97
0.12
90.92
1.5
0.29
Deep Ensemble-4 networks
99.36
0.99
0.08
92.4
1.8
0.26
Method
CIFAR100-ResNet50
Tmne- Ne55ResNet50
Accuracy %
ECE%
NLL
Accuracy% %
ECE%
NLL
Vanilla Training
71.57
19.1
1.58
46.71
25.2
2.95
MC-Dropout
71.57
1.48
46.72
3.17
Temperature Scaling
69.84
2.5
1.23
45.03
4.8
2.59
Label Smoothing
71.92
3.3
1.39
47.19
5.6
2.93
Mixup
71.85
1.44
46.89
2.66
TrustScore
71.57
10.9
1,43
46.71
19.2
2.75
DBLE
71.03
1.1
1.09
46.45
3.6
2.38
Deep E networks
73.58
1.3
0.82
51.28
2.4
1.81
Table 1: Test Accuracy, ECE and NLL of DBLE and baselines under scenarios.
5.2 RESULTS
Table 1 shows the results. every scenario tested, DBLE is comparable with vanilla training in test accuracy applying prototypical learning for classification problems does not hurt generalization of models on classification. On confidence calibration, DBLE performs all dingle-modal baselines on every scenario tested both ECE and NLL. Moreover, our method reaches comparable results with Deep Ensemble of networks with a smaller time complexity and parameter size. For MNIST-MLP our method outperforms Deep ensemble (4 networks) on ECE. Among other baselines, Temperature Scaling performs the best in every scenario, which because Temperature Scaling directly optimizes NLL on the small sub-training set. MC-dropout gives the least improvement on confidence calibration, especially on NI.I. The potential reason for MC-dropout's s limited improvement on NI.L can be that when applying dropout inference, it adds similar level of randomness on mis mis-classified samples and correctly classified samples. Therefore, the predictive likelihood the correct labels becomes smaller general, which leads to worse NLL. Compared Temperature Scaling, DBLE decouples classification and calibration, therefore achieves better calibration without sacrificing classification performance. Compared to MC-dropout, our model is trained to add randomness predictive distributions for mis-classified samples specifically, thus achieving significantly better calibration.
CIFARIO-VGGI and
5.3 COMPUTATIONAL COST
DBLE adds training time complexity and trainable parameters compared with vanilla training. However, compared with Deep Ensembles and other Bayesian methods, DBLE's cost is significantly less. For training time complexity, although DBLE requires two forwards passes for fo each update step (see Algorithm 1), its actual training time is less than twice of vanilla training since the number of iterations required until convergence is smaller. Empirically we observe that DBLE's total training time CIFAR 10-ARIO-VGGI is 1.4x of vanilla training and on CIFAR 1e05Re is 1.7x of vanilla training. While Deep Ensemble of 4 networks takes 4x vanilla training time. DBLE adds extra trainable parameters o compared with vanilla training. ó is the parameters of MLP in practice. It's size usually 1% 3% of the classification model, while ensembling DNNs in Deep Ensemble increases the size of trainable parameters 4x of vanilla training.
7

---

Published as a conference paper at ICLR 2020
Varilla training DBLE
2.75
Vanila training
DELE
2.50
0.20
2.25
0.15
2,00
1.75
0.19
1.50
1.25
60
70
30
100
70
80
100
Epoch
Epoch
Figure 4: Test ECE and NLL of the last 45 epochs during training on CIFAR-100. The learning rate of both vanilla training and DBLE is annealed by 10x at epoch 60, 70, 80.
Method
CIFAR100
Tiny-ImageNed
ECE%
NLL
ECE%
NLL
Vanilla Training
19.1
1.58
25.2
2.95
Learning with errors in vanilla training DBLE with calibration learning using all samples DBLE
18.3
1.43
20.9
2.61
18.9
1.54
24.8
2.87
1.1
1.09
3.6
2.38
Table 2: Ablation study on CIFAR-100 and Imny-ImageNet
5.4 ALGORITIMM ANALYSIS AND ABLATION STUDY
DBLE alleviates the NLL overfitting problem. NLL overfitting problem of vanilla training has been observed in (Guo et al., 2017) after annealing the learning rate, as the test accuracy goes up, the model overfits to NLL score (test NI.L starts to increase instead of decreasing or maintaining flat). Fig. 4 shows that this phenomenon is alleviated by DBLE. Under vanilla training, the model starts overfitting to NLL after the first learning rate annealing. DBLE on the other hand, decreases and maintains its test NLL every time after learning rate annealing. We see the same trend for test ECE as well - with vanilla training the model overfits to ECE after learning rate annealing while DBLE decreases and maintains its test ECE.
Distance-based representation space and learning from errors are both essential for DBLE. We conduct ablation studies to verify the effectiveness of the main design choices of our model, the distance-based space and learning calibration from training errors. Table 2 shows the results. In "Learning with errors in Vanilla training"" training. In other words, instead of updating & by maximizing Eq. 9, we update á by maximizing the softmax likelihood with z as logits and fe is updated with vanilla training. In "DBLE with calibration learning using all samples", we update & with all training samples by maximizing Eq. 9. We can see from the results that firstly, learning to calibrate with errors also helps vanilla training. improves calibration in vanilla training slightly since it to some extent, introduces more randomness in the decision making process for misclassified samples. However, the improvement is very small without the distance-based space. Secondly, we notice that in DBLE if we learn calibration with all training samples, significantly decreases DBLE's s performance on calibration. The potential reason is the model's 's underfitting to mis-classified training samples in learning to estimate the distance. Auxiliary analysis on these two can be found in Appendix A.3.
conduct calibration learning with errors in vanilla
CONCLUSION
Confidence calibration of DNNs, which has significant practical impacts, still remains an open problem. In this paper, have proposed Silance-BBsedd Learning from Errors (DBLE) to try to solve this problem. DBLE starts with applying prototypical learning to train the DNN classification model. It results in a distance- based representation space in which the model's performance sample is calibrated by the sample's 's distance to its ground -trut class center. DBLE then trains a confidence model with mis-classio classiisd training samples for confidence estimation. We have empirically shown the effectiveness of DBLE on various classification datasets and DNN architectures.
test
8

---

Published as a conference paper at ICLR 2020
7 ACKNOWLEDGMENTS
Discussions with Kihyuk Sohn are gratefully acknowledged.
REFERENCES
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. arXiv preprint 4rXiv:1505.05424.222 2015.
Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT'2010. Springer, 2010.
Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, Hyrum S. Anderson, Heather Roff, Gregory C. Allen, Jacob Steinhardt, Carrick Flynn, Seán Ó hEigeartaigh, Simon Beard, Haydn Belfield, Sebastian Farquhar, Clare Lyle, Rebecca Crootof, Owain Evans, Michael Page, Joanna Bryson, Roman Yampolskiy, and Dario Amodei. The malicious use of artificial intelligence: Forecasting. prevention, and mitigation. arXivv
2018.
Deng. Dong, R. Socher, L. Li, Li, and L.. Fei-Fei. ImageNet: Large-Scale Hierarchical Image Database. In CVPR, 2009.
Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning, volume 1. Springer series in statistics New York, 2001.
Yarin Gal and Zoubin Ghahramani. Dropout bayesian approximation: Representing model uncertainty in deep learning. In ICMI, 2016.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, 2011.
Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume MIT Press, 2016.
Alex Graves. Practical variational inference for neural networks. In NIPS, 2011.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In ICML, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Jonathan Heek and Nal Kalchbrenner. Bayesian inference for large scale image classification. arXiv preprint arXiv: 00..99999999999 2019.
Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust not to trust classifier. In NIPS, 2018
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 13 14, 2013.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, 2009.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. NIPS, 2017.
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient- based learning applied to document recognition. Proceedings of the IEEE, 86(11):227--2324 1998.
Rafael Müller, Simon Kornblith, and Geoffrey Hinton. When does label smoothing help? arXiv preprint arXiv: 1rXiv:/906.000 1et..999969.9... 2019.

---

Published as a conference paper at ICLR 2020
Mahdi Pakdaman Nacini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated proba- bilities using bayesian binning. In AAAI, 2015.
Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning. arXiv:/412.66/ 2014.
Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. arXiv: 2014.
Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. Tadam: Task dependent adaptive metric for improved few-shot learning. In NIPS, 2018.
Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. 2016.
Raanan Rohekar, Yaniv Gurwicz, Shami Nisimov, and Gal Novik. Modeling uncertainty by learning hierarchy of deep neural connections, arXiv preprint arXiv:1905. 2019.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NIPS, 2017.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. The journal of machine learning research, 2014.
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance initialization and momentum in deep learning. In 2013,
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR, 2016.
Sunil Thulasidasan, Gopinath Chennupati, Jeff Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. arXiv preprint arXiv:1905.11001 2019.
Chen Xing, Negar Rostamzadeh, Boris N Oreshkin, and Pedro O Pinheiro. Adaptive cross-modal few-shot learning. arXiv preprint arXiv: 1rX2v:/902.077 104, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint 1rX1v:1710.09412. 2017.
10

---

Published as a conference paper at ICLR 2020
A APPENDIX
A.1 ALGORITHM OF ONE UPDATE OF DBLE
One update of DBLE. M is the total number of classes in the training set, N is the number of classes in every episode, K is the number of supports for each class, Ko the number of queries for each class.
Algorithm 1:
Input: Training Drain = {(x; vi)]i i: yi E (1,.., M). D {(x,, Us) E Dunin y: = c).
# Build training episode ## Select N classes for episode
C RandomSample((1) M,, M)
## sample supports and querics for every class in e for in C do
S: f- Random trin" K) Q RandomSamplle De train Sc Ko)
end for
# Compute Loss
## Compute center representation for every class e in
for C'
c |ssl Le fo(8;)
end for
## Compute prototypical loss for classi fication
C(0)
for e C' do
for (x,, vi) in Qc do
h, fx(x;)
((A) ((0) + N-K [d(,,, Cy.) logE, exp(-d(h,, cx))]
end for
end for
# Compute confidence Loss
C(Ø) +
### Make predictions and track mis-classified training samples
M, = {}
for in C do
for (xi, yi) in O: do y = argmin, (d(,,, co)
#h, = fo(x))
if vo4 + yr then
MM AddT (()
end if end for end for # ## Compute confidence loss with mis-classified training samples for (x,, in Me do
Jo = 96 (h)) #h. = fo(u.)
<~ N(0,1)
2, = h +6-0x
(()) L(d) + N.K Cy.) + logE, p(pppd((,,))))) ck))]
end for
# Update with prototypical loss for classification
- -- VL())
# Update á with confidence loss
o vL(
A.2 IMPLEMENTATION AND TRAINING DETAILS OF DBLE AND BASELINES
For fair comparison with baseline methods, in every scenario tested, the network architecture is identical for DBLE and all other baselines. As regularization techniques such as BatchNorm, weight
11

---

Published as a conference paper at ICLR 2020
DBLE
Number of samples Dimensions
300000
DBLE with calibration learning using samples
250000
200000
150000
100000
50000
0.0
0.5
1.0
1.5
2.0
Standard Deviation Figure 5: The histogram of of models trained with all training samples vs. with mis-classified training samples on CIFAR-10.
decay observed affect confidence calibration (Guo al., 2017), also keep them constant while comparing to baselines in every scenario. All models are trained with stochastic gradient descent with momentum (Sutskever al., 2013). We use initial learning rate of 0. and fixed momentum coefficient of 0.9 for all methods tested. The learning rate scheduling is tuned according classification performance on validate set. All other hyperparameters of classification models for baselines are also chosen based on accuracy validation set.
There are several unique hyper-parameters in DBLE. We describe how we choose them in the following paragraph. In DBLE, we stop the training when the classification model converges. The confidence model 9a two-layer MLP with Dropout (Srivastava et al., 2014) added in between. we use ReLU non-linearity (Glorot et al., 2011) for the MI.. We fix the dropout rate as 0.5. The N (number of classes), K (number of shots) and the batch-size of the query set for every episode in DBLE's are firstly tuned according the classification performance comparable performance with vanilla training. Then the N. K and the batch-size the query set for every episode fine-tuned according to the DBLE's calibration performance validation set. At inference, we fix the number of representation sampling U in Eq. 10 as 20. This is because we empirically observed that U larger than 20 doesn't give performance improvements. We set the number of bins L in Eq. as 15 following (Guo et al., 2017).
validation set to reach
A.3 TRAINING THE CONFIDENCE WITH ALL TRAINING SAMPLES VS. WITH MIS-CLASSIFIED TRAINING SAMPLES
In addition to the final results we report in the ablation study (the last two rows of Table 2) comparing training the confidence with all training samples vs. with mis-classified training samples, we also checked statisties of a, which is the standard deviation predicted by the confidence model. For CIFAR-10 10, the mcan of of the model trained with all samples is 0.203, while that of the model trained with mis-classified training samples is 0.740. This suggests that if train the confidence model with all training samples, the model outputs for test samples are indeed smaller in general, which aligned with our intuition. The histogram of under these methods are shown in Figure 5.
12


Reviews:
[{"review_id": "rkgRPJHRYr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper describes an approach to improve the confidence on deep\nneural networks (DNN). The proposed approach uses a distance-based\napproach (distance to prototypes) and train using a confidence\nmodel with the classification model using mis-classified examples.\n\nIt first learns using a loss function based on pre-computed centroids\nof each class evaluated from the training examples. At inference it\nassigns the class with the closest center.\n\nTo estimate a confidence level, it learns a new model to estimate the\ndistance using only the misclassified examples.\n\nThe authors experimentally showed the benefits of the proposed approach\nover several methods.\n\nFinding prototypes and training with distances, on one hand, and\nevaluating confidence with Gaussian, on the other, assumes\nthat the classes are \"well defined\". Although the authors show\npromising results, it is not clear how well the proposed method will\nbehave in more challenging classification tasks where the classes are\nmixed.\n\nAlthough the authors show that estimating the confidence with\nmisclassified examples works better I will like to see a further\nanalysis in the paper.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "r1llj41atS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe paper proposes a method to do confidence calibration for deep neural networks. It uses standard\nepisodic training for prototypical networks, and first shows empirically that the distances of the\nembedded test point to its ground truth class center embedding (*not* the predicted class embedding)\nare indicative of the confidence of the prediction. Further it proposes to exploit this by training\nan auxiliary confidence prediction MLP carefully. To do so they demonstrate that the training needs\nto be done of erroneously predicted training examples cf. all the traning examples. They show\nresults with MLP+MNIST, VGG11+CIFAR10, ResNet50+CIFAR100 and ResNet50+TinyImageNet.\n\n\nDetailed comments:\nThe paper is interesting but largely empirical. It shows empirically that:\n1. when prototypical networks (and episodic training) is used the distance of test example to true\nclass center reflects the confidence.\n2. this does not hold when `vanilla' training is used\n3. an auxiliary MLP can be used to learn to predict this while training only with erroneously \nclassified training examples cf. all training examples\n                                      \nThe results are reported on three networks (MLP, VGG11 and ResNet50) on different benchmarks of \nimage classification. The confidence prediction improvements wrt baselines are non trivial, while \nkeeping the accuracy similar, and the computation cost lower than competing methods. Ablations \nstudies are also convincing.                    \n            \nI would have two broad critical comments on the paper:\n1. Would this generalize to other image classification tasks and datasets. Generally distance\n(embedding) based networks perform less than softmax based networks on bigger datasets, so an\nimmediate disadvantage if that happens, is that you would be trading off accuracy cf. vanilla\nnetworks, for better confidence prediction using the required distance based network here.\n2. The vanilla training is never formally detailed. I am assuming it was softmax + cross entropy loss \nwith gradient descent. Would some other loss be helpful? Specially the metric learning based losses like \ncontrastive or triplet losses come to mind, since they are also distance based. Does the method work\nwith prototypical networks only or it generalizes to other distance based methods as well?\n                                                                                    \nMinor comments:\nThe notations are a bit confusing sometimes, and require going back and forth a bit. Eg. \\mu is used\nfor representation of feature (Eq4) while it usually denotes a mean of some sort (so the reader's expectation\ncould be that it represents class center). Similarly, boldface p is used for class centers, which is\nagain a bit confusing as being a probability of some sort. In general the notations are different\nfrom the original prototypical networks paper (which I needed to revise); keeping them similar would\nhelp the reader. The contribution of the present paper is more than that anyway.\n\nThe erroneously classified training examples are called errors (eg. just before eq8). By errors one\ncould think that it is a difference between some sort of prediction and the ground truth. Explicitly\ncalling them erroneously classified training examples would help the reader as well.\n\nThe notation \\odot is not explained (before eq.9).\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HkxX4zVCKB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper addresses the (long-standing) problem of classification systems being able to output reliable confidence estimates on its own output. The selected approach is to use the distance to (previously computed) class centers in multi-class classification to help compute the confidence interval. The method is shown to have comparable results on well known datasets but higher efficiency than 2 rival methods: ensemble ANNs and Bayesian neural networks. I would point out that such confidence intervals are all intuitive and have no statistical basis or other independent means of empirical validation. Experienced practitioners are aware of this, but I see that the paper steers wisely clear of overambitious claims. The general intuition of hybrid supervised and unsupervised learning for C.I. (or ellipse) estimation is not new, but an effective and compact representation of this intuition in a DNN context is a valuable contribution, well-placed in the literature - I recommend acceptance.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers voted to accept this paper.
The AC recommends acceptance.
Decision: Accept (Poster)