Should Ensemble Members Be Calibrated?
Authors: Xixin Wu
Mark Gales
Venue:  (2021)

Abstract: Underlying the use of statistical approaches for a wide range of applications is the assumption that the probabilities obtained from a statistical model are representative of the “true” probability that event, or outcome, will occur. Unfortunately, for modern deep neural networks this is not the case, they are often observed to be poorly calibrated. Additionally, these deep learning approaches make use of large numbers of model parameters, motivating the use of Bayesian, or ensemble approximation, approaches to handle issues with parameter estimation. This paper explores the application of calibration schemes to deep ensembles from both a theoretical perspective and empirically on a standard image classification task, CIFAR-100. The underlying theoretical requirements for calibration, and associated calibration criteria, are first described. It is shown that well calibrated ensemble members will not necessarily yield a well calibrated ensemble prediction, and if the ensemble prediction is well calibrated its performance cannot exceed that of the average performance of the calibrated ensemble members. On CIFAR-100 the impact of calibration for ensemble prediction, and associated calibration is evaluated. Additionally the situation where multiple different topologies are combined together is discussed.

Under review as a conference paper at ICLR 2021
SHOULD ENSEMBLE MEMBERS BE CALIBRATED?
Anonymous authors
Paper under double-blind review
ABSTRACT
Underlying the use of statistical approaches for a wide range of applications is the assumption that the probabilities obtained from a statistical model are represen- tative of the "true" probability that event, or outcome, will occur. Unfortunately, for modern deep neural networks this is not the case, they are often observed to be poorly calibrated. Additionally, these deep learning approaches make use of large numbers of model parameters, motivating the use of Bayesian, ensemble approximation, approaches to handle issues with parameter estimation. This pa- per explores the application of calibration schemes to deep ensembles from both a theoretical perspective and empirically on a standard image classification task, CIFAR-ION 100. The underlying theoretical requirements for calibration, and associ- ated calibration criteria, are first described. It shown that well calibrated ensem- ble members will not necessarily yield well calibrated ensemble prediction, and if the ensemble prediction is well calibrated its performance cannot exceed that the average performance of the calibrated ensemble members. On CIFAR-100 the impact of calibration for ensemble prediction, and associated calibration is evalu- ated. Additionally the situation where multiple different topologies are combined together is discussed.
INTRODUCTION
Deep learning approaches achieve state-of-the-art performance in a wide range of applications, in- cluding image classification. However, these networks tend to be overconfident in their predictions, they often exhibit poor calibration. A system is well calibrated, if when the system makes pre- diction with probability of 0.6 then 60% of the time that prediction is correct. Calibration is very important deploying system, especially in risk-sensitive tasks, such as medicine (Tiang et al.) 2012), auto-driving (Bojarski et al.) 2016), and economics (Gneiting et al. [2007). It was shown by Niculescu-Mizil Caruana (2005) that shallow neural networks are well calibrated. However, Guo al. (2017) found that more complex neural network model with deep structures do not exhibit the same behaviour. This work motivated recent research into calibration for general deep learning systems. Previous research has mainly examined calibration based samples from the true data
Ji=l p(x, w), (()) e [WI,... WK) (Zadrozny & Elkan 2002 Vaicenavi- cius et al. 2019). This analysis relies on the limiting behaviour as + 400 to define a well
distribution (x( ( I
calibrated system
P(y (|P(j) = p) =p lim
N |ST1
p
YES!
where 5.¹ = ()))((()) = () = p.i N} and g()) the model prediction for ( 5(x,1) = if otherwise O. However, Eq. ( doesn't explicitly reflect the relation between Ply = (PP(ū)=;;)) = p) and the underlying data distribution p(æ, y). In this work we examine this explicit relationship and use it to define range of calibration evaluation criteria, including the standard sample- criteria.
One issue with deep-learning approaches is the large number of model parameters associated with the networks. Deep ensembles (Lakshminarayanan et al., 2017) is a simple, effective, approach for handling this problem. It has been found improve performance, of uncertainty. In recent literature there has been "contradictory" empirical observations about the relationship between the calibration of the members of the ensemble and the calibration of the final
well allowing measures

---

Under review as a conference paper at ICLR 2021
ensemble prediction (Rahaman & Thiery 2020; Wen et al. 2020). In this paper, examine the un- derlying theory and empirical results relating to calibration with ensemble methods. We found, both theoretically and empirically, that ensembling multiple calibrated models decreases the confidence of final prediction, resulting in an ill-calibrated ensemble prediction. To address this, strategies to calibrate the final ensemble prediction, rather than individual members, are required. Addition- ally we empiricaly examine the situation where the ensemble is comprised of models with different topologies, and resulting complexitypperfommnnce, requiring non-uniform ensemble averaging.
In this study, we focus on post-hoc calibration of ensemble, based on temperature annealing. Guo al. (2017) conducted thorough comparison of various existing post-hoc calibration methods and found that temperature scaling was a simple, fast, and often highly effective approach to calibration. However, standard temperature scaling acts globally for all regions of the input samples, i.e. all logits are scaled towards one single direction, either increasing or decreasing the distribution entropy. To address this constraint, that may hurt some legitimately confident predictions, we investigate the effect of region-specific temperatures. Empirical results demonstrate the effectiveness of this approach, with minimal increase in the number of calibration parameters.
2 RELATED WORK
Calibration is inherently related to uncertainty modeling. Two of the most important scopes of cali- bration are calibration evaluation and calibration system construction. One method to assessing cal- ibration is the reliability diagram (Vaicenavicius et al. 2019: Bröcker 2012). Though informative, It is still desirable have overall metric. Widmann et al. 2019) investigate different distances in the probability simplex for estimating calibration error. [Nxon et al. (2019) point out the problem of fixed spaced binning scheme, bins with few predictions may have low-bias but high-variance mea- surement. Calibration error measure adaptive to dense populated regions have also been proposed (Nixon et al. 2019). Vaicenavicius
al. (2019) treated the calibration evaluation hypotheses tests. All these approaches examine calibration criteria from a sample-base perspective, rather than as a function of the underlying data distribution which is used in the thoretical analysis in this work.
There are two main approaches to calibrating systems. The first is to recalibrate the uncalibrated systems with post-hoc calibration mapping, e.g. Platt scaling (Platt et al., 1999). isotonic regres- sion (Zadrozny & Elkan 2002). Dirichlet calibration (Kull et al., 2017 (019). The second is to directly build calibrated systems, via: (i) improving model structures, e.g. deep convolutional Gaus- sian processes (Tran et al.) 2019); (ii) data augmentation, e.g. adversarial samples (Hendrycks &
Dietterich] 2019; Stutz et al. (020) or Mixup (Zhang et al., 2018); (iii) minimize calibration error during training (Kumar et al. 2018). Calibration based on histogram binning (Zadrozny & Elkan 2001]. Bayesian binning (Naeini et al., 2015) and scaling binning (Kumar et 20199) are related our proposed dynamic temperature scaling, in the sense that the samples are divided into regions and separate calibration mapping are applied. However, our method can preserve the property that all predictions belonging one sample sum 1. The region-based classifier by Kuleshov & Liang
(2015) is also related to our approach.
Ensemble diversity has been proposed for improved calibration (Rafiery et al., 2005 Stickland & Murray, 2020). In Zhong & Kwok (2013), ensembles of SVM, logistic regressor, boosted decision trees are investigated, where the combination weights of calibrated probabilities is based on AUC of ROC. However, AUC is not comparable between different models as discussed in (Ashukha et al. 2020). In this work we investigate the combination of different deep neural network structures. The weights assigned to the probabilities is optimised using a lkelihood-basee metric.
3 CALIBRATION FRAMEWORK
Let X C Rdd be the d-dimensional input space and y = [..,,.,.,]] be the discrete output space consisting of classes. The true underlying joint distribution for the data is p(x,w) P(w)#)p(z), E X, w E y. Given some training data D ~ p(æ, w), model 0 is trained to predict the distribution P(W x=; 0) given observation features. For a calibrated system the average predicted posterior probability should equate to the average posterior of the underlying distribution for spe- cific probability region. Two extreme cases will always yield perfect calibration. First when the predictions that the same, and equal to the class prior for all inputs, P(wy a;; 0) P(w;). Sec-
2

---

Under review as a conference paper at ICLR 2021
ond the minimum Bayes' risk classifier is obtained, P(wy|æ; 0) calibration doesn't t imply high accuracy, as shown by the system predicting the prior distribution.
p(a,wy
Note that perfect
Σx p(ar,
3.1 DISTRIBUTION CALIBRATION
A system is calibrated if the predictive probability values can accurately indicate the portion of correct predictions. Perfect calibration for system that yields P(w|a; 0) when the training and test data are obtained form the joint distribution p(æ,, w) can be defined as:
Jt JEERZIEN))
Plwy a; O)p(x)dd
JEERZION P(wyla)p(xrddr Vp,wy,
(2)
R5(0.e)
{(wy 0) -p
(2, c) denotes the region of input space where the system predictive probability for class wj is sufficiently close, within error of to the probability p. A perfectly calibrated system will satisfy this expression for regions, the expected predictive probability (left side Eq. (2) is identical to the expected correctness, i.c., expected true probability (right side of Eq. (2).
R²(0, (0, E) defines the region which calibration is defined, For top-label calibration, only the most is modified to reflect this:
probable class is considered and the region defined in Eq. 3
wo arg max a 0), x}
(4)
Eq. (4) is a strict subset of Eq. 33. As the two calibration regions are different between calibration and top-label calibration, perfect calibration doesn't imply top-label calibration, and vise versa. A simple illustrative example of this property is given in A.3] Binary classification, exception this general rule, the regions for top-label calibration are equivalent to those for perfect calibration, i.e. R,, calibration for binary classification (Nguyen & O'Connor 2015).
= 2, is an
(,,0,)) c). Hence, perfect calibration is equivalent to top-label
Eq. 2 defines the requirements for a perfectly calibrated system. It is useful define metries that allow how close system is to perfect calibration to be assessed. Let the region calibration error be:
(0, = S (P(w) ax; 0) -
(5)
This then allows two forms of expected calibration losses to be defined
AC
1 K Σ c ( 6)
/
dp: ACCE(
1 K
Σ
f CT c
(6)
All Calibration Error (ACE) only considers the expected calibration error for particular probability, irrespective the class associated with the data (Hendrycks et al. 2019). Hence, All Class Calibra- tion Error (ACCE) that requires that all classes minimises the calibration eπror for all probabilities advocated by Kull al. 2019 Kumar al. (2019). Nixon al. (2019) propose the Thresholded Adaptive Calibration Error (TACE) to consider only the prediction larger than threshold, and it can be described as a special case ACCE by replacing the integral range. [aeini et al. (2015) also propose to only consider the region with maximum error.
Though measures such as ACE and ACCE require consistency of the expected posteriors with the true distribution, for tasks with multiple classes, particularly large numbers of classes, the same weight is given to the ability of the model to assign low probabilities to highly unlikely classes, and high probabilities artificially low scores. To address this problem it is more common to replace the regions in Eq. (5) with the tabelabel regions in Eq. (4) to give a topelabel calibration error E7 (,, (0,c). c). This then yields
the "correet" class. For systems with large numbers of classes this can yicld
In this section the references given refer to the sample-hased equivalent versions of the distributional cali- bration expressions in this paper using the same concepts, rather than identical expressions.
3

---

Under review as a conference paper at ICLR 2021
the expected top-label equivalents of ACCE and ACE, Expected Class Calibration Error (ECCE)
and Expected Calibration Error (ECE). Here for example ECE by Guo et al. 2017 is expressed as
ECE(O) = !. f
K
Σ
(P(w) a 0) ((w)(=))p((x)dd dp (0
5=1
!.
0 p) Conf (0.p) - acc(0,p))dt
where O(0,p) Σø= (0.3) p(æ)dd is the fraction observations that are assigned to that par-
ticular probability and Conf p) and Acc(O.p) p) are the ideal distribution accuracy and confidences from the model for that probability. For more details see the appendix.
3.2 SAMPLE-BASED CALIBRATION
Usually only samples from the true joint distribution are available. Any particular training set is drawn from the distribution to yield
D =
{ (z y00)
(o:"), y00 w(x, y E (wi,-.,wk).
The region defined in Eq. 3 is now changed to be indices of the samples:
(0,
[[Wy ( ;) pp S e, ( }
(9)
The sample-based version of "perfect" calibration Eq. 2 can then expressed as:
1
Σ (0)(
(gg()w)) Vp P(wyl 0) = (0,)) (0.)(0.))
(10)
e)l
(ES3(0,))
as N + oc. When considering finite data, in this case N samples, it is important to set appro- priately. Setting different yields different regions and leads different calibration results (Kumar et al. 2019). Thus it is important to specify when defining calibration for system.
Similarly, the distribution form of top-label calibration can be written in terms of samples Eq. with different regions considered:
S( filws arg max P(w (0) 0), x (1) E D}
(11)
The cumple-based calibration losses in region ( E) can be defined based on Eg. 10 10. For exam-
ple ACE Eq. (6) can be expressed in its sample-base based form (Hendrycks al. 2019)
ACE(O, NK
Σ
Σ
Σ
(P(wy (1 ((y(1,
(12)
PEP(c)
j=l iES)
where P(c) {plp min(1, (1, (2z 1)c}, E Z+}, and z is the set of positive integers. The
measure ECE relating Eq. (7). which only considers the top regions in Eq. (TT) can defined as Guo et al. (2017)
(C =
Σ
Σ
Σ
)()(0) (
(13)
N
pEP(e)
j=1
iES (0,c)
Σx 1 (0, (),+
Conf Conf(0, Acc(O.p)
(14)
N
It should be noted that for finite number of samples, the regions (, e) and S (0. derived from the samples can different from the theoretical regions, leading to difference between theoretical calibration error measures and the values estimated from the finite samples. This is also referred to

---

Under review as a conference paper at ICLR 2021
"estimator randomness" by Vaicenavicius et al. (2019). An example is given in A.3 to illustrate this mismatch.
The simplest region specification for calibration is to set 1. In this case, ( the "minimum" perfect calibration requirement for system with parameters 0 becomes
(0, 1)1 N, and
N P(wy ar (1)
1 N 1=1
Σ ((y(,,0))
Vug
(15)
N
This is also referred to as global calibration in this paper. Similarly, global top-label calibration can be defined
N
Σ
P P
12
Σ 8(y .0))
ge) = arg max (1; (0:
(16)
4 ENSEMBLE CALIBRATION
An interesting question when using ensembles is whether calibrating the ensemble members suf- ficient to ensure calibrated predictions. Initially the ensemble model will be viewed an approxi- mation to Bayesian parameter estimation. Given training data D D, the prediction class wi is:
P(wyl x0", .D (o-ss00))/(w//>a
P(w,læ* ())
1 P(w,læ' : M P(wj|a' ((m));
θ @(m)
p(0 D)
(17)
where Eq. 17 is ensemble, Monte-Carlo, approximation the full Bayesian integration, with (m) the m-th ensemble member parameters in the ensemble O. The predictions of ensemble and
members are i'n = arg max (P(wa=" ((m))))))))))))))))))))))))))))) ((m))), dh = arg max { ΣM= P(w) x"; 06m)) }
4.1 THFORETICAL ANALYSIS
For ensemble methods it is only important that the final ensemble prediction, dz. is well calibrated, rather than the individual ensemble members. It is useful to examine the relationship between this ensemble prediction and the predictions from the individual models when the ensemble members calibrated. Consider particular top-label calibration region for the ensemble prediction, R*))),). related to Eq.
(4) the following expression is true
M
M
[ (E.)(( M
Σ
P(je
x 0 ()) )p(x)dx S / (EER)(O..) M
Σ P(im x @(m) )p(x)dx (18) m=1
m=1
pp e e, x}. For all where the ensemble region is defined as (², (
regions ( the ensemble is more confident than the average confidence of individual mem- ber predictions. This puts bounds on the ensemble prediction performance if the resulting ensemble prediction is top-label calibrated, and all ensemble members yield the same region (e,,))) Here
[ (
P(ee|æ; O)p(x)dæ=
(19)
(aER(((0..)
From Eq. (18) the left hand-side of this expression, the ensemble prediction confidence, cannot be greater that than the average ensemble member confidence. If the regions associated with the ensemble prediction and members are the same, then for top-label calibrated members this average confidence is the same the average ensemble member accuracy. Furthermore, if the ensemble prediction is top-label calibrated, then this average ensemble member accuracy bounds the ensemble prediction accuracy. Under these conditions ensembling the members yields no performance gains.
The above bound holds with the assumption that the members are calibrated on the same regions. Proposition 3 in Appendix describes one trivial case when all members calibrated the same regions. Another case is the calibration on global regions. As shown in Proposition | at the global level, ensemble accuracy is still bounded.
5

---

Under review as a conference paper at ICLR 2021
Proposition 1. If all members and the corresponding ensemble are globally top-label calibrated, the ensemble performance no better than the average performance of the members:
M
N
Σ
S
8 (0)
(20)
N
Proof. If members and the ensemble are globally top-label calibrated,
P
P (
(1) g(m))
9m )
m
1,.., M
(21)
N
M
N
Σ
( x (9) p(())) =
5(
(22)
N
M
N
m==1
By definition,
P(UE (0; ((m)) S P(O (+) g(m))
P
(23)
Hence,
N
M
N
8 (t)
ye
(8)
Σ
8 (0) 0 )10)
(24)
N
M
N
m=1
However, this is not true for all-label calibration. In both cases, all-label calibrated members always yield all-label calibrated ensemble, no matter whether the ensemble accuracy exceeds the mean accuracy of members or (Example 2 in Appendíx gives illustration synthetic dataset).
Proposition If all members are global all-label calibrated, then the overall ensemble is global all-label calibrated.
Proof. If all members are global all-label calibrated, then
N Ywjj m = 1,... M
(0 =
5ly (,,y)
(25)
N
N
Hence,
N ())==
1 M
N
1 N =
P(wy
Σ
8(y (1) woy))
Σ N
8 ( wj)
(26)
N
M m=1
N
1=1
1=1
In general the regions are not the same, the ensemble accuracy is not bounded the above way. However, note that global level calibration is the minimum requirement of calibration. The above discussion based on regions still sheds light on the question of should the members be calibrated or not, though the final theoretical answer is still absent. It should be also noted that, global all-label calibration does not imply global top-label calibration, because the regions considered are different (as illustrated by Example I in Appendix). For the discussion so far, the ensemble members are combined with uniform weights, motivated from a Bayesian approximation perspective. When, for example, multiple different topologies are used members of the ensemble, non-uniform averaging of the members of the ensemble, reflecting the model complexities and performance may be useful. Propositions and will still apply.

---

Under review as a conference paper at ICLR 2021
4.2 TEMPERATURE ANNEALING FOR ENSEMBLE CALIBRATION
Calibrating ensembles can be performing using a function f E F. with some parameters, t, F [0, 1] [0, 1] for scaling probabilities. There are two modes for calibrating an ensemble: Pre-combination Mode. the function is applied to the probabilities predicted by members, prior to combining the members obtain ensemble prediction using a set of calibration parameters T.
M
(P(0E) Olm)), {(m)
Prro Del0: O)
(27)
M
Post-combination Mode. the function is applied to the ensemble predicted probability after com- bining members' predictions.
((1
M
Posnt (UE [x; 1,
Σ
P(ie la; ((m))
(28)
M
There are many functions for transforming predicted probability in the calibration literature, c.g. histogram binning, Platt scaling and temperature annealing. However, histogram binning shouldn't be adopted the pre-combination mode as scaling function f for calibrating multi-class ensemble, the transformed values may not yield valid PMF.
As shown in Guo et al. 2017). temperature scaling is a simple, effective, option for the map- ping function F. which scales the logit values associated with the posterior by a temperature f(z;t) = exp{= /tp/z/tt//,, exp(zy/t}. Here single temperature is used for scaling logits for all samples. This leads to the problem that the entropy of the predictions for all regions are either increased or decreased. From Eq. (2) the temperature can be made region specific.
exp exp(z/tr)
if max exp(z1} Σ, E Rr
(z; Σ, explzy/t,
(29)
To determine the optimal set of temperatures, the samples in the validation set divided into R regions based on the ensemble predictions (e.g. R2 = (0, 0.3). R₂ = [0.3, 0.6), and R3 [0.6, 1).. Each region has an individual temperature for scaling {R,, 1
4.3 EMPIRICAL RESULTS
Experiments were conducted on CIFAR-100 (and CIFAR-10 in the A.4). The data partition was 55,000/5,000/100 10,0000000000000 images for train/validationntess.. We train LeNet (LEN) (LeCun et al. 1998), DenseNet 100 and 121 (DSNI00. DSN121), (Huang et al. 2017) and Wide ResNet 28 (RSN) Kagoruyko & Komodakis 2016) following the original training recipes in each paper (more de- tails in A.4) The results presented are slightly lower than that in the original papers, 5,000 images were held-out to enable calibration parameter optimisation.
Figure examines exxamins the empirical performance of ensemble calibration on CIFAR-100 test set using the three trained networks. The top row shows that, with appropriate temperature scaling, the mem- bers are calibrated on different regions (because otherwise the accuracy values should be the same). The middle row shows the ECE of ensemble members and ensemble prediction at different tem- peratures. The optimal calibration temperature for the ensemble prediction are consistently smaller than those associated with the ensemble members. This indicates that the ensemble predictions are less confident than those of the members, as stated in Eq. (23). The bottom row of figures show the reliability curves when the ensemble members are calibrated with optimal temperature values, and the resulting combination. It is clear that calibrating the ensemble members, using temperature, does not yield calibrated ensemble prediction. Furthermore for all models the ensemble prediction is less confident than it should be, the line is above the diagonal. As discussed in Proposition this is necessary, the ensemble prediction is no better, which is clearly not the case for the performance plots in the top row. This ensemble performance is relatively robust to poorly calibrated ensemble members, with consistent performance over wide range of temperatures.
Table [ shows the calibration performance using three temperature scaling methods, pre-, post- and dynamic post-combination. The temperatures are optimized to minimize ECE (Liang et al.
7

---

Under review as a conference paper at ICLR 2021
(a) LEN
(b) DSNI0O
RSN
0.50
240
1.00
AU2.000
4.50
6010
Tampirtus
Tamparorr
Tampxssus
07
05
0
811
05
02
02
64 05 5. 60 0.2 2.4 0.5
0.3
10 32
02 65
86
Figure 1: Top-label calibration error and accuracy of members (mem) and the whole ensemble (ens) on CIFAR- 100 (test set) using LeNet, DenseNet and ResNet. "pre" denotes the calibration where shared temperature is applied to members before combination. The reliability curves shows the calibrated members and calibrated ensembles with optimal temperature values.
2020) on the validation data. We use the unbiased quadratic version of squared kernel calibration error (SKCE) with Laplacian kernel and kenel bandwidth chosen by median heuristic as one of the calibration error (Wirmann et al., 2019) All three methods effectively improve the ensemble prediction calibration, with the dynamic approach yielding the best performance. We further investigate the impact of region numbers on the dynamic approach, as shown in Figure It can be found that increasing the region number tends to improve the calibration performance, while requiring more parameters.
Finally, for the topology ensemble, weights were optimised using either maximum likelihood (Max LL) or area under curve (AUC) Zhong & Kwok (2013) (results in A.4p. In Figure 2 the ensemble of calibrated structures is shown to be uncalibrated, with reliability curves typically slightly above the diagonal line. When the ensemble prediction is calibrated it can be seen that the calibration for the ensemble prediction is lower than the individual calibration errors in Table (""pst" lines).
Traskde Mid.eiiiiiideee
LEN+ BEX-DDSNI0O-DSNIII-RY DSN 1ONIODSSSNIN
DSNI21+RSN
Figure 2: Reliability curves of weighted combination of calibrated structures, LEN, DSNI00, DSNI21 and RSN CIFAR-100. The weightes are estimated by Max LL. Each structure is an ensemble of 10 models.
8

---

Under review as a conference paper at ICLR 2021
Table 1: Temperature calibration techniques on CIFAR-100, calibration parameters optimized to minimize ECE on validation set. In the "pre" mode, each member is scaled with one separate temperature. "dyn." denotes dynamic temperature scaling in post-combination mode using region- based temperatures. Ranges indicate 220
Model
Cal.
Acc/()
NLL
ACCETJO
ACE(1O
ECCECIO
ECECIO
SKCE
49.20
19741
000009
30.2200.4
23.00:0.555
16223:0.39
11.55:0.399
2337=00.06
49.17
19641
1:0.1337
23.15:10.86
13.23:10.19
3.24::0.37
2..24.50.42
post
21.72#061
57341.24
13:22-0.23
dyn.
49.20
19280 0.0107
21.19:100.3
1.66
12.86-10.28
2:334:100
28.81
0.030
81.32
0 100015
16.1110.12
5,8110388
89210.39
131.0019
53.71 0.14
0.683
167320038
6.7220068
8.56:20.25
1.8:00332
$7.6422
uyn.
81.32
0.6781 : 0,0058
16.11:10..00
1.94
8.1:110355
1.3110..3
$7.17:20.66
82.09
0.6314 0,0022
15.743:024
3.64:0.35
8.58:10.18
1.54:10.34
0.150
DSN
poot
pro
82.69
82.70
0.6324--0000444
15.79:20338
15.8120444
376622065
3.58
8.50:10.18
8.57-10.17
1.56::0.33
162200200
59.61
82.09
0.6315 0.041
15.63--0.43
3.242-034
8.65-40.29
1.11+0.18
$7,87 $7,87=00.55
RSN
pnd
83.45
83:41
0.6129:0000018
0.6118+-00000
15.48-1-028
15.41:10.43
2.52:10.33
3.28++0.52
8.75-+0.115
8.75:1.117
1.2-+0020
188:10.26
00.75 00.75 --555
60.67:1001
83:45
0.6097 1: 0,0023
15.63-100311
2.33:.0.56
8.68:10.31
1,20-10.41
0.
029
0794
0998
33M
Tap Narder
Dagion Mobb
Tapor Nanbel
DSNI00
DSNI21
RSN
Impact of different region numbers on dynamic temperature annealing in calibration of ensembles of DSN100, DSN121 and RSN.
Figure 3:
Topology ensembles for CIFAR- 100, optimal weights using ML estimation. Calibrations of each topology and ensemble using post-combination mode ("post" in Table
Table 2:
Weight Est.
Comb. Weight
Acc.
Ens Cal.
NLL.
ACE
ECE
DSN100
DSN121
RSN
(%)
(10 (10-1)
(10 3)
0.5766
4.97
2.24
0.02
0.19
0.30
0.49
83.75
0.5698
1.42
1.20
0.5741
3.74
2.00
Max LL
0.22
0.30
0.48
83.80
0.5714
1.52
1.29
0.5816
3.64
2.06
0.44
0.56
83.86
0.5801
2.36
1.35
5 CONCLUSIONS
State-of-the oftt---atttthttttt deep learning models often exhibit poor calibration performance. In this paper two aspects of calibration for these models are investigated: the theoretical definition of calibration and associated attributes for both general and top-label calibration; and the application of calibration to ensemble methods that are often used in deep-learning approaches for improved performance and uncertainty estimation. ensure that the ensemble prediction is itself calibrated. The resulting ensemble predictions will be under-confident. requiring calibration functions to be optimised for the ensemble prediction, rather than ensemble members. These theoretical results backed-up by empirical analysis CIFAR- 100 heep-learning models, with ensemble performance being robust to poorly calibrated ensemble members but requiring calibration even with well calibrated members.
shown that calibrating members the ensemble not sufficient to

---

Under review as a conference paper at ICLR 2021
REFERENCES
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in-domain uncertainty estimation and ensembling deep learning. arXiv preprint urXiv:2002.06-4700 2020.
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp. Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. End to end learning for self-driving cars. arXiv preprint 164.0:160, 2016.
Jochen Bröcker. Estimating reliability and resolution of probability forecasts through decomposition of the empirical score. Climate dynamics, 39(3-4):655667, 2012.
Tilmann Gneiting, Fadoua Balabdaoui, and Adrian Raftery. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2): 243-268, 2007.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Weinberger. On calibration of modern neural networks. ICML, 2017.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. ICLR, 2019.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. ICLR, 2019.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
Xiaoqian Jiang, Melanie Osl, Jihoon Kim, and Lucila Ohno-Machado. Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Asso- ciation, 19(2):263-215 274, 2012.
Volodymyr Kuleshov and Percy S Liang. Calibrated structured prediction. In Advances in Neural Information Processing Systems, pp. 1482- 2015.
Meelis Kull, Telmo Silva Filho, and Peter Flach. Beta calibration: well-founded and easily imple- mented improvement on logistic calibration for binary classifiers. Artificial Intelligence and Statistics, pp. 623-631, 2017.
Meelis Kull, Miquel Perello Nieto, Markus Kängsepp. Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. In Advances in Neural Information Processing Systems, pp. 12316-12326, 2019.
Ananya Kumar, Percy S Liang, and Tengyu Ma. Verified uncertainty calibration. In Advances in Neural Information Processing Systems, pp. 3792-3803, 2019.
Aviral Kumar, Sunita Sarawagi, and Ujjwal Jain. Trainable calibration measures for neural networks from kernel mean embeddings. In International Conference Machine Learning, pp. 2805- 2814, 2018
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predic- tive uncertainty estimation using deep ensembles. In Advances in neural information processing systems, pp. 1002-641,, 2017.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 (6(11))2278-2333 1998.
Gongbo Liang, Yu Zhang, and Nathan Jacobs. Neural network calibration for medical imaging classification using dea regularization. In ICML. UDI, 2020.
10

---

Under review as a conference paper at ICLR 2021
Mahdi Pakdaman Naeini, Gregory F Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the.. AAAI Conference on Artificial Intel- ligence. AAAI Conference Artificial Intelligence, volume 2015, pp. 2901. NIH Public Access, 2015.
Khanh Nguyen and Brendan O'Connor. Posterior calibration and exploratory analysis for natural language processing models. EMNLP, 2015.
Alexandru Niculescu-Mizz and Rich Caruana. Predicting good probabilities with supervised learn- ing. In Proceedings of the 22nd international conference on Machine learning, pp. 625-632, 2005.
Jeremy Nixon, Michael W Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measur- ing calibration deep learning. In CVPR Workshops, pp. 38-41, 2019.
John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3):61-74, 1999.
Adrian E Raftery, Tilmann Gneiting, Fadoua Balabdaoui, and Michael Polakowski. Using bayesian model averaging to calibrate forecast ensembles. Monthly weather review, 133(5):1155-111) 2005.
Rahul Rahaman and Alexandre H Thiery. Uncertainty quantification and deep ensembles. arXiv preprint #xXiv:2007.08792, 2020.
Asa Cooper Stickland and lain Murray. Diverse ensembles improve calibration. ICML 2020 work- shop on Uncertainty Robustness in Deep Learning, 2020.
David Stutz, Matthias Hein, and Bernt Schiele. manfidence-caliraite adversarial training: Gener- alizing to unseen attacks. ICML. 2020 workshop on Uncertainty Robustness in Deep Learning, 2020.
Gia-Lac Tran, Edwin V Bonilla, John Cunningham, Pietro Michiardi, and Maurizio Filippone. Cali- brating deep convolutional gaussian processes. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1554-1563. PMIR, 2019.
Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, and Thomas B Schön. Evaluating model calibration in classification. Proceedings of Machine Learn- ing Research, 2019.
Yeming Wen, Ghassen Jerfel, Rafael Muller, Michael W Dusenberry, Jasper Snoek, Balaji Laksh- minarayanan, and Dustin Tran. Improving calibration of batchensemble with data augmentation. ICML 2020 workshop Uncertainty Robustness in Deep Learning, 2020.
David Widmann, Fredrik Lindsten, and Dave Zachariah. Calibration tests in multi-class classi- fication: A unifying framework. In Advances in Neural Information Processing Systems, pp. 12257-12267, 2019.
Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Icml, volume 1, pp. 609-616. Citeseer, 2001.
Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass proba- bility estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowl- edge discovery and data mining, pp. 694 699, 2002.
Sergey Zagoruyko and Nikos Komodakis. arXiv:1605.07146, 2016.
Wide residual networks.
arXiv preprint
Zhang Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: risk minimization. ICLR, 2018
Beyond empirical
Wenliang Zhong and James T Kwok. Accurate probability calibration for multiple classifiers. In Sheety-Third International Joint Conference on Artificial Intelligence. Citeseer, 2013.

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 THEORETICAL PROOF
Proposition 3. If all members are calibrated and the regions are the same, i.e., for different members g m) and ((m²)
() .)) = R)( ee Vp,uy, E 6 0
then the ensemble also calibrated on the same regions
bEEZZIO P(ay x ()p(x)dx
J )
P(wylz)p(z)ddz, Vp,ug, e
Proof. If
R)(0mm)
R0(mmm) e) Vp,uy, E 6
The ensemble is also calibrated and the regions are the same:
R)(0,() c)
x
1
P(Wy an; ((m)) - p S (((())
Vp,ug; 6 0
(30)
M
m==1
1 M
M
Σ
P(W x ((m) )p(x)
M Σ
P(Wy x (m)) )p(x)da
(0.e) M
m=1 (0
m==
P(w)l=)p(a)ddr
M
(ER3(
m=1
/
J
P(wy) (p)))
.))
/ P(w,l=)p(zx)dz
Proposition 4. When class number K > 2, if all members are globally top-label calibrated, then the ensemble is not necessarily global top-label calibrated.
Proof. Assume globally top-label calibrated members imply globally top-label calibrated ensemble, that is, given
P P(o
( x (0)))((m))
Σ
8
m I,.., M
(31)
N
the following true
1
Σ
P (0:0)
N Σ
1
0
(32)
N
If En, m, > O, such that (n))
(x) Um then it is possible to write
*
P(U (m)
x (n): θ
(n) (m); o)m)) (lm))) P(D
+ M
P(D (m) (m) (())
(33)
M
For top-label calibration there are no constraints on the second term Eq. (33 as the top- label for model 0 (m)) Thus there for member mn that only need to satisfy the following constraints
set of models that satisfy the top-label calibration constraints
< P(r (n) ao: (n); 0())
P(j(m))
(n) |a (n) ((m) <
(34)
12

---

Under review as a conference paper at ICLR 2021
and the standard sum-to-one constraint over all classes. Consider replacing member in of the semble with member having parameters O(m), to yield θ that satisfies
max ((wlax ( ) max (P(w ((w))(lm ) = D1%0!
(35)
P P ( (x )(m))
P(U (m) (
(
g(())
(36)
P P( (x) ((m)
(1)
P(te (m) ((3) +
(37)
where > O, and the standard sum-to-one constraint is satisfied, and all other predictions are unaltered. This results in the following constraints
max ((w)aa!)) 6)} max {P(wlae 0) (n)
(38)
P(UE (n) x (m); Q < P(iie
(m) (x( θ)
(39)
The accuracy of the two ensembles O and é are the same from Eq. (38). but the probabilities associated with those predictions cannot the same from Eq. (39) so both ensemble predictions cannot be calibrated, assuming that the ensemble prediction for O is calibrated
P(DE 0)
U
④
8
(0) yE
(40)
N
Hence there are multiple values of P(DE (=(); Q) for which all the models satisfy the top- calibration constraints, but these cannot all be consistent with Eq. (40). For the situation where
(
there is no sample or model where (n) * Um then the predictions for all models for all samples
the same the ensemble prediction, by definition there can be no performance gain.
A.2 GLOBAL GENERAL CALIBRATION AND TOP-LABEL CALIBRATION
To demonstrate the differences between global top-lahel calibration and global calibration, set of ensemble member predictions were generated using Algorithm 1, this ensures that the predictions are perfectly calibrated. Since the member predictions are perfectly calibrated, the ensemble members will be globally calibrated. Figure 4 (a) shows the performance in terms of ACE of the ensemble prediction the value of increases, note when 1 this global calibration version of ACE, It can be seen that as increases ACE decreases, and for the global case reduces to zero for the ensemble predictions as the theory states.
In terms of top-label calibration, as the ensemble members are perfectly calibrated, they will again be global top-label calibrated. This illustrated Figure where ECE zero for all ensemble members. For top-label calibration the value of ECE does not decrease to zero the - 1, again as the theory states. This is because the underlying probability regions associated with each of the members of the ensemble are different. Hence, even for perfectly calibrated ensemble members, the ensemble prediction is not global top-label calibrated.
A.3 TOY DATASETS
Example 1. In this example, show the difference between all-label calibration and top-label calibration which consider the different regions in Eq. (3) and Eq. (4).
Assuming p(œ) 1, the whole input space X' is consisted of three regions R1, R2 and Ra, and
[
JASR. p(x)da =
p(x)dx=
p(x)dx.
(41)
J Ra
13

---

Under review as a conference paper at ICLR 2021
8.7
o15 mam
0.12
0.06
0.10
0.05
0.04
0.08
601
000
0.00
04
08
08
1.0
0.0
0.2
0.4
0.6
1.0
(a) Calibration
(b) Top-label Calibration
Figure 4: ACE (calibration) and ECE (top-label calibration) of perfectly calibrated set of ensemble members, as the value of e varies.
The corresponding system prediction P and the true distribution P is:
W1
W2
W's
W1
0.5
0.4
0.05
0.05
R1
p = (P(wylor Ry:
0.3
0.4
0.2
0.1
R2
(42)
0.3
0.3
0.35
0.05/
Rs
W1
W2
W3
W4
0.5
0.4-T
0.05
0.05
R1
0.3 T
0
0.2
0.1
R₂
7>0
(43)
0.3 + r 0.3
0.35
0.05
R3
can be verified that
J P(Wyla, |z. ()p(a))
= / -Iere P(w)lx))(2z)dz Vuj.D,E-0
(44)
However, when p = =.
[ P(wylæ, O)p(arddx, 6 0
Jacke/000) P(willæ, ()
#
J
(45)
This example shows that combination of calibrated members yields uncalibrated en- semble. Algorithm generates the true data distribution p by sampling from a Dirichlet distribution with equal concentration parameters of 1. To generate the member predictions, the N samples randomly assigned to N/b i bins with size of b. In each bin, the member predictions P are equally the average true data distribution of the associated samples. This ensures that for each region, Eq. (2) holds for each member. However, regions for different member are different due to the random assignment. Therefore, the corresponding ensemble is not automatically calibrated. Figure shows that the ensemble is uncalibrated with ACE of 0.0697.
Example 2.
Example 3. In this example, we show that for a finite number of samples, the regions (f and S (,, derived from the samples is different from the theoretical regions, leading to difference between theoretical calibration error measures and the values estimated from the finite samples. Algorithm generates data with difference between finite sample-based calibration error and theo- retical error. The theoretical ACE of the predicted probabilities in Algorithm 2 is 32 However, the finite sample-based ACE
The true data distribution is ((w)(æ) size The type of bin that a sample is assigned to determines the predicted probability of sample. Considering each class wy, there are three types of bins:
p(x) 1. The samples in D are assigned to bins of
B both samples belong to class w,. B.5: only one sample is class wy.
14

---

Under review as a conference paper at ICLR 2021
Algorithm 1: Algorithm for generating calibrated members that yield uncalibrated ensemble
Result: // "
Ji=1
b = 2: //bin size, number samples in one bin;
K 4; //number of classes;
M = 10; //number of members;
N 1000000; //number of samples;
f Dirichlet( = 1), = 1, ..... N: //tuee data distribution, sampled from Dirichlet
distribution with equal concentration parameters of I;
= [1, 2.... NJ; // index vector;
for in ...,, M] do
I <- shufflc(I);
for j in [1.... [N/b]] do
B, (bx (j 1), min mn
PB, = 5/2 ElEB, p();
end
for i in
[1 M do
ml) pn,
end
end
Bp=u: both samples are not class wy.
1
6
9
P(x E B1) 16'
P(x E Bo.s)
P(x E Bb)
(46)
16'
16
then
/ J (P(w)l>:: 0) ((z)) ((x)
1 P(a))d
(p
(47)
(p -0-77 P( E Bp)
(48)
therefore,
ACE(O) = -
1
Σ
/
(P(w 12; 0) - P(ww)(x)) ((zz)
(49)
4 0
DE (0,0)
J=1
Σ
(p P(x E Bp)
9
(50)
32
(E(00..5.1)
A.4 ADDITIONAL EXPERIMENTAL RESULTS
In this section, show some comparison experiments the empirical results in Section We conducted experiments on CIFAR- 100 and CIFAR- 10 dataset. dissply the performance of inidividual models, LeNet, DenseNet 100, DenseNet 121, and wide ResNet 28- 10. All systems are trained with data augmentation of random cropping and horizontal flipping, and simple mean/std normalization. The original training/test images in CIFAR datasets 50,000/10,000. We hold out 5,000 images from training optimization, this leads to slight accuracy degradation compared to training with all 50,000 images. We have 10 runs of our experiment obtain the deviations.
validation set (10%) for temperature and combination weights
In Section 4.3 we presented ensemble calibration on CIFAR-100. The counterpart on CIFAR-10 is given in Table Other than separately specified, all sample based evaluation criteria in this paper use 15 bins following previous literature (Guo et al.; 2017). The temperatures pre-, post- and dy- namic post-combination modes optimized on the validation by minimizing ECE (Liang al.,
15

---

Under review as a conference paper at ICLR 2021
Algorithm 2: Algorithm for generating data with difference between finite sample-based ACE
and theoretical ACE.
Result: ( (())) Ji=1
b //bin size, number of samples in one bin;
K 4: //number of classes;
= [91..... 9v]; //vector ground-truth labels, where g E {l...., K... Numbers of labels for
different classes equal;
[1,2. NY; // index vector;
<- shuffle(T):
for j in [1,..., [ do
Bj (b* (j 1), (b (b+ J.
pn, EIEB (611 91, 5(2, 91, )...) "..,,,,KK, ((Kg)))
end
for i in [1, ..... N1 do
j 1
p(T) PB
end
2020), using SGD with learning rate 0.1 for 400 iterations. It can be observed that combination of DenseNet and ResNet improves the calibration performance, while combination LeNet doesn't t help. This is because LeNet is over-confident DenseNet and ResNet (as shown in Figure D.. Hence the simple ensemble combination doesn't help, but aggravates the calibration. The three temperature-base calibration methods effectively improve the system calibration performance on CIFAR-10 as well.
Table gives
gives the ensemble combination based on AUC weights (Zhong & Kwok 2013). The AUC
weights are much even than the Max LL weights in Table 2 The structures combined are first calibrated, nevertheless, applying post-combination calibration the ensemble obtains gains. We evaluated post-combination for topology combination in Table 7 The post- and dynamic post- combination methods are applied to calibrate the topologies and the topology ensemble. The dynamic temperature method shows clear advantage in obtaining calibrated ensemble of mutiple topologies.
Table 3: Individual model performance on CIFAR-100, Full data training (100%Train) and training with 5,000 images held out (90%Train) are presented. denotes results from the original papers.
Model
% Train
Acc.(%)
NLL
ACCE
ACE
ECCE
ECE
(%)
(10-1)
(10-1)
(10-2)
(10-2)
100
42.47
2.2730
21.07
7.62
13.05
1.82
LEN
90
42.12
2.2652
22.80
10.11
13.20
2.86
100"
76.21
DSN 100
100
76.71
1.1022
30.54
25.85
14.53
12.65
90
76.00
1.0591
30.22
24.55
13.91
11.72
100
80.00
0.8438
23.39
17.29
11.52
8.59
DSN 121
90
78.87
0.8981
24.83
19.00
11.99
9.41
100'
80.75
RSN
100
80.99
0.7711
18.67
8.68
10.25
5.02
90
80.06
0.7985
18.72
8.83
10.23
4.76
16

---

Under review as a conference paper at ICLR 2021
Table 4: Individual model performance on CIFAR-10. Full data training (100%Train) and training with 5,000 images held out (90%Train) are presented. denotes results from the original papers.
Model
% Train
Acc.(%)
NLL
ACCE
ACE
ECCE
ECE
(%)
(10 +)
(10-+)
(10-3)
(10 3)
100
75.07
0.7329
60.85
23.97
3,55
1.24
LEN
90
75.31
0,7280
77.17
44.49
4.16
2.09
100*
94.23
DSN 100
100
95.39
0.2003
61.04
54.74
3.04
2.77
90
94.92
0.2181
65.50
60.54
3.27
2.99
100
95.66
0.1832
61.56
56.63
3.01
2.87
DSN 121
90
95.53
0.1993
62.79
57.16
3.06
2.89
100*
96.00
RSN 28
100
95.98
0.1524
51.67
46.70
2.61
2.35
90
95.52
0.1692
58.07
52.70
2.89
2.62
Table 5: Calibration using temperature annealing on CIFAR-10. The temperatures are optimized to minimize ECE on validation set. In the 'pre' mode, each member is scaled with one separate temperature. 'dyn.' denotes dynamic temperature scaling in post-combination mode using region- based temperatures. The three structures investigated are LeNet 5, DenseNet 121, DenseNet 100 and Wide ResNet 28. Ranges indicate +2a.
Model
Cul.
Acc/%)
NLL
ACCECI
ACEZIO 4
ECCEL1 ?)
ECECO 2
SKCE/10 4
50.11
000003
137.7313222
BLIKE
0.55
9.1820.17
447,5011
LEN
poot
pre
80.11
80.04
0L5063
0.501520.003
$7.84::3000
766822.11
34.00-24.62
$1 $1.50:5.100
4.0120.24
36660.21
L1520.31
1.9120.34
568
54
3.74:0336
1.2910.52
56894117.07
96.19
0,0013
2801
1.95
1.1
0.35
17.27
DSN 100
96.19
000015
01227=00:01
28.274-2.44
21522422.56
13.27: 180
13:00:22117
147
14460010
0.35
0.54-20.12
#702444-331
87062-5419
dyn.
96.35
0000000000000
2718
15:31E1144
L4320012
0.8910.14
8950000
DSN 121
96.35
011002200012
2785:1 1.33
15.08:2:1 46
1,47200122
0.6720.13
554-5---00
post
dyn.
96.35
0.11912000002
27.66:1:1.6
14:00:22242
1,42:0.15
0.66:00.14
894
96.31
0.1081 0,000
25:14:1344
11.33:11.339
1.7
0.38:00.10
89L.64 1.07
RSN
pre
0.0075-000009
24.54:2044
1.3200.00
0.50-20.11
post
dyn.
96.51
0.1070 0,0007
23884:1154
8.57-11..4
1.30
0,47
883.58-1-5.00
Table 6: Topology ensembles for CIFAR-100, optimal weights based on AUC. Calibrations of each topology and ensemble using post-combination mode ("post" Table
Weight Est.
Comb. Weight
Acc.
Ens Cal.
NLL
ACE
ECE
LEN
DSN100
DSN121
RSN
(%)
(10-4)
(10-3)
0.6649
20.84
10.07
0.20
0.26
0.27
0.27
83.30
0.6096
2.60
2.00
0,5776
3.67
1.89
AUC
0.33
0.33
0.34
83.55
V
0.5787
3.03
1.38
0.5820
3.54
2.03
0.49
0.51
83.78
0.5805
2.30
1.34
17

---

Under review as a conference paper at ICLR 2021
Table 7: Topology combination on CIFAR-100, Dynamic mode uses region-based temperatures.
Comb.
Weight
Struct cal.
Ens Cal.
Acc.(%)
NLL
ACE
ECE
LEN
DSN100
DSNI21
RSN
0.33
0.33
0.33
83.54
0.5846
0.0007
0.0334
0.33
0.33
0.33
post
83.54
0.5769
0.0002
0.0132
0.33
0.33
0.33
dyn.
83.54
0.5751
0.0002
0.0099
0.33
0.33
0.33
post
83.57
0.5779
0.0004
0.0189
0.33
0.33
0.33
post
post
83.57
0.5778
0.0002
0.0118
0.33
0.33
0.33
dyn.
83.59
0,5771
0.0004
0.0211
0.33
0.33
0.33
dyn.
dyn.
83.59
0.5730
0.0002
0.0112
0.22
0.39
0.39
83.78
0.5822
0.0007
0.0327
0.22
0.39
0.39
post
83.78
0.5724
0.0002
0.0135
0.22
0.39
0.39
dyn.
83.78
0.5705
0.0002
0.0113
0.22
0.30
0.48
post
83.80
0.5741
0.0004
0.0200
0.22
0.30
0.48
post
post
83.80
0.5714
0.0002
0.0129
0.23
0.31
0.46
dyn.
83.75
0.5741
0.0004
0.0211
0.23
0.31
0.46
dyn.
dyn.
83.75
0.5700
0.0002
0.0153
0.25
0.25
0.25
0.25
83.34
0.7238
0.0032
0.1551
0.25
0.25
0.25
0.25
post
83.34
0.6043
0.0003
0.0196
0.25
0.25
0.25
0.25
dyn.
83.34
0.6027
0.0002
0.0122
0.25
0.25
0.25
0.25
post
83.25
0.6947
0.0025
0.1224
0,25
0.25
0.25
0.25
post
post
83.25
0.6262
0.0004
0.0218
0.25
0.25
0.25
0.25
dyn.
83.22
0.6978
0.0026
O. 0.1258
0.25
0.25
0.25
0.25
dyn.
dyn.
83.22
0.6184
0.0002
0.0137
0.01
0.19
0.42
0.38
83.74
0.5859
0.0008
0.0374
0.01
0.19
0.42
0.38
post
83.74
0.5710
0.0001
0.0114
0.01
0.19
0.42
0.38
dyn.
83.74
0.5703
0.0001
0.0085
0.02
0.19
0.30
0.49
post
83.75
0.5766
0.0005
0.0224
0.02
0.19
0.30
0.49
post
post
83.75
0.5698
0.0001
0.0120
0.02
0.21
0.30
0.47
dyn.
83.77
0.5784
0.0006
0.0282
0.02
0.21
0.30
0.47
dyn.
dyn.
83.77
0.5698
0.0002
0.0167
18


Reviews:
[{"review_id": "hH_B_GBwI1y", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- **Summary**:\nThis paper investigates the impact of different calibration strategy (pre-combination, post-combination and its dynamic variant) on the performance of a deep ensemble. It presents both theoretical and empirical proof to show that well-calibrated ensemble member does guarantee calibration in the final ensemble.\n\n- **Strength**:\n  * A coherent theoretical account for the issue of calibrating Deep ensembles. Accompanied by empirical evidence from CIFAR datasets.\n  * Although not stated explicitly, a new calibration approach (dynamic calibration) is introduced, which empirically leads to better performance.\n\n- **Weakness**\n  * Novelty may be limited: one central contribution of this paper is to provide a mathematical derivation to confirm the observation made in Rahaman and Thiery (2020) and Wen et al. (2020).  Although I appreciate author's work on providing mathematical explanation for recent empirical findings, I'm not sure if the submission in its current form  is contributing significant novel theoretical insight beyond the fact that ensemble prediction is less confidence, since max of the mean probability is no greater than mean of the max probabilities. On the other hand, the empirical investigation is conducted on a single vision task (CIFAR-10/-100). This paper can be made stronger by investigating synthetic situation where the ground truth is known, or extend experiment to also other data modalities (like  Guo et al. (2017)). \n\n  * Organization: Given the place of the new approach (dynamic temperature scaling) in the experiment, it might be worthwhile to devote some paragraph to introduce the procedure in more detail.\n\n- **Recommendation**: Based on reason stated in weakness, I recommend rejection since the either theoretical or the empirical contribution of this paper does not seem to be substantive enough for ICLR.", "extra_scores": {}, "extra_text": {}}, {"review_id": "baeJqVO5zc", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper makes an analysis of calibration in ensembles of deep learning models. Through some theoretical developments, the paper supports that a given ensemble cannot be more confident than the average individual members for regions where the ensemble is well calibrated. Empirical results, on CIFAR-100 and three different deep models, report a comparison of ensemble calibration, where calibration is done over all members in order to achieved a calibrated ensemble decision, over individual calibration of members with no feedback from the ensemble decisions. Results show that individual member calibration does not lead to calibrated ensembles, and as such calibrating directly on the ensemble output is required for obtained a proper evaluation of its uncertainty. Different ensemble calibration approaches are also compared.\n\nPros:\n- Overall well-written paper.\n- Straightforward proposal, simple yet meaningful on several aspects for better understanding of the link between calibration and ensembles.\n- Rely on theory to support some claims, which strengthen the proposal.\n\nCons:\n- The proposal is somewhat trivial, although I do not have knowledge that it has been investigated in detail elsewhere. Before reading the paper, I expected the results (i.e. calibration of individual members will not lead to calibrated ensemble decisions; calibration at the ensemble level is required), the paper is somewhat confirming this in a more explicit manner.\n- Evaluation on only one dataset (CIFAR-100) in the main paper, with another dataset for the appendix (CIFAR-10).\n- Results on CIFAR-10 in the appendix are not very compelling.\n- It is hard to make sense of the results in Table 1 and similar. Differences are small and difficult to interpret.\n- The explanations and organization of the paper are hard to following in some specific part.\n\nAlthough the paper is making a well-founded analysis of a hot topic in the last few years (i.e.,  ensembles are a way to evaluate uncertainty on decisions), I found it having some relatively trivial developments. And the conclusion is intuitive and expected. However, it is the first time I see this point well articulated, and the authors have made a good effort to develop theoretically backed explanations to support this.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "MjSf19FfIDl", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Update after the author response: I've read the other reviews, and agree with R2 and R3. I think the paper is useful (emphasizes you need to calibrate the final ensemble, not enough to calibrate members), and has some nice conceptual contributions (explaining that if ensemble accuracy > average member accuracy (which is usually the case), and the ensemble is calibrated even in just a global/weak sense, then the members must be uncalibrated). This could spur more research into conceptually analyzing ensembles, and seems interesting. But I understand the other reviewer's concerns that it's not clear what practical impact this will have, so I'm keeping my score at a 6 (instead of raising to a 7).\n\n#########################################################################\n\nSummary:\n\nThis paper tackles the problem of calibrating an ensemble. They show experimentally that calibrating all members of an ensemble is often not enough to calibrate the combined ensemble, so instead we need to calibrate the final predictions of the ensemble. Additionally, they show that using a different temperature parameter for different regions of outputs can improve calibration. They explain why if the ensemble members are top-label calibrated (even in a very weak sense they call “global” calibration”), and the ensemble is calibrated, then the ensemble is less accurate than the average member of the ensemble.\n\n#########################################################################\n\nReasons for score:\n\nThey make interesting observations about calibration of ensembles that could guide practitioners. For example, that it’s not enough to calibrate the members of the ensemble. They also raise an intriguing connection between calibration of ensemble members and ensemble accuracy, one would not expect a priori that if both are calibrated the ensemble would do worse than the average member. I could see this result being interesting to people who study ensembles as well. There are some weaknesses in writing and execution, but overall this paper is probably worth publishing if edited.\n\n#########################################################################\n\nPros:\n\n- I think it’s a nice observation that calibrating the members of an ensemble may not yield a calibrated ensemble. It’s easy to come up with toy examples where this is the case, but it’s interesting that this seems to be the case in practice.\n\n- They make an intriguing observation that if the ensemble members are in fact calibrated and the ensemble is calibrated, then the ensemble accuracy is at most the average member accuracy\n\n\n#########################################################################\n\nCons:\n\n- I believe the writing can be substantially simplified. The core ideas are simple and nice, but it takes a lot of effort to get to them, and I believe the authors should put in more work into making this understandable.\n\n- Some of the results seem unrealistic and can be omitted. For example in the start of section 4.1, the first couple of results require that the ensemble member regions and ensemble regions are the same. This seems rather unrealistic. The assumptions in prop 1 seem too strong to me. I’d remove the mentions of regions and I’d instead mention the other results (prop 2, 3, 4) in the main paper, Section 4.1. You could just move the propositions, and give some intuition for why the results are true. Removing regions should also considerably simplify the notation and setup.\n\n- I’m not quite sure what you mean in the intro when you say “Eq. (1) doesn’t explicitly reflect the relation between … and the underlying data distribution p(x, y)”. The definition in Equation (1) uses p(x, y). I’m not sure why all the definitions in 3.1 and 3.3 are defined in a way different from the standard ways in the calibration literature e.g. in Kull et al 2019 or Kumar et al 2019.\n\n- Temperature scaling is performed on logits, not on the actual probabilities. From equations 24, 25, and 26 it looks like you might be doing temperature scaling on the probability space (in equation 24, 25 the first argument to f is the probability, not the logit), which looks a bit odd.\n\n- Prop 4 should also hold when K = 2 (2 ensemble members) I believe. Happy to provide an example.\n\n- Some symbols are undefined. For example, \\delta(y^{(i)}, \\omega_j), I don’t believe \\delta is defined. I think it should be 1 if they are equal and 0 otherwise?\n\n#########################################################################\n\nQuestions and things to improve:\n\n- Please answer the cons above.\n\n- Ensembles are particularly useful because they tend to be more calibrated out of domain (Lakshminarayanan et al 2017). It could be useful to see which of these methods (calibrating the members, or the entire ensemble) is better calibrated when we have domain shift (e.g. training data = CIFAR-10, test data = CIFAR-10C, Hendrycks et al 2019).\n\n- Having confidence intervals for the calibration errors would be nice (and also using more modern, debiased estimators to estimate the calibration error) e.g. in Kumar et al 2019. \n\n#########################################################################\n\nAll cites mentioned are already in the paper, except: \nBenchmarking Neural Network Robustness to Common Corruptions and Perturbations. Dan Hendrycks, Thomas Dietterich. ICLR 2019.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2G17lz8QMr4", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- In general, my opinion is aligned with AnonReviewer1 the theory and the empirical contribution do not feel sufficient.  \n- I also agree with AnonReviewer3  and AnonReviewer4 but feel less excited about the prons and more worried about the cons. \n\nAt this point, I'm not against the acceptance of the paper, although I'm still staying on the rejection side. I'm increasing my score because we are at least talking about a borderline.\n\n--------------------------\n\nSummary:\n\nThe paper study calibration of ensembles of DNNs and its relation to the calibration of individual members of ensembles. The work demonstrates that i) members of an ensemble should not be calibrated, but the final ensemble may require calibration (especially if members of an ensemble are calibrated) ii) provide theoretical results to support the statement iii) propose an adaptive calibration scheme (dynamic temperature scaling) that uses different temperatures based on the confidence of a model. \n\n\nConcerns: \n\n1) The main question of the paper \"Should ensemble members be calibrated?\" feels trivial, because the community is aware of the simple example that provides an answer. The Deep Ensembles [Lakshminarayanan2017] have miscalibrated members---conventional DNNs, but the predictions of an ensemble are, in-most-cases, calibrated. Thus the answer is \"No\".\n\n2) The paper mostly is clearly written, but section 4.1 \"Theoretical Analysis\" is *extremely* hard to follow. Even though I re-read it many times, I'm still not sure if I understood it correctly. The most confusing part is the conclusion \"In practice, there is no constraint that the ensemble prediction should be calibrated, thus ensemble prediction calibration is required even for top-label calibrated members.\". It seems that no listed results were used to produce this statement.    \n\n3) The calibration of ensemble has been proposed in [Ashukha2020, 5 Discussion & Conclusion]. (\"The resulting ensemble predictions ..., requiring calibration functions to be optimized for the ensemble prediction, rather than ensemble members.\")\n\n4) The two main contributions (4.1 Theoretical Analysis, 4.2 Temperature Annealing for Ensemble Calibration) feels not related, they are basically two independent topics packed in the one paper.\n\n5) The empirical comparison exploits the calibrations score (e.g., ECE). ECE  is a biased estimate of true calibration with a different bias for each model, so it is not a valid metric to compare different models (see Vaicenavicius2019). The fact is even mentioned in the current paper (\"It should be noted that for finite number of samples ... \") but still is ignored in the empirical study.\n\nWhat I suggest is to use the squared kernel calibration error (SKCE) proposed in [Widmann2019] along with de facto standard, but biased ECE. The SKCE is an unbiased estimate of calibration. There might be some pitfalls of this metric that I'm not aware of, but the paper looks solid and convincing. Also, please put attention to Figure 83 in the arХiv version. Yes, ECE is the standard in the field, but it is the wrong standard that prevents us from meaningful scientific progress, so we should stop using it.\n\n6) The results provided in Table 1 seem to be close values (0.6119 vs 0.6129, etc.), so at least standard deviations need to be reported. Also, there is no mentioning of several runs per results in the text.\n\nThe paper toches the nice topics but, overall it feels like \"ok, but not enough\". The theory is interesting but it does not give us a lot of insides (maybe it's very subjective). The dynamic temperature scaling is not proofed to outperform the basslines. The contributions feel disconnected. The writing quality needs to be improved.  \n\nComments: \n\n1) As far as I can tell, the citation \"The weights assigned to the probabilities are either optimized using AUC as in (Ashukha et al., 2020) ...\" is incorrect, as there is no mentioning of optimizing weights using AUC in the paper.\n\n2) typo: It should be noted that for A finite number of sampleS\n\n[Lakshminarayanan2017] Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in neural information processing systems 2017 (pp. 6402-6413).\n\n[Ashukha2020] Ashukha A, Lyzhov A, Molchanov D, Vetrov D. Pitfalls of in-domain uncertainty estimation and ensembling in deep learning. ICLR, 2020. \n\n[Vaicenavicius2019] Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, and Thomas B Schon. Evaluating model calibration in classification. AISTATS, 2019.\n\n[Widmann2019] Widmann D, Lindsten F, Zachariah D. Calibration tests in multi-class classification: A unifying framework. In Advances in Neural Information Processing Systems 2019 (pp. 12257-12267). https://arxiv.org/pdf/1910.11385.pdf", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies ensemble calibration and the relationship between the calibration of individual ensemble member models with the calibration of the resulting ensemble prediction.  The main theoretical result is that individual ensemble members should not be individually calibrated in order to have a well-calibrated ensemble prediction.  While other recent work has found this to be the case in empirical results, this paper substantiates the empirical results through theoretical results. 

Pros:
* Theoretical study of ensemble calibration with meaningful insights

Cons:
* Contributions limited to theoretical study of known observation and dynamic temperature scaling.
* Dynamic temperature scaling is not shown to outperform baseline methods.
* Limited experimental validation: CIFAR-10/CIFAR-100.

The authors engaged in a extensive discussion with reviewers and made changes to their paper, including adding standard deviation results over multiple runs and the SKCE calibration measure.

Overall this is solid work and could be accepted to the conference; however, reviewers agree that parts of the work are lacking, in particular: 1. limited experimental evaluation (one type of task, one/two datasets only), and 2. given known literature the benefit of the derived theoretical results to practioners is not clear.  The discussions have been unable to resolve this disagreement.

Decision: Reject