Uncertainty Quantification and Deep Ensembles
Authors: Rahul Rahaman
Alexandre H. Thiery
Venue: NeurIPS 2021 Poster (2021)
Keywords: Uncertainty Quantification
Model averaging
Deep Ensemble
Low-data setting

Abstract: Deep Learning methods are known to suffer from calibration issues: they typically produce over-confident estimates. These problems are exacerbated in the low data regime. Although the calibration of probabilistic models is well studied, calibrating extremely over-parametrized models in the low-data regime presents unique challenges. We show that deep-ensembles do not necessarily lead to improved calibration properties. In fact, we show that standard ensembling methods, when used in conjunction with modern techniques such as mixup regularization, can lead to less calibrated models. This text examines the interplay between three of the most simple and commonly used approaches to leverage deep learning when data is scarce: data-augmentation, ensembling, and post-processing calibration methods. We demonstrate that, although standard ensembling techniques certainly help to boost accuracy, the calibration of deep ensembles relies on subtle trade-offs. We also find that calibration methods such as temperature scaling need to be slightly tweaked when used with deep-ensembles and, crucially, need to be executed after the averaging process. Our simulations indicate that, in the low data regime, this simple strategy can halve the Expected Calibration Error (ECE) on a range of benchmark classification problems when compared to standard deep-ensembles.

Uncertainty Quantification and Deep Ensembles
Rahul Rahaman
Department of Statistics and Data Science, National University of Singapore
rahul rahamanou nus edu
Alexandre H. Thiery
Department of Statistics and Data Science, National University of Singapore
a.h. thieryynus edu. sg
Abstract
Deep Learning methods are known to suffer from calibration issues; they typically produce over-confident estimates. These problems are exacerbated in the low data regime. Although the calibration of probabilistic models is well studied, calibrat- ing extremely over peer-parmmetrized models in the low-data regime presents unique challenges. We show that deep-ensembles do not necessarily Icad to improved calibration properties. In fact, we show that standard ensembling methods, when used in conjunction with modern techniques such mixup regularization, lead to less calibrated models. This text examines the interplay between three the most simple and commonly used approaches leverage deep learning when data is scarce: data-augmentation, ensembling, and post-processing calibration methods. Although standard ensembling techniques certainly help boost accuracy, we demonstrate that the calibration of deep ensembles relies on subtle trade-off's We also find that calibration methods such as temperature scaling need to be slightly tweaked when used with deep-ensembles and. crucially, need to be executed af- ter the averaging process. Our simulations indicate that this simple strategy can halve the Expected Calibration Error (ECE) on range of benchmark classification problems compared standard deep-ensembles in the low data regime.
Introduction
Overparametrized deep models can memorize datasets with labels entirely randomized [48]. is consequently not entirely clear why such extremely flexible models are able to generalize well on unseen data and trained with algorithms as simple as stochastic gradient descent, although a lot of progress these questions have recently been reported [8, 19, 2, 31, 39, 10].
The high capacity of neural network models, and their ability to easily overfit complex datasets, makes them especially vulnerable to calibration issues. In many situations, standard deep-learning approaches are known to produce probabilistic forecasts that are over-confident [16]. In this text, we consider the regime where the size of the training sets is very small, which typically amplifies these issues. This can lead to problematic behaviors when deep neural networks are deployed in scenarios where proper quantification of the uncertainty is necessary. Indeed, host of methods [22, 30, 40, 12, 37] have been proposed to mitigate these calibration issues, even though no gold standard has far emerged. Many different forms of regularization techniques [35, 48, 50] have been shown to reduce overfitting in deep neural networks. Importantly, practical implementations and
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

approximations of Bayesian methodologies [30, 44, 3, 14, 27, 38, 28] have demonstrated their worth in several settings. However, some of these techniques are not entirely straightforward to implement in practice. Ensembling approaches such as drop-outs [12] have been widely adopted, largely due to their ease of implementation. Recently, [1] provides a study on different ensembling techniques and describes pitfalls of certain metric for in-domain uncertainty quantification. Also subsequent to our work, several articles also studied the interaction between aata-augmentation and calibration issues. Importantly, the CAMixup approach is proposed as a promising solution in [42]. Furthermore, [47] analyzes the under-confidence of ensembles due to augmentations from a theoretical perspective. In this text, we investigate the practical use of beep-Ensembles [22, 4, 25, 41, 9, 16], straightforward approach that leads to state-of-thh- art performances in most regimes. Although deep- -eepmbeeseblee can be difficult to implement when training datasets are large (but calibration issues are less pronounced in this regime), the focus of this text is the data-scarce setting where the computational burden associated with deep-ensembles is not a significant problem.
Contributions: We study the interaction between three of the most simple and widely used methods for adopting deep-learning the low-data regime: ensembling, temperature scaling, and mixup data augmentation.
Despite the widely-held belief that model averaging improves calibration properties, show that, in general, standard ensembling practices do not lead to better-calibrated models. Instead, we show that averaging the predictions of a set of neural networks generally leads to less confident predictions: that is generally only beneficial in the oft-encountered regime when each network is overconfident. Although results are based Deep Ensembles, our empirical analysis extends to any class of model averaging, including hampling-based Bayesian Deep Learning methods.
We empirically demonstrate that networks trained with the mixup data auttaaugmettttion scheme, a widespread practice in computer vision, are typically mnder-confident. Consequently, subtle interactions between ensembling techniques and modern data augmentation pipelines have to be considered for proper uncertainty quantification. The typical distributional shift induced by the mixup data-aigmentttion strategy influences the calibration properties of the resulting trained neural networks. In these settings, a standard ensembling approach typically worsens the calibration issues.
Post-processing techniques such temperature scaling are sometimes regarded as compet- ing methods when comparing the performance of many modern model-avvraging techniques. Instead, to mitigate the under-confidence of model averaging, temperature scaling should be used in conjunction with deep-ensembling methods. More importantly, the order in which the aggregation and the calibration procedures are carried out greatly influences the resulting uncertainty quantification. These findings lead us to formulate the straight- forward Pool-Then- mho--Then-Callbrate strategy for post-processing deep-ensembles: stage, separately train deep models (2) in a second stage, fit a single temperature parameter by minimizing proper scoring rule (eg. cross-entropy) data regime, this simple procedure can halve the Expected Calibration Error (ECE) on a range of benchmark classification problems when compared to standard deep-ensembles. Although straightforward to implement, to the best of our knowledge this strategy has not been investigated in the literature prior to our work.
(1) in first
a validation set. In the low
2 Background
Consider a classification task with C > 2 possible classes y = (I, For sample æ E X, the quantity p(x) E A = {p E RC pi +... PC: = 1} represents a probabilistic prediction, often obtained as pxx) = JSM [fw (x)) for a neural network fw X + RC with weight w E RR and softmax function daM R + Ac. We set ((z) = arg max (xx and p(x) = max p(x).
Consider a training dataset D = (#,, J.N and denote by y E Ac the one-hot encoded version of the label y E y. A stochastic augmentation process Aug X x Ac + X' x Ac maps a pair (x. yy E X x Ac to another augmented pair (.c.HH).. In computer vision, standard augmentation strategies include rotations, translations, brightness and contrast manipulations. In this text, in addition to these standard agumentations, we also make use of the more recently proposed mixpp augmentation strategy [49] that has proven beneficial in several settings. For pair
Augmentation:

---

(x, E X' x Ac. its
version 0.) is defined as
I, = =/=+ (1-7)a, =, and V. = 7 + (1
for random coefficient y E (0, 1) drawn from a fixed mixing distribution often chosen as Beta(a, a), and random index J drawn uniformly within NV.....N).
Model averaging: Ensembling methods leverage a set of models by combining them into an aggregated model. In the context of deep learning, Bayesian averaging consists of weighting the predictions according to the Bayesian posterior [dw Drain) on the neural weights. Instead of finding an optimal set of weights by minimizing loss function, predictions by Pw (a)) E Ac the probabilistic prediction associated to sample x E X' and neural weight w, the Bayesian approach advocates to consider
averaged. Denoting
(prediction) = / (x) πI (dw Dimni) E Ac-
(1)
Designing sensible prior distributions is still an active area of research, and data tata-augmentation schemes, crucial in practice, are not entirely straightforward fit into this framework. Furthermore, the high-dimensional integral (1) is (extremely) intractable: the posterior distribution <ddw (Puain) multi-modal, high-dimensional, concentrated along low-dimensional structures, and any local exploration algorithm (eg. MCMC, Langevin dynamics and their variations) is bound to only explore a tiny fraction of the state space. Because of the typically large number of degrees of symmetries, many of these local modes correspond to essentially similar predictions, indicating that it is likely not necessary to explore all the modes in order to approximate (I) well. A detailed understanding of the geometric properties of the posterior distribution in Bayesian neural networks is still lacking, although a lot of recent progress has been made. Indeed, variational approximations have been reported to improve, in some settings, over standard empirical risk minimization procedures. Deep-ensembles can be understood as crude, but practical, approximations of the integral in Equation (1). The high-dimensional integral can be approximated by simple non-weighted average over several modes wi,..., W of the posterior distribution found by minimizing the negative log- oog-oosterior, or some approximations of it, with standard optimization techniques:
1
(prediction) = K Pw, (w) (r)+ Pwn (x) E Ac.
We refer the interested reader [34, 29, 45, 3] for different perspectives Bayesian neural networks. Although simple and not well understood, deep-ensembles have been shown to provide highly robust uncertainty quantification when compared to more sophisticated approaches [22, 4, 25, 41].
The article [16] proposes a class of post-processing calibra- tion methods that extend the more standard Platt Scaling approach [36]. Temperature Scaling, the simplest of these methods, transforms the probabilistic outputs p(z) E Ac into a tempered version Scale Scale(p(z), r] E Ac defined through the scaling function
Post-processing Calibration Methods:
Scale(p, r) = aSM (log = 7 (1 "....."''' v. Ecc,
1 (pl/,,...,
for a temperature parameter O and normalization 7 O. The optimal parameter T. >> is usually found by minimizing proper-scoring rules [13], often chosen as the negative log-likelihood, on validation dataset. Crucially, during this post-processing step, the parameters of the probabilistic model are kept fixed: the only parameter being optimized is the temperature 7 > 0. In the low-data regime, the validation set being also extremely small, we have empirically observed that the more sophisticated Vector and Matrix scaling post-processing calibration methods [16] do not offer any significant advantage over temperature scaling approach and in fact overfit the extremely small validation dataset as chosen by our setup.
The Expected Calibration Error (ECE) measures the discrepancy between prediction confidence and empirical accuracy. For a partition 0 = Co <... < cM = 1 of the unit interval and labelled set { V set Bm, (i Cm-1 < p(x.)) S cm)- The quantity ECE is then defined as
Calibration Metrics:
Bmm
ECE =
Σ
conf m accm
(3)
N
1
where
acCm
1(j(x)) = yi) and confm
Σ
(lx)).
(4)
Bm ie
[mm
ie
3

---

A model is calibrated if accm z confm for all S m S M. It is often instructive to display the associated reliability curve. i.e. the curve with confm on the x-axis and the difference (acc,. conf on the y-axis. Figure displays examples of such reliability curves. A perfectly calibrated model flat (i.e. accm confm m 0), while the reliability curve associated to an under-confident (resp. over-confident) model prominently lies above (resp. below) the flat line acCm. conf = 0. We |[z,,) 7.
sometimes also report the value of the Brier score [5] defined N Σ
Setup and implementation details: For our experiments, we use standard neural architectures. For CIFAR10/100 [21] we use ResNet18, 8, ResNet34 [17] for Imaaenettee/magewwea the Diabetic Retinopathy [7], similar to [26] we use the architecture (not containing any residual connection) from the 5th place solution of the associated Kaggle challenge. We also include the
[18], and for
results for LeNet [23] trained on the MNIST [24] dataset in the supplementary. very low number of training examples (CIFARIO: 1000, CIFAR 100: 5000, Image woof]: 5000, MNIST: 500) was used for all the datasets. However, we also show that our observations extend to full-data setups in 4. The validation dataset is chosen from the leftover training dataset. The test dataset is kept the original and is hidden during both training and validation step.
3 Empirical Observations
Linear pooling: It has been observed in several studies that averaging the probabilistic predictions of set of independently trained neural networks, i.e., deep-ensembles, often leads and better-calibrated forecasts [22, 4, 25, 41, 9]. Figure 1 displays the reliability curves across three different datasets of of K 30 independently trained neural networks, well as the reliability curves of the aggregated forecasts obtained by simply linear averaging the K = 30 individual probabilistic predictions. These results suggest that deep-ensembles consistently lead to predictions that are less confident than the ones of its individual constituents. This can indeed be beneficial in the often encountered situation when each individual neural network is overconfident. Nevertheless, this phenomenon should not be mistaken with an intrinsic property of deep ensembles to lead to better-calibrated forecasts. For example, and as discussed further in Section 4, networks trained with the popular mixup augnenttgmtnnioonn are typically under-confident, Ensembling such set of individual networks typically leads predictions that are even more under-confident.
accurate
CIFAR10 under-confident
CIFAR100 over-confident
Imagewoof near-calibrated
0.2
0.2
0.2
0.0
0.0
0.0
-0.2
0.4
0.6
0.8
1.0
0.4
0.6
0.8
1.0
0.4
0.6
0.8
1.0
Individual Model
Pooled Model
Figure 1: Confidence confm (x-axis) vs. Difference (accm confm) (y-axis): We plot Reliability Curves in this figure. see Section 2 'Calibration Metrics' for definitions. The plots display the curves of K 30 individual networks (blue) trained on three datasets (i.e. CIFARIO, CIFARI00 and Imagewoof), as well as the pooled estimates (red) obtained by averaging the K individual predictions. This linear averaging leads to consistently less confident predictions (i.e. higer values of (accm - conf,, )). It is only beneficial to calibration when each network is over-confident. It is typically detrimental to calibration when the individual networks are already calibrated, or under- confident.
Other BNN methods: It is important to point out that under-confidence of pooled predictions are not limited to Deep Ensembles. Other modern Bayesian Neural Network methods show similar properties. In table we can see that ensembles obtained by SWAG [30] and MC-Dropout [1 I]. two other popular model averaging techniques, are more under-confident than the individual models.
In order to gain some insights into this phenomenon, recall the definition of the entropy functional H AC + IR, defined as H(p) - - Pk log Pk- The entropy functional is concave on the probability simplex Ac, i.e. H(Ap + (1) 2 X H(P) + (1 >) H(q) for any p. q E Ac- Furthermore, tempering probability distribution p leads to increased entropy if > 1, can
k=

---

Dataset
Method
Single models
Ensemble
SWAG
3.17 + .27
4.36
CIFAR 10
MC-Dropout
6.55 + 110
7.59
SWAG
3.34 + .14
5.49
CIFAR 100
MC-Dropout
4.92 + .19
9.05
Table 1: ECE, as defined in Equation (3), of twenty individual models and the ensemble of SWAG
[30] and MC-Dropout [11] trained with mixup augmentation on full CIFAR 1001000) dataset. The ensemble is less calibrated than the individual models.
be proved by examining the derivative of the function + H(p'/TJ. The entropy functional is consequently natural surrogate measure of (lack of) confidence. The concavity property of the entropy functional shows that ensembling set of K individual networks leads to predictions whose entropies are higher than the average of the entropies of the individual predictions. We provide additional quantitative heuristics for this phenomenon in the Supplementary Material.
Entropy
Reliability
NLL
Accuracy
Histogram of distance
Unscaled ensemble
1.5
20%
80%
0%
60%
1.5
-10%
1.2
Tempsca eed
ensemble
1.5
20%
1.0
1.0
60%
0.5
0.5
-20%%
40%
75%
25%
50%
75%
0%
25%
50%
75%
25%
50%
75%
0.0
1.0
No mixup
0.2
0.5
0.8
1.0
Figure 2: Metric (y-axis) vs. Distance from the training set (percentile) (x-axis): Deep En- sembles trained on N = 1000 CIFARIO samples with different amount of mixup regularization a E (.2,.5..8, .8. 1). The x-axis represents quantiles of the distance to the CIFAR 10 training set (see Section 3 for details). The overall distribution of the distances is displayed in the last column. The first row describes the performances of standard Deep Ensembles trained with data-augmennation and several amounts of mixup regularization a. In the second row, before averaging the predictions of the members of the ensemble, each individual network is first temperature scaled on a validation set of size Nval = 50: this corresponds method (B) of Section 4.
Distance to the training set: In order to gain some additional insights into the calibration properties of neural networks trained small datasets, as well as the influence of the popular mixup augmenta- tion strategy, we examine several metrics (i.e., Accuracy, Reliability, Negative Log-likelihood (NLL), Entropy) as a function of the distance to the (small) training set Drain The 2nd column of Figure displays the mean Reliability (i.e., acc conf) as a function of the distance percentiles. We focus on the CIFAR 10 dataset and train our networks
balanced subset of N 1000 training examples. Since there is no straightforward and semantically meaningful distance between images, we first use an unsupervised method (i.e., labels were not used) for learning -iw-dimeniional and semantically meaningful representation of dimension d = 128. For these experiments, we obtained mapping 323,33 + 1228 where 1128 C TR. 128 denotes the unit sphere in 1228 with the SimCLR method the cosine
[6]. We used the distance d(x, y) (2) (2))ll2. which in this case is equivalent distance between the 128-dimensional representations of the CIFARIO images r and y. The distance of test image x to the training dataset is defined min(d(x, 9:) Y: E D J. We computed the distances to the training set for each image contained in the standard CIFAR 10 test set (last column of Figure 2). Not surprisingly, we note that the average Entropy, Negative Log- loglliielhood, and Error Rate all increase for test samples further away from the training set.
Over-confidence:
The second column represents the Reliability curve, but with bins (x-axis) as distance percentile, rather than confidence. The predictions associated with samples chosen further away from the training set have lower value of acc conf. This indicates that the over-confidence of the predictions increases (csp. lower mixup a) with the distance
5

---

to the training set. In other words, even if the entropy increases as the distance increases (as it should), calibration issues do not vanish as the distance the training set increases. This phenomenon is irrespective of the amount of mixup used for training the network.
Effect of mixup-auggmentation: The first row of Figure 2 shows that increasing the amount of mixup augmentation generally leads an increase entropy, decrease in over-confidence, as well as more accurate predictions (lower NLL and higher accuracy). Additionally, the effect less pronounced for 2 0.2. This is confirmed in Figure 3 that displays more generally the effect of the mixun-augmentttton on the reliability curves over four different datasets. In the supplementary we provide more analysis on this.
Temperature Scaling: Importantly, the second row of Figure 2 indicates that post- processing temperature scaling for the individual models almost washes-out all the differ- ences due the mixup-auggentation scheme. For this experiment, ensemble of K 30 networks is considered: before averaging the predictions, each network has been individ- ually temperature scaled by fitting temperature parameter (through negative likelihood minimization) on a validation set of size Nealid = 50.
Imagewoof 1k samples
CIFAR10 1k sample
CIFAR100 5k samples
Diabetic Retinopathy
0.0
0.0
0.0
0.0
-0.2
-0.2
-0.2
-0.2
-0.4
-0.4
-0.4
-0.4
04
0.6
06.
alpha=0.0
alpha=0.3
alpha=0.6
alpha=0.9
Ideal line
Figure 3: Confidence confm (x-axis) vs. Difference (acon confm) (y-axis): Reliability curve of single neural network trained with different amount of mixup- auuuuup-agmmetttionn on the Imagewoof, CIFARIO, CIFARI00 and Diabetic Retinopathy datasets. Increasing the amount of mixup augmenta- tion, in general, makes the predictions less dess-confident. The case a 0 corresponds to training without an mixup-auggmentttin, i.c. only using standard augmentation strategies.
Calibrating Deep Ensembles
In order to calibrate deep ensembles, several methodologies can be considered:
(A) Do nothing and hope that the averaging process intrinsically leads to better calibration
(B) Calibrate each individual network before aggregating all the results
(C) Simultancously aggregate and calibrate the probabilistic forecasts of each individual model.
(D) Aggregate first the estimates of each individual model before calibrating the pooled estimate.
Simple pooling/aggregattoon rules that do not require large number of tuning parameters are usually preferred, especially when training data is scarce [20, 46]. Such rules usually robust, conceptually easy to understand, and straightforward to implement and optimize. The standard and most commonly used average pooling of a set p I:K of K 2 2 probabilistic predictions p'),..., E Ac C RC is defined as
Agg(p::8g 1:K) p²
p
(5)
K
Replacing the averaging with the median operation leads to median pooling strategy, where the median taken component-wis and then normalized afterward to obtain the final probability prediction. Alternatively, trimmed linear pooling strategy removes a pre-defined percentage of outlier predictions before performing the average in 5.
Pool-Then- Taol-ahte (D): any of the aforementioned aggregation procedure can used as pooling strategy before fitting temperature т. by minimizing proper scoring rules on validation set. In all

---

our experiments, we minimized the negative log- likeliiod (i.e., cross-entropy). For a given set p of K 2 2 probabilistic forecasts, the final prediction is defined as
1:k
p
Scale
Ag I:K), ),+))
(6)
where Scale(p, 7) = OsM (log p/T). Note that the aggregation procedure can be carried out entirely independently from the fitting of the optimal temperature T.
CIFAR10
CIFAR100
Imagenette
0.2
0.2
0.2
0.0
0.0
0.0
-02
Individual [B] scaled madels
Individual [C] scaled madels
Pooled [B] scaled madels
Pooled [C] scaled models
Figure 4: Confidence conf.m (X-axis) vs. Difference (accm - confmm ) (y-axis): Reliability curve of: (light blue) each model calibrated with one temperature per model (i.e. individually temperature scaled), (dark blue) average of individually temperature scaled models (i.e. method [B]), (orange) each model scaled with a global temperature obtained with method [C], (red) result of method [C] that consists in simultancously aggregating and calibrating the probabilistic forecasts of each individual model. Datasets: a valintialiittion split of size 950 50 was used for the CIFARIO IO and IMAGENETTE datasets, and of size 4700 300 for the CIFAR 100 dataset. To avoid clutter, we omit method [D] for its similarity with method [C] in terms of performance.
Joint Pool-and-Calibne (C): there are several situations when the so-called end-to-end training strategy consisting in jointly optimizing several component of composite system leads increased performances [33, 32, 15]. In our setting, this means learning the optimal temperature т. concurrently with the aggregation procedure. The optimal temperature T, found by minimizing proper scoring rule Score(-) on validation Dvalad {I,,, Not.
arg min {T
1
Score(p), .8)
Dvalid
Dead
where p. = Agg S 1: (I:), 7)] E Ac denotes the aggregated probabilistic prediction for sample z,. In all our experiments, we have found it computationally more efficient and robust to use simple grid search for finding the optimal temperature; we used n = 100 temperatures equally spaced on a logarithmic scale in between Tmin 10- 2 and Tmax 10.
Importance of the Pooling and Calibration order: Figure shows calibration curves when in- dividual models are temperature scaled separately (i.e. group [B] of methods), well as when the models are scaled with a common temperature parameter (i.e. group [C] of methods). Further- more, the calibration curves the pooled model (group [B] and [C] of methods) are also displayed. More formally, the group [B] of methods obtains for each individual model 1 < k S K an optimal
temperature (k) 0 as solution of the optimization procedure
arg min
Score Sccale [p?.T].u.
Dvalid
where D E Ac denotes the probabilistic output of the 4th model for the 8th example validation dataset. The light blue calibration curves corresponds to the outputs Scale [p*, T. for K different models. The deep blue calibration curve corresponds the linear pooling of the individually scaled predictions. For the group [C] of methods, single common temperature T. > 0 is obtained as solution of the optimization procedure described equation
(7). The orange calibration curves generated using the predictions Scale [p* r.] and the red curve corresponds to the prediction
Agg Scale(p T=)]. Notice that when scaled separately (by -!(*), each of the individual models (light blue) close to being calibrated, but the resulting pooled model (deep blue) under-confident. However, when scaled by common temperature. the optimization chooses temperature т, that
7

---

CIFAR10
CIFAR100
IMAGENETTE
IMAGEWOOF
DIABETIC RETINOPATHY
20%
5%
10%
ECE
4%
10%
10%
5.54
5%
599
0%
A
A
D
C
8
C
1.05
2.20
.75
650
NLL
1.00
2,00
.70
645
.95
640
.05
A
A
0
D
C
0
C
50
.66
Brier
.30
48
.42
.28
46
665
Linear pooling
Median pooling
Trimmed linear pooling
Temp scaled single model
Figure 5: Pooling method (x-axis) Metric (y-axis): Performance of different pooling strategies (A-D) with K = 30 models trained with mixup-augmentationn (a = 1) across multiple datasets. The total datasets (training + validation) were of size N 1000 for CIFAR10 and Imagenette and Imagewoof, and N = 5000 for CIFAR100 and Diabetic Retinopathy. Experiments were executed 50 times on the same training data but different validation sets. The dashed red line represents baseline performance when single model was training with mixup augmentation (a = 1) and post-processed with temperature scaling.
makes the individual models (orange) slightly over-confident so that the resulting pooled model (red) is nearly calibrated. This reinforces the justifications in section 3, and it also shows the importance of the order of pooling and scaling.
Figure compares the four methodologies A-B-C-D identified the start of this section, with the three different pooling approaches Agg and Aggmed and Agg These methods are compared to the baseline approach (in dashed red line) consisting of fitting a single network trained with the same amount a = 1 of mixup augmentation before being temperature scaled. All the experiments are executed 50 times, on the same training set, but with 50 different validation sets of size N = 50 for CIFAR10, Imagenette, Imagewoof and N = 300 for CIFAR 100, and Noai 500 for the Diabetic Retinopathy dataset. The results indicate that on most metries and datasets, the (naive) method (A) consisting of simply averaging predictions is not competitive. Secondly, and as explained in the previous section, the method (B) consisting in first calibrating the individual networks before pooling the predictions is less efficient across metrics than the last two methods (C- D). Finally, the two methods (C (C-D D) perform comparably, the method (D) (i.e. pool- mool-then-calibratee being slightly more straightforward to implement. With regards to the pooling methods, the intuitive robustness of the median and trimmed-ave -veereddaveraggig approaches do not seem to lead to any consistent gain across metrics and datasets. Note that ensembling set of K 30 networks (without any form of post-processing) does lead to a very significant improvement in NLL and Brier score but leads serious deterioration of the ECE. The Tool-Then-Calirree keeps the gains in NLL/Brier score unaffected, without compromising calibration.
Importance of the validation set: it would be practically useful to be able to fit the temperature without relying on validation set. We report that using the training set instead (obviously) does not lead to better-calibrated models. We have tried to a different amount of mixup- nixup-augmeetttion (and other types of augmentation) on the training set for fitting the temperature parameter but have not been able to obtain satisfying results.
Role and effect of mixup-augmmentatiin:: the mixup augmentation strategy is popular and straight- forward to implement. As already empirically described in Section 3, increasing the amount of mixup-augmentation typically leads to decrease the confidence and increase in entropy of the predictions. This can be beneficial in some situations but also indicates that this approach should

---

Accuracy
ECE
NLL
Brier score
150%
105%
0075
95%
95%
100%
50%
90%
95%
0%
85%
80%
No mixup
alpha 0.2
alpha 0.5
alpha 0.8
alpha 1.0
Figure 6: Mixup strength (x-axis) vs. Metric (y-axis): Phol-Then-Calibee approach when applied to deep-ensemble of K 30 networks trained with different amount of mixup-angmeentttion on N = 1000 CIFAR10 training samples (Nval = 50 were used for validation). For each metric, we report the ratio of performance when compared the Pool-then- Pool-then-Calibrate method used without any form of in using the
(only standard data-auumentation) The results indicate a clear benefit conjunction with temperature scaling. The error bars represent variability due to the choice of different validation sets.
certainly be employed with care for producing calibrated probabilistic predictions. Contrarily to other geometric data-augmentation transformations such image flipping, rotations, and dilatations, the mixup strategy produces non-realistic images that consequently lie outside the data-min aata-maniioold of natural images: leading to large distributional shift. Mixup relies on a subtle trade-off between the increase in training data diversity, which can help mitigate over-fitting problems, and the distributional shift that can be detrimental to the calibration properties of the resulting method. Figure 6 compares the performance of the Pool-Thhen- Prol-Ihhen-aalibrate approach when applied to a deep ensemble of = 30 networks trained with different amounts of mixup-a. The results compared to the same approach (i.e. Chal-bhen-Callbratee with K = 30 networks) with no auguu--agmmnttiionn The results indicate clear benefit in using the mixup-augmentatiio in conjunction with temperature scaling.
Extension to full-data setting: Although classification accuracy is usually not an issue when data plentiful, the lack of calibration can indeed be still present when models are trained with aggressive data-auge aatta--ugmentttionn strategies (as is common nowadays): the distributional shift between (data- augmented) training samples and (ron-augmented) test samples when models are used in production can lead to significant calibration issues. Although we mainly focus on low-data setting, below in table 2 we show that conclusion extends to full-data setting well. We have investigated below the CIFAR 100 full dataset (ResNet architecture/ / no-mixup) setting under varying conditions.
Method
Accuracy
ECE
NLL
Brier
(I) Individual models (unscaled)
70.8 .36
9.8
331
1.17 1 ,01
0.411
Ensemble of models in (I)
78.4
5.9
0.782
0.308
(2) Individual models (temp scaled)
70.8 .36
2.1 = .4
1.07 = .01
0.396
Ensemble of models in (2)
78.4
13.2
0.859
0.331
Pool-then-calibrat
78.4
3.4
0.770
0.303
Table 2: In line with our discussion in Sec 3, we show that linear pooling (A) appears to be helping with calibration (2nd row) when individual models are mildly over-confident (1st row). but performs worse h4th row) than individual models even full-data setting (CIFAR 100 50K training) when the individual models are near-calibrated (3rd row). Our proposed pool-then-calibrut (D) has the best performance (5th row).
The first row reports the performance of individual models trained without mixup: the individual models are over-confídent, but not extremely over-confident (presumably because of the large number of samples). When these models are pooled make an ensemble in the second row, the pooled model is better calibrated. This is the setup that is usually studied in almost every early articles investigating the properties of deep-ensembles, hence leading to the conclusion that deep- deep-ensembling inherently brings calibration. When we make the individual models calibrated in the 3rd row, where we used temp-scaling but can also be due to the effect more aggressive data-augmentation schemes, the individual calibration naturally improves significantly. Nevertheless, when we pool these calibrated models to make an ensemble, the pooled model suffers from extreme under-confidence (4th row). Our proposed method pool- pool-then-calibree (5th row) performs well even in full-data setting.

---

performance: We show the method compared vanilla ensembling when the ensembles are trained on CIFARIO and tested on subset of CIFAR1O 100 classes which are visually different from CIFARIO. In table 3, we show the metric: difference between the medians of the in-class and out-of-class prediction entropy (higher
detection performance of our
better).
Single model variations
Deep Ensemble [A]
Pool-then-calibrate [D]
0.342 = 0.015
0.359
0.521
Table 3: Difference in median prediction entropy between in-class (CIFARIO) vs out-of-class
(CIFAR100 subset) dataset. Pool-then-calibrat brings significant improvement in terms of out-of- distribution detection.
Pool-then- Caalbrrae performs significantly better than vanilla ensemble in separating the predictions for in-class and out-of-class observations (45% more separation in terms of distance between medians). In table 4, we also show the performance when we run inference on the CIFARIO- dataset (Gaussian noise) after training our ensemble model the setting: 1000 samples of CIFARIO dataset with mixup 1.0. As expected, vanilla ensembling with linear pooling (A) has worse calibration than single models, while pool-tten-calibrate (D) improves score across the board.
Method
Accuracy
ECE
NLL
Brier
Individual models
59.38 = 0.05
6.57 0.006
1.237 + 0.012
0.549
0.005
Deep Ensemble [A]
64.63
15.13
1.145
0.511
Pool-then- Calibrate
Pool-then-aalibrate [D]
64.63
1.65
1.059
0,480
Table Inference on CIFAR-IOC IOC (Gaussian noise), trained on CIFARIO (IK sample). Pool-then-
calibrate [D] performs better while vanilla ensemble [A] has worse calibration than single models.
Additional experiments: In the supplementary, add more experiments on the effect of number of models in the ensemble, detailed numerical results for all datasets as well as MNIST, ablation study, and effect of different mixup levels all the metrics.
Cold posteriors: the article [43] reports gains in several metrics when fitting Bayesian neural networks tempered posterior of type n, (0) x T(0) where T(f) is the standard Bayesian posterior, for temperatures T smaller than one. Although not identical to our setting, it should be noted that all our experiments, the optimal temperature т, was consistently smaller than one. In our setting, this is because simply averaging predictions lead to under-confident results. We postulate that related mechanisms are responsible for the observations reported [43].
Discussion
The problem of calibrating deep-ensee enteeebeeeblles has received surprisingly little attention in the literature. In this text, we examined the interaction between three of the most simple and widely used methods for adopting deep-learning to the low-data regime: ensembling, temperature scaling, and mixup data augmentation. We highlight that ensembling itself does not lead to better-calibrated predictions, that the mixup augmentation strategy is practically important and relies on non-trivial trade-offs, and that these methods subtly interact with each other. Crucially, we demonstrate that the order in which the pooling and temperature scaling procedures are executed is important to obtaining calibrated deep-ensembles. We advocate the Poal-Then-Culibrate approach consisting of first pooling the individual neural network predictions together before eventually post-processing the result with simple and robust temperature scaling step.
Broader Impact
Producing well-calibrated probabilistic predictions is crucial to risk management, and when decisions that rely on the outputs of probabilistic models have to be trusted. Furthermore, designing well- calibrated models is crucial to the adoption of machine-learnin methods by the general public, especially in the field Al-driven medical diagnosis, since it intimately related to the issue of trust in new technologics.
10

---

References
[1] Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in- domain uncertainty estimation and ensembling in deep learning, 2021. 2
[2] Alberto Bietti and Julien Mairal. On the inductive bias of neural tangent kernels. In Advances in Neural Information Processing Systems, pages 12873-12884, 2019. 1
[3] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks, 2015. 2,3
[4] Hamed Bonab and Fazli Can. Less is more: comprehensive framework for the number of components of ensemble classifiers. arXiv preprinl arXiv: (709.02209 2017. 2,
[5] Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1-3, 1950. 4
[6] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002..5709, 2020. 5
[7] Jorge Cuadros and George Bresnick. EyePACS: An Adaptable Telemedicine System for Diabetic Retinopathy Screening. Journal of Diabetes Science and Technology, 3(3):509-516, May 2009.
[8] Gintare Karolina Dziugaite and Daniel Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint arXiv: 1108, 2017.
[9] Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspective. arXiv preprint arXiv: (tt::19122 1912.02757, 2019. 2,4
[10] Marylou Gabrié, Andre Manoel, Clément Luneau, Nicolas Macris, Florent Krzakala, Lenka Zdeborová, et al. Entropy and mutual information in models of deep neural networks. In Advances in Neural Information Processing Systems, pages 1821-1831, 2018.
[11] Y. Gal and Z. Ghahramani. Dropout bayesian approximation. International Conference on Machine Learning, 2016. 5,
[12] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference machine learning, pages 1050-1059,
2016. 1.2
[13] Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359-377 -785999--7--- 2007. 3
[14] A. Graves. Practical variational inference for neural networks. NIPS, 2011.
[15] Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska- Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471
2016.
[16] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. Proceedings of the 34 th International Conference Machine Learning, Sydney, Australia, PMLR 70, 2017, 2017. 2,
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, abs//511..03355 2015.
[18] Jeremy Howard. Imagenette and Imagewoof, 2018.
[19] Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization neural networks. In Advances in neural information processing systems, pages 8571-8580 2018. I
11

---

[20] Victor Richmond R Jose and Robert L Winkler. Simple robust averages of forecasts: Some empirical results. International journal of forecasting. 24(1):163-163 2008. 6
[21] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar- 10 (canadian institute for advanced research). 4
[22] B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA, 2017. 3,
[23] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1(4):541-551, 12 1989.
[24] Yann LeCun and Corinna Cortes, MNIST handwritten digit database. 2010.
[25] Stefan Lee, Senthil Purushwalkam, Michael Cogswell, David Crandall, and Dhruv Batra. Why m heads are better than one: Training a diverse ensemble of deep networks. arXiv preprint arXiv:1511.00 2015. 2,3,4
[26] Christian Leibig, Vaneeda Allken, Murat Seçkin Ayhan, Philipp Berens, and Siegfried Wahl. Leveraging uncertainty information from deep neural networks for disease detection. Scientific Reports, 7(1):17816, Dec 2017. 4
[27] C. Louizos and M. Welling. Structured and efficient variational deep learning with matrix gaussian posteriors. arXiv preprint (tth::/603047333 2016.
[28] D. J. C. MacKay. A practical bayesian framework for backpropagation networks. Neural Computation, (13):4484477 1992. 2
[29] David JC MacKay and David JC Mac Kay. Information theory, inference and learning algo- rithms. Cambridge university press, 2003. 3
[30] W. Maddox, T. Garipov, P. Izmailov, D. Vetrov, and A. G. Wilson. A simple baseline for bayesian uncertainty in deep learning. arXiv preprint 1rXiv:1902.02476, 2019. 1, 2, 4, 5
[31] Song Mei, Andrea Montanari, and Phan-Minh Nguyen. A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences, 115(33):E77655- E7671, 2018.
[32] Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andrew Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, al. Learning navigate in complex environments. arXiv preprint arXiv: 161 I.03673, 2016. 7
[33] Volodymyr Mnih, Koray Kavukeuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533 2015. 7
[34] Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
[35] Luis Perez and Jason Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv: 1712.0:7712.04000 2.0221, 2017.
[36] J. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances Large Margin Classifiers, 10(3), 1999. 3
[37] Lutz Prechelt. Early stopping-but when? In Neural Networks: Tricks of the trade, pages 55-69. Springer, 1998.
[38] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv: 4082, 2014. 2
12

---

[39] Grant M Rotskoff and Eric Vanden-Eijnden Neural networks as interacting particle systems: Asymptotic convexity of the loss landscape and universal scaling of the approximation error. arXiv preprini arXiv:/805.0003 1805.00915, 2018.
[40]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929-199) 2014.
[41] Christian Szegedy, Wei Liu, Yangging Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 1-9, 2015. 2, 3,4
[42] Yeming Wen, Ghassen Jerfel, Rafael Muller, Michael W. Dusenberry, Jasper Snoek, Balaji Lakshminarayanan, and Dustin Tran. Combining ensembles and data augmentation can harm your calibration, 2021. 2
[43] Florian Wenzel, Kevin Roth, Bastiaan Veeling, Jakub Swiątkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the bayes posterior in deep neural networks really? arXiv preprint arXiv:2002.024055 2020. 10
[44] Andrew Wilson, Zhiting Hu, Ruslan R Salakhutdinov, and Eric Xing. Stochastic variational deep kernel learning. In Advances in Neural Information Processing Systems, pages 2586-2594, 2016.
[45] Andrew Gordon Wilson and Pavel Izmailov. Bayesian deep learning and a probabílistic perspective of generalization. arXiv preprint arXiv:2002.08791, 2020. 3
[46] Robert L. Winkler, Yael Grushka-Cockayne Kenneth C. Lichtendahl, and Victor Richmond Jose. Probability forecasts and their combination: A research perspective. Decision Analysis, 16(4):239-260, 2019. 6
[47] Xixin Wu and Mark Gales. Should ensemble members calibrated?, 2021.
[48] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:/611.035300 2016.
[49] Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. CoRR, abs/1710.09412, 2017.
[50] Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology), -302)301-3 2005.
13


Reviews:
[{"review_id": "O3ji-Z7DNcz", "reviewer": "Reviewer_KgKq", "summary": "An empirical analysis of the interplay between deep ensembles, data augmentation and calibration with temperature scaling. The paper seeks to clear out several misconceptions about ensembling (eg. ensembling does not necessarily imply better calibration -- this is only true if each network in the ensemble is overconfident) and to provide practical recommendations (eg., order in which the ensemble aggregation and the calibration should be carried out).", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Overall appreciation (originality, quality, clarity, and significance)**\n- Clarity: A well motivated and very well written paper for which the main claims would benefit from more thorough / rigorous proofs to be more convincing. The paper is very easy to read throughout -- all key terms are adequately defined, the work is adequately structured and flows naturally\n- Quality: The work draws conclusions and provides guidance based on (sometimes limited) empirical evidence only (no theoretical proof; empirical analyses limited to 4 datasets, single model architecture for each). Some aspects of the experimental design might be problematic (see below).\n- Originality: The interplay between ensembling, calibration and data augmentation has been understudied to date -- this paper provides some interesting elements of response\n- Significance: limited to a couple insightful takeaways (the one about calibration vs aggregation ordering needs some clarification) \n\n**Clarifying questions / suggestions**\n- Introduction → Please clarify what you mean by “low data regime” -- this may mean different things to different people\n- Section 4 → In the descriptions of all experiments, you seem to only reference a training set (on which models are trained) and a validation set (on which the temperature scaling is tuned) but no test set. Are all results reported in Fig2-5 reported on the same validation set on which you perform the temperature scaling tuning? If so, the claim that “pool-then-calibrate” is better than “calibrate-then-pool” approach seems trivial. Conclusions drawn in section 4 (Fig5 in particular) would only be meaningful if results are reported on a separate test set\n- Section 4, line 200 → Did you look into other types of aggregation (e.g., geometric or harmonic mean)?\n- Related work (line 294-297) → This “related work” section feels a bit contrived as such. Would suggest expanding a bit more \n\n**Minor points**\n- Line 41 -- adapting (typo)\n- Line 49 -- augmentations (typo)\n- Line 180 -- drop “a” after “as well as” (typo)\n- Line 292-293 -- “When datasets are large, the differences in calibration performances between  the several methods are relatively minor” -- please cite\n- Line 311 -- drop capital letter for “Designing”", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Well covered overall", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "Mpi_u_spOcS", "reviewer": "Reviewer_LouX", "summary": "This paper empirically studies the calibration of deep ensembles in low-data settings, as well as the interaction between ensembling, post-hoc calibration (i.e. temperature scaling) and mixup data augmentation (as a form of regularization). The main experimental findings include that in the low-data regime, 1) ensembling neural networks (i.e. averaging their predictions) _reduces_ their confidence, 2) mixup regularization generally makes neural networks _underconfident_, 3) as a result, ensembling mixup-regularized models does _not_ improve calibration but instead exacerbates their underconfidence, and 4) temperature scaling can help improve the calibration of deep ensembles, but only if it is applied _after_ averaging the individual model predictions.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Post Rebuttal**\n\nThe author response clarifies some of the issues I had raised -- I therefore increase my score from weak rejection (5) to weak acceptance (6). I am still hesitant to recommend acceptance more enthusiastically due to the remaining issue of a low methodological contribution.\n\n---\n\nThis paper tackles an important open problem in practical deep learning (which is of significant interest to the NeurIPS community), namely how to obtain models that are well calibrated and therefore robust. It provides a comprehensive and well-executed empirical study with conclusive results and actionable suggestions that will be useful to deep learning practitioners. The manuscript is also very well-written and easy to follow. That being said, the methodological contribution and novelty is rather low, and the scope of the evaluation somewhat limited (by focusing on the low-data regime). Importantly, while the empirical results and resulting practical suggestions are certainly interesting and useful, previously published work has made very similar observations. In particular, it has been shown in prior work that Mixup + ensembling deteriorates calibration [1] (which the authors acknowledge, claiming that it succeeds their work) and that temperature scaling after ensemble averaging improves calibration [2] (which the authors seem to have missed). While this seems to compromise the novelty of this paper, I can certainly understand the authors claims that the work was conducted in concurrence (at least [1]), so I am somewhat torn. All in all, I view this paper as borderline, and am tending to recommend rejection — but I am happy to change my judgement and increase my score if the authors can convincingly address my concerns.\n\n\nDetailed comments are as follows:\n\n- The methodological contribution is rather low; the paper does not introduce a new method, but rather conducts a study to assess the interaction between different existing approaches, namely deep ensembling, Mixup regularization/augmentation and temperature scaling; the lack of a methodological contribution is not an issue in itself, but raises the bar for the empirical evaluation.\n- The empirical evaluation is generally well-executed and fairly comprehensive. It studies the interaction of practically relevant and widely-used techniques (ensembling, mixup augmentation & temperature scaling) in a variety of settings. The main conclusions are convincingly supported by the results and provide clear suggestions for practitioners. Also, the paper is generally very well written and enjoyable to read.\n- While the results are certainly interesting and useful, the main issue I see with this work is that previously papers (which have been published already) have made very similar observations. In particular:\n     - [1] also studied the interaction between ensembling and Mixup, reaching the same conclusion that they both individually reduce confidence, and that combining them therefore yields compounded underconfidence which deteriorates calibration. [1] then continues to argue that this is caused by the conflation of model and data uncertainty, and proposes CAMixup, which improves upon Mixup by adaptively adjusting the mixing parameter on a class-wise basis. The paper clearly mentions [1] in their related work section (l. 294-297), but claims that it is \"subsequent to our work\". I personally cannot comment on that, and would suggest the authors to clarify this directly with the area chair.\n    - [2] has a different goal to this paper ([2] studies issues with commonly-used calibration metrics and suggests to always use temperature scaling for a fair comparison of model calibration), but their empirical evaluation ultimately ends up with the same recommendation as this work, namely that applying temperature scaling on top of the ensemble predictive is beneficial to calibration. I appreciate that the paper under review has a different focus and also does a more comprehensive evaluation of the combination between temperature scaling and ensembling (in particular, assessing different ways/orders of combining the two), but this nevertheless seems to impact the novelty and potential significance of this work. I would appreciate the author's comments on this. It might be worth pointing out that the paper does not mention [2] anywhere, and that [2] was published at ICLR 2020, so it would likely not be considered as concurrent work.\n- One could argue that by focusing on the small-data regime, the scope of the evaluation becomes somewhat narrow. However, I believe that the small-data regime is important and therefore welcome studies focusing on it, as most other papers typically look at the large-data regime. That being said, it is clear that some of the conclusions drawn from this work do not translate beyond the small-data setting, which perhaps could be clarified even more. For example, the authors acknowledge that for large datasets, the differences in performance between the methods assessed are relatively minor (l 291-293). It would be great if the authors could clarify what exactly they mean by that (ideally by showing the larger-scale experiments that were conducted to reach this conclusion).\n\n\nQuestions:\n- In l.115-118 you state that „in the low-data regime, […] we have empirically observed that […] Vector and Matrix scaling post-processing calibration methods do not offer any significant advantage over the simple and robust temperature scaling approach.“; I did not find those experiments in the paper or appendix, were these just preliminary results that you did not intend to share? I believe that many people would be interested in seeing this comparison (me included), so it might be worth in including it in the appendix.\n- It might be clearer to describe the setup and implementation details (l. 285-293) at the beginning of the experiments, rather than at the end.\n- Some of the figure captions just describe the plots, but do not state the conclusions drawn from them. While I understand that this is personal preference, I personally find it easier to parse if the conclusions are summarized in the captions, to avoid having to jump back-and-forth between the text and figures.\n- In Fig. 2, for the unscaled ensemble, why is mixup with alpha=0.2 much more confident than the rest (e.g. when looking at entropy)?\n- Typo in l.249: „As regards the“ —> „With regards to the“ / „Regarding the“\n- The plot labels/legends are not consistent (compare e.g. Fig 2 and Fig 3/6, where the mixup parameters are labeled in different ways).\n- In l. 287, you mention that you took the 5th place solution from the Kaggle challenge for the Diabetic Retinopathy dataset; it would be great to have a justification for this.\n- Typo in l. 311: „Designing“ —> „designing“\n- In the references, paper [2] and [3] are identical and should be merged into a single reference.\n\n\n\n[1] Wen et al. 2021, \"Combining Ensembles and Data Augmentation can Harm your Calibration\" (https://arxiv.org/abs/2010.09875).\n\n[2] Ashukha et al. 2020, \"Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning\" (https://arxiv.org/abs/2002.06470).\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the paper describes the limitations of the work by clearly stating the assumptions under which the presented results hold. It also addresses its potential societal impact in sufficient detail.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "K5Bw7OlWs5", "reviewer": "Reviewer_SiXE", "summary": "This paper focuses on the calibration performance of deep ensembles of over-parameterized models. In particular, this work studies the interaction between data augmentation, ensembling and post-processing calibration such as temperature scaling. Each of the technique improves calibration when applied alone. However, the authors demonstrated that both mixup and temperature scaling might hurt calibration when naively combined with ensembling. The empirical evaluation identifies the underconfidence as the core issue in calibration degradation. It also shows that the proposed modification to temperature scaling can greatly improve calibration.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Both mixup and temperature scaling are prevailing methods to improve deep neural networks calibration. However, their interactions with deep ensembles received only little attention. This paper is a step towards understanding these interactions. It identifies that the compounding underconfidence is the core issue of worse calibration. Temperature scaling is generally used without extra caution but the authors showed that it need to be tweaked when combined with ensembels. In particular, the order of pooling and scaling affects the calibration of deep ensembles and the authors found that the Pool-Then-Calibrate approach gives the best result.\n\nI also have several concerns, one of the major concern is that the paper focuses on the empirical interaction between mixup, temperature scaling and deep ensembles. However, both mixup and temperature scaling are studied before (although not with deep ensembles). The fix proposed by this paper is tweaking the order of pooling and temperature scaling rather than any fundamentally different idea. Although some new insights are discovered, the algorithmic contribution is still limited. \n\nThe empirical evaluation is limited in the vision domain. The paper would be more convincing if it can show its conclusion can be extended to transformer models in the NLP domain. Moreover, one of the important benefits of ensembles is it can improve uncertainty estimation on out-of-distribution dataset. The paper can also be convincing if the authors can show consistent conclusion on CIFAR-10C and CIFAR-100C dataset.\n\nThe authors didn't state which neural network architectures are studied (it is in the appendix saying 30 ensembles of ResNet-18). It would be interesting if the paper can study the case for a small number of ensembles such as 4.\n\nAt the current moment, the cons outweigh the pros so I consider the paper is marginally below the boarderline.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors discussed the limitation.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies the quality of the uncertainty of ensembles of deep networks through the lens of calibration.  The authors demonstrate that contrary to common belief, ensembling can lead to even worse calibration.  They identify that an altered version of temperature scaling, a common strategy, can help to significantly improve the calibration of these ensembles.  During the discussion period a question came up regarding novelty compared to an existing published paper that was very related.  Given the timeline of that paper compared to this one, the reviewers were instructed to treat that paper as concurrent work.  

There was significant discussion for this paper during the author response period, multiple reviewers raised their scores and a consensus of weak accept was reached.  Quoting from the discussion, the reviewers all "agreed that 1) the paper studies an important problem (i.e. the interplay between ensembling, data augmentation and post-hoc calibration) that is of high relevance and interest to to the NeurIPS community, 2) the empirical evaluation fairly convincingly supports the key hypotheses (with some minor caveats in terms of breadth of experimental settings), and 3) the resulting practical suggestions (e.g. to apply post-hoc calibration after ensembling) are interesting and useful. Furthermore, there were no concerns raised w.r.t. the writing, which was clear and of high quality."  However, the reviewers agreed that there is not a strong strong algorithmic, methodological, or theoretical contribution and thus no reviewers were willing to champion the paper with a rating higher than a marginal accept.  

The fact that all reviewers were unwilling to recommend an "accept" rating or champion this paper seems to indicate that while they believe it meets the bar for acceptance, they don't think it will be tremendously impactful in its current form.  This appears to be partially owing to the fact that the paper has been on arxiv for a considerable time and in the meantime other papers have reported similar results.  Therefore, the recommendation is to accept the paper as a poster.  The paper could be more impactful if there was some additional theoretical justification for the observed phenomenon or some additional analysis, e.g. from the loss landscape perspective.
Decision: Accept (Poster)