Title: PITFALLS OF IN-DOMAIN UNCERTAINTY ESTIMATION AND ENSEMBLING IN DEEP LEARNING

Abstract: Uncertainty estimation and ensembling methods go hand-in-hand. Uncertainty estimation is one of the main benchmarks for assessment of ensembling performance. At the same time, deep learning ensembles have provided state-of-the-art results in uncertainty estimation. In this work, we focus on in-domain uncertainty for image classification. We explore the standards for its quantification and point out pitfalls of existing metrics. Avoiding these pitfalls, we perform a broad study of different ensembling techniques. To provide more insight in this study, we introduce the deep ensemble equivalent score (DEE) and show that many sophisticated ensembling techniques are equivalent to an ensemble of only few independently trained networks in terms of test performance.

1. INTRODUCTION:

Deep neural networks (DNNs) have become one of the most popular families of machine learning models. The predictive performance of DNNs for classification is often measured in terms of accuracy. However, DNNs have been shown to yield inaccurate and unreliable probability estimates, or predictive uncertainty (Guo et al., 2017) . This has brought considerable attention to the problem of uncertainty estimation with deep neural networks.

There are many faces to uncertainty estimation. Different desirable uncertainty estimation properties of a model require different settings and metrics to capture them. Out-of-domain uncertainty of the model is measured on the data that does not follow the same distribution as the training dataset (out-of-domain data). Out-of-domain data can include images corrupted with rotations or blurring, adversarial attacks (Szegedy et al., 2013) or data points from a completely different dataset. The model is expected to be resistant to data corruptions and to be more uncertain on out-of-domain data than on in-domain data. On the contrary, in-domain uncertainty of the model is measured on data taken from the training data distribution, i.e. data from the same domain. In this setting the model is expected to produce reliable probability estimates, e.g. the model shouldn't be too overconfident in its wrong predictions.

Pitfalls of metrics:

We show that many common metrics of in-domain uncertainty estimation (e.g. log-likelihood, Brier score, calibration metrics, etc.) are either not comparable across different models or fail to provide a reliable ranking. We address some of the stated pitfalls and point out more reasonable evaluation schemes. For instance, although temperature scaling is not a standard for ensembling techniques, it is a must for a fair evaluation. With this in mind, the calibrated log-likelihood avoids most of the stated pitfalls and generally is a reasonable metric for in-domain uncertainty estimation task.

Pitfalls of ensembles Equipped with the proposed evaluation framework, we are revisiting the evaluation of ensembles of DNNs-one of the major tools for uncertainty estimation. We introduce the deep ensemble equivalent (DEE) score that measures the number of independently trained models that, when ensembled, achieve the same performance as the ensembling technique of interest. The DEE score allows us to compare ensembling techniques across different datasets and architectures using a unified scale. Our study shows that most of the popular ensembling techniques require averaging predictions across dozens of samples (members of an ensemble), yet are essentially equivalent to an ensemble of only few independently trained models.

Missing part of ensembling In our study, test-time data augmentation (TTA) turned out to be a surprisingly strong baseline for uncertainty estimation and a simple way to improve ensembles. Despite being a popular technique in large-scale classification, TTA seems to be overlooked in the community of uncertainty estimation and ensembling.

2. SCOPE OF THE PAPER:

We use standard benchmark problems of image classification which comprise a common setting in research on learning ensembles of neural networks. There are other relevant settings where the correctness of probability estimates can be a priority, and ensembling techniques are used to improve it. These settings include, but are not limited to, regression, language modeling (Gal, 2016) , image segmentation (Gustafsson et al., 2019) , active learning (Settles, 2012) and reinforcement learning (Buckman et al., 2018; Chua et al., 2018) .

We focus on in-domain uncertainty, as opposed to out-of-domain uncertainty. Out-of-domain uncertainty includes detection of inputs that come from a completely different domain or have been corrupted by noise or adversarial attacks. This setting has been thoroughly explored by (Ovadia et al., 2019) .

We only consider methods that are trained on clean data with simple data augmentation. Some other methods use out-of-domain data (Malinin & Gales, 2018) or more elaborate data augmentation, e.g. mixup (Zhang et al., 2017) or adversarial training (Lakshminarayanan et al., 2017) to improve accuracy, robustness and uncertainty.

We use conventional training procedures. We use the stochastic gradient descent (SGD) and use batch normalization (Ioffe & Szegedy, 2015) , both being the de-facto standards in modern deep learning. We refrain from using more elaborate optimization techniques including works on superconvergence (Smith & Topin, 2019) and stochastic weight averaging (SWA) (Izmailov et al., 2018) . These techniques can be used to drastically accelerate training and to improve the predictive performance. Thus, we do not comment on the training time of different ensembling methods since the use of these and other more efficient training techniques would render such a comparison obsolete.

A number of related works study ways of approximating and accelerating prediction in ensembles. The distillation mechanism allows to approximate the prediction of an ensemble by a single neural network (Hinton et al., 2015; Balan et al., 2015; Tran et al., 2020) , whereas fast dropout (Wang & Manning, 2013) and deterministic variational inference (Wu et al., 2018) allow to approximate the predictive distribution of specific stochastic computation graphs. We measure the raw power of ensembling techniques without these approximations.

All of the aforementioned alternative settings are orthogonal to the scope of this paper and are promising points of interest for further research.

3. PITFALLS OF IN-DOMAIN UNCERTAINTY ESTIMATION:

No single metric measures all the desirable properties of uncertainty estimates obtained by a model of interest. Because of this, the community is using many different metrics in an attempt to capture the quality of uncertainty estimation, such as the Brier score (Brier, 1950) , log-likelihood (Quinonero-Candela et al., 2005) , metrics of calibration (Guo et al., 2017; Nixon et al., 2019) , performance of misclassification detection (Malinin & Gales, 2018) , and threshold-accuracy curves Here TACE is reported for VGG16BN model on CIFAR-100 dataset and is evaluated at the optimal temperature. (Lakshminarayanan et al., 2017) . In the section we highlight the pitfalls of the aforementioned metrics, and demonstrate that these pitfalls can significantly affect evaluation, changing the ranking of the methods.

Notation:

We consider a classification problem with a dataset that consists of N training and n testing pairs (x i , y * i ) ∼ p(x, y), where x i is an object and y * i ∈ {1, . . . , C} is a discrete class label. A probabilistic classifier maps an object x i into a predictive distribution p(y | x i ). The predictive distribution p(y | x i ) of a deep neural network is typically defined by the softmax function p(y | x) = Softmax(z(x)/T ), where z(x) is a vector of logits and T is a scalar parameter standing for the temperature of the predictive distribution. This scalar parameter is usually set to T = 1 or is tuned on a validation set (Guo et al., 2017) . The maximum probability max c p(y = c | x i ) is called the confidence of a classifier p on an object x i . I[•] denotes the indicator function throughout the text.

3.1. LOG-LIKELIHOOD AND BRIER SCORE:

The average test log-likelihood LL = 1 n n i=1 log p(y = y * i | x i ) is a popular metric for measuring the quality of in-domain uncertainty of deep learning models. It directly penalizes high probability scores assigned to incorrect labels and low probability scores assigned to the correct labels y * i . LL is sensitive to the softmax temperature T . The temperature that has been implicitly learned during training can be far from optimal for the test data. However, a nearly optimal temperature can be found post-hoc by maximizing the log-likelihood on validation data. This approach is called temperature scaling or calibration (Guo et al., 2017) . Despite its simplicity, the temperature scaling results in a notable improvement in the LL.

ensembling techniques (see Figure 10 ), this effect is most noticeable on experiments with data augmentation on ImageNet (Figure 1 ).

We introduce a new metric called the calibrated log-likelihood that is the log-likelihood at the optimal temperature.

The calibrated log-likelihood considers a model and a post-training calibration as a unified system, targeting to measure all models in the equal conditions of optimal temperature. That allows to avoid measuring calibration error that can be eliminated by a simple temperature scaling. The metric significantly affects the results of the comparison. For example, in Figure 10 the differences between Bayesian (VI, K-FAC, SWAG, dropout) and conventional non-Bayesian networks become much less pronounced, and in most cases making conventional non-Bayesian networks match the performance of Bayesian ones (VI, K-FAC, Dropout) on ResNet110, ResNet164, and WideResNet.

We show how to obtain an unbiased estimate of the calibrated log-likelihood without a held-out validation set in Section 3.5.

LL also demonstrates a high correlation with accuracy (ρ > 0.86), that in case of calibrated LL becomes even stronger (ρ > 0.95). That suggests that while (calibrated) LL measures the uncertainty of the model, it still has a significant dependence on the accuracy and vice versa. A model with higher accuracy would likely have a higher log-likelihood. See Figure 9 in Appendix C for more details.

Brier score BS = 1 n 1 C n i=1 C c=1 (I[y * i = c] -p(y = c | x i ))

2 has also been known for a long time as a metric for verification of predicted probabilities (Brier, 1950) . Similarly to the log-likelihood, the Brier score penalizes low probabilities assigned to correct predictions and high probabilities assigned to wrong ones. It is also sensitive to the temperature of the softmax distribution and behaves similarly to the log-likelihood. While these metrics are not strictly equivalent, they show a high empirical correlation for a wide range of models on CIFAR-10, CIFAR-100 and ImageNet datasets (see Figure 8 in Appendix C ).

3.2. MISCLASSIFICATION DETECTION:

Detection of wrong predictions of the model, or misclassifications, is a popular downstream problem relevant to the problem of in-domain uncertainty estimation. Since misclassification detection is essentially a binary classification problem, some papers measure its quality using conventional metrics for binary classification such as AUC-ROC and AUC-PR (Malinin & Gales, 2018; Cui et al., 2019; Możejko et al., 2018) . These papers use an uncertainty criterion like confidence or predictive entropy H[p(y | x i )] as a prediction score.

While these metrics can be used to assess the misclassification detection performance of a single model, they cannot be used to directly compare misclassification performance across different models. Correct and incorrect predictions are specific for every model, therefore, every model induces its own binary classification problem. The induced problems can differ significantly, since different models produce different confidences and misclassify different objects. In other words, comparing such metrics implies a comparison of performance of classifiers that solve different classification problems. Such metrics are therefore incomparable.

AUCs for misclassification detection cannot be directly compared between different models.

While comparing AUCs is incorrect in the setting of misclassification detection, it is correct to compare these metrics in many out-of-domain data detection problems. In that case, both objects and targets of the induced binary classification problems remain the same for all models. All outof-domain objects have a positive label and all in-domain objects have a negative label. Note that this condition does not necessarily hold in the problem of detection of adversarial attacks. Different models generally have different inputs after an adversarial attack, so such AUC-based metrics might still be flawed.

3.3. CLASSIFICATION WITH REJECTION:

Accuracy-confidence curves are another way to measure the performance of misclassification detection. These curves measure the accuracy on the set of objects with confidence max c p(y = c | x i ) above a certain threshold τ (Lakshminarayanan et al., 2017) and ignoring or rejecting the others.

The main problem with accuracy-confidence curves is that they rely too much on calibration and the actual values of confidence. Models with different temperatures have different numbers of objects at each confidence level which does not allow for a meaningful comparison. To overcome this problem, one can swhich from thresholding by the confidence level to thresholding by the number of rejected objects. The corresponding curves are then less sensitive to temperature scaling and thus allow to compare the rejection ability in a more meaningful way. Such curves have been known as accuracyrejection curves (Nadeem et al., 2009) . In order to obtain a scalar metric for easy comparisons, one can compute the area under this curve, resulting in AU-ARC (Nadeem et al., 2009) .

3.4. CALIBRATION METRICS:

Informally speaking, a probabilistic classifier is calibrated if any predicted class probability is equal to the true class probability according to the underlying data distribution (see (Vaicenavicius et al., 2019) for formal definitions). Any deviation from the perfect calibration is called miscalibration. For brevity, we will use pi,c to denote p(y = c | x i ) in the current section.

Expected calibration error (ECE) (Naeini et al., 2015) is a metric that estimates model miscalibration by binning the assigned probability scores and comparing them to average accuracies inside these bins. Assuming B m denotes the m-th bin and M is overall number of bins, the ECE is defined as follows:

EQUATION

where

acc(B) = |B| -1 i∈B I[arg max c pi,c = y * i ] and conf(B) = |B| -1 i∈B pi,y * i .

A recent line of works on measuring calibration in deep learning (Vaicenavicius et al., 2019; Kumar et al., 2019; Nixon et al., 2019) outlines several problems of the ECE score. Firstly, ECE is a biased estimate of the true calibration. Secondly, ECE-like scores cannot be optimized directly since they are minimized by a model with constant uniform predictions, making the infinite temperature T = +∞ its global optimum. Thirdly, ECE only estimates miscalibration in terms of the maximum assigned probability whereas practical applications may require the full predicted probability vector to be calibrated. Finally, biases of ECE on different models may not be equal, rendering the miscalibration estimates incompatible. Similar concerns are also discussed by Ding et al. (2019) .

Thresholded adaptive calibration error (TACE) was proposed as a step towards solving some of these problems (Nixon et al., 2019) . TACE disregards all predicted probabilities that are less than a certain threshold (hence thresholded), chooses the bin locations adaptively so that each bin has the same number of objects (hence adaptive), and estimates miscalibration of probabilties across all classes in the prediction (not just the top-1 predicted class as in ECE). Assuming that B TA m denotes the m-th thresholded adaptive bin and M is the overall number of bins, TACE is defined as follows:

EQUATION

where objs(B TA , c)

= |B TA | -1 i∈B TA I[y * i = c] and conf(B TA , c) = |B TA | -1 i∈B TA pi,c

. Although TACE does solve several problems of ECE and is useful for measuring calibration of a specific model, it still cannot be used as a reliable criterion for comparing different models. Theory suggests that it is still a biased estimate of true calibration with different bias for each model (Vaicenavicius et al., 2019) . In practice, we find that TACE is sensitive to its two parameters, the number of bins and the threshold, and does not provide a consistent ranking of different models, as shown in Figure 2 .

3.5. CALIBRATED LOG-LIKELIHOOD AND TEST-TIME CROSS-VALIDATION:

There are two common ways to perform temperature scaling using a validation set when training on datasets that only feature public training and test sets (e.g. CIFARs). The public training set might be divided into a smaller training set and validation set, or the public test set can be split into test and validation parts (Guo et al., 2017; Nixon et al., 2019) . The problem with the first method is that the resulting models cannot be directly compared with all the other models that have been trained on the full training set. The second approach, however, provides an unbiased estimate of metrics such as log-likelihood and Brier score but introduces more variance.

In order to reduce the variance of the second approach, we perform a "test-time cross-validation". We randomly divide the test set into two equal parts, then compute metrics for each half of the test set using the temperature optimized on another half. We repeat this procedure five times and average the results across different random partitions to reduce the variance of the computed metrics.

4. A STUDY OF ENSEMBLING & DEEP ENSEMBLE EQUIVALENT:

Ensembles of deep neural networks have become a de-facto standard for uncertainty estimation and improving the quality of deep learning models (Hansen & Salamon, 1990; Krizhevsky et al., 2009; Lakshminarayanan et al., 2017) . There are two main directions of training ensembles of DNNs: training stochastic computation graphs and obtaining separate snapshots of neural network parameters.

Methods based on the paradigm of stochastic computation graphs introduce some kind of random noise over the weights or activations of deep learning models. When the model is trained, each sample of the noise corresponds to a member of the ensemble. During test time, the predictions are averaged across the noise samples. These methods include (test-time) data augmentation, dropout (Srivastava et al., 2014; Gal & Ghahramani, 2016) , variational inference (Blundell et al., 2015; Kingma et al., 2015; Louizos & Welling, 2017) , batch normalization (Ioffe & Szegedy, 2015; Teye et al., 2018; Atanov et al., 2019) , Laplace approximation (Ritter et al., 2018) and many more.

Snapshot-based methods aim to obtain sets of weights for deep learning models and then to average the predictions across these weights. The weights can be trained independently (e.g. deep ensembles (Lakshminarayanan et al., 2017) ), collected on different stages of a training trajectory (e.g. snapshot ensembles (Huang et al., 2017) and fast geometric ensembles (Garipov et al., 2018) ), or obtained from a sampling process (e.g. MCMC-based methods (Welling & Teh, 2011; Zhang et al., 2019) ). These two paradigms can be combined. Some works suggest construction of ensembles of stochastic computation graphs (Tomczak et al., 2018) , while others make use of the collected snapshots to construct a stochastic computation graph (Wang et al., 2018; Maddox et al., 2019) .

In this paper we consider the following ensembling techniques: deep ensembles (Lakshminarayanan et al., 2017) , snapshot ensembles (SSE by Huang et al. (2017) ), fast geometric ensembling (FGE by Garipov et al. (2018) ), SWA-Gaussian (SWAG by Maddox et al. (2019) ), cyclical SGLD (cSGLD by Zhang et al. (2019) ), variational inference (VI by Blundell et al. (2015) ), K-FAC Laplace approximation (Ritter et al., 2018) , dropout (Srivastava et al., 2014) and test-time data augmentation (Krizhevsky et al., 2009) . These techniques were chosen to cover a diverse set of approaches keeping their predictive performance in mind.

All these techniques can be summarized as distributions q m (ω) over parameters ω of computation graphs, where m stands for the technique. During testing, one can average the predictions across parameters ω ∼ q m (ω) to approximate the predictive distribution

EQUATION

For example, a deep ensemble of S networks can be represented in this form as a mixture of S Dirac's deltas q DE (ω) = 1 S S s=1 δ(ω -ω s ), centered at independently trained snapshots ω s . Similarly, a Bayesian neural network with a fully-factorized Gaussian approximate posterior distribution over the weight matrices and convolutional kernels ω is represented as q VI (ω) = N (ω | µ, diag(σ 2 )), µ and σ 2 being the optimal variational means and variances respectively.

If one considers data augmentation as a part of the computational graph, it can be parameterized by the coordinates of the random crop and the flag for whether to flip the image horizontally or not. Sampling from the corresponding q aug (ω) would generate different ways to augment the data during inference. However, as data augmentation is present by default during the training of all othe A deep ensemble equivalent score (DEE) of a model is equal to the minimum size of a deep ensemble (an ensemble of independently train networks) that achieves the same performance as the model under consideration. The score is measured in the number of models (higher is better). The area between average lower and upper bounds of DEE is shaded. The plot demonstrates that all of the ensembling techniques are far less efficient than deep ensembles during inference and fail to produce the same level of performance as deep ensembles. The comparison that is normalized on training time is presented in Appendix A.

mentioned ensembling techniques, it is suitable to study it in combination with these methods and not as a separate ensembling technique. We perform such an evaluation in Section 4.3.

Typically, the approximation (equation 3) requires K independent forward passes through a neural network, making the test-time budget directly comparable across all methods.

4.1. DEEP ENSEMBLE EQUIVALENT:

Most ensembling techniques under consideration are either bounded to a single mode, or provide positively correlated samples. Deep ensemble, on the other hand, is a simple technique that provides independent samples from different modes of the loss landscape, which, intuitively, should result in a better ensemble. Therefore deep ensembles can be considered a strong baseline for the performance of other ensembling techniques given a fixed test-time computation budget.

Comparing the performance of ensembling techniques is, however, a challenging problem. Different models on different datasets achieve different values of metrics; their dependence on the number of samples is non-trivial, and varies depending on a specific model and dataset. Values of the metrics are thus lacking in interpretability as the gain in performance has to be compared against a modeland dataset-specific baseline.

Aiming to introduce perspective and interpretability in our study, we introduce the deep ensemble equivalent score that employs deep ensembles to measure the performance of other ensembling techniques. Specifically, the deep ensemble equivalent score answers the following question:

What size of deep ensemble yields the same performance as a particular ensembling method?

Following the insights from the previous sections, we base the deep ensemble equivalent on the calibrated log-likelihood (CLL). Formally speaking, we define the deep ensemble equivalent (DEE) for an ensembling method m and its upper and lower bounds as follows: 

EQUATION

EQUATION

4.2. EXPERIMENTS:

We compute the deep ensemble equivalent (DEE) of various ensembling techniques for four popular deep architectures: VGG16 (Simonyan & Zisserman, 2014) , PreResNet110/164 (He et al., 2016) , and WideResNet28x10 (Zagoruyko & Komodakis, 2016) on CIFAR-10/100 datasets (Krizhevsky et al., 2009) , and ResNet50 (He et al., 2016) on ImageNet dataset (Russakovsky et al., 2015) . We use PyTorch (Paszke et al., 2017) for implementation of these models, building upon available public implementations. Our implementation closely matches the quality of methods that has been reported in original works. Technical details on training, hyperparameters and implementations can be found in Appendix B. The source code and all computed metrics are available on GitHub1 .

As one can see on Figure 3 , ensembling methods clearly fall into three categories. SSE and cSGLD outperform all other techniques except deep ensembles and enjoy a near-linear scaling of DEE with the number of samples on CIFAR datasets. The investigation of weight-space trajectories of cSGLD and SSE (Huang et al., 2017; Zhang et al., 2019) suggests that these methods can efficiently explore different modes of the loss landscape. In terms of the deep ensemble equivalent, these methods do not saturate unlike other methods that are bound to a single mode. We found SSE to still saturate on ImageNet. This is likely due to suboptimal hyperparameters of the cyclic learning rate schedule. More verbose results are presented in Figures 11-13 and in Table 5 and Table 8 in Appendix C.

In our experiments SSE typically outperforms cSGLD. This is mostly due to the fact that SSE has a much larger training budget. The cycle lengths and learning rates of SSE and cSGLD are comparable, however, SSE collects one snapshot per cycle while cSGLD collects three snapshots. This makes samples from SSE less correlated with each other while increasing the training budget threefold. Both SSE and cSGLD can be adjusted to obtain a different trade-off between the training budget and the DEE-to-samples ratio. We reused the schedules provided in the original papers (Huang et al., 2017; Zhang et al., 2019) . Being more "local" methods, FGE and SWAG perform worse than SSE and cSGLD, but still significantly outperform "single-snapshot" methods like dropout, K-FAC Laplace approximation and variational inference. We hypothesize that by covering a single mode with a set of snapshots, FGE and SWAG provide a better fit for the local geometry than models trained as stochastic computation graphs. This implies that the performance of FGE and SWAG should be achievable by singlesnapshot methods. However, one might need more elaborate posterior approximations and better inference techniques in order to match the performance of FGE and SWAG by training a stochastic computation graph end-to-end (as opposed to SWAG that constructs a stochastic computation graph post-hoc).

The deep ensemble equivalent curves allow us to notice the common behaviour of different methods, e.g. the relation between deep ensembles, snapshot methods, advanced local methods and singlesnapshot local methods. They also allow us to notice inconsistencies that may indicate a suboptimal choice of hyperparameters. For example, we find that SSE on ImageNet quickly saturates, unlike SSE on CIFAR datasets (Figure 3 ). This may indicate that the hyperparameters used on ImageNet are not good enough for efficient coverage of different modes of the loss landscape. We also find that SSE on WideResNet on CIFAR-10 achieves a DEE score of 100 on approx. 70 samples (Figure 12 ). This may indicate that the members of the deep ensemble for this dataset-architecture pair are underfitted and may benefit from longer training or a different learning rate schedule. Such inconsistencies might be more difficult to spot using plain calibrated log-likelihood plots.

4.3. TEST-TIME DATA AUGMENTATION IMPROVES ENSEMBLES FOR FREE:

Data augmentation is a time-honored technique that is widely used in deep learning, and is a crucial component for training modern DNNs. Test-time data augmentation has been used for a long time to improve the performance of convolutional networks. For example, multi-crop evaluation has been a standard procedure for the ImageNet challenge (Simonyan & Zisserman, 2014; Szegedy et al., 2015; He et al., 2016) . It is, however, often overlooked in the literature on ensembling techniques in deep learning. In this section, we study the effect of test-time data augmentation on the aforementioned ensembling techniques. To keep the test-time computation budget the same, we sample one random augmentation for each member of an ensemble. Figure 5 reports the calibrated log-likelihood on combination of ensembles and test-time data augmentation for ImageNet. Other metrics and results on CIFAR-10/100 datasets are reported in Appendix C. We have used the standard data augmen-tation: random horizontal flips and random padded crops for CIFAR-10/100 datasets, and random horizontal flips and random resized crops for ImageNet (see more details in Appendix B).

Test-time data augmentation (Figure 4 ) consistently improves most ensembling methods, especially on ImageNet, where we see a clear improvement across all methods (Figure 5 and Table 7 ). The performance gain for powerful ensembles (deep ensembles, SSE and cSGLD) on CIFAR datasets is not as dramatic (Figures 14-15 and Table 4 ). This is likely due to the fact that CIFAR images are small, making data augmentation limited, whereas images from ImageNet allow for a large number of diverse samples of augmented images. On the other hand, while the performance of "single-snapshot" methods (e.g. variational inference, K-FAC Laplace and dropout) is improved significantly, they perform approximately as good as an augmented version of a single model across all datasets.w Interestingly, test-time data augmentation on ImageNet improves accuracy but decreases the uncalibrated log-likelihood of deep ensembles (Table 7 in Appendix C). Test-time data augmentation breaks the nearly optimal temperature of deep ensembles and requires temperature scaling to reveal the actual performance of the method, as discussed in Section 3.1. The experiment demonstrates that ensembles may be highly miscalibrated by default while still providing superior predictive performance after calibration.

We would like to note that test-time data augmentation does not always break the calibration of an ensemble, and, on the contrary, test-time data augmentation often improves the calibration of an ensemble. In our experiments, decalibration was caused by the extreme magnitude of a random crop, that is conventionally used for ImageNet augmentation. Using less extreme magnitude of the random crop fixes decalibration, that makes test-time data augmentation a more practical method that provides out-of-the-box calibration. Although, as we demonstrated earlier, there is no guarantee that any ensemble is calibrated out-of-the-box. If we are willing to apply post-hoc calibration, the final performance can be much better with more severe augmentations.

5. DISCUSSION & CONCLUSION:

We have explored the field of in-domain uncertainty estimation and performed an extensive evaluation of modern ensembling techniques. Our main findings can be summarized as follows:

• Temperature scaling is a must even for ensembles. While ensembles generally have better calibration out-of-the-box, they are not calibrated perfectly and can benefit from the procedure. A comparison of log-likelihoods of different ensembling methods without temperature scaling might not provide a fair ranking, especially if some models happen to be miscalibrated. • Many common metrics for measuring in-domain uncertainty are either unreliable (ECE and analogues) or cannot be used to compare different methods (AUC-ROC, AUC-PR for misclassification detection; accuracy-confidence curves). In order to perform a fair comparison of different methods, one needs to be cautious of these pitfalls. Our takeaways are aligned with the take-home messages of Ovadia et al. (2019) that relate to indomain uncertainty estimation. We also observe a stable ordering of different methods in our experiments, and observe that deep ensembles with few members outperform methods based on stochastic computation graphs.

A large number of unreliable metrics inhibits a fair comparison of different methods. Because of this, we urge the community to aim for more reliable benchmarks in the numerous setups of uncertainty estimation. 

A IS "EFFICIENT" TRAINING OF ENSEMBLES EFFICIENT AT ALL?:

Yes! If you care most about training time efficiency. All snapshot based methods (SSE, cSGLD and FGE) on average (Figure 6 ) tend to achieve the same performance as deep ensembles using 2-5× less training epochs on CIFAR datasets.

The gain comes at a cost of inference efficiency and memory consumption. "Efficient" snapshotbased methods require to store much more samples of weights compared to deep ensembles making inference significantly more expensive (up to ×25) given the same predictive performance.

You need to get lucky with hyperparameter choice. While on average "efficient" snapshot-based methods require less training resources, they might be completely inefficient if your hyperparameter choice is sub-optimal (see Figure 7 ). Such hyperparameters as the maximum learning rate, the length of learning rate decay cycle, the snapshot saving schedule can all impact the performance significantly.

This analysis assumes that we use only conventional training procedure. Many models can most likely be trained, stored and executed much more efficiently with methods like super-convergence, stochastic weight averaging, compression, distillation and others. These methods are out of the scope of the paper but are interesting topics for future research. The current choice of hyperparameters may also not be optimal. We reuse the hyperparameters used in the original papers.

B EXPERIMENTAL DETAILS:

Implementations of deep ensembles, SWAG, FGE and K-FAC Laplace are heavily based on the original PyTorch implementations of stochastic weight averaging (SWA)2 and SWAG3 . Implementations of cyclical MCMC and snapshot ensembles are based on the original implementation of cyclical MCMC4 . We hypothesize that the optimal hyperparameters of ensembling methods may vary widely depending on the computational budget and the number of samples in the ensemble. Searching for the optimal values for each configuration is outside the scope of this paper so we stick to the originally proposed hyperparameters whenever possible.

Implied probabilistic model Conventional neural networks for classification are usually trained using the average cross-entropy loss function with weight decay regularization hidden inside an optimizer in a deep learning framework like PyTorch. The underlying optimization problem can be written as follows:

EQUATION

where {(x i , y * i )} N i=1 is the training dataset of N objects x i with corresponding labels y * i , λ is the weight decay scale and p(j | x i , w) denotes the probability that a neural network with parameters w assigns to class j when evaluated on object x i .

The cross-entropy loss defines a likelihood function p(y * | x, w) and weight decay regularization, or L 2 regularization corresponding to a certain Gaussian prior distribution p(w). The whole optimization objective then corresponds to the maximum a posteriori inference in the following probabilistic model:

EQUATION

EQUATION

) log p(w) = -N λ 2 w 2 + const ⇐⇒ p(w) = N w 0, (N λ) -1 I (9)

In order to make the results comparable across all ensembling techniques, we used the same probabilistic model for all methods, choosing fixed weight decay parameters for each architecture. We used the softmax-based likelihood for all models. We also use the fully-factorized zero-mean Gaussian prior distribution with variances σ 2 = (N λ) -1 where the number of objects N and the weight decay scale λ are dictated by the particular datasets and neural architectures as defined in the following paragraph.

Conventional networks:

To train a single network on CIFAR-10/100, we used SGD with batch size of 128, momentum 0.9 and model-specific parameters, i.e. the initial learning rate (lr init ), the weight decay coefficient (wd), and the number of optimization epochs (epoch). Specific hyperparameters are shown in Table 1 . The models were trained with a unified learning rate scheduler that is shown in equation 10. All models have been trained using data augmentation that consists of horizontal flips and a random crop of 32 pixels with a padding of 4 pixels5 . The standard data normalization has also been applied. Weight decays, initial learning rates, and the learning rate scheduler were taken from (Garipov et al., 2018) paper. Compared with hyperparameters of (Garipov et al., 2018) , we increased the number of optimization epochs since we found that all models were underfitted. While the original WideResNet28x10 network includes a number of dropout layers with p = 0.3 and is trained for 200 epochs, we find that the WideResNet28x10 underfits in this setting and requires a longer training. Therefore, we used p = 0 which reduces training time while bearing 

EQUATION

On ImageNet dataset we used ResNet50 with default hyperparameters taken from PyTorch examples6 . Specifically, we used SGD with momentum 0.9, batch size of 256, initial learning rate 0. Dropout Binary dropout (or MC dropout) (Srivastava et al., 2014; Gal & Ghahramani, 2016 ) is one of the most widely known ensembling techniques. It involves putting a multiplicative Bernoulli noise with a parameter p over the activations of either a fully connected layer or a convolutional layer, averaging predictions of the network w.r.t. noise at test-time. Dropout layers were applied to VGG and WideResNet networks on CIFAR-10 and CIFAR-100 datasets. Dropout for VGG was applied to fully connected layers with p = 0.5. Two dropout layers were applied: one before the first fully connected layer and one before the second one. While the original version of VGG for CIFARs (Zagoruyko, 2015) exploits more dropout layers, we observed that any additional dropout layer deteriorates the performance of the model in ether deterministic or stochastic mode. Dropout for WideResNet was applied in accordance with the original paper (Zagoruyko & Komodakis, 2016) with p = 0.3. Dropout usually increases the time needed to achieve convergence. Because of this, WideResNet networks with dropout were trained for 400 epochs instead of 300 epochs for deterministic case, and VGG networks have always been trained with dropout. All the other hyperparameters were the same as in the case of conventional models.

Variational Inference Variational Inference (VI) approximates the true posterior distribution over weights p(w | Data) with a tractable variational approximation q θ (w) by maximizing a so-called variational lower bound L (eq. 11) w.r.t. the parameters of variational approximation θ. We used a fully-factorized Gaussian approximation q(w) and Gaussian prior distribution p(w). Table 2 : Optimal noise scale for K-FAC Laplace for different datasets and architectures. For ResNet50 on ImageNet, the optimal scale found was 2.0 with test-time augmentation and 6.8 without test-time augmentation.

EQUATION

In the case of such a prior, the probabilistic model remains consistent with the conventional training which corresponds to MAP inference in the same probabilistic model. We used variational inference for both convolutional and fully-connected layers where variances of the weights were parameterized by log σ. For fully-connected layers we applied the local reparameterization trick (LRT, (Kingma et al., 2015) ).

While variational inference provides a theoretically grounded way to approximate the true posterior, it tends to underfit deep learning models in practice (Kingma et al., 2015) . The following tricks are applied to deal with it: pre-training (Molchanov et al., 2017) or equivalently annealing of β (Sønderby et al., 2016), and scaling β down (Kingma et al., 2015; Ullrich et al., 2017) .

During pre-training we initialize µ with a snapshot of weights of a pre-trained conventional model, and initialize log σ with a model-specific constant log σ init . The KL-divergence -except for the term corresponding to the weight decay -is scaled with a model-specific parameter β. The weight decay term is implemented as a part of the optimizer. We used a fact that the KL-divergence between two Gaussian distributions can be rewritten as two terms, one of which is equivalent to the weight decay regularization.

On CIFAR-10 and CIFAR-100 we used β equal to 1e-4 for VGG, ResNet100 and ResNet164 networks, and β equal to 1e-5 for WideResNet. Log-variance log σ init was initialized with -5 for all models. Parameters µ were optimized with SGD in the same manner as in the case of conventional networks except that the initial learning rate lr init was set to 1e-3. We used a separate Adam optimizer with a constant learning rate of 1e-3 to optimize log-variances of the weights log σ. Pretraining was done for 300 epochs, and after that the remaining part of training was done for 100 epochs. On ImageNet we used β = 1e-3, lr init = 0.01, log σ init = -6, and trained the model for 45 epochs after pre-training.

K-FAC Laplace:

The Laplace approximation uses the curvature information of the appropriately scaled loss function to construct a Gaussian approximation to the posterior distribution. Ideally, one would use the Hessian of the loss function as a covariance matrix and use the maximum a posteriori estimate w M AP as a mean of the Gaussian approximation:

EQUATION

EQUATION

In order to keep the method scalable, we use the Fisher Information Matrix as an approximation to the true Hessian (Martens & Grosse, 2015) . For K-FAC Laplace, we use the whole dataset to construct an approximation to the empirical Fisher Information Matrix, and use the π correction to reduce the bias (Ritter et al., 2018; Martens & Grosse, 2015) . Following (Ritter et al., 2018) , we find the optimal noise scale for K-FAC Laplace on a held-out validation set by averaging across five random initializations. We then reuse this scale for networks trained without a hold-out validation set. We report the optimal values of scales in Table 2 . Note that the optimal scale is different depending on whether we use test-time data augmentation or not. Since the data augmentation also introduces some amount of additional noise, the optimal noise scale for K-FAC Laplace with data augmentation is lower.

Snapshot ensembles Snapshot ensembles (SSE) (Huang et al., 2017) is a simple example of an array of methods which collect samples from a training trajectory of a network in weight space to construct an ensemble. Samples are collected in a cyclical manner: during each cycle the learning rate goes from a large value to near-zero and snapshot of weights of the network is taken at the end of the cycle. SSE uses SGD with a cosine learning schedule defined as follows:

EQUATION

where α 0 is the initial learning rate, T is the total number of training iterations and M is the number of cycles.

For all datasets and models hyperparameters from the original SSE paper are reused. For CIFAR-10/100 length of the cycle is 40 epochs, maximum learning rate is 0.2, batch size is 64. On ResNet50 and ImageNet length of the cycle is 45 epochs, maximum learning rate is 0.1, batch size is 256.

Cyclical SGLD Cyclical Stochastic Gradient Langevin Dynamics (cSGLD) (Zhang et al., 2019) is a state-of-the-art ensembling method for deep neural networks pertaining to stochastic Markov Chain Monte Carlo family of methods. It bears similarity to SSE, e.g. it employs SGD with a learning rate schedule described with the equation 16 and training is cyclical in the same manner. Its main differences from SSE are the introduction of gradient noise and the capturing of several snapshots per cycle, both of which can aid in sampling from posterior distribution over neural network weights efficiently.

Some parameters from the original paper are reused: length of cycle is 50 epochs, maximum learning rate is 0.5, batch size is 64. Number of epochs with gradient noise per cycle is 3 epochs. This was found to yield much higher predictive performance and better uncertainty estimation compared to the original paper choice of 10 epochs for CIFAR-10 and 3 epochs for CIFAR-100.

Finally, the results of cyclical Stochastic Gradient Hamiltonian Monte Carlo (SGHMC), (Zhang et al., 2019) which reportedly has marginally better performance compared with cyclical SGLD, could not be reproduced with any value of SGD momentum term. Because of this, we only include cyclical SGLD in our benchmark.

FGE Fast Geometric Ensembling (FGE) is an ensembling method that is similar to SSE in that it collects weight samples from a training trajectory to construct an ensemble. Its main differences from SSE are pretraining, short cycle length and a piecewise-linear learning rate schedule:

EQUATION

)

Hyperparameters of the original implementation of FGE are reused. Model pretraining is done with SGD for 160 epochs according to the standard learning rate schedule described in equation 10 with maximum learning rates from Table 1 . After that, a desired number of FGE cycles is done with one snapshot per cycle collected. For VGG the learning rate is changed with parameters α 1 = 1e -2, α 2 = 5e -4 in a cycle with cycle length of 2 epochs. For other networks the learning rate is changed with parameters α 1 = 5e -2, α 2 = 5e -4 with cycle length of 4 epochs. Batch size is 128.

SWAG SWA-Gaussian (SWAG) (Maddox et al., 2019) is an ensembling method based on fitting a Gaussian distribution to model weights on the SGD training trajectory and sampling from this distribution to construct an ensemble.

Like FGE, SWAG has a pretraining stage which is done according to the standard learning rate schedule described in equation 10 with maximum learning rates from Table 1 . After that, training continues with a constant learning rate of 1e-2 for all models except for PreResNet110 and PreRes-Net164 on CIFAR-100 where it continues with a constant learning rate of 5e-2 in accordance with the original paper. Rank of the empirical covariance matrix which is used for estimation of Gaussian distribution parameters is set to be 20. 

:

Source code: https://github.com/bayesgroup/pytorch-ensembles

https://github.com/timgaripov/swa

https://github.com/wjmaddox/swa_gaussian

https://github.com/ruqizhang/csgmcmc/tree/master/experiments

Compose([RandomHorizontalFlip(), RandomCrop(32, padding=4)])

https://github.com/pytorch/examples/tree/ee964a2/imagenet

Compose([RandomResizedCrop(224), RandomHorizontalFlip()])

https://pytorch.org/docs/stable/torchvision/models.html

Figures and Tables:

Figure 2: Thresholded adaptive calibration error (TACE) is highly sensitive to the threshold and the number of bins. It does not provide a consistent ranking of different ensembling techniques.Here TACE is reported for VGG16BN model on CIFAR-100 dataset and is evaluated at the optimal temperature.

Figure 3: The deep ensemble equivalent score (DEE) for different numbers of samples on CIFAR-10, CIFAR-100, and ImageNet datasets averaged across different deep convolutional architectures.A deep ensemble equivalent score (DEE) of a model is equal to the minimum size of a deep ensemble (an ensemble of independently train networks) that achieves the same performance as the model under consideration. The score is measured in the number of models (higher is better). The area between average lower and upper bounds of DEE is shaded. The plot demonstrates that all of the ensembling techniques are far less efficient than deep ensembles during inference and fail to produce the same level of performance as deep ensembles. The comparison that is normalized on training time is presented in Appendix A.

Figure4: An illustration of test-time augmentation (TTA) for an ensemble. We apply every member of an ensemble to a separate random augmentation of an image. The predictions of all members are averaged to produce a final prediction of an ensemble. In our experiments, TTA leads to a significant boost of the performance for most of the ensembling techniques on ImageNet with a sufficient computational budget (see Figure5).

Figure 6: The mean cost of training of an ensemble vs. the mean deep ensemble equivalent score. Each marker on the plot denotes one snapshot of weights.

Figure 8: The average log-likelihood vs the Brier score on a test dataset for different ensemble methods on CIFAR-10 (a) and CIFAR-10 (b) and ImageNet (c) datasets. While not being equivalent, these metrics demonstrate a strong linear correlation. The correlation coefficient is denoted as ρ.

Figure 9: Log-likelihood vs accuracy for different ensembles before (a, c, e) and after (b, d, f) calibration. Both plain log-likelihood and especially calibrated log-likelihood are highly correlated with accuracy.

Figure10: A side-by-side comparison of log-likelihood and calibrated log-likelihood on CIFAR-10 (a) and CIFAR-100 (b) datasets. On CIFAR-10 the performance of one network becomes close to dropout, variational inference (vi), and K-FAC Laplace approximation (kfacl) after calibration on all models except VGG. On CIFAR-100 deep ensembles move to the first position in the ranking after calibration on WideResNet and VGG. See Section 3.1 for details on the calibrated log-likelihood.

Many popular ensembling techniques require dozens of samples for test-time averaging, yet are essentially equivalent to a handful of independently trained models. Deep ensembles dominate other methods given a fixed test-time budget. The results indicate, in particular, that exploration of different modes in the loss landscape is crucial for good predictive performance. • Methods that are stuck in a single mode are unable to compete with methods that are designed to explore different modes of the loss landscape. Would more elaborate posterior approximations and better inference techniques shorten this gap? • Test-time data augmentation is a surprisingly strong baseline for in-domain uncertainty estimation. It can significantly improve other methods without increasing training time or model size since data augmentation is usually already present during training.

Table content: <table/>

Hyperparameters of models trained on CIFARs for single-model evaluation no significant effect on final model performance in our experiments.

Table content: <table><tr><td>Model</td><td colspan="2">lr init epochs wd</td></tr><tr><td>VGG</td><td>0.05 400</td><td>5e-4</td></tr><tr><td>PreResNet110</td><td>0.1 300</td><td>3e-4</td></tr><tr><td>PreResNet164</td><td>0.1 300</td><td>3e-4</td></tr><tr><td colspan="2">WideResNet28x10 0.1 300</td><td>5e-4</td></tr></table>

1, weight decay 1e -4. Training included data augmentation 7 (scaling, random crops of size 224 × 224, horizontal flips), normalization and learning rate scheduler lr = lr init • 0.1 epoch//30 where // denotes integer division. We only deviated from standard parameters by increasing the number of training epochs from 90 to 130. Or models achieve top-1 error of 23.81 ± 0.15 that closely matches the accuracy of ResNet50 provided by PyTorch which is 23.85 8 . Training of one model on a single NVIDIA Tesla V100 GPU takes approximately 5.5 days.

Table content: <table/>

Dropout 5.81 vs 5.52 ↓ 5.82 vs 5.34 ↓ 5.79 vs 5.21 ↓ 0.225 vs 0.187 ↓ 0.224 vs 0.177 ↓ 0.223 vs 0.167 ↓ SWA-Gaussian 5.66 vs 5.65 ≈ 5.49 vs 5.36 ≈ 5.25 vs 5.08 ↓ 0.182 vs 0.180 ≈ 0.171 vs 0.166 ↓ 0.160 vs 0.152 ↓ Cyclic SGLD 6.56 vs 6.07 ↓ 5.71 vs 5.47 ↓ 4.84 vs 4.88 ≈ 0.196 vs 0.186 ↓ 0.176 vs 0.169 ↓ 0.147 vs 0.147 ≈ Fast Geometric Ens. 5.95 vs 5.64 ↓ 5.69 vs 5.36 ↓ 5.10 vs 4.98 ≈ 0.187 vs 0.177 ↓ 0.178 vs 0.167 ↓ 0.155 vs 0.150 ↓ VGG16 Deep Ensembles 4.79 vs 4.90 ≈ 4.57 vs 4.73 ↑ 4.39 vs 4.55 ↑ 0.158 vs 0.162 ↑ 0.148 vs 0.152 ↑ 0.134 vs 0.136 ↑ CIFAR-10 Single model 5.83 vs 5.55 ↓ 5.83 vs 5.35 ↓ 5.83 vs 5.19 ↓ 0.223 vs 0.183 ↓ 0.223 vs 0.174 ↓ 0.223 vs 0.166 ↓ Variational Inf. (FFG) 5.63 vs 5.43 ↓ 5.50 vs 5.25 ↓ 5.46 vs 5.07 ↓ 0.192 vs 0.182 ↓ 0.184 vs 0.169 ↓ 0.175 vs 0.154 ↓ KFAC-Laplace 5.82 vs 5.49 ↓ 5.82 vs 5.32 ↓ 5.80 vs 5.17 ↓ 0.203 vs 0.177 ↓ 0.201 vs 0.171 ↓ 0.200 vs 0.164 ↓ Snapshot Ensembles 5.52 vs 5.61 ≈ 5.00 vs 5.03 ≈ 4.54 vs 4.64 ↑ 0.176 vs 0.178 ↑ 0.160 vs 0.160 ≈ 0.137 vs 0.137 ≈ SWA-Gaussian 4.56 vs 4.47 ≈ 4.46 vs 4.34 ↓ 4.34 vs 4.16 ↓ 0.143 vs 0.142 ≈ 0.139 vs 0.135 ↓ 0.131 vs 0.125 ↓ Cyclic SGLD 5.32 vs 4.88 ↓ 4.55 vs 4.35 ↓ 3.83 vs 3.59 ↓ 0.156 vs 0.148 ↓ 0.138 vs 0.131 ↓ 0.115 vs 0.111 ↓ Fast Geometric Ens. 4.83 vs 4.59 ↓ 4.73 vs 4.44 ↓ 4.28 vs 4.14 ↓ 0.141 vs 0.138 ≈ 0.137 vs 0.132 ↓ 0.126 vs 0.121 ↓ ResNet110 Deep Ensembles 3.77 vs 3.70 ↓ 3.63 vs 3.58 ≈ 3.53 vs 3.45 ↓ 0.117 vs 0.118 ≈ 0.112 vs 0.112 ≈ 0.106 vs 0.105 ↓ CIFAR-10 Single model 4.69 vs 4.32 ↓ 4.69 vs 4.23 ↓ 4.69 vs 4.10 ↓ 0.150 vs 0.137 ↓ 0.150 vs 0.134 ↓ 0.150 vs 0.131 ↓ Variational Inf. (FFG) 4.91 vs 4.66 ↓ 4.72 vs 4.43 ↓ 4.60 vs 4.25 ↓ 0.149 vs 0.145 ↓ 0.144 vs 0.138 ↓ 0.140 vs 0.130 ↓ KFAC-Laplace 5.14 vs 4.43 ↓ 4.90 vs 4.27 ↓ 4.78 vs 4.17 ↓ 0.160 vs 0.139 ↓ 0.153 vs 0.134 ↓ 0.147 vs 0.130 ↓ Snapshot Ensembles 4.85 vs 4.78 ≈ 4.16 vs 4.18 ≈ 3.52 vs 3.48 ≈ 0.153 vs 0.150 ↓ 0.132 vs 0.130 ≈ 0.107 vs 0.106 ↓ SWA-Gaussian 4.21 vs 4.17 ≈ 4.21 vs 4.04 ≈ 4.02 vs 3.84 ↓ 0.130 vs 0.130 ≈ 0.128 vs 0.126 ≈ 0.121 vs 0.116 ↓ Cyclic SGLD 4.97 vs 4.67 ↓ 4.30 vs 4.01 ↓ 3.66 vs 3.48 ↓ 0.149 vs 0.141 ↓ 0.131 vs 0.125 ↓ 0.110 vs 0.107 ↓ Fast Geometric Ens. 4.49 vs 4.29 ↓ 4.36 vs 4.15 ↓ 4.09 vs 3.85 ↓ 0.134 vs 0.131 ↓ 0.130 vs 0.126 ↓ 0.119 vs 0.115 ↓ ResNet164 Deep Ensembles 3.51 vs 3.58 ↑ 3.50 vs 3.46 ≈ 3.34 vs 3.30 ↓ 0.113 vs 0.114 ↑ 0.107 vs 0.107 ≈ 0.100 vs 0.101 ↑ CIFAR-10 Single model 4.52 vs 4.15 ↓ 4.52 vs 4.08 ↓ 4.52 vs 3.98 ↓ 0.144 vs 0.131 ↓ 0.144 vs 0.128 ↓ 0.144 vs 0.126 ↓ Variational Inf. (FFG) 4.78 vs 4.41 ↓ 4.66 vs 4.22 ↓ 4.55 vs 4.05 ↓ 0.151 vs 0.142 ↓ 0.146 vs 0.135 ↓ 0.141 vs 0.128 ↓ KFAC-Laplace 4.77 vs 4.26 ↓ 4.65 vs 4.21 ↓ 4.60 vs 4.08 ↓ 0.151 vs 0.135 ↓ 0.146 vs 0.132 ↓ 0.142 vs 0.127 ↓ Snapshot Ensembles 4.50 vs 4.42 ↓ 3.89 vs 3.87 ≈ 3.50 vs 3.35 ↓ 0.144 vs 0.142 ↓ 0.124 vs 0.123 ≈ 0.104 vs 0.104 ≈ Dropout 3.70 vs 3.62 ≈ 3.63 vs 3.52 ↓ 3.64 vs 3.44 ↓ 0.120 vs 0.117 ↓ 0.119 vs 0.114 ↓ 0.117 vs 0.111 ↓ SWA-Gaussian 3.53 vs 3.59 ≈ 3.34 vs 3.34 ≈ 3.28 vs 3.24 ≈ 0.111 vs 0.114 ≈ 0.105 vs 0.107 ↑ 0.101 vs 0.102 ≈ Cyclic SGLD 4.09 vs 3.95 ≈ 3.63 vs 3.58 ≈ 3.19 vs 3.22 ≈ 0.128 vs 0.125 ↓ 0.114 vs 0.112 ↓ 0.099 vs 0.100 ↑ Fast Geometric Ens. 3.95 vs 3.90 ≈ 3.77 vs 3.64 ↓ 3.34 vs 3.27 ↓ 0.120 vs 0.120 ≈ 0.113 vs 0.113 ≈ 0.102 vs 0.102 ≈ WideResNet Deep Ensembles 3.11 vs 3.21 ↑ 3.01 vs 3.02 ≈ 2.83 vs 2.91 ↑ 0.097 vs 0.103 ↑ 0.095 vs 0.098 ↑ 0.090 vs 0.094 ↑ CIFAR-10 Single model 3.70 vs 3.53 ↓ 3.70 vs 3.45 ↓ 3.70 vs 3.40 ↓ 0.124 vs 0.117 ↓ 0.125 vs 0.114 ↓ 0.124 vs 0.113 ↓ Variational Inf. (FFG) 4.15 vs 4.05 ≈ 3.94 vs 3.80 ↓ 3.64 vs 3.63 ≈ 0.134 vs 0.136 ≈ 0.127 vs 0.126 ↓ 0.117 vs 0.116 ↓ KFAC-Laplace 3.90 vs 3.63 ↓ 3.88 vs 3.58 ↓ 3.83 vs 3.50 ↓ 0.124 vs 0.115 ↓ 0.122 vs 0.113 ↓ 0.120 vs 0.111 ↓ Snapshot Ensembles 3.68 vs 3.74 ≈ 3.33 vs 3.35 ≈ 2.89 vs 2.90 ≈ 0.119 vs 0.122 ↑ 0.105 vs 0.107 ↑ 0.090 vs 0.093 ↑

Table content: <table><tr><td/><td/><td/><td/><td/><td/><td colspan="2">CIFAR100</td></tr><tr><td/><td/><td/><td/><td colspan="3">Error (%) central crop (ensemble size 10)</td><td colspan="2">Negative calibrated log-likelihood central crop (ensemble size 50 )</td></tr><tr><td colspan="2">Model</td><td colspan="2">Method</td><td colspan="3">1 test-time aug. (ensemble size 10) 5 10</td><td colspan="2">100 test-time aug. (ensemble size 50) 1 5</td><td>10</td><td>100</td></tr><tr><td/><td/><td/><td/><td/><td colspan="2">Error (%)</td><td/><td>Negative calibrated log-likelihood</td></tr><tr><td colspan="2">Model 20 21 22 23 24 25 26 Top-1 error (%)</td><td colspan="2">Method</td><td>5 VGG16</td><td>10</td><td colspan="2">100 0.75 0.80 0.85 0.90 0.95 1.00 1.05 Neg. calibrated log-likelihood 1.10</td><td>5</td><td>10 VGG16</td><td>100</td></tr><tr><td/><td/><td>Single model</td><td>K-FAC-L FFG VI</td><td colspan="2">SWAG FGE cSGLD SSE</td><td>DE</td><td>Single model</td><td>K-FAC-L FFG VI</td><td>SWAG FGE cSGLD SSE</td><td>DE</td></tr><tr><td/><td/><td/><td/><td>ResNet110</td><td/><td/><td/><td>ResNet110</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>0.85</td></tr><tr><td>Top-1 error (%)</td><td>18 19 20 21 22</td><td/><td/><td/><td/><td>Neg. calibrated log-likelihood</td><td>0.65 0.70 0.75 0.80</td></tr><tr><td/><td>17</td><td/><td/><td/><td/><td/><td>0.60</td></tr><tr><td/><td/><td>Single model</td><td>K-FAC-L FFG VI</td><td colspan="2">SWAG FGE cSGLD SSE</td><td>DE</td><td>Single model</td><td>K-FAC-L FFG VI</td><td>SWAG FGE cSGLD SSE</td><td>DE</td></tr><tr><td/><td/><td/><td/><td>ResNet164</td><td/><td/><td/><td>ResNet164</td></tr><tr><td>Top-1 (%)</td><td>17 18 19 20 21</td><td/><td/><td/><td/><td>Neg. calibrated log-likelihood</td><td>0.60 0.65 0.70 0.75 0.80</td></tr><tr><td/><td/><td>Single model</td><td>K-FAC-L FFG VI</td><td colspan="2">SWAG FGE cSGLD SSE</td><td>DE</td><td>Single model</td><td>K-FAC-L FFG VI</td><td>SWAG FGE cSGLD SSE</td><td>DE</td></tr><tr><td/><td/><td/><td/><td>WideResNet</td><td/><td/><td/><td>WideResNet</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>0.80</td></tr><tr><td>Top-1 error (%)</td><td>16 17 18 19</td><td/><td/><td/><td/><td>Neg. calibrated log-likelihood</td><td>0.60 0.65 0.70 0.75</td></tr><tr><td/><td/><td>Single model</td><td>K-FAC-L FFG VI</td><td colspan="2">SWAG FGE cSGLD SSE</td><td>DE</td><td>Single model</td><td>K-FAC-L FFG VI</td><td>SWAG FGE cSGLD SSE</td><td>DE</td></tr></table>

Peer review (iclr2020):
The paper provide an extensive review of current advances in uncertainty estimation in neural networks with the analysis of drawbacks of currently used uncertainty metrics and comparison on scale the recent method to estimate uncertainty. The paper covers a lot of uncertainty metrics and a wide range of methods. The paper focuses on in-domain uncertainty estimation complementing the recent similar review on out-of-domain uncertainty estimation. It seems that the paper provides the analysis missing in the current literature. Whereas as mentioned Yukun Ding in a public comment there is a related work on identifying issues with popular uncertainty metrics, the mentioned paper is missing the through comparison of the methods for estimating uncertainty. Such kind of thorough analysis (especially performed on scale on large datasets) and comparison is of obvious interest to the community as well as objective comparison of the current state-of-the-art. The paper is clearly written and easy to follow. Based on this, I believe this is a strong technical paper and it should be accepted. However, the analysis in the paper is not overwhelmingly exhaustive. Some of the arguments on that are listed below. Below is the list of comments/thoughts for potential improvement of the paper: 1. “In this case, a model is expected to provide correct probability estimates:” – may be not the best choice of words, because for out of domain uncertainty estimation we still expect a model to provide correct probability estimates 2. The first paragraph on page 3 seems to better fit in Section 2, for example, on the very beginning of Section 2. 3. “Comparison of the log-likelihood should only be performed at the optimal temperature.” and others alike – personally, I do not support this kind of formatting for a scientific paper 4. “can produce an arbitrary ranking of different methods. <…> Empirically,” – in the current form it seems that the first statement is somehow theoretically justified and then additionally it is confirmed empirically in this paper. I believe that the authors use empirical observation itself as the justification of the first statement, if that the case it should be reworded here. For example, “can produce an arbitrary ranking of different methods as we show below/ as we show empirically. We demonstrate that the overall …” If my belief is incorrect and there are other grounds that justify the first statement that it is required a reference after this statement. 5. Italic and non-italic LL usage is unclear 6. Having “Brier score” emphasised as a paragraph, it seems that there should be a paragraph log-likelihood as well 7. “In that case, both objects and targets of induced binary classification problems remain fixed for all models” – do the authors consider in this case all out-of-domain objects as having a positive class and all in-domain objects as having a negative class? Because the models are still going to make individual misclassification mistakes 8. Figure 2 – legend occupies too much space of the plot occluding almost a third part of the plot. Maybe taking the legend out of the plot to the right and squeezing the plot to make a room for the legend would be a better solution 9. In eq. (4) and (5) subscript DE is not defined 10. “SSE and cSGLD outperform all other techniques except deep ensembles” –cSGLD was not applied on ImageNet, therefore this statement is a bit misleading 11. Colour of SWAG in Figure 3 is not very clear. Only excluding other colours I can determine which line is SWAG. Similar to cSGLD, is seems that SWAG was not applied on ImageNet. Why is that if that is the case? And it should be clearly stated at least in experimental setup in Supplementary For colours in general, lines in legends are very thin and it is difficult to assess their colour. I appreciate the authors compare a lot of methods and therefore have to use a lot of colours, but it is quite difficult to assess them even on screen not to mention if the paper is printed out. Could the authors please use thicker lines in legends at least? 12. “Being more “local” methods” – without any context in the main paper this referral to “local” methods is unclear. Also it is good to add a reference to Appendix review of the considered methods in the main text. 13. Missing details of what kind of augmentation is used in Section 4.3. Is it the same as training augmentation specified in Supplementary? It would require a reference to Supplementary 14. “(Figure 1, Table REF)” – missing number for Table 15. “Our experiments demonstrate that ensembles may be severely miscalibrated by default while still providing superior predictive performance after calibration.” – unclear which experiments demonstrate this and superior in comparison to what 16. The issues of uncalibrated log-likelihood and TACE are clearly shown in the paper, whereas the issues with misclassification detection are only verbally discussed. An illustrative example, at least a toy thought example, could really improve the paper here 17. The chosen main performance metric is not very convincingly motivated. It is clear why it is based on calibrated log-likelihood, but it is not very convincing why one cannot just used calibrated log-likelihood as a performance metric, why one should base the metric on deep ensembles instead. Also from the long-term perspective, if the community comes up with methods clearly outperforming deep ensembles, the metric would need to be based on one of these new methods 18. There is an indirect uncertainty metric that is not mentioned in the paper – uncertainty used in active learning (see, e.g., Hernández-Lobato and Adams, 2015. Probabilistic backpropagation for scalable learning of Bayesian neural networks) 19. Figures 4 and 5 are too small 20. “the original PyTorch implementations of SWA” – SWA is not considered in the paper 21. “hidden inside an optimizer … The actual underlying optimization problem” – it seems that the ICLR audience should be familiar with “actual optimization problems” rather than using blindly the optimizer. It is always good to explicitly write down an equation that is used in a paper, but this wording seems a bit off for ICLR 22. “hat{p}(y^?_i = j | x_i, w) denotes the probability that a neural network with parameters w assigns to class j when evaluated on object x_i” – it should be hat{p}(y_i = j | x_i, w), y^*_i is observed 23. a. Why was dropout applied only for limited number of architectures and not applied on ImageNet at all? b. Why wasn’t cSGLD applied on ImageNet 24. “On CIFAR-10/100 parameters from the original paper are reused” – it is better to repeat the reference here Minor: 1. Font size in eq. (10) should be the same as the rest of the paper 2. “Or models achived top-1 error of”: “Or” - ?, “achived” -> achieved 3. “for a 45 epoch form a per-trained model”: “form” -> “from”?