Uncertainty for deep image classifiers on out of distribution data. 
Authors: Tiago Salvador
Alexander Iannantuono
Adam M Oberman
Venue:  (2021)
Keywords: uncertainty
confidence
out of distribution
outlier exposure
classification

Abstract: In addition to  achieving high accuracy, in many applications, it is important to  estimate the probability that a model prediction is correct. Predictive uncertainty is particularly important on out of distribution (OOD) data where accuracy degrades.  However, models are typically overconfident, and model calibration on OOD data remains a challenge. In this paper we propose a simple post hoc calibration method that significantly  improves on benchmark results [Ovadia et al 2019] on a wide range of corrupted data.  Our method uses outlier exposure to properly calibrate the model probabilities.

Under review as a conference paper at ICLR 2021
UNCERTAINTY FOR DEEP IMAGE CLASSIFIERS ON OUT- OF-DISTRIBUTION DATA.
A Anonymous authors Paper under double-bling review
ABSTRACT
In addition to achieving high accuracy, in many applications, it is important estimate the probability that a model prediction correct. Predictive uncertainty is particularly important out-of-distribution (OOD) data where accuracy de- grades. However, models are typically overconfident, and model calibration on OOD data remains challenge. In this paper we propose simple post hoc cal- ibration method that significantly improves on benchmark results (Ovadia et al.] (2019) on wide range of corrupted data. Our method uses outlier exposure properly calibrate the model probabilities.
PREDICTIVE UNCERTAINTY
When machine learning model makes a prediction, we want to know how confident (or uncertain)
we should be about the result. Uncertainty estimates are useful for both in distribution and out- of-distribution (OOD) data. Predictive uncertainty addresses this challenge by endowing model predictions with estimates of class membership probabilities. The baseline method for predictive
uncertainty is to simply use the softmax probabilities the model, y ysoltmas (x) softmaxx (x)),
as surrogate for class membership probabilities (Hendrycks & Gimpel 2017). Here f(xx) denotes
the model outputs. Other approaches include temperature scaling (Guo et al. (017), dropout (al &
Ghahramani 2016; Srivastava et al. 2014), and model ensembles (Lakshminarayanan et al. 2017
as well as Stochastic Variational Bayesian Inference (SVBI) for deep learning (Blundell et al. 2015
Graves 2011 Louizos & Welling 2016 2017 Wen al. 2018), among others.
All methods suffer from some degree of calibration error, which the difference between pre- dicted eтror rates and actual error rates, measured by collecting data into bins based Pmax max; prootmax bins. The standard measurement of calibration error is the expected calibration er-
(ECE) Guo al. (2017), although other measures have been used (see Nguyen al. (2015). Hendrycks & Gimpel (017))) including the Brier score (DeGroot & Fienberg 1983), which is also used in Ovadia al. (2019).
1.1 OUR RESULTS
We are interested in image classification problems, in particular the CIFAR-10 and Imagenet 2012 datasets, in the setting of distribution covariate shift, where the data has been corrupted by known transformation of unknown intensity. These corruptions are described in Hendrycks & Di- etterich (2019). Our starting point is the work of Ovadia al. (2019) which offers large-scale benchmark of existing state-of-the-art methods for evaluating uncertainty on classification problems under dataset shift by providing the softmax model outputs. One of main take-aways from the work Ovadia et al. (201)) is that, unsurprisingly, the quality of the uncertainty predictions deteriorates significantly along with the dataset shift. In order to be able to calibrate for different intensity levels of unknown corruptions, we make use of surrogate calibration sets, which are corruptions of the data by a different (known) corruption. Then, when given an image (or sample of images), we first estimate the corruption level, and then recalibrate the model probabilities based the surrogate representative calibration set. The latter step is done with a simple statistical calibration step, con- estimate the
verting the model outputs into calibrated uncertainty estimates. Surprisingly corruption level just using the model outputs. We focus on the probability of correct classification, using the Pmax values.

---

Under review as a conference paper at ICLR 2021
(a) ImageNet
Vanilla
Temp Scaling
Ensemble
Dropout
LL Dropout
LL SVI
Single Image
Mult Image
0.05
Compuptin Intersiiy
(b)
CIFAR-10
Vanilla
Temp Sceling
Ensemble
Drepout
Dropout
SVI
LL SVI
0.30
0.25
0.20
Ovadia Singte Image Image
0.15
0.05
0.2.3.5
012349
0
012345 Comupttinn intersity
012349 012345
0.2.2
Figure I: Comparison of the benchmark implementation (Ovadia et al. 2019). versus our single and multiple image methods. Mean Expected Calibration Error (ECE) across different corruptions types, for fixed corruption intensity going from 0 to 5. Each box represents different uncertainty method. levels of intensity with the greatest improvement
The ECE decreases across almost all methods at the higher intensities. See Tables and 2 in the Appendix for numerical comparisons.
In Figure we compare each benchmark uncertainty method with our two methods. The single image method determines the appropriate calibration using only single image. The multiple image method uses a sample of images (all drawn from the same corruption level and type) to choose the calibration set. More images allow for a better choice of the calibration set, further reducing the calibration етгоr. As shown in the figure, the ECE decreases across almost all methods and levels of intensity with the greatest improvement the higher intensities. The Brier scores give similar results (see Figure 5) below). We also reproduce figure in Ovadia et al. (2019) which gives a whisker plot of the distribution the values.
Our results are based on the fact that data distribution shift typically leads to overconfident models (Nguyen et al. 2015): the Pmax values are above the true probability, and so they themselves are shifted. This allows us use the Pmax distribution shift as surrogate for data distribution shift and ultimately significantly reduce the calibration error using a purely statistical approach. In practice, perform the model recalibration for the different corruptions and intensities based simply the Pmax distribution shift, which we detect using surrogate corrupted calibration sets Crucially, the corruption used to generate the surrogate corrupted datasets is left out of the test set. The calibrated probabilities are visualized using histograms in Figure 2 In Figure 3, we can see the Pmax distributions for the chosen surrogate corrupted calibration sets and for specific test corruption.
1.2 OTHER RELATED WORK
Models trained on a given dataset are unlikely to perform as well on a shifted dataset (Hendrycks & Dietterich 2019), Moreover, there are inevitable tradeoff's between accuracy and robustness (Chun et al. 2020). Training models against corruptions can fail to make models robust to new corruptions Vasiljevic et al. 2016 Geirhos et al., 2018). Hendrycks et al. 2019) deals with anomaly detection, the task of distinguishing between anomalous and in-distribution data. They propose an approach called Outlier Exposure (OE) that consists in training anomaly detectors auxiliary dataset
2

---

Under review as a conference paper at ICLR 2021
(a) Imagenet
Ovadia
Single Image
Image
1.0
Calibration
1.0
Calibration
1.0
Calibration
Test
Test
0.8
0.8
0.8
0.6
0.6
0.4
0.2
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
ECE 10.938%
ECE
3.055%
Brier 0,178
Brier
0.165
Brier 0.164
(h) CIFAR-10
Ovadia al
Single Image
Multi Image
1.0
Callbration
1.0
Calibration
1.0
Calibration
0.8
0.8
0.8
0.6
0.6
0.6
0.2
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
ECE 13.492%
ECE = 2.681%
ECE 3.136%
Brier 0,172
Brier 0.148
Brier - 0.149
Figure 2: Visualization of calibration errors for the Vanilla method on corrupted images on ImageNet and CIFAR-10 using the elastic transform corruption with intensity 4. The x-axis corresponds to the Pmax values and the y-axis to the confidence estimates of proriect The blue histogram is the calibration probabilities, the orange is the test probabilities, and the brown is where both overlap. The ECE is large (lower is better) for the Vanilla method at higher corruption levels, due to the probability shift. The Brier score (lower better) is also improved. The gap between the orange and blue curves represents the calibration error. Notice that we used 30 equally sized bins, so in the CIFAR- plot, there are very few values below .4, which why the first bin wide.
of outliers. Similarly in Hendrycks et al. (2020), the authors propose AUGMIX, a method that improves both robustness and uncertainty measures by exposing the model to perturbed images during training. [Shao et al. (2000) propose a confidence calibration method that uses an auxiliary class separate mis-classified samples from correctly classified ones which thus allowing the mis- classified samples to be assigned with a low confidence, [ado et al. (2020) argues that the internal activations of the deep models also suffer from covariate shift in the presence of OOD images. Thus they propose to recompute the batch norm at prediction time using sample of the unlabeled images from the test distribution improving the accuracy and ultimately the calibration. Park al. (2020) and Wang et al. (2020) focus on the more general problem of unsupervised domain adaptation where one assumes assumes to have unlabeled examples from the test distribution which may only share the same classification classes as the train distribution. Park et al. (2020) propose approach based on importance weighting to correct for the covariate shift in the data, together with learning an indistinguishable feature map between training and test distributions. Waan et al.) (2020) extend the temperature scaling method into domain adaption achieving more accurate calibrations with lower bias and variance without introducing any hyperparameters.
3

---

Under review as a conference paper at ICLR 2021
10-
3.00
6:25
3.50
6.75
100
0.03
025
0.50
0.75
1.00
6:38
325
6.50
2775
1.0
3.00
0:3
6.50
0.55
1.0
2.00
6.25
0.50
6.75
0.00 323
675
1.0
1.
3.00 0.25
930
175
1.00
1.02
0.25
5,0
0.75
1.00
0.0
025
055
0.75
1.0
200
625
250
5.5
1.0 edt
6.5 555 6.75 102
ate
025
958
075
110
Actumoy
Acamay
3.84
Accuray
Acammcy
Accunoy: 8.67
Figure 3: Histogram of the Pmax values for the vanilla method on CIFAR-10. Top: calibration sets {0). {0.1], {0.2], {0.3), {0.4), and {0,5} (left to right) with contrast corruption. Bottom: test set with elastic transform corruption with increased intensities ranging from to 5 (left to right) and SVHN dataset (right most picture). The multi image method outputs calibrated probabilities based the surrogate calibration set (0, (0.3)-(0.3))(0)) 30, 4), 4).(0, (0,5), respectively.
BACKGROUND
2.1 CLASSIFICATION AND LABEL PROBABILITIES
Predictive uncertainty seeks estimate the probability that an input belongs to each class,
polass (z) P = 1: x] for each i E y
(1)
Here we write I E X for data, y E E = [L....,K] for labels.
In the benchmark methods, the softmax of the model outputs, p softmax(f (x)) used as a surrogate the class probabilities. In the case of the vanilla and temperature scal- ing methods, f(x) is simply the model outputs. Similarly, for the Ensemble Dropout methods, p panftmmx (x) represents the average of the probability vectors over the multiple models or queries of the model, respectively.
Generally speaking, these softmax values are not an accurate prediction of the class probabilities pk pelase (Domingos & Pazzani 1996). Here focus the correct classification
p correct
P [y [
where the classification of the model is given by f(x) = arg max fz)). Guo et al. (017)) showed that Pmax max, p pooftmax usually overestimates coorrett We can extend our method to top correctness, as well as to other quantities of interest, such as, using different binning methods, based on structured predictors (Kuleshov & Liang 2015).
METHOD
Problem definition We are given model (or ensemble of models) trained given dataset Ptrain. We want to have calibrated error estimates on an unknown (different) data set Prest, which could have different levels of corruption. Since one model cannot be calibrated all of the possible (different) datasets, we want to allow for multiple calibrations, and apply them accordingly.
We study two cases: (i) we have single image drawn from unknown distribution, (ii) we have multiple images, each drawn from the same unknown distribution. In the latter case, we use the full test (we obtain similar results using 100 images).
3.1 CALIBRATING FOR DATASET SHIFT
We choose C' distinct calibration sets generated from shifted distributions p CALL These sets are chosen to have different representative degrees of corruption intensity. Each calibration set leads
4

---

Under review as a conference paper at ICLR 2021
(a) ImageNet
Vedi
0.7
digll
12
(h) CIFAR-10
Miria
Comuptiin
Figure 4: Comparison of the benchmark implementation OOaddia et al. 2019) versus our single and multiple image methods. Expected Calibration Error (ECE) distribution across different corruptions types, for fixed corruption intensity going from O to 5. Each box represents different uncertainty method.
ImageNet
Vanilla
Temp Scaling
Ensemble
Dropout
LL Dropout
LL SVI
5 5015
0.14
Single Image
0.13
Muthi Image
0.12
Compupton Intersity
(b) CIFAR-10
Vanilla
Temp Scaling
Ensemble
Dropout
Dropout
SVI
LL SVI
0.30
0.25
0.20
Dvadi Single Image
0.15
Muti Image
Computio Intersity
Figure 5: Comparison of the benchmark implementation (Oadiia et al. 2019) versus our single and multiple image methods. Mean Brier score across different corruptions types, for fixed corruption intensity going from 0 to S. Each box represents different uncertainty method. See Tables and for numerical comparisons.
to different uncertainty estimate for a given Pmax value. We adaptively choose the calibration set given image (Single Image Method) or test of images (Multiple Image Method).

---

Under review as a conference paper at ICLR 2021
(a) ImageNet
0.22
Temp Scaling
Drspuut
Comuption intonsity
(b) CIFAR-10 10
04
Mirila
0.5
Treemole
Dransl
Arait b
Comuptonn intersity
Figure 6: Comparison of the benchmark implementation (Ovadia et al. 2019) versus our single and multiple image methods. Brier score distribution across different corruptions types, for fixed corruption intensity going from O to 5. Each box represents different uncertainty method. See Tables and in the Appendix for numerical comparisons.
For each calibration dataset SCALJ, we convert the Pmax values that calibration set into the Peorrect values, as follows. Recall that Pmax max; pootimax, i..., the model's probability for the predicted class.
(i) Evaluate the model Pinax values on the calibration set, PCAL. (Pmmax x E SCAL.)). Record the probability density h CATT..( of the Pmax values a histogram, by binning the Pmax using equally spaced bins.
(ii) Define the calibrated model probabilities by
CAL. (x) P (y ((x) Pmmax(:)),
Poorrect
(3)
using ground truth labels on the calibration set SCAL... These probabilities are computed using a histogram in the following way. Partition SCAL. {a1,..., Fmmm into bins B, according its Pmax values. Given an image x with Pmax( E B, approximate Peorrect (x)
Σ
1((x)==((a,)
B2
(x;))
See (Oberman et al. 2020 for more details.
Single Image Method: Given single image drawn from unknown distribution Prests estimate the likelihood that the corruption level of the image corresponds to each of the calibration sets, and then take the corresponding weighted average of the calibrated probilities. Ideally, in the
single image method we would like to obtain di (Pmax) close to one for the calibration set whose Pmax distribution looks closely the Pmax distribution of the test images.
(i) The probability that the Pmax value came from a given calibration set, making the standard assumption that the priori likelihoods of the calibration sets are all equal,
h CALL (Pmax)
qs (Praax)
ΣS=1 CALL (Pmax)
Then the calibrated probability, conditional on each of the calibration sets, given by
tast Peorrect
(x)
Σ
45 (Pmax(())) CAL. (x)

---

Under review as a conference paper at ICLR 2021
Vanilla
Temp Stalnn
Ensembbee
Dropout
LL Dropout
SVI
L1 SWI
of po/10
333 1:0 CB
Figure 7: Confidence of CIFAR-10 (bottom) trained models on entirely OOD data (SVHN dataset Netzer et al.] 2011)). The benchmark method (blue) has highest confidence. The single and multi image method are much less confidence on OOD data.
Multiple Image Method: Given a sample of images Stest {o1,.... Fm drawn from an un- known distribution, Prest, where m 1.
(i) Record the corresponding model Pmax values, plest (Pmax (x) x E Stest], and com- pute the mean jr.
(ii) Compare to the means H1j of PCAL and find the closest mean for the calibration sets, i arg miny 10u - /pyl- Set
test Peorrect (x)
CAL. Peorrect (ar).
We simpler formula for the multiple image method, because with multiple samples, know- ing the mean is sufficient to estimate the correct calibration set.
3.2 PRACTICAL IMPLEMENTATION OF OUR METHOD
In Ovadia al. 2019) the additional methods used include (LL) Approximate Bayesian inference for the parameters of the last layer only (Riquelme et al., 2018). (LL SVI) Mean field stochastic variational inference on the last layer, (LLL Dropout) Dropout only on the activations before the last layer. We refer to Ovadia et al. (2019) for more details on how each method was implemented. All these methods ultimately use the softmax probabilities as a surrogate for the class probabilities. The difference between the methods is how these probabilities are obtained.
The distributional shift on ImageNet used 16 corruption types with corruption intensity on a scale from 0 to 5: various forms of Noise, and Blur as well as Pixelate, Saturate, Brightness, Contrast, Fog and Frost, etc. See Figure S3 in (Ovadia et al. 2019).
We used the published softmax value of each of the methods from the benchmark dataset in Ovadia jet al.) (2019). We selected Contrast as the corruption to use for calibration, removing from the test set. We calibrated our models using equally sized bins, each of following calibration sets: (0). [0.1]. {0.2). {0.3), {0,4), and {0,5). For instance the set 0, corresponds to clean images and respective corruption with intensity level of 1. Heuristically we always want clean images in our calibration set while having different shifted means as a result increasingly corrupted images (see Figure 3) Without the clean images, the single image method would become uncalibrated for in-distribution images as the calibration sets would have a disproportional amount of corrupted images. For both CIFAR-I0 and Imagenet, we select 5000 images for calibration, which are shared across the different calibration sets. This means that for instance the calibration set corresponding {0, 1} contains a total of 10000 images: the selected 5000 clean images and their corrupted counterparts at an intensity level of 1. A more sophisticated combination of calibration sets could lead to improvements.
We reproduced the figures in Ovadia et al. (2019) with small adjustment: we measure the ECE and Brier score just for Peorrect. rather than for p However this made a negligible difference to the values. In addition, ECE can depend on the binning procedures (equally spaced equally sized). Equally sized bins are more effective for calibration since they reduce statistical error. They do however lead to different bin edges different calibration sets, which required combining the bins. This can be done by refinement (which we used here) or simple by one dimensional density estimation (Wasserman 2006 Chapter 6).
7

---

Under review as a conference paper at ICLR 2021
(a) Fashion MNIST
Vanilla
Temp
Ensemble
Drepout
plyMit
630
333
10
C22
68
68
00 82
X6 Cl
(b) Not-MNIST
Vanill
Temp Scaling
Ensemble
Dropoct
Dropout
SVI
L1
of pV/022
Figure 8: Confidence of MINST trained models on entirely OOD data: Pashion-MNIST 2017) and Not-MNIST (Bulatov] 201I). Our proposed methods are significantly less confident on OOD than the benchmark method.
MNIST (Xiao al.,
3.3 DISCUSSION OF RESULTS
Both our single and multi image methods improve the benchmark widely across methods and corruption intensity levels, as shown in Figure I and Tables [ and 2 which report the mean ECE scores for each model and across different corruption types, for fixed corruption intensity going from to 5. The multi image method performs better than the single image with a few exceptions. This is natural consequence of using more images to better estimate the corruption level. While the ensemble method remains overall the best method, the gap to other methods is significantly reduced.
Moreover, test the OOD detection performance. We evaluate the CIFAR- trained models the SVHN dataset (Netzer et al. (011]) in addition to MNIST trained models (for these we used rotation form the surrogate calibration sets) on Pashion-MNIST (Xiao al.) 2017) and Not-MNIST (Xiao et al. 2017). Ideally, the models should not be confident when presented with this completely OOD data. As we can see in Figures and both the single and multi images methods result methods that are significantly less confident when compared to the benchmark data (Ovadia et al. 2019). We can explain the increased performance by looking at the Pmax distributions depicted Figure 3 For the SVHN dataset, the multi image method recalibrated the model based calibration set (0,5): the dataset shift correctly captured by the Pmax shift and the probabilities are recalibrated accordingly. By exposing the model to corrupted images at the calibration stage, it now "knows what it does not know". We note that instead of requiring class predictions for all classes, our method only requires Paaax values, which is considerably smaller data set. Moreover, compile these values into a histogram with 30 bins, making the added cost of our method negligible.
4 CONCLUSIONS
Increasingly we asking models trained on given dataset to perform out of distribution data. Our work focused on uncertainty estimates, in particular, an estimate of the probability that our model classification is correct. In contrast to most deep uncertainty work, use a purely statistical approach to reduce the calibration error of deep image classifiers under dataset shift. The approach model agnostic, so it can be applied future models.
Previous work has shown that uncertainty estimates degrade on corrupted data, as measured by the expected calibration error. The greater the mismatch between training data and test data, the greater
8

---

Under review as a conference paper at ICLR 2021
the degradation of uncertainty estimates. We overcome this limitation introducing method which allows given model to be better calibrated to multiple corruption intensities. Our method works by no longer requiring that model outputs approximate class probabilities. We add simple extra calibration step, and detect the level of corruption of data, which allows the use calibrations tuned the corruption level of the data.
Table Comparison Imagenet of the benchmark implementation (Ovadia al. 2019 versus our single and multiple image methods. Numerical values of the means of ECE scores across different corruptions types, for fixed corruption intensity going from 0 to 5.
Corruption Intensity
Method
2
3
(Ovadia
0.0327
0.0534
0.0792
0.1130
0.1571
0.1997
Vanilla
(Single Image)
0.0382
0.0348
0,0300
0.0543
0.0855
0,1218
(Multi Image)
0,0201
0.0208
0.0282
0.0459
0.0533
0,0733
(Ovadia al.)
0.0187
0.0323
0.0523
0.0835
0.1237
0.1649
Temp Scaling
(Single Image)
0,0334
0.0329
0.0292
0.0631
0.0828
0,1183
(Multi Image)
0.0196
0.0207
0.0285
0.0454
0.0531
0.0733
(Ovadia et al.)
0.0222
0.0318
0.0253
0.0269
0.0470
0.0883
Ensemble
(Single Image)
0.0158
0.0273
0.0232
0.0262
0.0447
0.0793
(Multi Image)
0.0221
0.0138
0.0181
0.0291
0.0301
(Ovadia ul.)
0.0196
0.0282
0.0232
0.0446
0.0759
0.1242
Dropout
(Single Image)
0.0251
0.0303
0.0420
0.0681
0.1046
(Multi Image)
0.0161
0.0179
0.0314
0.0441
0.0755
0.0775
(Ovadia
0.0236
0.0296
0.0491
0.0783
0.1189
0,1659
LL Dropout
(Single Image)
0.0233
0.0280
0.0232
0.0442
0.0752
0.1160
(Multi Image)
0.0152
0.0176
0.0246
0.0452
0.0579
0.0761
(Oadiaa el al.)
0.0454
0.0571
0.0824
0.1153
0.1547
0.1937
SVI
(Single Image)
0.0312
0.0312
0.0258
0.0522
0.0845
0.1220
(Multi Image)
0.0146
0.0186
0.0252
0.0465
0.0531
0.0751
Table 2: Comparison on CIFAR-10 of the benchmark implementation Ovadia et al. (2019) versus our single and multiple image methods. Numerical values of the means ECE scores across different corruptions types, for fixed corruption intensity going from 0 to 5.
Coruuptinn Intensity
Method
Test
2
3
(Ovadia et al.)
0.0475
0.1009
0.1379
0.1748
0,2214
0,2782
Vanilla
(Single Image)
0.0170
0.0423
0.0590
0.0776
0.1098
0.1485
(Multi Image)
0.0127
0.0444
0.0643
0.0794
0.0908
0.1194
(Ovadia et al.)
0.0110
0.0410
0.0603
0.0822
0.1167
0.1591
Temp Scaling
(Single Imaaee
0.0150
0.0595
0.0781
0.1097
(Multi Image)
0.0134
0.0468
0.0672
0.0839
0.0925
0.1225
(Ovadia
0.0187
0.0283
0.0453
0.0718
0.1068
0.1509
Ensemble
(Single Image)
0.0141
0.0277
0.0458
0.0703
0.1005
0.1427
(Multi Image)
0.0149
0.0183
0.0355
0.0521
0.0766
0,0999
(Ovadia al.)
0.0097
0.0228
0.0331
0.0542
0.0903
0.1380
Dropout
(Single Image)
0.0127
0.0257
0.0339
0.0531
0.0868
0.1319
(Multi Image)
0.0108
0.0225
0.0338
0.0515
0.0843
0.1254
(Ovadia et al.)
0.0494
0.0999
0.1452
0.1971
0.2584
0.3338
L.I. Dropout
(Single Image)
0.0430
0.0474
0.0533
0.0792
0.1233
0.1789
(Multi Image)
0.0178
0.0289
0.0414
0.0718
0.1022
0.1489
(Ovadia al.)
0.0224
0.05-42
0.0853
0.1216
0.1608
0,2251
SVI
(Single Image)
0.0326
0.0452
0.0547
0.0733
0.1077
0.14588
(Multi Image)
0.0133
0.0290
0.0419
0.0593
0.0928
0.1231
(Ovadia
0.0491
0.1011
0.1481
0.1984
0.2472
0.3106
LL SVI
(Single Image)
0,0200
0.0382
0.0626
0.0940
0.1296
0.1763
(Multi Image)
0.0100
0.0312
0.0599
0.1002
0.1255
0.1642

---

Under review as a conference paper at ICLR 2021
REFERENCES
Charles Blundell, Julien Comebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. volume 37 of Proceedings of Machine Learning Research, pp. 1613-1622. PMLR, 2015.
Y. Bulatov. Notmnist dataset, 2011. URL o9/noctmisstddatsst..oo html
www. 2011
Sanghyuk Chun, Seong Joon Oh, Sangdoo Yun, Dongyoon Han, Junsuk Choe, and Youngjoon Yoo. An empirical evaluation on robustness and uncertainty of regularization methods. arXiv preprint arXiv:2003.03879, 2020.
Morris H DeGroot and Stephen E Fienberg. The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statistician), 32(1-2):12-22, 1983.
Pedro Domingos and Michael Pazzani. Beyond independence: Conditions for the optimality of the simple bayesian classi er. In Proc. 13th Inil. Conf. Machine Learning, pp. 105-112, 1996.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. volume 48 of Proceedings of Machine Learning Research, pp. 1050- 1059, New York, New York, USA, 20-22 Jun 2016. PMLR. URL (ttt:tt::/// htttt:://proceed.ngs. mlr. wwtppppsssssepepepee /ttp//////aall5./ html
Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation humans and deep neural networks. Advances in neural informa- fion processing systems, pp. 7538-7550, 2018.
Alex Graves.
Practical variational inference for neural networks. In J. Shawe-Taylor, R. S. Zemel, L. Bartlett, F. Pereira, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Curran Associates, Inc., 2011.
Systems 24, pp. 1348-23566
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On Calibration of Modern Neural Networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney. NSW, Australia. 6-11 August 2017, pp. 1321-1330, 2017.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. Proceedings of the International Conference on Learning Represen- tations, 2019.
Dan Hendrycks and Kevin Gimpel. A bascline for detecting misclassified and cut-of-distributie examples in neural networks. In Proceedings of International Conference Learning Represen- tations, 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. Proceedings of the International Conference on Learning Representations, 2019.
Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. AugMix: A simple data processing method to improve robustness and uncertainty. Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Volodymyr Kuleshov and Percy Liang. Calibrated structured prediction. In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (eds.), Advances in Neural In-
formation Processing Systems 28: Anmual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp. 3474-3482, 2015. URL http: 7/paerrs nips cc /
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 6405-6416, 2017.
Christos Louizos and Max Welling. Structured and efficient variational deep learning with matrix gaussian posteriors. volume 48 of Proceedings of Machine Learning Research, pp. 1708-1716, New York, New York, USA, 20-22 Jun 2016. PMLR.
10

---

Under review as a conference paper at ICLR 2021
Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, IAMIICCII, 17, pp.
2017.
Zachary Nado, Shreyas Padhy, D Sculley, Alexander D'Aourr, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. In ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011. 2011. URL http://ufldia stanford. edu/ www.shumbes...eoommmmom
Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pp. htttttss:
2015. doi: 10. 1109/CCPP.20155.29664. URL https:://o..........ettoo http:///oooorg//////// 1109//VPR. 2015. 7298640
Adam Oberman, Chris Finlay, Alexander lannantuono, and Tiago Salvador. Calibrated top-1 uncer- tainty estimates for classification by score based models. In ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning. 2020.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? Evalu- ating predictive uncertainty under dataset shift. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Sys- tems 32, pp. 13991-14002. Curran Associates, Inc., 2019.
Sangdon Park, Osbert Bastani, James Weimer, and Insup Lee. Calibrated prediction with covari- shift via unsupervised domain adaptation. In Silvia Chiappa and Roberto Calandra (eds.). The 23rd International Conference on Artificial Intelligence and Statisties, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy], volume 108 of Proceedings of Machine Learning Research, pp. 3219-3299 PMLR, 2020. UtLhttp: http://ppoceeen...eeeepp mlr pwa.s/prrss//11 park20b. html
Carlos Riquelme, George Tucker, and Jasper Roland Snoek. Deep bayesian bandits showdown. In Proceedings of the International Conference on Learning Representations, 2018.
Zhihui Shao, Jianyi Yang. and Shaolei Ren. Calibrating deep neural network classifiers on out-of- distribution datasets, 2020.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1999-1958, 06 2014.
Igor Vasiljevic, Ayan Chakrabarti, and Gregory Shakhnarovich. Examining the impact of blur on recognition by convolutional networks. arXiv preprint 1611v:1611.05760 2016.
Ximei Wang, Mingsheng Long, Jianmin Wang, and Michael I. Jordan. Transferable calibration with lower bias and variance in domain adaptation, 2020.
Larry Wasserman. All of nonparametric statistics. Springer Science & Business Media, 2006.
Yeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, and Roger Grosse. Flipout: Efficient pseudo- independent weight perturbations mini-batches. In International Conference Learning Representations, 2018.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms, 2017.
11

---

Under review as a conference paper at ICLR 2021
A ABLATION AND CROSS VALIDATION STUDY
We start by investigating the impact of the choice of corruption for the calibration set. Ideally, the choice of corruption should be representative of the distribution of corruptions, so mild corruption or very strong corruption would give slightly worse results. At the same time, here we demonstrate that choosing different corruption should not significantly degrade the results.
In Figure 2 we perform cross-vales voss-valldation study over the choice of corruption used to generate the calibration sets (always leaving out of the corruptions used at test time). We plot the mean and variance of the ECE across different validation corruptions types.
For CIFAR-10, both the single and multi image method are robust to the choice of validation cor- ruption. On ImageNet, the multi image method the improvement is consistent across the validation corruption chosen except for the dropout method. As for the single image method the calibration at lower levels of intensity is degraded using certain corruptions, for example the glass blur corrup- tion. However, this seems to caused by the strength of the corruption: the accuracy on glass blur level was roughly half that of clean images. We hypothesize that better results could be obtained by simply having the corruption strength be proportional to the loss of accuracy, as is the case of the contrast corruption (see Figure 3) In practice, the choice of corruption for single image method should be such that the method remains calibrated for in-distribution images.
We investigate well the impact of adding corrupted images to the calibration sets, In order to do so, we compare the results of our proposed method and the benchmark from Ovadia et al. (2019) with only clean images like the
with the calibration obtained from using a single calibration method proposed in (Oherman et al., 2020). We refer to it as the Top1 binning method. As we can see in Figure I0 if the classifier is not well-calibrated, e.g., the vanilla or the LL SVI classifiers, there is a consistent improvement across all corruption intensity levels, with the improvement being only marginal for ImageNet. Moreover, when the classifier well-calibrated, Top1 binning does not improve calibration (e.g. the Temp Scaling classifier) or even decreases it at high levels of corruption intensity (e.g. Ensemble and Dropout).
Finally, we explore why the method works in practice. Figure 11 shows us that without any cal- ibration the ECE scores become higher when the mismatch between the Pmax distribution of the training set and the Pmax distribution of the test set increases. Here measure the mismatch in terms of the Pmax means, the same criteria used in the multi image method. These qualitative results confirmed by the Pearson's correlation coefficient. This correlation justifies why detecting the Puna distribution shit allows us to significantly improve the calibration of the different methods: in practice our proposed methods perform the recalibration of the model based the calibration set whose Pmax distribution is closest to the Pmax distribution of the test set. Moreover, one notices the higher the correlation, the bigger the calibration improvement provided by both our single and multi image methods. For instance, Dropout has the lowest Pearson's 's r score and it is also the method where we notice the least improvement. On the other hand, Vanilla has the largest improvement and also the highest Pearson's monson'sr' r score.
B TABLES OF BRIER METRICS
Table 3 and Table 4 report the mean Brier scores for each model and dataset across different cor- ruption types, for fixed corruption intensity going from 5. The Brier scores can be computed directly from the data, without binning. The ranking provided by the Brier scores is quite similar that provided by the ECE. On ImageNet the only difference is Ensemble at corruption level 1. On CIFAR-10 10 there were two ranking differences.
C TABLES OF ECE METRICS ACCROSS DIFFERENT CORRUPTIONS
Table 5 and Table 6 report the ECE scores for the vanilla model across different corruption types and intensities ranging from 0 to for ImageNet and CIFAR-10 10. respectively. Contrast is the corruption used for to form the calibration sets.
12

---

Under review as a conference paper at ICLR 2021
(a) ImageNet
Vanilla
Temp Scalling
Ensemble
Dropout
LL Dropout
LL SVI
Dvadia
0 3.110
Single Image
0.05
0.00
Computtin intersity
(h) CIFAR-10
Temp Scaling
Dropout
0.20
Ovadla Singte image
0.15
0.10
0.00
012545
01234
Comppiion Intersity
Figure 9: Figure shows us the mean ECE across different corruptions types, for fixed corruption when contrast used in the calibration sets. Here we show how those
intensity going from means change when different corruptions are used in the calibration set. For CIFAR-10, 10, our pro- posed methods are robust to the choice corruption used in the calibration set, while for ImageNet the choice of the corruption is import, in particular for the single image method.
(a) ImageNet
Vanilla
Temp Scaling
Ensemble
Dropout
LL Dropout
SVI
0.15
Dvadie
Singla Image
Nulti Image
Calibration
0.05
Corruption Intensity
Vanilla
Temp Scaling
Ensemble
Dropout
LL Dropout
0.30
Dvadiaa
0.15
Single
Image
0.10
Top Calibration
0.00
245
Comuptiin Intensity
(b) CIFAR-10
Figure 10: Comparison the benchmark implementation Ovadia et al. 2019). versus our proposed
single and multiple image methods and Top1 binning (Oberman al. 2020) for CIFAR-10. Mean Expected Calibration Error (ECE) across different corruptions types, for fixed corruption intensity going from 0 to 5.
13

---

Under review as a conference paper at ICLR 2021
(a) ImageNet
(b) CIFAR-10
Figure 11: ECE (pre-calibration versus 11 test H taann and corresponding Pearson's 's score, where test train denote the Pmax of the test and training set, respectively. Each point in the plot represents a different corruption at different level of intensity.
Table 3: Comparison on Imagenet of the benchmark implementation (Ovadia et al. 2019 versus single and multiple image methods. Numerical values of the means of the Brier scores across different corruptions types, for fixed corruption intensity going from 0 to 5.
Corruption Intensity
Method
Test
2
3
(Ovadia el ul.)
0.1217
0.1567
0.1693
0.1759
0.1790
0.1721
Vanilla
(Single Image)
0.1225
0.1545
0,1621
0.1621
0.1567
0.1417
(Multi Image)
0.1213
0.1535
0.1620
0.1607
0.1472
0.1230
(Ovadia
0.1209
0.1544
0.1647
0.1681
0.1669
0.1558
Temp Scaling
(Single Image)
0.1221
0.1545
0,1620
0.1615
0.1554
0.1395
(Multi Image)
0.1214
0.1536
0.1619
0.1603
0.1467
0.1225
(Ovadia al.)
0.1136
0.1478
0.1579
0.1579
0.1502
0.1305
Ensemble
(Single Image)
0.1135
0.1476
0.1579
0.1581
0.1504
0.1303
(Multi Image)
0.1142
0.1468
0.1575
0.1584
0.1485
0.1221
(Ovadia et al.)
0.1284
0.1547
0.1608
0.1584
0.1550
0.1369
Dropout
(Single Immaee)
0.1290
0.1557
0.1615
0.1583
0.1536
0.1335
(Multi Image)
0.1289
0.1617
0.1504
0.1537
(Ovadia et al.)
0.1194
0.1512
0.1609
0.1628
0.1603
0.1506
LL Dropout
(Single Image)
0.1197
0.1513
0.1585
0.1567
0.1492
0.1337
(Multi Image)
0.1195
0.1506
0.1586
161566
0.1437
0.1186
(Ovadia
0.1291
0.1558
0.1642
0.1632
0.1610
0.1557
LVT
(Single Image)
0.1278
0.1526
0.1559
0..486
0.1390
0.1266
(Multi Image)
0.1269
0.1519
0.1558
0.1475
0.1295
0.1078
14

---

Under review as a conference paper at ICLR 2021
Table 4: Comparison of CIFAR-10 of the benchmark implementation Ovadia al. (2019) versus our single and multiple image methods. Numerical values of means Brier scores across different corruptions types, for fixed corruption intensity going from 0 to 5.
Corruption Intensity
Method
(Ovadia al.)
0.0694
0.1236
0,1620
0.1971
0.2357
0,2804
Vanilla
(Single Image)
0.0644
0.1061
0.1352
0.1600
0,1848
0.2120
(Multi Image)
0.0640
0.1070
0.1367
0.1597
0.1787
0,2021
(Ovadia al.)
0.0633
0.1053
0.1345
0.1593
0.1845
0.2122
Temp Scaling
(Single Imagee)
0.0636
0.1054
0.1345
0.1588
0.1
0.2096
(Multi Image)
0.0634
0.1066
0.1364
0.1597
0.1766
0.2007
(Ovadia et al.)
0.0443
0.0803
0.1129
0.1426
0.1696
0.2033
Ensemble
(Single Imagee
0.0442
0.0803
0.1135
0,1433
0.1697
0,2026
(Multi Image)
0.0447
0.0791
0.1112
0.1366
0.1592
(Ovadia al.)
0.0644
0.0883
0.1122
0.1378
0.1672
0,2024
Dropout
(Single Image)
0.0647
0.0885
0.1123
0.1376
0.1668
0.2014
(Multi Image)
0.0648
0.0882
0.1123
0.1373
0.1652
0.1981
(Ovadia
0.0736
0.1228
0.1673
0.2162
0.2703
0.3343
L.I. Dropout
(Single Image)
0.0735
0.10
0.1357
0.1679
0.2005
0.2370
(Multi Image)
0.0689
0.1031
0.1336
1.1658
0.1920
0.2225
(Ovadia et al.)
0.0744
0.1111
0.1436
0.1763
0,2140
0.2543
SVI
(Single Image)
0.0765
0.1081
0.1349
0,1610
0,1890
0.2173
(Multi Image)
0,0738
0.1055
0.1322
0.1570
0.1822
0.2063
(Ovadia et al.)
0.0704
0.1238
0.1710
0.2176
0.2611
0.3139
LL
(Single Image)
0.0658
0.1053
0.1407
0.1737
0.2032
0.2361
(Multi Image)
0.0652
0.1039
0.1398
0.1753
0.2012
0.2319
15

---

Under review as a conference paper at ICLR 2021
Table Comparison ImageNet of the benchmark implementation Ovadia (2019) versus our single and multiple image methods for the vanilla classifier. Numerical values of ECE scores for different corruptions different intensity levels going from 0 to The contrast corruption was used to form the calibration sets.
Corruption Intensity
Coruptiin
Test
(Ovadia et al.)
0.0327
0.0333
0.0348
0.0371
0.0436
0.0533
Brightness
(Single Image)
0.0382
0.0436
0.0:25
0.0:127
0.0396
0,0315
(Multi Image)
0.0201
0.0209
0,0225
0.0106
0.0059
0,0073
(Ovadia et aL)
0.0327
0.0417
0.0:144
0.0628
0.0862
0.1026
Defocus Blur
(Single Image)
0.0382
0.0483
0.0392
0.0325
0.0350
0.0423
(Multi Image)
0.0201
0.0192
0.0376
0.0950
0.0493
0.0533
(Ovadia
0.0327
0.0266
0.0881
0.0595
0.1094
0,2633
Elastic Transform
(Single Image)
0.0382
0.0576
0.0183
0.0257
0.0306
0.1745
(Multi Image)
0.0201
0.0208
0.0173
0.0109
0.0252
0.0904
(Ovadia et al.)
0.0327
0.0531
0.0699
0.0972
0.1288
0,1985
Fog
(Single Image)
0,0382
0.0302
0.0497
(Multi Image)
0.0201
0.0143
0,0194
0.0149
0.0465
0.0378
(Ovadia et al.)
0.0327
0.0528
0,0385
0.1375
0.1515
U.1735
Frost
(Single Image)
0.0382
0.0308
0.0252
0.0578
0.0699
0.0935
(Multi Image)
0.0201
0.0099
0,0180
0.0298
0.0158
0.0204
(Ovadia et al.)
0.0327
0.0311
0,0464
0,0750
0.1206
Gaussian Blur
(Single Image)
0.0382
0.0527
0,0405
0,0333
0.0494
0.09433
(Mulii Image)
0.0201
0.0161
0.0407
0,0138
0.0331
0,0404
(Ovadia et al.)
0.0327
0.0693
0.1025
0.1684
0.2552
0.2976
Gaussian Noise
(Single Image)
0.0382
0.0155
0.0272
0,0814
D.1659
0.2121
(Multi Image)
0.0201
0.0262
0.0472
0.0804
0.0780
0.1436
(Ovadia al.)
0.0327
0.0495
0.0745
0,1624
0.1720
0.1647
Glass Blur
(Single Image)
0.0382
0.0354
0,0220
0.0837
0.0956
0.0930
(Multi Image)
0.0201
0.0105
0,0123
0.0184
0.0372
0.0379
(Ovadia et al.)
0.0327
0.1157
0.1543
0,1892
0.2681
0.3035
Impulse
(Single Image)
0,0382
0.0337
0.0666
0,1002
0.1783
(Multi Image)
0.0201
0.0595
0,0638
0.1003
0.0910
0.1466
(Ovadia et al.)
0.0327
0.0640
0.0684
0.1042
0.1371
0. 1367
Pixelate
(Single Image)
0.0382
0.0209
0.0171
0.0284
0.0534
0.0522
(Multi Image)
0.0201
0.0207
0.0236
0.0483
0.0511
0.05-11
(Ovadia et al.)
0.0327
0.0512
0.0298
0.0561
0.1121
Saturate
(Single Image)
0.0382
0.0311
0.0264
0.0494
0.0299
0.0327
(Multi Image)
0.0201
0.0095
0.0138
0.0177
0.0091
0.0237
(Ovadia et al.)
0.0327
0.0682
0.1165
0.1814
0.2863
0.3297
Shot Noise
(Single Image)
0.0382
0.0190
0,0353
0,0931
0.1971
0,2420
(Multi Image)
0.0201
0.0243
0.0609
0.0941
0.1124
0.1654
(Ovadia al.)
0.0327
0.0309
0.0574
0.1056
0.1674
U.2401
Spatter
(Single Image)
0.0382
0.0464
0.0266
0.0273
0.0808
0.1503
(Multi Image)
0.0201
0.0136
0.0099
0.0196
0.0804
U.15:36
(Ovadia et al.)
0.0327
0.0587
0.0804
0.1702
0.2294
0,2863
Speckle Noise
(Single Image)
0.0382
0. 0.0235
0,0142
0.0826
0.1395
0,19633
(Multi Image)
0.0201
0.0158
0.0248
0.0819
0.1430
0.1093
(Ovadia et al.)
0.0327
0.0553
0.0885
0.11499
0.1413
0.1605
Zoom Blur
(Single Image)
0.0382
0.0303
0.0274
0.0467
0.0678
0.0807
(Multi Image)
0.0201
0.0117
0.0531
0.0207
0.0157
16

---

Under review as a conference paper at ICLR 2021
Table 6: Comparison of CIFAR-I0 of the benchmark implementation Ovadia et al. (0019) versus single and multiple image methods for the vanilla classifier. Numerical values of ECE scores for different corruptions at different intensity levels going from 0 5. The contrast corruption was used form the calibration sets.
Corruption Intensity
Corruption
Test
3
4
5
(Ovadia et al.)
0.0475
0.0465
0,0482
0.0541
0.0596
0.0697
Brightness
(Single Image)
0.0170
0.0180
0.0191
0.0178
0.0192
0.0233
(Multi Image)
0.0127
0.0151
0.0113
0.0147
0.0161
0.0184
(Ovadia al.)
0.0475
0.0478
0.0526
0.0874
0.1282
0,2555
Defocus Blur
(Single Image)
0.0170
0.0150
0.0177
0.0193
0.03418
0.0919
(Multi Image)
0.0127
0.0120
0.0133
0.0214
0.0336
0.0418
(Ovadia et al.)
0.0475
0.0735
0.0732
0.1004
D.1349
0.1737
Elastic Transform
(Single Image)
0.0170
0.0211
0.0199
0.0287
0.0268
0.0553
(Multi Image)
0.0127
0.0168
0.0144
0.0282
0.0314
0.0648
(Ovadia et al.)
0.0475
0.0459
0.0473
0.0579
0.0749
0.1946
Fog
(Single Image)
0.0170
0.0187
0.0239
0.0284
0.0265
0.0675
(Mulii Image)
0.0127
0.0123
0.0142
0.0140
0.07ST
(Ovadia al.)
0.0475
0.2240
0.2401
U.3589
Frost
(Single Image)
0.0170
0.0267
0.0499
0.1083
0.1258
0.2255
(Multi Image)
0.0127
0.0282
0.0623
0.1211
0.1354
0.2423
(Ovadia et al.)
0.0475
0.0482
0.0899
0.1565
0.2468
0.3729
Gaussian Blur
(Single Image)
0.0170
0.0145
0.0436
0.0904
0,1910
(Multi Image)
0.0127
0.0115
0.0240
0.0529
0.0296
0.0949
(Ovadia al.)
0.0475
0.1712
0.2781
0,3833
0,4266
0.44166
Gaussian Noise
(Single Image)
0.0170
0.0627
0.1349
0,2253
0.2633
U.2786
(Multi Image)
0.0127
0.0744
0.1556
0.1439
0.1716
0.1872
(Ovadia et al.)
0.0475
0.4228
0.3902
0.3492
D.4673
U.4207
Glass Blur
(Single Image)
0.0170
0.2732
0.2379
0.2034
0.3107
0,2643
(Multi Image)
0.0127
0.2936
0,2593
0.2234
0.3323
0,2868
(Ovadia et al.)
0.0475
0.1226
0.2027
0,2518
0.3267
0.3794
Impulse Noise
(Single Image)
0.0170
0.0475
0.0906
0.1197
0.1663
0, 0,2073
(Mulii Image)
0.0127
0.1031
0.1366
0.0980
(Ovadia al.)
0.0475
0.06553
0.0975
0.1331
0.2836
0.4390
Pixelate
(Single Image)
0.0170
0.0179
0,0349
0.0513
0.1676
0.3039
(Multi Image)
0.0127
0.0187
0.0419
0.0659
0.1826
0.3220
(Ovadia et al.)
0.0475
0.0659
0.0713
0.0507
0.0658
0.0995
Suturate
(Single Image)
0.0170
0.0245
0,0216
0.0170
0.0267
(Multi Image)
0.0127
0.0194
0.0249
0.0117
0.0214
U.0399
(Ovadia et al.)
0.0475
0.1082
0.1769
0.3012
0.3579
0.4046
Shot Noise
(Single Image)
0.0170
0.0360
0.0741
0.1546
0.2041
112406
(Multi Image)
0.0127
0.0433
0.0811
0.1759
0.1232
0.1510
(Ovadia et al.)
0.0475
0.0726
0.1052
0.1233
0.1193
0,1828
Sputter
(Single Image)
0.0170
0.0244
0.0395
0.0443
0.0488
0.0897
(Multi Image)
0.0127
0.0236
0.0463
0.0514
0.0614
0. 1083
(Ovadia et al.)
0.0475
0.1155
0.2138
0.2440
0.3118
0.3601
Speckle Noise
(Single Image)
0.0170
0.0339
0.0985
0.1176
0.1626
0.1992
(Multi Image)
0.0127
0.0475
0.1109
0.1337
0.0930
0.1102
(Ovadia al.)
0.0475
0.0817
0.1022
0.1374
0.1765
0. 0.2387
Zoom Blur
(Single Image)
0.0170
0.0241
0.0225
0.0199
0.0447
0.08466
(Multi Image)
0.0127
0.0235
0.0197
0.0297
0.0606
0.0322
(Ovadia et al.)
0.0475
0.0554
0.1240
0.1410
0.1224
0.05-40
Translation
(Single Image)
0.0170
0.0410
0.0421
0.0385
0.0237
(Multi Image)
0.0127
0.0147
0.0477
0.0453
0.0488
0.0176
17


Reviews:
[{"review_id": "wQOb6VeBda", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the problem of providing calibrated predictions for out-of-distribution data. They propose algorithms for both calibrating predictions given a single image from the unknown distribution as well as given multiple images from the unknown distribution. They propose an algorithm that estimates which “calibration distribution” the novel image came from, and then use calibrated predictions for this distribution. They evaluate their approach on a standard image datasets including CIFAR-10 and ImageNet, and show that their approach outperforms existing work.\n\nPros\n- Important problem\n\nCons\n- The approach claims to work on out-of-distribution data, but assumes the possible novel distributions are known\n- Missing related work\n\nThe approach proposed by the authors is fundamentally flawed: while they do not directly assume to know which “unknown distribution” the novel image is from, they assume it is from one of a small set of possibilities. This information is not assumed in existing work, and fundamentally alters the problem, making it simple to address and uninteresting.\n\nThe proposed approach is also very simplistic, which is not a flaw in and of itself but is a consequence of the extra knowledge they assume. In particular, given this extra information, the authors simply predict which shifted distribution the novel example is from, and then use the calibrated prediction for that distribution.\n\nIn practice, this problem is important for handling unanticipated distribution shifts in production. If the distribution shift is known and anticipated, then a much more natural approach would be to simply use data augmentation to generate data from the shifted distribution and train the model on this extra data.\n\nIn addition, there recent work in this area that the authors do not cite, for instance:\n\nPark et al., Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptation. In AISTATS 2020.\n\nWang et al., Transferable Calibration with Lower Bias and Variance in Domain Adaptation. In NeurIPS 2020.\n\n-------------------------------------------------------------------------------------------------------------------------------\n\nPost rebuttal: I have updated my score based on the clarification provided by the authors. My remaining concern is that I still think the baselines considered by the authors is incomplete. In particular, the calibration under distribution shift techniques can still be applied, just using either just a single test image or their set of multiple test images. Admittedly, this approach would probably not perform well for a single image, but in Table 5, it seems like oftentimes multiple images are needed to even beat Ovadia et al. (2019).", "extra_scores": {}, "extra_text": {}}, {"review_id": "fkn1w5z6TnF", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The submission proposes a very simple and seemingly effective method for improving uncertainty estimates of predictions for corrupted data. The main idea is to calibrate predictive confidences assuming access to an “exposure” set of corruptions. In particular, the paper uses contrast-corrupted data to calibrate predictive confidences and shows improvements for the types of corruptions discussed in [1] (leaving out contrast). \n\n(+) The improvements seem to be fairly consistent, and the method is simple and intuitive. It is interesting to know that such improvements can be had, i.e. calibration on one type of corruption is transferrable to other types to some extent.\n\n(-) While the 'single image method’ is more or less intuitive, the motivation for the ‘multiple image method' is less clear. For example, why not compare the distribution of p_maxes in S^deploy to the distribution of p_maxes in each p^CAL? Or, why not do a similar weighted averaging as in the single image method, with divergences between these two distributions providing the weights?\n\n(-) While this paper mostly builds on prior work, it would be interesting to see calibration advantages on more than just the artificially corrupted sets. The prior work [2] that this submission builds upon reports results across a fairly wide range of tasks and out-of-distribution types. It would be more interesting to see if such calibrations with a specific set can be relevant for more realistic OOD cases as well, as in [2].\n\nThe sentence \"Heuristically we always want clean images in our calibration set while having different shifted means” is not clear, could the authors elaborate? Why might we prefer such a calibration set? This might be an important point for the reader, since it motivates the particular choice of contrast-corruption.\n\nAn obvious baseline would be to perform the same recalibration procedure, but with the validation set (i.e. non-corrupt data), to figure out if corrupt sets in particular are required for calibration. I suspect they are, since performance at such sets are likely to be poorer, which would allow for more calibration room over a larger “error”-space.  \n\nOverall I think this paper could be interesting in that it informs us of the possibility of transferring recalibration from a corruption-exposure procedure, which to my knowledge is a novel reporting. More experiments as described above would improve the paper, by making a more compelling case for such exposure-based recalibration techniques.\n\n[1] Benchmarking neural network robustness to common corruptions and perturbations, Hendrycks and Dietterich\n\n[2] Can you trust your model’s uncertainty? Ovadia et al.\n\n\nPost Rebuttal:\nThanks for the response, and the new experiments. I continue to think that this is a nice simple method that works well enough to be interesting. I retain my initial rating.", "extra_scores": {}, "extra_text": {}}, {"review_id": "P_phbTML3Bk", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Thank you for the additional experiments. Especially, Figure 7 and 8 look promising.\nMy conclusion from the experiments is that the \"contrast\" corruption (used for validation) seems to be general enough, in the sense that for many other corruptions, encountered at test time, the performance is good.\nHowever, as AnonReviewer1, I am not sure about why the methodology seems to work well for very different types corruptions at test time, and completely OOD data (Figure 7 and 8). \nMore empirical/theoretical analysis would be nice.\nIncreased rating to 6.\n\n------\n\nSummary:\nThe paper addresses the important problem of over-confident predictions on out-of distribution (OOD) data. \nFocusing on image classification, they propose to calibrate probabilities on a validation dataset which contains images that were artificially corrupted in various ways (noise, blur, brightness etc.). This is different from traditional post-hoc methods which use an uncorrupted validation data set (iid assumption). This way they can explicitly calibrate probabilities for the non-iid assumption.\nTheir proposed methods (Single Image and Multiple Image Method) seem to be new, and their experiments suggests that for certain types of corruptions their method is effective (as measured by ECE and shown by calibration diagrams in Figure 2).\n\nStrong points:\n\n- Even when the type of corruption in the validation dataset is different from the corruption in the test set, the proposed method can considerably improve over existing methods.\n- The proposed method, similar to other post-hoc calibration methods, can be used in combination with any model. \n- The illustrations, in particular Figure 2, are well done.\n\nUnclear/Weak points:\n\n- The proposed method relies on set of pre-specified set of corruptions that are used to generate the validation data. Therefore, I think it is important to throughly evaluate the  impact when the corruption used for validation is different from the one in the test data.\nHowever, results are only reported when the validation corruption is \"Contrast\".\nFurthermore, it seems the results reported are the ones on the test data averaged over several types of corruption. However, it would be interesting to see for what types of test corruption the method work/not works.\n\n- It would be interesting to see results when the test data is completely OOD data, like using SVHN dataset for testing predictions of a model that was trained on CIFAR-10 (see Ovadia et al, 2019). \n\n- Some intuition/analysis should be given about the proposed method.\nFor example, in \"Single Image Method\" (i) and (ii): \nMy understanding is that the authors want to achieve that \nif a test sample is corrupted by type A, then q_i(max) should be close to one for the calibration set which type of corruption is similar to type A.\n\n- At least in the Appendix:\nSome more formal description of how Equation (3) is calculated should be given\n\n- The formula for computing $L_j(p_{max}(x))$ in \"Single Image Method\" (i) should be given.\n\n- The notation is slightly unclear, what does the \"m\" in $P^{CAL, j}_m$ mean?\n\n- I am not sure what this means:\n\"on each of the calibration sets determined by combinations of intensity given by: {0}, {5,0}, {5,4,0}, {5,4,3,0}, {5,4,3,2,0}, and {5,4,3,2,1,0}.\"\n\n- What are sizes of each calibration set in the experiments? How many calibration sets are there?\n\n- I am also not sure about this sentence:\n\"Heuristically we always want clean images in our calibration set while having different shifted means.\" Does it mean that you would like to have both clean images and corrupted images in the validation dataset?\n\n\nMinor points:\n- it should probably be $L_i(p_{max}) / ...$ and not $L_j(p_{max}) / ...$  in the equation in paragraph \"Single Image Method\" (i).\n\n- It might be better to rename $P^{deploy}_m$ as $P^{test}_m$, to confirm to the standard terminology of train/validation/test data.\n\n- in \"of p_max under each of the calibration models, using the probability density\",\nI think it is easier to understand if \"calibration models\" -> \"calibration sets\".\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Kppxu78l1q", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this work, the authors propose a post-hoc calibration method for potentially OOD data that relies on estimation of the \"degree\" of corruption for new test data. The rely on the benchmark provided in Ovadia et al. as a basis for their analysis. Ovadia et al. assessed common measurements of uncertainty such as Brier score, ECE, and entropy over a variety of datasets including MNIST + translations/rotations, CIFAR-10 and CIFAR-10C, and ImageNet and ImageNet-C for a variety of models such as vanilla neural networks, SVI, ensembles, and Dropout. By using corrupted versions of these common datasets, Ovadia et al. could evaluate how uncertainty estimates vary under dataset shift. In this work, the authors aim to improve the calibration of probabilities obtained from these models. They start by establishing a calibration set where they derive $p_{correct}$ from a sample of $p_{max}$. Then, depending on how many test images they are evaluating, they use a single image or multiple image method to attempt to determine which calibration set (of which there can be many depending on the number of corruption levels considered), the test images are closest to. Then they \"correct\" the model's probability estimate by weighting over the calibration sets. They show on CIFAR-10 and ImageNet that their method results in lower ECE over varying levels of corruptions. \n\nStrengths: \n* This method could be applied post-hoc to a variety of models \n* Seems fast to compute \n* Results in better calibration estimates\n\nWeaknesses: \n* The terminology of the paper is ill-defined. For instance, I don't think the authors ever explicitly define $p_{max}$. I understood it by context, but the presentation could be much clearer. \n* When is it clear to use the \"Multiple Image Method\"? How can one be sure that $\\{x_1,...x_m\\}$ come from the same distribution? \n* It was not explained why \"contrast\" was chosen as the calibration set. \n* How can this method be applied when the OOD corruption is very far away from \"contrast\"? Could you evaluate how your method performs on corruptions such as translations and rotations? \n\nUltimately, I don't think that this method was presented in a clear enough fashion or has sufficiently demonstrated an ability to generalize to new types of corruptions and am rating this a 4 because of these reasons. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method to improve the calibration of neural networks on out-of-distribution (OOD) data.

The authors show that their method can be applied post-hoc to existing methods and that it improves calibration under distribution shift using the benchmark in Ovadia et al. 2019.

However, reviewers felt that the theoretical justification for why this works is unclear (see detailed comments by R1 and R4), and some of the choices are not well-justified. Revising the paper to address these concerns with additional theoretical and/or empirical justifications should improve the clarity and strengthen the paper.

I encourage the authors to revise and resubmit to a different venue.

Decision: Reject