U NCERTAINTY FOR DEEP IMAGE CLASSIFIERS ON OUTOF - DISTRIBUTION DATA .
Anonymous authors
Paper under double-blind review
A BSTRACT
In addition to achieving high accuracy, in many applications, it is important to estimate the probability that a model prediction is correct.
Predictive uncertainty is particularly important on out-of-distribution (OOD) data where accuracy degrades.
However, models are typically overconﬁdent, and model calibration on
OOD data remains a challenge.
In this paper we propose a simple post hoc calibration method that signiﬁcantly improves on benchmark results (Ovadia et al., 2019) on a wide range of corrupted data.
Our method uses outlier exposure to properly calibrate the model probabilities.
P REDICTIVE UNCERTAINTY
When a machine learning model makes a prediction, we want to know how conﬁdent (or uncertain) we should be about the result.
Uncertainty estimates are useful for both in distribution and outof-distribution (OOD) data.
Predictive uncertainty addresses this challenge by endowing model predictions with estimates of class membership probabilities.
The baseline method for predictive uncertainty is to simply use the softmax probabilities of the model, p softmax ( x ) = softmax( f ( x )) , as a surrogate for class membership probabilities (Hendrycks & Gimpel, 2017). Here f ( x ) denotes the model outputs.
Other approaches include temperature scaling (Guo et al., 2017), dropout (Gal &
Ghahramani, 2016; Srivastava et al., 2014), and model ensembles (Lakshminarayanan et al., 2017), as well as Stochastic Variational Bayesian Inference (SVBI) for deep learning (Blundell et al., 2015;
Graves, 2011; Louizos & Welling, 2016; 2017; Wen et al., 2018), among others.
All methods suffer from some degree of calibration error, which is the difference between predicted error rates and actual error rates, as measured by collecting data into bins based on p max = max i p softmax i bins.
The standard measurement of calibration error is the expected calibration error (ECE)
Guo et al.
(2017), although other measures have been used (see Nguyen et al.
Hendrycks & Gimpel (2017)), including the Brier score (DeGroot & Fienberg, 1983), which is also used in Ovadia et al.
1. O UR RESULTS
We are interested in image classiﬁcation problems, in particular the CIFAR-10 and Imagenetdatasets, in the setting of distribution covariate shift, where the data has been corrupted by an unknown transformation of unknown intensity. These corruptions are described in Hendrycks & Dietterich (2019).
Our starting point is the work of Ovadia et al. (2019) which offers a large-scale benchmark of existing state-of-the-art methods for evaluating uncertainty on classiﬁcation problems under dataset shift by providing the softmax model outputs. One of main take-aways from the work of Ovadia et al. (2019) is that, unsurprisingly, the quality of the uncertainty predictions deteriorates signiﬁcantly along with the dataset shift.
In order to be able to calibrate for different intensity levels of unknown corruptions, we make use of surrogate calibration sets, which are corruptions of the data by a different (known) corruption.
Then, when given an image (or sample of images), we ﬁrst estimate the corruption level, and then recalibrate the model probabilities based on the surrogate representative calibration set.
The latter step is done with a simple statistical calibration step, converting the model outputs into calibrated uncertainty estimates.
[-to Vanilla 0 1 2 3 
 Temp Scaling 0 1 2 3 
 Ensemble 0 1 2 3 
-]Surprisingly we can estimate the corruption level just using the model outputs.
[-Dropout 0 1 2 3 
 LL Dropout 0 1 2 3 
 LL SVI Corruption Intensity-]We focus on the probability of correct classiﬁcation, using the p max values.
[Figure]
In Figure 1 [-2 
 [Figure]-]we compare each benchmark uncertainty method with our two methods.
[-Ensemble 0 1 2 3 
 Dropout 0 1 2 3 
 LL Dropout 0 1 2 3 
 SVI 0 1 2 3 
-]The single image method determines the appropriate calibration set using only a single image.
The multiple image method uses a sample of images (all drawn from the same corruption level and type) to choose the calibration set.
More images allow for a better choice of the calibration set, further reducing the calibration error.
As shown in the ﬁgure, the ECE decreases across almost all methods and levels of intensity with the greatest improvement at the higher intensities.
The Brier scores give similar results (see Figure 4 below).
We also reproduce a ﬁgure in Ovadia et al.
(2019) which gives a whisker plot of the distribution of the values.
Our results are based on the fact that data distribution shift typically leads to overconﬁdent models
Nguyen et al.
We make the observation that overconﬁdent models have p max values which are above the true probability, which correspond to a shifted p max distribution shift.
Thus we use p max distribution shift as a surrogate for data distribution shift.
This allows us to signiﬁcantly reduce calibration error using a purely statistical approach .
In practice, we perform the model recalibration for different corruption intensities based simply on the p max distribution shift, which we detect using surrogate corrupted datasets to calibrate the models on other corruptions. Crucially, the corruption used to generate the surrogate corrupted datasets is left out of the test set.
The calibrated probabilities are visualized using histograms in Figure 2.
1. O THER R ELATED WORK
Models trained on a given dataset are unlikely to perform as well on a shifted dataset (Hendrycks &
Dietterich, 2019).
Moreover, there are inevitable tradeoffs between accuracy and robustness (Chun et al., 2020).
Training models against corruptions can fail to make models robust to new corruptions
[Figure]
, and the brown is where both overlap.
The ECE is large (lower is better) for the Vanilla method at higher corruption levels, due to the probability shift.
The Brier score (lower is better) is also improved.
The gap between the orange and blue curves represents the calibration error.
Notice that we used 30 equally sized bins, so in the
CIFAR-10 plot, there are very few values below .4, which is why the ﬁrst bin is wide. (Vasiljevic et al., 2016; Geirhos et al., 2018). Hendrycks et al.
(2019) deals with anomaly detection, the task of distinguishing between anomalous and in-distribution data.
They propose an approach called Outlier Exposure (OE) that consists in training anomaly detectors on an auxiliary dataset of outliers.
Similarly in Hendrycks et al. , the authors propose AUGMIX, a method that improves both robustness and uncertainty measures by exposing the model to perturbed images during training. Shao et al. (2020) propose a conﬁdence calibration method that uses an auxiliary class to separate mis-classiﬁed samples from correctly classiﬁed ones which thus allowing the misclassiﬁed samples to be assigned with a low conﬁdence.
Nado et al.
(2020) argues that the internal activations of the deep models also suffer from covariate shift in the presence of OOD images.
Thus they propose to recompute the batch norm at prediction time using a sample of the unlabeled images from the test distribution improving the accuracy and ultimately the calibration.
Park et al.
(2020) and Wang et al.
(2020) focus on the more general problem of unsupervised domain adaptation where one assumes assumes to have unlabeled examples from the test distribution which may only share the same classiﬁcation classes as the train distribution.
Park et al.
(2020) propose an approach based on importance weighting to correct for the covariate shift in the data, together with learning an indistinguishable feature map between training and test distributions.
Wang et al.
(2020) extend the temperature scaling method into domain adaption achieving more accurate calibrations with lower bias and variance without introducing any hyperparameters.
[Figure]
B ACKGROUND 2. C LASSIFICATION AND LABEL PROBABILITIES
Predictive uncertainty seeks to estimate the probability that an input x belongs to each class,
[Equation]
Here we write x ∈ X for data, y ∈ Y
= { 1 , . .
, K } for labels.
In the benchmark methods, the softmax of the model outputs, p softmax ( x ) = softmax( f ( x )) are used as a surrogate for the class probabilities.
In the case of the vanilla and temperature scaling methods, f ( x ) is simply the model outputs.
Similarly, for the Ensemble or Dropout methods, p softmax ( x ) represents the average of the probability vectors over the multiple models or queries of the model, respectively.
Generally speaking, these softmax values are not an accurate prediction of the class probabilities p classk ( x ) (Domingos & Pazzani, 1996).
Here we focus on the correct classiﬁcation
[Equation]
where the classiﬁcation of the model is given by ˆ y ( x ) = arg max f i ( x ) .
Guo et al.
(2017) showed that p max = max i p softmax i usually overestimates p correct .
We can extend our method to topcorrectness, as well as to other quantities of interest, such as, using different binning methods, based on structured predictors (Kuleshov & Liang, 2015).
M ETHOD
Problem deﬁnition We are given a model (or ensemble of models) trained on a given dataset ρ train .
We want to have calibrated error estimates on an unknown (different) data set ρ test , which could have different levels of corruption.
Since one model cannot be calibrated on all of the possible (different) datasets, we want to allow for multiple calibrations, and apply them accordingly.
We study two cases: (i) we have a single image drawn from an unknown distribution, or (ii) we have multiple images, each drawn from the same unknown distribution.
In the latter case, we use the full test set (we obtain similar results using 100 images). [Figure] [Figure] [Figure]
3. C ALIBRATING FOR DATASET SHIFT
We choose C distinct calibration sets generated from shifted distributions ρ CAL ,j .
These sets are chosen to have different representative degrees of corruption intensity.
Each calibration set leads to a different uncertainty estimate for a given p max value.
We adaptively choose the calibration set given an image (Single Image Method) or a test set of images (Multiple Image Method).
For each calibration dataset S CAL ,j , we convert the p max values on that calibration set into the p correct values, as follows. Recall that p max = max i p softmax
i , i.e., the model’s probability for the predicted class.
(i) Evaluate the model p max values on the calibration set, P CAL ,j = { p max ( x ) | x ∈ S CAL ,j } .
Record the probability density h CAL ,j ( x ) , of the p max values as a histogram, by binning the p max using equally spaced bins.
(ii) Deﬁne the calibrated model probabilities by
[Equation]
using ground truth labels on the calibration set S CAL ,j .
These probabilities are computed using a histogram in the following way.
Partition S CAL ,j = { x 1 , . . .
, x m } into bins B i according its p max values.
Given an image x with p max ( x ) ∈ B i , approximate p CAL ,j correct ( x ) as
[Equation] See (Oberman et al., 2020) for more details.
Single Image Method:
Given a single image x drawn from an unknown distribution ρ test , we estimate the likelihood that the corruption level of the image corresponds to each of the calibration sets, and then take the corresponding weighted average of the calibrated probilities. Ideally, in the single image method we would like to obtain q i ( p max ) close to one for the calibration set whose p max distribution looks closely to the p max distribution of the test images. (i)
The probability that the p max value came from a given calibration set, making the standard assumption that the a priori likelihoods of the calibration sets are all equal, is
[Equation]
(ii)
Then
the
calibrated probability, conditional on each of the calibration sets, is given by
[Equation]
[Figure] Multiple Image Method: Given a sample of images S test = { x 1 , . . .
, x m } drawn from an un- distribution ρ test
(i) Record the corresponding model p max values, P
test = { p max ( x
) | x ∈ S test } , and compute the mean µ .
(ii) Compare to the means µ j of P CAL and ﬁnd the closest mean for the calibration sets, i =
arg min j | µ − µ j | .
Set p
[Equation]
We can use a simpler formula for the multiple image method, because with multiple samples, knowing the mean is sufﬁcient to estimate the correct calibration set.
3. P RACTICAL IMPLEMENTATION OF OUR METHOD
In Ovadia et al.
the additional methods used include (LL) Approximate Bayesian inference for the parameters of the last layer only (Riquelme et al., 2018), (LL SVI)
Mean ﬁeld stochastic variational inference on the last layer, (LL Dropout) Dropout only on the activations before the last layer.
We refer to Ovadia et al.
for more details on how each method was implemented.
All these methods ultimately use the softmax probabilities as a surrogate for the class probabilities.
The difference between the methods is how these probabilities are obtained.
The distributional shift on ImageNet used 16 corruption types with corruption intensity on a scale from 0 to 5: various forms of Noise, and Blur as well as Pixelate, Saturate, Brightness, Contrast,
Fog and Frost, etc.
See Figure S3 in (Ovadia et al., 2019).
We used the published softmax value of each of the methods from the benchmark dataset in Ovadia et al.
We selected Contrast as the corruption to use for calibration, removing it from the test set .
We calibrated our models using equally sized bins, on each of following calibration sets: { 0 } , { 0,1 } , { 0,2 } , { 0,3 } , { 0,4 } , and { 0,5 } . For instance the set 0 , 1 corresponds to clean images and respective corruption with an intensity level of 1.
Heuristically we always want clean images in our calibration set while having different shifted means as a result of increasingly corrupted images (see Figure 6).
Without the clean images, the single image method would become uncalibrated for in-distribution images as the calibration sets would have a disproportional amount of corrupted images.
For both CIFAR-10 and Imagenet, we select 5000 images for calibration, which are shared across the different calibration sets.
This means that for instance the calibration set corresponding to { 0 , 1 } contains a total of 10000 images: the selected 5000 clean images and their corrupted counterparts at an intensity level of 1.
A more sophisticated combination of calibration sets could lead to improvements.
We reproduced the ﬁgures in Ovadia et al.
with a small adjustment: we measure the ECE and Brier score just for p correct , rather than for p classk .
However this made a negligible difference to the values.
In addition, ECE can depend on the binning procedures (equally spaced or equally sized).
Equally sized bins are more effective for calibration since they reduce statistical error.
They do however lead to different bin edges on different calibration sets, which required combining the bins.
This can be done by reﬁnement (which we used here) or simple by one dimensional density estimation (Wasserman, 2006, Chapter 6). [Figure] [Figure] Ovadia
[Figure]
al Single Image Multi Image
3. D ISCUSSION OF THE RESULTS
Both our single and multi image methods improve on the benchmark widely across methods and corruption intensity levels, as shown in Figure 1 and Tables 1, and 2, which report the mean ECE scores for each model and across different corruption types, for ﬁxed corruption intensity going from 0 to 5.
The multi image method performs better than the single image with a few exceptions.
This is a natural consequence of using more images to better estimate the corruption level.
While the ensemble method remains overall the best method, the gap to other methods is signiﬁcantly reduced.
Moreover, we test the OOD detection performance. We evaluate the CIFAR-10 trained models on the
SVHN dataset (Netzer et al., 2011), in addition to MNIST trained models (for these we used rotation to form the surrogate calibration sets) on Fashion-MNIST (Xiao et al., 2017) and Not-MNIST (Xiao et al., 2017).
Ideally, the models should not be conﬁdent when presented with this completely OOD data. As we can see in Figures 7 and 8, both the single and multi images methods result in methods that are signiﬁcantly less conﬁdent when compared to the benchmark data (Ovadia et al., 2019).
We note that instead of requiring class predictions for all classes, our method only requires p max values, which is a considerably smaller data set.
Moreover, we compile these values into a histogram with 30 bins, making the added cost of our method negligible.
C ONCLUSIONS
Increasingly we are asking models trained on a given dataset to perform on out of distribution data.
Our work focused on uncertainty estimates, in particular, an estimate of the probability that our model classiﬁcation is correct.
In contrast to most deep uncertainty work, we use a purely statistical approach to reduce the calibration error of deep image classiﬁers under dataset shift.
The approach is model agnostic, so it can be applied to future models.
Previous work has shown that uncertainty estimates degrade on corrupted data, as measured by the expected calibration error.
The greater the mismatch between training data and test data, the greater the degradation of uncertainty estimates.
We overcome this limitation introducing a method which allows a given model to be better calibrated to multiple corruption intensities.
Our method works by no longer requiring that model outputs approximate class probabilities.
We add a simple extra calibration step, and detect the level of corruption of data, which allows the use calibrations tuned to the corruption level of the data. [Table] [Table] A A BLATION AND CROSS VALIDATION STUDY
We start by investigating the impact of the choice of corruption for the calibration set.
Ideally, the choice of corruption should be representative of the distribution of corruptions, so a mild corruption or a very strong corruption would give slightly worse results. At the same time, here we demonstrate that choosing a different corruption should not signiﬁcantly degrade the results. In Figure 9 we perform a cross-validation study over the choice of corruption used to generate the calibration sets (always leaving it out of the corruptions used at test time). We plot the mean and variance of the ECE across different validation corruptions types.
For CIFAR-10, both the single and multi image method are robust to the choice of validation corruption.
On ImageNet, the multi image method the improvement is consistent across the validation corruption chosen except for the dropout method.
As for the single image method the calibration at lower levels of intensity is degraded using certain corruptions, for example the glass blur corruption.
However, this seems to be caused by the strength of the corruption: the accuracy on glass blur level 1 was roughly half that of clean images.
We hypothesize that better results could be obtained by simply having the corruption strength be proportional to the loss of accuracy, as is the case of the contrast corruption (see Figure 6)
In practice, the choice of corruption for single image method should be such that the method remains calibrated for in-distribution images.
We investigate as well the impact of adding corrupted images to the calibration sets.
In order to do so, we compare the results of our proposed method and the benchmark from Ovadia et al.
(2019) with the calibration obtained from using a single calibration set with only clean images like the method proposed in (Oberman et al., 2020).
We refer to it as the Top1 binning method.
As we can see in Figure 10, if the classiﬁer is not well-calibrated, e.g., the vanilla or the LL SVI classiﬁers, there is a consistent improvement across all corruption intensity levels, with the improvement being only marginal for ImageNet.
Moreover, when the classiﬁer is well-calibrated, Top1 binning does not improve calibration (e.g. the Temp Scaling classiﬁer) or even decreases it at high levels of corruption intensity (e.g. Ensemble and Dropout).
{+Vanilla 0 1 2 3 
+}
{+Temp Scaling 0 1 2 3 
+}
{+Ensemble 0 1 2 3 
+}
{+Dropout 0 1 2 3 
+}
{+LL Dropout 0 1 2 3 
+}
LL SVI
Corruption Intensity
E C E
Ovadia et al Single Image
Multi Image Top 1 Calibration (a) ImageNet 0 1 2 3 4 5 0.
{+Vanilla 0 1 2 3 
+}
{+Temp Scaling 0 1 2 3 
+}
{+Ensemble 0 1 2 3 
+}
{+Dropout 0 1 2 3 
+}
{+LL Dropout 0 1 2 3 
+}
{+SVI 0 1 2 3 
+}
LL SVI
Corruption Intensity
[Figure]
Ovadia et al Single Image
Multi
[Figure]
Calibration [Figure] B T ABLES OF B RIER M ETRICS Table 3 and Table 4 report the mean Brier scores for each model and dataset across different corruption types, for ﬁxed corruption intensity going from 0 to 5.
The Brier scores can be computed directly from the data, without binning.
The ranking provided by the Brier scores is quite similar that provided by the ECE.
On ImageNet the only difference is Ensemble at corruption level 1.
On
CIFAR-10 there were two ranking differences.
C T ABLES OF ECE M ETRICS ACCROSS DIFFERENT CORRUPTIONS
Table 5 and Table 6 report the ECE scores for the vanilla model across different corruption types and intensities ranging from 0 to 5 for ImageNet and CIFAR-10, respectively.
Contrast is the corruption used for to form the calibration sets.
[Table] [Table] [Table]
[Table]