Bayesian OOD detection with aleatoric uncertainty and outlier exposure
Abstract
Typical Bayesian approaches to OOD detection use epistemic uncertainty.
Surprisingly from the Bayesian perspective, there are a number of methods that successfully use aleatoric uncertainty to detect OOD points (e.g. Hendryks et al. 2018).
In addition, it is diﬃcult to use outlier exposure to improve a Bayesian OOD detection model, as it is not clear whether it is possible or desirable to increase posterior (epistemic) uncertainty at outlier points.
We show that a generative model of data curation provides a principled account of aleatoric uncertainty for OOD detection.
In particular, aleatoric uncertainty signals a speciﬁc type of OOD point: one without a well-deﬁned class-label, and our model of data curation gives a likelihood for these points, giving us a mechanism for conditioning on outlier points and thus performing principled Bayesian outlier exposure.
Our principled Bayesian approach, combining aleatoric and epistemic uncertainty with outlier exposure performs better than methods using aleatoric or epistemic alone.
1. Introduction
The most typical approach to Bayesian OOD distribution detection uses epistemic uncertainty (Lakshminarayanan et al., 2017;
Malinin and Gales, 2018; Choi et al., 2018; Wen et al., 2019;
Malinin et al., 2020; Postels et al., 2020).
We have epistemic uncertainty when ﬁnite training data fails to pin down the classiﬁer’s ideal outputs in all regions of the input space (Der Kiureghian and Ditlevsen, 2009; Fox and ¨Ulk¨umen, 2011; Kendall and
Gal, 2017).
Importantly, the amount of epistemic uncertainty will vary depending on how close a given test point is to the training data.
Close to the training data, the classiﬁer’s predictive distribution is reasonably well-pinned-down and there is little epistemic uncertainty.
In contrast, far from the training data, the classiﬁer’s predictive distribution is more uncertain, and this uncertainty can be used to detect OOD data.
In contrast, aleatoric uncertainty is the irreducible output “noise” that is left over when there is no uncertainty in the parameters (e.g. because a lot of training data is available).
Some work using Bayesian epistemic uncertainty for OOD detection explicitly rejects the use of aleatoric uncertainty (Malinin and Gales, 2018; Malinin et al., 2020; Wen et al., 2019; Choi et al., 2018; Postels et al., 2020), while other work implicitly combines aleatoric and epistemic uncertainty by looking at the overall predictive entropy (Lakshminarayanan et al., 2017; Izmailov et al., 2021; Ovadia et al., 2019; Maddox et al., 2019).
Surprisingly from the Bayesian perspective, there are a large number of methods that successfully use aleatoric uncertainty alone to detect OOD points (Hendrycks and Gimpel, 2016; Liang et al.,
2017; Lee et al., 2017; Liu et al., 2018; Hendrycks et al., 2018).
In addition, many of these methods can be trained on OOD points, in a process known as outlier exposure (OE).
When suitable OOD data is available, OE leads to dramatic increases in performance (Hendrycks et al., 2018).
However, developing a principled Bayesian method using OE is diﬃcult.
In particular, a principled Bayesian formulation would involve treating the OE points as providing extra terms in the likelihood.
However, it not currently clear how to create a likelihood for outlier points.
Instead, current OE methods use a variety intuitively reasonable objectives, which have no interpretation as log-likelihoods and thus cannot be combined with Bayesian inference.
In this paper, we provide a principled account of how to incorporate aleatoric uncertainty and outlier exposure into Bayesian OOD detection methods.
In particular, we consider a model of the curation process applied during the original creation of datasets such as CIFAR10 and ImageNet.
Critically, this curation process is designed to ﬁlter out a speciﬁc set of
OOD points: data points without a well-deﬁned class label .
For simplicity, we will refer to these points as OOD for the remainder of the paper.
For instance, if we try to classify an image of a radio as cat vs dog, there is no well-deﬁned class-label, and we should not include that image in the training set.
We model curation as a consensus-formation process.
In particular, we give each the image to multiple human annotators: if the image has a welldeﬁned label (Fig. 1 left and middle), they will all agree, consensus will be reached and the datapoint will be included in the dataset.
In contrast, if the human annotators are given an image with an undeﬁned class label (Fig. 1 right), all they can do is to choose randomly, in which case they disagree, consensus will not be reached and the datapoint will be excluded from the dataset.
Critically, that random ﬁnal choice corresponds to aleatoric , not epistemic uncertainty.
If we ask a human to classify an image of a radio as cat vs dog, the issue certainly is not that the human annotator is uncertain about the radio’s degree of “cat-ness” or “dog-ness”.
The issue is that we are forcing the human to answer a fundamentally nonsensical question, and the only reasonable response is to choose randomly.
That random choice thus corresponds to aleatoric uncertainty, and thus aleatoric uncertainty can signal that the point is OOD, and has an undeﬁned class-label.
Our model gives a likelihood for being OOD (or having an “undeﬁned class-label”) in terms of the underlying classiﬁer probabilities, allowing us to incorporate outliers in principled Bayesian inference.
We ﬁnd that our approach, incorporating OE and aleatoric uncertainty with
Bayes performs better than a standard Bayesian approach without OE, and better than a standard aleatoric uncertainty based approach with OE (e.g. Hendrycks et al., 2018).
2. Background: A model for data curation
In the introduction, we brieﬂy noted that diﬀerent annotators will agree about the class label when that class label is well-deﬁned, but will disagree for OOD inputs without a well-deﬁned class-label (if only because they are forced to the label the image and the only thing they can do is to choose randomly).
Interestingly, a simpliﬁed generative model which considers the probability of disagreement amongst multiple annotators has already been developed to describe the process of data curation (Aitchison, 2020, 2021).
In data curation, the goal is to exclude any OOD images to obtain a high-quality dataset containing images with well-deﬁned and unambiguous class-labels.
Standard benchmark datasets in [Figure] image classiﬁcation have indeed been carefully curated.
For instance, in CIFAR-10, graduate student annotators were instructed that “It’s worse to include one that shouldn’t be included than to exclude one”, then Krizhevsky et al.
(2009) “personally veriﬁed every label submitted by the annotators”.
Similarly, when ImageNet was created, Deng et al.
(2009) made sure that a number of Amazon Mechanical Turk (AMT) annotators agreed upon the class before including an image in the dataset.
Aitchison (2021) proposes a generative model of data curation that we will connect to the problem of OOD detection.
Given a random input, X , drawn from P ( X ), a group of
S annotators (indexed by s ∈ { 1 , . . .
, S } ) are asked to assign labels Y s ∈ Y to X , where
Y = { 1 , . . .
, C } represents the label set of C classes.
If X is OOD, annotators are instructed to label the image randomly.
We assume that if the class-label is well-deﬁned, suﬃciently expert annotators will all agree on the label, so consensus is reached, Y 1 =
Y 2 = · · · = Y S , and the image will be included in the dataset.
Any disagreement is assumed to arise because the image is OOD, and such images are excluded from the dataset.
In short, the ﬁnal label
Y is chosen to be Y 1 if consensus was reached and Undef otherwise (Fig.
2B).
[Equation]
From the equation above, we see that
Y ∈ Y ∪ { Undef } , that is, Y could be any element from the label set Y if annotators come to agreement or Undef if consensus is not reached.
Suppose further that all annotators are IID (in the sense that their probability distribution over labels given an input image is the same).
Then, the probability of Y ∈ Y can be
[Equation]
written as
[Equation]
where we have abbreviated the single-annotator probability as p y ( X )
=
P ( Y s =
y | X, θ ).
When consensus is not reached (noconsensus), we have:
[Equation]
Notice that the maximum of Eq.(3) is achieved when the predictive distribution is uniform: p y = 1 /C, ∀ y ∈ Y , as can be shown using a Lagrange multiplier γ to capture the normalization constraint,
[Equation]
The value of p y with maximal L is independent of y , so p y is the same for all y ∈ Y , and we must therefore have p y = 1 /C .
In addition, the minimum of zero is achieved when one of the C classes has a probability p y ( X )
= 1.
Therefore, an input with high predictive (aleatoric) uncertainty is, by deﬁnition , an input with a high probability of disagreement amongst multiple annotators, which corresponds to being OOD.
3. Methods
We are able to form a principled log-likelihood objective by combining Eq.(2) for inputs with a well-deﬁned class-label (denoted by D in ) and Eq.(3) for OOD inputs without a well-deﬁned class label (denoted by D out ).
However, this model was initially developed for cold-posteriors (Aitchison, 2021) and semi-supervised learning (Aitchison, 2020) where the noconsensus inputs were not known and were omitted from the dataset.
In contrast, and following Hendrycks et al.
(2018), we use proxy datasets for OOD inputs, and explicitly maximize the probability of inputs from those proxy datasets having being OOD (Eq. 8).
Importantly, now that we explicitly ﬁt the probability of an undeﬁned class-label, we need to introduce a little more ﬂexibility into the model.
In particular, a key issue with the current model is that more annotators, S , implies a higher chance of disagreement hence implying more OOD images.
Thus, arbitrary choices about the relative amount of training data with well-deﬁned and undeﬁned class-labels might cause issues.
To avoid any such issues, we modify the undeﬁned-class probability by including a base-rate or bias parameter, c , which modiﬁes the log-odds for well-deﬁned vs undeﬁned class-labels.
In particular, we deﬁne the logits to be,
[Equation]
where p y ( X ) is the single-annotator probability output by the neural network.
[Equation]
with c = 0, this reverts to Eq.(2) and (3), while non-zero c allow us to modify the ratio of well-deﬁned to undeﬁned class-labels to match that in the training data.
Of course, we do not have the actual datapoints that were rejected during the data curation process, so instead D out is a proxy dataset (e.g. taking CIFAR-10 as D in , we might use downsampled ImageNet with 1000 classes as D out ).
The objective is,
[Equation]
where λ represents the relative quantity of inputs with undeﬁned to well-deﬁned class-labels.
We use λ = 1 both for simplicity and because the inclusion of the bias parameter, c , should account for any mismatch between the “true” and proxy ratios of inputs with well-deﬁned and undeﬁned class-labels.
In addition, we use a ﬁxed value of S = 10 as is suggested by
Aitchison (2021) and we learn c via backpropagation during training.
Lastly, since our objective is a well-deﬁned likelihood function that jointly models D in and D out , we can easily turn our model into a fully Bayesian one by adding a prior distribution on the neural network parameters, θ , and then perform approximate inference approaches (e.g. stochastic gradient Markov chain Monte Carlo) to estimate the posterior distribution over θ .
The use of Bayesian inference in our approach allows us to incorporate both epistemic uncertainty and aleatoric uncertainty when detecting OOD samples and we will show in next section that combining two types of uncertainty together can lead to performance superior than using either of them alone.
4. Results
In this section, we demonstrate the eﬀectiveness of our approach via large scale image classiﬁcation experiments with CIFAR-10 and CIFAR-100 as D in , and downsampled ImageNet
[Table] [Table]
D test is the testing out-of-distribution dataset.
( and standard error computed over 6 runs of
) as our training D out .
We considered two diﬀerent baselines, in addition to our method.
First, in “BNN”, we followed the usual OOD detection procedure for Bayesian neural networks in training on our in-distribution dataset, D in , using SGLD, and ignored our OOD dataset, D out , as it is not clear how to incorporate these points into a classical Bayesian neural network.
Second, in “OE”, we used the non-Bayesian method of Hendrycks et al.
(2018), which trains on D in , and incorporates an objective that encourages uncertainty on
D out .
Our method uses a BNN, as in the BNN baseline, but additionally trains on D out using the log-likelihood from Eq.(3) to increase uncertainty on those points.
The network architecture is chosen to be a 40-2 Wide Residual Network (Zagoruyko and Komodakis, 2016) for all experiments.
OE was trained directly using the code from Hendrycks et al.
For BNN and our approach, we used Cyclical Stochastic Gradient MCMC (Zhang et al., 2020) to perform approximate inference over the network parameters.
In addition, we used a temperature of 0.1 on the likelihood for the baseline BNN, so as to match S =in the labelled likelihood for our model (Eq. 2 Aitchison, 2020).
In addition, the OOD score is chosen to be the predictive distribution’s total uncertainty for all experiments (see
Appendix A.3).
Results
Broadly, we found that our approach gave superior performance to the OE and
BNN using FPR95 on a wide range of test datasets D test that the model was not trained on (Table 1).
5. Conclusion
We developed a likelihood for undeﬁned class-label samples (a subset of OOD points), and used it to integrate OE methods within principled Bayesian inference.
The resulting
Bayesian OE method gave superior performance to other methods, including pure aleatoric uncertainty and Bayesian methods without OE.
Appendix A. Experiment details A.1. Test datasets
At test time, we evaluate the models’ OOD detection ability on a number of OOD datasets that the model was not trained on, as proposed in Hendrycks and Gimpel (2016) and as implemented by Hendrycks et al.
Isotropic zero-mean Gaussian noise with σ = 0 .2.
Rademacher noise where each dimension is − 1 or 1 with equal probability.
Blobs data of algorithmically generated amorphous shapes with deﬁnite edges 4.
Texture data (Cimpoi et al., 2014) of textural images in the wild. 5.
SVHN (Netzer et al., 2011) which contains 32x32 colour images of house numbers.
A.2. OOD metric
OOD detection is in essence a binary classiﬁcation problem.
It is therefore sensible to use metrics for binary classiﬁcation to evaluate a model’s ability to detect OOD inputs.
In particular, we adopt the false positive rate at N % true positive rate (FPR N ), which computes the probability of an input being misclassiﬁed as having an undeﬁned class-label (false positive) when at least N % of the true inputs with undeﬁned class-labels are correctly detected (true positive).
In practice, we would like to have a model with low FPR N % since an ideal model should detect nearly all inputs with undeﬁned class-labels while raising as few false alarms as possible.
In our experiments, we let N = 95.
A.3. OOD score
At test time, to distinguish between OOD and in-distribution examples, we need a score that measures the model’s uncertainty.
There are several model-speciﬁc choices.
In our model one can choose to use the OOD probability (Eq. 3) as the score.
Hendrycks and Gimpel (2016); Hendrycks et al.
(2018) use the negative maximum softmax probability (Hendrycks and Gimpel, 2016).
However, to ensure a fair comparison, we used one metric that makes sense for all methods considered, the total uncertainty (Depeweg et al., 2018).
The total uncertainty is the entropy of the predictive distribution, marginalising over uncertainty in the neural network parameters, H [ p ( y | x ∗ )], which equals the sum of aleatoric uncertainty and epistemic uncertainty.
Note that the total uncertainty from OE only contains aleatoric uncertainty since the model is fully deterministic.
In contrast, the standard BNN and our
BNN approach both have aleatoric and epistemic uncertainty.
The key diﬀerences is that in our model, the aleatoric uncertainty is shaped by outlier exposure, whereas in a standard
BNN, the aleatoric uncertainty is determined solely by the in-distribution data.