I MPROVING CONFIDENT - CLASSIFIERS FOR O UT - OFDISTRIBUTION DETECTION
Anonymous authors
Paper under double-blind review
A BSTRACT
Discriminatively trained neural classiﬁers can be trusted, only when the input data comes from the training distribution (in-distribution).
Therefore, detecting outof-distribution (OOD) samples is very important to avoid classiﬁcation errors.
In the context of OOD detection for image classiﬁcation, one of the recent approaches proposes training a classiﬁer called “conﬁdent-classiﬁer” by minimizing the standard cross-entropy loss on in-distribution samples and minimizing the KL divergence between the predictive distribution of OOD samples in the low-density “boundary” of in-distribution and the uniform distribution (maximizing the entropy of the outputs).
Thus, the samples could be detected as OOD if they have low conﬁdence or high entropy.
In this paper, we analyze this setting both theoretically and experimentally.
We also propose a novel algorithm to generate the “boundary” OOD samples to train a classiﬁer with an explicit “reject” class for
OOD samples.
We compare our approach against several recent classiﬁer-based
OOD detectors including the conﬁdent-classiﬁers on MNIST and Fashion-MNIST datasets.
Overall the proposed approach consistently performs better than others across most of the experiments.
I NTRODUCTION
Discriminatively trained deep neural networks have achieved state of the art results in many classiﬁcation tasks such as speech recognition, image classiﬁcation, and object detection.
This has resulted in deployment of these models in real life applications where safety is paramount (e.g., autonomous driving).
However, recent progress has shown that deep neural network (DNN) classiﬁers make overconﬁdent predictions even when the input does not belong to any of the known classes (Nguyen et al. (2015)).
This follows from the design of DNN classiﬁers that are optimized over in-distribution data without the knowledge of OOD data.
The resulting decision boundaries are typically “unbounded/open” as shown in Figure 1a resulting in over-generalization (Spigler (2019),
Scheirer et al.
There have been many approaches proposed to address this problem under the umbrella of OOD detection 1 .
Lee et al.
(2018a) propose to explicitly train a classiﬁer using the OOD samples generated by a GAN (Goodfellow et al.
(2014a)).
They empirically try to show that, for effective OOD detection, the generated OOD samples should follow and be close to the low-density boundaries of in-distribution, and the proposed GAN training indeed tries to do that.
A multi-class softmax
DNN classiﬁer is trained with in-distribution samples to minimize the standard cross-entropy loss (minimizing the output entropy) and the generated OOD samples are trained to minimize a KL loss that forces the classiﬁer’s predictive distribution to follow a uniform one (maximizing the output entropy).
The resulting classiﬁer is called a “conﬁdent-classiﬁer”.
One can then classify a sample as being in or out-of distribution based on the maximum prediction probability or the entropy of the output.
Sricharan & Srivastava (2018) also follow a similar approach with slight modiﬁcations.
Contribution.
One of the key assumptions in Lee et al. (2018a) and Sricharan & Srivastava (2018) is that the effect of maximizing the entropy for OOD samples close to the low-density boundaries of in-distribution might also propagate to samples that are far away from in-distribution.
This training is expected to result in “bounded/closed” regions in input space with lower entropy over the in-distribution, and the rest of the region (corresponding to OOD), with higher entropy.
The ideal
[Figure]
decision boundary in such a scenario would be as shown in Figure 1b.
We ﬁnd that even though such a solution exists, the proposed training algorithm is unlikely to reach it.
We justify this both theoretically and experimentally for a ReLU network (network with ReLU activation units) that was indeed used in Lee et al.
(2018a).
Assuming training with OOD samples close to the in-distribution boundary, we ﬁnd that having an explicit reject class for OOD samples results in a solution close to the one depicted in Figure 1b.
Therefore we propose to use such a classiﬁer instead.
We give intuitive arguments to justify the proposal.
This forms the ﬁrst key contribution of our paper.
Moreover, with toy experiments (refer to section D in appendix) on low-dimensional synthetic data, we analyze if GAN can indeed produce samples that can follow the low-density boundaries of indistribution.
We ﬁnd that, even though GAN produces samples close to the low-density boundaries of in-distribution, it is unable to cover the whole boundary, thus resulting in a sub-optimal OOD detector when trained on such samples.
We therefore propose a novel algorithm to generate “boundary” OOD samples using a manifold learning network, (e.g., variational auto-encoder (VAE)) and show that the generated samples are diverse and cover the in-distribution boundaries better than the method proposed in Lee et al.
(2018a).
The resulting classiﬁer trained with those samples improves the OOD detection results.
This forms the second key contribution of our paper.
B ACKGROUND
Lee et al.
(2018a) propose a joint training of GAN and a classiﬁer based on the following objective:
[Equation]
where (b)+(c) is the modiﬁed GAN loss and (a)+(b) is the classiﬁer loss ( θ is the classiﬁer’s parameter) called the conﬁdence loss.
The difference from the regular GAN objective is the additional KL loss in (1), which when combined with the original loss, forces the generator to generate samples in the low-density boundaries of the in-distribution ( P in ( x ) ) space.
β is a hyper-parameter that controls how close the OOD samples are to the in-distribution boundary.
For the classiﬁer, the KL loss pushes the OOD samples generated by GAN to produce a uniform distribution at the output, and therefore have higher entropy.
This enables one to detect OOD samples based on the entropy or the conﬁdence at the output of the classiﬁer.
W HY MINIMIZING CONFIDENCE LOSS IS INSUFFICIENT FOR OOD DETECTION
Let f : R d → R K be the neural network function that maps input in R d to K output classes (input to the softmax layer).
Let f k : R d
→ R be the function that maps the input to output for a speciﬁc class k ∈ { 1 , 2 , 3 ...K } .
For a neural network with afﬁne activations (e.g., ReLU and Leaky ReLU), each f k is a continuous piece-wise afﬁne function over a ﬁnite set of polytopes, { Q 1 , Q 2 , · · · , Q M } such that R d
=
(cid:83) Ml =1 Q l , as described in Croce & Hein (2018).
This means that each f k is afﬁne within each Q l ( l ∈ { 1 , 2 , 3 ...M } ).
If the input space is R d , some of these polytopes stretch to inﬁnity (grow without bounds).
Let Q ∞ l ≡ Q l denote these “inﬁnity polytopes”.
The choice of the neural network structure and the weights deﬁne f k ’s. Figure 2a illustrates these polytopes and f k ’s for a simple 3-class ReLU classiﬁer, where the input space is R .
In this example, there are 4 polytopes in which Q ∞ 1 and Q ∞ 4 stretch to inﬁnity.
[Figure]
’s in Q ∞ 4 ( k ∗ = { 2 , 3 } ).
For sufﬁciently small x , there is a unique k ∗ = 3 in Q ∞ 1 .
Hein et al.
(2019) mathematically show that a ReLU classiﬁer (with softmax output) produces arbitrarily high conﬁdence predictions (approaching 1) far away from the training data in almost all directions on an unbounded input space.
This happens over Q ∞ l ’s. Their results are summarized as follows.
For any x ∈ R d , there exists a β l > 0 such that for all α l ≥ β l , α l x ∈ Q ∞ l .
Let f lk ( x ) =
(cid:104) v lk , x (cid:105)
+ a lk be the piece-wise afﬁne function for class k over Q l .
Let k ∗ = arg max k (cid:104) v lk , β l x (cid:105) 2 .
Then, as α l → ∞ , the conﬁdence for input α l x for class k ∗ becomes arbitrarily high if k ∗ is unique.
i.e,
[Equation]
But if there are multiple k ∗ ’s, arbitrarily large conﬁdence values cannot be obtained far away from the in-distribution in the direction of x .
For instance, as shown in Figure 2b 3 , for Q ∞ 4 , k ∗ = { 2 , 3 } and therefore arbitrarily high conﬁdence predictions cannot be achieved as α l → ∞ .
Having multiple k ∗ ’s for every Q ∞ l is highly unlikely, given that we are dealing with ﬂoating point numbers and also that it is not explicitly enforced during training.
Therefore, arbitrarily high conﬁdence values far away from the in-distribution are likely inevitable.
The above analysis is for the case where the input domain is unbounded ( R d ).
For bounded domains (for example, [0 , 1] d for images), as pointed out in Hein et al.
(2019), since we cannot let α l → ∞ , the above analysis cannot be directly applied to get arbitrary high conﬁdence values.
However the above technique in principle can be applied to increase the prediction conﬁdence for samples far away from the in-distribution.
Hein et al.
(2019) conduct experiments to support the claim.
The theoretical analysis of which can be done as follows.
Let Z d represent the bounded input domain.
Similar to the unbounded case, let Q ∞ l denote “inﬁnity polytopes” that stretch till the bounds of the input domain Z d .
For any x ∈ Z d , there exists a β l > 0 such that for all α l ≥ β l , α l x ∈ Q ∞ l .
Let f lk ( x ) =
(cid:104) v lk , x (cid:105)
+ a lk be the piece-wise afﬁne function for class k over Q l .
Let k ∗ = arg max k (cid:104) v lk , β l x (cid:105) .
Then, as α l increases, the conﬁdence for input
α l x for class k ∗ keeps increasing until the bounds of the domain is reached if k ∗ is unique.
If f k ∗ ( β l x ) >> f k ( β l x ) ∀ k (cid:54) =
k ∗ (ignoring the effect of bias term for simplicity), the conﬁdence for input α l x for the class k ∗ is very high.
Therefore, even for the case of bounded input space, one can obtain conﬁdence predictions for
OOD samples high enough for it to be considered as in-distribution samples.
Corollary.
The higher the conﬁdence of the output, the lower is the entropy.
Hence a direct corollary of Hein et al.
(2019)’s result is that the entropy of the classiﬁer output for data far away from the in-distribution data in all directions would almost always be arbitrarily low (approaching 0) like the in-distribution samples.
This makes it almost impossible to detect OOD samples based on the conﬁdence or the entropy of the classiﬁer outputs.
For the case of bounded input domain, as one can increase the prediction conﬁdence for OOD samples far from the in-distribution, the entropy of classiﬁer output also decreases making those OOD samples to be classiﬁed as in-distribution samples.
Therefore, the approaches in Lee et al.
(2018a)
and Sricharan & Srivastava (2018) would not be applicable.
A DDING AN EXPLICIT “ REJECT ” CLASS
When OOD samples are generated close to the in-distribution and follow its low-density boundaries as proposed in Lee et al.
(2018a)
and Sricharan & Srivastava (2018), we recommend adding an explicit reject class for OOD samples instead of minimizing the loss in Eq.1(b).
Let the resulting classiﬁer be called the reject-classiﬁer.
By adding an explicit reject class, our goal is to obtain a decision boundary close to the ideal decision boundary shown in Figure.1b, where the decision boundary of a K + 1 classiﬁer divides the input space into regions such that the in-distribution region is classiﬁed as one of the ﬁrst K classes and the rest of the region as the K + 1 th class, i.e., the reject class.
The intuition on how such a decision boundary can be obtained is as follows.
The arbitrarily high conﬁdence predictions happen in polytopes that stretch to inﬁnity (or stretch till the bounds of input space in case of bounded input space).
Each of the “inﬁnity polytopes” has its own class (or classes), k ∗ (or k ∗ ’s) where high conﬁdence predictions occur.
If adding an explicit “reject” class results in k ∗ = reject-class for all the “inﬁnity polytopes” (i.e there is only one k ∗ ), the arbitrarily high conﬁdence predictions would only happen at the reject class for OOD samples far-off from training data.
Therefore, these samples will be detected as OOD.
We argue that in reject-classiﬁer training, since we explicitly maximize the prediction conﬁdence of K + 1 th -class for boundary OOD samples, we expect the same effect to persist for OOD samples far from the in-distribution as well (i.e., k ∗ =
K + 1 ) resulting in close to ideal decision boundaries depicted in
Figure.1b.
This claim is supported by our experiments on a toy dataset (Figure. 5) and the superior performance of the reject-classiﬁer over the conﬁdent-classiﬁer on MNIST and Fashion MNIST datasets (Table. 1).
Note that how close the resulting decision boundary of the reject-classiﬁer is to the ideal one depends on how well the OOD samples follow the in-distribution boundary.
We ﬁnd that the method proposed in Lee et al.
(2018a) to generate boundary OOD samples is not diverse enough as evidenced by experiments shown in the appendix (Section. D).
Therefore we propose a novel approach for boundary OOD sample generation which is described in the next section that results in better boundary OOD samples that cover the in-distribution boundary quite effectively.
This evident from our experiments described in the appendix (Section.
D).
Lee et al.
(2018a) indeed experiment with adding an explicit reject class instead of using a conﬁdentclassiﬁer, but the results are found to be worse.
But this is because instead of using the boundary
OOD samples they use another natural image dataset called “seen OOD” similar to Hendrycks et al.
(2019) to train the classiﬁer.
However for images, it is difﬁcult to represent the entire OOD space with a small number of samples such methods may not perform that well.
Moreover as pointed out in both Lee et al.
(2018a)
and Hendrycks et al.
(2019), as these “seen OOD” samples aren’t diverse, when used to train a reject classiﬁer they can overﬁt to these training OOD samples.
However we use boundary OOD samples that can guide the decision boundary of the classiﬁer to be bounded around the in-distribution regions as depicted in Figure.
1b
Note that both the reject-classiﬁer and the conﬁdent-classiﬁer use boundary OOD samples for training.
The conﬁdent-classiﬁer tries to equalize f k ’s for boundary OOD samples (i.e., maximize the en-
[Figure]
tropy of output predictions) and expect this to persist over OOD samples far from the in-distribution as well (i.e., have multiple k ∗ ’s).
The reject-classiﬁer on the other hand maximizes the prediction conﬁdence of K + 1 th for boundary OOD samples and expects it to persists over OOD samples far from the in-distribution (i.e., have a single k ∗ at k = K + 1 ).
conﬁdence f k .
In the unbounded case, for a conﬁdent-classiﬁer, while it is proven that one can almost always ﬁnd arbitrarily high conﬁdence regions far from the in-distribution, for a reject classiﬁer we can still expect those OOD samples to be classiﬁed as belonging to the K + 1 th class.
In the bounded case too as shown previously, one can obtain decreasingly low entropy regions far from the in-distribution for the conﬁdent classiﬁer whereas for the reject-classiﬁer it is similar to the unbounded case.
As evident from the experimental results in Figure.
5 we can indeed ﬁnd OOD samples with low entropy for the conﬁdent-classiﬁers without stretching to inﬁnity, whereas for the reject-classiﬁer, all those OOD samples far away from the in-distribution are correctly classiﬁed as OOD.
The results in Table.further reinforces the superiority of the reject-classiﬁer over the conﬁdent classiﬁer.
O UT - OF - DISTRIBUTION S AMPLE G ENERATION
The proposed approach leverages the following generic assumptions (Cayton (2005), Narayanan &
Mitter (2010), Rifai et al. (2011)) that hold true for a wide range of problems, primarily for image data, which is the data used to validate our approach.
The manifold hypothesis states that the higher dimensional real-world data in the input space is likely concentrated on a much lower-dimensional sub-manifold.
The multi-class manifold hypothesis states that, if data contains multiple classes, different classes correspond to disjoint sub-manifolds separated by low-density regions in the input space.
To fully cover the “boundary” of in-distribution, we identify two categories of OOD samples that are to be generated.
As shown in Figure 3, Type I) are the OOD samples that are close but outside the in-distribution sub-manifolds; Type II) are the OOD samples that are on the sub-manifolds but close to the “boundary” of the in-distribution.
5. OOD SAMPLES OUTSIDE THE DATA MANIFOLD
These samples are obtained by adding small perturbations to in-distribution samples that are concentrated on the manifold.
These perturbations should be added in directions such that the resulting samples should fall outside the manifold.
The directions locally normal to the data-supporting manifold can be thought of as the directions that are less likely to contain in-distribution samples and the tangent directions as the more likely ones.
Therefore we add perturbations in the normal directions to get OOD samples.
Deep generative models such as VAEs (Kingma & Welling, 2013) and GANs (Goodfellow et al., 2014b) can model the data manifold of observations x ∈ X through corresponding latent variables z ∈ Z via a mapping function g : Z → X as x = g ( z ) .
With a choice of reasonably lower
[Figure]
dimensional z and a ﬂexible generative function g , the model can efﬁciently represent the true data manifold.
Following the multi-class manifold hypothesis , we use a conditional generative model that is conditioned over the class labels.
For our experiments, we use a conditional variational autoencoder (CVAE).
Let h : X → Z and g : Z → ˆ X denote the encoder and decoder functions of CVAE respectively.
The tangent space of the manifold at a point x ∈ X is given by the column space of the Jacobian
[Equation]
Let N ( x ) denote the null-space of J T ( x ) (left null space of J ( x ) ).
Then the basis vectors of N ( x ) span the normal bundle of the manifold at x .
Let v ( x ) ∼ N ( x ) be a randomly sampled unit vector from N ( x ) , then the perturbed sample is given by,
[Equation]
where β ∈ R is a hyper-parameter that controls how far the perturbed sample is from the indistribution point.
In our experiments, we use a stochastic β that is uniformly sampled from in the range [0 . 1 , 1 . 0] .
As discussed before, for better OOD detection, the boundary samples generated should be diverse; because the proposed approach generates OOD samples by randomly perturbing every in-distribution training sample, the diversity of the generated samples is ensured.
This is visually apparent from the experimental results on a 3D-dataset shown in section D of appendix.
Figure 4 illustrates the perturbed samples for MNIST and Fashion MNIST datasets.
One can observe that the perturbations added mostly modify the background pixels than the object pixels.
This is because the normal directions to the manifold mostly represent least variance components of the image.
5. OOD SAMPLES ON THE DATA MANIFOLD
These are the samples that are in the low-density regions of the input space but close to the indistribution boundaries on the manifold.
For a variational auto-encoder, the aggregate posterior q ( z ) (Makhzani et al., 2015) is given by,
[Equation]
[Equation]
) where p in ( x ) is the probability density function of in-distribution and q ( z / x ) is the approximate posterior.
Assuming a smooth decoder, the high-density regions in the aggregate posterior can be thought of as corresponding to densely populated regions in the input space, and the input space density would gradually decrease as we sample away from the high-density regions in the aggregate posterior.
Therefore the in-distribution boundary on the manifold can be approximated by regions at a distance away from the high-density areas where the density dips below a certain threshold.
For our experiments, we approximate q ( z ) with a uni-modal Gaussian distribution whose mean
ˆ µ and co-variance ˆΣ are estimated using the encoder mappings of in-distribution samples.
We use Mahalanobis distance as a criterion to determine the distance from the mean to sample and generate the required OOD samples.
Let r be the Mahalanobis distance from the mean of q ( z ) that encompasses 95% of the training data.
The OOD samples are generated by decoding the uniformly sampled samples from the latent space over the surface of a hyper-ellipsoid (Rubinstein, 1982) deﬁned by 5, where ˆ µ z and ˆΣ z are the mean and co-variance estimates of q ( z ) , respectively.
[Equation]
It is fair to assume a uni-modal Gaussian distribution for q ( z ) as we ﬁt a Gaussian per class.
Moreover, a substantial gain in the ODD detection results when the classiﬁer is trained with these samples can also be taken as evidence pointing towards the validity of such an assumption.
The generated OOD samples described in 5.1 and 5.2 are then used to train an n + 1 class softmax classiﬁer, where the n + 1 th class represents the OOD class.
E XPERIMENTS
Experiments 5 are divided into 2 sections; the ﬁrst section explains the toy experiments on a lowdimensional dataset to support our theoretical analysis of the conﬁdent-classiﬁer, the second section gives details of OOD detection experiments on MNIST and Fashion MNIST ((Xiao et al., 2017)) using the proposed method.
6. L IMITATIONS OF C ONFIDENT -C LASSIFIERS
In these experiments, the input space is R 2 and the in-distribution consists of 2-classes.
The samples for each of these classes are generated by sampling from 2 Gaussians with identity co-variances and means (-10, 0) and (10, 0) respectively, on the Cartesian coordinates.
Anything outside 3 standard deviations (Mahalanobis distance) from the in-distribution means is considered OOD.
The architecture of the neural network used is similar to the one used in Lee et al.
(2018a), which is a
ReLU-classiﬁer with 2 fully-connected hidden layers with 500 neurons each.
Following the case in Lee et al.
(2018a), for training, OOD samples are generated close to the indistribution as shown in Figure 5a.
For testing, OOD samples are uniformly sampled from a 2D box [ − 50 , 50] 2 excluding the in-distribution regions.
[Figure]
From Figure 5b, we observe that the ReLU-classiﬁer trained to optimize conﬁdence loss results in highly conﬁdent predictions for many OOD samples far from the in-distribution data.
This renders the classiﬁer ineffective at classifying the in and out of distribution samples based on the maximum prediction score (conﬁdence) or the entropy of the output.
However, from Figure 5c, for a classiﬁer trained with explicit reject class, the test OOD samples are indeed classiﬁed as OOD.
This supports the aforementioned intuitions in 4.
Note that these are not the results speciﬁc to a certain architecture of the neural network.
Experiments with different hyper-parameters such as the number of hidden neurons, changing input dimensions, using sigmoid activation functions instead of ReLU lead to similar results.
We remark however that for sigmoid networks, the results were not as extreme (in terms of the number of OOD samples with high-conﬁdence) as for ReLU networks.
This is understandable because sigmoid activation outputs will not produce arbitrarily large values, unlike the ReLU counterparts.
6. MNIST AND F ASHION MNIST EXPERIMENTS
We validated our approach on MNIST and Fashion MNIST as in-distribution datasets and several other OOD datasets.
For all MNIST as in-distribution experiments, we use a CVAE with a latent dimension of 8, and for Fashion MNIST, the latent dimension is set to 10.
We compare our approach against the recent classiﬁer-based OOD detectors such as conﬁdent-classiﬁer, ODIN and Mahalanobis distance-based approach without feature ensemble (MD) 6 .
The architecture for both CVAE and the classiﬁer used are shown in the appendix.
Both the networks are trained till convergence.
6.2. OOD DATASETS
MNIST is used as an OOD dataset for Fashion MNIST as in-distribution, and vice-versa.
For
MNIST 0-4 experiment, we use images in class 0 through 4 as in-distribution and class 5 throughas OOD.
We use both character datasets and noise generated images as OOD datasets.
The character datasets are Omniglot
(Lake et al., 2015), EMNIST-letters (Cohen et al., 2017) and NotMNIST (Bulatov, 2011).
The noise generated images are described below.
Gaussian noise includes gray-scale images, where each pixel is sampled from an independent normal distribution with 0.5 mean and unit-variance.
Uniform noise includes gray-scale images where each pixel is sampled from an independent uniform distribution in the range [0 , 1] .
Sphere OOD contains images sampled from the surface of a 784 dimensional hyper-sphere centered at the origin with a radius equal to the maximum Euclidean distance of in-distribution samples from the origin and reshaped to 28 × 28 .
This is used to show the effectiveness of our approach not only on the datasets that are restricted to a ﬁnite range such as [0 , 1] d for images in [0 , 1] d but also for a general case of R d .
6.2. E VALUATION METRICS FOR OOD DETECTION
We experimented with two different metrics as OOD score to determine if the given input sample is in or out of distribution.
OOD class probability is the n + 1 th class prediction probability.
Indistribution max probability is the maximum prediction probabilities of the in-distribution classes.
A higher (lower) OOD class probability (in-distribution max probability) indicates a higher probability of a sample being OOD.
Except for MNIST 0-4 experiments, we ﬁnd that the former metric gives the best results.
We report only the best score in Table 1.
We use the area under the ROC curve (AUROC ↑ ), the area under the precision-recall curve (AUPR ↑ ), the false positive rate at 95% true positive rate (FPR95 ↓ ) and the detection error as the metrics for evaluation.
These metrics are commonly used for evaluating OOD detection methods (Lee et al., 2018a;
Hendrycks et al., 2019).
The details of which are in the appendix.
6.2. D ETECTION R ESULTS
Table 1 compares our approach with other approaches for experiments on MNIST and Fashion
MNIST as in-distribution datasets.
Since the classiﬁer is trained with OOD samples, there is a possibility of reduction in the classiﬁcation accuracy of in-distribution classes in comparison to training without OOD class.
We therefore report classiﬁcation accuracy of a classiﬁer trained with and without OOD samples.
We ﬁnd that there is no signiﬁcant change in accuracy.
Training our
[Table] [Table]
requires tuning hyper-parameter such as β from Eq.
4, OOD class weight, and learning rate.
The hyper-parameters were chosen based on the in-distribution classiﬁcation accuracy and the AUROC of the validation generated OOD samples and the random noise datasets.
For all our experiments we use a stochastic β uniformly sampled in the range [0.1, 1], OOD class weight is set to 0.1, while the weights for the rest of the classes is set to 1.0, and Adadelta (Zeiler, 2012) is the optimizer used with learning rates of 0.1 and 0.01 for Fashion-MNIST and MNIST experiments, respectively.
We do not tune the hyper parameters per OOD dataset unlike ODIN and Mahalanobis distance-based approaches, where the perturbation magnitude is tuned per OOD dataset.
Even without this advantage, our method still performs better than these baselines for most of the OOD datasets.
We would like to remark that our approach gives good OOD detection results consistently on all the OOD datasets used unlike the baselines compared.
This indicates that our approach is robust to change in OOD datasets.
C ONCLUSION
We have shown in the paper that the conﬁdent-classiﬁer almost always has OOD samples that produce high conﬁdence outputs (in the contexts described earlier).
We provided empirical evidence that favor using an explicit “reject” class instead.
However, the ODD detection capabilities of a reject-classiﬁer depend on the extent to which the generated OOD samples follow the low-density boundaries of in-distribution.
We also propose a novel algorithm for generating “effective” OOD samples for training an n + 1 -class classiﬁer for OOD detection and the results for most of the experiments on gray-scale datasets are consistently better for our approach in comparisons to other methods compared.
For future research, we would like to investigate its effectiveness on non-grayscale datasets such as CIFAR and TinyImageNet.
However we would like to point out that the nullspace calculation for colored images is computationally quite expensive, hence requires a larger compute (refer appendix E).
A A PPENDIX B R ELATED W ORK
There have been many approaches in the literature proposed to address the problem of OOD detection in the context of image data.
Most of the successful ones are either generative ((Pidhorskyi et al., 2018;
Wang et al., 2017; Ren et al., 2019)) or classiﬁer-based approaches (Hendrycks & Gimpel, 2016; Hendrycks et al., 2019; DeVries & Taylor, 2018; Liang et al., 2018; Lee et al., 2018b).
Generative approaches either explicitly or implicitly estimate the input density or use reconstruction error as a criterion to decide if input belongs to OOD.
Classiﬁer-based approaches, on the other hand, incorporate OOD detection as a part of the classiﬁer network.
The approach proposed in this paper belongs to the latter category.
Therefore we limit our related work discussion to only classiﬁer-based approaches.
Typical discriminatively trained classiﬁers that model the conditional probability P ( y | x ) without any additional constraints, by deﬁnition can make reliable classiﬁcation decisions only on indistribution data.
For out-of-distribution data, the classiﬁer output is arbitrary.
Moreover, any meta information from the output of the classiﬁer or the features learned are also conditioned on the data belonging to in-distribution.
Therefore this information in-principle cannot be used to ascertain if the input is in or out of distribution.
However, most of the recent approaches ((Hendrycks & Gimpel, 2016; Lee et al., 2018b; Liang et al., 2018; DeVries & Taylor, 2018))in the literature follow this approach.
Hendrycks & Gimpel (2016) propose a baseline approach to detect OOD inputs, called max-softmax by thresholding the maximum softmax output of a pre-trained classiﬁer.
Liang et al.
(2018) improve upon this using temperature scaling (ODIN, (Guo et al., 2017)) and adding input perturbations.
The assumptions is that these changes result in larger separation between in and out of distribution data in terms of their output predictions.
Lee et al.
(2018b) propose an approach based on the assumption that the class-conditional features of a softmax classiﬁer follow a Gaussian distribution.
Therefore,
Mahalanobis distance (MD) from the mean of the Gaussian is used as a score to detect OOD.
This is then combined with input perturbations similar to ODIN to enhance the OOD detection results.
This method obtains state-of-the-art results on most of the baseline datasets used in OOD detection literature.
Despite good results, the method can be seen as OOD detection on feature space rather than pixel space not conforming to the usual deﬁnition of OOD (By deﬁnition, the in-distribution, p in ( x ) is deﬁned for x ∈ X in pixel space, and hence OOD is also deﬁned in the same space).
Hence the effectiveness of the method highly depends on the features learned by the classiﬁer, and also there is no guarantee that the optimization algorithm forces the features to follow a Gaussian distribution.
Hendrycks et al.
(2019) propose to train a classiﬁer with a conﬁdence loss where OOD data is sampled from a large natural dataset.
Hein et al.
(2019) also follow a similar approach using a conﬁdence loss and uniformly generated random OOD samples from the input space.
In addition, they not only minimize the conﬁdence at the generated OOD samples, but also in the neighbourhood of those samples.
However, because both these approaches use the conﬁdence-loss, they suffer from the problems explained in this paper.
Moreover, such approaches are only feasible for input spaces where it is possible to represent the support of OOD with ﬁnite samples (assuming uniform distribution over OOD space).
This is not possible when the input space is R d , whereas the method proposed in this paper is still applicable.
Geifman et al.
(2018) propose to use Bayesian prediction uncertainties given by MC-Dropout (Gal &
Ghahramani, 2016) for OOD detection.
However, on the theoretical front, the Bayesian uncertainty measure only characterizes the uncertainty in in-distribution.
Therefore in principle should not be applied to OOD detection.
C T OY EXPERIMENT ON GENERAL OOD SAMPLES
In this case, both train and test OOD samples are uniformly sampled from a 2D box [ − 50 , 50]excluding the in-distribution regions.
From Figure 6, we observe that both conﬁdence loss and reject class based classiﬁers are able to distinguish in and out of distribution samples effectively.
Therefore, there is no clear winner between the two.
However as mentioned previously, such approaches are only feasible for input spaces where (approximately) representing the entire OOD region with a ﬁnite number of samples is possible.
This is deﬁnitely not possible for example when the input space is R d .
D G ENERATING OOD SAMPLES USING A GAN VS O UR APPROACH
Lee et al.
(2018a) propose to generate OOD samples in the low-density regions of in-distribution by optimizing a joint GAN-classiﬁer loss, (1).
With a toy experiment, they show that the generator indeed produces such samples and also these samples follow the “boundary” of the in-distribution data.
However, in the experiment, they use a pre-trained classiﬁer.
The classiﬁer is pre-trained to optimize the conﬁdence loss on in-distribution and OOD samples sampled close to the in-distribution.
Therefore the classiﬁer already has the knowledge of those OOD samples.
When GAN is then trained following the objective in (1), GAN likely generates those OOD samples close to the indistribution.
But it is evident that this setting is not realistic as one cannot have a fully informative prior knowledge of those OOD samples if our objective is to generate them.
Therefore, we experiment by directly optimizing (1) where the classiﬁer is not pre-trained.
The indistribution data for the experiment is obtained by sampling over the surface of a unit sphere from its diagonally opposite quadrants to form 2 classes respectively as shown in Figure 7.
We ﬁnd that (with much hyper-parameter tuning), even though GAN ends up producing OOD samples close to
[Figure] [Figure]
the in-distribution, it does an unsatisfactory job at producing samples that could follow the entire indistribution boundary.
Moreover, there is less diversity in the generated samples which make them ineffective at improving the classiﬁer performance in OOD detection.
Our intuition is that the loss (1(b)+1(c)) that forces the generator of the GAN to generate samples in the high entropy regions of the classiﬁer doesn’t necessarily enforce it to produce samples that follow the entire in-distribution boundary.
The inability of GANs to generate such samples for a simple 3D dataset indicates that it would be even more difﬁcult in higher dimensions.
In comparison to the GAN based boundary OOD generation, our approach as visually apparent from Figure 8 produces samples that cover the in-distribution boundary quite effectively.
While it is difﬁcult to visualize how well the off-manifold OOD samples cover the boundary, one can imagine them having a good coverage on the off-manifold boundary as they are obtained by perturbing each training sample in the direction given by the null-spaces.
Hence the diversity of the OOD samples is ensured.
For on-manifold boundary OOD samples, as evident from Figure 8c, it forms a closed boundary around the in-distribution points.
E D ISCUSSION ON SAMPLE GENERATION COMPLEXITY
For generating OOD samples outside the manifold, we randomly sample from the left-null-space of the Jacobian as described earlier.
But the complexity of this step depends on the number of basis vectors in the null-space and its dimensions.
For the MNIST case, with the input dimensions 28 ×and latent dimension of 8, there are 776 basis vectors in the left-nullspace, each of dimension(i.e.
, 28 × 28 ).
For colored images such as CIFAR and TinyImagenet, the number of basis vectors are almost 3 times of that for gray-scaled images.
To cover the in-distribution boundary effectively in all directions, many OOD samples for each in-distribution training sample are to be generated by taking random linear combinations of the basis vectors, which is quite expensive.
This gives a quantitative measure of effective OOD sample complexity.
However, we ﬁnd that only a few OOD samples are sufﬁcient to guide the decision boundary of the classiﬁer to be bounded around the in-distribution regions evidenced by their OOD detection results.
[Figure]
F E XPERIMENTAL A RCHITECTURE
The encoder and the decoder parts of the CVAE architecture, and the classiﬁer used are described in Figure 9a, 9b and 9c respectively.
The latent dimension ( d ) is chosen per dataset.
For MNIST, d = 8 and for Fashion MNIST, d = 10 .
The number of features after the convolutions in the encoder is represented by f .
“cond x” is the one hot representation of class labels.
k in the classiﬁer architecture represents the number of classes in the training data.
G M ETRICS D EFINITIONS
The deﬁnitions of metrics used to evaluate OOD detection are as follows.
FPR at 95% TPR is the probability of an OOD input being misclassiﬁed as in-distribution when 95% of in-distribution samples are correctly classiﬁed as in-distribution (i.e, the true positive rate (TPR) is at 95%).
True positive rate is calculated as, TPR =
TP TP + FN , where TP and FN denote the true positives and false negatives, respectively.
The false positive rate (FPR) is computed as
FPR =
FP FP + TN , where FP and TN denote the false positives and true negatives, respectively.
Detection error is the minimum mis-classiﬁcation probability over all possible thresholds over the
OOD score.
We assume that the test set contains equal number of in and out of distribution samples.
AUROC is the area under the receiver operating characteristic curve, which is a threshold independent metric.
ROC curve is a plot of TPR versus FPR.
AUROC can be interpreted as the probability that a positive example is assigned a higher detection score than a negative example.
For a perfect detector, AUROC is 100%.
AUPR is the Area under the Precision-Recall (PR) curve.
PR curve is a plot of precision ( TP = ( TP + FP ) ) versus recall ( TP = ( TP + FN ) ).
The metric AUPR-In and AUPR-Out represent the area under the PR curve depending on if in or out of distribution data are speciﬁed as positives, respectively.
[Figure]
H M ORE R ESULTS
We report the OOD detection results for OOD detection methods based on softmax-score (Hendrycks & Gimpel (2016)), uncertainty of classiﬁer obtained via MC-dropout (Gal & Ghahramani (2016)), and mutual information between predictions and model posterior (Gal et al.
The results are shown in Table 2.
[Table]
F-MNIST
MNIST 84.5/68.9/19.
[Table] [Table]
[Table] [Table]
[Table] [Table]
9 EMNIST-letters 88.1/77.8/37.
61.3/72.2/87.9 NotMNIST 83.1/67.0/24.
[Table]
[Table]
8/93.6 Omniglot 39.0/32.5/26.
[Table]
[Table] [Table]
[Table]
Gaussian-Noise 99.1/98.5/72.
[Table]
[Table] [Table]
4 Uniform-Noise 96.9/96.5/48.
Sphere-OOD 97.2/71.2/1.
[Table]
MNIST0MNIST5- 18.2/15.7/
[Table] [Table]
NotMNIST
[Table]