Improving Confident-Classifiers For Out-of-distribution Detection
Authors: Sachin Vernekar
Ashish Gaurav
Vahdat Abdelzad
Taylor Denouden
Rick Salay
Krzysztof Czarnecki
Venue:  (2020)
Keywords: Out-of-distribution detection
Manifold
Nullspace
Variational Auto-encoder
GAN
Confident-classifier

Abstract: Discriminatively trained neural classifiers can be trusted, only when the input data comes from the training distribution (in-distribution).  Therefore, detecting out-of-distribution  (OOD)  samples is very important to avoid classification errors. In the context of OOD detection for image classification,  one of the recent approaches proposes training a classifier called “confident-classifier” by minimizing the standard cross-entropy loss on in-distribution samples and minimizing the KLdivergence between the predictive distribution of OOD samples in the low-density“boundary” of in-distribution and the uniform distribution (maximizing the entropy of the outputs).  Thus, the samples could be detected as OOD if they have low confidence or high entropy.  In this paper, we analyze this setting both theoretically and experimentally.  We also propose a novel algorithm to generate the“boundary” OOD samples to train a classifier with an explicit “reject” class for OOD samples.  We compare our approach against several recent classifier-based OOD detectors including the confident-classifiers on MNIST and Fashion-MNISTdatasets.  Overall the proposed approach consistently performs better than others across most of the experiments.

Under review as a conference paper at ICLR 2020
IMPROVING CONFIDENT-CLASSIIIER FOR OUT-OF- DISTRIBUTION DETECTION
Anonymous authors Paper under double-blind review
ABSTRACT
Discriminatively trained neural classifiers can be trusted, only when the input data comes from the training distribution (in-distribution). Therefore, detecting out- of-distribution (OOD) samples is very important to avoid classification errors. In the context of OOD detection for image classification, one of the recent ap- proaches proposes training a classifier called "confident-classiferr" by minimizing the standard cross-entropy loss on in-distribution samples and minimizing the KI. divergence between the predictive distribution of OOD samples in the low-density "boundary" in-distribution and the uniform distribution (maximizing the en- tropy of the outputs). Thus, the samples could be detected as OOD if they have low confidence or high entropy. In this paper, we analyze this setting both theo- retically and experimentally. We also propose novel algorithm to generate the "boundary" OOD samples to train a classifier with an explicit "reject" class for OOD samples. We compare our approach against several recent classifier- OOD detectors including the confident-classifiers on MNIST and Fashion-MNIST datasets. Overall the proposed approach consistently performs better than others across most of the experiments.
INTRODUCTION
Discriminatively trained deep neural networks have achieved state of the results in many clas- sification tasks such as speech recognition, image classification, and object detection. This has resulted in deployment of these models in real life applications where safety is paramount (e.g.. autonomous driving). However, recent progress has shown that deep neural network (DNN) clas- sifiers make overconfident predictions even when the input does not belong to any of the known classes (Nguyen et al. (2015)). This follows from the design of DNN classifiers that are optimized over in-distribution data without the knowledge of OOD data. The resulting decision boundaries are typically "unbounded/open" as shown in Figure La resulting in over-generalization (Spigler (2019). Scheirer et al. (2012)).
There have been many approaches proposed to address this problem under the umbrella of OOD detection Lee et al. (2018a) propose to explicitly train classifier using the OOD samples gener- ated by GAN (Goodfellow et al. (2014b)). They empirically try to show that, for effective OOD detection, the generated OOD samples should follow and be close to the low-density boundaries of in-distribution, and the proposed GAN training indeed tries to do that. A multi-class softmax DNN classifier is trained with in-distribution samples to minimize the standard cross-entropy loss (minimizing the output entropy) and the generated OOD samples are trained to minimize KL loss that forces the classifier's predictive distribution to follow uniform one (maximizing the output entropy). The resulting classifier is called "confident-classiife as being in or out-of distribution based on the maximum prediction probability or the entropy of the output, Sricharan & Srivastava (2018) also follow similar approach with slight modification Contribution. One of the key assumptions in Lee et al. (20185 8a) and Sricharan & Srivastava (2018) that the effect of maximizing the entropy for OOD samples close to the low-density boundaries of in in-distribution might also propagate to samples that are far away from in-distribution. This train- ing is expected to result in "bounded/closed" regions in input space with lower entropy over the in-distribution, and the rest of the region (corresponding OOD), with higher entropy. The ideal
One can then classify sample
Related work section is in the appendix due space constraints.

---

Under review as a conference paper at ICLR 2020
P=digit 2!
x
9 OOD
22
22
2
222 22
2
2
72/44
44
33
3 3 3
21
44
4
3
XX
4
4
3
3
9
3
K
X
X
X
(a)
(b)
Figure Figure shows how the decision boundaries would change and become more bounded when typical classifier is trained with auxiliary ("reject") class containing OOD samples. (a) The unbounded decision boundaries of a typical 4-class classifier. Digit is incorrectly classified digit with very high confidence. (b) A 5-class classifier trained with OOD samples 'x' that are close to in-distribution and form the fifth ("reject") class, resulting in bounded decision boundaries. Digit is correctly classified belonging the "reject" (OOD) class.
decision boundary in such scenario would be as shown in Figure 1b We find that even though such a solution exists, the proposed training algorithm unlikely to reach it. We justify this both theoretically and experimentally for ReLU network (network with ReLU activation units) that was indeed used in Lee al. 8a). Assuming training with OOD samples close to the in-distribution boundary, we find that having an explicit reject class for OOD samples results in solution close to the one depicted in Figure lb Therefore we propose to use such a classifier instead. We give intuitive arguments to justify the proposal. This forms the first key contribution of our paper.
Moreover, with toy experiments (refer to oection[jin appendix) on liw-dimeniional synthetic data, analyze if GAN can indeed produce samples that follow the low-density boundaries of in- distribution. We find that, even though GAN produces samples close to the low-density boundaries of in-distribution,
unable cover the whole boundary, thus resulting in sub-optimal OOD detector when trained on such samples. We therefore propose novel algorithm to generate "bound- ary" OOD samples using a manifold learning network, (e.g.. variational auto-encoder ( (VAE)) and show that the generated samples are diverse and cover the in-distribution boundaries better than the method proposed in Lee al. (2018a). The resulting classifier trained with those samples improves the OOD detection results. This forms the second key contribution of our paper.
BACKGROUND
Lee et al. 8a) propose a joint training of GAN and a classifier based on the following objective:
min max min (..) log Pa = (] 3 Pa (LL(U
G D
+ E Br D(x)] Ere(s) [log(1 D(x))]
(I)
where (b)+(c) is the modified GAN loss and (a)+() is the classifier loss (0 the classifier's param- eter) called the confidence loss. The difference from the regular GAN objective is the additional KI. loss (T) which when combined with the original loss, forces the generator to generate samples in the low-density boundaries of the in-distribution (x)) space. 3 is a hyper-parameter that con- trols how close the OOD samples are to the in-distribution boundary. For the classifier, the KL loss pushes the OOD samples generated by GAN to produce uniform distribution the output, and therefore have higher entropy. This enables one to detect OOD samples based on the entropy or the confidence the output of the classifier.
2

---

Under review as a conference paper at ICLR 2020
3 WHY MINIMIZING CONFIDENCE LOSS IS INSUFFICIENT FOR OOD DETECTION
Let f: IR RK be the neural network function that maps imput in R K output classes (input to the softmax layer). Let fa : R² + RR be the function that maps the input to output for a specific class neural network with affine activations (c.g., ReLU and Leaky ReLU), each fa is a continuous piece-wise affine function over finite set of polytopes, (Q1, Q2 Q } such that Rd - Q1, described in Croce & Hein (018). This means that each fa is affine within each Qi (1 E (1.2,3...MM) If the input space is Rd. some of these polytopes stretch to infinity (grow without bounds). Let O = Qi denote these "infinity polytopes". The choice of the neural network structure and the weights define s. Figure 2a illustrates these polytopes and f& 'ss for simple 3-class ReLU classifier, where the input space is R. In this example, there are 4 polytopes in which Q and QI stretch infinity.
k E 2,, 3. J. For
fz
fa
f2
f2
fi
f1
x
Q
Q2
Q3
Q
Q
Q2
Q3
Q
(a)
(b)
Figure 2: fk's and Q for an example 3-class ReLU classifier where the input x E R. Q and Q infinity polytopes. (a) For sufficiently large (small) there is unique k-" in QT (k' = in Q+)). (b) For sufficiently large x, there are multiple k* in Q (k* {2, ()) ). For sufficiently small x, there is unique A=* = in QT
Hein et al. (2019) mathematically show that ReLU classifier (with softmax output) produces ar- bitrarily high confidence predictions (approaching 1) far away from the training data in almost all directions unbounded input space. This happens over Q.'s. s. Their results summarized as follows.
For any Rd, there exists Br > 0 such that for all aq 2 BB, anæ e Q Let (x)
(v) .0) + dg be the piece-wise affine function for class k over Qt. Let k- arg maxe ( 3,ar Then, as on + co, the confidence for input aræ for class k* becomes arbitrarily high if k" is unique. i.e
etc- (axe)
lim
K eliaa)))
But if there are multiple ks arbitrarily large confidence values cannot be obtained far away from the in-distribution in the direction of x. For instance, as shown in Figure 2t for Q k- 3) and therefore arbitrarily high confidence predictions cannot be achieved as a₁ + co. Having multiple k 's for every Q is highly unlikely, given that we are dealing with floating point numbers and also that it is not explicitly enforced during training. Therefore, arbitrarily high confidence values far away from the in-distribution are likely inevitable.
The above analysis for the case where the input domain is unbounded (Rd). For bounded domains (for example, [0, 1 for images), as pointed out in Hein et al. (2019), since we cannot let ar the above analysis cannot directly applied to get arbitrary high confidence values. However the above technique in principle can be applied to increase the prediction confidence for samples far away from the in-distribution. Hein et al. (2019) conduct experiments to support the claim. The theoretical analysis of which can be done as follows. Let 2d represent the bounded input domain. Similar to the unbounded case, let Q denote "infinity polytopes" that stretch till the
Note, A' arg max; (p)) arg max ( Bax), Vou 3. Also note, we define k* only for infinity polytopes. Nooee for RR, k arg max (slope of (o1z)) (= arg max: (negative slope (f4z()))) as or (ar -00)
3

---

Under review as a conference paper at ICLR 2020
bounds of the input domain Zd. For any E E", there exists a 3 > 0 such that for all aq 2 Br, ogæ E Qi Let (x)) (oi, () + be the piece-wise affine function for class k over Qt. Let arg mag (uf, B/a)) Then, as oj increases, the confidence for input ogæ for class k" keeps increasing until the bounds of the domain is reached if k- is unique. If fs- (Brx) >> (3(rr) Vk + k' (ignoring the effect of bias term for simplicity), the confidence for input aty for the class k" is very high. Therefore, even for the case of bounded input space, one can obtain confidence predictions for OOD samples high enough for it to be considered in-distribution samples.
Corollary. The higher the confidence of the output, the lower is the entropy. Hence direct corollary of Heii al. (2019)'s 's result is that the entropy of the classifier output for data far away from the in-distribution data in all directions would almost always be arbitrarily low (approaching 0) like the in-distribution samples. This makes it almost impossible to detect OOD samples based on the confidence or the entropy of the classifier outputs. For the case of bounded input domain, as one can increase the prediction confidence for OOD samples far from the in-distribution, the entropy of classifier output also decreases making those OOD samples to be classified as in on-distribution samples. Therefore, the approaches in Lee et al. (2018a) and (Sricharan & Srivastaval (2018) would not be applicable.
ADDING AN EXPLICIT "REJECT" CLASS
When OOD samples are generated close to the in-distribution and follow its low-density boundaries as proposed in [ee et al. (2018a) and |Sricharan & Srivastava (2018). we recommend adding an explicit reject class for OOD samples instead of minimizing the loss in [g Let the resulting classifier be called the reject-classifie. By adding an explicit reject class, our goal is to obtain decision boundary close to the ideal decision boundary shown in Figure[[b] where the decision boundary of K classifier divides the input space into regions such that the in-distribution region is classified one of the first K classes and the rest of the region as the K + 1 class, ie., the reject class. The intuition on how such a decision boundary can be obtained is as follows. The arbitrarily high confidence predictions happen in polytopes that stretch to infinity (or stretch till the bounds of input space in case of bounded input space). Each of the "infinity polytopes" has its own class (or classes), (or ks*s) where high confidence predictions occur. If adding an explicit "reject" class results in k. = reject-class for all the "infinity polytopes" (i.e there is only one k*), the arbitrarily high confidence predictions would only happen at the reject class for OOD samples far-off from training data. Therefore, these samples will be detected as OOD. We argue that in reject-classifier training, since we explicitly maximize the prediction confidence of K + 1tth for boundary OOD samples, we expect the same effect to persist for OOD samples far from the in-distribution as well (i.e., Kn = K + 1) resulting in close to ideal decision boundaries depicted in Figure Tb This claim is supported by our experiments on toy dataset (Figure. 5) and the superior performance of the reject-classiffer over the confident-classifie on MNIST and Fashion MNIST datasets (Table. [ Note that how close the resulting decision boundary of the reject-classifier is to the ideal one depends on how well the OOD samples follow the in-distribution boundary. We find that the method proposed in Lee al. (2018)) to generate boundary OOD samples is not diverse enough as evidenced by experiments shown in the appendix (Section. D). Therefore we propose a novel approach for boundary OOD sample generation which described in the next section that results in better boundary OOD samples that cover the in-distribution boundary quite effectively. This evident from our experiments described in the appendix (Section. D)
Lee et al. 2018a) indeed experiment with adding an explicit reject class instead of using a confident- classifier but the results are found to be worse. But this is because instead of using the boundary OOD samples they use another natural image dataset called "seen OOD" similar to Hendrycks et al.) 2019) to train the classifier. However for images, it is difficult to represent the entire OOD space with small number of samples such methods may not perform that well. Morcover pointed out in both Lee et al. (201 8a) and Hendrycks et al. (2019), as these "seen OOD" samples aren't 't diverse, when used to train reject classifier they can overfit to these training OOD samples. However use boundary OOD samples that can guide the decision boundary of the classifier to be bounded around the in-distribution regions depicted in Figure. [b
Note that both the classf-e and the confident-classifie use boundary OOD samples for train- ing. The confident-classiffer tries equalize fi 's for boundary OOD samples (i.e., maximize the en-
4

---

Under review as a conference paper at ICLR 2020
(a) Type
(b) Type
Figure 3: Categories of OOD samples that we generate: (a) Type (yellow), which includes samples that are close to the data but outside the in-distribution sub-manifolds, and (b) Type II (black), which includes samples that lie on the in-distribution sub-manifolds and trace the in-distribution boundary; in-distribution clusters are represented through blue and red points.
tropy of output predictions) and expect this to persist over OOD samples far from the in-distribution well (i.e., have multiple s)) The reject-classifier confidence K + 1th for boundary OOD samples and expects it to persists over OOD samples far from the in-distribution (i.e., have single k" at k K + 1). confidence fs. In the unbounded case, for confident-classifeer, while it is proven that one can almost always find arbitrarily high confidence regions far from the in-distribution, for reject classifier samples to be classified as belonging to the K + 1th class. In the bounded case too shown previously, one can obtain decreasingly low entropy regions far from the in-distribution for the con- fident classifier whereas for the reject-classifier it is similar to the unbounded case. As evident from the experimental results in Figure. [ we can indeed find OOD samples with low entropy for the confident-classifees without stretching to infinity, whereas for the reject-classifier, all those OOD samples far away from the in-distribution are correctly classified as OOD, The results in Table. further reinforces the superiority of the reject-classifier over the confident classifier.
the other hand maximizes the prediction
can still expect those OOD
OUT-OF-DISTRIBUTION SAMPLE GENERATION
The proposed approach leverages the following generic assumptions (Cayton (2005), Narayanan & Mitter (2010), Rifai et al. (20IT) that hold true for wide range of problems, primarily for image data, which is the data used to validate our approach.
The manifold hypothesis states that the higher dimensional real-world data in the input space is likely concentrated on a much diveediimnsional man-fonifo.d.
The multi-class manifold hypothesis states that, if data contains multiple classes, different classes correspond to disjoint sub-manifolds separated by low-density regions in the input space.
To fully cover the "boundary" of in-distribution, are to be generated. As shown in Figure 3 Type I) are the OOD samples that are close but outside the in-distribution sub-manifolds; Type II) are the OOD samples that the sub-manifolds but close to the "boundary" of the in-distribution.
identify two categories of OOD samples that
5.1 OOD SAMPLES OUTSIDE THE DATA MANIFOLD
These samples are obtained by adding small perturbations to in-distribution samples that are con- centrated on the manifold. These perturbations should be added directions such that the resulting samples should fall outside the manifold. The directions locally normal to the data-supporting man- ifold can be thought of as the directions that are less likely to contain in-distribution samples and the tangent directions the more likely ones. Therefore add perturbations in the normal directions to get OOD samples.
Deep generative models such VAEs (Kingma & Welling 2013) and GANs Goodffelow al. 2014b) can model the data manifold observations x E X through corresponding latent variables E Z via mapping function g 7 X g(2). With choice of reasonably lower

---

Under review as a conference paper at ICLR 2020
/
F
8
9
0
02
1
8
2
3
4
2
G 89
0 S 7
8
S
2 9 0
52
4
2 8:
3 7
1 3,7
7 G
5 /
4 8
7 r
F 6
4 4 04
55 24
6
4
9
0 8477
1
(a) MNIST (Type I)
(b) MNIST (Type II)
(c) F-MNIST (Type I)
(d) F-MNIST (Type
Figure 4: Generated OOD samples using the proposed method; Type OOD samples typically modify the background pixels (normal components have the least variance), while Type If OOD samples modify the object pixels.
dimensional z and a flexible generative function 9, the model can efficiently represent the true data manifold. Following the multi-class manifold hypothesis, that is conditioned over the class labels. For our experiments, we use a conditional variational auto- encoder (CVAE).
a conditional generative model
Let h : X Z and g : Z + X denote the encoder and decoder functions of CVAE respectively. The tangent space of the manifold at a point x: E X is given by the column space of the Jacobiar
dg(z)
J J(x)
Oz
z = h(x)
Let (x)) denote the null-space (T((a (x) (left null space of J (x)). Then the basis vectors of N(x) span the normal bundle of the manifold at Let v(x) ~ (x)) be randomly sampled unit vector from N (z), then the perturbed sample is given by,
Bu(
where B E R is a distribution point. In our experiments, we use stochastic 3 that is uniformly sampled from in the range 1.0). As discussed before, for better OOD detection, the boundary samples generated should be diverse; because the proposed approach generates OOD samples by randomly perturbing every in-distribution training sample. the diversity of the generated samples is ensured. This is visu- ally apparent from the experimental results on 3D-dataset shown in section appendix. Figure 4 iillustrates the perturbed samples for MNIST and Fashion MNIST datasets. One can observe that the perturbations added mostly modify the background pixels than the object pixels. This because the normal directions to the manifold mostly represent least variance components of the image.
that controls how far the perturbed sample from the in-
5.2 OOD SAMPLES ON THE DATA MANIFOLD
These are the samples that are in the low-density regions of the input space but close to the in- distribution boundaries the manifold.
For a variational auto-encoder, the aggregate posterior (zx) Makhzani et al. 2015 is given by,
<(2) g(2 (x) (m)
where Pin (x) is the probability density function of in-distribution and ((z/x)) is the approximate posterior. Assuming a smooth decoder, the high-density regions in the aggregate posterior can be thought of as corresponding to densely populated regions in the input space, and the input space density would gradually decrease as we sample away from the high-density regions in the aggregate posterior. Therefore the in-distribution boundary on the manifold can be approximated by regions at a distance away from the high-density areas where the density dips below a certain threshold. For our experiments, approximate q(z) with uni-modal Gaussian distribution whose mean
Whiiee is stochastic, we just its mean estimate for generating OOD samples outside the manifold.

---

Under review as a conference paper at ICLR 2020
and co-variance Σ estimated using the encoder mappings of in-distribution samples. We use Mahalanobis distance as a criterion to determine the distance from the mean to sample and generate the required OOD samples. Let be the Mahalanobis distance from the mean of q(z) that encompasses 95% of the training data. The OOD samples are generated by decoding the uniformly sampled samples from the latent space
the surface of hyper-ellipsoid (Rubinstein 1982) defined by 5 where ü. and Σg are the mean and co-variance estimates of q(z), respectively,
(= - A.) "= =
(5)
is fair to assume a uni-modal Gaussian distribution for ((x) as we a Gaussian per class. More- substantial gain in the ODD detection results when the classifier is trained with these samples can also be taken as evidence pointing towards the validity of such an assumption.
The generated OOD samples described in 5.1 and 5.2 then used to train classifier, where the n + class represents the OOD class.
class softmax
EXPERIMENTS
Experiments are divided into sections; the first section explains the toy experiments on a low- dimensional dataset support theoretical analysis of the confident-classifeer, the second section gives details of OOD detection experiments on MNIST and Fashion MNIST ((Xiao et al. (017)) using the proposed method.
6.1 LIMITATIONS OF
In these experiments, the input space is IR2 and the in-distribution consists of 2-classes. The samples for each of these classes are generated by sampling from 2 Gaussians with identity co-variances and means (-10, 0) and (10. 0) respectively, dard deviations (Mahalanobis distance) from the in-distribution means is considered OOD, The architecture of the neural network used is similar to the used in Lee et al. (2018a), which is Rellsssasrffirr with 2 fully-connected hidden layers with 500 neurons each.
the Cartesian coordinates. Anything outside 3 stan-
Following the case in Lee al. 10). for training, OOD samples generated close to the in- distribution as shown in Figure 5a For testing, OOD samples are uniformly sampled from 2D box 50, 50]2 excluding the in-distribution regions.
-20
aero
(0.6,
(0.5
20
-20
(a)
(b)
(c)
Figure 5: Plots for boundary OOD samples experiments. (a) Training data in 2D. (b) Maximum prediction output on test data for confident-classifer (c) Classification output of classifier with a "reject" class on test data (TC = true class, PC = predicted class).
we observe that the Rel.U-classifier trained to optimize confidence loss results in highly confident predictions for many OOD samples far from the in-distribution data. This renders the classifier ineffective at classifying the in and out of distribution samples based on the maximum prediction score (confidence) the entropy of the output. However, from Figure 5c for classifier
From Figure 5b
*Code: http://gitthh.cmm////200001/111

---

Under review as a conference paper at ICLR 2020
trained with explicit reject class, the test OOD samples are indeed classified OOD. This supports the aforementioned intuitions in 4
Note that these are not the results specific to certain architecture of the neural network. Ex- periments with different syper-parametee such as the number of hidden neurons, changing input dimensions, using sigmoid activation functions instead of ReLU lead to similar results. We remark however that for sigmoid networks, the results were not as extreme (in terms of the number of OOD samples with high-confidence) as for ReLU networks. This understandable because sigmoid acti- vation outputs will not produce arbitrarily large values, unlike the ReLU counterparts.
6.2 MNIST AND FASIION MNIST EXPERIMENTS
We validated our approach on MNIST and Fashion MNIST as in-distribution datasets and several other OOD datasets. For all MNIST as in-distribution experiments, we use CVAE with latent di- mension of 8, and for Fashion MNIST, the latent dimension is set to 10. We compare our approach against the recent classifier- based OOD detectors such as confident-classifee,, ODIN and Maha- lanobis distance-based approach without feature ensemble (MD The architecture for both CVAE and the classifier used are shown in the appendix. Both the networks are trained till convergence.
6.2.1 OOD DATASETS
MNIST is used as an OOD dataset for Fashion MNIST as in-distribution, and vice-versa. For MNIST 0-4 experiment,
images in class 0 through 4 in-distribution and class 5 through as OOD. We use both character datasets and noise generated images as OOD datasets. The charac- ter datasets are Omniglot (Lake al. 2015), EMNIST-letters (Cohen al. 2017) and NotMNIST [Bulatov 2011). The noise generated images are described below.
Gaussian noise includes gray-scale images, where each pixel is sampled from an independent nor- mal distribution with 0.5 mean and unit-variance.
Uniform noise includes gray-scale images where each pixel is sampled from an independent uni- form distribution in the range [0, 1].
Sphere OOD contains images sampled from the surface of a 784 dimensional hyper-sphere centered at the origin with a radius equal to the maximum Euclidean distance of in-distribution samples from the origin and reshaped to 28 x 28. This is used show the effectiveness of our approach not only on the datasets that are restricted to a finite range such as [0, 1]a for images in [0, 1]d but also for general case of I
6.2.2 EVALUATION METRICS FOR OOD DETECTION
We experimented with two different metrics as OOD score to determine if the given input sample is in out of distribution. OOD class probability is the + 1 class prediction probability. In- distribution max probability is the maximum prediction probabilities of the in-distribution classes. A higher (lower) OOD class probability (in-distribution max probability) indicates higher proba- bility of sample being OOD. Except for MNIST 0-4 experiments, we find that the former metric gives the best results. We report only the best score in Table T We use the area under the ROC curve (AUROCT), the area under the precision-recall curve (AUPR1), the false positive rate at 95% true positive rate (FPR95.) and the detection error as the metries for evaluation. These metrics are commonly used for evaluating OOD detection methods (Lee et al. 2018a Hendrycks et al. 2019). The details of which are the appendix.
6.2.3 DETECTION RESULTS
Table compares our approach with other approaches for experiments on MNIST and Fashion MNIST in-distribution datasets. Since the classifier is trained with OOD samples, there is pos- sibility of reduction in the classification accuracy of in-distribution classes in comparison to training without OOD class. We therefore report classification accuracy of classifier trained with and with- out OOD samples. We find that there is no significant change in accuracy. Training our method
Resulls for 3 other approaches in the appendix
8

---

Under review as a conference paper at ICLR 2020
Table OOD detection results
FPR at TPR
Detection Erre]
AUPR
AUPR
ID Model
AUROCT
0OD
Out?
Inf
lace before 00D/
after OOD)
Ouss/Confieett hww.aanddeeentaassiieeoDDDNDDDD
F-MNIST
0.0/7.900.494.2
0,2/5.6/1 8/119
100.0/98. 5/99 8/86.6
100.0/98.//99889
1ML0/98 19t.84/99./777. 8/74.0
EMNIST-lettors
1.//31.//257/33 223.7/31 2
3.0/13.2/1 3.16
99.6/93. 0266693.09944444 0/94349949492 2
99.6/99../9443/99
99.6/92.4/94, 1/94,1/933/
NoTMNIST
0.0/26.5/1 1,33334
0.0/12.3/6,900
100.0/94, 8.9
100.0/93.9/1000000
100.0/93 100.0/93.8/97.79922 8/97.7/993
MNIST
(99.0/9899)
Omniglot
0.090.00.0998.5
0.011 O/O 2/46,9
100.0/ 1000000000011000 100.0/
100.0/100.0/1000 10..0000/00/
100.0/100.0/ 100000/550000000..5000
Gausian-Noise
0.0/0.000.099.99
0.040.09..024.66
100.0/100.0/ 100.0000000000...
100.0/110.00110000711
ww0.0/100.01100.002
Uniform-NNise
0.0/0.0.0.052..6
4.9/0.090.0/26.4
000.0/100.0/100.000
100.0/100.6/1000075
104.0/100.0/10000
Sphere OOD
0.0/11.60..050.4
0.1/6.6/1 4/14,9
100.0/96. 8/99 8/87.66
100.0/97.8/9999999
00..//9..2/99.777
MNIST
4.1/87.4/70.224 2/2.44
4.2/36.3/28..8 3/26./328.////.
98,2/65, 000552277..22 .2999.
(1N.0/64..877.3/99
EMNIST letters
6.4/87.3/83.5/11
5.4/4188/1 3.67.3
97.9/61. (166.1
06.8//60.0/62.0099
98.5/61 (66666666666 6/98.
NotMNIST
0.8/8.9/80..24 2/7.2
22/33.2/33, 9/5.8
99.7/73. 7/699 3/97.5
99,5/73. 0/6 00997474
99.8/72. 0177224/70.5999
F-MNIST
Omniglot
0.0/59.8/9.66 6/58.4
0.9/22. 1/7, 1/26..8
99,8/85 1/979/83./ 2
99, 99/5 8/977////44/
99.6/85. 1/988 2/83.288.4
(91.991.2)
Gaussian-Naise
0.0/32.2/4.5/99.
0,2/9 /63 8/19.9
99.8/95, 8/98, 0V80.0
99, 9/96. 7/96.7/977777 O
99.5/94, 7/95. 19566666.
Uniform-Noise
0.2/71.0/99441
6.3/1,.4/2..00 7/3.3
99,8/88, 6/74, 7/98.0 9
99.8/91 8/82//8//20 2/9.9999..2
99.8/82.900 .6997.9
Sphere-OOD
0.6/99.//100.00.00
0.8/50.0/50.000.0
99. 09.7/29.6/0.22 6/0.25/0222.0.00
99) 990.//3..1 w/...//9/..../////
99 A/33.3/33.3..//111///
MNISTS-9
17.2/21 9/20,4/50.00
10.0/12.0/1 1,5/44,4
95.1/92.9/99 4/92.3
94,0/92. L/993
94,9/93 6/94, 2/90.
F-MNIST
0.2/1.7/2.0044
1.6/3.1/3 4/15.11
99.8/9944/99.4 4/92.5
99, 8/99///999999////
99.7/99.3/99 1/999/99999999999999
EMNIST-leteers
11722.122.4/12..
3.8/12.4/1 3. 3.9/7.6
99,2/92.9/922 9/9..99999999999
99.3392.0//D www..099904496....
99,1/93, 6/93,2/97.
(99.8/99.)
4
NatMNIST
0.0/10.9/28.00 0/2.8
3.1/7.7/1..3/3.
100.0/97. 5/93 55999.3
100.0/97. 9/9.2/99//99
10L.0/97 6/93/7/99/
Omniglot
3.000.0/..4000
0.0/0. 6/0.4
100.0/100.0/99, 1/100.9
100,0/100.00
1w..0/100..099..1.. 3/100,0
100.0/100.00 ww9.000.....00 8/100.0
Gaussian Noise
0.0/0.0N0.60.22
0,0/0.000.1 1/2.4
00.00110.00100009977
W...0/100.0/1000000
100.0/100.0/9 2/92.7/92.2
Uniform-Noise
0.000.00..025.9
0.0/9.0004455
100.0/100.0/99.00
100.0/ 10000/90009990000 9/97.6
100.0/100.099 (/89
Sphere-OOD
0.077.10../22.7
0.1/5.5/2.0/6.
100.0/98.20 21998/2/999.0 6/96.5
100,0/98,00 6/99,7999777
100.0/97.4/9 4/93.3/93.
F-MNISTS-9
19.7/55 8/29. 2/75.8
12.3/17. 1/14.6/26.4
92.5/89.5/92 1/79.5
88.7/90.2/99 1.
94.3/87.1 1/9.887/92.8/77..
MNIST
1.8/67.353..68 5/5.3//20
2.3/23.6/2 1/3.4
99.5/83. 5/86.///699 4/99//99.00 O
99, 4//44/8.//86. 1/99.
99.6/81. 7/85, 7/98.5
EMNIST-leters
1.2/7166/8844
2.4/24,2/200 6/273
99.6/22 6/97.9977.
8887
99.7/79 8/67,9/9669
F-NNISTO-4
(94.8/94.8)
NotMNIST
0,2/76.0/57. 7/1 1.0
1,2/26. 12//268/23.666 6/8.0
99,9 99,9/79. 9/84, 1/97.0
99.8/8 19.88/1133/83
99.9/77. 1/83, 9/97.2
Omniglot
L0/62.3/15.5/
2.5/1 8. 3/9 1/7.1
99.5/88. 5/97.5
199.6/90.6/9666 0/97.9
99.3/85. 8/96, 7/95.
Goussian Noise
0.000.30..699...
0.4/2.089.4141.
100.0/99.7/100 7/1000///4
100,0/99, 8/10.00/62.
WWW.0/99.7/100.00
Uniform-Noiss
0.0/9 8/1.3/6633
0.3/5.4/3.0/88. 0/8.5
100.0/98. 1/99 2/95.0
100.0/9.6699900 019.866999.499666 4/96.6
10L.0997 w.8.559..9// 9/90.
Sphere-OOD
0.0/89.6/ (199)/995.500..
0.0/38.3/4 6/0.0
100.0/65 8/59 8/100.0
100.0/67.7662.4/100
100.0/61 1..99/55.00 1/0000
requires tuning hyper-panee eyper-parameter such as 33 from Eq. 4, OOD class weight, and learning rate. The hyper-parameters were chosen based on the in in-distribution classification accuracy and the AUROC of the validation generated OOD samples and the random noise datasets. For all our experiments we use a stochastic /3 uniformly sampled the range 10. 1J, OOD class weight is set to 0.1, while the weights for the rest of the classes is set to 1.0, and Adadelta (Zeiler 2012) the optimizer used with learning rates of 0.1 and 0.01 for Fashion-MNIST and MNIST experiments, respectively. We do not tune the hyper parameters per OOD dataset unlike ODIN and Mahalanobis distance- ap- proaches, where the perturbation magnitude tuned per OOD dataset. Even without this advantage, our method still performs better than these baselines for most of the OOD datasets.
We would like to remark that our approach gives good OOD detection results consistently on all the OOD datasets used unlike the baselines compared. This indicates that our approach is robust to change in OOD datasets.
7 CONCLUSION
We have shown in the paper that the confident-classifier almost always has OOD samples that pro- duce high confidence outputs (in the contexts described carlier). We provided empirical evidence that favor using an explicit "reject" class instead. However, the ODD detection capabilities of a reject-classifier depend on the extent to which the generated OOD samples follow the low-density boundaries of in-distribution. We also propose novel algorithm for generating "effective" OOD samples for training an n + 1-class classifier for OOD detection and the results for most of the ex- periments on gray-scale datasets are consistently better for our approach in comparisons to other methods compared. For future research, we would like to investigate its effectiveness on non- non-gray- scale datasets such as CIFAR and TinyImageNet. However we would like to point out that the nullspace calculation for colored images is computationally quite expensive, hence requires a larger compute (refer appendix E).

---

Under review as a conference paper at ICLR 2020
REFERENCES
Yaroslav Bulatov. Notmnist dataset. Google (Books/OCR), Tech. Rep.(Online). Available:
http://yarolavvvb blogspot. it/201 122201100/ootmmistaaaaaaaa html, 2, 2011.
Cayton. Algorithms for manifold learning. Technical Report CS2008-0923, UCSD., 2005.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. Emnist: an extension of
mnist to handwritten letters. arXiv preprint arXiv:1702.05373, 2017.
Francesco Croce and Matthias Hein. randomized gradient-free attack on relu networks. arXiv preprint arXiv:1811.11444 2018.
Terrance DeVries and Graham W Taylor. Learning confidence for out-of-distribttoon detection in 2018.
neural networks. arXiv preprint
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059, 2016.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. CoRR, 703.1900.,,900, 2017.
Yonatan Geifman, Guy Uziel, and Ran Elniv..v. Boosting uncertainty estimation for deep neural classifiers. CoRR, 2018.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, S. Ozair D. Warde-Farley, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems (NIPS), 2014a.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014b.
C. Guo, G. Pleiss, Y. Sun, and Q. Weinberger. calibration of modern neural. In International Conference on Machine Learning (ICML), 2017.
M. Hein, M. Andriushchenko, and Bitterwolf. Why relu networks yield high-confidence pre- dictions far away from the training data and how to mitigate the problem. arXiv preprint arXiv: 18X2.0881..0570. 2019.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-of-distribution. In International Conference on Learning Representations (ICLR), 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations (ICLR), 2019.
Diederik P Kingma and Max Welling, arXiv:1312.6114, 2013.
Auto-encoding variational bayes.
arXiv preprint
Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. Human-level concept learning through probabilistic program induction. Science. 350-6266) 2015. URL https:
7/science wweeeeecceeeimneeeeeee org/ content/350 /3/////5/66666633333
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confiuenne--calibraeed classifiers
for detecting -itto--distribution samples. International Conference on Learning Representations (ICLR), 2018a.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distributiion samples and adversarial attacks. arXiv preprint arXiv:/807.008:: 1807003888, 20180 8b.
Shiyu Liang Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distributio image detection in neural networks. In International Conference on Learning Representations (ICLR), 2018.
10

---

Under review as a conference paper at ICLR 2020
Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, lan Goodfellow, and Brendan Frey. Adversarial autoencoders. arXiv preprint arXiv: 2015.
H Narayanan and S Mitter. Sample complexity of testing the manifold hypothesis. Advances in Neural Information Processing Systems, 2010.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High con- fidence predictions for unrecognizable images. In Computer Vision and Pattern Recognition (CVPR), 2015.
S. Pidhorskyi, R. Almohsen, D. A. Adjeroh, and G. Doretto. Generative probabilistic novelty de- tection with adversarial autoencoders. In Advances in neural information processing systems (NeuRIPS), 2018.
Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A. DePristo, Joshua V. Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. arXiv preprint arXiv: (90::99660000 1906.02855,,, 2019.
Salah Rifai, Yann N, Dauphin, Pascal Vincent, Yoshua Bengio, and Xavier Muller. The manifold tangent classifier. Advances in Neural Information Processing Systems, 2011.
R. Y. Rubinstein. Generating random vectors uniformly distributed inside and the surface of different regions. European Journal of Operational Research, 1982.
W. Scheirer, A. Rocha, A. Sapkota, and T. E. Boult. Towards open set recognition. IEEE transac- tions on pattern analysis and machine intelligence. 2012.
G. Spigler. Denoising autoencoders overgeneralization in neural networks. arXiv preprint 17Xiv:1709.04762, 2019.
K. Sricharan and A. Srivastava. Building robust classifiers through generation of confident of distribution examples. Third workshop on Bayesian Deep Learning (NeurIPS), 2018.
W. Wang, A. Wang, A. Tamar, X. Chen, and P. Abbeel. Safer classification by synthesis. arXiv preprint arXiv:1711.000 00554,, 2017.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms, 2017.
Matthew D Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv: 121.55121,.5701, 2012.
A APPENDIX
B RELATED WORK
There have been many approaches in the literature proposed address the problem of OOD de- tection in the context of image data. Most of the successful ones are either generative (Pidhorskyy
al. 2018 Wang al. 2017 Ren al. 2019)) or classifier-based approaches (Hendrycks Gim-
pel 2016 Hendrycks et al. 2019; De Vries & Taylor 2018 Liang et al. 2018 Lee et al. (018b))
Generative approaches either explicitly or implicitly estimate the input density or use reconstruction emor as criterion decide if input belongs to OOD. Classifier- approaches, on the other hand, incorporate OOD detection as a part of the classifier network. The approach proposed this paper belongs to the latter category. Therefore we limit our related work discussion to only classifier-based approaches.
Typical discriminatively trained classifiers that model the conditional probability P(y(z) with- out any additional constraints, by definition can make reliable classification decisions only on in- distribution data. For disttiibbbtttiinn data, the classifier output is arbitrary. Moreover, any meta information from the output of the classifier or the features leamned are also conditioned on the data belonging in-distribution. Therefore this information in-principle cannot be used to ascertain if

---

Under review as a conference paper at ICLR 2020
the imput is in out of distribution. However, most of the recent approaches ((Hendrycks & Gim- pel 2016 Lee et al. 2018b Liang et al. 2018 De Vries & Taylor 2018 jn the literature follow this approach.
Hendrycks & Gimpel (2016) propose a baseline approach to detect OOD inputs, called mox--sofmmxx by thresholding the maximum softmax output of a pre-trained classifier. Liang al. (2018) improve upon this using temperature scaling (ODIN, (Guo et al.) (017)) and adding input perturbations. The assumptions is that these changes result in larger separation between in and out of distribution data in terms of their output predictions. [ee et al. (2018b) propose an approach based on the assumption that the cass-conniiional features of a softmax classifier follow a Gaussian distribution. Therefore, Mahalanobis distance (MD) from the mean of the Gaussian is used as a score to detect OOD. This then combined with input perturbations similar to ODIN to enhance the OOD detection results. This method obtains state-of-the-art results on most of the baseline datasets used in OOD detection literature. Despite good results, the method can be seen as OOD detection on feature space rather than pixel space not conforming to the usual definition of OOD (By definition, the in-distribution, (i)(x) is defined for x X in pixel space, and hence OOD is also defined in the same space). Hence the effectiveness of the method highly depends on the features learned by the classifier, and also there is no guarantee that the optimization algorithm forces the features to follow a Gaussian distribution. [Hendrycks et al., (2019) propose train a classifier with confidence loss where OOD data is sampled from a large natural dataset. Hein et al.] (2019) also follow a similar approach using confidence loss and uniformly generated random OOD samples from the input space. In addition, they not only minimize the confidence at the generated OOD samples, but also in the neighbourhood of those samples. However, because both these approaches use the confidence-loss, they suffer from the problems explained in this paper. Moreover, such approaches are only feasible for input spaces where is possible to represent the support of OOD with finite samples (assuming uniform distribution over OOD space). This is not possible when the input space is Rd, whereas the method proposed in this paper is still applicable.
Geifman et al. (2018) propose to use Bayesian prediction uncertainties given by MC-Dropout (Gal & Ghahramani] 2016) for OOD detection. However, on the theoretical front, the Bayesian uncertainty measure only characterizes the uncertainty in in-distribution. Therefore in principle should not be applied to OOD detection.
C TOY EXPERIMENT ON GENERAL OOD SAMPLES
In this case, both train and test OOD samples are uniformly sampled from a 2D box [-50, 50 excluding the in-distribution regions. From Figure observe that both confidence loss and reject class based classifiers are able to distinguish in and out distribution samples effectively. Therefore, there is clear winner between the two. However as mentioned previously, such approaches only feasible for input spaces where (approximately) representing the entire OOD region with a finite number of samples is possible. This is definitely not possible for example when the input space is Rdd
D GENERATING OOD SAMPLES USING A GAN VS OUR APPROACH
Lee et al. (2018a) propose to generate OOD samples in the low-density regions of in-distribution by optimizing a joint GAN-classifier loss, . With a toy experiment, they show that the generator indeed produces such samples and also these samples follow the "boundary" of the in in-distribution data. However, in the experiment, they use a pre- pre-trained classifier. The classifier is pre-trained to op- timize the confidence loss on in-distribution and OOD samples sampled close to the in-distribution. Therefore the classifier already has the knowledge of those OOD samples. When GAN then trained following the objective in (I). GAN likely generates those OOD samples close to the in- distribution. But it is evident that this setting is not realistic as one cannot have a fully informative prior knowledge of those OOD samples if our objective is to generate them.
Therefore, we experiment by directly optimizing ([) where the classifier is not pre-trained The in- distribution data for the experiment is obtained by sampling over the surface of unit sphere from its diagonally opposite quadrants to form classes respectively as shown in Figure 7 We find that (with much hyper-parameter tuning), even though GAN ends up producing OOD samples close to
12

---

Under review as a conference paper at ICLR 2020
TCO TO
TCO
16.2
20
-10
01.6.0 0
[0... 11
-10
(2.5.0.) 0.81
(3)
(b)
(c)
Figure 6: Plots for general OOD samples experiments.
(a) Training data in 2D. (b) Maximum
prediction output on test data for confident-classiffer. (c) Classification output of classifier with a "reject" class on test data.
875
1.00 9:0 635 o7n
3
(a) 3d plot
2d projection
Figure 7: Generated OOD samples using a joint training of a GAN and a confident-classiffer. We observe that the generated OOD samples don't 't cover the entire in distribbutiio boundary.
the in-distribution, it does an unsatisfattory job at producing samples that could follow the entire in- distribution boundary. Moreover, there is less diversity in the generated samples which make them ineffective at improving the classifier performance in OOD detection. Our intuition is that the loss [[)) that forces the generator of the GAN to generate samples in the high entropy regions of the classifier doesn't necessarily enforce to produce samples that follow the entire in-distribution boundary. The inability of GANs to generate such samples for simple 3D dataset indicates that it would be even more difficult higher dimensions.
In comparison to the GAN based boundary OOD generation, approach visually apparent from Figure [ produces samples that cover the in-distribution boundary quite effectively. While is difficult to visualize how well the off-manifold OOD samples them having a good coverage on the off-manifold boundary as they are obtained by perturbing each training sample in the direction given by the null-spaces. Hence the diversity of the OOD samples is ensured. For on mn-manifold boundary OOD samples, as evident from Figure Sc it forms closed boundary around the in in-distribution points.
the boundary,
can imagine
E DISCUSSION ON SAMPLE GENERATION COMPLEXITY
For generating OOD samples outside the manifold, we randomly sample from the left-null-ppace of the Jacobian as described earlier. But the complexity this step depends on the number basis vectors in the null-space and its dimensions. For the MNIST case, with the input dimensions 28 28 and latent dimension of 8, there are 776 basis vectors the left-nullspace, each of dimension 784 (i.e.. 28 28). For colored images such CIFAR and Tiny Imagenet. the number of basis vectors almost times of that for gcayld images. To cover the in nn-distribution boundary effectively in all directions, many OOD samples for each in-distribution training sample be generated by taking
13

---

Under review as a conference paper at ICLR 2020
random linear combinations of the basis vectors, which is quite expensive. This gives quantitative measure effective OOD sample complexity. However, we find that only a few OOD samples are sufficient guide the decision boundary of the classifier to be bounded around the in-distribution regions evidenced by their OOD detection results.
05
40
-10
-25
56
16
(a)
(b)
(c)
Figure 8: Generated boundary OOD samples using approach. (a) 3d plot of in-distribution data with out-of-manifold boundary OOD samples.
(b) 3d plot of in-distribution data with on-manifold boundary OOD samples. (c) 2d projection of in-distribution data with on-manifold boundary sam- ples to show that they cover the in-distribution boundary on the manifold.
F EXPERIMENTAL ARCHITECTURE
The encoder and the decoder parts of the CVAE architecture, and the classifier used are described in Figure Pa 9b and [c respectively. The latent dimension (d) is chosen per dataset. For MNIST, d 8 and for Fashion MNIST, encoder is represented by f. "cond_x" is the one hot representation of class labels. in the classifier architecture represents the number of classes in the training data.
10. The number of features after the convolutions in the
G METRICS DEFINITIONS
The definitions of metrics used to evaluate OOD detection are as follows.
FPR at 95% TPR is the probability of OOD input being misclassified in-distribution when 95% of in-distribution samples are correctly classified as in-distribution (i.e, the true positive rate TP (TPR) is at 95%). True positive rate is caleulated as, TPR TP4FN. where TP and FN denote the true positives and false negatives, respectively. The false positive rate (FPR) is computed as FP
FPR = FP4TN: where FP and TN denote the false positives and true negatives, respectively.
Detection error
minimum miscclassification probability over all possible thresholds over OOD score. We assume that the test set contains equal number of in and out of distribution samples.
AUROC is the area under the receiver operating characteristic curve, which is a threshold indepen- dent metric. ROC curve is plot of TPR versus FPR. AUROC can be interpreted the probability that a positive example is assigned a higher detection score than negative example. For a perfect detector, AUROC is 100%.
AUPR is the Area under the Precision- Recaal (PR) curve. PR curve is plot of precision (TP- (TP + FP)) versus recall (TP (TP + FN)). The metric AUPR-In and AUPR-Out represent the area under the PR curve depending on or out of distribution data are specified as positives, respectively.
14

---

Under review as a conference paper at ICLR 2020
zmean
(28, 28)
16F, K3
32F, K3
(f)
Dense
Conv2D
Conv2D
Dense16
relu
relu
Dense(d)
relu
(d)
cond (10)
zvar
(a) Encoder architecture
32F, K3
16F, K3
1F, K3
reconstructed
Dense1 16
Dense(f)
Conv2D Transpose
Conv2D Transpose
Conv2D Transpose
(d)
relu
relu
relu
sigmoid
relu
cond_x x (k)
(b) Decoder architecture
(28, 28)
32F, K3
64F, K3
MaxPoode 20
propoutio .)
Ropouite .)
Flatten
Dense122
Dense
Conv2D
Conv2D
relu
relu
relu
softmax
(c) Classifier architecture
H MORE RESULTS
We report the OOD detection results for OOD detection methods based softmax- (Hendrycks & Gimpel (2016)). uncertainty of classifier obtained via MC-dropout (Gal Ghahra-
mani (2016)), and mutual information between predictions and model posterior (Gal et al. (2017).
The results are shown in Table
15

---

Under review as a conference paper at ICLR 2020
Table 2: Results
FPR 95% TPR 1
Detection Emor
AUROCT
AUPR
AUPR
ID Model
OOD
Out?
Inf
Softmux/MC Dropoou/Muuuulll
F-MNIST
1.61/2.1/44.3
3.3/3.5/144.5
09.5/99.4/00.
99 5/99999599..5
99.5/99.488..2
EMNIST-leteer
28.0/24.5/23.0
12/6/11.2/11.1
93.6/94.6995
93.4/94.4994
93.09441/94.7
NoMMNSS
13.1/12.5/22.1
7.1/6.7/9.1
97.4/97.6PM 6/95.5
97.8/97. 9/969
97.0/97.1/44.
MNIST
Omnigloc
0.0/0.0/9..8
0.5/0.6/19.5
100.0/100.0/844
100.0/100 LV888.7
100.0/100.077 0/.03.
Goussian-Noise
0.0/0.0/100.0
0.0/0.0/49.3
100.0/100.0/177
100.0/100.0/311
100.0/100.0/344
100.0/100.0642.9
Sphere-OOD
0.8/1.37766
2.6/3.1/4.2
09.3//9.2/97..
99.5/99.499..3
99, 19.1999.//331
VNIST
84.5/68.9/19
34.5/25.0/11
70.6/82.2/93.5
70.9/83.0/911.
68. 68.2/80.3/94,
EMNIST-lctters
8/77 8/37.6
41.1/33. 1/21.1
62.3/73 4/84.2
62.3/73.2779
61.3/72 2/27.2/87
NotMNIST
83.1/67.0/22 0//4422
1/14.3
1.79971.7791.7
66.A/80.3/87.6
F-NNIST
Omniglot
39.0/32. 5/26.8 8
17.1/14.8/10.4
91.493.5995
91.933 4/93379995.6
91.8/95 91.8/5555/33.
Gaussian- Noise
19.1/98.5722.2
879./
1
Uniform-Nooe Noise
96.9/96.5/48.8 8
29.4/24.2/16
70.1/76. 1/76.8/90.9
78.6/84.0/92..
58. 1/64.8///87 8/87.
Sphere OOD
97.2/71.2/1.8
7.02/
48.4/88.2/98 2/99.6
50.6/9 4/99.5
47. 11..7/22.499
MNISTS-9
18.2/15.7/15.3
10.6/9.719.9
V3.0/93.199 1/92.8
94,8/95 4/95.
3.1/4.478.5
99
2/99.0/98,4
7/98
EMNIST-LETTERS
25.3/21 66/1 4
12.8/12.2/12.2
02.8/93.5/93.
90.6/91 6/9
93.3/93.919
MNISTO-4
NocMNIST
21.4/16.2/15.9
30,44/9.309.4
95.5/96.4/96.
95 5/96.
02.00/55. 9/96.4
Omniglot
0,2/0.181.5
1.7/1.9/24
99.4/99.4999
99.6/99.6/99
28.99/9.0/98.3 8
Gaussian Noise
0.000000.0
0.38).4/1.0
99.7/99. 7/98.8
99.8 8/99 .4
98, 9/98 9/95.8
Uniform- Noise
0.0/0.00.00
0.6/0.9/2.2
99.7/99.699 6/98.3
99 (19.8/99.9999 8/99.0
1/99.0/95.2
Sphere OOD
1.0119/2.5
2.8/3.4/3.6
99.2/99.0998.6
99.4/99.31 9/99.1
98.6/98.5/97
F-MNISTS-9
73.5/67.1/33
26.1/23.4 4/17.3
80,1/83.4/88
80.4/83.3/87 7.76
78. 78.1782.0/91.2 0/91
MNIST
44.9/93.9/73.9
15.9/66.0/16.
91 1.9991 4/8777
91 91.2/99. 3/89..
69.8/66.6 1
26.7/25 4/20.5
80.4/82.2/887
8/.1//2.8/86,
62.6/81 5/97.9
F-NNISTO-4
NorMNIST
71.9/67 0/38./
24.3/20.8/17.0
82.8. 1/90.8
84.3/87.7 7/904
80. 2/92//344/91
Omniglot
44.8/41 3/29.6
1/93
Gaussian-Nare Noise
0.3/0.5.98.3
2.3/2.3/12.2
99 3/877.
99.8/99 92.4
99. 99.7/99.7.744
27.7/20.5/8.4
96.4/99.//98.0
8/98.6
95.5/96
Sphere-OOD
86.5/67.3/10.7
33.7/20.477.8
71.6/862297.3
13.9688.0/96.7
67.3883.099788
16


Reviews:
[{"review_id": "SJlnCEcaFS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "** post rebuttal start **\n\nAfter reading reviews and authors' response, I decided not to change my score.\nI recommend to strengthen their theoretical justification or make their method scalable to improve their work.\n\n\nDetailed comments:\n\n2. \"Moreover, their results on some of the gray-scale experiments are significantly worse compared to ours.\"\n-> If you are talking about the comparison in MNIST-variants, please note that experimental results on MNIST cannot be seriously taken unless there is a strong theoretical background; especially, MNIST-variants are too small to talk about the scalability of the method. It is hard to convince readers only with results in MNIST-variants, unless the method has a strong theoretical justification.\nHowever, if your claim is true for general gray-scale images, e.g., preprocessing CIFAR to be in gray scale, then you may add supporting experiments about it.\n\n4. Again, if the method is only applicable to MNIST-variants due to its computational complexity while it has no strong theoretical justification, I can't find benefits from it.\n\n** post rebuttal end **\n\n\n\n- Summary:\nThis paper proposes to improve confident-classifiers for OOD detection by introducing an explicit \"reject\" class. Although this auxiliary reject class strategy has been explored in the literature and empirically observed that it is not better than the conventional confidence-based detection, the authors provide both theoretical and empirical justification that introducing an auxiliary reject class is indeed more effective.\n\n\n- Decision and supporting arguments:\nWeak reject.\n\n1. Though the analysis is interesting, it is not applicable to both benchmark datasets and real-world cases. Including the benchmark datasets they experimented, the input to the model is in general bounded, e.g., natural images are in RGB format, which is typically normalized to be bounded in [0,1]. Therefore, the polytopes would not be stretched to the infinity in most cases.\nOn the other hand, note that softmax classifiers produce a high confidence if the input vector and the weight vector of a certain class are in the same direction (of course feature/weight norm also matters, but let's skip it for simplicity). Therefore, if there is an auxiliary reject class, only data in the same direction will be detected as OOD; in other words, OOD is \"modeled\" to be in the same direction with the weight vector of the auxiliary reject class. However, the conventional confidence-based detection does not model OOD explicitly. Since OOD is widely distributed over the data space by definition, modeling such a wide distribution would be difficult. Thus, the conventional approach makes more sense to me.\n\n2. The experiment is conducted only on MNIST variations, so it is unclear whether their claim is true on large-scale datasets and real-world scenario.\nWhy don't you provide some experimental results on other datasets commonly used in other OOD detection papers, such as CIFAR, SVHN, TinyImageNet, and so on?\n\n\n- Comments:\n1. In section 4, the authors conjectured the reason why the performance of reject class in Lee et al. (2018a) was worse is that the generated OOD samples do not follow the in-distribution boundaries well. I think Appendix E in the Lee et al.'s paper corresponds to this reasoning, but Lee et al. actually didn't generate OOD samples but simply optimized the confidence loss with a \"seen OOD.\" Lee et al. didn't experiment on MNIST variations but many natural image datasets. So, it is possible that the auxiliary reject class strategy is only effective in MNIST variations. I suggest the authors to do more experiments on larger datasets to avoid this criticism.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "S1xdZMcnKS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Comments on rebuttal\n\nI don’t think that the authors made a valid argument to address my concerns about theoretical justification and experiments. As I mentioned in the review, the assumption and statements in the paper are not clear to me. Moreover, I think the authors should evaluate their methods on more realistic cases. Because of that, I’d like to keep my score. \n\n====\n\n[Summary]\n\nTo detect out-of-distribution (OOD) samples, the authors proposed to add an explicit \"reject\" class instead of producing a uniform distribution and OOD sample generation method. They showed that the proposed method can perform better than several OOD detectors on MNIST and Fashion-MNIST datasets.\n\n[Detailed comments]\n\nI'd like to recommend a \"weak reject\" due to the following reasons:\n\n1. Justification is not clear: The authors argue that arbitrarily large confidence values cannot be obtained if there are multiple K*. However, how can we guarantee that there are multiple K* only by introducing the additional class? Could the authors elaborate this more? Also, I'm not sure that the theoretical justifications are really valid because we usually consider bounded input space. \n\n2. Experimental results are not convincing: in the paper, only grayscale datasets, such as MNIST and FMNIST, are considered to evaluate the proposed method and I think it is not enough. I would be appreciated if the authors can provide more evaluations on various datasets (e.g., CIFAR, SVHN, TinyImageNet) and deep architectures (e.g., DenseNet and ResNet) similar to [Hendrycks 19, Liang' 18, Lee' 18].\n\n[Questions]\n\n1. Introducing additional class increases the number of parameters and can suffer from overfitting. Could the authors comment on overfitting issues? \n\n2. Could the authors compare the performance of the proposed method with the ensemble version of MD? \n\n3. Instead of generated OOD samples, could the authors report the performance with explicit OOD samples similar to [Hendrycks' 19]? \n\n[Hendrycks' 19] Hendrycks, D., Mazeika, M. and Dietterich, T.G., Deep anomaly detection with outlier exposure. In ICLR, 2019.\n\n[Lee' 18] Lee, K., Lee, H., Lee, K. and Shin, J., Training confidence-calibrated classifiers for detecting out-of-distribution samples. In ICLR, 2018.\n\n[Liang' 18] Liang, S., Li, Y. and Srikant, R., Enhancing the reliability of out-of-distribution image detection in neural networks In ICLR, 2018.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HJeHSCR-5r", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an algorithm to generate boundary OOD positive/negative samples to train a classifier for OOD samples. The algorithm is based on the theoretical analysis on why confidence value could be high in unbounded polytopes. CVAE is used as a generative model to get new samples. Experiments are conducted on MNIST and Fashon-MNIST datasets, which are used as OOD and in-distribution, and vice versa. Other datasets are also used for OODs. Comparison are made with Confident-Classifier, ODIN, and Mahalanobis distance-based approach, and the proposed method outperforms the others.\nOverall the paper is well-written and well-organized. The proposed method is based on the idea from theoretical analysis, and is reasonable and valid. There are only a couple of things to point out: First, the methods to compare, such as Confident-Classifier and ODIN, are not so strong. Thus, I am not sure whether the performance of the proposed algorithm is dramatically better. Second, I would like to see the sensitivity analysis of the proposed method, because there are several hyper-parameters as mentioned in the paper.\nHowever, I like the method and could be accepted as an ICLR paper.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper improves the previous method for detecting out-of-distribution  (OOD) samples. 

Some theoretical analysis/motivation is interesting as pointed out by a reviewer. I think the paper is well written in overall and has some potential.

However, as all reviewers pointed out, I think experimental results are quite below the borderline to be accepted (considering the ICLR audience), i.e., the authors should consider non-MNIST-like and more realistic datasets. This indicates the limitation on the scalability of the proposed method. 

Hence, I recommend rejection.
Decision: Reject