Are all outliers alike?  On Understanding the Diversity of Outliers for Detecting OODs
Authors: Ramneet Kaur
Susmit Jha
Anirban Roy
Venue:  (2021)
Keywords: OOD
out of distribution
trust
model confidence
DNN
deep learning

Abstract: Deep neural networks (DNNs) are known to produce incorrect predictions with very high confidence on out-of-distribution (OOD) inputs. This limitation is one of the key challenges in the adoption of deep learning models in high-assurance systems such as autonomous driving, air traffic management, and medical diagnosis. This challenge has received significant attention recently, and several techniques have been developed to detect inputs where the model's prediction cannot be trusted. These techniques use different statistical, geometric, or topological signatures. This paper presents a taxonomy of OOD outlier inputs based on their source and nature of uncertainty. We demonstrate how different existing detection approaches fail to detect certain types of outliers. We utilize these insights to develop a novel integrated detection approach that uses multiple attributes corresponding to different types of outliers. Our results include experiments on CIFAR10, SVHN and MNIST as in-distribution data and Imagenet, LSUN, SVHN (for CIFAR10), CIFAR10 (for SVHN), KMNIST, and F-MNIST as OOD data across different DNN architectures such as ResNet34, WideResNet, DenseNet, and LeNet5.

Under review as a conference paper at ICLR 2021
ARE ALL OUTLIERS ALIKE? ON UNDERSTANDING THE DIVERSITY OF OUTLIERS FOR DETECTING OODS
Anonymous authors Paper under double-bling blind review
ABSTRACT
Deep neural networks (DNNs) are known to produce incorrect predictions with very high confidence on out-of-distribution (OOD) inputs. This limitation is one of the key challenges in the adoption of deep learning models in high-assurance systems such as autonomous driving, air traffic management, and medical diag- nosis. This challenge has received significant attention recently, and several tech- niques have been developed to detect inputs where the model's prediction cannot be trusted. These techniques use different statistical, geometric, topological signatures. This paper presents a taxonomy of OOD outlier inputs based on their source and nature of uncertainty. We demonstrate how different existing detec- tion approaches fail to detect certain types of outliers. We utilize these insights develop a novel integrated detection approach that uses multiple attributes cor- responding to different types of outliers, Our results include experiments on CI- FARIO, SVHN and MNIST as in-distribution data and Imagenet, LSUN, SVHN (for CIFARI0), CIFAR10 (for SVHN), KMNIST, and F-MNIST as OOD data across different DNN architectures such as ResNet34, WideResNet, DenseNet, and LeNet5.
INTRODUCTION
Deep neural networks (DNNs) have achieved remarkable performance-levels many areas such as computer vision (Gkioxari et al. 2015). speech recognition (Hannun et al. 20014), and text analy- sis (Majumder et al. 2017). ) But their deployment in the safety-critical systems such as self-driving vehicles (Bojarski et al., 2016). aircraft collision avoidance (Julian & Kochenderter, [007)) and medical diagnoses (De Fauw al.] (0188) is hindered by their brittleness. One major challenge is the inability of DNNs to be self-aware of when new inputs are outside the training distribution and likely to produce incorrect predictions. It has been widely reported literature (Guo et al., 2017a Hendrycks & Gimpel 2016) that deep neural networks exhibit overconfident incorrect predictions on inputs which are outside the training distribution. The responsible deployment of deep neural network models in high-assurance applications necessitates detection of diit----dittiittioonn (OOD) data so that DNNs can abstain from making decisions on those.
Recent approaches for OOD detection consider different statistical, geometric topological sig- natures in data that differentiate OODs from the training distribution. For example, the changes in the softmax scores due to input perturbations and temperature scaling have been used detect OODs (Hendrycks & Gimpel 2016 Liang et al., 2017] Guo et al., (017b). Papernot & McDaniel (0018) use the conformance among the labels of the nearest neighbors while Tack al. (2020) use cosine similarity (modulated by the norm of the feature vector) to the nearest training sample for the detection of OODs. Lee et al. (2018) consider the Mahalanobis distance of an input from the in-distribution data to detect OODs. Several other metrics such as reconstruction error (An & Cho, 2015), likelihood (ratio of the distance to the nearest class different from the predicted class and the distance to the predicted class) (Jiang et al. 2018), density function (Liu et al. 2020 Hendrycks et al. 201 probability distribution of the softmax scores (Lee et al. 2017]] Hendrycks et al. [019b] Tack al.) 2020 (Hendrycks et al.) (2119) have also been used to detect OODs. All these methods attempt to develop uniform approach with single signature to detect all OODs accompanied by empirical
between the in-distribution and OOD samples (Ren et al. 2019). trust scores

---

Under review as a conference paper at ICLR 2021
evaluations that use datasets such CIFAR1O as in-distribution data and other datasets such as SVHN as OOD.
Our study shows that OODs be of diverse types with different defining characteristics. Conse- quently, an integrated approach that takes into account the diversity of these outliers needed for effective OOD detection. We make the following three contributions in this paper:
Taxonomy of OODs. We define taxonomy of OOD samples that classify OODs into different
types based aleatoric vs epistemic uncertainty (Hällermeier & Waegeman, 2019). distance from the predicted class vs the distance from the tied training distribution, and uncertainty in the principal components uncertainty in non-principal components with low variance.
Incompleteness of existing uniform OOD detection approaches. We examine the limitations of the state-of-the-rtt approaches to detect various types of OOD samples. We observe that not all outliers alike and existing approaches fail to detect particular types of OODs. We use a toy dataset comprising two halfmoons as two different classes to demonstrate these limitations.
An integrated OOD detection approach. We propose integrated approach that detect dif- ferent types of OOD inputs. We demonstrate the effectiveness of our approach on several bench- marks, and compare against state-of-the-ars OOD detection approaches such as the ODIN (Liang et al. 2017) and Mahalanobis distance method (Lee et al. 2018).
2 OOD TAXONOMY AND EXISTING DETECTION METHODS
DNNs predict the class new input based on the classification boundaries learned from the sam- ples of the training distribution. Aleatory uncertainty is high for inputs which close to the classification boundaries, and epistemic uncertainty is high when the input is far from the learned distributions of all classes (Hora) 1996 Hüllermeier & Waegeman, 2019). Given the predicted class of DNN model on a given input, we can observe the distance of the input from the distribution of this particular class and identify it as an OOD if this distance is high. We use this top-down infer- ence approach to detect this type of OODs which are characterized by an inconsistency in model's prediction and input's distance from the distribution of the predicted class. Further, typical inputs to DNNs are high-dimensional and can be decomposed into principal and non-principal components based on the direction of high variation; this yields another dimension for classification of OODs. We, thus, categorize OOD using the following three criteria.
the OOD associated with higher epistemic or aleatoric uncertainty, i.e., the input away from in-distribution data it be confused between multiple classes?
2. Is the epistemic uncertainty of an OOD sample unconditional or is it conditioned on the class predicted by the DNN model?
3. Is the OOD outlier due to unusually high deviation in the principal components of the data or due to small deviation in the non-principal (and hence, statistically invariant) components?
Figure demonstrates different types of OODs which differ along these crite- ria. Type OODs have high epistemic uncertainty and are away from the in- distribution data. Type 2 OODs have high epistemic uncertainty with respect to each of the 3 classes even though approx- imating all in-distribution (ID) data using single Guassian distribution will miss these outliers. Type 3 OODs have high aleatoric uncertainty as they are close to the decision boundary between class O and class I. Type and 5 have high epis- temic uncertainty with respect to their closest classes. While Type OODs are far from the distribution along the princi- pal axis, Type OODs vary along a rel- atively invariant axis where even small
OOD Types
In dist classes
Figure 1: The different types of OODs in a 2D space with three different classes. The class distributions represented as Gaussians with black boundaries and the tied distribution of all training data is Gaussian with red boundary.
2

---

Under review as a conference paper at ICLR 2021
deviation indicates that the sample is an OOD.
Limitations of Existing Detection Methods. We empirically demonstrate the limitations of existing OOD detection methods on two-dimensional (2D) half-moon dataset with two classes. As shown in Figure 2 consider three clusters of OOD samples: cluster A (black), B (brown) and C(red). Figure 2 (right) shows the 2D penultimate features of the classifier.
In- classes
In-dist classes
0
OOD Clusters A
0.64
OOD Clusters A
B
C
Figure 2: The cluster A (black), cluster B (brown), and cluster C (red) clusters represent the OOD due to epistemic uncertainty the tied training distribution, epistemic uncertainty in the class- conditional training distribution, and the aleatoric uncertainty in the class-conditional distribution, respectively. (Left) shows the training data the 2 half-moon classes and the 3 OOD clusters the input space along with the trained classifier's boundary and its softmax scores. (Right) shows the II) samples and the OODs after projection to the 2D) feature space (penultimate layer) of the DNN.
Different approaches differ in their ability detect different OOD types illustrated in Figure
Figure aa shows that the Mahalanobis distance (Lee et al. 2018) from the mean and tied co- variance of all the training data in the feature space cannot detect OODs in the clusters B and C corresponding to consiconnitional epistemic uncertainty and aleatoric uncertainty, respectively. It attains the overall true negative rate (TNR) 39.09% the 95% positive rate (TPR).
Figure b)b shows that the softmax prediction probability (SPB) (Hendrycks & Gimpel 2016 cannot detect the OODs cluster corresponding to high epsitemic uncertainty. The TNR at 95% TPR) reported by the SPB technique is 60.91% %.
Figure 3 c) shows that class-wise Principal Component Analysis (PCA) (Hoffmann) 2007 cannot detect OODs in cluster C corresponding high alcatoric uncertainty. We performed PCA of the two classes separately in the feature space and used the minimum reconstruction error to detect OODs. This obtained overall TNR of 80.91 95% TPR).
Figure 3 shows that K-Nearest Neighbor (kNN) (Papernot & McDaniel 2018 conformance in the labels of the nearest neighbors cannot detect OODs in clusters A and B with high epistemic uncertainty. The overall TNR (at 95% TPR) reported by this technique is 15%.
These observations can explained by the focus different detection techniques on measuring different forms of uncertainty. This motivates our integrated OOD detection method.
INTEGRATED OOD DETECTION METHOD
Complementary information about different OOD types can be used to detect wider range of OODs. Figure shows the improvement in the TNR of the OOD detector composed with informa- tion about different classes of OODs the two half-moons dataset. Non-confornity in the labels of the nearest neighbors captures OODs in cluster C. Mahalanobis distance from the tied in-distribution detects OODs in cluster A. Reconstruction error from the PCA of the 2 class distributions captures OODs in cluster B. Softmax scores further strengthens the OOD detection by reporting OODs in cluster C that undetected by the other three methods.
The integrated OOD detection approach, thus, uses the following attributes, each specialized in detecting specific type (or combination of types) of OODs:
3

---

Under review as a conference paper at ICLR 2021
OODs
0.96
OODs
Undetected Detected
Undetected
Detected
0.72
In-dist classes
0.64
In-dist classes
0
1
0.48
(a) Estimating distance tied in-distribution fails to detect OOD clusters and C.
(b) Techniques based on softmax score tect OODs in cluster A.
OODs
OODs
Undetected
Undetected
Detected
Detected
In-dist classes
In-dist classes
0
0
(c) Estimating distance from the class-wise in- distribution fails detect OODs in cluster
(d) Non-conformance among the nearest neighbors fails to detect OODs in cluster and
Figure 3: Detected OODs are shown in blue and undetected OODs are in red. Different techniques fail to detect different types of OODs.
1. Mahalanobis distance from the in-distribution density estimate that considers either tied (Lee et al.) 2018) or class-wise covariance estimate. This attribute captures the overall or class- conditional epistemic uncertainty of an OOD, Our refinement also use class-wise covariance significantly improves detection of OODs when coupled with PCA approach described below.
Conformance measure among the variance of the Annoy (Bernhardsson 2018) nearest neighbors calculated as the Mahalanobis distance of the input's conformance the closest class confor- mance. Our experiments found this to be very effective in capturngg aleatoric uncertainty. This new attribute is fusion of nearest-neighbor and Mahalanobis distance methods in literature.
3. Prediction confidence of the classifier as the maximum softmax score the perturbed input where the perturbation used the same as ODIN approach (Liang et al. 2017). This boosts the detection of high aleatoric uncertainty by sharpening the class-wise distributions.
4. Reconstruction error using top 40% of PCA components where the components are obtained via class conditional PCA of the training data. This boosts the detection high class-wise epistemic uncertainty by eliminating irrelevant features.
This fusion of attributes from existing tfate-of-the-an detection methods and new attributes was found to the most effective integrated appraoch capable of detecting the different types of OODs. We evaluated it several benchmarks discussed in Section with ablation study in Appendix.
4

---

Under review as a conference paper at ICLR 2021
Undetected
Undatacted
Delected
Delected
In.dist classes
000s
0ODs
Undetacted
Undetectad
Detected
Detected
In-dist classes
In-dist diasses
Figure 4: Complementary information about different types of OODs improves detection. (Top-left) 15% TNR with non-conformance among the labels of the nearest neighbors. (Top-right) Adding Mahalanobis distance over the tied in in-distribution improves TNR to 54.09 %. (Bottom-left) Adding Class-wise PCA further improves TNR to 95.91 % TNR. (Bottom-right) Adding softmax score fur- ther improves TNR to 99.55 %c TPR is 95% in all the cases.
EXPERIMENTAL RESULTS
Attributes forming the signature of the OOD detector used the experiments The signature of the OOD detector used in the experiments is the weighted sum of four attributes, one from each of the following four categories:
1. Distance from the in-distribution density estimate: We use mahalanobis distance of the input with respect to the closest class conditional distribution. The parameters this distance chosen from one of the following two categories:
empirical class means and tied empirical covariance of training samples empirical class means and empirical class covariance of training samples
2. Reconstruction error: We perform class conditional PCA empirically from the training samples. We use minimum reconstruction error of the input from the top 40% eigen vectors of the class conditional eigen spaces.
3. Prediction confidence of the classifier: We maximum value of the temperature scaled softmax scores (S) on the perturbed input. Perturbations to the input (x) are made according the following equation (Liang al. 2017)
= esign( V V,looSy(a;77 ((x;T))
(1)
The values of the magnitude of noise (c) and the temperature scaling parameter (T) chosen from one of the following three categories:
0 and T
0 and T 10
0.005 and T' 10
4. Conformance measure among the nearest neighbors: We compute m-dimensional feature vector to capture the conformance among the input's nearest neighbors the train- ing samples, where is the dimension of the input. We call this m-dimensional feature vector as the conformance vector. The conformance vector is calculated by taking the mean deviation along each dimension of the nearest neighbors from the input. We hypoth- esize that this deviation for the in-distribution samples would vary from the OODs due to aleatory uncertainty.
5

---

Under review as a conference paper at ICLR 2021
The value of the conformance measure is calculated by computing mahalanobis distance of the input's s conformance vector to the closest class conformance distribution. Similar to the distance for the in-distribution density estimate, the parameters of this mahalanobis distance are chosen from the following two categories:
empirical class means and tied empirical covariance on the conformance vectors of the training samples
empirical class means and empirical class covariance on the conformance vectors of the training samples
The value of the number of the nearest neighbors is chosen from the set {10, 20, 30, 40, 50) via validation. We used Annoy (Approximate Nearest Neighbors Oh Yeah) (Bernhardsson 2018 to compute the nearest neighbors.
The weights of the four attributes forming the signature of the OOD detector are generated in the following manner. We use small subset (1000 samples) of both the in-distribution and the gen- crated OOD data to train a binary classifier using the logistic loss. The OOD data used to train the classifier is generated by perturbing the in-distribution data using the Fast Gradient Sign attack (FGSM) (Goodfellow et al. 2014). The trained classifier (or OOD detector) is then evaluated on the real OOD dataset at the True Positive Rate of 95%. The best result, in terms of the highest TNR on the validation dataset (from the training phase of the OOD detector), from the twelve combinations of the aforementioned sub-categories (one from each of the four attributes) are then reported on the test (or real) OOD datasets.
Datasets and metries. We evaluate the proposed integrated OOD detection on benchmarks such as CIFARIO (Krizhevsky et al., 2009) and SVHN (Netzer et al., 2011 We consider standard met- rics (Hendrycks & G 2016]] Liang et al., 2017 Lee et al. 2018) such as the true negative rate (TNR) at 95% true positive rate (TPR), the area under the receiver operating characteristic curve (AUROC), area under precision recall curve (AUPR), and the detection accuracy (DTACC) to eval- uate our performance.
DNN-based classifier architectures. To demonstrate that the proposed approach generalizes across various network architectures, we consider wide range of DNN models such as ResNet (He et al.) 2016), WideResNet (Zagrruyko & Komodakis 2016), and DenseNet (Huang et al., 2017).
Comparison with the statc-of-the-art. We compare our approach with the three state-of-the-art approaches: SPB (Hendrycks & Gimpel 2016), ODIN (Liang et al. 2017). and Mahalanobis (Lee eet al. 2018). For the ODIN method, the perturbation noise is chosen from the set {0, 0.0005, 0.001, 0.0014, 4, 0.002, 0.0024, 0.005, 0.01, 0.05, 0.1, 0.2). and the temperature T is chosen from the set {1, 10, 100, 1000). These values are chosen from the validation set of the adversarial samples of the in-distribution data generated by the FGSM attack. For the Mahalanobis method, we consider their best results obtained after feature ensemble and input preprocessing with the hyperparameters of their OOD detector tuned on the in-distribution and adversarial samples generated by the FGSM attack. The magnitude of the noise used in pre-proces pre-processing of the inputs is chosen from the set {0.0, 0,01, 0.005, 0.002, 0.0014,00 0.001, 0.0005}.
CIFAR1O. With CIFARIO as in-distribution, we consider SVHN (Netzer al., 2011). Tiny- Imagenet (Deng et al.] 2009). and LSUN (Yu et al. 2015) as the OOD datasets. For CIFAR10, we consider two DNNs: ResNet50, and WideResNet. Table shows the results.
SVHN. With SVHN as in-distribution, we consider CIFAR1O, Imagenet, and LSUN and as the OOD datasets. For SVHN, we use the DenseNet classifier. Table [ shows the results.
Key observations. We do not consider pre-processing of the inputs in our integrated OOD detector. Even without input re-processing and with the exception of CIFARIO OOD dataset for SVHN in- distribution trained on DenseNet, could perform equally well (and even out-perform in most of the cases) as the Mahalanobis method on its best results generated after pre-processing the input.
We also consider Subset-CIFAR 100 as OODs for CIFARIO. Specifically, from the CIFAR100 classes, we select sea, road, bee, and butterfly as OODs which are visually similar to the ship, auto- mobile, and bird classes in the CIFAR1O, respectively. Thus, there be numerous OOD samples due to aleatoric and class-conditional epistemic uncertainty which makes the OOD detection chal- lenging. Figure shows the t-SNE (Maaten & Hinton 2008) plot of the penultimate features from

---

Under review as a conference paper at ICLR 2021
Table I: Comparison of TNR, AUROC, DTACC, AUPR with SPB, ODIN and Mahalanobis methods
In-dist
OOD Dataset
Method
TNR
AUROC
DTACC
AUPR
(model)
CIFARIO (ResNet55)
SVHN
SPB
44.69
97.31
86.36
87.78
ODIN
63.57
93.53
86.36
87.58
Mahalanobis
72.89
91.53
85.39
73.80
Ours
85.90
95.14
90.66
80.01
Imagenet
SPB
42.06
90.8
84.36
92.6
ODIN
79.48
96.25
90.07
96.45
Mahalanobis
94.26
97.41
95.16
93.11
Ours
95.19
97.00
96.02
90.92
LSUN
SPB
48.37
92.78
86.97
94.45
ODIN
87.29
97.77
92.65
97.96
Mahalanobis
98.17
99.38
97.38
98.69
Ours
99.36
99.65
98.57
98.96
CIFARIO (WideResNet)
SVHN
SPB
45,46
90.10
82.91
82.52
ODIN
57.14
89.30
81.14
75.48
Mahalanobis
85.86
97.21
91.87
94.69
Ours
88.95
97.61
92.46
92.84
LSUN
SPB
52.64
92.89
86.81
94.13
ODIN
79.60
96.08
89.74
96.23
Mahalanohis
95.69
98.93
95.41
98.99
Ours
98.84
99.63
97.72
99.25
SVHN (DenseNet)
Imagenet
SPB
79.79
94.78
90.21
97.2
ODIN
79.8
94.8
90.2
97.2
Mahalanobis
99.85
99.88
98.87
99.95
Ours
98.02
98.34
98.00
97.05
LSUN
SPB
77.12
94.13
89.14
96.96
ODIN
77.1
94.1
89.1
97.0
Mahalanobis
99,99
99.91
99.23
99,97
Ours
99.74
99.79
99.08
99.65
CIFAR1O
SPB
69.31
91.9
86.61
95.7
ODIN
69.3
91.9
86.6
95.7
Mahalanobis
97.03
98.92
96.11
99.61
Ours
94.87
98.41
94.97
98.76

---

Under review as a conference paper at ICLR 2021
Airplane
Automobile
Cat
Deer
Dog
Frog
Horse
Ship
Truck
OOD
Figure 5: t-SNE plot of the penultimate layer feature space of ResNet50 trained on CIFAR1O. We show four OOD images from the SCIFAR 100. OOD and OOD 2 are far from the distributions of all classes and thus represent OODs due to epistemic uncertainty. OOD 3 and OOD are OODs due to aleatoric uncertainty as they lie closer to two class distributions. Third OOD is closer to the cat and frog classes of the ID and forth OOD is closer to the airplane and automobile classes of the ID. Mahalanobis distance cannot detect these OODs but our integrated approach can detect them.
the ResNet50 model trained on CIFAR10. We show 4 examples of OODs (2 due to epistemic and due to aleatoric uncertainty) from Subset-CIFAR100 These OODs were detected by integrated approach but missed by the Mahalanobis approach.
These observations justify the effectiveness of integrating multiple attributes detect OOD samples.
Additional experimental results in the appendix. We also compare the performance the in- tegrated OOD detector with the SPB, ODIN and Mahalanobis detector in supervised settings, as reported by the Mahalanobis method for OOD detection (Lee et al., 2018). These results include experiments on CIFARIO, I0, SVHN and MNIST as in-distribution data and Imagenet,LSUN, SVHN (for CIFARIO), CIFARIO (for SVHN), KMNIST, and F-MNISTas OOD data across different DNN architectures such as ResNet34, WideResNNet,ensseNe,, and LeNet5. All these results, along with the ablation studies on OOD detectors with single attributes are included in the Appendix. In al- most all of the reported results in the Appendix, our OOD detector could outperform the compared state-of-the-arr methods with improvements of even 2X higher TNR at 95% TPR in some cases.
DISCUSSION AND FUTURE WORK
Recent techniques propose refinement in the training process of the classifiers for OOD detection. Some of these techniques include fine-tuning the classifier's 's training with an auxiliary cost function for OOD detection (Hendrycks et al., 2019a Liu al. 2020). Other techniques make use of self- supervised models for OOD detection (Tack et al.] 2020; Hendrycks et al.) 2019b). We perform preliminary experiments to compare the performance of these techniques with our integrated OOD detector that makes use of the feature space of the pre-trained classifiers to distinguish in-distribution samples from OODs. Our approach does not require modification the training cost function of the original task. These results reported in the Appendix. We consider making use of the feature space of in our OOD detection technique as a promising prospective future work. Another direction of the future work detection (Liu et al. 2020; Hendrycks et al.] (209a; Tack et al.) 2020 [enddryck et al., (019b) as attributes (or categories of attributes) forming the signature of the integrated OOD detector. Another
explore the score functions used in these refined training processes for OOD
8

---

Under review as a conference paper at ICLR 2021
avenue of future work is explore OOD generation techniques other than adversarial examples generated by the FGSM attack for training of the integrated OOD detector.
CONCLUSION
We introduced a taxonomy of OODs and proposed an integrated approach to detect different types of OODs. Our taxonomy classifies OOD the nature of their uncertainty and we demonstrated that no single wate-offthhearr approach detects all these OOD types. Motivated by this observation, we formulated integrated approach that fuses multiple attributes target different types of OODs. We have performed extensive experiments on a synthetic dataset and several benchmark datasets (e.g., MNIST, CIFARIO, SVHN). Our experiments show that our approach can accurately detect various types of OODs coming from a wide range of OOD datasets such as KMNIST, Fashion- MNIST, SVHN, LSUN, and Imagenet. We have shown that our approach generalizes over multiple DNN architectures and performs robustly when the OOD samples are similar to in-distribution data.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Jinwon An and Sungzoon Cho. Variational autoencoder hased anomaly detection using reconstruc- tion probability. Special Lecture on IE, 2(1):1-18, 2015.
Erik Bernhardsson. Annoy, 2018. URL
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp. Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. End to end learning for self-driving cars. arXiv preprint 1604.07/60 2016.
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. arXiv preprint arXiv: 1812.0118 2018.
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelli- gence and statistics, pp. 215-223, 2011.
Jeffrey De Fauw, Joseph R Ledsam, Bernardino Comera--Predees Stanislav Nikolov, Nenad Toma- sev, Sam Blackwell, Harry Askham, Xavier Glorot, Brendan O' D'Donoghue, Daniel Visentin, et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature medicine, 24(9):1342-1350 2018
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and patlern recognition, pp. 248-255. Icee, 2009.
Georgia Gkioxari, Ross Girshick, and Jitendra Malik. Contextual action recognition with r* cnn. In Proceedings of the IEEE international conference on computer vision, pp. 1080-1088, 2015.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6653 2014.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. arXiv preprint arXiv:1706.0453 706.04599 2017a.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. arXiv preprint arXiv:1706.04559 2017
Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al. Deep speech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5555 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136. 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. International Conference on Learning Representations, 2019a
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using supe-supe learning can improve model robustness and uncertainty. In Advances in Neural Information Processing Systems, pp. 15663-15674, 2019b.
Heiko Hoffmann. Kernel pca for novelty detection. Pattern recognition, 40(3):863-874, 2007.
Stephen Hora. Alcatory and epistemic uncertainty in probability elicitation with an example from hazardous waste management. Reliability Engineering & System Safety, 54(2-3):217-223, 1996.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
10

---

Under review as a conference paper at ICLR 2021
Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: A tutorial introduction. arXiv preprint 0rxiv:1910.094:3 2019.
Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust classifier. In Advances in neural information processing systems, pp. 5541-5552, 2018
Kyle D Julian and Mykel J Kochenderfer. Neural network guidance for uavs. In AIAA Guidance, Navigation, and Control Conference, pp. 1743, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrateed classifiers
for detecting out- out-of-distrinn of-distribution samples, arXiv preprint arXiv:1711.093255, 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Process- ing Systems, pp. 7167-7177, 2018.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.0000 2017.
Weitang Liu, Xiaoyun Wang, John Owens, and Sharon Yixuan Li. Energy-based out-of ofdtsiribittti detection. Advances in Neural Information Processing Systems, 33, 2020.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, (ANNo))2579-2600) 2008
Navonil Majumder, Soujanya Poria, Alexander Gelbukh, and Erik Cambria. Deep learning- document modeling for personality detection from text. IEEE Intelligent Systems, 32(2):74-79, 2017.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning, 201 1.
Nicolas Papernot and Patrick McDaniel. Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning. arXiv preprint
2018.
Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distri detection. In Advances in Neural Information Processing Systems, pp. 14707-14718, 2019.
Jihoon Tack, Sangwoo Mo. Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via con- trastive learning on distributionally shifted instances. Advances Neural Information Processing Systems, 33, 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist. a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint arXivv 2017.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:/506.0333 2015.
Sergey Zagoruyko and Nikos Komodakis. 16Xiv:1605.07146, 2016.
Wide residual networks.
arXiv preprint
11

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 DEFINING OODS DUE TO EPISTEMIC AND ALEATORIC UNCERTAINTY
In general, let there be l classes C1, .2... Cs and the distribution of training data for each class is p(z|c)). The overall training distribution is denoted by p(z). Now, given a new input 2 to the trained DNN model M, let é M ( denote the predicted class. The flowchart in Figure shows different sources of uncertainty that could make I an OOD.
Model
output
P(R)
Is P(RIe) low?
Is there = with P(fcc) high?
2 is not OOD
Input
low?
No
No
No
Yes
Yes
Yes
& is OOD
.2 is OOD (high conditional epistemic uncertainty)
8 is OOD (high aleatoric uncertainty)
(high epistemic uncertainty)
Figure 6: OODs due to High Epistemic and Aleatoric Uncertainty
A.2 ADDITIONAL EXPERIMENTAL RESULTS
We first present preliminary results for comparison with the OOD detection techniques based on fine-tuning of the classifiers (Hendrycks et al. 2019a; Liu et al. 2020 Tack et al. [2200] Hendrycks al. 2019b). We then present our results on various vision datasets and different architectures the pre-trained DNN based classifiers for these datasets in comparison to the ODIN, the Mahalanobis and the SPB methods in supervised settings. Finally, we then report results from the ablation study on OOD detection with individual attributes and compare it with our integrated approach OOD detection.
A.2.1 COMPARISON WITH THE OOD DETECTION TECHNIQUES BASED ON REFINEMENT OF THE TRAINING PROCESS FOR CLASSIFIERS
Recent techniques propose refinement in the training process of the classifiers for OOD detection. Some of these techniques include fine-tuning the training of classifiers with trainable cost func- tion for OOD detection (Hendrycks et al.) [0019]] Liu et al., 2020),, superviee training of the classifiers to enhance OOD detection (Tack et al. 2020 Hendrycks et al. 2019b) etc.
We perform preliminary experiments to compare the performance of these techniques with our in- tegrated OOD detector that uses features of the pre-trained classifiers to distinguish in-distribution samples from OODs. Table 2 compares TNR (at 95% TPR), AUROC and AUPR for the energy based OOD detector (Liu et al. 2020) and our integrated OOD detector
CIFARIO with pre- trained WideResNet model. The integrated OOD detector was trained on in-distribution and adver- sarial samples generated by FGSM attack. Table 3 compares the results of the WideResNet model trained on CIFARIO 10 and fine-tuned with the outlier exposure from the 80 Million Tiny Images with our OOD detector that uses features from the pre-trained WideResNet model trained CIFAR 10. Since 80 Million Tiny Images dataset is no longer available for use, we used a small subset of Im- ageNet (treated as OOD dataset for CIFAR1O and SVHN datasets (Lee et 2018)) for generating OODs for training of the integrated OOD detector. Table 4 compares the OOD detection perfor- mance of the sulf-super training based OOD detector with our method. We trained our OOD detector with the in-distribution CIFARIO as in-distribution samples and adversarial samples gener- ated by FGSM attack from the test dataset of CIFARIO as OODs. The trained OOD detector was then tested on LSUN OODs and the results are reported in Table 4 With ResNet-50 the classi- fier for CIFAR 10, we trained our OOD detector with the in-distribution CIFAR 10 as in-distribution samples and adversarial samples generated by FGSM from the test dataset of CIFAR10 as OODs. The trained OOD detector was then tested on SVHN as OODs and these results are compared with the contrastive based learning for OOD detection Taak al. 2020) in table
12

---

Under review as a conference paper at ICLR 2021
Table 2: Results with Energy based OOD detector (Liu et al. 2020 Our method.
In-dist (model)
OOD dataset
TNR (TPR=95%)
AUROC
AUPR
CIFARIO (WideResNet)
SVHN LSUN
64.41 / 88.95 72.42 98.84
90.96/97.66
97.64/92.88
94.24 / 99.63
98.67 99.25
Table 3: Results with Outlier Exposure based OOD detector
(Heddrycks et al. 2019a) /
Our method.
In-dist (model)
OOD dataset
TNR
AUROC
AUPR
(TPR=95%) %)
CIFARIO (WideResNet)
SVHN
95.64 92.53
99.74/96.62
Table 4: Results with self-superviser learning based OOD detector (Hendrycks et al. 2019b / Our method.
In-dist
OOD
TNR (TPR=95%)
AUROC
DTACC
(model)
dataset
CIFAR10 (WideResNet)
LSUN
/1.3/98.84
93.2/99.63
71.0/97.72
Table 5: Results with contrastive learning based OOD detector (Tack al. 2020 / Our method.
In-dist
OOD
TNR (TPR=95%
AUROC
DTACC
AUPR
(model)
dataset
CIFARIO (ResNet50)
SVHN
97.2 82.88
99.5 196.98
96.7 / 99.74
99.6 / 94.71
13

---

Under review as a conference paper at ICLR 2021
A.2.2 COMPARISON WITH THE TFATE-OF-THE-ARE OOD DETECTION METHODS IN SUPERVISED SETTINGS ON PRE- TRRINED CLASSIFIERS
We compare our results with the state-of-the-art methods in supervised settings, as reported by the Mahalanobis method for OOD detection (Leee et al., 2018). In supervised settings, small subset of the real OOD dataset is used in the training of the OOD detector.
Datasets and metrics. We evaluate the proposed integrated OOD detection on benchmarks such as MNIST (LeCun et al., 1998), CIFARIO (Krizhevsky et al. 2009), and SVHN (Netzer et al., 2011).
We consider standard metrics (Hendryeks & Gimpel| 2006 Liang et al. 2017 Lee et al. 2018) such as the true negative rate (TNR) at % true positive rate (TPR), the area under the receiver operating
characteristic curve (AUROC), area under precision recall curve (AUPR) with both in-distribution and OODs as positive samples (AUPR IN and AUPR OUT respectively), and the detection accuracy (DTACC) to evaluate our performance.
DNN-based classifier architectures. To demonstrate that the proposed approach generalizes across various network architectures, we consider a wide range of DNN models such as Lenet (LeCun) et al. 1998). ResNet (He et al. 2016). and DenseNet (Huang al. 2017).
Comparison with the ofate-of-the-art. We compare our approach with the three state-of-the-an approaches: SPB (Hendrycks & Gimpel 2016). ODIN (Liang et al., (017) and Mahalanobis (Lee) cct al. 2018). Since, these experiments are performed in supervised settings, we fix T = 10 and 0.005 for generating results from the ODIN method. For Mahalanobis distance, we consider the distance in the penultimate layer feature space as well as features from all the layers of the DNN without preprocessing of the input in either settings.
MNIST. With MNIST as in- di-dstribbutonn, we consider KMNIST (Clanuwat et al. 2018)) and Fashion- MNIST(F-MNIST) (Xiao et al., 2017) as OOD datasets. For MNIST, we use the LeNet5 (LeCun) eet al. 1998) DNN. Results in terms of TNR (at 95% TPR), AUROC, and DTACC are reported in tables 6 7 and 8 Table 6 shows the results with the features from the penultimate layer in comparison to the ODIN and Mahalanobis methods. Table/]shows the results with the features from all the layers in comparison to the Mahalanobis method. T shows the results with the features from the penultimate layer in comparison to the SPB method. In all these settings, our approach outperforms the of-e-of-thh-arr art approaches for both the OOD datasets. Results comparison to AUPR IN and AUPR OUT are shown in table Here also, our technique out-performs all the three OOD detectors on all the test cases.
CIFAR1O. With CIFAR10 as in-distribution, we consider STL10 (Coates et al. 2011), SVHN (Net- zer et al. 201T), Imagenet (Deng et al. 2009), LSUN (Yu et al. 2015), and a subset of CIFARIO0 (SCIFAR (OO) (Krizhevsky et al.] 2009 as OOD datasets. For CIFARIO, we consider three DNNs: DenseNet, ResNet34, and ResNet50. Results in terms TNR (at 95% TPR), AUROC, and DTACC reported in tables 7 and g Table shows the results with the features from the penultimate layer in comparison to the ODIN and Mahalanobis methods. Table shows the results with the fea- tures from all the layers in comparison to the Mahalanobis method. T shows the results with the features from the penultimate layer in comparison to the SPB method. Results in comparison to AUPR IN and AUPR OUT are shown in tables 10 and 12 Here also, the integrated OOD detection technique could out-perform the other three detectors on most of the test cases.
Note that images from STL10 and the subset of CIFARI00 are quite similar to CIFARIO images. Furthermore, from the CIFAR 100 classes, we select sea, road, bee, and butterfly as OODs which are visually similar to the ship, automobile, and bird classes in the CIFAR1O. respectively.
SVHN. With SVHN as in-distribution, we consider STL CIFAR1O, Imagenet, LSUN and, SCI- FAR100 as OOD datasets. For SVHN we consider two DNNs: DenseNet and ResNet34. Results in terms of TNR (at 95% TPR), AUROC, and DTACC are reported in tables 6 7 and 8 Table 6 shows the results with the features from the penultimate layer in comparison to the ODIN and Ma- halanobis methods. Table shows shows the results with the features from all the layers in comparison to the Mahalanobis method. Table 8 shows the results with the features from the penultimate layer in comparison to the SPB method. Results in comparison to AUPR IN and AUPR OUT are shown in tables [ and [4] Here also, the integrated OOD detection technique could out perforrr the other three detectors most of the test cases.
14

---

Under review as a conference paper at ICLR 2021
Key observations. As shown in Table 6 and Table 7 and Table 8 approach outperforms the state-of-the-arr on all three datasets and with various DNN architectures. On CIFAR1O, in terms of the TNR metric, approach with Resnet50 outperforms Mahalanobis by 56% when SVHN is OOD and our approach with Resnet34 outperforms ODIN by 36% when LSUN is OOD.
While considering STL10 and Subset-CIFAR datasets are quite similar to CIFAR-10 images. Thus, there can be numerous OOD samples due to aleatoric and class-conditional epistemic uncertainty which makes detection challenging. Although our performance is low on the STLI0 dataset, it still outperforms the state-of-the-ar For instance, the proposed approach achieves 27% better TNR score than the Mahalanobis using ResNet50. On SVHN, in terms of the TNR metric, our approach outperforms ODIN and Mahalanobis by 63% and 13%, respectively on SCIFAR 100 using ResNet34. The above observations justify the effectiveness of integrating multiple attributes to detect OOD samples.
OODs for CIFARIO, the images from both these
A.2.3 ABLATION STUDY
We report ablation study on OOD detection with individual attributes and compare it with our in- tegrated approach the penultimate feature space of the classifier in the supervised settings as described in the previous section. We call the OOD detector with Mahalanobis distance estimated on class mean and tied covariance (Lee et al., 2018) as Mahala-Tied Detector based Mahalanobis distance estimated on class mean and class covariance is referred as Mahala-Class. Similarly con- formance among the K-nearest neighbors (KNN) measured by Mahala- -ied and Mahala-Class referred as KNN-Tied and KNN-Class respectively in these experiments. Results for this study CIFARIO 10 with DenseNet architecture, SVHN with DenseNet and ResNet34 architectures are shown in Tables 15
16 and respectively.
The integrated approach could out out-perform all the single attribute based OOD detector in all the tested cases due detection of diverse OODs. An important observation made from these experi- ments is that the performance of the single attribute based methods could depend on the architecture of the classifier. For example, while the performance of PCA really bad in case of DenseNet (for both CIFAR1O and SVHN) as compared to all other methods, it could out-perform all but the integrated approach for SVHN ResNet34.
15

---

Under review as a conference paper at ICLR 2021
Table 6: Results with ODIN/Mahalanobis/OUUU method. The best results are highlighted.
In-dist (model)
OOD dataset
TNR (TPR=95%)
AUROC
DTACC
MNIST (LeNet5)
KMNIST F-MNIST
67.72 17.72/80552/1 90.52/91.88
92.98 96.53 99..3
85.99 99.82/00////// 94.01
58.47 63.33 74.49
90.76/ 94.11 /95.5
83.21 / 87.76 90.98
CIFARIO
STL10
8.89/9.23. 15.29
56.31 022.16/0 63.96
55.38 59.57 61.02
(DenseNet)
SVHN
69.96 83.63 91.29
92.02 97.1 1 98.38
94.1/91.26/ 93.228
Imagenet
61.03 /9.33 77.81
91.4/ 90.32 95.98
83.85 83.08 / 89.74
LSUN
71.89 / 46.63
84.34
94.37 / /97.27
87.72 / 84.93 92.1
SCIFARI00
35.06 20.33 38.78
80.18/ 80.4/ 90.58
72.58
/ 74.15 85.35
CIFARIO (ResNer34)
STL10
10.63 / 13.9 17.4
61.56/ 66.47 67.52
59.22 62.75 63.7
SVHN
72.85 53.16 88.2
93.85 /3.85/ 97.69
85.4 89.18997733 92.14
Imagenet
46.54/ 68.41 74.53
90.45 /2.0995.0/// /95.73
83.06 / 88.63 89.73
LSUN
45.16/77.53
81.23
89.63 923399.51/900 96.87
81.83 90.64 91.19
SCIFARI00
37 38.39 61.11
86.13/ 0/////888///7777
78.5 82.51 /9053
CIFAR10 (ResNet50)
STL10
12.19 10.33 / 16
60.29 60.29/61.951 61.95 66.39
58.57 59.36/ 62.28
SVHN
86.61 34.49/ 91.06
84.41 /8.19 /91.98
91.25 76.72 93.2
Imagenet
73.23 / 29.48 75.96
94.91 / 84.3 95.79
88.23 / 77.19/ 89.26
LSUN
80.72 32.11 81.38
96.51
87.09 96.93
90.59 80.07 91.779
SCIFAR100
47.44 / 21.06 / 48.33
86.16/ 77.42/ 92.98
78.69 / 71.43 88.27
SVHN
STL10
45.91 / 81.66
87.76
77.6 96.97 97.63
72.62 / 92.29 93.35
(DenseNet)
CIFARIO
37.23
80.82 86.42
73.14 / 96.8 97.37
68.92 92.27 92.86
Imagenet
62.76
85.44 93.44
85.41 /97.29 98.38
79.94 / 93.39 94.53
LSUN
62.91
76.87 89.73
86.06 99.6/96.3//777 97.73
80.04 92.43 93.55
SCIFAR100
48.17 86.06 / 96.72
/8.94/97.43/ /98.24
73.72 / 93.02 96.26
SVHN
STL10
35.14/ 85.3 90.9
67.05 97.19/
66.19/ 93.41 94.34
(ResNet34)
CIFARIO
32.6 85.03 90.34
66.75 / /97.64
65.37 93. 94.29
Imagenet
41.8 84.46 89.82
73 96.95 97.59
69.84 93.14 94.32
LSUN
35.92 78.38 85.46
68.6 / 96.17 97.09
66.75 / 99.98 93.17
SCIFAR100
36.67 / 86.61 99.61
68.01 /97.3 98.47
67.26 /93.6/ /9.36
16

---

Under review as a conference paper at ICLR 2021
Table 7: Results with Mahalanobis/Our method with feature ensemble. The best results are high- lighted.
In-dist (model)
OOD dataset
TNR (TPR-95%)
AUROC
DTACC
MNIST (LeNet5)
KMNIST F-MNIST
96 98.8 99.9 99.98
99.19/ 99.65
95.56 97.3
99.95 99.96
98.98 99.17
CIFARIO (DenseNet)
STL10
16.44 / 22.94
72.4 / 75.23
66.69 / 69.31
SVHN
92.4 98.23
98.41 99.49
93.97 97.02
Imagenet
96.46 98.8
99.16/ 99.63
95.74 97.55
LSUN
98.09 99.664
99.47 99.85
96.76 98.45
SCIFARI00
27.33 46.5
83.7 / 92.17
77.08 86.69
CIFARIO
STL10
26.14 29.8
76.23 76.46
70.33 70.94
(ResNet34)
SVHN
91.53 99.07
98.4 99.32
93.63 96.27
Imagenet
99.41 97.09 99.11
99.47 99.58
96.31 96.91
LSUN
98.67
99.71 99.81
97.56 98.14
SCIFARI00
38.89
/ 62.78
88.8 / 94.23
82.14 90.16
CIFARIO (ResNet50)
STL10
26.36
30.83
73.74 76.73
67.37 70.4
SVHN
84.44
98.59
96.56 99.65
90.63 97.43
Imagenet
97.87 / 99.46
99.58 99.84
97.09 98.22
LSUN
99.21
99.83
99.64 99.91
98.39 199.21
SCIFAR100
29.33 / 55
80.26 / 91.48
74.51 86.42
SVHN
STL10
97.31 98.76
99.14 99.47
96.23 97.24
(DenseNet)
CIFARIO
96.36 99.66
98.8 99.16
95.7 / 96.34
Imagenet
99.89 99.82
99.88 99..9
98.85 98.95
LSUN
99.99 /99997
99.91 99.91
99.26 199.18
SCIFARI00
99.33 100
99.53 99.78
97.89 98.95
SVHN
STL10
98.44 98.88
99.31
99.52
96.91 97.4
(ResNet34)
CIFAR10
/ /9.888
99.31
99.52
96.91 97.4
Imagenet
99.83 99.87
99.85
99.91
99.07 99.07
LSUN
99.87 99.99
99.83 / 99.95
99.5 99.47
SCIFAR100
99.83 100
99.72 99.91
98.33/ 99.56
17

---

Under review as a conference paper at ICLR 2021
Table Experimental results with SPB/Our method. The best results are highlighted.
In-dist (model)
OOD dataset
TNR (TPR=95%)
AUROC
DTACC
MNIST (LeNet5)
KMNIST F-MNIST
69.33 / 91.82
93.24 98.3
86.88 / 99.01
52.69 / 74.49
89.19 95.55
82.77 990.98
CIFARIO (DenseNet)
STL10
15.64 15.29
64.15 63.96
62.12 61.02
SVHN
39.22 / 91.29
88.24 98.38
82,41 /93.28
Imagenet
40.13 77.81
89.3
95.98
82.67 89.74
LSUN
48.38 84.34
92.14/97.27
86.22 / 92211
SCIFARI00
34.11/38.77 38.78
85.53 90.58
79.18 85.35
CIFARIO (ResNet34)
STL10
1
65.88/ 67.52
62.85 / 63.7
SVHN
32.47 / 88.2
89.88 97.69
85.06 35.06/92.14
Imagenet
44.72 74.53
91.02/ 95.73
85.05 89.773
LSUN
28.44/81.22
91.04 96.87
85.26 / 99.19
SCIFARI00
38.17 61.11
88.91 94.74
82.34 990.53
CIFARIO (ResNet50)
STL10
15.33 16
66.68 66.39
63.47 62.28
SVHN
44.69 / 91.06
97.31 / 91.98
86.36 /93.2
Imagenet
42.06 75.96
90.8 /95.79
84.36 89.26
LSUN
48.37 / 81.38
92.78 / 96.93
86.97 991.79
SCIFARI00
36.39 48.33
89.09 92.98
83.37 88.27
SVHN
STL10
72.87
87.76
92.79 / 97.63
87.76 93.35
(DenseNet)
CIFARIO IO
69.31
86.42
91.9 / 97.37
86.61 92.86
Imagenet
79.79
93.44
94.78 98.38
90.21 994.53
LSUN
77.12
89.73
94.13 97.73
89.14 93.55
SCIFARI00
76.94
96.72
94.18 98.24
89.57 99..26
SVHN (ResNet34)
STL10
79.57 / 99.59
93.84
99.72
90.83 /88.06
CIFAR1O
78.26
90.34
92.92 97.64
90.03 994.29
Imagenet
79.02
89.82
93.51 97.59
90.44 94.32
LSUN
74.29 / 85.46
91.58 97.09
88.96 93.17
SCIFAR100
81.28
99.61
94.62 /98.47
91.48 997.36
18

---

Under review as a conference paper at ICLR 2021
Table Experimental Results with MNIST on Lenet5 for AUPR IN and AUPR OUT. The best results are highlighted.
OOD Dataset
Layer
Method
AUPR IN
AUPR OUT
Penultimate
KMNIST
Baseline
92.47
92.41
ODIN
92.65
92.69
Mahalanobis
96.69
96.2
Ours
98.48
98.13
Fashion-MNIST
Baseline
87.98
87.89
ODIN
90.94
89.99
Mahalanobis
95.24
91.94
Ours
96.64
92.96
All
KMNIST
Mahalanobis
99.22
99.18
Ours
99.67
99.64
Fashion-MNN MNIST
Mahalanobis
99.95
99.94
Ours
99.96
99.96
19

---

Under review as a conference paper at ICLR 2021
Table 10: Experimental Results with CIFARIO on DenseNet for AUPR IN and AUPR OUT. The best results are highlighted.
OOD Dataset
Layer
Method
AUPR IN
AUPR OUT
Penultimate
STL10
Baseline
64.55
59.37
ODIN
60.24
50.17
Mahalanobis
65.86
53.87
Ours
66.01
58.5
SVHN
Baseline
74.53
94.09
ODIN
80.49
97.05
Mahalanobis
94.13
98.78
Ours
96.26
99.37
Imagenet
Baseline
90.88
86.74
ODIN
91.32
90.55
Mahalanobis
91.32
88.6
Ours
96.22
95.68
LSUN
Baseline
93.68
89.83
ODIN
94.65
93.39
Mahalanobis
92.71
87.74
Ours
97.6
96.74
Subset CIFAR 100
Baseline
96.65
50.08
ODIN
95.14
47.64
Mahalanobis
95.68
37.86
Ours
98.18
54.83
All
STL10
Mahalanobis
77.21 78.29
63.45
Ours
68.14
SVHN
Mahalanobis
96.72
99.31
Ours
98.57
99.81
Imagenet
Mahalanobis
99.19
99.13
Ours
99.62
99.54
LSUN
Mahalanobis
99.49 99.82
99.45
Ours
99.85
Subset CIEAR 100
Mahalanobis
96.58 98.53
42.41
Ours
59.7
20

---

Under review as a conference paper at ICLR 2021
Table 11: Experimental Results with CIFARIO on ResNet34 for AUPR IN and AUPR OUT. The best results are highlighted.
OOD Dataset
Layer
Method
AUIN
AUOUT
Penultimate
STL.0
Baseline
67.17
59.74
ODIN
64.22
53.83
Mahalanobis
68.48
59.47
Ours
68.78
61.52
SVHN
Baseline
85.4
93.96
ODIN
86.46
97.55
Mahalanobis
91.19
96.14
Ours
94.7
99.1
Imagenet
Baseline
92.49
88.4
ODIN
92.11
87.46
Mahalanobis
95.77
94.02
Ours
96.32
94.99
LSUN
Baseline ODIN
92.45
88.55
91.58
86.5
Mahalanobis
97.08
95.78
Ours
97.36
96.29
Subset CIFAR100
Baseline
97.77
55.62
ODIN
97.05
51.57
Mahalanobis
97.71
54.11
Ours
99.06
64.53
All
STL10
Mahalanobis Ours
77.59
71.15
77.32
72.38
SVHN
Mahalanobis Ours
96.46
99.37
98.37
99.73
Imagenet
Mahalanobis Ours
99.48
99.48
99.59
99.58
LSUN
Mahalanobis Ours
99.71
99.71
99.8
99.82
Subset CIFAR100
Mahalanobis Ours
97.75
52.28
98.74
65.99
21

---

Under review as a conference paper at ICLR 2021
Table 12: Experimental Results with CIFARIO as on ResNet50 for AUPR IN and AUPR OUT. The best results are highlighted.
OOD Dataset
Layer
Method
AUIN
AUOUT
Penultimate
STL.0
Baseline
67.47
60.83
ODIN
62.79
55.04
Mahalanobis
65.14
54.43
Ours
68.54
59.75
SVHN
Baseline
87.78
95.61
ODIN
93.17
99.03
Mahalanobis
71.88
92.54
Ours
95.38
99.34
Imagenet
Baseline
92.6
87.98
ODIN
95.16
94.45
Mahalanobis
86.14
80.6
Ours
96.22
95.23
LSUN
Baseline
94.45
90,41
ODIN
96.9
96.01
Mahalanobis
89.34
82.87
Ours
97.53
96.03
Subset CIFAR100
Baseline
97.72
55.29
ODIN
96.67
60.62
Mahalanobis
94.49
36.12
Ours
98.72
59.3
All
STL10
Mahalanobis Ours
75.6
69.32
77.79
73.22
SVHN
Mahalanobis Ours
91.89
98.58
99.12
99.86
Imagenet
Mahalanobis Ours 99.84
99.56
99.6
99.84
LSUN
Mahalanobis Ours
98.91
99.75
99.72
99.93
Subset CIFAR100
Mahalanobis Ours
94.1
42.96
97.54
64.12
22

---

Under review as a conference paper at ICLR 2021
Table 13: Experimental Results with SVHN as on DenseNet for AUPR IN and AUPR OUT. The best results are highlighted.
OOD Dataset
Layer
Method
AUIN
AUOUT
Penultimate
STL10
Baseline
97.01
82.02
ODIN
89.18
63.71
Mahalanobis
99.2
87.32
Ours
99.36
90.31
CIFARIO
Baseline
95.7
82.8
ODIN
84.32
60.32
Mahalanobis
98.94
88.91
Ours
99.09
91.59
Imagenet
Baseline
97.2
88.42
ODIN
90.95
79.59
Mahalanobis
99.12
90.22
Ours
99.4
95.15
LSUN
Baseline
96.96
87.44
ODIN
92.03
79.98
Mahalanobis
98.84
85.79
Ours
99.17
92.92
Subset CIFAR100
Baseline
99.39
63.21
ODIN
97.24
45.23
Mahalanobis
99.82
72.35
Ours
99.88
68.25
All
STL10
Mahalanobis Ours
99.75
96.51
99.77
98.18
CIFAR10
Mahalanobis Ours
99.6
95.39
99.69
97.21
Imagenet
Mahalanobis Ours
99.96
99.59
99.96
99.74
LSUN
Mahalanobis Ours
99.97
99.7
99.95
99.74
Subset CIFAR100
Mahalanobis Ours
99.97
91.41
99.98
94.54
23

---

Under review as a conference paper at ICLR 2021
Table 14: Experimental Results with SVHN as on ResNet34 for AUPR IN and AUPR OUT. The best results highlighted.
OOD Dataset
Layer
Method
AUIN
AUOUT
Penultimate
STL10
Baseline
96.63
84.15
ODIN
84.03
47.26
Mahalanobis
99.89
97.5
Ours
99.93
98.2
CIFARIO
Baseline
95.06
85.66
ODIN
80.69
50.49
Mahalanobis
99.04
88.62
Ours
99.17
91.17
Imagenet
Baseline
95.68
86.18
ODIN
84.62
58.28
Mahalanobis
99
88.39
Ours
99.19
90.77
LSUN
Bascline ODIN
94.19
83.95
82.37
53.12
Mahalanobis
98.73
85.11
Ours
99.03
89.03
Subset CIFARI00
Baseline ODIN
99.35
64.38
95.57
23.04
Mahalanobis
99.81
64.4
Ours
99.89
67.9
All
STL.0
Mahalanobis Ours
99.7
97.03
99.84
97.86
CIFARIO
Mahalanobis Ours
99.7
97.03
99.84
97.86
Imagenet
Mahalanobis Ours
99.86
99.1
99.93
99.62
LSUN
Mahalanobis Ours
99.82
98.85
99.98
99.64
Subset CIFAR100
Mahalanobis
99.98
93.59
Ours
99.99
95.75
24

---

Under review as a conference paper at ICLR 2021
Table 15: Ablation study with CIFAR1O DenseNet. The best results are highlighted.
OOD dataset
Method
TNR (TPR=95%) %)
AUROC
DTACC
AUPR IN
AUPR
OUT
SVHN
Mahala- -ied
83.63
97.1
91.26
94.13
98.78
Mahala-Class
71.73
95.16
87.92
90.77
97.98
KNN-Tied
84.07
97.18
91.32
94.2
98.84
KNN-Class
77.95
96.19
89.68
92.06
98.45
SPB
39.22
88.24
82.41
74.53
94.09
ODIN
69.96
92.02
84.1
80.49
97.05
PCA
2.46
55,89
56.36
35.42
74.12
Integrated(Oun)
91.29
98.38
93.28
96.26
99.37
Imagenet
Mahala-Tice
49.33
90.32
83.08
91.32
88.6
Mahala-Class
53.11
92.16
85.3
93.42
90.29
KNN-Tied
51.36
90.73
83.31
91.75
88.87
KNN-Class
57.94
92.74
86.01
93.67
91.28
SPB
40.13
89.3
82.67
90.88
86.74
ODIN
61.03
91.4
83.85
91.32
90.55
PCA
4.66
58.68
57.19
60.66
54.42
Integrated(Our)
77.81
95.98
89.74
96.22
95.68
LSUN
Mahala- Tied
46.63
91.18
84.93
92.71
87.74
Mahala-Class
58.53
93.82
88.16
95.15
91.43
KNN-Tied
51.48
92.25
85.96
93.75
89.13
KNN-Class
65.17
94.57
88.6
95.61
92.61
SPB
48.38
92.14
86.22
93.68
89.83
ODIN
71.89
94.37
87.72
94.65
93.39
PCA
2.06
53.26
54.88
57.08
49.33
Integrated(Our)
84.34
97.27
92.1
97.6
96.74
25

---

Under review as a conference paper at ICLR 2021
Table 16: Ablation study with SVHN on DenseNet. The best results are highlighted.
OOD dataset
Method
TNR (TPR=95%)
AUROC
DTACC
AUPR IN
AUPR
OUT
CIFARIO
Mahala-Tice
80.82
96.8
92.27
98.94
88.91
Mahala-Clat
82.99
97.11
92.83
99.05
89.71
KNN-Tied
69.99
95.58
90.77
98.52
84.3
KNN-Class
74.52
96.01
91.21
98.64
85.99
SPB
69.31
91.9
86.61
95.7
82.8
ODIN
37.23
73.14
68.92
84.32
60.32
PCA
5.27
65.82
64.83
86.62
33.51
Integrated(Our)
86.6
97.41
92.88
99.11
91.76
Imagenet
Mahala-Tice -ied
85.44
97.29
93.39
99.12
90.22
Mahala-Clen
77.66
96.83
93.17
98.98
88.59
KNN-Tied
65.76
94.67
89.59
98.18
80.16
KNN-Class
73.44
95.69
90.68
98.55
84.28
SPB
79.79
94.78
90.21
97.2
88.42
ODIN
62.76
85,41
79.94
90.95
79.59
PCA
5.16
65.08
65.39
86.65
32.83
Integrated(Our)
93.46
98.39
94.54
99.41
95.16
LSUN
Mahala- Tied
76.87
96.37
92.43
98.84
85.79
Mahala-Class
69.44
96.05
92.4
98.74
84.89
KNN-Tied
59.64
93.71
88.22
97.83
77.17
KNN-Class
66.96
94.77
89.45
98.21
81.27
SPB
77.12
94.13
89.14
96.96
87.44
ODIN
62.91
86.06
80.04
92.03
79.98
PCA
3.19
62.66
64.7
85.72
30.37
Integrated(Our)
89.73
97.73
93.55
99.17
92.92
26

---

Under review as a conference paper at ICLR 2021
Table 17: Ablation study with SVHN on ResNet34. The best results are highlighted.
OOD dataset
Method
TNR (TPR=95%)
AUROC
DTACC
AUPR IN
AUPR
OUT
SCIFARI00
Mahala-Tied
86.61
97.3
93.6
99.81
64.4
Mahala-Clat
88.44
97.7
94.19
99.84
69.59
KNN-Tied
84.67
96.82
92.83
99.76
61.08
KNN-Class
83.72
96.83
93.05
99.77
57.58
SPB
81.28
94.62
91.48
99.35
64.38
ODIN
36.67
68.01
67.26
95.57
23.04
PCA
89.94
97.81
94.52
99.84
70.83
Integrated(Our)
99.61
98.47
97.36
99.89
67.9
LSUN
Mahala-Tiee
78.38
96.17
91.98
98.73
85.11
Mahala-Clan Class
81.51
96.71
92.44
98.91
87.63
KNN-Tied
77.61
95.98
91.34
98.61
85.56
KNN-Class
78.77
96.05
91.45
98.62
85.71
SPB
74.29
91.58
88.96
94.19
83.95
ODIN
35,92
68.6
66.75
82.37
53.12
PCA
82.93
96.88
92.74
98.97
88.27
Integrated(Our)
85.46
97.09
93.17
99.03
89.03
CIFARIO
Mahala-Tics Tied
85,03
97.05
93.15
99.04
88.62
Mahala-Class -lasss
86.84
97.41
93.48
99.15
90.37
KNN-Tied
82.17
96.65
92.24
98.87
87.63
KNN-Class
83.24
96.73
92.38
98.9
87.67
SPB
78.26
92.92
90.03
95.06
85.66
ODIN
32,67
66.75
65.37
80.69
50.49
PCA
88.18
97.55
93.83
99.2
90.77
Integrated(Our)
90.34
97.64
94.29
99.17
91.17
27


Reviews:
[{"review_id": "qurbtGXSRj-", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\n\nThis paper introduces a novel taxonomy for OOD outliers. The authors analyze current OOD detection approaches and uncover their limitations. They propose to fuse several existing approaches into a combined one and extensively evaluate it on various data sets (CIFAR,10, SVNH, MNIST, STL10, ImageNet, etc.). The proposed integrated OOD detection approach clearly shows superior performance.\n\n##########################################################################\n\nReasons: \n\nOverall, I vote for accepting. The authors make several key contributions: The introduce a novel OOD taxonomy, analyse current OOD detection approaches on a toy data set, propose an integrated OOD detection approach, which shows a superior performance in their extensive evaluation.\n\n##########################################################################\n\nPros:\n\n* Introduction of a sound and helpful OOD taxonomy\n* Limitation analysis of state-of-the-art OOD detection algorithms\n* Proposal of a new integrated approach to detect different kind of OOD inputs that unifies the advanatges of underlying algorithms.\n* Extensive evaluation of new approach shows clearly superior performance. On a variety of data sets (CIFAR,10, SVNH, MNIST, STL10, ImageNet, etc.) the proposed approach outperforms the baselines on all evaluation criteria (TNR, AUROC, DTACC, AUPR IN, AUPR OUT) for various classifier neural network architectures (LeNet, ResNet, DenseNet).\n\n##########################################################################\n\nCons:\n\n* The demonstration of the limitations of current OOD detection algorithms is solely empirical (based on a toy data set). Theoretic motivations (if possible) would be a great addition.\n* Similarly, a sound theoretical derivation for the proposed integrated approach is lacking. \n* Further toy data sets beyond the two half moon data set would be helpful to better understand the implications of all algorithms.\n\n##########################################################################\n\nQuestions during rebuttal period: \n\nPlease address and clarify the cons above ", "extra_scores": {}, "extra_text": {}}, {"review_id": "kp-9WM2Q2Y1", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "-- Paper Summary: \nThe paper presents the idea of fusion of attributes from existing sota ood detection methods to achieve higher detection performance.\n\n-- Review :\n\n- The three criteria presented in section two are questions rather than criteria. It is better to be re-worded into criteria. \n\n- Figure 1 suggests the \"tied distribution of all training data\" is different than the combination of \"class distributions\". I wish authors  could explain the difference between Type 4 and 5 in the ood sample taxonomy.\n\n- The relation between five types of OOD with three criteria for OOD categorization is not clear. \n\n- The visualization in all figures could be improved:\n    - figure 1: too many colors. better to use different shape or numbers directly in the figure.\n    - figure 5: not necessary to include, hard to see and comprehend.\n    - the total number of figures can be reduced by eliminating some and combining others. \n\n- What was the reason to choose a subset of cifar100 as ood test set but not the whole dataset? \n\n- Authors emphasize reporting detection TNR in the manuscript while FNR is missing from the measurements. I suggest authors either report both or use threshold agnostic metrics like area under precision recall curve (AUPR) or area under receiver operating curve (AUROC) for reporting as in the Table. \n\n- I can't find an explanation and/or discussion on the final detection score and it's hyperparametere. \n\n- The results from the Mahanalobis technique [7] does not match the original paper. If authors did not use a subset of ood samples for tuning, it should be reported in the paper.\n\n-- Strengths: \n- interesting taxonomy of ood samples and the following conclusion for integrated detection score. \n\n\n-- Weaknesses: \n- limited on contribution\n- no discussion on final detection score and its hyperparameters.\n- comparison with more recent techniques including Outlier Exposure [1], Self-supervised reject classifier [2], Geometric self-superivised learning [3,4], and contrastive learning [5,6] are missing in this paper. \n\n\n[1] Hendrycks, D., Mazeika, M., & Dietterich, T. (2018, September). Deep Anomaly Detection with Outlier Exposure. ICLR 2019\n\n[2] Mohseni, Sina, et al. \"Self-Supervised Learning for Generalizable Out-of-Distribution Detection.\" AAAI. 2020. \n\n[3] Hendrycks, D., Mazeika, M., Kadavath, S., & Song, D. (2019). Using self-supervised learning can improve model robustness and uncertainty. In Advances in Neural Information Processing Systems (pp. 15663-15674). \n\n[4] Golan, Izhak, and Ran El-Yaniv. \"Deep anomaly detection using geometric transformations.\" Advances in Neural Information Processing Systems. 2018. \n\n[5] Tack, J., Mo, S., Jeong, J., & Shin, J. (2020). Csi: Novelty detection via contrastive learning on distributionally shifted instances. arXiv preprint arXiv:2007.08176. \n\n[6] Winkens, J., Bunel, R., Roy, A. G., Stanforth, R., Natarajan, V., Ledsam, J. R., ... & Cemgil, T. (2020). Contrastive training for improved out-of-distribution detection. arXiv preprint arXiv:2007.05566. [8] Liu, Hao, and Pieter Abbeel. \"Hybrid discriminative-generative training via contrastive learning.\" arXiv preprint arXiv:2007.09070 (2020).\n\n[7] Lee, K., Lee, K., Lee, H., & Shin, J. (2018). A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems (pp. 7167-7177).\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "cPhM2rIxXx", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a taxonomy of OODs and proposed an integrated approach to detect different types of OODs. Their taxonomy classifies OOD on the nature of their uncertainty and they show that no single state-of-the-art approach detects all these OOD types. Motivated by this observation, they combine multiple existing OOD detection methods to detect various types of OODs. \n\nIn general, this paper is easy to understand. But I have the following concerns: \n\n1. Lack of discussions about some important related work. They only compare their method to ODIN and Mahalanobis methods. But there are some other OOD detection methods which also achieve state-of-the-art results, such as [1][2][3]. Could the authors compare their method to these methods? \n\n2. In their taxonomy, they consider examples that are very close to in-distribution as OOD. I am wondering whether we should treat those examples as OOD since they are too close to the in-distribution. I think previous works like ODIN and Mahalanobis all assume that OOD inputs are far away from the in-distribution. In the experimental setup, they consider STL10 as an OOD dataset for CIFAR10. But STL10 contains CIFAR10 alike images. It is unconvincing that we should treat those images as OOD. And I think the classifier trained on CIFAR10 may have correct predictions on some of those images. Could the authors explain why we should treat those images as OOD? \n\n3. I am wondering whether the analysis for the simple two-dimensional dataset could be applied to high-dimensional datasets. In the high-dimensional space, their conclusion about which method detects which type of OOD may not hold. Could the authors explain it? \n\n4. In Appendix A.2.1, they mention that the best results from the twelve combinations of the aforementioned sub-categories (one from each of the four attributions) are reported. Could the authors explain how they select the best results? Do they use the test OOD data to select the best results? \n\n5. Could the author describe how they integrate the existing state-of-the-art detection methods in detail? It is hard for me to understand what they exactly do in their proposed method.  \n\n--------- After Reading the Updated Paper ----------\n\nThanks for the update. After reading the revised paper, I still have some major concerns:\n\n1. The current experiments performed are not enough to demonstrate the effectiveness of the proposed method. The old experiment results (Table 6, 7, 8) are not convincing since the authors train a binary classifier as an OOD detector using a subset of the test OOD data, which is not realizable in practice. We should assume that the test OOD data are unknown during learning the OOD detector. The new experimental results where they train the binary classifier using adversarial examples generated on in-distribution data (follow the Mahalanobis method) in Table 1 are limited. For example, on CIFAR10, they only report results for ResNet50 and WideResNet, but I also want to know the results for DenseNet (Mahalanobis method [4] performs very well on CIFAR10/SVHN using DenseNet under the same setting).\n\n2. Some experimental details about their method are missing. The authors mention that they train 12 binary classifiers and then select the best one on the validation dataset. But they don't provide the details about the validation dataset, which is critical for their results. Based on their previous response, it seems they use a subset of test OOD data to select the best classifier, which is not allowed I think. Based on the current description of experimental settings, it is hard for me to evaluate the reported results.\n\n3. The proposed approach needs a lot of hyper-parameters (4 attributes, 12 combinations, the weights of the binary classifier, etc) and it is unclear how to tune these hyper-parameters and how they would affect the results. The current ablation study is limited I think.\n\n4. This paper doesn't have rigorous analysis for why integrating different attributions would improve OOD detection. I think this is an empirical paper but the experiments provided are not sufficient to demonstrate the effectiveness of the proposed method.\n\nTo clarify, I didn't agree to raise the score previously. What I said was that the previous paper needed significant revision and I could not recommend acceptance. I still have some major concerns after reading the revised paper. Thus, I keep the same rating and think the paper is not ready for publication. I hope the authors could keep improving their paper. \n\n \n[1] Hendrycks, Dan, Mantas Mazeika, and Thomas Dietterich. \"Deep anomaly detection with outlier exposure.\" arXiv preprint arXiv:1812.04606 (2018).\n\n[2] Liu, Weitang, et al. \"Energy-based Out-of-distribution Detection.\" arXiv preprint arXiv:2010.03759 (2020).\n\n[3] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. \"Simple and scalable predictive uncertainty estimation using deep ensembles.\" Advances in neural information processing systems. 2017.\n\n[4] Lee, Kimin, et al. \"A simple unified framework for detecting out-of-distribution samples and adversarial attacks.\" Advances in Neural Information Processing Systems. 2018.", "extra_scores": {}, "extra_text": {}}, {"review_id": "UoSVYi8ptpj", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\nThe authors explore the different kinds of outliers and show that the methods previously proposed detect different kinds of OOD and not a single one can detect them all. The authors propose an interesting study of the different kind of outlier on synthetic data which  illustrates well the different characteristics of the outlier types. The authors then propose to combine different methods to increase the OOD detection rate. Experiments are conducted on 3 images classification datasets using different deep neural networks. For each dataset, samples from other databases are introduced as outliers and must be detected. The combination method yield better detection rates than baseline methods in almost all configurations. \n\n\n##########################################################################\n\n Reasons for score: \n\nThe main idea of the paper is simple : combine different OOD detection metrics to increase the detection rate on different types of outliers. The proposed method indeed increases the OOD detection rate for almost all the experimental settings tested by the authors. However, the method to create the OOD samples is always the same: in-distribution samples come from a database whereas out of distribution samples are drawn from another database. It would be interesting to show that the method also increases the detection rate of outliers inside a given database. This could be done by reporting the classification rate of the DNN in an abstaining scheme : if the OOD metric is greater than a threshold, the sample is not classified (rejected). If the OOD detection method is useful, the classification rate of the DNN can be freely increased by increasing the threshold and rejecting more and more samples. \n\nThe author do not justify their choice of the combination method. Computing all the OOD metrics can be computationaly expensive, is it necessary to compute them all ? Are this combination of metric the best ? In which conditions ?\n\nThe combination method should be described in the body of the paper, not in appendix.\n\nGuo 2017 appears twice in the bibliography.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: **Problem Significance**  This paper introduces an interesting taxonomy of OODs and proposed an integrated approach to detect different types of OODs. The AC agrees on the importance of a fine-grained characterization of outliers given the large OOD uncertainty space. 

**Technical contribution** The key idea of the paper is to combine the predictions from multiple existing OOD detection methods. While the AC recognizes the effort made by the authors to address the review comments, reviewers have several major standing concerns regarding limited contributions, insufficient analysis, and lack of clarity. The AC agrees with reviewers that the paper is not ready yet for ICLR publication, and can be further strengthened by:

- (R1) reporting the computational cost for the integrated approach. The inference time for approaches such as Mahalanobis is typically a few times more expensive than the MSP baseline. The cumulative time for calculating all four scores may be non-negligible. Authors are encouraged to analyze the performance tradeoff in a future revision. 
- (R2 & R3) discussing the effect of hyper-parameters tuning, which be overly complicated in practice as it involves combinations of multiple methods that each have multiple parameters to tune. 
- (R3) comparing with more recent development on OOD detection and move the new results to the main paper. The AC also thinks it's worth discussing the connection and comparison to methods on quantifying uncertainty via Bayesian probabilistic approaches.
- (R2 & R4) more rigorous analysis of the benefits of the proposed integrated approach, both empirically and theoretically. Based on Table 7, the performance of using Mahalanobis alone is almost competitive as the proposed approach (except for the CIFAR10-CIFAR100 pair). This may deem further careful examination to understand what value other components are adding, and in what circumstance. 
- (R2, R3 & R4) More discussion on the implication of the taxonomy and algorithm in the high-dimensional space would be valuable. The 2D toy dataset might be too simple to reflect the decision boundary as well as uncertainty space learned by NNs. Moreover, it's important to justify further how aleatoric and epistemic uncertainty manifests in the current experiments using NNs. For example, epistemic uncertainty can arise due to the use of limited samples or due to the model uncertainty associated with the model regularization. 

Recent work by Hsu et al. [2] also attempt to define a taxonomy of OOD inputs (based on semantic shift and domain shift), which can be relevant for the authors. 

**Recommendation** Three knowledgeable reviewers have indicated rejection. The AC discounted R4's rating due to the less familiarity in this area and lack of participation in the post-rebuttal discussion. 

[1] Richard Harang, Ethan M. Rudd. Towards Principled Uncertainty Estimation for Deep Neural Networks
[2] Hsu et al. Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data

Decision: Reject