Are all outliers alike?  On Understanding the Diversity of Outliers for Detecting OODs

1 INTRODUCTION . Deep neural networks ( DNNs ) have achieved remarkable performance-levels in many areas such as computer vision ( Gkioxari et al. , 2015 ) , speech recognition ( Hannun et al. , 2014 ) , and text analysis ( Majumder et al. , 2017 ) . But their deployment in the safety-critical systems such as self-driving vehicles ( Bojarski et al. , 2016 ) , aircraft collision avoidance ( Julian & Kochenderfer , 2017 ) , and medical diagnoses ( De Fauw et al. , 2018 ) is hindered by their brittleness . One major challenge is the inability of DNNs to be self-aware of when new inputs are outside the training distribution and likely to produce incorrect predictions . It has been widely reported in literature ( Guo et al. , 2017a ; Hendrycks & Gimpel , 2016 ) that deep neural networks exhibit overconfident incorrect predictions on inputs which are outside the training distribution . The responsible deployment of deep neural network models in high-assurance applications necessitates detection of out-of-distribution ( OOD ) data so that DNNs can abstain from making decisions on those . Recent approaches for OOD detection consider different statistical , geometric or topological signatures in data that differentiate OODs from the training distribution . For example , the changes in the softmax scores due to input perturbations and temperature scaling have been used to detect OODs ( Hendrycks & Gimpel , 2016 ; Liang et al. , 2017 ; Guo et al. , 2017b ) . Papernot & McDaniel ( 2018 ) use the conformance among the labels of the nearest neighbors while Tack et al . ( 2020 ) use cosine similarity ( modulated by the norm of the feature vector ) to the nearest training sample for the detection of OODs . Lee et al . ( 2018 ) consider the Mahalanobis distance of an input from the in-distribution data to detect OODs . Several other metrics such as reconstruction error ( An & Cho , 2015 ) , likelihood-ratio between the in-distribution and OOD samples ( Ren et al. , 2019 ) , trust scores ( ratio of the distance to the nearest class different from the predicted class and the distance to the predicted class ) ( Jiang et al. , 2018 ) , density function ( Liu et al. , 2020 ; Hendrycks et al. , 2019a ) , probability distribution of the softmax scores ( Lee et al. , 2017 ; Hendrycks et al. , 2019b ; Tack et al. , 2020 ; Hendrycks et al. , 2019a ) have also been used to detect OODs . All these methods attempt to develop a uniform approach with a single signature to detect all OODs accompanied by empirical evaluations that use datasets such as CIFAR10 as in-distribution data and other datasets such as SVHN as OOD . Our study shows that OODs can be of diverse types with different defining characteristics . Consequently , an integrated approach that takes into account the diversity of these outliers is needed for effective OOD detection . We make the following three contributions in this paper : • Taxonomy of OODs . We define a taxonomy of OOD samples that classify OODs into different types based on aleatoric vs epistemic uncertainty ( Hüllermeier & Waegeman , 2019 ) , distance from the predicted class vs the distance from the tied training distribution , and uncertainty in the principal components vs uncertainty in non-principal components with low variance . • Incompleteness of existing uniform OOD detection approaches . We examine the limitations of the state-of-the-art approaches to detect various types of OOD samples . We observe that not all outliers are alike and existing approaches fail to detect particular types of OODs . We use a toy dataset comprising two halfmoons as two different classes to demonstrate these limitations . • An integrated OOD detection approach . We propose an integrated approach that can detect different types of OOD inputs . We demonstrate the effectiveness of our approach on several benchmarks , and compare against state-of-the-art OOD detection approaches such as the ODIN ( Liang et al. , 2017 ) and Mahalanobis distance method ( Lee et al. , 2018 ) . 2 OOD TAXONOMY AND EXISTING DETECTION METHODS . DNNs predict the class of a new input based on the classification boundaries learned from the samples of the training distribution . Aleatory uncertainty is high for inputs which are close to the classification boundaries , and epistemic uncertainty is high when the input is far from the learned distributions of all classes ( Hora , 1996 ; Hüllermeier & Waegeman , 2019 ) . Given the predicted class of a DNN model on a given input , we can observe the distance of the input from the distribution of this particular class and identify it as an OOD if this distance is high . We use this top-down inference approach to detect this type of OODs which are characterized by an inconsistency in model ’ s prediction and input ’ s distance from the distribution of the predicted class . Further , typical inputs to DNNs are high-dimensional and can be decomposed into principal and non-principal components based on the direction of high variation ; this yields another dimension for classification of OODs . We , thus , categorize an OOD using the following three criteria . 1 . Is the OOD associated with higher epistemic or aleatoric uncertainty , i.e. , is the input away from in-distribution data or can it be confused between multiple classes ? 2 . Is the epistemic uncertainty of an OOD sample unconditional or is it conditioned on the class predicted by the DNN model ? 3 . Is the OOD an outlier due to unusually high deviation in the principal components of the data or due to small deviation in the non-principal ( and hence , statistically invariant ) components ? Figure 1 demonstrates different types of OODs which differ along these criteria . Type 1 OODs have high epistemic uncertainty and are away from the indistribution data . Type 2 OODs have high epistemic uncertainty with respect to each of the 3 classes even though approximating all in-distribution ( ID ) data using a single Guassian distribution will miss these outliers . Type 3 OODs have high aleatoric uncertainty as they are close to the decision boundary between class 0 and class 1 . Type 4 and 5 have high epistemic uncertainty with respect to their closest classes . While Type 4 OODs are far from the distribution along the principal axis , Type 5 OODs vary along a relatively invariant axis where even a small deviation indicates that the sample is an OOD . Limitations of Existing Detection Methods . We empirically demonstrate the limitations of existing OOD detection methods on a two-dimensional ( 2D ) half-moon dataset with two classes . As shown in Figure 2 , we consider three clusters of OOD samples : cluster A ( black ) , B ( brown ) and C ( red ) . Figure 2 ( right ) shows the 2D penultimate features of the classifier . Different approaches differ in their ability to detect different OOD types as illustrated in Figure 3 . • Figure 3 ( a ) shows that the Mahalanobis distance ( Lee et al. , 2018 ) from the mean and tied covariance of all the training data in the feature space can not detect OODs in the clusters B and C corresponding to class-conditional epistemic uncertainty and aleatoric uncertainty , respectively . It attains the overall true negative rate ( TNR ) of 39.09 % at the 95 % true positive rate ( TPR ) . • Figure 3 ( b ) shows that the softmax prediction probability ( SPB ) ( Hendrycks & Gimpel , 2016 ) can not detect the OODs in cluster A corresponding to high epsitemic uncertainty . The TNR ( at 95 % TPR ) reported by the SPB technique is 60.91 % . • Figure 3 ( c ) shows that class-wise Principal Component Analysis ( PCA ) ( Hoffmann , 2007 ) can not detect OODs in cluster C corresponding to high aleatoric uncertainty . We performed PCA of the two classes separately in the feature space and used the minimum reconstruction error to detect OODs . This obtained overall TNR of 80.91 % ( at 95 % TPR ) . • Figure 3 ( d ) shows that K-Nearest Neighbor ( kNN ) ( Papernot & McDaniel , 2018 ) nonconformance in the labels of the nearest neighbors can not detect OODs in clusters A and B with high epistemic uncertainty . The overall TNR ( at 95 % TPR ) reported by this technique is 15 % . These observations can be explained by the focus of different detection techniques on measuring different forms of uncertainty . This motivates our integrated OOD detection method . 3 INTEGRATED OOD DETECTION METHOD . Complementary information about different OOD types can be used to detect a wider range of OODs . Figure 4 shows the improvement in the TNR of the OOD detector composed with information about different classes of OODs on the two half-moons dataset . Non-conformity in the labels of the nearest neighbors captures OODs in cluster C. Mahalanobis distance from the tied in-distribution detects OODs in cluster A. Reconstruction error from the PCA of the 2 class distributions captures OODs in cluster B. Softmax scores further strengthens the OOD detection by reporting OODs in cluster C that are undetected by the other three methods . The integrated OOD detection approach , thus , uses the following attributes , each specialized in detecting a specific type ( or a combination of types ) of OODs : 1 . Mahalanobis distance from the in-distribution density estimate that considers either tied ( Lee et al. , 2018 ) or class-wise covariance estimate . This attribute captures the overall or classconditional epistemic uncertainty of an OOD . Our refinement to also use class-wise covariance significantly improves detection of OODs when coupled with PCA approach described below . 2 . Conformance measure among the variance of the Annoy ( Bernhardsson , 2018 ) nearest neighbors calculated as the Mahalanobis distance of the input ’ s conformance to the closest class conformance . Our experiments found this to be very effective in capturing aleatoric uncertainty . This new attribute is a fusion of nearest-neighbor and Mahalanobis distance methods in literature . 3 . Prediction confidence of the classifier as the maximum softmax score on the perturbed input where the perturbation used is the same as ODIN approach ( Liang et al. , 2017 ) . This boosts the detection of high aleatoric uncertainty by sharpening the class-wise distributions . 4 . Reconstruction error using top 40 % of PCA components where the components are obtained via class conditional PCA of the training data . This boosts the detection of high class-wise epistemic uncertainty by eliminating irrelevant features . This fusion of attributes from existing state-of-the-art detection methods and new attributes was found to be the most effective integrated appraoch capable of detecting the different types of OODs . We evaluated it on several benchmarks as discussed in Section 4 with ablation study in Appendix . 4 EXPERIMENTAL RESULTS . Attributes forming the signature of the OOD detector used in the experiments The signature of the OOD detector used in the experiments is the weighted sum of four attributes , one from each of the following four categories : 1 . Distance from the in-distribution density estimate : We use mahalanobis distance of the input with respect to the closest class conditional distribution . The parameters of this distance are chosen from one of the following two categories : • empirical class means and tied empirical covariance of training samples • empirical class means and empirical class covariance of training samples 2 . Reconstruction error : We perform class conditional PCA empirically from the training samples . We use minimum reconstruction error of the input from the top 40 % eigen vectors of the class conditional eigen spaces . 3 . Prediction confidence of the classifier : We use maximum value of the temperature scaled softmax scores ( S ) on the perturbed input . Perturbations to the input ( x ) are made according to the following equation ( Liang et al. , 2017 ) x̃ = x− sign ( −∇xlogSŷ ( x ; T ) ) ( 1 ) The values of the magnitude of noise ( ) and the temperature scaling parameter ( T ) are chosen from one of the following three categories : • = 0 and T = 0 • = 0 and T = 10 • = 0.005 and T = 10 4 . Conformance measure among the nearest neighbors : We compute an m-dimensional feature vector to capture the conformance among the input ’ s nearest neighbors in the training samples , where m is the dimension of the input . We call this m-dimensional feature vector as the conformance vector . The conformance vector is calculated by taking the mean deviation along each dimension of the nearest neighbors from the input . We hypothesize that this deviation for the in-distribution samples would vary from the OODs due to aleatory uncertainty . The value of the conformance measure is calculated by computing mahalanobis distance of the input ’ s conformance vector to the closest class conformance distribution . Similar to the distance for the in-distribution density estimate , the parameters of this mahalanobis distance are chosen from the following two categories : • empirical class means and tied empirical covariance on the conformance vectors of the training samples • empirical class means and empirical class covariance on the conformance vectors of the training samples The value of the number of the nearest neighbors is chosen from the set { 10 , 20 , 30 , 40 , 50 } via validation . We used Annoy ( Approximate Nearest Neighbors Oh Yeah ) ( Bernhardsson , 2018 ) to compute the nearest neighbors . The weights of the four attributes forming the signature of the OOD detector are generated in the following manner . We use a small subset ( 1000 samples ) of both the in-distribution and the generated OOD data to train a binary classifier using the logistic loss . The OOD data used to train the classifier is generated by perturbing the in-distribution data using the Fast Gradient Sign attack ( FGSM ) ( Goodfellow et al. , 2014 ) . The trained classifier ( or OOD detector ) is then evaluated on the real OOD dataset at the True Positive Rate of 95 % . The best result , in terms of the highest TNR on the validation dataset ( from the training phase of the OOD detector ) , from the twelve combinations of the aforementioned sub-categories ( one from each of the four attributes ) are then reported on the test ( or real ) OOD datasets . Datasets and metrics . We evaluate the proposed integrated OOD detection on benchmarks such as CIFAR10 ( Krizhevsky et al. , 2009 ) and SVHN ( Netzer et al. , 2011 ) . We consider standard metrics ( Hendrycks & Gimpel , 2016 ; Liang et al. , 2017 ; Lee et al. , 2018 ) such as the true negative rate ( TNR ) at 95 % true positive rate ( TPR ) , the area under the receiver operating characteristic curve ( AUROC ) , area under precision recall curve ( AUPR ) , and the detection accuracy ( DTACC ) to evaluate our performance . DNN-based classifier architectures . To demonstrate that the proposed approach generalizes across various network architectures , we consider a wide range of DNN models such as , ResNet ( He et al. , 2016 ) , WideResNet ( Zagoruyko & Komodakis , 2016 ) , and DenseNet ( Huang et al. , 2017 ) . Comparison with the state-of-the-art . We compare our approach with the three state-of-the-art approaches : SPB ( Hendrycks & Gimpel , 2016 ) , ODIN ( Liang et al. , 2017 ) , and Mahalanobis ( Lee et al. , 2018 ) . For the ODIN method , the perturbation noise is chosen from the set { 0 , 0.0005 , 0.001 , 0.0014 , 0.002 , 0.0024 , 0.005 , 0.01 , 0.05 , 0.1 , 0.2 } , and the temperature T is chosen from the set { 1 , 10 , 100 , 1000 } . These values are chosen from the validation set of the adversarial samples of the in-distribution data generated by the FGSM attack . For the Mahalanobis method , we consider their best results obtained after feature ensemble and input preprocessing with the hyperparameters of their OOD detector tuned on the in-distribution and adversarial samples generated by the FGSM attack . The magnitude of the noise used in pre-processing of the inputs is chosen from the set { 0.0 , 0.01 , 0.005 , 0.002 , 0.0014 , 0.001 , 0.0005 } . CIFAR10 . With CIFAR10 as in-distribution , we consider SVHN ( Netzer et al. , 2011 ) , TinyImagenet ( Deng et al. , 2009 ) , and LSUN ( Yu et al. , 2015 ) as the OOD datasets . For CIFAR10 , we consider two DNNs : ResNet50 , and WideResNet . Table 1 shows the results . SVHN . With SVHN as in-distribution , we consider CIFAR10 , Imagenet , and LSUN and as the OOD datasets . For SVHN , we use the DenseNet classifier . Table 1 shows the results . Key observations . We do not consider pre-processing of the inputs in our integrated OOD detector . Even without input pre-processing and with the exception of CIFAR10 OOD dataset for SVHN indistribution trained on DenseNet , we could perform equally well ( and even out-perform in most of the cases ) as the Mahalanobis method on its best results generated after pre-processing the input . We also consider a Subset-CIFAR100 as OODs for CIFAR10 . Specifically , from the CIFAR100 classes , we select sea , road , bee , and butterfly as OODs which are visually similar to the ship , automobile , and bird classes in the CIFAR10 , respectively . Thus , there can be numerous OOD samples due to aleatoric and class-conditional epistemic uncertainty which makes the OOD detection challenging . Figure 5 shows the t-SNE ( Maaten & Hinton , 2008 ) plot of the penultimate features from the ResNet50 model trained on CIFAR10 . We show 4 examples of OODs ( 2 due to epistemic and 2 due to aleatoric uncertainty ) from Subset-CIFAR100 . These OODs were detected by our integrated approach but missed by the Mahalanobis approach . These observations justify the effectiveness of integrating multiple attributes to detect OOD samples . Additional experimental results in the appendix . We also compare the performance of the integrated OOD detector with the SPB , ODIN and Mahalanobis detector in supervised settings , as reported by the Mahalanobis method for OOD detection ( Lee et al. , 2018 ) . These results include experiments on CIFAR10 , SVHN and MNIST as in-distribution data and Imagenet , LSUN , SVHN ( for CIFAR10 ) , CIFAR10 ( for SVHN ) , KMNIST , and F-MNISTas OOD data across different DNN architectures such as ResNet34 , WideResNet , DenseNet , and LeNet5 . All these results , along with the ablation studies on OOD detectors with single attributes are included in the Appendix . In almost all of the reported results in the Appendix , our OOD detector could outperform the compared state-of-the-art methods with improvements of even 2X higher TNR at 95 % TPR in some cases .

Summary: This paper introduces a novel taxonomy for OOD outliers. The authors analyze current OOD detection approaches and uncover their limitations. They propose to fuse several existing approaches into a combined one and extensively evaluate it on various data sets (CIFAR,10, SVNH, MNIST, STL10, ImageNet, etc.). The proposed integrated OOD detection approach clearly shows superior performance.