Informative Outlier Matters: Robustifying Out-of-distribution Detection Using Outlier Mining

1 INTRODUCTION . Out-of-distribution ( OOD ) detection has become an indispensable part of building reliable open-world machine learning models ( Amodei et al. , 2016 ) . An OOD detector determines whether an input is from the same distribution as the training data , or a different distribution ( i.e. , out-of-distribution ) . The performance of the OOD detector is central for safety-critical applications such as autonomous driving ( Eykholt et al. , 2018 ) or rare disease identification ( Blauwkamp et al. , 2019 ) . Despite exciting progress made in OOD detection , previous methods mostly focused on clean OOD data ( Hendrycks & Gimpel , 2016 ; Liang et al. , 2018 ; Lee et al. , 2018 ; Lakshminarayanan et al. , 2017 ; Hendrycks et al. , 2018 ; Mohseni et al. , 2020 ) . Scant attention has been paid to the robustness aspect of OOD detection . Recent works ( Hein et al. , 2019 ; Sehwag et al. , 2019 ; Bitterwolf et al. , 2020 ) considered worst-case OOD detection under adversarial perturbations ( Papernot et al. , 2016 ; Goodfellow et al. , 2014 ; Biggio et al. , 2013 ; Szegedy et al. , 2013 ) . For example , an OOD image ( e.g. , mailbox ) can be perturbed to be misclassified by the OOD detector as in-distribution ( traffic sign data ) . Such an adversarial OOD example is then passed to the image classifier and trigger undesirable prediction and action ( e.g. , speed limit 70 ) . Therefore , it remains an important question to make out-of-distribution detection algorithms robust in the presence of small perturbations to OOD inputs . In this paper , we begin with formally formulating the task of robust OOD detection and providing theoretical analysis in a simple Gaussian data model . While recent OOD detection methods ( Hendrycks et al. , 2018 ; Hein et al. , 2019 ; Meinke & Hein , 2019 ; Mohseni et al. , 2020 ) have leveraged auxiliary OOD data , they often sample randomly uniformly from the auxiliary dataset . Contrary to the common practice , our analysis reveals a key insight that the majority of auxiliary OOD examples may not provide useful information to improve the decision boundary of OOD detector . Under a Gaussian model of the data , we theoretically show that using outlier mining significantly improves the error bound of OOD detector in the presence of non-informative auxiliary OOD data . Motivated by this insight , we propose Adversarial Training with informative Outlier Mining ( ATOM ) , which justifies the theoretical intuitions above and achieves state-of-the-art performance on a broad family of classic and adversarial OOD evaluation tasks for modern neural networks . We show that , by carefully choosing which OOD data to train on , one can significantly improve the robustness of an OOD detector , and somewhat surprisingly , generalize to unseen adversarial attacks . We note that while hard negative mining has been extensively used in various learning tasks such as object recognition ( Felzenszwalb et al. , 2009 ; Gidaris & Komodakis , 2015 ; Shrivastava et al. , 2016 ) , to the best of our knowledge , we are the first to exploit the novel connection between hard example mining and OOD detection . We show both empirically and theoretically that hard example mining significantly improves the generalization and robustness of OOD detection . To evaluate our method , we provide a unified framework that allows examining the robustness of OOD detection algorithms under a broad family of OOD inputs , as illustrated in Figure 1 . Our evaluation includes existing classic OOD evaluation task – Natural OOD , and adversarial OOD evaluation task – L∞ OOD . Besides , we also introduce new adversarial OOD evaluation tasks – Corruption OOD and Compositional OOD . Under these evaluation tasks , ATOM achieves state-ofthe-art performance compared to eight competitive OOD detection methods ( refer to Appendix B.3 for a detailed description of these methods ) . On the Natural OOD evaluation task , ATOM achieves comparable and often better performance than current state-of-the-art methods . On L∞ OOD evaluation task , ATOM outperforms current state-of-the-art method ACET by a large margin ( e.g . on CIFAR-10 , outperforms it by 53.9 % ) . Under the new Corruption OOD evaluation task , where the attack is unknown during training time , ATOM also achieves much better results than previous methods ( e.g . on CIFAR-10 , outperform previous best method by 30.99 % ) . While almost every method fails under the hardest Compositional OOD evaluation task , ATOM still achieves impressive results ( e.g . on CIFAR-10 , reduce the FPR by 57.99 % ) . The performance is noteworthy since ATOM is not trained explicitly on corrupted OOD inputs . In summary , our contributions are : • Firstly , we contribute theoretical analysis formalizing the intuition of mining hard outliers for improving the robustness of OOD detection . • Secondly , we contribute a theoretically motivated method , ATOM , which leads to state-of- the-art performance on both classic and adversarial OOD evaluation tasks . We conduct extensive evaluations and ablation analysis to demonstrate the effectiveness of informative outlier mining . • Lastly , we provide a unified evaluation framework that allows future research examining the robustness of OOD detection algorithms under a broad family of OOD inputs . 2 PRELIMINARIES . In this section , we formulate the problem of robust out-of-distribution detection , and provide background knowledge on the use of auxiliary data for OOD detection . Problem Statement . We consider a training dataset Dtrainin drawn i.i.d . from a data distribution PX , Y , where X is the sample space and Y = { 1 , 2 , · · · , K } is the set of labels . A classifier f ( x ) is trained on the in-distribution , PX , the marginal distribution of PX , Y . The OOD examples are revealed during test time , which are from a different distribution QX , potentially with perturbations added . The task of robust out-of-distribution detection is to learn a detector G : x→ { −1 , 1 } , which outputs 1 for x from PX and output −1 for a clean or perturbed OOD example x from QX . Formally , let Ω ( x ) be a set of small perturbations on an OOD example x . The detector is evaluated on x from PX and on the worst-case input inside Ω ( x ) for an OOD example from QX . The false negative rate ( FNR ) and false positive rate ( FPR ) are defined as : FNR ( G ) = Ex∼PXI [ G ( x ) = −1 ] , FPR ( G ; QX , Ω ) = Ex∼QX max δ∈Ω ( x ) I [ G ( x + δ ) = 1 ] . ( 1 ) Note that no data from the test OOD distribution QX are available for training . Use of Auxiliary Data for OOD Detection While it is impossible to anticipate the test OOD data distributions QX for training , recent works ( Hendrycks et al. , 2018 ; Hein et al. , 2019 ; Meinke & Hein , 2019 ; Mohseni et al. , 2020 ; Liu et al. , 2020 ) have shown the promise using auxiliary data as a proxy for estimating the decision boundary between in- vs. OOD data . The idea is illustrated in Figure 1 , where outlier data is randomly sampled to regularize the model outputs ( e.g. , low confidence for OOD data and high confidence for in-distribution data ) . Formally , we assume the auxiliary OOD dataset Dauxiliaryout is sampled from a different distribution UX . The difference between the auxiliary data UX and test OOD data PX raises the fundamental question of how to effectively leverage Dauxiliaryout for improving learning the decision boundary between in- vs. OOD data . 3 THEORETICAL ANALYSIS : INFORMATIVE OUTLIERS MATTER . In this section , we present a theoretical analysis1 that motivates the use of informative outlier mining for OOD detection . To establish formal guarantees , we use a Gaussian data model to model data PX , QX , and UX . Different from previous work by Schmidt et al . ( 2018 ) and Carmon et al . ( 2019 ) , our analysis gives rise to a separation result with or without informative outlier mining for OOD detection . To this end , we note that while hard negative mining has been explored in different domains of learning ( e.g . object detection , deep metric learning , please refer to Section 6 for details ) , the vast literature of out-of-distribution detection has not explored this idea . Moreover , most uses of hard negative mining are on a heuristic basis , but in our case , the simplicity of the definition of OOD ( see Section 2 ) allows us to derive precise formal guarantees , which further differs us from previous studies of hard negative mining . As a remark , our analysis also establishes formal evidence of the importance of using auxiliary outlier data for OOD detection , which is lacking in the current OOD detection studies . We refer readers to Section A for these results . At a high level , our analysis provides two important insights : ( 1 ) First , we show that a detection algorithm can work very well if all data is informative ; yet it can fail completely in a natural setting where we mix informative auxiliary data with non-informative auxiliary data . ( 2 ) Second , we show that , tweaking the algorithm with simple thresholding to choose mildly hard auxiliary data ( in our setting they are exactly the informative ones ) , can lead to good detection performance . Combining both thus provides direct evidence about the importance of hard negative mining for OOD . Gaussian Data Model . We now describe the Gaussian data model , inspired by the model in ( Schmidt et al. , 2018 ; Carmon et al. , 2019 ) , but with important adjustment to the OOD detection setting . In particular , our setting has a family Q of possible test OOD distributions and have only indistribution data for training , modeling that the test OOD distribution is unknown for training . Given µ ∈ Rd , σ > 0 , ν > 0 , we consider the following model : • PX ( in-distribution data ) : N ( µ , σ2I ) ; The in-distribution data { xi } ni=1 is drawn from PX . • QX ( out-of-distribution data ) can be any distribution from the family Q = { N ( −µ + v , σ2I ) : v ∈ Rd , ‖v‖2 ≤ ν } . 1Due to lack of space , proofs are deferred to Appendix A . • Hypothesis class of OOD detector : G = { Gθ ( x ) = sign ( θ > x ) : θ ∈ Rd } . A concrete instance of the model is defined by a set of parameter values for d , µ , σ , and ν ; see Appendix A.2 for the family of instances we analyze . While the Gaussian model may be much simpler than the practical data , its simplicity is desirable for our analytical purpose for demonstrating the insights . Furthermore , the analysis in this simple model has implications for more complicated and practical methods , which we present in Section 4 . Finally , the analysis can be generalized to mixtures of Gaussians which models practical data much better . Below , we consider the FNR and the FPR under ` ∞ perturbations of magnitude . Since QX is not accessible at training time , our goal is to bound supQX∈Q FPR ( G ; QX , Ω∞ , ( x ) ) . Failing a good detector by mixing non-informative auxiliary data . We start by considering the case where all auxiliary data are informative : That is , all auxiliary { x′i } n ′ i=1 come from uniform mixture of the possible test OOD distributions in Q . In this case , it is straightforward to show that a simple averaging-based detector , θ̂n , n′ = 1 n+ n′ n∑ i=1 xi − n′∑ i=1 x̃i , ( 2 ) performs very well ( See Proposition 4 in the appendix ) . Unfortunately , this detector can be easily failed by considering the following simple auxiliary data distribution Umix , which mixes the ideal auxiliary data with non-informative data • Umix ( Non-ideal mixture ) : Umix is a uniform mixture of N ( −µ , σ2I ) and N ( µo , σ2I ) with µo = 10µ . Importantly , the distribution Umix models the case where the auxiliary OOD data has some noninformative outliers , and also with a small probability mass of samples ( e.g. , tail of N ( −µ , σ2I ) ) in the support of in-distribution . In this case , the simple average method leads to E [ θ̂n , n′ ] = −7µ/4 with a large error , since θ̂n , n′ is misled by auxiliary data from N ( µo , σ2I ) or tail of N ( −µ , σ2I ) in the support of in-distribution . Fixing the detector with informative outlier mining . We now show an important modification to the detection algorithm by using informative outlier mining , which leads to good detection performance . Specifically , we first use in-distribution data to get an intermediate solution : θ̂int = 1 n ∑n i=1 xi . Then , we use a simple thresholding mechanism to only pick points with mild confidence scores , which removes non-informative outliers . Specifically , we only select outliers x̃ whose confidence scores f ( x̃ ) = 1/ ( 1 + e−x̃ > θ̂int/d ) fall in an interval [ a , b ] . The final solution θ̂om is −1 times the average of the selected outliers . We can prove the following : Proposition 1 . ( Error bound with outlier mining . ) For any ∈ ( 0 , 1/2 ) and any integer n0 > 0 , there exist a family of instances of the Gaussian data model such that the following is true . n′ auxiliary OOD data from Umix specified above . There exist thresholds a and b for θ̂om and a universal constant c > 0 such that if the number of in-distribution data n ≥ c ( n0 log d + √ dn0 ) and the number of auxiliary data n′ ≥ ( d+ n0 · 4 2 ) √ d/n0 , then θ̂om has small errors:2 Eθ̂omFNR ( Gθ̂om ) ≤ 10 −3 , Eθ̂om sup QX∈Q FPR ( Gθ̂om ; QX , Ω∞ , ( x ) ) ≤ 10 −3 . ( 3 ) Intuitively , the mining method removes the misleading points ( most points in N ( µo , σ2I ) and tail of N ( −µ , σ2I ) in the support of in-distribution ) . Outliers selected in this way are mostly informative and thus give an accurate final detector , which justifies outlier mining in the presence of non-informative data . If we compare this bound with that for the case without auxiliary data ( Proposition 3 ) , we can see that for sufficiently high dimension d , with the same amount of in-distribution data , any algorithm without outliers must fail but our outlier mining method can learn a good detector . We also note that our analysis and the result also hold for many other auxiliary data distributions Umix , and the particular Umix used here is for the simplicity of demonstration ; see the appendix for more discussions . In the following section , we design a practical algorithm based on this insight and present empirical evidence of its effectiveness . 2The error bound in the proposition can be made arbitrarily small and with high probability . The current bound is presented for simplicity .

Summary: In this paper the authors propose a method for training a classifier to be more effective at OOD (out of distribution) detection. Many OOD detection methods work by utilizing an auxiliary dataset as examples of OOD-ness. This is the approach taken in this paper and OOD is trained as being a k+1 classification class.  When training the OOD class the proposed method allows for adversarial perturbation of the OOD examples to help improve training. This is a pretty common technique in deep learning, see for example "Deep Robust One Class Classification."  Finally the main novelty of the method proposed by the authors is to sort a collection of OOD examples and sort according to the OOD score of the current model and use the "qNth" to be presented as OOD examples during the next epoch during training. The authors term this "Informative Outlier Mining." The authors demonstrate that this method works well experimentally.