Adversarial Distributions Against Out-of-Distribution Detectors

Out-of-distribution ( OOD ) detection is the task of determining whether an input lies outside the training data distribution . As an outlier may deviate from the training distribution in unexpected ways , an ideal OOD detector should be able to detect all types of outliers . However , current evaluation protocols test a detector over OOD datasets that cover only a small fraction of all possible outliers , leading to overly optimistic views of OOD detector performance . In this paper , we propose a novel evaluation framework for OOD detection that tests a detector over a larger , unexplored space of outliers . In our framework , a detector is evaluated with samples from its adversarial distribution , which generates diverse outlier samples that are likely to be misclassified as in-distribution by the detector . Using adversarial distributions , we investigate OOD detectors with reported near-perfect performance on standard benchmarks like CIFAR-10 vs SVHN . Our methods discover a wide range of samples that are obviously outlier but recognized as indistribution by the detectors , indicating that current state-of-the-art detectors are not as perfect as they seem on existing benchmarks . 1 INTRODUCTION . Identifying whether an input datum lies outside the training data distribution is one of the canonical problems in machine learning . Over its long history , the problem has been called by multiple names , including novelty detection ( Markou & Singh , 2003 ) , outlier detection ( Hawkins , 1980 ) , oneclass classification ( Japkowicz et al. , 1995 ) , and more recently , out-of-distribution ( OOD ) detection ( Hendrycks & Gimpel , 2016 ) . Investigation of the problem has resulted in a number of real-world applications , for example , medical diagnosis ( Li et al. , 2019 ) and inspection of defective parts and products ( Bergmann et al. , 2019 ) . The relevance of OOD detection is growing beyond these applications , as an OOD detector is considered an essential component of a trustworthy machine learning system . For example , without a reliable OOD detector , an image classifier trained to classify cats and dogs may incorrectly classify a human as belonging to one of these two classes ( Hendrycks et al. , 2019b ) . In order to advance the development of reliable OOD detection algorithms , a more comprehensive evaluation protocol is needed . Current evaluation protocols adopted by the community provide a distorted view of a detector ’ s performance for two reasons . First , OOD detectors are tested over a small fraction of possible outliers . Detectors are typically evaluated using test OOD datasets chosen by a researcher . Since the test OOD datasets do not cover the entire space of outliers , there may exist untested outliers that the detector fails to classify correctly , even though the detector perfectly detects the chosen test OOD points as shown in Figure 1 . To assess a detector ’ s performance in a more comprehensive and systematic way , a method is needed to test a detector over a larger , unexplored space beyond what is covered by the test OOD datasets . Second , the current evaluation protocol neglects the worst-case behavior of an OOD detector . The average performance metric is often not sufficient to build trust on a detector , because in safetycritical applications even a single mistake can result in fatal consequences . A detector should be tested adversarially , through an active search for its worst-case failure mode , i.e. , an outlier that is classified with maximal confidence as an inlier . Such failure cases may reveal weaknesses of the tested detector , and provide informative clues for building a better detection algorithm . In this paper , we propose a novel evaluation protocol of OOD detectors that addresses the above-mentioned limitations of current evaluation methods . We first formulate the notion of adversarial search against an OOD detector , a search problem of finding an outlier that a detector classifies as an inlier with the greatest confidence . A detector may have more than one significant failure mode , and it is more informative to find a set of diverse failure cases instead of the most critical one . To that end , we propose the adversarial distribution against an OOD detector , which generates outlier samples that are likely to be misclassified by the given detector . Measuring OOD detection performance against samples from a detector ’ s adversarial distribution gives an accurate and finer-grained assessment on its performance . To ensure that samples from an adversarial distribution are indeed outliers , the distribution needs to be supported on a zero-inlier space , a set without any overlap to the inlier distribution . Meanwhile , the zero-inlier space should be large so that we may observe meaningful failure mode of OOD detectors within the space . However , finding such a space is generally challenging , as the true boundary between inliers and outliers is unknown . We circumvent this challenge by building a generative model over known outliers . Our construction of zero-inlier spaces contain diverse samples . We implement 11 previously proposed OOD detectors and investigate their behavior using their adversarial distributions . Among the tested detectors , 8 detectors report near-perfect OOD detection performance on a popular benchmark , CIFAR-10 ( in ) vs SVHN ( out ) , effectively being indistinguishable with respect to their performance . Our investigation reveals that the 8 detectors in fact have diverging degrees of detection quality outside the SVHN test set . Our methods also lead to several interesting insights that suggest techniques for improving OOD detection . Our main contributions can be summarized as follows : • We propose the adversarial search and adversarial distributions , which can be used to evaluate OOD detection algorithms beyond a pre-defined test OOD dataset ; • We provide practical techniques to define the space of outliers that contains samples outside the test OOD dataset ; • By examining the state-of-the-art OOD detectors , we show that OOD detectors with seemingly equivalent performance differ significantly in their failure modes ; Related Work Testing the worst-case performance of an algorithm is highly related to evaluating adversarial robustness of the detector . Detection of adversarially perturbed outliers is investigated in previous literature where perturbation is assumed to be restricted in a small norm-ball Hein et al . ( 2019 ) ; Meinke & Hein ( 2020 ) ; Bitterwolf et al . ( 2020 ) . The idea of using an autoencoder during an adversarial attack is investigated in Tu et al . ( 2019 ) . The idea of generating outliers are investigated in the context of improving OOD detection Chen et al . ( 2020 ) or adversarial attack Song et al . ( 2018 ) . In Section 2 , we provide essential preliminaries . Section 3 formulates adversarial search and adversarial distributions , and Section 4 introduces techniques to construct a zero-inlier space . Our main experimental results are presented in Section 5 , with deeper discussions provided in Section 6 . Section 7 concludes the paper . 2 BACKGROUND : OUT-OF-DISTRIBUTION DETECTION . 2.1 DEFINITION . We consider a probability distribution of interest Pin , samples from which we consider as inliers . Each sample is represented as a D-dimensional real-valued vector . The probability density function of Pin is denoted as pin ( x ) . We write the support of Pin as Sin = { x|pin ( x ) > 0 } ⊂ X ⊂ RD , where X is the data space , the set of all possible values for data , which we assume to be compact . Then , we define OOD-ness as follows : Definition 1 . A vector x is out-of-distribution ( OOD ) , or an outlier , with respect to Pin if x does not belong to the support of Pin , i.e. , x 6∈ Sin . Conversely , x is in-distribution when x ∈ Sin . A distribution Q having the support SQ is OOD to Pin , when SQ ∩ Sin = ∅ . Another popular definition of OOD-ness characterizes a vector x as OOD when the vector belongs to a density sub-level set , x ∈ { x|pin ( x ) ≤ η } ( Steinwart et al. , 2005 ) . However , the density sub-level set does not provide a consistent characterization of the OOD-ness . A vector classified as OOD in one coordinate may not be classified as OOD in a different coordinate , because a probability density function can be arbitrarily distorted via an invertible coordinate transform as pointed out in ( Lan & Dinh , 2020 ) . On the contrary , our definition of OOD using the density support provides an invariant characterization of outliers with respect to such transforms . An OOD detector f ( x ) : RD → R is a function which outputs a larger value for an input more likely to be an outlier . A test vector x∗ is classified as OOD f ( x∗ ) > ηf for the threshold ηf . A detector score refers to the function value of f ( x ) . We shall assume f ( x ) is bounded . In this paper , we consider a black-box setting , where any information other than the function value of f ( x ) , such as its gradient , is not accessible . An OOD detector is normally trained using an in-distribution datasetDin , a set of iid samples from Pin . However , some OOD detectors utilize additional datasets other than Din . 2.2 EVALUATION OF OOD DETECTORS . The currently accepted evaluation protocol for OOD detectors relies on one or multiple test OOD datasets Dout , which contain a finite number of samples considered as OOD with respect to Pin by human prior knowledge . When Pin is a distribution of images , test OOD sets are often chosen from separately published image datasets of which contents are different from that of Pin . For example , when an image dataset of animals is selected as in-distribution , a set of digit images can be used as a test OOD dataset . Given a test OOD dataset , an OOD detector f ( x ) performs the binary classification against indistribution dataset , and the quality of the classification is considered as an indicator for how good f ( x ) is as an OOD detector . The classification result is summarized using metrics such as the area under the receiver operating characteristic curve ( AUROC or AUC ) . AUC is a preferred metric in a number of literature , as it does not require the specification of the threshold ηf . AUC score of 1.0 indicates the perfect classification , and AUC of 0.5 implies the random guess . The research community has been focusing on a few representative in-distribution and OOD dataset pairs , such as CIFAR-10 ( in ) vs SVHN ( out ) and Fashion-MNIST ( in ) vs MNIST ( out ) . These dataset pairs become popular after the reports showing that OOD detectors built upon deep generative models , such as PixelCNN++ ( Salimans et al. , 2017 ) or Glow ( Kingma & Dhariwal , 2018 ) , fail to detect outliers ( Hendrycks et al. , 2019a ; Nalisnick et al. , 2019 ) . In fact , the generative-modelbased detectors score AUC lower than 0.5 , meaning that SVHN images are more strongly perceived as CIFAR-10 than the actual CIFAR-10 images by the detectors . This observation spurred intense research efforts , and now there are multiple OOD detectors achieving AUC scores higher than 0.9 or even higher than 0.99 on CIFAR-10 vs SVHN as listed in Section 5 . Given the near-perfect detection score , we question whether the detectors are indeed good OOD detectors beyond the tested examples .

Summary: The paper proposes a novel evaluation framework for out-of-distribution (OOD) detection under worst-case scenarios. While existing benchmarks use real samples from datasets outside the training distribution, the authors propose instead to learn an adversarial outlier distribution against OOD detectors using an autoencoder model, with an auxiliary binary classifier that filters out inlier samples. Empirical experiments on CIFAR-10 (inlier) and SVHN/CelebA (outlier) datasets show that standard OOD benchmarks tend to produce overoptimistic results, and that prior methods with similar scores on standard benchmarks have diverging performance outside the predefined OOD test sets.