Exploring the Limits of Out-of-Distribution Detection

1 Introduction . Deep neural networks are increasingly used in high-stakes applications such as healthcare [ Roy et al. , 2021 , Ren et al. , 2019 ] . Safe deployment of models requires that models not only be accurate but also be robust to distribution shift [ Amodei et al. , 2016 ] . Neural networks can assign high-confidence predictions to mis-classified inputs [ Guo et al. , 2017 , Lakshminarayanan et al. , 2017 ] as well as test inputs that do not belong to one of the training classes [ Nguyen et al. , 2015 ] . This motivates the need for methods that can reliably detect out-of-distribution ( OOD ) inputs . There has been a lot of progress in detecting OOD inputs including methods based on discriminative models [ Hendrycks and Gimpel , 2016 , Lee et al. , 2018 , Liang et al. , 2017 , Liu et al. , 2020 ] as well as methods based on deep generative models [ Nalisnick et al. , 2019 , Zhang et al. , 2020 ] . The difficulty of the OOD detection task depends on how semantically close the outliers are to the inlier classes . Winkens et al . [ 2020 ] distinguish between near-OOD tasks which are harder and far-OOD tasks which are easier , as evidenced by the difference in state-of-the-art ( SOTA ) for area under the receiver operating characteristic curve ( AUROC ) . For instance , for a model trained on CIFAR-100 ( which consists of classes such as mammals , fish , flowers , fruits , household devices , trees , vehicles , insects , etc ) , a far-OOD task would be detecting digits from the street-view house numbers ( SVHN ) dataset as outliers . For the same model , detecting images from the CIFAR-10 dataset ( which consists of the following 10 classes : airplane , automobile , bird , cat , deer , dog , frog , horse , ship , truck ) would be considered a near-OOD task , which is more difficult as the classes are semantically similar . There has been impressive progress on far-OOD detection , for instance there are several approaches which can achieve AUROC close to 99 % on CIFAR-100 ( in ) vs SVHN ( out ) task , cf . [ Sastry and Oore , 2020 ] . However , the state-of-the-art for near-OOD detection is much lower , for instance the SOTA AUROC for CIFAR-100 ( in ) vs CIFAR-10 ( out ) task is around 85 % ⇤Equal contribution . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . [ Zhang et al. , 2020 ] which is considerably lower than the SOTA for far-OOD tasks . Similar trends are observed in other modalities such as genomics where the SOTA AUROC of near-OOD detection is only 66 % [ Ren et al. , 2019 ] . Improving the SOTA for these near-OOD detection tasks and closing the performance gap between near-OOD detection and far-OOD detection is one of the key challenges in ensuring the safe deployment of models . Large-scale pre-trained transformers have led to significant accuracy improvements in multiple domains , cf . Bidirectional Encoder Representations from Transformers ( BERT ) for text [ Devlin et al. , 2018 ] , Vision Transformers ( ViT ) for images [ Dosovitskiy et al. , 2021 ] , Contrastive Language–Image Pre-training ( CLIP ) trained on image-text pairs [ Radford et al. , 2021 ] . We show that classifiers obtained by fine-tuning large-scale pre-trained transformers are significantly better at near-OOD detection . Intuitively , large-scale pre-training makes classifiers less vulnerable to shortcut learning [ Geirhos et al. , 2020 ] , making these representations better suited for near-OOD detection . Figure 1 visualizes two-dimensional PCA projections of representations from residual networks ( ResNet ) [ He et al. , 2016 ] trained on CIFAR-100 and ViT model pre-trained on ImageNet-21k and fine-tuned on CIFAR-100 ; we can observe that representations obtained by fine-tuning pre-trained transformers are better suited at near-OOD detection than representations from ResNet just trained on CIFAR-100 . Motivated by real-world applications which demand very high level of OOD detection for safe deployment , we explore variants of outlier exposure to further improve OOD detection . We show that pre-trained transformers are particularly well-suited at leveraging known outliers due to their highquality representations ( see Figure 1 ) . We systematically vary the number of outlier examples per class , and show that even a handful of known outliers can significantly improve OOD detection . We refer to this setting as few-shot outlier exposure . For multi-modal pre-trained transformers , we explore a new form of outlier exposure that leverages names of outlier classes without any accompanying images , and show that this can significantly improve OOD detection for zero-shot classification . In summary , our contributions are the following : • We show that pre-trained transformers lead to significant improvements on near-OOD benchmarks . Concretely , we improve the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 from 85 % ( current SOTA ) to 96 % using ViT pre-trained on ImageNet-21k , and improve the AUROC on a genomics OOD detection benchmark from 66 % ( current SOTA ) to 77 % using BERT . • We show that pre-trained transformers are well-suited for few-shot outlier exposure . With just 10 labeled examples per class , we can improve the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 to 99 % , and improve the AUROC of OOD detection on genomics to 86 % . • We explore OOD detection for pre-trained multi-modal image-text transformers in the zero-shot classification setting , and show that just using the names of outlier classes as candidate text labels for CLIP , we can achieve AUROC of 94.8 % on CIFAR-100 vs CIFAR-10 task . On easier far-OOD tasks such as CIFAR- { 100 , 10 } vs SVHN , we achieve AUROC of 99.6 % and 99.9 % respectively . 2 Background and Related work . Notation We assume that we have an in-distribution dataset Din of ( xin , yin ) pairs where x denotes the input feature vector , and yin 2 Y in : = { 1 , . . . , K } denotes the class label . Let Dout denote an outof-distribution dataset of ( xout , yout ) pairs where yout 2 Yout : = { K+1 , . . . , K+O } , Yout\Y in = ; . Depending on how different Dout is from Din , we categorize the OOD detection tasks into nearOOD and far-OOD . We first study the scenario where the model is fine-tuned only on the training set Dintrain without any access to OOD data . The test set contains Dintest and Douttest for evaluating OOD performance using AUROC . Next , we explore the scenario where a small number of OOD examples are available for training , i.e . the few-shot outlier exposure setting . In this setting , the training set contains Dintrain S Doutfew shot , where |Doutfew shot| is often smaller than 100 per OOD class . 2.1 Methods for detecting OOD inputs . We describe a few popular techniques for detecting OOD inputs using neural networks . Maximum over softmax probabilities ( MSP ) A baseline method for OOD detection is to use the maximum softmax probability as the confidence score , i.e . scoremsp ( x ) = maxc=1 , ... , K p ( y = c|x ) [ Hendrycks and Gimpel , 2016 ] . While being slightly worse than other techniques , its simplicity and performance make it an ideal baseline . Mahalanobis distance Lee et al . [ 2018 ] proposed to fit a Gaussian distribution to the classconditional embeddings and use the Mahalanobis distance for OOD detection . Let f ( x ) denote the embedding ( e.g . the penultimate layer before computing the logits ) of an input x . We fit a Gaussian distribution to the embeddings of the training data , computing per-class mean µc = 1Nc P i : yi=c f ( xi ) and a shared covariance matrix ⌃ = 1N P K c=1 P i : yi=c f ( xi ) µc f ( xi ) µc > . The Mahalanobis score ( negative of the distance ) is then computed as : scoreMaha ( x ) = minc ⇣ 1 2 f ( x ) µc ⌃ 1 f ( x ) µc > ⌘ . Outlier exposure Hendrycks et al . [ 2018 ] proposed outlier exposure which leverages a large dataset of known outliers . For classification problems , the model is trained to predict uniform distribution over labels for these inputs . Thulasidasan et al . [ 2021 ] proposed to use a single outlier class as the ( K + 1 ) th class for a ( K + 1 ) -way classification problem . Roy et al . [ 2021 ] showed that leveraging the labels of known outliers ( rather than assigning all known outliers to a single class ) can further improve OOD detection performance . 2.2 Pre-training neural networks . Architectures using self-attention , typically based on the Transformer [ Vaswani et al. , 2017 ] , often combined with large-scale pre-training directly on raw text , have been very popular for natural language processing ( NLP ) tasks in recent years [ Devlin et al. , 2018 , Dai and Le , 2015 , Peters et al. , 2018 , Howard and Ruder , 2018 , Radford et al. , 2018 , Raffel et al. , 2019 ] . Often followed by fine-tuning on a smaller , downstream dataset , large-scale pre-training techniques lead to highly informative embeddings that are broadly useful in natural language tasks . The advantage of the transformer architecture is its ability to scale to very large model sizes , reaching up to 1 trillion parameter mark [ Fedus et al. , 2021 ] . Large pre-trained models , such as GPT-3 [ Brown et al. , 2020 ] , have shown the potential of large-scale task-agnostic pre-training in language . The Vision Transformer ( ViT ) [ Dosovitskiy et al. , 2021 ] has shown that self-attention combined with large-scale pre-training is a viable strategy for vision tasks as well . The performance of ViT is comparable to other state-of-the-art models , while being more efficient to train . Its ability to quickly fine-tune to a smaller , downstream dataset , generalizing even in a few-shot regime , makes it an attractive backbone for tasks such as out-of-distribution ( OOD ) detection . In this paper , we use ViT pre-trained on ImageNet-21k [ Ridnik et al. , 2021 ] . Multi-modal text-image transformers such as CLIP ( Contrastive Language-Image Pre-Training ) [ Radford et al. , 2021 ] pre-train on 400 million ( image , text ) pairs from the internet to learn to predict a raw text caption from an image , and by doing so develop state-of-the-art visual representations and their natural language counterparts . Radford et al . [ 2021 ] showed that CLIP improves robustness to natural distribution shift . In this paper , we use the shared image-text embedding to introduce a new zero-shot OOD detection method ( Section 5 ) . Hendrycks et al . [ 2019a ] show that pre-training improves OOD detection for non-transformer architectures . Self-supervised learning techniques have also been shown to improve OOD detection ; Hendrycks et al . [ 2019b ] use rotation prediction and Winkens et al . [ 2020 ] use contrastive training to improve near-OOD detection . Robustness of pre-trained transformers Hendrycks et al . [ 2020 ] show that pre-trained transformers improve OOD detection in NLP . Pre-trained BERT has been has used as a backbone for OOD detection in language , cf . [ Liu et al. , 2020 ] . Unlike them , we focus on vision and genomics modalities , and specifically on near-OOD detection benchmarks . Investigating the robustness of pre-trained ViT is an active research area and there are several concurrent papers exploring robustness of ViT to adversarial perturbations and common corruptions ( ImageNet-C ) . Bhojanapalli et al . [ 2021 ] show the robustness of pre-trained transformers to input perturbations , and of transformers to layer removal . Caron et al . [ 2021 ] demonstrate many emerging properties in self-supervised ViTs , while Shao et al . [ 2021 ] , Mahmood et al . [ 2021 ] show that they are more robust to adversarial perturbations , and Naseer et al . [ 2021 ] that they have less texture bias . Paul and Chen [ 2021 ] , Minderer et al . [ 2021 ] show that ViT is more robust to distribution shift and natural adversarial examples [ Paul and Chen , 2021 ] , and Mao et al . [ 2021 ] propose a robust ViT . To the best of our knowledge , we are the first to show that pre-trained ViT can significantly improve near-OOD detection in vision benchmarks , and show that few-shot outlier exposure can further improve performance .

Summary: This paper studies the few-shot outlier detection setting where a few examples from outlier classes are provided to the ODD detector.  The main motivation behind this work are situations that require high level of ODD detection. It is shown that classifiers trained by fine-tuning large-scale pre-trained transformers are significantly better at near-ODD detection. Simulations are performed on well-known data sets for visual classification and genomic sequences. Performance evaluation are reported in term of AUROC scores. However, it would have been more convenient to report results in terms of FPR at 95% scores which allow to better understand if the performance gains with respect to similar methods are relevant. The reported gains are not very significant but consistent.