Exploring the Limits of Out-of-Distribution Detection
Year: 2022

Abstract:
Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to 96% using Vision Transformers pre-trained on ImageNet-21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks.

heading: 1 Introduction
text: Deep neural networks are increasingly used in high-stakes applications such as healthcare [Roy et al., 2021, Ren et al., 2019]. Safe deployment of models requires that models not only be accurate but also be robust to distribution shift [Amodei et al., 2016]. Neural networks can assign high-confidence predictions to mis-classified inputs [Guo et al., 2017, Lakshminarayanan et al., 2017] as well as test inputs that do not belong to one of the training classes [Nguyen et al., 2015]. This motivates the need for methods that can reliably detect out-of-distribution (OOD) inputs. There has been a lot of progress in detecting OOD inputs including methods based on discriminative models [Hendrycks and Gimpel, 2016, Lee et al., 2018, Liang et al., 2017, Liu et al., 2020] as well as methods based on deep generative models [Nalisnick et al., 2019, Zhang et al., 2020].
The difficulty of the OOD detection task depends on how semantically close the outliers are to the inlier classes. Winkens et al. [2020] distinguish between near-OOD tasks which are harder and far-OOD tasks which are easier, as evidenced by the difference in state-of-the-art (SOTA) for area under the receiver operating characteristic curve (AUROC). For instance, for a model trained on CIFAR-100 (which consists of classes such as mammals, fish, flowers, fruits, household devices, trees, vehicles, insects, etc), a far-OOD task would be detecting digits from the street-view house numbers (SVHN) dataset as outliers. For the same model, detecting images from the CIFAR-10 dataset (which consists of the following 10 classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck) would be considered a near-OOD task, which is more difficult as the classes are semantically similar. There has been impressive progress on far-OOD detection, for instance there are several approaches which can achieve AUROC close to 99% on CIFAR-100 (in) vs SVHN (out) task, cf. [Sastry and Oore, 2020]. However, the state-of-the-art for near-OOD detection is much lower, for instance the SOTA AUROC for CIFAR-100 (in) vs CIFAR-10 (out) task is around 85%
⇤Equal contribution.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
[Zhang et al., 2020] which is considerably lower than the SOTA for far-OOD tasks. Similar trends are observed in other modalities such as genomics where the SOTA AUROC of near-OOD detection is only 66% [Ren et al., 2019]. Improving the SOTA for these near-OOD detection tasks and closing the performance gap between near-OOD detection and far-OOD detection is one of the key challenges in ensuring the safe deployment of models.
Large-scale pre-trained transformers have led to significant accuracy improvements in multiple domains, cf. Bidirectional Encoder Representations from Transformers (BERT) for text [Devlin et al., 2018], Vision Transformers (ViT) for images [Dosovitskiy et al., 2021], Contrastive Language–Image Pre-training (CLIP) trained on image-text pairs [Radford et al., 2021]. We show that classifiers obtained by fine-tuning large-scale pre-trained transformers are significantly better at near-OOD detection. Intuitively, large-scale pre-training makes classifiers less vulnerable to shortcut learning [Geirhos et al., 2020], making these representations better suited for near-OOD detection. Figure 1 visualizes two-dimensional PCA projections of representations from residual networks (ResNet) [He et al., 2016] trained on CIFAR-100 and ViT model pre-trained on ImageNet-21k and fine-tuned on CIFAR-100; we can observe that representations obtained by fine-tuning pre-trained transformers are better suited at near-OOD detection than representations from ResNet just trained on CIFAR-100.
Motivated by real-world applications which demand very high level of OOD detection for safe deployment, we explore variants of outlier exposure to further improve OOD detection. We show that pre-trained transformers are particularly well-suited at leveraging known outliers due to their highquality representations (see Figure 1). We systematically vary the number of outlier examples per class, and show that even a handful of known outliers can significantly improve OOD detection. We refer to this setting as few-shot outlier exposure. For multi-modal pre-trained transformers, we explore a new form of outlier exposure that leverages names of outlier classes without any accompanying images, and show that this can significantly improve OOD detection for zero-shot classification.
In summary, our contributions are the following:
• We show that pre-trained transformers lead to significant improvements on near-OOD benchmarks. Concretely, we improve the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 from 85% (current SOTA) to 96% using ViT pre-trained on ImageNet-21k, and improve the AUROC on a genomics OOD detection benchmark from 66% (current SOTA) to 77% using BERT.
• We show that pre-trained transformers are well-suited for few-shot outlier exposure. With just 10 labeled examples per class, we can improve the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 to 99%, and improve the AUROC of OOD detection on genomics to 86%.
• We explore OOD detection for pre-trained multi-modal image-text transformers in the zero-shot classification setting, and show that just using the names of outlier classes as candidate text labels for CLIP, we can achieve AUROC of 94.8% on CIFAR-100 vs CIFAR-10 task. On easier far-OOD tasks such as CIFAR-{100, 10} vs SVHN, we achieve AUROC of 99.6% and 99.9% respectively.

heading: 2 Background and Related work
text: Notation We assume that we have an in-distribution dataset Din of (xin, yin) pairs where x denotes the input feature vector, and yin 2 Y in := {1, . . . ,K} denotes the class label. Let Dout denote an outof-distribution dataset of (xout, yout) pairs where yout 2 Yout := {K+1, . . . ,K+O},Yout\Y in = ;. Depending on how different Dout is from Din, we categorize the OOD detection tasks into nearOOD and far-OOD. We first study the scenario where the model is fine-tuned only on the training set Dintrain without any access to OOD data. The test set contains Dintest and Douttest for evaluating OOD performance using AUROC. Next, we explore the scenario where a small number of OOD examples are available for training, i.e. the few-shot outlier exposure setting. In this setting, the training set contains Dintrain S Doutfew shot, where |Doutfew shot| is often smaller than 100 per OOD class.

heading: 2.1 Methods for detecting OOD inputs
text: We describe a few popular techniques for detecting OOD inputs using neural networks. Maximum over softmax probabilities (MSP) A baseline method for OOD detection is to use the maximum softmax probability as the confidence score, i.e. scoremsp(x) = maxc=1,...,K p(y = c|x) [Hendrycks and Gimpel, 2016]. While being slightly worse than other techniques, its simplicity and performance make it an ideal baseline. Mahalanobis distance Lee et al. [2018] proposed to fit a Gaussian distribution to the classconditional embeddings and use the Mahalanobis distance for OOD detection. Let f(x) denote the embedding (e.g. the penultimate layer before computing the logits) of an input x. We fit a Gaussian distribution to the embeddings of the training data, computing per-class mean µc = 1Nc P i:yi=c f(xi) and a shared covariance matrix ⌃ = 1N P K c=1 P i:yi=c f(xi)
µc f(xi) µc
> . The Mahalanobis score (negative of the distance) is then computed as:
scoreMaha(x) = minc ⇣ 1 2 f(x) µc ⌃ 1 f(x) µc >⌘ .
Outlier exposure Hendrycks et al. [2018] proposed outlier exposure which leverages a large dataset of known outliers. For classification problems, the model is trained to predict uniform distribution over labels for these inputs. Thulasidasan et al. [2021] proposed to use a single outlier class as the (K + 1)th class for a (K + 1)-way classification problem. Roy et al. [2021] showed that leveraging the labels of known outliers (rather than assigning all known outliers to a single class) can further improve OOD detection performance.

heading: 2.2 Pre-training neural networks
text: Architectures using self-attention, typically based on the Transformer [Vaswani et al., 2017], often combined with large-scale pre-training directly on raw text, have been very popular for natural language processing (NLP) tasks in recent years [Devlin et al., 2018, Dai and Le, 2015, Peters et al., 2018, Howard and Ruder, 2018, Radford et al., 2018, Raffel et al., 2019]. Often followed by fine-tuning on a smaller, downstream dataset, large-scale pre-training techniques lead to highly informative embeddings that are broadly useful in natural language tasks. The advantage of the transformer architecture is its ability to scale to very large model sizes, reaching up to 1 trillion parameter mark [Fedus et al., 2021]. Large pre-trained models, such as GPT-3 [Brown et al., 2020], have shown the potential of large-scale task-agnostic pre-training in language.
The Vision Transformer (ViT) [Dosovitskiy et al., 2021] has shown that self-attention combined with large-scale pre-training is a viable strategy for vision tasks as well. The performance of ViT is comparable to other state-of-the-art models, while being more efficient to train. Its ability to quickly fine-tune to a smaller, downstream dataset, generalizing even in a few-shot regime, makes it an attractive backbone for tasks such as out-of-distribution (OOD) detection. In this paper, we use ViT pre-trained on ImageNet-21k [Ridnik et al., 2021].
Multi-modal text-image transformers such as CLIP (Contrastive Language-Image Pre-Training) [Radford et al., 2021] pre-train on 400 million (image, text) pairs from the internet to learn to predict a raw text caption from an image, and by doing so develop state-of-the-art visual representations and their natural language counterparts. Radford et al. [2021] showed that CLIP improves robustness to natural distribution shift. In this paper, we use the shared image-text embedding to introduce a new zero-shot OOD detection method (Section 5). Hendrycks et al. [2019a] show that pre-training improves OOD detection for non-transformer architectures. Self-supervised learning techniques have
also been shown to improve OOD detection; Hendrycks et al. [2019b] use rotation prediction and Winkens et al. [2020] use contrastive training to improve near-OOD detection.
Robustness of pre-trained transformers Hendrycks et al. [2020] show that pre-trained transformers improve OOD detection in NLP. Pre-trained BERT has been has used as a backbone for OOD detection in language, cf. [Liu et al., 2020]. Unlike them, we focus on vision and genomics modalities, and specifically on near-OOD detection benchmarks. Investigating the robustness of pre-trained ViT is an active research area and there are several concurrent papers exploring robustness of ViT to adversarial perturbations and common corruptions (ImageNet-C). Bhojanapalli et al. [2021] show the robustness of pre-trained transformers to input perturbations, and of transformers to layer removal. Caron et al. [2021] demonstrate many emerging properties in self-supervised ViTs, while Shao et al. [2021], Mahmood et al. [2021] show that they are more robust to adversarial perturbations, and Naseer et al. [2021] that they have less texture bias. Paul and Chen [2021], Minderer et al. [2021] show that ViT is more robust to distribution shift and natural adversarial examples [Paul and Chen, 2021], and Mao et al. [2021] propose a robust ViT. To the best of our knowledge, we are the first to show that pre-trained ViT can significantly improve near-OOD detection in vision benchmarks, and show that few-shot outlier exposure can further improve performance.

heading: 3 Near-OOD detection on image classification benchmarks

heading: 3.1 Fine-tuning the Vision Transformer
text: We use the Vision Transformer (ViT) architecture [Dosovitskiy et al., 2021] and its pretrained model checkpoints.2 The checkpoints are pre-trained on ImageNet-21k [Deng et al., 2009]. We fine-tune the full ViT architecture on a downstream task that is either the CIFAR-10 or CIFAR-100 classification problem (using a TPU in Google Colab). Once the model is finetuned, we get its pre-logit embeddings (the layer immediately preceding the final layer) for the train and test sets of CIFAR-10 and CIFAR-100 to use for OOD tasks. We use the maximum over softmax probabilities (labeled as MSP) and the Mahalanobis distance (labeled as Maha).
The results are summarized in Table 1 and Figure 2. We observe that the MSP baseline yields surprisingly good results when used on top of a large pre-trained transformer that has been fine-tuned on the in-distribution training set. The Mahalanobis distance technique improves OOD detection even further. Applying Mahalanobis distance to a pre-trained ViT fine-tuned on CIFAR-100, we achieve AUROC of 96% on CIFAR-100 vs CIFAR-10, significantly improving over the previous SOTA of
2https://github.com/google-research/vision_transformer
85% using a hybrid model [Zhang et al., 2020]. To study the effect of model architecture, we also evaluate OOD performance on another large-scale pre-trained model, Big Transfer (BiT) [Kolesnikov et al., 2019]3, as a comparison to ViT. We use the BiT-M R50x1 and R101x3 models pre-trained on ImageNet-21k, and fine-tune the full model architecture on CIFAR-10 and CIFAR-100 respectively. The results are shown in Table 1. For both directions, the AUROCs for BiT are lower than that for ViT. More importantly, BiT uses a different model architecture, ResNet, instead of a transformer, which may explains the large difference in the OOD performance. As an additional ablation, we fine-tuned the MLP-Mixer pre-trained on ImageNet-21k [Tolstikhin et al., 2021]4, a high-performance all-MLP architecture for vision, and compared its performance to the Vision Transformer (ViT) and BiT. The summary of our results can be found in Table 1. We observe that MLP-mixer outperforms BiT as well, which adds additional evidence that pre-training helps architectures such as ViT and MLP-mixer more than it helps BiT.
Due to semantic similarity between classes in CIFAR, this task is hard for humans as well.5 We also evaluated the performance of our approach on popular far-OOD benchmarks such as CIFAR-* vs SVHN and CIFAR-* vs Textures, and achieve very high AUROC values of around 98% or higher, see Table 7 in Appendix C. We mainly focus on the difficult near-OOD as this is a more challenging and realistic problem; many methods can achieve high AUROC on the easier far-OOD benchmarks, but do not perform as well in near-OOD tasks, cf. [Winkens et al., 2020, Table 1] which compares many methods on near-OOD and far-OOD tasks.
Since ViT models are typically pre-trained using a large labeled set, we ran additional ablations to understand how much of the improvement is due to supervision vs transformers. To assess the role of supervised pre-training, we compared the results to a ViT pre-trained on ImageNet-21k in a self-supervised way that does not use any labels. We took a pre-trained checkpoint from Caron et al. [2021], and fine-tuned it on CIFAR-100. The results are shown as DINO ViT-B_16 in Table 2. Since the fine-tuned test accuracy is lower for DINO ViT-B_16, we also include an ViT-B_16 that was fine-tuned for fewer steps to achieve comparable test accuracy as DINO ViT-B_16. Note that even though DINO ViT-B_16 is pre-trained without labels, the AUROC is significantly higher than the current SOTA for CIFAR-100 vs CIFAR-10. The difference between DINO ViT-B_16 vs early stopped ViT-B_16 shows the difference due to supervision during pre-training. We also experimented with larger ViT models such as ViT-L_16 and ensembles, and found that they improve AUROC of OOD detection to 98% on CIFAR-100 vs CIFAR-10 task. We found that the OOD detection is lower for ViT models with lower fine-tuned test accuracy, see Appendix C.1 for these results. Hence, we believe that better strategies for unsupervised pre-training and fine-tuning can further improve OOD detection performance. In Section 4, we explore unsupervised pre-training for transformers to improve near-OOD detection in genomics.
3https://github.com/google-research/big_transfer 4https://github.com/google-research/vision_transformer 5To approximately estimate the human performance, we performed the CIFAR-100 vs CIFAR-10 near-OOD detection ourselves. We set-up a simple GUI to randomly sample images from both CIFAR-100 and CIFAR-10 datasets, where the task was to identify images that belong to the CIFAR-10 classes. Note that this setup is easier for humans as they only have to remember 10 classes in CIFAR-10 (as opposed to the 100 classes in CIFAR-100). The estimated human performance was 96% AUROC which demonstrates the difficulty of the task. We do not claim that this is the best possible human performance, we will open source the code so that readers can estimate their OOD detection performance themselves. Further details are available in Appendix A.

heading: 3.2 Few-shot outlier exposure using ViT
text: In Section 3.1, we demonstrated that fine-tuning a pre-trained ViT model can improve near-OOD detection (with relatively simple OOD detection techniques such as MSP and Mahalanobis distance). Figure 1 shows that the representations from fine-tuned ViT models are well-clustered. This motivates few-shot outlier exposure, where we assume just a handful of known outliers (and optionally their labels). This setting is motivated by real-world applications which require high-quality OOD detection and teams are willing to collect a handful of known outlier examples (rather than just rely on modeling approaches) to improve safety. Another setting is the case where models are being continuously re-trained; once an outlier is detected, it is desirable to include that in the training corpus to encourage the model to correctly detect similar examples as outliers in the future.
The general approach is shown in Figure 3. By using the in-distribution training set Dintrain with K classes and a small number of known OOD examples from Doutfew shot with O classes, we train a simple classifier h(·) that maps an embedding vector z to a probability vector p 2 RK+O, which concatenates the in- and out-of-distribution classes. We considered two types of outlier exposure, one which assumes access to outlier labels (similar to [Roy et al., 2021]) and one where all the outlier examples are collapsed to a single (K + 1)th class (similar to [Thulasidasan et al., 2021]). For models pre-trained with labels (such as ViT), we use a linear classifier. For models that use unsupervised pre-training (e.g. genomics in Section 4), we use a shallow multi-layer perceptron (MLP) with a single hidden layer so that fine-tuning can learn discriminative features. We use the sum of the probabilities of all K in-distribution classes as the confidence score for the OOD detection task, scoreoe(x) = p(in|x) = P c=1,...,K p(y = c|x). When training the MLP h(·) with few-shot OOD examples, there could be many more examples of the in-distribution data than the small number of OOD data. We therefore oversample the OOD inputs by a factor that we calculate as (|Dintrain|/|Doutoe |)(O/K). This makes the training classes approximately balanced during training. We used a single layer MLP from scikit-learn [Pedregosa et al., 2011], batch size 200, L2 penalty of 1, learning rate 0.001 with Adam, maximum of 1,000 iterations. Algorithm 1 and Algorithm 2 describe the details of training and scoring.
Algorithm 1 Few-shot outlier exposure training
1: Input: In-distribution train set Dintrain = {(x, y)} with K classes, out-of-distribution train subset Doutfew shot = {(x, y)} with O classes, oversampling factor , a pretrained feature extractor f(·) : x ! z, a simple classification head h(·) : z ! p 2 RK+O. 2: Initialize: Initialize h(·) at random, generate random batches from Dintrain S Douttrain, oversampling Douttrain by . 3: for train_step = 1 to max_step do 4: loss = CrossEntropy(h(f(x)), y) 5: SGD update of h(·) w.r.t loss 6: end for
Algorithm 2 Few-shot outlier inference 1: Input:
In-distribution test set Dintest = {(x, y)} with K classes, out-ofdistribution test subset Douttest = {(X, y)} with O classes, a pre-trained f(·) : x ! z from inputs to embedding vectors, a trained classification head h(·) : z ! p 2 RK+O. 2: Compute scoreinoe(x),x 2 Dintest 3: Compute scoreoutoe (x),x 2 Douttest 4: Compute AUROC based on the scores.
We systemically vary the number of known outliers from 1-10 examples per OOD class and evaluate the OOD detection performance. We also show results for higher numbers of examples per class for
completeness as it illustrates how quickly the performance saturates. Figure 4 and Table 3 show the few-shot outlier exposure results for CIFAR-100 vs CIFAR-10 and CIFAR-10 vs CIFAR-100. We evaluate performance of the pre-trained transformer (without any fine-tuning on CIFAR-*) as well as a fine-tuned transformer. We observe that even with 1-10 known outliers per class, we can achieve 99% AUROC for near-OOD detection on CIFAR-100 vs CIFAR-10. Interestingly, we observe that having labels for outliers is less important when the transformer is fine-tuned on in-distribution (dashed vs solid red lines) than in the scenario where the transformer is not fine-tuned (dashed vs solid blue lines). Intuitively, the embeddings obtained by fine-tuning a pre-trained transformer are well-clustered, so just a handful of known outliers can significantly improve OOD detection, as illustrated in Figure 9 in Appendix B.

heading: 4 Near OOD detection of genomic sequences
text: We investigate OOD detection in genomics as another input modality for near-OOD detection. Ren et al. [2019] proposed a benchmark dataset6 for OOD detection in genomics, motivated by the real-world problem of bacteria identification based on genomic sequences. Real bacteria sequencing data can contain approximately 60-80% of sequences from unknown classes that have not been studied before. Hence, a classifier trained on all known classes so far will be inevitably asked to predict on genomes that do not belong to one of the known classes. Since different bacteria classes are discovered gradually over the years, Ren et al. [2019] use a set of 10 bacteria classes that were discovered before the year 2011 as in-distribution classes, a set of 60 bacteria classes discovered between 2011-2016 as the validation OOD, and a set of 60 different bacteria classes discovered after 2016 as the test OOD. The training set only contains genomic sequences of in-distribution classes. The validation and test sets contain sequences from both in-distribution and OOD classes. The genomic sequence is of fixed length of 250 base pairs, composed by characters of A, C, G, T. In 6https://www.tensorflow.org/datasets/catalog/genomics_ood
the previous work, 1-dimensional Convolutional Neural Networks (1D CNN) were used to build the classifier for the 10 in-distributional classes, and the maximum of softmax probabilities (MSP) and Mahalanobis distance were used for OOD detection. The best AUROC for MSP was only 66.14%, and 62.41% for Mahalanobis distance [Ren et al., 2019].
Similar to the previous section, we explore the usefulness of pre-trained transformers and fine-tuning for near-OOD detection. Unsupervised pre-training and fine-tuning approach has been applied to several bio-informatic problems, such as protein function prediction [Elnaggar et al., 2020, Dohan et al., 2021, Littmann et al., 2021], protein structure prediction [Rives et al., 2021], and predicting promoters, splice sites and transcription factor binding sites [Ji et al., 2020], but it has not yet been studied how pre-training could help near-OOD detection.
Unsupervised BERT pre-training and supervised fine-tuning We first pre-train the transformer model in an unsupervised fashion as in BERT to capture biologically relevant properties, following Dohan et al. [2021]. For unlabeled in-distribution sequences in the training set, we randomly mask the characters in the sequence at the rate of 0.15, feed the masked sequence into transformer-based model of 8 heads and 6 layers and embedding dimension 512, and predict the masked characters. To boost the performance, we add the unlabeled validation data to the training set for pre-training. In the fine-tuning stage, we load the pre-trained transformer model, mean pool the embeddings over the positions, and add a single linear projection classification head for 10 in-distribution classes on top of the embeddings. The setup is shown in Figure 5a. All the parameters in the model including those in the pre-trained transformer and those in the classification head are fine-tuned using the labeled training data. The model is pre-trained for 300,000 steps using learning rate of 0.001 and Adam optimizer [Kingma and Ba, 2014] on TPU, and the accuracy for predicting the masked token is 48.35%. The model is fine-tuned for 100,000 steps at the learning rate of 0.0001, and the classification accuracy is 89.84%. We use the validation in-distribution and validation OOD data to select the best model checkpoint for each of the two methods and evaluate on test set.
The results are reported in Table 4. It can be seen that using the approach of pre-training transformer and fine-tuning, the OOD detection performance is significantly improved, from 64.75% to 77.49%
for Mahalanobis distance, and from 65.84% to 73.53% for MSP. The in-distribution accuracy also improves a bit, from 85.93% to 89.84%. We also study the relationship between the genetic distance and the AUROC of OOD detection for the 60 test OOD classes. We compute the genetic distance using the popular alignment-free method dS2 which is based on the similarity between the word frequencies of the two genomes [Ren et al., 2018, Reinert et al., 2009]. Studies have shown that this genetic distance reflects true evolutionary distances [Chan et al., 2014, Bernard et al., 2016]. For each of the 60 OOD test classes, we use the minimum genetic distance between this OOD class to any of the 10 in-distribution classes as the final distance measure. Figure 5 shows the AUROC and the minimum distance for each of the 60 OOD classes. We expect the AUROC is higher as the distance is greater. Using the baseline 1D CNN model, we did not see obvious correlation between the AUROC and the minimum distance, with r2 = 0.0000, based on Mahalanobis distance. The AUROC based on MSP method has positive correlation to the minimum distance, with r2 = 0.1190. After we use the pre-trained+fine-tuned transformer, both MSP and Mahalanobis distance methods have significantly higher AUROC overall, and the positive correlation between the minimum distance and the AUROC is more prominent than for the baseline model.
Few-shot outlier exposure Given that pre-trained and fine-tuned model improves the OOD performance, we next explore the idea of few shot outlier exposure to further boost the performance. We randomly select 1, 2, 5, 10, 100 examples per test OOD class and add them to the training set respectively. For each input x in the training set, we extract its corresponding embedding vector z from the above pre-trained and fine-tuned model (or alternatively the model without fine-tuning). We construct a single layer perceptron network of 1024 units for classifying each individual to in-distribution classes and OOD classes, as shown in Figure 3. At inference time, we use the sum of the probability of in-distribution classes as the final confidence score for OOD detection. Additionally, we also tried the idea of collapsing all OOD classes into one single class (as in [Thulasidasan et al., 2021]) for comparison. The model is
trained for 10,000 steps with the learning rate of 0.001. The best model checkpoint is selected based on the highest AUROC on a small set of validation dataset disjoint from the test set.
Results are shown in Figure 6. We observe that exposing to just a small number of OOD examples significantly improves the OOD performance, increasing AUROC from 76.73% to 88.48%. As expected, using the embeddings from the fine-tuned model (blue lines) is better than that from the model without fine-tuning (purple lines). Also, using the outlier labels (purple solid line) has a slightly better performance than collapsing the OOD classes into a single class (purple dashed line) using the pre-trained embeddings without fine-tuning.

heading: 5 Using candidate labels with multi-modal text-image models such as CLIP
text: Multi-modal transformers such as CLIP [Radford et al., 2021], which are pre-trained on image-text pairs, have been shown to perform well on zero-shot classification tasks. We show that such multimodal transformers open the door to new forms of outlier exposure which can significantly improve out-of-distribution (OOD) detection in the zero-shot classification setting. Our goal is to show that multi-modal transformers can leverage a weaker form of outlier exposure than the few-shot outlier exposure assumption in previous sections, and improve their safety for zero-shot classification.
We use the pre-trained CLIP model7 (specifically ViT-B/32) that was trained on 400 million (text, image) pairs from the internet. Its image encoder can map an image I into an embedding vector zimage(I), while its text encoder can do the same for a string T as ztext(T ). By choosing a set of D candidate labels for an image, the similarity between the embedding of the candidate label Ti and an image I can be used as the ith component of the image’s embedding vector z as zi = ztext(Ti) · zimage(I). 7https://github.com/openai/CLIP
Zero-shot outlier exposure In the zero-shot classification setting, the candidate labels are chosen to describe the semantic content of the in-distribution classes (e.g. names of the classes). We propose to include the candidate labels related to the out-of-distribution classes, and utilize this knowledge as a very weak form of outlier exposure in multi-modal models. This could be relevant in applications, where we might not actually have any outlier images for fine-tuning but we might know the names or descriptions of outlier classes.
Our proposed procedure is shown in Figure 7. We choose two groups of candidate labels, indistribution and out-of-distribution labels (e.g. CIFAR-100 and CIFAR-10 class names). We produce an embedding vector z for each image I , apply softmax to get probabilities as p = softmax(z). Those get split to p(in|x) = P i2in pi, and p(out|x) = P i2out pi, where p(in|x) + p(out|x) = 1. Similar to Figure 3, we use scoreoe(x) = p(in|x) as the confidence score. By choosing the candidate labels to represent the in-distribution and OOD dataset we would like to distinguish (e.g. CIFAR-100 and 10), we can get a very informative score that leads to AUROC above previous SOTA, despite no exposure to the training set of the in-distribution (zero-shot). Our results are shown in Table 5, with additional results in Table 10 of Appendix C.2.

heading: 6 Conclusion
text: We focus on the challenging problem of near-OOD detection. We show that fine-tuning large-scale pre-trained transformers and using few-shot outlier exposure can significantly improve the SOTA. On the CIFAR-100 vs CIFAR-10 visual OOD detection benchmark, we improve the SOTA AUROC from 85% to 96% (without outlier exposure) and 99% (with outlier exposure), essentially closing the gap between SOTA and the ideal performance. On a challenging genomics benchmark, we improve the SOTA from 66% to 77% using BERT (without outlier exposure) and 88% (with outlier exposure). We also show that multi-modal pre-trained transformers open the door to new, weaker forms of outlier exposure which only use names of OOD inputs; we apply this to CLIP and achieve AUROC of 94.7% in the zero-shot classification setting. We believe that our findings will be of interest to the research community (and inspire the creation of harder near-OOD benchmarks) as well as practitioners working on safety-critical applications.

heading: Acknowledgements
text: We thank Abhijit Guha Roy, Jim Winkens, Jeremiah Liu, Lucas Beyer and the anonymous reviewers for helpful feedback. We thank David Dohan and Andreea Gane for the helpful advice on BERT genomics model pre-training. We thank Basil Mustafa for providing the BiT model checkpoints. We thank Matthias Minderer for his helpful advice on ViT. We thank Winston Pouse for useful discussions on human performance.

Review summaries:
This paper studies the few-shot outlier detection setting where a few examples from outlier classes are provided to the ODD detector.  The main motivation behind this work are situations that require high level of ODD detection. It is shown that classifiers trained by fine-tuning large-scale pre-trained transformers are significantly better at near-ODD detection. Simulations are performed on well-known data sets for visual classification and genomic sequences. Performance evaluation are reported in term of AUROC scores. However, it would have been more convenient to report results in terms of FPR at 95% scores which allow to better understand if the performance gains with respect to similar methods are relevant. The reported gains are not very significant but consistent.  

* The paper presents detailed experiments which show that pre-trained large scale transformers significantly improves OOD detection performance. * The paper especially focuses on the case where in-distribution (InD) and OOD samples are close to each other (near OOD) (e.g. when CIFAR100 is InD and CIFAR10 is OOD, SoTA AUROC is improved from 85% to 96% ). * It is demonstrated that transformers can yield better well-separated representations for each class on InD dataset compared to ResNet without even fine tuning on InD samples. Separation of the representations become even better after fine-tuning in InD dataset (see Fig. 1). This motivates that transformers can be better for OOD detection compared to ResNet-like architectures and improves the SoTA. * The paper presents experiments in different settings on multiple datasets: * First, OOD detection performance between CIFAR10 <-> CIFAR100 is evaluated. The results demonstrate that fine-tuning transformers significantly improves SoTA OOD detection performance. Additional experiments are presented where it is assumed that a few examples from each class of target OOD dataset is available (called outlier exposure). This leads to further boost in OOD detection performance. * The second experiments are performed on a genomics dataset where InD dataset consist of 10 known bacteria classes and OOD dataset consist of another 60 bacteria classes which are not seen during training. The results are consistent with the CIFAR10-CIFAR100 experiments.  * Finally, experiments on multimodal text-image models are presented where candidate labels are used for OOD detection. The results show that it leads to significant performance improvement.