Exploring the Limits of Out-of-Distribution Detection
Authors: Stanislav Fort
Jie Ren
Balaji Lakshminarayanan
Venue: NeurIPS 2021 Poster (2021)
Keywords: out-of-distribution detection
outlier exposure

Abstract: Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to more than 96% using Vision Transformers pre-trained on ImageNet21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% to 77% using transformer and unsupervised pre-training.  To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that  pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR-10  can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. For multi-modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard OOD benchmark tasks. 

Exploring the Limits of Out-of-Distribution Detection
Stanislav Fort" Stanford University sfort1@stanford.edt
Jie Ren" Google Research, Brain Team jjen@googge..co com
Balaji Lakshminarayanan Google Research, Brain Team balajiln@googl com
Abstract
Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers signifi- cantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 tection, we improve the AUROC from 85% (current SOTA) to 96% using Vision Transformers pre-trained ImageNet-21k. On challenging genomies OOD de- tection benchmark, we improve the AUROC from 66% to 77% using transformers and unsupervised pre-training. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained transformers are particularly well-suited for outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR- 10 can be improved 98.7% with just image per OOD class, and 99.46% with 10 images per OOD class. For multi-mon modal image-text pre-trained transformers such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information without any accompanying images, and show that this outperforms previous SOTA on standard vision OOD benchmark tasks.
CIFAR-I0 OOD de-
Introduction
Deep neural networks are increasingly used in high-stakes applications such as healthcare [Roy et al., 2021. Ren et al., 2019]. Safe deployment of models requires that models not only be accurate but also be robust to distribution shift [Amodei et al.. 2016]. Neural networks can assign high-confidence predictions to mis-classified inputs [Guo al., 2017, Lakshminarayanan et al., 2017] as well as test inputs that do not belong to one of the training classes [Nguyen et al., 2015). This motivates the need for methods that can reliably detect -iitt---iittiittioon (OOD) inputs. There has been a lot of progress in detecting OOD inputs including methods based discriminative models [Hendrycks and Gimpel, 2016, Lee et al., 2018, Liang et al., 2017, Liu et al., 2020] as well as methods based on deep generative models INalisnick et al., 2019, Zhang et al., 2020].
The difficulty of the OOD detection task depends on how semantically close the outliers are to the inlier classes. Winkens et al. [2020] distinguish between near-OOD tasks which are harder and far-OOD tasks which are easier, evidenced by the difference in state-of-the-an (SOTA) for area under the receiver operating characteristic curve (AUROC). For instance, for a model trained on CIFAR-100 (which consists of classes such as mammals, fish, flowers, fruits, household devices, trees, vehicles, insects, etc), far-OOD task would be detecting digits from the street-view house numbers (SVHN) dataset as outliers. For the same model, detecting images from the CIFAR-10 10 dataset (which consists of the following 10 classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck) would be considered a near-OOD task, which is more difficult as the classes are semantically similar. There has been impressive progress far-OOD detection, for instance there are several approaches which can achieve AUROC close to 99% on CIFAR- (in) vs SVHN (out) task, cf. [Sastry and Oore, 2020]. However, the state-of-the-an for near-OOD detection is much lower, for instance the SOTA AUROC for CIFAR- 100 (in) vs CIFAR- 10 (out) task is around 85%
Equul contribution.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

ResNet trained on CIFAR-100
VIT without finetuning
VIT finetuned on CIFAR-100
630
dip class
des class
dis cless
die care
In dis clans
ditt dars
000 clsss
OCD clsss
0C0 class
210
PCA dlim 2
PCA di
300
270
PCA
270
90
PCA
PCA dim
A two-dimensional PCA projection of the space of embedding vectors for 3 models, with examples of 2 in-distribution (from CIFAR- 100) and I out-of-distributee class (from CIFAR-10). The color coding shows the Mahalanobis outlier score, while the points are projections of embeddings of members of the in-distribution CIFAR- 100 classes "sunflowers" (black plus signs) and "turtle" (yellow crosses), and the OOD CIFAR-10 class "automobile" (red circles). The left panel shows ResNet-20 trained on CIFAR-100, which assigns low Mahalanobis distance to OOD inputs and leads overlapping clusters of class embeddings. The ViT pre-trained on ImageNet-21k (middle panel) is able to distinguish classes from each other well, but does not lead to well-separated outlier scores. ViT fine-tuned on CIFAR-I00 (right panel) great at clustering embeddings based on class, as well as assigning high Mahalanobis distance to OOD inputs (red).
Figure 1:
(Zhang al., 2020] which is considerably lower than the SOTA for For-OOD tasks. Similar trends observed in other modalities such as genomics where the SOTA AUROC of near-OOD detection is only 66% [Ren al., 2019]. Improving the SOTA for these near-OOD detection tasks and closing the performance gap between near-OOD detection and far-OOD detection is one of the key challenges in ensuring the safe deployment of models.
Large-scale pre-trained transformers have led to significant accuracy improvements in multiple domains, cf. Bidirectional Encoder Representations from Transformers (BERT) for text [Devlin et al., 2018). Vision Transformers (ViT) for images [Dosovitskiy et al., 2021], Contrastive Language-Image Pre-training (CLIP) trained on image-text pairs [Radford et al., 2021]. We show that classifiers obtained by fine-tuning large-scale pre-trained transformers are significantly better at near-OO1) detection. Intuitively, large-scale pre- pre-training makes classifiers less vulnerable to shortcut learning [Geirhos et al., 2020], making these representations better suited for near-OOD detection. Figure visualizes two-dimensional PCA projections of representations from residual networks (ResNet) [He et al., 2016] trained on CIFAR- 100 and ViT model pre-trained on ImageNet-22 Net-21k k and fine-tuned on CIFAR-100; 100; we can observe that representations obtained by fine-tuning pre-trained transformers are better suited at near-OOD detection than representations from ResNet just trained on CIFAR- 100.
Motivated by real-world applications which demand very high level of OOD detection for safe deployment, we explore variants of outlier exposure to further improve OOD detection. We show that pre-trained transformers are particularly well-suited at leveraging known outliers due to their high- quality representations (see Figure 1). We systematically vary the number of outlier examples per class, and show that even handful of known outliers can significantly improve OOD detection. We refer to this setting as few-shot outlier exposure. For multi-modal pre-trained transformers, we explore a new form of outlier exposure that leverages names of outlier classes without any accompanying images, and show that this can significantly improve OOD detection for zero-shot classification.
In summary, contributions are the following:
We show that pre-trained transformers lead to significant improvements near-OOD benchmarks. Concretely, we improve the AUROC of OOD detection on CIFAR- vs CIFAR-10 from 85% (current SOTA) to 96% using ViT pre-trained on ImageNet-2 and improve the AUROC genomics OOD detection benchmark from 66% (current SOTA) to 77% using BERT.
We show that pre-trained transformers are well-suited for few-shot outlier exposure. With just 10 labeled examples per class, we can improve the AUROC of OOD detection on CIFAR-100 vs CIFAR- 10 to 99%, and improve the AUROC of OOD detection on genomics to 86%.
We explore OOD detection for pre-trained multi-modal image-text transformers the zero-shot classification setting, and show that just using the names of outlier classes as candidate text labels for CLIP, we can achieve AUROC of 94.8% on CIFAR-100 vs CIFAR-10 task. On easier far-OOD tasks such as CIFAR- (100, 10) vs SVHN, we achieve AUROC of 99.6% and 99.9% respectively.

---

2 Background and Related work
Notation We assume that we have in-distribution dataset Din of (æ1 y pairs where æ denotes the input feature vector, and y E e := {1..... ......K K} denotes the class label. Let Donrt denote an out- of-distribution dataset of (æ yf pairs where y y == {K +1,-- ................................ K+0), y nn W. Depending on how different Dontt is from Din, we categorize the OOD detection tasks into near- OOD and far-OOD. We first study the scenario where the model is fine-tuned only on the training set Din train without any access to OOD data. The test set contains Diest and Dost for evaluating OOD) performance using AUROC. Next, we explore the scenario where small number of OOD examples are available for training, i.e. the few-shot outlier exposure setting. In this setting, the training set contains Din few where D shot! is often smaller than 100 per OOD class.
2.1 Methods for detecting OOD inputs
We describe few popular techniques for detecting OOD inputs using neural networks.
Maximum over softmax probabilities (MSP) A baseline method for OOD detection is to use the maximum softmax probability as the confidence score, i.e. scoremsp (a:) = max.==,...ff ply = c/ [Hendrycks and Gimpel, 2016]. While being slightly worse than other techniques, its simplicity and performance make it an ideal baseline.
Mahalanobis distance Lee al. [2018] proposed to fit Gaussian distribution the class- conditional embeddings and use the Mahalanobis distance for OOD detection. Let [(x) de- note the embedding (e.g. the penultimate layer before computing the logits) of an input æ. We fit Gaussian distribution to the embeddings of the training data, computing per-class
mean Me N. Ei (ax and shared covariance matrix Σ N K Einy (f(x.)
me) (f(x,) mo) The Mahalanobis score (negative of the distance) is then computed as: scoreMaha (x) = min, (f(((z) - -ame))--))--- -f(x))=)))
Outlier exposure Hendrycks et al. [2018] proposed outlier exposure which leverages large dataset of known outliers. For classification problems, the model is trained to predict uniform distribution over labels for these inputs. Thulasidasan et al. [2021] proposed to use a single outlier class as the (K + 1 class for (K + 1)yy classification problem. Roy et al. [2021] showed that leveraging the labels of known outliers (rather than assigning all known outliers to single class) can further improve OOD detection performance.
2.2 Pre-training neural networks
Architectures using self-attention, typically based on the Transformer [Vaswani et al., 2017], often combined with large-scale pre-training directly on raw text, have been very popular for natural language processing (NIP) tasks in recent years [Devlin et al., 2018, Dai and Le, 2015, Peters et al.. 2018, Howard and Ruder, 2018, Radford et al., 2018, Raffel et al., 2019]. Often followed by fine-tuning on a smaller, downstream dataset, large-scale pre-training techniques Icad to highly informative embeddings that are broadly useful in natural language tasks. The advantage of the transformer architecture is its ability to scale to very large model sizes, reaching up to trillion parameter mark [Fedus et al., 2021]. Large pre-trained models, such GPT-3 [Brown et al., 2020], have shown the potential of large-scale task-agnostic pre-training in language.
The Vision Transformer (ViT) (Dosovitskiy et al., 2021] has shown that self-attention combined with large-scale pre-training a viable strategy for vision tasks as well. The performance of ViT is comparable to other state-of-the-art models, while being more efficient to train. Its ability to quickly fine-tune to a smaller, downstream datasct, generalizing even in a sew-shot regime, makes an attractive backbone for tasks such -uttof-distribution (OOD) detection. In this paper, use ViT prentrained on ImageNet-22 [Ridnik et al., 2021].
Multi-modal text-image transformers such as CLIP (Contrastive Language- Image Pre- Fre-Training) [Radford et al., 2021] pre-train on 400 million (image, text) pairs from the internet to learn predict a raw text caption from an image, and by doing so develop state-of-the-a art visual representations and their natural language counterparts. Radford et al. [2021] showed that CLIP improves robustness to natural distribution shift. In this paper, we use the shared image-text embedding to introduce new zero-shot OOD detection method (Section 5). Hendrycks et al. [2019a] 9a] show that pre-training improves OOD detection for non-transformer architectures. Self-supervised learning techniques have
3

---

also been shown to improve OOD detection; Hendrycks et al. [2019b] use rotation prediction and Winkens al. [2020] use contrastive training to improve near-OOD detection.
Robustness of pre-trained transformers Hendrycks et al. [2020] show that pre-trained transform- ers improve OOD detection in NLP. Pre-trained BERT has been has used as a backbone for OOD detection in language, cf. [Liu et al., 2020]. Unlike them, focus vision and genomics modalities, and specifically on near- near-OOD detection benchmarks. Investigating the robustness of pre-trained ViT is an active research area and there are several concurrent papers exploring robustness of VIT to adversarial perturbations and common corruptions (ImageNet-C). Bhojanapalli et al. [2021] show the robustness of pre-trained transformers to input perturbations, and of transformers to layer removal. Caron et al. [2021] demonstrate many emerging properties self-supervised ViTs, while Shao et al. [2021]. Mahmood et al. [2021] show that they are more robust adversarial perturbations, and Naseer et al. [2021] that they have less texture bias. Paul and Chen [2021]. Minderer et al. [2021] show that VIT is more robust to distribution shift and natural adversarial examples [Paul and Chen, 2021], and Mao et al. [2021] propose a robust ViT. To the best of our knowledge, we are the first to show that pre-trained ViT can significantly improve near- -OD detection in vision benchmarks, and show that few-shot outlier exposure can further improve performance.
3 Near-OOD detection on image classification benchmarks
3.1 Fine-tuning the Vision Transformer
We the Vision Transformer (ViT) archi- tecture [Dosovitskiy et al., 2021] and its pre- trained model checkpoints. The checkpoints are pre-trained on ImageNet-21k [Deng et al.,
100
CIFAR-100 CIFAR-10
100
CIFAR-10 - CIFAR-100
AUROC (%)
95
Juman
95.52%
AUROC (1)
47.7%%
08
98
85.6%
2009]. We fine-tune the full VIT architecture on a downstream task that is either the CIFAR-IE 10 or CIFAR- 100 classification problem (using TPU in Google Colab). Once the model is fine- tuned, we get its pre-logit embeddings (the layer immediately preceding the final layer) for the train and test sets of CIFAR-IO and CIFAR-IO0 to use for OOD tasks. We the maximum over softmax probabilities (labeled as MSP) and the Mahalanobis distance (labeled Maha).
80
95
soedets for open
Hydori mosdals openy exeent
NyT foned
Eay
Regm swree crfak
Respy tureed
Figure 2: Left: CIFAR-100 vs CIFAR-I0 OOD AUROC for previous ttate-of-the-eee Zhang et al. [2020], our fine-tuned ViT with two different back- bones (ViT-B 16 and R50+VIT-B __6)). Right: CIFAR-10 vs CIFAR- 100 OOD task.
Table 1: ImageNet-21k pre-trained V
fine-tuned the in-distribution training set.
fine-tuned
In- distribution
Out- distribution
Mahalanobis AUROC
MSP AUROC
Model
test accuracy
BiT-M R50x1
CIFAR- 100
87.01%
CIFAR-IE
81.71%
81.15%
BÎT-M 1101x3
CIFAR-
91.55%
CIFAR-10 10
90.10%
83.69%
ViT-B_16 16
CIFAR- 100
90.95%
CIFAR- 10
95.53%
91.89%
R50+viT-B
CIFAR-100
91.71%
CIFAR-10
96.23%
92.08%
MLP-Mixer-B.
CIFAR-100
90.40%
CIFAR- 10
95.31%
90.22%
BiT-M R50x1
CIFAR-10
97.47%
CIFAR- 100
95.52%
85.87%
BiT-M 1101x3
CI
97.36%
CIFAR-100 100
94.55%
85.34%
VIT-B 16
CIFAR-10
98.10%
CIFAR-
98.42%
97.68%
R50+V _16
CIFAR-10
98.70%
CIFAR-100
98.52%
97.75%
MLP-Mixer-B_16
CIFAR-10 97.58%
CIFAR-100
97.85%
96.28%
The results are summarized in Table and Figure 2. We observe that the MSP baseline yields surprisingly good results when used on top of large pre-trained transformer that has been fine-tuned on the in-distribution training set. The Mahalanobis distance technique improves OOD detection even further. Applying Mahalanobis distance to a pre-trained ViT fine-tuned on CIFAR-100. we achieve AUROC of 96% on CIFAR-100 vs CIFAR- 10, significantly improving over the previous SOTA of
Pnttps://githun http.////ooooooeeeeeerrr//iiaaaa transffrrrrrr

---

85% using a hybrid model [Zhang et al., 2020]. To study the effect of model architecture, we also evaluate OOD performance on another large-scale pre-trained model. Big Transfer (BiT) [Kolesnikov et al., 2019]², as a comparison to VIT. We use the BiT-M R50x I and R 10113 models pre-trained on ImageNet-21K and fine-tune the full model architecture on CIFAR-I 10 and CIFAR-I00 respectively. The results are shown in Table 1. For both directions, the AUROCs for BiT are lower than that for ViT. More importantly, BiT uses a different model architecture, ResNet, instead may explains the large difference in the OOD performance. As an additional ablation, we fine-tuned the MLP-Mixer pre-trained on ImageNet-21K [Tolstikhin et al., 2021J',
transformer, which
perhh-peffommncc all-MLP architecture for vision, and compared its performance to the Vision Transformer (ViT) and BiT. The summary of our results can be found in Table 1. We observe that MLP-mixe outperforms BiT as well, which adds additional evidence that pre-training helps architectures such ViT and MLP-mixer more than it helps BiT.
Due to semantic similarity between classes in CIFAR, this task is hard for humans as well. We also evaluated the performance of our approach on popular far- farOOOD benchmarks such as CIFAR-* vs SVHN and CIFAR-* # vs Textures, and achieve very high AUROC values of around 98% higher, see Table 7 in Appendix C. We mainly focus on the difficult near-OOD as this is a more challenging and realistic problem; many methods can achieve high AUROC on the easier far-OOD benchmarks, but do not perform as well in near-OOD tasks, cf. [Winkens et al., 2020, Table 1] which compares many methods near-OOD and far-OOD tasks.
Since ViT models are typically pre-trained using a large labeled set, we ran additional ablations to understand how much of the improvement is due to supervision vs transformers. To assess the role of supervised pre-training, we compared the results to a VIT pre-trained on ImageNet-21k in a self- seff-supervised way that does not use any labels. We took a pre-trained checkpoint from Caron et al. [2021], and fine-tuned it on CIFAR-100. The results are shown as DINO VIT-B 16 in Table 2. Since the fine-tuned test accuracy is lower for DINO ViT-B_16 we also include an VIT-B 16 that was fine-tuned for fewer steps to achieve comparable test accuracy as DINO ViT-B_ 16. Note that even though DINO ViT-B 16 is pre-trained without labels, the AUROC is significantly higher than the current SOTA for CIFAR- 100 vs CIFAR-10. The difference between DINO ViT-B __16 vs early stopped ViT-B 16 shows the difference due supervision during pre- mre-tranning. We also experimented with larger ViT models such as ViT-L 16 and ensembles, and found that they improve AUROC of OOD detection to 98% on CIFAR-100 vs CIFAR-10 task. We found that the OOD) detection is lower for ViT models with lower fine-tuned test accuracy, see Appendix C. for these results. Hence, we believe that better strategies for unsupervised pre-training and fine-tuning can further improve OOD detection performance. In Section 4, we explore unsupervised pre-training for transformers to improve near-OOD detection in genomies.
Table 2: Additional ablations to measure the effect of supervised pre-training. indicates self- supervised pre-training without labels.
fine-tuned test accuracy
In- distribution
Out- distribution
Mahalanobis AUROC
MSP AUROC
Model
DINO ViT-B 16"
CIFAR- 100
88.95%
CIFAR- 10
88.78%
81.25%
ViT-B _ (early stop)
CIFAR- 100
88.71%
CIFAR-10
93.05%
88.82%
ViT-B 16
CIFAR-100
90.95%
CIFAR-10
95.53%
91.89%
R50+ViT-B_IC
CIFAR-100
91.71% %
CIFAR-10
96.23%
92.08%
VYT-L_16 ViT ensemble
CIFAR-100
94.73%
CIFAR-10 10
97.98%
94.28%
CIFAR-
CIFAR-10 10
98.11%
95.15%
"https://githuu.comm// com./google-rreeeacch/bbt http:rccccbbbigt.tteeeeeeee transfer "nttps://github. com/ googleerreeearcchviiiii. "To approximately estimate the human performance, we performed the CIFAR-100 vs CIFAR-10 near-OOD detection ourselves. We set-up simple GUI to randomly sample images from both CIFAR-100 and CIFAR-10 datasets, where the task was to identify images that belong to the CIFAR-I0 classes. Note that this setup is casier for humans as they only have to remember 10 classes CIFAR-I0 (as opposed to the 100 classes in CIFAR- 100). The estimated human performance was 96% AUROC which demonstrates the difficulty of the task. We do not claim that this is the best possible human performance, we will open source the code so that readers estimate their OOD detection performance themselves. Further details available in Appendix A.
http://iiiiootttaaaasttteeeeeee transformer
5

---

3.2 Few-shot outlier exposure using VIT
In Section 3.1, we demonstrated that fine-tuning a pre-trained ViT model can improve near-OOD detection (with relatively simple OOD detection techniques such as MSP and Mahalanobis distance). Figure I shows that the representations from fine-tuned ViT models are well-clustered. This motivates few-shot outlier exposure, where we assume just handful of known outliers (and optionally their labels). This setting is motivated by real-world applications which require high-quality OOD detection and teams are willing to collect a handful of known outlier examples (rather than just rely on modeling approaches) to improve safety. Another setting is the case where models are being continuously re-trained; once an outlier is detected, is desirable to include that in the training corpus to encourage the model to correctly detect similar examples as outliers in the future.
p(e1(x) p(cz(x)
Sum
p X
In-distribution
:
p(ck(x)
Input
Embedding
Softmax
Pretrained Encoder
Simple Classifier
p|cK+1|x
01
12
Sum
00
p(out|x)
Few-shot OOD
PicK
Backpropagation
Figure 3: Few-shot outlier exposure with pre-trained transformers. The OOD samples are used to fine-tune simple classifier (linear classifier for ViT which uses supervised pre-training, and shallow MLP with one hidden layer for genomics which uses unsupervised pre-trane pre-training). We use the in-distribution classes addition to multiple OOD classes (when labels available), or single OOD class. The confidence score is the sum of probabilities corresponding to the in-distribution classes
The general approach is shown in Figure 3. By using the in-distribution training set Dirain with K classes and a small number of known OOD examples from Doutt Dewos with O classes, we train simple classifier h(-) that maps an embedding vector = to a probability vector p E R RK+ which concatenates the in- and out-of-distribution classes. We considered two types of outlier exposure, one which assumes access to outlier labels (similar to [Roy et al., 2021]) and one where all the outlier examples are collapsed to single (K + 1)th class (similar to [Thulasidasan et al., 2021]). For models pre-trained with labels (such as ViT), we use a linear classifier. For models that use unsupervised pre-training (e.g. genomics in Section 4), we use a shallow multi-layer perceptron (MLP) with a single hidden layer so that fine-tuning can learn discriminative features. We use the sum of the probabilities of all K in-distribution classes as the confidence score for the OOD detection task, score (x) = p(in(z)) = E==I....K p(y = c/z). When training the MLP h(-) with few-shot OOD examples, there could be many more examples of the in-distribution data than the small number of OOD data. We therefore oversample the OOD inputs by factor that we calculate as [[Dirail///PPe oe (O ((OK)) /.. This makes the training classes approximately balanced during training. We used single layer MLP from scikit-leand [Pedregosa et al., 2011] IJ. batch size 200, L2 penalty of 1, learning rate 0.001 with Adam, maximum of 1,000 iterations. Algorithm and Algorithm describe the details of training and scoring.
Algorithm Few-shot outlier exposure training
Algorithm 2 Few-shot outlier inference
1: Input:
In-distribution train set Din ((x, w)} with K classes, out-of-distrriutioo train subset Dout fhoo ()) with O classes, oversampling factor T, a pre- trained feature extractor f-) : x + z, a simple classi- fication head ((-): z -> p E IK+
I: Input:
In-distribution test set Din {xx v)} with K classes, out-of- distribution test subset Dynt ((X,v) with O classes, a pre-trained f()) : - from inputs to embed- ding vectors, a trained classification head h(-) = +
2: Initialize: Initialize h(-) random, generate random
batches from Dir U Diain oversampling Dout by T.
E R O
3: for train train_step__e = to max step do
4: loss = CrossEntropy(h(
2: Compute score (w), x E Din
5: SGD update of 16(-) w.r.t loss
3: Compute score (())
(z)) x E Delit
6: end for
4: Compute AUROC based on the scores.
We systemically vary the number of known outliers from 1-10 examples per OOD class and evaluate
the OOD detection performance. We also show results for higher numbers of examples per class for

---

completeness as it illustrates how quickly the performance saturates. Figure 4 and Table 3 show the few-shot outlier exposure results for CIFAR- 100 vs CIFAR- 10 and CIFAR-I0 vs CIFAR-I00. We evaluate performance of the pre-trained transformer (without any fine-tuning on CIFAR-*) as well as a fine-tuned transformer. We observe that even with 1-10 known outliers per class, we can achieve 99% AUROC for near-OOD detection on CIFAR-100 vs CIFAR-10. Interestingly, we observe that having labels for outliers is less important when the transformer is fine-tuned on in-distribution (dashed vs solid red lines) than in the scenario where the transformer is not fine-tuned (dashed vs solid blue lines). Intuitively, the embeddings obtained by fine-tuning a pre-trained transformer are well-clustered, just handful of known outliers can significantly improve OOD detection, as illustrated in Figure in Appendix B.
In-distribution CIFAR-100 OOD CIFAR-10
In-distribution CIFAR-10 OOD CIFAR-100 CIFAR-100 outlier exposure
CIFAR-10 outlier exposure
100
99.02%
99.7%
100
98. 96%
11%
98
98
7%
99.52%
96.23%
98-
98
98.52%
96-
AUROC (%)
95.9%
AUROC (%)
95.9%
94-
use cutlier label
35.1%
turre CIFAR-100
ignore outleer lube!
92-
no finetuning
94
90
Gat cuilier
no finetuning
human performance
ighare
92
luned en CFA--30
tured OFAR-10 use butller label
ighore label
Hybrid models for open sets
human performance
88
Hybrid madels
ne finetaning label
tuned on CIFAR-10 Mahalanobis
86
Mahalaneate
90
ignore outler late
2
1
2 10
100
1k
4k
0 1
3
10
40
100
400
Outlier examples per CIFAR-10 class
Outlier examples per CIFAR-100 class
Figure 4: The effect of few-shot outlier exposure and fine-tuning on CIFAR- 100 vs CIFAR-10 (left) and CIFAR-10 vs CIFAR-100 (right) using R50+ 16 pre-trained on ImageNet-21k. Fine-tuning on ni-distribution (red) prior to outlier exposure outperforms no tine-tuning (blue).
Table ImageNet-211 pre-trained ViT (optionally fine-tuned on in-distribution), with an additional final layer that was trained using the in-distribution train set and small number of examples of the OOD train set (including the OOD class labels, corresponding to the solid curves in Figure 4). (a) CIFAR-100 vs CIFAR- 10 AUROC results.
(b) CIFAR- 10 vs CIFAR- 100 AUROC results.
Number of OOD (CIFAR-10)
Number of OOD (CIFAR-100)
R+VIT (without fine-tuning)
R+VIT fine-tuned on CIFAR-100
R+VIT (without finc-tuning)
R+VIT fine-tuned on CIFAR- 10
examples
examples
per class
per class
88.73 1.88%
98.70 + 0.08%
94.35 + 0.05%
98.96 0.05%
92.94 0.55%
99.02 1 0.15%
2
95.10 1 0.30%
99.11 0.04%
3
93.25 + 0.59%
99. + 0.11%
3
95.60 + 0.01%
99.17 + 0.03%
10
95.73 = 0.31%
99.46 = 0.01%
10
96.42 + 0.02%
99.29 + 0.02%
100
97.70 1 0.01%
99.67 1 0.01%
100
97.38 1 0.01%
99.50 1 0.01%
4 Near OOD detection of genomic sequences
We investigate OOD detection in genomics as another input modality for near-OOD detection. Ren et al. [2019] proposed a benchmark d for OOD detection in genomics, motivated by the real-world problem of bacteria identification based genomic sequences. Real bacteria sequencing data can contain approximately 60-80% of sequences from unknown classes that have not been studied before. Hence, classifier trained on all known classes far will be inevitably asked to predict on genomes that do not belong to one of the known classes. Since different bacteria classes discovered gradually over the years, Ren al. [2019] use set of 10 bacteria classes that were discovered before the year 2011 I as in-distribution classes, a set of 60 bacteria classes discovered between 201 1-2016 as the validation OOD, and a set of 60 different bacteria classes discovered after 2016 as the test OOD. The training set only contains genomic sequences of in-distribution classes. The validation and test sets contain sequences from both in-distribution and OOD classes. The genomic sequence is of fixed length of 250 base pairs, composed by characters of A, C, G, T. In
"https://www htttp::/www..onsooooooooooo/ www.://wwww..ortttttssssss
7

---

the previous work, 1-dimensional Convolutional Neural Networks (ID CNN) were used to build the classifier for the 10 in-distributional classes, and the maximum of softmax probabilities (MSP) and Mahalanobis distance were used for OOD detection. The best AUROC for MSP was only 66. 14%, and 62.41% for Mahalanobis distance [Ren et al., 2019].
Similar to the previous section, we explore the usefulness of pre-trained transformers and fine-tuning for near-OOD detection. Unsupervised pre-training and fine-tuning approach has been applied to several bio-informatic problems, such as protein function prediction [EInaggar et al., 2020, Dohan al., 2021, Littmann al., 2021], protein structure prediction [Rives al., 2021], and predicting promoters, splice sites and transcription factor binding sites [Ji et al., 2020], but it has not yet been studied how pre-training could help near- near-OOD detection.
Bacilus Eschericha
marmonala
90
A G
C
G C
Plazsitiation head
Hoannooo
70
-60
GUROC
50
40
30
Protrain finetune, Mahalanobis,
Pretrain and finetune. MSP. Po0.155
20
1D CNN, MSP, ro00.115
Mahalanobis, P=0.000
? T C
A G G
10
0.10
Pre-training
Finelune
Minimum distance to In-distribution
(a) BERT pre-training and fine-tuning for genomics.
(b) AUROC of near-OOD detection
Figure 5: (a) Model architecture for BERT pre-training and fine-tuning- The unsupervised pre-training model uses a transformer encoder to predict the masked token (shown in red). The fine-tuned model adds simple classification head (a single linear projection) to predict in-distribution classes. (b) The relationship between the minimum genetic distance and the AUROC scores for the 60 OOD test classes. The pre-train+fine-urr based MSP and Mahalanobis distance methods have significantly higher AUROC overall, and the positive correlation between the minimum distance and the AUROC more prominent than for the baseline model.
Unsupervised BERT pre-training and supervised fine-tuning We first pre-train the transformer model in an unsupervised fashion as BERT to capture biologically relevant properties, following Dohan et al. [2021]. For unlabeled in-distribution sequences in the training set, randomly mask the characters in the sequence at the rate of 0.15, feed the masked sequence into transformer-be -based model of heads and layers and embedding dimension 512, and predict the masked characters. To boost the performance, we add the unlabeled validation data to the training set for pre-training. In the fine-tuning stage, we load the pre-trained transformer model, mean pool the embeddings over the positions, and add a single linear projection classification head for 10 in-distribution classes top of the embeddings. The setup shown in Figure 5a. All the parameters the model including those in the pre-trained transformer and those in the classification head are fine-tuned using the labeled training data. The model pre-trained for 300,000 steps using learning rate of 0.001 and Adam optimizer [Kingma and Ba, 2014] on TPU, and the accuracy for predicting the masked token is 48.35%. The model is fine-tuned for 100,000 steps at the learning rate of 0.0001, and the classification accuracy is 99.84% We use the validation in-distribution and validation OOD data to select the best model checkpoint for each of the two methods and evaluate on test set.
Table Genomics OOD BERT pre-trained and fine-tuned on the in in-distribution training set. Error bars represent standard deviation over 3 runs. See Table 1l in Appendix D for AUPRC and FPR95.
Mahalanobis
MSP
Model
Test Accuracy
AUROC
AUROC
ID CNN [Ren et al., 2019] BERT pre-train and fine-tune
85,93-10.11 11..0.11% %
64.75..0.3%
65.84.10.469
89.8410.00%%
77.4910.004%
73.53+10.03%
The results are reported Table It can be seen that using the approach of pre-training transformer and fine-tuning, the OOD detection performance is significantly improved, from 64.75% to 77.49%

---

for Mahalanobis distance. and from 65.84% to 73.53% for MSP. The in n--iisributtin accuracy also improves a bit, from 85.93% to 89.84%. We also study the relationship between the genetic distance and the AUROC of OOD detection for the 60 test OOD classes. We compute the genetic distance using the popular alignment-free method dz which is based on the similarity between the word frequencies of the two genomes [Ren et al., 2018, Reinert et al., 2009]. Studies have shown that this genetic distance reflects true evolutionary distances [Chan et al., 2014, Bernard et al., 2016]. For each of the 60 OOD test classes, we use the minimum genetic distance between this OOD class to any of the 10 in-distribution classes as the final distance measure. Figure 5 shows the AUROC and the minimum distance for each of the 60 OOD classes. We expect the AUROC is higher the distance is greater. Using the baseline ID CNN model. we did not see obvious correlation between the AUROC and the minimum distance, with r² 0.0000, based on Mahalanobis distance. The AUROC based on MSP method has positive correlation to the minimum distance, with r = 0.1190. transformer, both MSP and Mahalanobis distance methods have significantly higher AUROC overall, and the positive correlation between the minimum distance and the AUROC is more prominent than for the baseline model.
After we use the meetrained+ffne-tuune
Few-shot outlier exposure Given that pre-trained and fine-tuned model improves the OOD perfor- mance, we next explore the idea of few shot outlier exposure to further boost the performance. We ran- domly select 1, 2, 5, 10, 100 examples per test OOD class and add them to the training set respectively. For each input æ: in the training set, we extract its corresponding embedding vector z from the above pre-trained and fine-tuned model (or alternatively the model without fine-tume mie-tuning). We construct a single layer perceptron network of 1024 units for classifying each individual to nn-distribution classes and OOD classes, as shown in Figure 3. At inference time, we use the sum of the probability of in-distribution classes the final confidence score for OOD detec- tion. Additionally, we also tried the idea of collapsing all OOD classes into one single class (as in [TThlasii- dasan et al., 2021J) for comparison. The model is trained for 10,000 steps with the learning rate of 0.001. The best model checkpoint is selected based on the highest AUROC on a small set of validation dataset disjoint from the test set.
90
85
80
76 73%
AUROC
75
finetuned use outler label
70
finetured outter label
cutlier ignore
66
66.14%
65
finetured wi out er exposure
60
2
5 10
100
Outlier examples per OOD class
Figure 6: Few-shot outlier exposure for ge- nomics OOD. The x-axis shows the number of outliers per class that the model was exposed to. The y-axis is OOD AUROC in % The shading shows the standard deviation over 3 runs. See Table 12 for exact numbers.
Results are shown in Figure 6. We observe that exposing just small number of OOD examples significantly improves the OOD performance, increasing AUROC from 76.73% to 88.48%. As expected, using the embeddings from the fine-tuned model (blue lines) is better than that from the model without fine-tuning (purple lines). Also, using the outlier labels (purple solid line) has a slightly better performance than collapsing the OOD classes into single class (purple dashed line) using the pre-trained embeddings without fine-tuning.
Using candidate labels with multi-modal text-image models such as CLIP
Multi-modal transformers such as CLIP [Radford et al., 2021], which are pre-trained on image-text pairs, have been shown to perform well on zero-shot classification tasks. We show that such multi- modal transformers open the door to new forms of outlier exposure which can significantly improve out-of-distribune (OOD) detection in the zero-shot classification setting. Our goal is to show that multi-modal transformers can leverage a weaker form of outlier exposure than the few-shot outlier exposure assumption in previous sections, and improve their safety for zero-shot classification.
We use the pre- rre-trained CLIP model? (specifically ViT-B/32) that was trained on 400 million (text, image) pairs from the internet. Its image encoder can map an image I into an embedding vector
Zimage (I). while its text encoder can do the same for a string T as (Te)) By choosing a set of D candidate labels for an image, the similarity between the embedding of the candidate label T, and an image I can be used as the 7th component of the image's embedding vector as 2; =
Ztext (T,) (7)
Phttps:/ htttp::///ittubb www com/openai/CLIP

---

In In-distibutioon candidate
"dinosaur" "elephant" "boy"
"bird"
Out out-distributton
candidatte
Text
Ti
T2
T3
T5
Text
"cat"
Encoder
Encoder
"ship"
Embedding vector
softmax
Probability
Image to embed
Image
Σ
+ p(out|x)
Encoder
[inn|x)
Figure 7: Using candidate text labels and an image-text multi-modal model (CLIP) to produce an embedding vector for OOD detection. We use two sets of candidate labels, evaluate the semantic alignment of the image with each label, apply softmax, and use the sum of probabilities in the first (in) set OOD score. Note that this is zero-shot classification and the model is not fine-tuned and does not leverage any in-distribution or OOD images/labels. It only uses the names of the classes (or other informative words) candidate labels and works well due the strong pre-training of CLIP.
Zero-shot outlier exposure In the zero-shot classification setting, the candidate labels are chosen describe the semantic content of the in-distribution classes (e.g- names of the classes). We propose to include the candidate labels related to the ontoof-distribution classes, and utilize this knowledge as very weak form of outlier exposure in multi-modal models. This could be relevant in applications. where we might not actually have any outlier images for fine-tuning but we might know the names or descriptions of outlier classes.
Our proposed procedure shown in Figure 7. We choose two groups of candidate labels, in- distribution and an embedding vector x for each image I, apply softmax to get probabilities as p = softmax(2)) (=). Those get split p[in (r) Eiein Pi, and p(out (r)) Eienut p;, where p(in|ar) p(out [2r) 1. score (x) p(in
labels (e.g. CIFAR-100 and CIFAR- 10 class names). We produce
Similar to Figure 3, labels to represent the in-distribution and OOD dataset we would like to distinguish (c.g. CIFAR-100 get very informative score that leads to AUROC above previous SOTA, despite no exposure to the training set the in-distribution (zero-shot). Our results are shown in Table 5, with additional results in Table 10 of Appendix C.2.
the confidence score. By choosing the candidate
and 10),
Table 5: Zero-shot OOD detection using image-text multi-mode models. We compare CLIP that uses only the names of in-distribution classes (baseline) and compare it to proposed variant that uses just the names of aut-ofddistribution classes as candidate labels. Even in the zero-shot setting (without any fine-tuning on either in-distribution or OOD dataset), outperform previous SOTA.
Distribution
Distribution 2
Labels 1
Labels 2
AUROC
CIFAR-100
CIFAR 10
CIFAR- 100 names
69.49%
CIFAR-100
CIFAR-10
CIFAR- 100 names
CIFAR-10 names
94.68%
CIFAR-10
CIFAR-
CIFAR-IC names
89.17%
CIFAR-10
CIFAR-100
CIFAR- 10 names
CIFAR 100 names
94.68%
CIFAR-100
SVHN
CIFAR-ION names
93.05%
CIFAR-100
SVHN
CIFAR-100 names
["number"]
99.67%
CIFAR-10
SVHN
CIFAR- 10 names
96.90%
CIFAR- 10
SVHN
CIFAR- 10 names
["number"] 99.95%
6 Conclusion
We focus on the challenging problem of near-OOD detection. We show that fine-tuning large-scale pre-trained transformers and using few-shot outlier exposure can significantly improve the SOTA. On the CIFAR-I00 vs CIFAR-10 visual OOD detection benchmark, we improve the SOTA AUROC from 85% to 96% (without outlier exposure) and 99% (with outlier exposure), essentially closing the gap between SOTA and the ideal performance. On a challenging genomics benchmark, we improve the SOTA from 66% 77% using BERT (without outlier exposure) and 88% (with outlier exposure). We also show that multi-modal pre-trained transformers open the door to new, weaker forms of outlier exposure which only use names of OOD inputs; we apply this to CLIP and achieve AUROC of 94.7% in the zero-shot classification setting. We believe that our findings will be of interest to the research community (and inspire the creation harder near-OOD benchmarks) as well practitioners working criticali applications.
10

---

Acknowledgements
We thank Abhijit Guha Roy, Jim Winkens, Jeremiah Liu, Lucas Beyer and the anonymous reviewers for helpful feedback. We thank David Dohan and Andreea Gane for the helpful advice on BERT genomies model pre-training. We thank Basil Mustafa for providing the BiT model checkpoints. We thank Matthias Minderer for his helpful advice on VIT. We thank Winston Pouse for useful discussions on human performance.
References
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in Al safety. arXiv preprint arXiv: 06565, 2016.
Guillaume Bernard, Cheong Xin Chan, and Mark A Ragan. Alignment-free microbial phyloge-
nomics under scenarios of sequence divergence, genome rearrangement and lateral genetic transfer. Scientific reports, 6(1):1-12, 2016.
Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and Andreas Veit. Understanding robustness of transformers for image classification, 2021.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Vosss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in sule-super vision transformers, 2021.
Cheong Xin Chan, Guillaume Bernard, Olivier Poirion, James M Hogan, and Mark A Ragan. Inferring phylogenies of evolving sequences without multiple sequence alignment. Scientific reports, 4: 6504, 2014.
M. Cimpoi, Maji, Kokkinos, Mohamed, and Vedaldi. Describing textures in the wild, In CVPR, 2014.
Andrew M Dai and Quoc V Le. Semi-supervised sequence learning. In NeurlPS, 2015.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: /eXxvv///000 /810,04805 2018.
David Dohan, Andreea Gane, Maxwell L Bileschi, David Belanger, and Lucy Colwell. Improving protein function annotation via unsupervised pre-training: Robustness, efficiency, and insights. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pages 2782-2792 2021.
Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn. Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rihawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, al. Prottrans: Towards cracking the language of life's code through self-supervised deep learning and high performance computing. arXiv preprint arXiv:2007.062)) 06200. 2020.
William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, 2021.

---

Robert Geirhos, Hom--eerik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shortcut learning deep neural networks. Nature Machine Intelligence, 2(11 11:6665-7733 2020.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks, arXiv preprint 1:Xiv:1706.045355 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas G Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: 18:2: 04666, 2018.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning, 2019a.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supprvised learning can improve model robustness and uncertainty. arXiv preprint urXiv:/906 /2340, 2019b.
Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. Pretrained transformers improve iit-of-distribution robustness, arXiv preprint arXiv:2004.06100, 2020.
Jeremy Howard and Sebastian Ruder.
Universal language model fine-tuning for text classification. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018. URL http:t/:/// /ttt::://dx.co doi. org/ 10. 18653/v1/p18- 1031.
Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. Dnabert:
pre-trained bidirectional encoder representations from transformers model for lna-lang in genome. bioRxiv, 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. arXiv preprint arXiv:1912.11370, 6(2):8, 2019.
Balaji Lakshminarayaman, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In NeurlPS, 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurlPS, 2018
Shiyu Liang. Yixuan Li, and R Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv: 2017.
Maria Littmann, Michael Heinzinger, Christian Dallago, Tobias Olenyi, and Burkhard Rost. Embed- dings from deep learning transfer go annotations beyond homology. Scientific reports, 11(1):1-14, 2021.
Jeremiah Zhe Liu. Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax- Weiss, and Balaji Lakshmi- narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. NeurIPS, 2020.
Kaleel Mahmood, Rigel Mahmood, and Marten van Dijk. On the robustness of vision transformers to adversarial examples, 2021.
Xiaofeng Mao, Gege Qi, Yuefeng Chen, Xiaodan Li, Ranjie Duan, Shaokai Ye, Yuan He, and Hui Xue. Towards robust vision transformer, 2021.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks, 2021.
12

---

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Hybrid models with deep and invertible features. In ICML, 2019.
Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transformers, 2021.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Anh Nguyen, Jason Yosinski, and Jeff Clune. predictions for unrecognizable images. In CVPR, pages 427-436, 2015.
Deep neural networks are easily fooled: High confidence
Sayak Paul and Pu Chen. Vision transformers are robust learners. arXiv preprini arXiv:2/05.07581, 2021.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. JMLR,
2011.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations, 2018.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language under- standing by generative pre-training. 2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2019.
Gesine Reinert, David Chew, Fengzhu Sun, and Michael Waterman. Alignment-free sequence comparison (I): statistics and power. Journal of Computational Biology, 16(12):1615-164, 2009.
Jie Ren, Xin Bai, Yang Young Lu, Kujin Tang, Ying Wang, Gesine Reinert, and Fengzhu Sun. Alignment-free sequence analysis and applications. Annual Review of Biomedical Data Science, 1: 93-1 14, 2018
Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A DePristo, Joshua V Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distributio detection. NeurIPS, 2019.
Tal Ridnik, Emanuel Ben Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor Imagenet-21k pretraining for the masses, 2021.
Alxxander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning 250 million protein sequences. Proceedings of the National Academy of Sciences, 118(15), 2021.
Abhijit Guha Roy, Jie Ren, Shekoofeh Azizi, Aaron Loh, Vivek Natarajan, Basil Mustafa, Nick Pawlowski, Jan Freyherg, Yuan Liu, Zach Beaver, Nam Vo, Peggy Bui, Samantha Winter, Patricia MacWilliams, Greg S. Corrado, Umesh Telang, Yun Liu, Taylan Cemgil, Alan Karthikesalingam, Balaji Lakshminarayanan, and Jim Winkens. Does your dermatology classifier know what doesn't t know? Detecting the long-tail of unseen conditions. arXiv preprint arXi::2104003829, 2021.
Chandramouli Shama Sastry and Sageev Oore. Detecting out-of- examples with Gram matrices. In /CML, 2020.
Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin- Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of visual transformers, 2021.
13

---

Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. arXiv preprint arXiv:2 100.7000, 2021.
Sunil Thulasidasan, Sushil Thapa, Sayera Dhaubhadel, Gopinath Chennupati, Tanmoy Bhattacharya, and Jeff Bilmes. A simple and effective baseline for out-of disttibbtitiiittioonn detection using abstention. 2021. URL https: //pp///ee/e/eieeee//
lya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung. Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mip-mixer: An all-mlp architecture for vision, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017.
Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R. Ledsam, Patricia MacWilliams, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, Taylan Cemgil, S. M. Ali Eslami, and Olaf Ronneberger. Contrastive training for improved out-of-distribttion detection, arXiv preprint arXiv:2007.05566 2020.
Hongjie Zhang, Ang Li, Jie Guo, and Yanwen Guo. Hybrid models for open set recognition. European Conference Computer Vision, 2020.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
14


Reviews:
[{"review_id": "maLaa_NZcz1", "reviewer": "Reviewer_nQCh", "summary": "This paper finds and demonstrates that large-scale pre-trained transformers can significantly improve the state-of-the-art  on a range of near OOD tasks across different data modalities. The main contribution is that they find an efficient way to apply pre-trained transformer to OOD tasks and boost the performance significantly.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Though not surprised about this finding and conclusion, it is impressive that the authors provide detailed study and experiments on different comparisons. The paper is easy to read and follow, this would provide an important method to deal with OOD based on pretrained transformers. Also, beyond OOD, this paper could also bring some insights about how powerful the pretrained transformer will be for various vision tasks.\n\nStrength:\nI like the analytical experiments in the paper, such as Figure 1. The figure is very clear and easy to understand.\nThe paper is well-written and easy to follow.\nThe proposed method is simple but effective and could bring some insights for future research.\n\nConcerns:\nIn line 123, the authors mention that `Self-supervised learning techniques have also been shown to improve OOD robustness`, what’s the underlying difference between the self-supervised methods and pretrained model. Does large-scale data-driven pretrained model should be the main issue for OOD tasks? Is there some cases that pretrained strategy works worse than self-supervised learning techniques? If yes or no, could the authors explain it?\n\nIn line 134, the authors claim that `We found that we get better generalization and higher quality embeddings when we do not use data augmentation for fine-tuning.` I am curious about which data augmentation the authors used for the experiments, is there any explanation for this part?\n\nIn general the whole paper is very clear and easy to follow. While I still have concerns regarding only conduct OOD experiments on CIFAR10 and a few relevant very small datasets with very low resolution ($32 \\times 32$). Since the pretrained transformer is trained with large-scale datasets, so it is possible that the transformers already learn some related patterns. The improvement is significant, but wondering have the authors tried on other dataset with higher resolutions. If yes, how it goes with the pre-trained model? I feel like it might be harder to achieve a much higher performance on those dataset based on the pretrained model using current straight-forward strategy. I will change my scores based on the explanation of this part.\n\n------------------- post rebuttal-----------------------\nThanks for the rebuttal, I am satisfied with the explanation, but still conservative about the potential usage of this method for the community, therefore, I keep my score unchanged.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Since the pretrained transformer is trained with large-scale dataset, while in this paper, the comparisons are primarily based on CIFAR10 and CIFAR100, which is too small to convince people thoroughly. It would be suggested that the authors could apply the same strategy to various OOD tasks and justify the efficiency and robustness of the proposed method.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6-7"}, "extra_text": {}}, {"review_id": "c5J-oyPWIHV", "reviewer": "Reviewer_qcJH", "summary": "The paper experimentally examines the effect of using pre-trained (transformer) models for OOD detection and reports the following three results: i) pre-trained transformers lead to significant improvements on near-OOD benchmarks; ii) pre-trained transformers are well-suited for few-shot outlier exposure; iii) exploring pre-trained multi-modal image-text transformers in the zero-shot classification setting, just using the names of outlier classes as candidate text labels for CLIP can achieve good OOD detection performance.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think this is a good paper that contains a lot of useful experimental results. But, on the other hand, I feel that its academic impact is slightly insufficient for a NeurIPS paper. \n\nThe three results can be summarized by the effectiveness of pretrained Transformer models for OOD detection. However, the effectiveness of pretraining for OOD detection is already known, as shown in, e.g., the following paper. (It is a problem that it is not cited.) \n\nHendrycks et al., Using Pre-Training Can Improve Model Robustness and Uncertainty, ICML2019\n\nThe paper’s claim (i) is that the two, i.e., pretraining + transformer, are important for improved performance. However, is it true? I suspect that not the transformer model but pretraining (on ImageNet21k) is the essence. Thus, even ResNet/EfficeinetNet models would show similarly good performance if they are pretrained on ImageNet21k and achieve similar ID classification accuracy. Table 1 shows comparisons with BiT-M R50x1, but they may not be fair since the compared models differ in the ID classification accuracy. Thus, the paper lacks sufficient evidence for the above claim. If my guess is true (i.e., the transformer is not necessary), then the results reported in this paper may not be so novel.\n\nRegarding claim (ii), the method of few-shot outlier exposure is already known. It is unclear how novel it is to use transformer models for the backbone network. \n\nRegarding claim (iii), the proposed method may be regarded as a new CLIP application but remains one of many of its applications. \n\nIn addition, one may question if we can still call it OOD detection even when we use pretrained models, especially those trained on a dataset sharing its classes with the OOD dataset. For instance, I suspect that most (or even all?) of CIFAR100 classes are the ImageNet21k classes. Can we call it OOD if an input belongs to a class learned in the pretraining?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "If my concerns above are not resolved, they will be limitations of this study. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "Odx5R29QQgz", "reviewer": "Reviewer_dPMr", "summary": "* The paper presents detailed experiments which show that pre-trained large scale transformers significantly improves OOD detection performance.\n* The paper especially focuses on the case where in-distribution (InD) and OOD samples are close to each other (near OOD) (e.g. when CIFAR100 is InD and CIFAR10 is OOD, SoTA AUROC is improved from 85% to 96% ).\n* It is demonstrated that transformers can yield better well-separated representations for each class on InD dataset compared to ResNet without even fine tuning on InD samples. Separation of the representations become even better after fine-tuning in InD dataset (see Fig. 1). This motivates that transformers can be better for OOD detection compared to ResNet-like architectures and improves the SoTA.\n* The paper presents experiments in different settings on multiple datasets:\n* First, OOD detection performance between CIFAR10 <-> CIFAR100 is evaluated. The results demonstrate that fine-tuning transformers significantly improves SoTA OOD detection performance. Additional experiments are presented where it is assumed that a few examples from each class of target OOD dataset is available (called outlier exposure). This leads to further boost in OOD detection performance.\n* The second experiments are performed on a genomics dataset where InD dataset consist of 10 known bacteria classes and OOD dataset consist of another 60 bacteria classes which are not seen during training. The results are consistent with the CIFAR10-CIFAR100 experiments. \n* Finally, experiments on multimodal text-image models are presented where candidate labels are used for OOD detection. The results show that it leads to significant performance improvement.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n- The paper presents extensive performance evaluation existing OOD detection methods on transformers architectures. The results presented in the paper demonstrate significant improvement over SoTA in challenging near OOD detection tasks. \n- Despite of the fact that the paper doesn't propose a novel algorithm, the presented experiments are quite extensive.\n- Transformers are popular architectures which achieve SoTA performance in variety of tasks. The paper demonstrate that they not only achieve high prediction performance, but also achieves significant OOD detection performance which makes it an ideal choice for safety-critical applications.\n\n**Weaknesses**\nI don't see any major weakness of the paper. There are just a few points that needs a bit clarification for me and a few suggestions for improvements.\n- According to Fig. 1, ViT achieves perfect separation of classes without even fine-tuned on InD dataset. This is quite surprising to me because this mean that transformers can identify images from same classes without seeing any training examples. Is it because ViT is pre-trained on very large datasets? I think some intuition about how transformers can achieve this would be quite useful.\n- In the outlier exposure experiments, a few examples from each class of target OOD are used to improve the OOD detection performance. However, target OOD dataset is usually unknown in real applications. So, although teams are willing to annotate some, they may not overlap with the target OOD dataset. For example, let's consider the scenario where CIFAR10 is InD, sample OOD data available for outlier exposure is CIFAR100, and OOD samples given to the model in test time is SVHN. Do the model assign SVHN samples to one of the InD or OOD classes?\n- One generic way of performing outlier exposure could be using adversarial samples as OOD and testing how they generalise to various unseen OOD datasets.\n- In lines 278-279, the paper mentions that the best model is checkpointed based on best validation AUROC. However, the best model is usually checkpointed based on the best prediction accuracy rather than the best OOD detection performance. How would the results change if the models were saved based on the best prediction accuracy? Do this affect the OOD detection performance significantly?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- Please see the weaknesses section for some limitations of the paper.\n- I don't see any potential negative social impact.\n", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "88rYWxye78s", "reviewer": "Reviewer_s3xZ", "summary": "This paper studies the few-shot outlier detection setting where a few examples from outlier classes are provided to the ODD detector.  The main motivation behind this work are situations that require high level of ODD detection. It is shown that classifiers trained by fine-tuning large-scale pre-trained transformers are significantly better at near-ODD detection. Simulations are performed on well-known data sets for visual classification and genomic sequences. Performance evaluation are reported in term of AUROC scores. However, it would have been more convenient to report results in terms of FPR at 95% scores which allow to better understand if the performance gains with respect to similar methods are relevant. The reported gains are not very significant but consistent. \n", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main limitation of this work relies on the assumption that out-of-distribution samples can be labelled and be available to train the detector. I strongly believe this is not the case in many of practical scenarios where these samples evolve over time and this cannot be given in advance to the learner. On the other hand, the paper lacks of novelty in terms of exceptions for a NeurIPS work. The proposed method relies on the addition of the soft-probabilities of the concatenated  predictions for in and out distributions. Finally, it is not clear how much this method improve on state-of-the-art methods since most of the existent literature on ODD detection does not assume of the presence of ODD samples from the same distribution than for testing. This also brings the question of how much the present method degrades  if the available ODD samples do not correspond to the same testing distributions. Additional simulations would have been beneficial to clarify this issue. Finally, I believe that FPR at 95% scores are much more relevant than  AUROC scores which have not been reported. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I do not think the authors addressed the main limitation related to the impossibility of disposing of the ODD samples corresponding to the testing distribution. As for the societal impact, the investigated topic (e.g., ODD detection) is very relevant and can have significant impact on real-word systems.", "review_text": "", "extra_scores": {"ethical_concerns": "There is no ethical concerns to be reported. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours "}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers are in agreement that the paper results are clearly presented and thoroughly validated, and the takeaways on ViTs helping with (few-shot) OOD identification should be of broad interest to the community.
Decision: Accept (Poster)