Practical Evaluation of Out-of-Distribution Detection Methods for Image Classification
Authors: Engkarat Techapanurak
Takayuki Okatani
Venue:  (2021)
Keywords: out-of-distribution
novel class detection
domain shift
concept drift

Abstract: We reconsider the evaluation of OOD detection methods for image recognition. Although many studies have been conducted so far to build better OOD detection methods, most of them follow Hendrycks and Gimpel's work for the method of experimental evaluation. While the unified evaluation method is necessary for a fair comparison, there is a question of if its choice of tasks and datasets reflect real-world applications and if the evaluation results can generalize to other OOD detection application scenarios. In this paper, we experimentally evaluate the performance of representative OOD detection methods for three scenarios, i.e., irrelevant input detection, novel class detection, and domain shift detection, on various datasets and classification tasks. The results show that differences in scenarios and datasets alter the relative performance among the methods. Our results can also be used as a guide for practitioners for the selection of OOD detection methods.

Under review as a conference paper at ICLR 2021
PRACTICAL EVALUATION OF OUT-OF-DISTRIBUTION DETECTION METHODS FOR IMAGE CLASSIFICATION
Anonymous authors Paper under double-blind review
ABSTRACT
We reconsider the evaluation of OOD detection methods for image recognition. Although many studies have been conducted so far build better OOD detection methods, most them follow Hendrycks and Gimpel's 's work for the method of experimental evaluation. While the unified evaluation method is necessary for a fair comparison, there is a question of if its choice of tasks and datasets reflect real-world applications and if the evaluation results can generalize to other OOD detection application scenarios. In this paper, we experimentally evaluate the per- formance of representative OOD detection methods for three scenarios, i.e., irrel- evant input detection, novel class detection, and domain shift detection, on various datasets and classification tasks. The results show that differences scenarios and datasets alter the relative performance among the methods. Our results can also used as a guide for practitioners for the selection of OOD detection methods.
INTRODUCTION
Despite their high performance on various visual recognition tasks, convolutional neural networks (CNNs) often show unpredictable behaviors against out-of-distribution (OOD) inputs, i.e., those sampled from different distribution from the training data. For instance, CNNs often classify irrel- evant images to one of the known classes with high confidence. A visual recognition system should desirably be equipped with ability to detect such OOD inputs upon its real-world deployment.
There are many studies of OOD detection that are based on diverse motivations and purposes. How- far the recent studies targeted at visual recognition concerned, most of them follow the work of Hendrycss & Gimpel (2017), which provides formal problem statement of OOD detection and experimental procedure evaluate the performance of methods. Employing this procedure, the recent studies focus mainly on increasing detection accuracy, where the performance is measured using the same datasets.
On the one hand, the employment of the experimental procedure has arguably bought about the rapid progress of research in short period. On the other hand, little attention has been paid how well the employed procedure models real-world problems and applications. They are diverse in purposes and domains, which obviously cannot be covered by the single problem setting with a narrow range of datasets.
In this study, to address this issue, consider multiple, more realistic scenarios of the application of OOD detection, and then experimentally compare the representative methods. To be specific, we consider the three scenarios: detection of irrelevant inputs, detection of novel class inputs, and detection of domain shift. The first two scenarios differ in the closeness between ID samples and OOD samples.
Unlike the first two, domain shift detection is not precisely OOD detection. Nonetheless, it is the same as the other two in that what we want is to judge if the model can make a meaningful inference for a novel input. In other words, we can generalize OOD detection to the problem of judging this. Then, the above three scenarios are naturally fallen into the same group of problems, and becomes natural to consider applying OOD detection methods to the third scenario. It is noteworthy that domain shift detection has been poorly studied in the community. Despite many demands from practitioners, there is no established method in the context of deep learning for image elassification.

---

Under review as a conference paper at ICLR 2021
Based on the above generalization of OOD detection, we propose meta-approach in which any OOD detection method can be used as its component.
For each of these three scenarios, compare the following methods: the confidence-basel bascline (Hendrycks & Gimpel 2017). MC dropout (Gal & Ghahramani 2016). ODIN (Liang et al. 2017).
cosine similarity (Techapanurak et al. 2019: Hsu et al. 2020), and the Mahalanobis detector (Lee)
et al. 2018). Domain shift detection is studied in (Elsahar & Gallé 2019) with natural language processing tasks, where proxy-A distance (PAD) is reported to perform the best; thus we test it in our experiments.
As for choosing the compared methods, we follow the argument shared by many recent studies (Shafaci et al. 2019; Techapanurak et al. 2019 Yu & Aizawa 2019. Yu et al. 2020: Hsu et al.) 2020) that OOD detection methods should not assume the availability of explicit OOD samples at training time. Although this may sound obvious considering the nature of OOD, some of the recent methods (e.g.. [Ling et al.] (2017);[ee Lee et al. (2018) use certain amount of OOD samples as validation data to determine their hyperparameters. The recent studies, (Shafaei et al., 2019) Techapanurak et al. 2019), show that these methods do perform poorly when encountering OOD imputs sampled from different distribution from the assumed one at test time. Thus, for ODIN and the Mahalanobis detector, we employ their variants (Hsu et al. 2020 Lee et al. 2018)) that can work without OOD samples. The other compared methods do not need OOD samples.
The contribution of this study are summarized as follows. i) Listing three problems that practitioners frequently encounter, evaluate the existing OOD detection methods on each of them. ii) We show a practical approach to domain shift detection that is applicable to CNNs for image classification. iii) We show experimental evaluation of representative OOD detection methods on these problems, revealing each method's effectiveness and ineffectiveness in each scenario.
2 PROBLEMS AND METHODS
2.1 PRACTICAL SCENARIOS OF OOD DETECTION
We consider image recognition tasks in which a CNN classifies a single image x into one of C known classes. The CNÑ is trained using pairs of x and its label, and x is sampled according to ~ p(z)). At test time, it will encounter an unseen input I, which is usually from p(r) but is sometimes from p' (x)) a different, unknown distribution. In this study, we consider the following three scenarios.
Detecting Irrelevant Inputs The new input x does not belong to any of the known classes and is out of concern. Suppose we want to build a smartphone app that recognizes dog breeds. We train a CNN on a dataset containing various dog images, enabling it to perform the task with reasonable accuracy. We then point the smartphone could classify the image as a Bull Terrier with high confidence. Naturally, we want to avoid this by detecting the irrelevance of x. Most studies of OOD detection assumes this scenario for evaluation.
sofa and shoot its image, feeding it to classifier. It
The input & belongs to a novel class, which differs from any of C known classes, and furthermore, we want our CNN learn to classify later, e.g., after additional training. For instance, suppose we are building a system that recognizes insects in the wild, with an ambition make it cover all the insects on the earth. Further, suppose an image of one of the endangered (and thus rare) insects is inputted to the system while operating it. If we can detect it as novel class, we would be able update the system in several ways. The problem the same as the first scenario in that we want to detect whether to samples of the learned classes, or equivalently, p/ (xx) is more close to p(z). arguably making the detection more difficult. Note that in this study, we don't consider distinguishing whether is an irrelevant input or a novel class input, for the sake of simplicity. We left for a future study.
Detecting Novel Classes
p(x)) or not. The difference is that x is more similar
Detecting Domain Shift The input belongs to one of C' known classes, but its underlying dis- tribution is pzx), not p(z). We are especially interested in the case where a distributional shift p(x) + p' (z) occurs either suddenly gradually while running system for the long term. Our CNN may or may not generalize beyond this shift to p' (z). Thus, we want to detect if it does not. If can do this, would take some actions, such as re-training the network with new training
2

---

Under review as a conference paper at ICLR 2021
data (Elsahar & Gallé 2019). We consider the case where no information is available other than the incoming inputs z's.
A good example is surveillance system using camera deployed outdoor. Let us assume the images' quality deteriorates after some time since its deployment, for instance, due to the camera's aging. Then, the latest images will follow different distribution from that of the training data. Unlike the above two cases where we have to decide for a single input, we can use multiple inputs; we should, especially when the quality of input images deteriorate gradually as time goes.
The problem here has three differences from the above two scenarios. First, the input is a valid sample belonging
known class, neither an irrelevant sample nor novel class sample. Second, we are basically interested in the accuracy of our CNN with the latest input(s) and not in whether x ~ p(z) or p' (zr). Third, as mentioned above, we can use multiple inputs { for the judgment.
Additional remarks on this scenario. Assuming a temporal sequence of inputs, the distributional shift is also called concept drift (Gama et al. (2044). It includes several different subproblems, and the one considered here called virtual concept drift its terminology. Mathematically. concept drift occurs when p(xr,) :) changes with time. It is called virtual when p(x; changes while p(ylar) does not change. Intuitively, this the case where the classes (i.e., concept) remain the same but p(z)) changes, demanding the classifier to deal with inputs drawn from p' p1.... Then, are usually interested in predicting if æ lies in a region of the data space for which our classifier is well trained and can correctly classify it. If not, we might want to retrain our classifier using additional data or invoke unsupervised domain adaptation methods (Ganin & Lempitsky 2015 Tzeng et al.) 2017).
2.2 COMPARED METHODS
We select five representative OOD detection methods that do not use real OOD samples to be en- countered at test time.
Baseline: Max-softmax Hendrycks & Gimpel (017)) showed that the maximum of the softmax outputs, or confidence, can be used detect OOD inputs. We use it as the score of an input being in-distribution (ID). We will refer to this method as Baseline. It is well known that the confidence can be calibrated using temperature to better represent classification accuracy (Guo al., 2017 Li & Hoiem 2020). We also evaluate this calibrated confidence, which will be referred to as Calib.
MC Dropout The confidence (i.e., the max-softmax) is also thought of as a measure of uncer- tainty of prediction, but it captures only aleatoric uncertainty (Hüllermeier & Waegeman 2019). Bayesian neural networks (BNNs) can also take epistemic uncertainty into account, which is theo- retically more relevant to OOD detection. MC (Monte-Carlo) dropout (Gal & Ghahramani, 2016) is an approximation of BNNs that is computationally more efficient than an ensemble of networks LLassminnaayanaan et al. 2017). To be specific, using dropout (Srivastava et al., (214)) at test time provides multiple prediction samples, from which the average of their max-softmax values is calcu- lated and used as ID score.
Cosine Similarity It is recently shown in Techapanurak et al. (2019) Hsu et al. (2020) that using scaled cosine similarities the last layer of a CNN. similar to the angular softmax for metric learn- ing, enables accurate OOD detection. To be specific, the method first computes cosine similarities between the feature vector of the final layer and class centers (or equivalently, normalized weight vectors for classes). They are multiplied with a scale and then normalized by softmax to obtain class scores. The scale, which is the inverse temperature, is predicted from the same feature vector. These computations are performed by single layer replacing the last layer of a standard CNN. The maximum of the cosine similarities (without the scale) gives IDD score. The method is free of hyperparameters for OOD detection. We will refer to it as Cosine.
ODIN (with OOD-sample Free Extension) ODIN was proposed by Liang et al. (2117) to improve Baseline by perturbing an input + x + (8x) in the direction dx of maximally increasing the max softmao and also by temperature scaling. Thus, there are two hyperparameters, the perturbation size and the temperature T. In Liang et al. (2017), they are chosen by assuming the availability of explicit OOD samples. Recently, Hsu et al. (2020) proposed to select e t argmax, Σ yx + sgg((zr)) where ye is the max-sofumax and the summation is taken over IDD samples in the
3

---

Under review as a conference paper at ICLR 2021
validation set. As for the temperature, they set T = 1000. ID score is given by yr + e sgn(8x)). To distinguish from the original ODIN, we refer to this as ODIN".
Mahalanobis Detector The above three methods are based on the confidence. Another approach is to formulate the problem as unsupervised anomaly detection. Lee et al. (2018) proposed to model the distribution of intermediate layer's activation by Gaussian distribution for each class but with a shared covariance matrix among the classes. Given an input, the Mahalanobis distance concerning the predicted class is calculated at each layer. A score for OOD is given by the weighted sum of those calculated at different layers. The weights are predicted by logistic regression, which is determined by assuming the availability of OOD samples. To be free from the assumption, another method is suggested that generates adversarial examples from ID samples and regard them as OOD samples. also reported in (Hsu et al., 2020) that setting all the weights to one works reasonably well. We evaluate the last two methods that do not need OOD samples. Although the original method optionally uses input perturbation similar to ODIN, we do not use because our experiments show that its improvement is very small despite its high computational cost.
Effects of Fine-tuning a Pre-trained Network It has been well known that fine-tuning a pre- trained network on downstream task improves its prediction accuracy, especially when small amount of training data is available. was pointed out in (He et al., 2019) that the improvement is little when there is sufficient training data. Hendrycks et al. (2019) then show that even in that case, using a pre-trained network helps increase the overall robustness of the inference. It includes improved OOD detection performance, in addition robustness adversarial attacks, better cali- bration of confidence, robustness to covariate shift. However, their experimental validation is per- formed only on single configuration with few datasets. It remains unclear if the improvement can generalize to a broader range of purposes and settings that may differ in image size, the number of training samples, and ID/OOD combinations.
EXPERIMENTAL RESULTS
We use Resnet-50 (He et al. 2016) for a base network. We use it as is for Baseline, ODIN", and Mahalanobis, which share the same networks with the same weights, which will be referred to as Standard. We apply dropout to the last fully-connected layer with p = 0.5 and draw ten samples for MC dropout. We modify the last layer and the loss function for Cosine, following TTchapanurak eet al. (2019). We use the ImageNet pre-trained model provided by the Torchvision libraryfor their pre-trained models. We employ AUROC to evaluate OOD detection performance with the first two scenarios, following previous studies.
3.1 DETECTION OF IRRELEVANT INPUTS
We employ the following five tasks and datasets: dog
breed recognition (120 classes and 10,222 images;
Khosla et al. (20IT)), plant seeding classification (12
classes and 5,544 images; Giselsson al. (017)).
Food-101 (101 classes and 101,000 images; Bossard
al. (2014)) CUB-200 (200 classes and 11.788 images;
Figure 1: Example images for the five
Welinder et al. (2000), and Stanford Cars (196 classes
datasets.
and 18.185 images: Krause al. (203)). These datasets will be referred to as Dog. Planr, Food, Bird, and Cars. They are diverse in terms of image contents, the number of classes, difficulty of tasks (e.g., fine- aooe-grrined/coare-ggiinee grained)), etc. Choosing one of the five as ID and training network on it, we regard each of the other four OOD, measuring the OOD detection performance of each method on the 5 x ID-OOD combination. We train each net- work for three times to measure the average and standard deviation for each configuration. Table [ shows the accuracy of the five datasets/tasks for the three networks (i.c., Standard, MC dropout, and Cosine) trained from seratch and fine-tuned from pre-trained model, respectively. It is seen that there is large gap between wrining-froms from-scratch and fine-tuning a pre-trained model for the datasets with fewer training samples.
Figure ( shows the average AUROC of the compared OOD detection methods for each ID dataset over the four OOD datasets and three trials for each. The error bars indicate the minimum and
4

---

Under review as a conference paper at ICLR 2021
Table 1: Classification accuracy (mean and standard deviation in parenthesis) of the three networks on the five datasets/tasks.
Train-from-scratch MC dropout
Fine-tuning Standard
Dataset
Standard
Cosine
MC drepout
Cosine
Dog
26.7(3.4)
28.912.8)
36.((1.4)
79.4(0.1)
79.3(0.3)
78.5(0.3)
Plant
94.1(0.4)
94.7(0.2)
95.8(0.9)
95.2(0.6)
95.5(0.5)
92.7(2.6)
Food
75.5(1.0)
76.4(0.2)
76.6(0.1)
80.5(0.0)
80.7(0.1)
79.2(0.1)
Bird
24.7(0.9)
28.5(0.6)
31.3(2.4)
71.90.3)
72.4(0.4)
70.1(0.3)
Car
18.2(3.8)
22.0(1.6)
36.0(6.2)
77.6(0.3)
77.7(0.3)
73.7(0.6)
Fine-tun mee-tuning Trom-scratch
AUROC
25-
50-
25-
Plant
Food
Bird
Baseline
Callb.
drapout
(sum)
ladv)
Figure 2: OOD detection performance of the compared methods. 'Dog' indicates their performance when Dog is ID and all the other four datasets are OOD, etc. Each bar shows the average AUROC of method, and the error bar indicates its minimum and maximum values. Upper: The networks are trained from scratch. Lower: Pre-trained models are fine-tuned.
maximum of AUROC. The full results for each of the twenty ID-OOD pairs are reported in Tables S and [6] in Appendix AA] The upper row of Fig. 2 shows the results with the networks trained from scratch. It is seen that the ranking of the compared methods are mostly similar for different ID) datasets. For the five datasets, Cosine is consistently among the top group; Mahalanobis will be ranked next, since it performs mediocre for Dog and Food. For the tasks with low classification accuracy, Dog, Bird, and Car, as shown Table [ the OOD detection accuracy tends to be also low; however, there is tendency in the ranking of the OOD detection methods depending the ID classification accuracy.
The lower of Fig. 2 shows the results with the fine-tuned networks. It is first observed for any dataset and method that the OOD detection accuracy is significantly higher than the networks trained from scratch. This reinforces the argument made by [Hendrycks al. (2019) that the of pre-trained networks improves OOD detection performance. Furthermore, the performance increase lot higher for several cases than reported their experiments that use CIFAR- 1A/R-00/100 and Tiny ImageNet (Deng et al. 2009). The detection accuracy is pushed to mear-maximum for each case. Thus, there only little difference among the methods; Cosine and Mahalanobis(sum) shows slightly better performance for some datasets.
3.2 DETECTION OF NOVEL CLASSES
We conducted two experiments with different datasets. The first experiment uses the Oxford-IIII Pet dataset (Parkhi et al. 2012), consisting of 25 dog breeds and 12 cat breeds. We use only the dog breeds and split them into 20 and breeds. We then train each network on the first 20 dog breeds using the standard train/test splits per class. The remaining five breeds (i.e., Scottish Terrier, Shiba Inu, Staffordshire Bull Terrier, Wheaten Terrier, Yorkshire Terrier) are treated as OOD. should be noted that the ImageNet dataset contains 118 dog breeds, some of which overlap with them. We intentionally leave this overlap to simulate a similar situation that could occur in practice. In the second experiment, use the Food-101 dataset. We remove eight classe contained in the
Apple Pie, Breakfast Burrito, Chocolate Mousse, Gaucamole, Hamburger, Hot Dog, Ice Cream, Pizza
5

---

Under review as a conference paper at ICLR 2021
Table 2: Novel class detection performance of the compared methods measured by AUROC.
Dog
Food-A
Method
From-scratch
Fine-tuning
From-scratte
Fine-tuning
Baseline
61.1(0.8)
88.7(1.0)
82.5(0.1)
84.6(0.2)
Calib.
62.9(0.5)
86,5(1.1)
83.4(0.1)
84.8(0.2)
MC dropout
61.7(0.44)
89.8(0.8)
82.7(0.1)
84.7(0.1)
Cosine
(8.3(
94.1(0.8)
83.7(0.1)
85.7(0.3)
ODIN*
59.9(0.5)
85.3(1.4)
77.4(0.1)
74.7(0.3)
Maha. (sum)
52.3(1.2)
78.7(1.4)
$1.3(0.1)
61.9(0.5)
Maha. (adv)
49.0(0.2)
65.8(1.5)
51.8(0.3)
57.1(7.2)
ImageNet dataset. We split the remaining 93 classes into 46 and 47 classes, called Food-A and -B, respectively. Each network is trained on Food-A. We split Food-A into 800/100/100 samples per class to form train/val/test sets. Treating Food-B OOD, evaluate the methods' performance.
Table shows the methods' performance of detecting OOD samples (i.e., novel samples). In the table separate the Mahalanobis detector and the others; the latter all based confidence or its variant, whereas Mahalanobis is not. The ranking of the methods is similar between the two experiments. Cosine attains the top performance for both of the two training methods. While this is similar to the results of irrelevant sample detection (Fig. 2 the gap to the second best group (Baseline, Calib., and MC dropout) is much larger here; this is significant for training from scratch. Another difference is that neither variant of Mahalanobis performs well; they are even worse than Baseline. This will be attributable the similarity between ID) and OOD samples here. The classi- fication accuracy of the original tasks, Dog and Food-A are given in Table in Appendix B
3.3
DETECTION OF DOMAIN SHIFT
3.3.1 PROBLEM FORMULATION
Given a network trained on a dataset D,, we wish to estimate its classification error different dataset De. In practice, a meta-system monitoring the network estimates the classification error on each of the incoming datasets D.") DD(2) : which are chosen from the incoming data stream. It issues alert if the predicted error for the latest D(2") is higher than the pre-fixed target.
We use an OOD score S for this purpose. To be specific, given D; = {=:])=1,...... we calculate an average of the score 3- = Σ. S./n, where S; is the OOD score for IG; note that an OOD score is simply given by negative ID) score. We want to use S to predict the classification error eTT =-==> Σ; 1-11- 11y 14)/n, n, where y and are a prediction and the true label, respectively. Following Elsahar Gallé (2019), we train a regressor f to do this, as eIr ~ f(S). We assume multiple labeled datasets D. are available, each of which do not share inputs with D3 D,, Choosing two-layer MLP for /, we train it on D, 's plus D..- As they have labels, we can get the pair of err and 5 for each of them. Note that D, does not have labels.
is reported in Elsahar & Gallé (2019) that Proxy-A Distance (PAD) (Ben-David et al., 2007) performs well on several NLP tasks. Thus, we also this method (rigorously, the one called PAD* in their paper) for comparisons. It first trains a binary classifier using portions of D. and D. to distinguish the two. Then, the classifier's accuracy evaluated on the held-out samples of D. and Dr, which is used as a metric of the distance between their underlying distributions. Intuitively, the classification is easy when their distance is large, and vice versa. We train using 1 (mean absolute error) for S as in the previous work.
3.3.2 DOMAIN SHIFT BY IMAGE CORRUPTION
We first consider the case when the shift caused by the deterioration of image quality. An example is surveillance camera deployed in an outdoor environment. Its images initially of high quality, but later their quality deteriorates gradually or suddenly due to some reason, e.g., dirt on the lens, failure of focus adjustment, seasonal/climate changes, etc. We want detect it if affects classifi-

---

Under review as a conference paper at ICLR 2021
Table 3: Errors of the predicted classification error by the compared methods.
Food-A (From-scratch)
Food-A (Fine-tuning)
ImageNet
Method
MAE
RMSE
MAE
RMSE
MAE
RMSE
Baseline
15.8(3.0)
20.5(3.7)
6.4(1.3)
7.9(16)
4.6(0.8)
6.3(1.0)
Calib.
15.0(2.9)
19.6(3.4)
6.3(1.3)
7.9(1.6)
4.3(0.8)
6.0(1.0)
MC dropout
15.3(2.7)
19.7(3.0)
5.8(1.1)
7.2(1.4)
4.0(0.7)
5.3(0.9)
Cosine
6.6(1.3)
8.2(1.6)
6.1(1.6)
7.5(2.2)
3.8(0.9)
4.7(1.1)
ODIN=
14.7(1.9)
17.4(2.2)
8.9(1.3)
10.8(1.4)
9.1(0.8)
12.3(1.)
Moha. (sum)
15.3(1.5)
18,4(1.8)
15.6(1.5)
18.9(2.1)
15.1(22)
18.5(2.9)
Moha. (adv)
14,3(1.5)
17.5(2.0)
19.1(15.8)
24.1(27.6)
16.1(17)
19.6(2.6)
PAD
16.3(1.5)
19.2(1.9)
17.5(1.3)
20.5(1.6)
11.0(1.1)
12.9(1.2)
cation accuracy. To simulate multiple types of image deterioration, we employ the method and code for generating image corruption developed by Hendrycks & Dietterich (2019). It can generate 19 types of image corruptions, each of which has tive levels of severity.
We consider two classification datasets/tasks, Food-A (i.e., 46 selected classes from Food-10 101d as explained in Sec. (32) and ImageNet (the original 1,000 object classification). For Food-A, we first train each network on the training split, consisting only of the original images. We divide the test split into three sets, 1,533, 1,533, and 1 1,534 images, respectively. The first one is used for DD as is (i.c., without corruption). We apply the image corruption method to the second and third sets. To be specific, splitting the 19 corruption types into 6 and 13, we apply the 6 corruptions to the second set to make D.'s 's, and the 13 corruptions to the last to make Dj 's. As each corruption has five severity levels, there are 30(= 6 x 5) D. 's and 65 13 5) D, ss. The former is used for training f (precisely, 20 are used for training and 10 are for validation), and the latter is for evaluating f-
For ImageNet, we choose 5,000, 2,000, and 5,000 images from the validation split without overlap. We use them to make D., D.'s 's and D,'s 's, respectively. As with Food-A, we apply the 6 and 13 types of corruption the second and third sets, making 30 D.'s and 65 Dj's respectively.
For the evaluation of f. we calculate mean absolute error (MAE) and root mean squared error (RMSE) of the predicted етT over the 65 D,'s. We repeat this for 20 times with different splits of image corruptions (19 + 6 + 13), reporting their mean and standard deviation.
Table 3 shows the results for Food-A and ImageNet. (The accuracies of the original classification tasks of Food-A and ImageNet are reported in Table and Table i0 Appendix Bnd C) It is seen for both datasets that Cosine achieves the top-level accuracy irrespective of the training methods. For Food-A, using a pre-trained network boosts the performance for the confidence-based methods (i.e., from Baseline to ODIN"), resulting in that MC dropout performs the best; Cosine attains almost the same accuracy. On the other hand, Mahalanobis and PAD do not perform well regardless of the datasets and training methods. This well demonstrates the difference between detecting the distributional shift p(x) + p' (a:) and detecting the deterioration of classification accuracy. We show scatter plots of S vs. eir in Fig. and in Appendix C which provides similar, even clearer, observation.
3.3.3 OFFICE-31
To study another type of domain shift, we employ the Office-31 dataset (Saenko et al. 2010). which popular in the study of domain adaptation. The dataset consists of three subsets, Amazon, DSLR, and Webcam, which share the same 31 classes and are collected from different domains. We train our CNNs on Amazon and evaluate the compared methods in terms of prediction accuracy of clas- sification errors for samples in DSLR and Webcam. The classification accuracy of the CNNs on Amazon is provided Table II in Appendix D
To obtain D,'s 's for training f. we employ the same image corruption methods Sec. 3.3.2 we apply them to Amazon samples to create virtual comain-shifted samples. The effectiveness of modeling the true shifted data, i.e., DSLR and Webcam, with these samples is unknown and needs to be experimentally validated. If this works, it will be practically useful. Specifically, we split the test splits of Amazon containing 754 images evenly into sets. We use one for D, and the other for
7

---

Under review as a conference paper at ICLR 2021
Table 4: Errors of the predicted classification error by the compared methods on 50 sample subsets of DSLR and Webcam. The CNN is trained on Amazon and the regressor f is trained using corrupted images of Amazon.
Train mein-from--scateee
Fine-tume tinettuning
Method
MAE
RMSE
MAE
RMSE
Baseline
12.1(3.1)
14.6(3.1)
10.6(2.3)
11.7(2.3)
Calib.
9.5(3.2)
11.5(3.4)
9.7(2.3)
10.8(2.3)
MC dropout
8.0(1.9)
10.1(2.7)
9.3(1.7)
10.5(1.7)
Cosine
5.6(1.5)
6.8(1.7)
8.5(2.0)
10.0(2.))
ODIN*
7.3(2.5)
8.1(2.6)
13.4(3.6)
15.3(3.6)
Maha. (sum)
18.8(4.7)
20.7(3.9)
7.9(2.0)
9.7(2.1)
Maha. (adv)
34.1(15.8)
40.0(22.5)
8.2(2.1)
9.9(222)
PAD
10.6(2.2)
12.1(2.6)
16.8(3.4)
18.3(3.2)
creating D.'s. We apply all the types of corruption, yielding 95(= 19 x 5) D ss We then split them into those generated by four corruptions and those generated by the rest; the latter used for training f. and the former is used for the validation. We iterate this for 20 times with different random splits of the corruption types, reporting the average over 20 x trials, there are three CNN models trained from different initial weights.
To evaluate each method (i.e., f based on a OOD score), we split DSLR and Webcam into subsets containing 50 samples, yielding 18 D2 's in total. We apply f to each of them, reporting the average error of predicting classification errors. Taole the results. well in both training methods. The two variants of Mahalanobis show good performance when using a pre-trained model, but this may be better considered a coincidence, as explained below. Figure shows the scatter plots of OOD score classification error for each method. The green dots indicate D. 's, corrupted Amazon images used for training f. and the blue ones indicate D; 's, subsets from DSLR and Webcam containing 50 samples each. For the method for which the green dots distribute with narrower spread, the regressor I will yield more accurate results. Thus, is from Fig. 3 that both Mahalanobis's tend have large spread, meaning that they could perform poorly depending on incoming domain-shifted data. Cosine and MC dropout have narrower spread, confirming their performance Table 4 Other results for DSLR and Webcam subsets with a different number of samples are provided in Appendix D
observed that Cosine works
Baceline Inn
Calla
MC drapout
Cosine
ODIN
Maha. (Sumi
Maha. [Adv]
PAD
From
Error
23
-108-C15-399-33
66
3xc
Fime-unning
-26
-235
-015
4.022
-30235
S/c 704
06
-22
66
0OD score
Figure 3: OOD score the true classification error for 95( 19 5) Do's (comrupted Amazon im- ages used for training the regressor f: in green), 18 D, 's (subsets of DSLR and Webcam containing 50 samples each; in blue), and D, (original Amazon images; in red).
3.4 ANALYSES OF THE RESULTS
We can summarize our findings in the following. Using pre-trained network has shown im- provements in all the scenarios, confirming the report of Hendrycks et al. (2019). ii) The detector using cosine similarity consistently works well throughout the three scenarios. The method will be the first choice if it is acceptable to modify the network's 's final layer. iii) The Mahalanobis detector, SOTA method, works well only for irrelevant input detection. This not contradictory with the
8

---

Under review as a conference paper at ICLR 2021
previous reports, since they employ only this very scenario. The method fits Gaussian distribution to ID samples belonging to each class and uses the same covariance matrix for all the classes. This strategy might work well on easy cases when incoming OOD samples are mapped distantly from the Gaussian distributions. However, such a simple modeling method will not work in more challeng- ing cases. For instance, incoming OOD samples could be mapped near the IID distributions, in novel class detection. In such cases, the ID sample distribution needs to be very precisely modeled, for which the assumption of Gaussian distributions with a single covariance matrix inadequate. iv) Domain shift detection requires detecting classification accuracy deterioration, not detecting a distributional shift of inputs, contrary to its name. This theoretically favors the confidence-bai based methods; they (particularly MC dropout) indeed work well, when used with pre-trained network. However, the Mahalanobis detector is more like an anomaly detection method, although its similar- ity with softmax classifier is suggested in (Lee et al. 2008)). An input sample for which the network can make a correct classification can be detected as an "anomaly' by the Mahalanobis detector.
4 RELATED WORK
Many studies of OOD detection have been conducted so far, most of which are proposals of new methods; those not mentioned above include (Vyas et al. [018 Yu & Aizawa 2019: [Sastry & Oore, 2020 Zisselman & Tamar 2020; Yu et al. 2020). Experimental evaluation similar to our study but on the estimation of the uncertainty of prediction is provided in (Ovadia et al. 2019).
In (Hsu et al. 2020), the authors present a scheme for conceptually classifying domain shifts in two axes, semantic shift and non-semantic shift. Semantic shift (S) represents OOD samples coming from the distribution of an unseen class, and non-semantic shift (NS) represents to OOD samples coming from an unseen domain. Through the experiments using the DomainNet dataset (Peng al. 2019), they conclude that OOD detection is more difficult in the order of S > NS > S+NS.
In this study, we classify the problems into three types from an application perspective. One might view this as somewhat arbitrary and vague. Unfortunately, Hsu et al.'s scheme does not provide help. For instance, according to their scheme, novel class detection S, and domain shift is NS. However, it is unclear which to classify irrelevant detection between S and S+NS. Moreover, their conclusion (i.c., S > NS > S+NS) does not hold for our results; the difficulty depends on the closeness between classes and between domains. After all, we think that only applications can determine what constitutes domain and what constitutes classes. Further discussion will be left for future study.
As mentioned earlier, the detection of domain shift in the context of deep learning has not been well studied in the community. The authors are not aware of study for image classification and find only a few Elsahar & Gallé (2019) even when looking at other fields. On the other hand, there are a large number of studies of domain adaptation (DA); (Ganin & Lempitsky 2015) Tzeng al. 2017] Zhang et al.) 2017: Toldo et al., 22020] Zou et al. 2019) to name a few. It is to make a model that has learned a task using the dataset of a particular domain adapt to work on data from a different domain. Researchers have been studied several problem settings, e.g., closed-set, partial, open-set set, and boundless DA (Toldo et al.] 2020). However, these studies all assume that the source and target domains are already known; no study considers the case where the domain of incoming inputs is unidentified. Thus, they do not provide a hint of how to detect domain shift.
5 SUMMARY AND CONCLUSION
In this paper, we first classified OOD detection into three scenarios from an application perspective, i.e. irrelevant input detection, novel class detection, and domain shift detection. We have presented a meta-approach to be used with any OOD detection method to domain shift detection, which has been poorly studied in the community. We have experimentally evaluated various OOD detection methods on these scenarios. The results show the effectiveness of the above approach to domain shift several as well as several findings such as which method works which scenario.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems, 2007.
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 mining discriminative com- ponents with random forests. In Proceedings of the European Conference on Computer Vision, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the Conference on Computer Vision and Pattern Recognition, 2009.
Hady Elsahar and Matthias Gallé. To annotate not? predicting performance drop under domain shift. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing, 2019.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model un- certainty in deep learning. In Proceedings of the International Conference on Machine Learning, 2016.
João Gama, Indré Žliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey concept drift adaptation. ACM computing surveys (CSUR). 46(4):1-37, 2014.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In
Proceedings of the International Conference on Machine Learning, 2015.
Thomas Mosgaard Giselsson, Rasmus Nyholm Jørgensen, Peter Kryger Jensen, Mads Dyrmann, and Henrik Skov Midtiby. A public image database for benchmark of plant seedling classification algorithms. 1711::171..0544 2017.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Proceedings of the International Conference on Machine Learning, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. Proceedings of the Conference on Computer Vision and Puttern Recognition, 2016.
Kaiming He, Ross Girshick, and Piotr Dollár. Rethinking imagenet pre- pre-training. In Proceedings of the International Conference Computer Vision, 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness. to common cor- ruptions and perturbations. Proceedings of the International Conference on Learning Repre- sentations, 2019.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of out-rf-distribution examples in neural networks. Proceedings of the International Conference on Learning Rep- resentations, 2017.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In Proceedings of the International Conference on Machine Learning, 2019.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of- distribution image without learning from out-of-distribution data. In Proceedings of the Confer- ence on Computer Vision and Pattern Recognition, 2020.
Eyke Hüllermeier and Willem Waegeman. Alcatoric and epistemic uncertainty in machine learning: A tutorial introduction. arXiv:19100004457 20119
Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine- grained image categorization: Stanford dogs. In Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop Fine- Grained Visual Categorization, 2011.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fne-grained categorization. In Proceedings of the International Workshop on 3D Representation and Recog- nition, 2013.
10

---

Under review as a conference paper at ICLR 2021
Balaji Lakshminarayanan. Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Process- ing Systems, 2018
Zhizhong Li and Derek Hoiem. Improving confidence estimates for unfamiliar examples. In Pro- ceedings of the Conference on Computer Vision and Pattern Recognition, 2020.
Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the reliability out-of-distribution image detec- tion in neural networks. In Proceedings of the International Conference on Learning Representa- lions, 2017.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Process- ing Systems, 2019.
Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar. Cats and dogs. In Proceedings of the Conference on Computer Vision and Puttern Recognition, 2012.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi- source domain adaptation. In Proceedings of the International Conference on Computer Vision, 2019.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Proceedings of the European Conference on Computer Vision, 2010.
Chandramouli S Sastry and Sageev Oore. Detecting out-of-distribution examples with gram matri- ces. In Proceedings of the International Conference on Machine Learning, 2020.
Alireza Shafaei, Mark Schmidt, and James J. Little, A less biased evaluation of outto--distribution sample detectors. Proceedings of the British Machine Vision Conference, 2019.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way prevent neural networks from overfitting. The journal of machine learning research, 15()):192-1158, 2014.
Engkarat Techapanurak. Suganuma Masanori, and Takayuki Okatani. Hyperparameter-ffre out-of- distribution detection using softmax of scaled cosine similarity. arXiv:1905.1063 2019.
Marco Toldo, Andrea Maracani, Umberto Michieli, and Pietro Zanuttigh. Unsupervised domain adaptation in semantic segmentation: a review. arXiv:2005. 10876, 2020.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the Conference Computer Vision and Pattern Recognition, 2017.
Apoorv Vyas, Nataraj Jammalamadaka, Xia Zhu, Dipankar Das, Bharat Kaul, and Theodore Willke. Out-of-distribution detection using ensemble of self supervised leave-out classifiers. In Proceedings of the European Conference on Computer Vision, 2018
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florain Schroff, Serge Belongie, and Pietro Perona. Caltech-UCSD Birds 200. Technical report, California Institute of Technology, 2010.
Qing Yu and Kiyoharu Aizawa. Unsupervised out-of-distribation detection by maximum classifier discrepancy. In Proceedings of the International Conference on Computer Vision, 2019.
Sehun Yu, Dongha Lee, and Hwanjo Yu. Convolutional neural networks with compression com- plexity pooling for out-of distrribubt image detection. In Proceedings of the International Joint Conference Artificial Intelligence, 2020.

---

Under review as a conference paper at ICLR 2021
Yang Zhang, Philip David, and Boqing Gong. Curriculum domain adaptation for semantic seg- mentation of urban scenes. In Proceedings of the International Conference on Computer Vision, 2017.
Ev Zisselman and Aviv Tamar. Deep residual flow for out of distribution detection. In Proceedings of the Conference on Computer Vision and Pattern Recognition, June 2020.
Yang Zou, Zhiding Yu, Xiaofeng Liu, BVK Kumar, and Jinsong Wang. Confidence regularized self-training. Proceedings of the International Conference on Computer Vision, 2019.
12

---

Under review as a conference paper at ICLR 2021
A DETECTION OF IRRELEVANT INPUTS
A.1 ADDITIONAL RESULTS
In our experiment for irrelevant imput detection, using five datasets, consider every pair of them, one for ID and the other for OOD. In the main paper, we reported only the average detection accuracy over four such pairs for an ID dataset. We report here the results for all the ID-OOD pairs. Tables S 6nd show the performance of the compared methods for training from scratch and for fine-tuning of pre-trained network.
Table 5: The OOD detection performance (AUROC) for networks trained from scratch. DI=Dog. D2=Plant, D3=Food, D4=Bird, and D5=Cat.
In In-Distribution
Method
OOD
D1
D2
D3
D4
D5
78.1(1.1)
88.9(0.7)
59.2(1.0)
61.4(3.9)
D2
76.1(12.4)
63.6(4.5)
32.7(5.6)
70.8(21.))
Buseline
D3
65.5(4.2)
75.9(4.6)
54.1(1.2)
56.3(4.4)
D4
72.1332)
74.8(2.1)
88.0(0.8)
61.2(4.4)
D5
73.4(2.4)
79.5(5.4)
93.5(1.0)
53.0(0.9)
DI
76.1(1.3)
91.6(0.6)
65.1(0.6)
67.2(3.5)
83.2(12.2)
65.3(5.3)
39.6(3.9
76.4(20.3)
Calib,
D3
70.4(5.4)
73.6(7.6)
58.2(1.5)
60.3(5.6)
D4
78.9(3.4)
72.8(1.)
90.8(0.8)
66.5(4.5)
D5
81.43.0)
77.7(6.4)
95.7(0.8)
54.4(1.7)
DI
82.6(2.8)
89.3(0.2)
61.2(1.1)
67.0(1.9)
D2
82.509.9)
67.4(7.7)
43.8(7.4)
84.4(8.8)
MC dropout
68.6(1.6)
84.7(2.6)
57.1(1.6)
61.3(2.8)
73.7(1.6)
79.0(2.0)
88.3(044)
67.3(1.4)
D5
75.9(2.4)
84.6(4.6)
94.7(0.0)
57.0(0.7)
93.9(2.2)
96.9(0.2)
74.8(0.5)
89.7(2.3)
D2
96.6(1.8)
94.1(0.5)
51.1(13.9)
98. ((1.4))))
Cosine
D3
85.0(1.4)
94.4(2.5)
78.8(1.9)
81.7(4.7)
D4
90.9(0.7)
94.1(1.4)
97.6(0.1)
90.6(2.7)
92.5(0.5)
96.3(0.9)
99.5(0.0)
73.6(2.5)
55.8(16.6)
74.2(0.9)
54.5(2.1)
63.5(2.1)
78.4(3.1)
55.2(9.2)
58.6(11.8)
76.5(6.6)
ODIN*
D3
69.3(6.1)
54.9(21.9)
62.5(3.0)
68.7(2.0)
D4
82.3(2.4)
48.9(15.1)
82.3(0.6)
70.3(1.5)
D5
81.2(3.8)
57.0(18.1)
89.6(0.7)
51.6(2.7)
D1
99.7(0.1)
68.6(0.8)
67.5(1.8)
63.6(3.0)
D2
87.8(3.4)
97.8(0.6)
75.7(4.5)
92.7(5.9)
Maha. (sum)
73.2(2.7)
99.8(0.1)
87.1(1.4)
78.4(2.8)
D4
75.9(1.2)
99.7(0.1)
87.2(0.8)
76.2(2.0)
D5
53.9(2.7)
99.8(0.1)
63.9(2.6)
83.2(1.8)
DI
80.9(10.8)
65.6(3.1)
59.8(3.6)
35.2(8.2)
D2
67.9(6.0)
98.4(0.4)
58.2(8.5)
51.3(23.0)
Maha, (adv)
D3
41.0(4.2)
87.0(8.2)
59.2(3.5)
39.5(9.7)
D4
26.2(3.3)
87.3(6.1)
71.0(2.4)
33.5(15.2)
D5
35.5(6.6)
70.9(19.4)
41.1(6.6)
74.2(9.7)
13

---

Under review as a conference paper at ICLR 2021
Table 6: The OOD detection performance (AUROC) for fine-tuned networks from pre- pre-trained model. Dl=Dog, D2=Plant, D3=Food, D4=Bird, and D5=Cat.
In In-Distribution
Method
OOD
DI
D2
D3
D44
D5
D1
96.3(0.4)
91.4(0.5)
96.4(0.6)
99.3(0.5)
D2
100.0(0.0)
54.3(8.2)
96.9(2.8)
87.4(4.1)
Baseline
D3
99.7(0.0)
96.1(0.7)
99.2(0.2)
97.1(0.6)
D4
99.0(0.2)
97.2(0.4)
91.1(0.5)
97.6(0.6)
D5
100.0(0.0)
95.2(1.5)
95.9(1.2)
99.8(0.1)
D1
97.4(0.4)
92.3(0.5)
94.2(0.9)
98.4(1.1)
D2 100.0(0.0)
55.0(8.3)
92.9(6.0)
73.0(6.2)
Calib.
D3 99.4(0.1)
97.1(0.6)
98.2(0.5)
94.2(1.1)
D4 98.1(0.5)
98.1(0.3)
91.8(0.5)
94.7(1.1)
DS
100.0(0.0)
96.9(1.1)
96.4(1.1)
99.7(0.1)
D1
95.7(1.8
92.7(0.5)
96.8(0.4)
99.6(0.2)
D2
100.0(0.0)
61.4(7.3)
99.3(0.4)
88.0(7.1)
MC dropout
D3
99.7(0.0)
96.1(0.9)
99.3(0.2)
97.9(0.8)
D4
98.9(0.1)
97.2(0.7)
91.5(0.7)
98.7(0.5)
D5
100.0(0.0)
96.4((.1)
97.1(0.7)
99.9(0.0)
D1
99.3(0.5)
96.3(0.0)
97.5(0.2)
99.4(0.0)
D2
99.5(0.2)
86.7(6.1)
100.0(0.0)
98.6(0.4)
Cosine
D3 99.5(0.1)
99.5(0.2)
99.4(0.1)
99.0(0.3)
D4 99.5(0.0)
99.6(0.3)
96.0(..3)
99.1(0.2)
D5
99.8(0.0)
99.8(0.1)
99.2(0.2)
99.6(0.0)
DI
93.1(1.7)
86.2(0.7)
92.2(0.6)
99.8(0.1)
D2
99.6(0.2)
53.5(16.5)
98.0(0.9)
91.8(2.8)
ODIN*
D3 96.6(0.6)
92.7(3.3)
96.8(0.5)
98.2(0.2)
D4
97.2(0.6)
95.7(1.3)
90.0(0.3)
99.6(0.1)
D5
100.0(0.0)
98.6(0.1)
96.2(0.7)
99.5(0.2)
D1
100.0(0.0)
93.2(0.5)
99.0(0.1)
99.6(0.0)
D2
99.7(0.0)
97.5(0.2)
99.8(0.0)
99.9(0.0)
Maha, (sum)
D3
98.9(0.0)
100.0(0.0)
99.3(0.1)
99.8(0.0)
D4
97.4(0.1)
100.0(0.0)
96.4(0.1)
99.5(0.0)
D5
98.1(0.0)
100.0(0.0)
92.8(0.4)
99.1(0.0)
D1
99.1(1.0)
86.9(1.5)
98.2(0.1)
99.6(0.0)
D2
99.7(0.0)
82.7(6.2)
99.7(0.0)
100.0(0.0)
Maha. (adv)
D3
98.5(0.1)
99.0(1.0)
98.3(0.2)
99.8(0.0)
D4
92.2(1.1)
98.9(1.2)
92.7(1.1)
99.5(0.0)
D5
98.1(0.1)
99.6(0.4)
88.1(1.3)
98.9(0.1)
14

---

Under review as a conference paper at ICLR 2021
B DETECTION OF NOVEL CLASSES
B.1 CLASSIFICATION ACCURACY OF THE BASE TASKS
In our experiments for novel class detection, we employ two datasets, Dog and Food-A. Table shows the classification accuracy for each of them. It is seen that for Dog. using a pre-trained model boosts the accuracy. There is tendency similar that seen Table [ that Cosine outperforms oth- ers in training from scratch. For Food-A, using pre-trained model shows only modest improvement due to the availability sufficient number samples.
Table 7: Classification accuracy for the two tasks, Dog (20 dog breeds classification) and Food-A (46 food class classification), for which novel class detection is examined.
mrin-from-ssratch
Fine-tuning
Dataset
Standard
MC dropout
Cosine
Standard
MC dropout
Cosine
Dog
51.9(0.8)
51.1(0.8)
64.2(1.4)
95.7(0.1)
95.6(0.2)
96.0(0.1)
Food-A
83.4(0.3)
84.0(0.2)
83.6(0.1)
87.5(0.2)
87.5(0.2)
86.0(0.1)
B.2 ADDITIONAL RESULTS
one of the experiments explained Sec. 3.2, we use only dog classes from the Oxford-IIIT Pet dataset. We show here additional results obtained when using cat classes. Choosing nine from 12 cat breeds contained in the dataset, we train the networks on classification test novel class detection using the remaining three breed classes. In another experiment, Food-A for ID and Food-B for OOD. We report here the results for the reverse configuration. Table shows the classification accuracy of the new tasks. Table shows the performance of the compared methods on the novel class detection. A similar observation to the experiments of Sec. 3.2 can be made.
these nine breeds and
Table 8: Classification accuracy for Car (9 cat breed classification) and Food-B (47 food class clas- sification).
Random Init. MC dropout
Fine-Tuning MC dropout
Dataset
Standard
Cosine
Standard
Cosine
Cat
60.9(0.8)
57.8(1.2)
64.1(0.9)
89.5(0.3)
88.8(0.3)
88.9(0.5)
Food-B
83.6(0.3)
84.3(0.4)
83.7(0.3)
87,6(0.2)
87.4(0.1)
86.2(0.1)
Table 9: Novel class detection performance (AUROC) of the compared methods. The OOD samples for Cat and Food-B are the held-out 3 cat breeds and Food-A, respectively.
Cat
Food-B
Method
From-scratch.
Fine- Tune From-scratch Fine-Tune
Baseline
57.7(0.1)
72.1(1.9)
81.5(0.2)
84.2(0.2)
Calib.
58.6(0.4)
70.9(2.0)
82.5(0.2)
84.5(0.2)
MC dropout
57.6(1.3)
72.7(1.1)
82.0(0.2)
84.5(0.3)
Cosine
63.6(1.1)
73.7(2.1)
81.7(0.2)
84.8(0.3)
ODIN*
52.3(0.7)
72.6(0.7)
74.4(0.7)
73.6(0.3)
Maha. (sum)
43.9(0.5)
62.2(0.4)
51.8(0.2)
64.2(001)
Maha. (adv)
50.0(3.0)
61.1(2.1)
52.6(0.4)
52.8(9.8)
15

---

Under review as a conference paper at ICLR 2021
C DETECTION OF DOMAIN SHIFT (IMAGE CORRUPTION)
C.1 CLASSIFICATION ACCURACY ON IMAGENET
Table TO shows the accuracy of the three networks used by the compared OOD detection methods for 1,000 class classification of the ImageNet dataset. We use center-cropping at test time. The cosine network shows lower classification accuracy here.
Table 10: Classification accuracy on ImageNet for each network.
Dataset
Standard
MC drepout
Cosine
ImageNet 74.6(0.1) 75.5(0.3)
72.4(0.1)
C.2 SCATTER PLOTS OF OOD SCORE VS. CLASSIFICATION ERROR
In Sec. 3.3.2, we showed experimental results domain shift detection using Food-A. Given a set Dj of samples, each of the compared methods calculates OOD score 5 for it, from which the average classification eтror eIT over samples from Dj is predicted. Figure 4 shows seatter plots showing the relation between the OOD score S and the true classification enror for number of datasets (i.e., D2's). s). We have 95(= 19 x 5) such datasets, each containing images undergoing one of the combinations of 19 image corruptions and 5 severity levels. The method with narrower spread of dots should provide more accurate estimation. These scatter plots well depict which method works well and which does not, which agrees well with Table The holds true for the plots for ImageNet shown in Fig.
Swoire
trppout
Cochee
Mana (amm
Hata (Axy)
Fromacath
Frne-tiing Etror
-8.5 <056 -15
-07 -18 -0
-035 -000 -22
.120
4.000
16
-32
de
109
-6.6 -630 -125
-67 -151 -435
4.7 -038 -035
-0.0 +635 -335
4.01
355
586
156
-26
-25
8.5
-0.2
66
000 score
Figure 4: OOD score vs. classification error for 95( 19x5 datasets, i.e., D, and D, (corrupted Food-A images).
Swvelie
Di
MC trepout
Cecine
0394
Mona (Smm)
MATA (AS))
MAD
Error
48
40
24
40
46
de
48
46
-02
-155
-031
3800
4000
2504 500 7500
-1.
-05
-34
-3 60
000 score
Figure 5: OOD score vs. classification error for 95(= 19 datasets, i.e., D, and D, (corrupted ImageNet images).
16

---

Under review as a conference paper at ICLR 2021
D DETECTION OF DOMAIN SHIFT (OFFICE-31)
D.1 CLASSIFICATION ACCURACY OF THE BASE TASKS
Table IT shows the classification accuracy of the three networks used by the compared methods for the different domain datasets of Office-31. These networks are trained only on Amazon.
Table 11: The classification accuracy of the three networks trained on Amazon for Amazon, DSLR, and Webcam.
Fine-tuning Standard
Dataset
Standard
MC dropout
Cosine
MC dropout
Cosine
Amazon
63.0(1.3)
63,9(2.5)
67.1(1.8)
87.8(0.7)
87.1(0.1)
87.8(0.3)
DSLR
9.6(0.6)
7.8(1.6)
10,0(1.0)
77.1(1.))
78.9(1.3)
74.2(0.9)
Webcam
7.2(1.9)
6.8(1.2)
11.1(0.6)
73.8(1.3)
74.1(2.2)
67.8(0.3)
D.2 ADDITIONAL, RESULTS
As with the experiments on image corruption, we evaluate how accurately the compared methods predict the classification enror incoming datasets. D,'s 's. Table 4 and Fig. B show the error of the predicted classification accuracy and the scatter plots of the OOD score and the true classification accuracy, where 'D''s 's created by splitting DSLR and Webcam into containing 50 samples. We show here additional results obtained for D,'s 's created differently. Table 12 and Fig, 6 show the prediction errors and the scatter plots for D, containing 30 samples. Table and Fig, show those for Dj 's of 100 samples. Table 14 and Fig. g show those for using the entire DSLR and Webeam for Dj thus there are only two D.'s. 's. The standard deviations are computed for x 3 trials (20 for random splitting of corruption types for train/val and 3 for network models trained from random initial weights), as explained Sec. 3.3.3
Table Errors the predicted classification error by the compared methods on 30 sample subsets of DSLR and Webcam. The CNN is trained on Amazon and the regressor f is trained using corrupted images of Amazon.
Train-from-scrattee
Fine- tune-tuning
Method
MAE
RMSE
MAE
RMSE
Baseline
13.3(2.2)
17.5(2.7)
10.9(2.0)
12.8(2.2)
Calih.
9.8(2.4)
12.6(2.7)
9.9(2.1)
11.9(2.3)
MC dropout
9.7(2.4)
12.1(3.5)
10.0(1.6)
11.7(1.6)
Cosine
6.7(1.2)
8.3(1.5)
9.5(1.8)
11.5(1.))
ODIN*
8.1(2.4)
9.3(2.6)
13.5(3.6)
16.1(3.8)
Maha. (sum)
19.5(3.8)
22.0(2.9)
9.1(1.9)
11.2(2.0)
Maha. (adv)
33.6(15.7)
40.0(22.5)
9.2(1.8)
11.3(1.9)
PAD
13.1(2.9)
14.7(3.2)
13.9(2.3)
16.2(2.3)
17

---

Under review as a conference paper at ICLR 2021
Calla
MC dropout
(Sum)
[Adv]
from sommtcrtte
Fine Error
45 -5 -4
-5
45
43
41
-006
4,000
30339
1000
-L0 -035 -15---35--2
-34
-11 06
3x
-5 25
-24
-035 -329 -015
2,006
-003
19:
59
-08
-02
0OD score
Figure 6: OOD score vs. classification error for 9- 19 x 5) Do's (corrupted Amazon images used for training the regressor f: green), D,'s 's (subsets DSLR and Webcam containing 30 samples each; in blue), and D, (original Amazon images; in red).
Table 13: Errors of the predicted classification error by the compared methods 100 sample subsets of DSLR and Webcam. The CNN is trained on Amazon and the regressor f is trained using corrupted images of Amazon.
frain-from--cratchh
Fine-tuning
Method
MAE
RMSE
MAE
RMSE
Baseline
11.1(3.2)
12.9(3.4)
10.2(2.7)
11.0(2.6)
Calih.
8.7(3.2)
10.4(3.7)
9.2(2.7)
10.0(2.6)
MC drooou
6.9(2.5)
8.2(2.9)
8.8(1.9)
9.7(2.0)
Cosine
4.6(1.6)
5.5(1.7)
7.9(2.1)
9.0(2.3)
ODINs
6.8(3.1)
7.3(311
13.4(3.8)
15.1(3.7)
Maha. (sum)
18.3(4.7)
19.9(3.9)
7.4(2.4)
8,5(2.4)
Maha. (adv)
33.6(15.3)
39.1(22.0)
7.7(2.2)
8.7(222)
PAD
8.6(2.3)
9.3(2.2)
18.7(3.7)
19.3(3.7)
Baselnn
Callb
MC dropout
Cosine
ODIN
Maha. [Sum)
Maha. [Adv]
PAD
Fime Error
da
66
369
-31
-04
-8
45 -4 203 -2
-015
4.026 .0035
53c 174
04
-04
-32
06
Figure 7: OOD classification error for 95(= 19 5) Do's (corrupted Amazon images used for training the regressor f: in green), D 's (subsets of DSLR and Webcam containing 100 samples each; in blue), and D, (original Amazon images; in red).
18

---

Under review as a conference paper at ICLR 2021
Table 14: Errors of the predicted classification error by the compared methods on the entire set of DSLR and Webcam. The CNN trained on Amazon and the regressor trained using corrupted images of Amazon.
Train-from-scratch
Fine-tuning
Method
MAE
RMSE
MAE
RMSE
Baseline
9.1(2.1)
10.4(2.5)
10.0(2.1)
10.1(2.1)
Calih.
8.3(2.9)
9.6(3.5)
9.0(2.3)
9.0(2.2)
MC drpoou
6.9(2.9)
7.8(.5)
9.2(1.4)
9.2(1.4)
Cosine
4.0(1.8)
4.6(1.8)
7.5(2.2)
7.5(2.2)
ODIN
6,5(3.0)
6.7(3.0)
12.7(3.5)
14.1(3.7)
Maha, (sum)
17.8(4.8)
19.2(3.7)
7.5(1.9)
7.6(1.9)
Maha. (adv)
27.7(14.0)
32.9(20.9)
7.7(199
7.9(1.9
PAD
6.3(1.9)
6.5(1.8)
19.1(2.5)
19.3(2.6)
Base
Calib
MC drapout
Cosine
ODIN'
Maha. [Sum]
Maha. [Adv]
PAD
Toomscratch
Error
45 -M
4.25
.0023
1000
-Los -0.1 -253-
6o
Tine treetuning
36
-33
-6
-66
24
45
-3
28
45
44
-05 -32
-025
4,006
-3033
25c 53c
12c
-10
-35
04
-04
-32
06
0OD
Figure 8: OOD score vs. classification error for 95(- 19 x 5) D.'s (corrupted Amazon images used for training the regressor f; green), D, 's (the entire set of DSLR and Webcam; blue), and D, (original Amazon images; in red).
19

---

Under review as a conference paper at ICLR 2021
E EFFECTIVENESS OF ENSEMBLES
An ensemble of multiple models is known to performs better than MC-dropout we considered in the main experiments for estimation of uncertainty etc. also known be better approximation to Bayesian networks. Thus, we experimentally evaluate ensembles. We consider an ensemble of five models and train each model in two ways, i.e.. "from-scratee" and "fine-tuning"" " We randomly initialize all the weights of each model for the former. We initialize the last layer randomly and other layers with the pre-traine preitrained model's 's weights for the latter. We evaluate ensembles for Baseline and Cosine. Tables 15 T6 17 and 18 show the results for the three scenarios. In the tables, "(con.)" means confidence is used as an ID score, or equivalently, negative confidence used as an OOD score. "(en.)" means the entropy is used as OOD score.
can observe the following from tables:
An ensemble of models performs better than single model. This is always true for Base- line. The same is true for Cosine except for domain shift detection. (The reason is not clear.)
An ensemble of Baseline models still performs lower than single Cosine model for most cases. It sometimes shows better performance for fine-tuned models, but the margin is small.
Using entropy OOD score tends to show slightly better performance than using confi- dence.
We conclude that Cosine's 's superiority remains true even when we take ensembles into consideration.
Table 15: Irrelevant input detection performance of the ensemble models.
Method
From-scratch
Fine-Tune
Baseline (con.)
61.4(12.1)
97.7(3.1)
Ensemble (con.)
67.8(13.7)
98.3(2.2)
Baseline (en.)
64.8(13.7)
99.2(0.9)
Ensemble (en.)
73.4(15.3)
99.5(0.5)
Cosine
83.9(11.4)
99.0(0.7)
Ensemble cosine
85.7(12.9)
99.1(0.7)
Table 16: Novel class detection performance of the ensemble models.
Dog
Food-A
Method
Prom-scratch Fine- Tune From-scratch
Fine-Tune
Baseline (con.)
61.1(0.8)
88.7(1.)
82.5(0.1)
84.6(0.2)
Ensemble (con.)
64.7
89.5
84.3
86.0
Baseline (en.)
61.6(0.7)
90.0(1.0)
83.3(0.1)
85.4(0.2)
Ensemble (en.)
65.7
90.8
85,0
86.8
Cosine
68.8(1.3)
(0.1(0.8)
83.7(0.1)
85.7(0.3)
Ensemble cosine
72.0
94.4
85.2
86.8
20

---

Under review as a conference paper at ICLR 2021
Table 17: Errors of the predicted classification error by the ensemble models.
Food-A (From-scratch)
Food-A (Fine-tuning)
ImageNet
Method
MAE
RMSE
MAE
RMSE
MAE
RMSE
Baseline (con.)
15.8(3.0)
20.5(3.7)
6,4(1.3)
7.9(1.6)
4.6(0.8)
6.3(1.0)
Ensemble (con.)
12.9(2.3)
17.1(2.4)
5.6(1.3)
7.0(1.8)
4.0(0.8)
5.5(1.1)
Baseline (en.)
16.8(3.2)
21.6(3.6)
6.6(1.0)
8.4(1.3)
4.7(0.8)
6.7(1.1)
Ensemble (en.)
14.6(2.0)
19.3(2.5)
6.0(0.9)
7.5(1.3)
3.9(0.4)
5.7(0.6)
Cosine
6.6(1.3)
8.2(1.6)
6.1(1.6)
7.5(2.2)
3.8(0.9)
4.7(1.1)
Ensemble cosine
7.3(1.3)
9.0(1.4)
6.4(1.6)
8.0(2.2)
4.2(1.0)
5.2(1.3)
Table 18: Errors of the predicted classification error by the ensemble models on 50 sample subsets of DSLR and Webcam.
Train-frome from-scratch
Fine-tuning
Method
MAE
RMSE
MAE
RMSE
Baseline (con.)
12.1(3.1)
14.6(3.1)
10.6(2.3)
11.7(2.3)
Ensemble (con.)
10.4(2.2)
11.8(2.2)
9.0(1.1)
10.9(1.2)
Baseline (en.)
11.5(2.5)
12.7(2.4)
11.4(2.)
13.7(2.9)
Ensemble (en.)
11.2(2.2)
12.6(2.1)
7.5(1.0)
8.7(1.1)
Cosine
5.6(1.5)
6.8(1.7)
8.5(2.0)
10.0(2.))
Enxemble cosine
5.3(1.1
7.1(1.5)
8.6(1.6)
10.3(1.)
21

---

Under review as a conference paper at ICLR 2021
F ADDITIONAL DETAILS OF EXPERIMENTAL SETTINGS
F.1 TRAINING OF THE NETWORKS
As is mentioned in the main paper, employ Resnet-50 in all the experiments, For the optimiza- tion, we use SGD with the momentum set to 0.9 and the weight decay set to 10-4, The learning rate starts at 0.1, and then divided by 10 depending on the performance of the validation dataset.
To tnn-tune a pre- pre- trained network, we use the learning rate of 0.001 for the standard network and that with MC dropout. For network used with Coxine, we use the learning rate 0.001 to the backbone part and higher learning rate of 0.1 to the fully-connected layer; the weight decay for the fully-connected layer is set to following Techapanurak et al. (2019) and Hsu et al. (2020).
F.2 DATASETS
Table T9 shows the specification of the datasets used in our experiments. Note that modify some of the dataset and use them in several experiments. In the experiments of domain shift detection, we employed image corruption simulate/model domain shift. The example of the corrupted images are shown in Fig. 9
Table 19: Specifications of the datasets used in the experiments.
Dataset
# of classes
# of samples
Ave. image size
Used in
Dog Breeds
120
10,222
443 x 387
Scction 3.1
Plant Seeding
12
5,544
357 x 356
Section 3.1
101
101
101,000
496 475
Sections 3.1, 3.2, and 3.3.2
CUB-200
200
11,788
468
Section 3.1
Stanford Car
196
16,185
700 x 483
Section 3.1
Oxford-IIl IIT Pet
37
7,393
437 x 391
Section 3.2
ImageNet
1000
1,281,167
482 418
Section 3.3.2
Office-31
31
4,110
418 418
Section 3.3.3
Clean
Blastic Transform
Frost
Motion Blur
Frosted Glass Blur
Fog
Food-A
ImageNet
Amazon
Figure Examples the corrupted images obtained by applying different types of image corruption (Hendrycks & Dietterich 2019)
22


Reviews:
[{"review_id": "s-fdW8oAaMX", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n----------------\nThe goal of this paper is to evaluate methods for detecting out-of-distribution samples in a more comprehensive fashion than prior work. To this end, it distinguishes between three different cases of OOD samples:\n\n(1) detecting irrelevant inputs: defined in this case as being \"out of concern\" for a task of interest, e.g. when classifying dogs is the task of interest, irrelevant inputs would be images of food, plants, birds, and cars\n\n(2) detecting novel classes: e.g. if the task of interest is classifying 15 types of dog, a novel class would be an unseen breed of dog\n\n(3) detecting domain shift: that is detecting samples that belong to a known class but stem from a different generating distribution due to image corruptions (a la Hendrycks &  Dietterich, 2019)  or a different imaging setup/setting (a la Saenko et al., 2010)\n\nAn impressive number of experiments are carried out in which the same set of OOD detection methods are compared across these different scenarios. From the results, it is concluded that using a pre-trained network improves OOD detection (in line with previous results), and that not all methods perform well across the different settings, but one particular method does: cosine similarity (Techapanurak et al., 2019; Hsu et al., 2020).\n\nStrengths & Weaknesses:\n----------------------------------\nGiven the dizzying amount of papers continually being released in any ML subfield, any attempt to step back and evaluate recent methods on even footing are always welcome in my book and can provide more value than adding yet another method to the literature. I appreciate the effort to distinguish between different cases of OOD detection as well as the amount of experiments with a good selection of methods including some very recent ones.\n\nDespite the above, I have reservations about this paper:\n\nWhile there are a lot of experiments, the novel conclusions are sparse and very little space is afforded for a discussion of these: a single paragraph on page 8. It's a bit difficult to see the forest from the trees. \n\nLooking beyond the individual results, the key insight here to me is that there are different kinds of OOD detection and that it's important to consider these different settings when evaluating a method. However, Hsu et al. (CVPR '20) also distinguish between two settings in their evaluation: semantic and non-semantic shift. These results are unfortunately not discussed in this submission. To be clear, here the distinction is a bit more nuanced. The first two settings (1) \"irrelevant inputs\" and (2) \"novel classes\" correspond to different kinds of semantic shift (the latter to fine-grained classes, e.g. dog breeds), and the last setting \"domain shift\" corresponds to non-semantic shift. While I agree that (1) and (2) are real and distinct practical scenarios, I don't think either implies a specific degree of \"semantic closeness\". What is \"novel\" and what is \"irrelevant\" is application-dependent. I would not tie \"novel\" and \"irrelevant\" to certain sets of classes, but instead focus on the effect of \"semantic closeness\" on OOD performance as a more nuanced way of looking at semantic shift. \n\nNow a detailed discussion of the experiments themselves:\n\nThe experiments on \"detection of irrelevant inputs\" (3.1) confirms results from Hendrycks et al. (2019) that using a pre-trained network is better for OOD detection than training from scratch. Given the choice of datasets, the OOD detection is almost perfect (above 95%) except when the in-distribution is the food dataset. This leads me to question the choice of datasets, because I'm certain that OOD detection remains a difficult problem these almost perfect results notwithstanding.\n\nThe experiments on \"detection of novel classes\" (3.2) find that within the food dataset, four methods perform almost identically well (Baseline, Calib, MC dropout, Cosine), and outperform ODIN* and Mahalanobis. The conclusion is similar from the Dog experiment, except that Cosine performs the best by some margin. This contradicts the results in Hsu et al. 2020, where it is shown that both ODIN* and a variant of Mahalanobis significantly outperform Baseline across different settings. Here it is chalked up to the similarity between in-distribution and out-distribution images.\n\nThere are also a couple of problems with the experimental setting: The dataset of dog images contains 25 classes, some of which overlap with the Imagenet classes. This violates the OOD aspect. The authors explain that they \"intentionally leave this overlap to simulate a similar situation that could occur in practice\", but this really isn't convincing. In the case of the food dataset on the other hand, \"eight classes [out of 101] contained in the ImageNet dataset\" are removed. Perhaps it is inconvenient to retrain an ImageNet classifier without the offending dog classes, but then maybe another dataset should have been selected. This dataset contamination was not discussed in section 3.1, despite the use of the same dataset. Was this not an issue?\n\nThe final set of experiments on \"detection of domain shift\" (3.3) were confusing to me. I found the description of the problem formulation as well as the experimental setting (3.3.1, first part of 3.3.2) hard to followm especially the second paragraph of 3.3.1. Here also the goal is now to predict the classification error from the OOD scores, which departs from the setting considered for the first two sets of experiments. I would recommend cutting out section 3.3.2 and spending more time on analysing the results as I don't see what these experiments add to the paper.\n\nAlso, a new method is suddenly introduced (PAD*) and I'm not sure why. The way it's trained also appears to contradict the following statement in the introduction \"we follow the argument shared by many recent studies [...] that OOD detection methods should not assume the availability of explicit OOD samples at training time\". PAD* involves \"train[ing] a binary classifier using portions of D_s and D_t to distinguish the two\".\n\nConclusion:\n----------------\n\nA lot of effort went into conducting a large number of experiments with the goal of comparing several OOD detection methods across different scenarios. While I appreciate this effort and agree that it makes sense to consider these different scenarios when comparing methods, I don't think there are sufficiently novel insights to warrant acceptance with very little discussion of the results.\n\nEvaluating OOD detection on fine-grained vs. non-fine-grained classes is interesting, and I would have liked to see more exploration of this. Unfortunately, the choice of datasets for these experiments makes the task appear to be too easy in one case, and in the other case is not truly OOD due to overlap with ImageNet classes.\n\nGiven that this is an evaluation of existing methods with little analysis, I would have been more inclined to accept if this paper established a convincing benchmark that could be easily adopted by researchers in this area, but due to the aforementioned problems this is not really the case. Perhaps one could consider DomainNet (Peng et al. ICCV '19) as a dataset that would support such an analysis at different semantic levels together with the non-semantic component. Additionally, one could add the image corruptions proposed by Hendrycks & Dietterich.", "extra_scores": {}, "extra_text": {}}, {"review_id": "RylU-l1e3ky", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "#### Summary\nThis paper is about a comparison of methods for out of distribution detection on image classification. The authors compare along three different lines: Irrelevant inputs, novel classes, and domain shift. The results show that cosine similarity consistently outperforms other methods across all sub-tasks.\n\nThe comparison made in this work is novel, as most OOD detection methods are evaluated on standard academic datasets (CIFAR10/100, SVHN, etc), which could hide issues that are presented in more realistic scenarios.\n\n\n#### Reasons for Score\nI believe that this paper touches a subject of importance to practitioners and researchers, the performance of OOD detection methods in more realistic settings and with three different sub-tasks inside the OOD detection task.\nThe evaluations are made correctly, in a variety of datasets, ensuring the robustness of the conclusions that were made.\nMy only concerns with this paper are the lack of comparisons with ensembles, which could change which method works best.\n\n\n#### Pros\n- The evaluation is comprehensive, with a variety of datasets, properly selected, ensuring robust results and conclusions. There is also a variety of tasks inside out of distribution detection (new inputs, novel classes, and domain shift).\n- The evaluation produces strong conclusions, with one method (cosine similarity) clearly outperforming the rest. This provides good evidence for practitioners to select OOD detection methods in the future.\n- The evaluation is made with a practical point of view, motivated by real-world examples, that differ from purely academic benchmarks. This is very useful for practitioners.\n- A nice appendix that provides detailed results that can be used for future comparison.\n\n\n#### Cons\n- I believe there is a missing reference that is very relevant for this work, namely Ovadia et al. \"Can You Trust Your Model’s Uncertainty? Evaluating\nPredictive Uncertainty Under Dataset Shift\" NeurIPS 2019. Many methods evaluated in Ovadia et al. like ones relying on variational inference are not evaluated in this work, so I suggest to take a look and include additional baselines. Note that Ovadia et al. also defines corruption methods for evaluating dataset shift.\n- I think there are missing methods to compare, for example ensembles. The paper mentions that MC-Dropout \"is an approximation of BNNs that is computationally more efficient than en ensemble of networks\". I do not think this is true, ensembles and MC-Dropout are equally slow in predicting a forward pass, but an ensemble usually performs better in OOD detection and produces higher quality uncertainty (see Lakshminarayanan et al. 2017, Figure 3 and Ovadia et al. 2019). If MC-Dropout is used for comparison, then Ensembles should also be used since generally ensembles outperform MC-Dropout in terms of uncertainty quality. Note that this might change which method works better, or even create new baselines, as ensembling can be combined with cosine similarity OOD detection.\n\n\n#### Questions for Rebuttal Period\nWhy were ensembles not considered as one of the evaluated methods?\n\n\n#### Minor Issues\n- In Section 3.1, please mention on which dataset are models pre-trained, I assume it is on ImageNet, but this needs to be clearly specified.\n- Please mention in the Appendix that number of forward passes used in MC-Dropout\n- I think method should be separated into two groups, one where OOD detection capabilities are given by the quality of uncertainty (like Bayesian NNs, MC-Dropout, Ensembles, etc), and other methods where an score is produced specifically for OOD detection and are not direct measures of uncertainty (not probabilities). This could shed a light on why methods work so differently.\n- Sections 2.2 and 3.3.1 could be rewritten to clearly describe the OOD score produced by each method, for example in some cases the ID score is describe, how is the OOD score derived in these cases? This is important since an OOD score is used in Section 3.3.\n- For probability-based methods like MC-Dropout, entropy of the output probabilities can also be used as an OOD score.\n- In Figure 3, the x labels for Maha (Sum) are too close to each other and might hinder reading, I recommend to rewrite as 25K/50K/75K or rotate them a bit.\n- For more clarity and a self-contained work, the authors could add a summary description of each method in mathematical terms to the appendix.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "CxosSnRwWIN", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\n \nThe paper empirically analyzes the evaluation framework of the current OOD detection systems for the image recognition task, specifically the evaluation described in [1] using Max-softmax and calibrated confidence. They motivate the paper by the necessity of having better evaluation for OOD detection to be reflective of real world scenarios. The addressed problem is interesting and valuable for the field as many of the defined OOD datasets, and evaluation metrics may not cover many real-world scenarios. They specifically addressed three scenarios, inputs that i) are irrelevant to the task ii) are from novel classes and iii) are from another domain (domain shift), which for the first 2 scenarios, they only evaluate them as unseen classes and not distinguish between them. Based on my understanding of the paper, they compare 5 OOD detection methods from the literature, suggest a few test datasets/scenarios and conclude using cosine similarity is consistently favorable for evaluation, and the choice of using confidence-based methods in case of domain shift detection scenarios.\t\t\t\t\t\n\n##########################################################################\t\n[Q]: questions, and comments on the paper. [S] strength [W] weakness\n\t\t\nQ1: Questions about the choices of the datasets:\n\n1.1. What are the reasoning for the choices of datasets? \n1.2. Why are these datasets considered reflective of real world scenarios? For example, why didn’t you analyze the models on some OOD datasets like Imagenet-A, -O, -P.\n\nQ2: Could you elaborate more on the specific contributions of the paper?\n\nQ3: What does “genuine OOD samples” mean in section 2.2?\n\nQ4: There is no related work part In the paper.\n\nQ5: Many recent and relevant papers are not cited or compared against. As an example, it is mentioned in the Domain shift detection [2] study in NLP, that is not the focus of the paper. However, there are multiple recent works in Computer Vision also addressing the domain or distributional shift in vision scenarios. In particular, for the example surveillance system scenarios mentioned in page 2, there are datasets and papers addressing the domain shift that should be cited such as [3 (a review paper), 4] or [5]. \n\nQ6: The following statement from the paper is inaccurate as a general statement. Which kind of pre-training contributes to performance improvement? In addition, there are many scenarios where pre-training or fine-tuning lead to reduced performance on seen classes or on the original data distribution. \n“(i) Using a pre-trained network always contributes to performance improvements, confirming the study of Hendrycks et al [1]. (2019). ”\n\nQ7: For the evaluation using fine-tuning, the impact of the results on the seen classes (the current classes in the dataset), are not re-evaluated. In many cases, fine-tuning in a new distribution, contributes to reduced performance in seen classes or former distribution. I recommend checking the literature of generalized zero-shot learning and specifically the use of harmonic mean as a measurement.\n\n##########################################################################\nReason for the score: \n\nS1: The paper is well-motivated and addresses an interesting problem. \n\nS2: I believe there is room for improvement.\n\nW1: The contributions of the paper are not clear. The ablation studies to compare the approaches are not \tsufficient. \n\nW2: I am not convinced the current set up can be sufficient for generalizable decisions on choice of evaluations.\n\nW3: Choices of the datasets for comprehensive guidance is unclear.\n\nW4: There is no related work part  in the paper and multiple recent and relevant papers are not being compared against or cited.\t\n\t\t\t\t\t\n##########################################################################\t\t \t \t \t\t\nReferences:\t\t\t\t\t\n[1] Hendrycks and Kevin Gimpel, A baseline for detecting misclassified and out-of-distribution examples in neural networks, ICLR 2017 \n\n[2] Elsahar et al, To annotate or not? predicting performance drop under domain shift, IJCNLP 2019.\n\n[3] Toldo et al, Unsupervised Domain Adaptation in Semantic Segmentation: a Review, CVPR 2020 \n\n[4] Zhang et al, Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes, CVPR 2017.\n\n[5] Ganin et al, Unsupervised Domain Adaptation by Backpropagation, ECCV 2016.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "0xTC88DhWJs", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper provides an empirical evaluation of various OoD methods . The authors distinguish three different operating scenarios:\n\n+ Irrelevant inputs: in this case, the inputs  simply need to be rejected.\n+ Novel classes: detecting inputs that are of the same overall category as that used in training, but from an unseen class of that category.\n+ Domain shift, where measurement artifacts and other disturbances cause a shift in the measured p(x) even though the underlying inputs belong to known classes\n\nMultiple methods are compared, using different image datasets to compare OoD detection performance using the AUROC as the evaluation metric.\n\nPros \n\n+ Paper is reasonably well written, and easy to follow\n+ Distinguishing different types of OoD scenarios is a clarifying aspect and useful in this line of work.\n+ Shows which methods work under what scenarios.\n\nCons\n+ The biggest concern with this type of work is lack of novelty.  The results are not surprising (as the authors themselves have noted). Fine-tuning/pre-training gives a strong performance boost for OoD detection (as shown in the work of Hendrycks et al 2019) and cosine similarity has been shown to be better than softmax for OoD detection (as shown in the work of Techapanurak et al. (2019); Hsu et al. (2020)).  Further, there is no real discussion or insights into why certain methods work in some scenarios and not others. This results in a rather underwhelming paper for  the reader, and while there are useful takeaways from the paper, as it stands, it does not meet the bar of ICLR acceptance. \n\nReferences\n* Dan Hendrycks, Kimin Lee, and Mantas Mazeika Using pre-training can improve model robustness\nand uncertainty. In Proceedings of the International Conference on Machine Learning, 2019.\n\n* Engkarat Techapanurak, Suganuma Masanori, and Takayuki Okatani. Hyperparameter-free out-ofdistribution detection using softmax of scaled cosine similarity. arXiv:1905.10628, 2019.\n\n* Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data. In Proceedings of the Conference on Computer Vision and Pattern Recognition, 2020.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an OOD evaluation framework under three categories: irrelevant input detection, novel class detection, and domain shift detection. As with several reviewers, the AC recognizes the importance and effort to distinguish between different cases of OOD detection, as well as the amount of experimental comparison across several prominent methods in literature (MSP, MC-dropout, cosine similarity, ODIN, Mahalanobis).  

Despite being well-motivated, three knowledgeable reviewers find the paper not ready yet for publication at ICLR. The AC recommends a rejection, given the standing major concerns from the reviewers. The AC is hopeful that the paper can be significantly improved by 

- sufficiently discussing and highlighting the novel insights of the results. 
- a more rigorous definition of  "novel" vs. "irrelevant" inputs. There seem to be overlapping definitions between what Hsu et al. considered vs. this paper. In particular,  Hsu et al distinguish i) samples of a novel class but in a known domain, called semantic shift (S), and ii) samples of known class in a novel domain, called non-semantic shift (NS), both of which are reconsidered in this paper. Therefore, the novelty of this submission is more precisely to distinguish within the category of semantic shift. The AC agrees that this might deem some more rigorous measurement and definition of "semantic closeness". 
- The AC also finds the evaluation of domain shift in Section 3.3.2 may be potentially misleading the community, as it falls out of the standard OOD scope. The notion of common corruption is closer to the robustness problem (which is how ML model predictions changes w.r.t some delta changes in the input space). The changes may not be substantial enough to be "out-of-distribution".  
Decision: Reject