Title: DETECTING OUT-OF-DISTRIBUTION SAMPLES USING LOW-ORDER DEEP FEATURE STATISTICS Anonymous

Abstract: The ability to detect when an input sample was not drawn from the training distribution is an important desirable property of deep neural networks. In this paper, we show that a simple ensembling of first and second order deep feature statistics can be exploited to effectively differentiate in-distribution and out-of-distribution samples. Specifically, we observe that the mean and standard deviation within feature maps differs greatly between in-distribution and out-of-distribution samples. Based on this observation, we propose a simple and efficient plug-and-play detection procedure that does not require re-training, pre-processing or changes to the model. The proposed method outperforms the state-of-the-art by a large margin in all standard benchmarking tasks, while being much simpler to implement and execute. Notably, our method improves the true negative rate from 39.6% to 95.3% when 95% of in-distribution (CIFAR-100) are correctly detected using a DenseNet and the out-of-distribution dataset is TinyImageNet resize. The source code of our method will be made publicly available.
Input pre-proc.

1. INTRODUCTION:

In the past few years, deep neural networks (DNNs) (Goodfellow et al., 2016) have settled as the state-of-the art-techniques in many difficult tasks in a plurality of domains, such as image classification (Krizhevsky et al., 2012) , speech recognition (Hinton et al., 2012; Han et al., 2018) , and machine translation (Bahdanau et al., 2014; van den Oord et al., 2017) . This recent progress has been mainly due to their high accuracy and good generalization ability when dealing with realworld data. Unfortunately, DNNs are also highly confident when tested against unseen samples, even if the samples are vastly different from the ones employed during training (Hendrycks & Gimpel, 2017) . Moreover, several works have shown that such deep networks are easily fooled by minor perturbations to the input (Goodfellow et al., 2014; Su et al., 2017) . Obtaining a calibrated confidence score from a deep neural network is a problem under continuous investigation (Hendrycks & Gimpel, 2017 ) and a major thread in artificial intelligence (AI) safety (Amodei et al., 2016) . In fact, knowing when the model is wrong or inaccurate has a direct impact in many production systems, such as self-driving cars, authentication and disease identification (Akhtar & Mian, 2018; Goswami et al., 2018) , to name a few. Guo et al. (2017) showed that despite producing significantly low classification errors, DNNs confidence scores are not faithful estimates of the true certainty. Their experiments confirmed that depth, width, weight decay, and batch normalization are the main reasons for overconfident scores. Moreover, they demonstrated that a simple and yet powerful method of temperature scaling in the softmax scores is an effective way to improve calibrate a DNNs confidence. While calibrating the classifier's output to represent a faithful likelihood from the training (in-distribution) data has effective solutions, the problem of detecting whether or not the samples are generated from a known distribution (out-of-distribution), is still an open problem (Hendrycks & Gimpel, 2017) .

One straightforward approach to calibrate the classifier's confidence in order to detect samples whose distribution differs from the training samples distribution is to train a secondary classifier that digests both in-distribution (ID) and out-of-distribution (OOD) data so that anomalies are scored differently from ID samples, as performed in Hendrycks et al. (2018) . Re-training a network, however, can be computationally intensive and may even be intractable, since the number of OOD samples is virtually infinite. Other solutions rely on training both classification and generative neural net-Table 1 : Summary comparison of the characteristics of recent related methods. Test complexity refers to the required number of passes over the network. Training data is the number of samples for which the methods were calibrated against (with all standing for the whole training set). AUROC is the area under receiver characteristic curve (detailed in Section 4). Performance shown is for DenseNet trained on CIFAR-100 and using TinyImageNet (resized) as OOD dataset.

works for OOD using a multi-task loss (Lee et al., 2018a) , or using unsupervised fully convolutional networks as done by Sabokrou et al. (2016) to detect OOD in video samples. All these methods, however, have a major drawback: they require re-training a modified model using a different loss function possibly with additional parameters, which increases the computational burden, and demands access to the entire original (and probably huge) training data.

In this work, we propose a novel OOD sample detection method that explores low-level statistics from feature layers. The statistics are obtained directly from the batch normalization layers (Ioffe & Szegedy, 2015) , requiring no extra computations during training time, no changes to the network architecture and loss functions, nor preprocessing of the input image. During test time, the proposed method extracts statistics from selected layers and combines them into an OOD-ness score via a linear classifier. Throughout this paper, we observe that the mean and standard deviation of a given channel in a layer differ greatly between ID and OOD samples, which naturally motivates their use as features to be employed by an OOD detector. By selecting the BN layers of a network, we are able to normalize the features according to the learned batch statistics. The effectiveness of the proposed method is validated in two state-of-the-art DNN architectures (DenseNet and ResNet) (Huang et al., 2017; He et al., 2015; 2016; Zagoruyko & Komodakis, 2016) that are trained for image classification tasks in popular datasets. The proposed approach achieves state-of-the-art performance, surpassing all competitors by a large margin in all tested scenarios, while being much more efficient. Notably, our method only requires one forward pass while Liang et al. (2018) ; Lee et al. (2018b) ; Vyas et al. (2018) require two forward and one backward passes.

The rest of the paper is organized as follows. Section 2 describes prior work on OOD samples detection. Section 3 introduces the proposed method, whereas Section 4 details all experiments and compares the results with state-of-the-art methods. Finally, we draw our conclusions in Section 5.

2. PREVIOUS WORK:

In this section, we describe recent prior work on OOD detection methods. A summary of all methods described can be seen in Table 1 . Hendrycks & Gimpel (2017) proposed a baseline method based on the posterior distribution (i.e. softmax scores). They showed that well-trained models tend to produce higher scores for ID samples than for OOD ones. Hence their method comprises of applying a threshold on the softmaxnormalized output of a classifier. If the largest score is below the threshold, then the sample is considered OOD. Liang et al. (2018) continued the aforementioned line of work and proposed the Out-of-Distribution detector for Neural networks (ODIN), which includes a temperature scaling T ∈ R + * to the softmax classifier as in Guo et al. (2017) . The authors in ODIN argued that a good manipulation of T eases the separation between in-and out-of-distribution samples. Allied to that, they also incorporated small perturbations to the input (inspired by Goodfellow et al. (2014) ) whose goal is to increase the softmax score of the predicted class. Liang et al. (2018) found that this kind of perturbation has a stronger effect on ID samples than on OOD ones, increasing the separation between ID and OOD samples. ODIN outperforms the baseline method (Hendrycks & Gimpel, 2017 ) by a fair margin; however, it is three times slower due to the two forward and one backward passes needed to preprocess the input, while (Hendrycks & Gimpel, 2017) only requires one forward pass. Vyas et al. (2018) describes a novel loss function, called margin entropy loss, over the softmax output that attempts to increase the distance between ID and OOD samples. During training, they partition the training data itself into ID and OOD by assigning samples labeled as certain classes as OOD and use the different partitions to train an ensemble of classifiers that are then used to detect OOD samples during test time. They also use the input pre-processing step proposed in Liang et al. (2018) , including temperature scaling. Lee et al. (2018b) is the most recent work on OOD detection that we have knowledge of. They show that the posterior distribution defined by a generative classifier (under Gaussian discriminant analysis) is equivalent to that of the softmax classifier, and the generative classifier eases the separation between in-and out-of-distribution samples. The confidence score is defined using the Mahalanobis distance between the sample and the closest class-conditional Gaussian distribution. They argue that abnormal samples can be better characterized in the DNN feature space rather than the output space of softmax-based posterior distribution as done in previous work (e.g., ODIN). Samples are pre-processed similarly as done in ODIN, but the confidence score is increased instead of the softmax one. To further improve the performance, they also consider intermediate generative classifiers for all layers in the network. The final OOD sample detector is computed as an ensemble of confidence scores, chosen by training a logistic regression on validation samples. This method also shows remarkable results for detection of adversarial attacks and for incremental learning.

3. PROPOSED SOLUTION:

An OOD detector should incorporate information from the training data in a natural manner, without being directly influenced by the loss function, which is intrinsically related to the task which could be well-defined for ID samples but be meaningless for most OOD samples. Moreover, if the OOD method is more dependent on the training distribution, it should be able to be applied to a wide variety networks, and not be designed specifically for a given architecture.

Conv Batch Norm Non-lin. input Conv Batch Norm Classifier output µ (1) c , σ (1) c X (1) X (L) µ (L) c , σ (L) c

Linear decision function

µ (l) c , σ (l) c stat.

m(1) , s(1)

stat. stat. m(l) , s(l) m(L) , s(L)

OOD-ness score

Figure 1 : An illustration of the complete proposed model. At each BN layer, we extract the input, normalize it using the running statistics, and compute the first and second order features. The outputs are fed to a linear decision function to predict if the input sample is out-of-distribution or not.

Our method is based on a very simple observation. Input samples with different distributions generate statistically distinct feature spaces in a DNN. In other words, the deep features of an ID sample are distinct from those of an OOD one. Moreover, when using a normalization scheme, such as BN, the features are normalized by the statistics of the ID batch during training, possibly leading to feature statistics that are more similar to the batch statistics, as depicted in Figure 2 .

The main problem then becomes how to summarize the feature space distribution for ID samples in a way that best discriminates between ID and OOD samples. In this work we show that using the first and second order statistics within each feature map performs remarkably well for this task.

Figure 1 shows the proposed OOD detector, highlighting the feature statistics that are extracted for each channel at each BN layer. These statistics are then combined through a learned linear decision function, providing a measure of OOD-ness. The way we compute the per-channel statistics Figure 2 : Visualization of the normalized BN latent space distribution for one ID sample (a), and one OOD sample (b) at BN layers l = 1, 5, 10, 15, 20, 25 from a WideResNet-28-10. We also show the normal distribution for comparison. In the mid layers, the (arbitrary) ID CIFAR-10 sample (a) is more similar to the batch stastitics than the (arbitrary) OOD TinyImageNet(r) sample (b). Bars: normalized feature histograms. Blue curve: KDE. Orange curve: N (0, 1) (always the same inside each column but scaled for visualization). More in Appendix A.

is formalized in Section 3.1. Finally, the linear classifier used to combine the statistics from the different layers is described in Section 3.2.

3.1. LOW-ORDER FEATURE STATISTICS:

In the previous section, we motivated that characterizing the feature-space distributions might lead to a robust OOD detector. As a first approach, one could model these distributions using a nonparametric method to estimate the distribution of the features for each channel, which requires the computation of sufficient statistics using the training data or using a parametric method to fit the distribution (Bishop, 2007) , which are both computationally intensive.

Here, we propose to use only the mean and standard deviation computed along the spatial dimension for each channel to summarize the per-channel distribution. As it will be shown, these two statistics are sufficient to distinguish between ID and OOD. Moreover, because the mean and standard deviation of each channel are normalized by the running mean and variance computed during training by the BN layers (Ioffe & Szegedy, 2015) , these statistics can be naturally combined within each layer to produce effective features for OOD detection. We describe such a procedure in what follows.

Given the l-th BN layer with input tensor X ∈ R C×W ×H , the output BN

EQUATION

where γ

(l) c > 0 and β (l)

c are the per channel per layer learned scaling and shifting parameters, > 0 is a small constant value for numerical stability, µ

(l) c ∈ R and [σ (l)

c ] 2 ∈ R + are the mean and variance estimated through a moving average using the batch statistics during training, and Z (l) ∈ R C×W ×H is the normalized feature tensor. It is worth noting that the statistics are calculated independently for each channel c at each layer l.

In this paper, we conjecture that low-order statistics computed from either X (l) c,i,j or Z (l) c,i,j can be used to discriminate between ID and OOD samples. Given the unnormalized input X 

EQUATION

and the features for the normalized tensor Z (l) c,i,j are defined as

EQUATION

i.e., the normalized mean feature m 

3.2. AGGREGATION OF FEATURE STATISTICS:

Intra-layers aggregation. The features derived from low-order statistics (equation 3) can be readily used to train a predictor for ID/OOD discrimination. Of course, if they were produced for every feature map in the network, this would result in a feature vector of very high dimension, typically tens of thousands. Instead, we propose to combine these features within each BN layer, so that in the end we obtain one pair of features per layer: average mean and average variance. Taking advantage of the fact that features are approximately normalized by BN's running statistics, we propose to simply average them for all channels within a layer. Thus, each layer yields the following features, for the normalized case:

EQUATION

)

where C l is the number of channels in layer l. Note that doing this aggregation amounts to computing the mean and standard deviation of all normalized features at the given layer. Using averages of the low-order statistics could lead to issues in deeper layers, where activations are in general concentrated over fewer number of channels. In this case, the mean of the statistics over channels might not be an appropriate data reduction function. Nevertheless, as we show in the experiments section, this did not impact the performance of the proposed method, but more investigation is warranted. 

Inter:



4. EXPERIMENTS:

In this section, we evaluate the effectiveness of the proposed method in state-of-the-art deep neural architectures, such as DenseNet (Huang et al., 2017) and Wide ResNet (Zagoruyko & Komodakis, 2016) , on several computer vision benchmark datasets: CIFAR (Krizhevsky, 2009) , TinyImageNet, a subset of ImageNet (Deng et al., 2009) , LSUN (Yu et al., 2015) , and iSUN (Xu et al., 2015) . We also use Gaussian noise and uniform noise as synthetic datasets. This evaluation protocol is the de facto standard in recent OOD detection literature (Hendrycks & Gimpel, 2017; Liang et al., 2018; Vyas et al., 2018; Lee et al., 2018b) . All experiments were performed on four models trained from scratch (each one initialized with a different random seed) for each architecture, to account for variance in the model parameters. The code to reproduce all results is publicly available1 .

4.1. SETUP:

Datasets: For backbone training, we use CIFAR-10 and CIFAR-100 datasets which have 10 and 100 classes respectively, both containing 50,000 images in the training set and 10,000 images in the test set. At test time, the test images from CIFAR-10 (CIFAR-100) are considered as ID (positive) samples. For OOD (negative) datasets, we test with datasets containing natural images, such as TinyImageNet resize and crop, LSUN resize and crop, and iSUN, as well as synthetic datasets, such as Gaussian/uniform noise, which is the same dataset setup as in Liang et al. (2018) ; Hendrycks & Gimpel (2017) . This is summarized in Table 2 . For all datasets, we did the validation/test set split following the procedure in Liang et al. (2018) . Just for reproducibility, 1000 samples from the test set are separated in a validation set used for fitting the logistic regressor and hyper-parameter tuning.

The remaining samples (unseen for both backbone model and OOD detector) are used for testing. 12) . For Wide ResNet, we also follow Liang et al. (2018) , with L = 28 and widen factor of 10 (WRN-28-10). All hyperparameters are identical to their original papers. All networks were trained using stochastic gradient descent with Nesterov momentum (Ruder, 2016) and an initial learning rate of 0.1. We train the DenseNet-BC-100-12 for 300 epochs, with batch size 64, momentum 0.9, weight decay of 10 -4 and decay the learning rate by a factor of 10 after epochs 150 and 225. We train the WRN-28-10 for 200 epochs, with batch size 128, momentum 0.9, weight decay 0.0005, and decay the learning rate by a factor of 10 after epochs 60, 120, and 160. Table 3 shows each network error rate over 4 independent runs each one initialized with a different random seed. 2 ). Using both mean and standard deviation as input (from equation 4), we have 50 features for WRN-28-10 models, and 198 features for DenseNet-BC-100-12 models. The training was performed using 5-fold cross-validation with the 2 minimization and the regularization factor being chosen as the best one (according to the 5-folds) among 10 values linearly spaced in the range 10 -4 and 10 4 .

Evaluation metrics: To evaluate the proposed method, we use the following metrics:

1. True negative rate (TNR) at 95% true positive rate (TPR). Let TP, TN, FP, and FN be the true positive, true negative, false positive, and false negative, respectively. The TNR is defined as TN/(TN+FP) whereas TPR is defined as TP/(TP+FN). 2. Area under the receiver operating characteristic curve (AUROC). AUROC is the area under the FPR=1-TNR against TPR curve.

4.2. MANIFOLD VISUALIZATION:

We applied t-SNE (L. van der Maaten, 2008) to visualize our high-dimensional feature space in order to see the similarities between ID/OOD samples. For this, we used one of the WRN-28-10 models trained with CIFAR-10 as ID dataset. We fitted the t-SNE using the ID and all OOD validation samples together using both mean and standard deviation features, and the result is shown in Figure 3 using a perplexity of 30. It is clear from the visualization that the proposed features are concentrated around well-defined clusters. Both synthetic OOD datasets have clear distinct behavior from the natural images ones, and it is straightforward to differentiate them. TinyImageNet (c), LSUN (c) are similar and have some intersection with TinyImageNet (r), LSUN (r). Interestingly, the clustering seems to reflect the dataset generation method (resizing or cropping 

4.3. PAIRWISE FITTING:

In this section, following other methods in the literature, we adjust the linear classifier for each ID/OOD pair. That is, for each pair of ID/OOD datasets, a different OOD classifier is trained using their respective validation samples.

We performed a comparison between the proposed method and four recent methods described in Section 2 (Hendrycks & Gimpel, 2017; Lee et al., 2018b; Liang et al., 2018; Vyas et al., 2018) 2018) results, we reimplemented the method using their reference implementation 3,4 . For ODIN (Liang et al., 2018) , we employed the same procedure as described by the authors to tune the methods parameters. A detailed description of the procedure can be found in Appendix B.1. For Lee et al. (2018b) and Vyas et al. (2018) , we use the values presented on their papers (Table 2 and Table 3 , respectively).

The results are compiled in Table 4 . Notably, our method outperforms the baseline and ODIN methods by a large margin, and yields better results than Lee et al. (2018b) and Vyas et al. (2018) in all tested cases without requiring any preprocessing, or changing the backbone model. In fact, when setting the OOD-ness threshold to obtain 95% TPR, our method is able to correctly detect all OOD samples from the test partition. We argue that the pairwise fitting scenario presented in the previous section is a limited performance measurement. In fact, many practical applications have OOD samples that do not come from a single distribution mode, and it might be infeasible to collect data from the many different modes of the OOD distribution (in general, some modes are unknown during training). A good OOD detector should be able to correctly identify samples from OOD distibutions for which its parameters have not been adjusted to. With this in mind, we propose a different, harder task in which an OOD detector is fitted to one, or a few, OOD datasets and then it is tested on all OOD datasets available. We note that this is not a standard practice in previous works, like in Liang et al. (2018) . We begin by evaluating the generalization ability of our detector in some preliminary experiments, which motivate our decisions in the choice of OOD datasets for fitting the model and feature selection.

Selecting Features: We evaluate the individual impact of each of the proposed features (i.e., layers average mean and standard deviation) by comparing the performance of the classifier with different features as inputs. In this experiment, we fit the linear classifier to a specific OOD validation dataset and test on all of the OOD test datasets available. Our performance metric is the TNR @ 95% TPR averaged over all the tested datasets. Table 5 presents the performance of the classifier for the WRN-28-10, averaged of the four available models, with CIFAR-100 as ID dataset. The classifier fitted using only standard deviation features still achieved very good performance generalizing very well to unknown OOD datasets. Since we are interested in designing an OOD detector that is able to differentiate ID samples from any OOD sample, all our results from now on are presented using the averaged standard deviations per layer as the only features used to train the linear decision function.

Table 5 : Generalization over unseen OOD datasets using different features (only mean, only std, and both). For each OOD validation set, we fit the linear model to it and tested against all OOD test sets. The results displayed are TNR @ 95% TPR averaged over the OOD datasets using WRN-28-10 as the backbone model and CIFAR-100 as ID dataset. As motivated in the previous section, we use the averages of normalized standard deviation features (equation 4). The obtained logistic regressor was then evaluated on the remaining OOD test datasets (unseen by both backbone training and logistic regressor). This procedure was then repeated for each possible OOD dataset, and the results are summarized in Table 6 . We see that all classifier fitted using only natural images are capable to generalize well over all other OOD sets, while this is not entirely true when fitting on random noise datasets. Also, fitting to all OODs validation sets (penultimate row), we can achieve even higher TNR scores over all test sets.

Using no OOD Dataset: To further evaluate the effectiveness of the method, we also tested the extreme case where no OOD samples are available for training. To do this, we used an unsupervised algorithm (one-class SVM (Manevitz & Yousef, 2002) with RBF kernel), and we only fitted to ID samples (i.e., no OOD samples are seen in the training step). The unsupervised results are summarized in the last row of Table 6 . As one can see, even the unsupervised method shows reasonable performance; showing again that in the proposed feature space the ID/OOD samples have different behavior. This corroborates the assumption that these features are a good indicator of OOD-ness.

Table 6 : Generalization to unseen OOD sets using CIFAR-100 as ID dataset and the WRN-28-10 backbone model. Performance of the OOD detector when the logistic regression is fit using 1000 samples of a given OOD dataset and then evaluated with respect to other OOD test datasets using only "std" as features. Results are TNR @ 95% TPR formatted as "mean (std)". Comparison with ODIN (Liang et al., 2018) : We compare the generalization capabilities of our method with the state-of-the-art technique ODIN (Liang et al., 2018) , in this new harder task. We fit both OOD classifier to maximize their detection performance on TinyImageNet (c) and Gaussian validation sets (i.e., 2000 OOD samples), and evaluate on all OOD test datasets. For our model, we use standard deviation features as inputs. For ODIN, we tune its hyperparameters using the grid search described in B.1. The results, presented in Table 7 , show that our method outperforms ODIN by a large margin, indicating better generalization to samples from unseen OOD datasets.

Effect of feature normalization::

We evaluated how much it helps to use the batch statistics computed by BN. As shown in Table 8 , normalizing the latent space using the BN statistics before computing the features has clear advantages.

Reduced validation data::

We study if our method can correctly detect OOD samples even when a small number of samples is available. Figure 4 shows the TNR @ 95% TPR for WRN 28-10 trained on CIFAR-10 (CIFAR-100), where only 27, 45, 75, 150, 300, 700, 1.5k, 3k (all) test images (ID + TinyImageNet crop + Gaussian, equally divided) are used to fit 25 coefficients of our logistic regressor. Using our method, only 27 images (from each ID and OOD), are necessary to achieve an average of 87.6% of TNR @ 95% TPR. Figure 4 : Averaged TNR @ 95% TPR over the OOD datasets using only a few samples to fit the logistic regressor (for WRN-28-10). The logistic regressor was fitted using TinyImageNet (c) and Gaussian validation sets using only s(l) as features. This experiment shows, in accordance to our results, that CIFAR-100 is more difficult to differentiate from other OOD datasets than CIFAR-10. Table 8 : TNR @ 95% TPR for computing the averaged mean and standard deviation from unnormalized/normalized BN latent space. The backbone model is the WRN 28-10 using CIFAR-10 as ID samples, and the logistic regressor was fitted using TinyImageNet (c) and Gaussian validation sets. The results are formatted as "mean (std)".

Normalization TinyImageNet (c) TinyImageNet (r) LSUN (c) LSUN (r) iSUN No 99.8 (0.1) 92.2 (1.0) 99.8 (0.1) 95.4 (0.6) 93.2 (1.1) Yes 99.9 (0.0) 97.1 (1.3) 100.0 (0.0) 99.4 (0.4) 98.6 (0.9)

5. CONCLUSION:

Deep neural networks trained to maximize classification performance in a given dataset are extremely adapted to said dataset. The statistics of activations throughout the network for samples from the training distribution (in-distribution) are remarkably stable. However, when a sample from a different distribution (out-of-distribution) is given to the network, its activation statistics depart greatly from those of in-distribution samples. Based on this observation, we propose a very simple yet efficient method to detect out-of-distribution samples. Our method is based on computing averages of low-order statistics at the batch normalization layers of the network, and then use them as features in a linear classifier. This procedure is much simpler and efficient than current stateof-the-art methods, and outperforms them by a large margin in the traditional ID/OOD fitting task (as proposed in previous works). We evaluated all methods in the challenging task of fitting on a single OOD dataset and testing on samples from other (unseen) datasets. In this harder scenario, our method generalizes well to unseen OOD datasets, outperforming ODIN by an even larger margin. Moreover, we show some preliminary results that even in the extreme case where no OOD samples are used for the training (unsupervised) we get reasonable performance.

A SUPPLEMENTARY RESULTS FOR SECTION 3 tuning is carried out on each pair of in-and out-of-distribution samples, which is the same procedure presented in Liang et al. (2018) and Lee et al. (2018b) . For reproducibility, a grid search is employed considering T ∈ {1, 1000} and with 21 linearly spaced values between 0 and 0.004 plus [0.00005, 0.0005, 0.0011].

B.2 ADDITIONAL BACKBONE MODEL:

We also train a Wide ResNet with L = 40 and widen factor of 4 (WRN-40-4) using the same training setup in Section 4, and the results for both training on CIFAR-10 and CIFAR-100 is depicted in Table 9 .

Table 9 : Average test error rates (%) and standard deviation (in parenthesis) over 4 runs.

Architecture CIFAR-10 CIFAR-100 WRN-40-4 4.41 (0.12) 20.83 (0.15)

B.3 OTHER EVALUATED METRICS:

To compare all methods described in Section 2 to our proposed method, we also use the following additional metrics:

1. Area under precision-recall curve (AUPR). The precision is evaluated as TP/(TP+FP) and recall in this case is the TPR. The AUPR-in (AUPR-out) is defined when in-(out-of)distribution samples are considered as the positive ones.

B.4 SELECTING OOD DATASET USING SVHN AS ID:

We also tested using the Street View House Number (SVHN) dataset (Netzer et al., 2011) as ID and DenseNet BC 100-12 as backbone model. The pre-trained model is from Lee et al. (2018b) , which has a test error rate of 3.58%. The results are displayed in Table 10 .

Table 10 : Generalization to unseen OOD sets using SVHN as ID dataset and the DenseNet BC 100-12 backbone model. Performance of the OOD detector when the logistic regression is fit using 1000 samples of a given OOD dataset and then evaluated with respect to other OOD test datasets using only "std" as features. Results are TNR @ 95% TPR. In this experiment, we evaluate the effect of ensembling across the BN layers, as described in Section 3.2, using a logistic regression. As shown in Figure 7 , aggregating confidence scores of intermediate layers indeed improves the detection performance of OOD samples. The left-most bin indicates the TNR at 95% of TPR using only the last BN layer. As we go to the right, we aggregate the shallower BN layers, and the right-most bin indicates that all the BN layers are being used. Another important insight is that deeper layers (on the left of Figure 7 ) contribute more for the separation between in-and out-of-distribution samples.

B.6 OOD DETECTION COMPARISON:

We also extended our Figure 7 : TNR @ 95% TPR obtained when aggregating information from multiple layers. The leftmost bin corresponds to the deepest (last) layer, and the rightmost bin to the first BN layer in the network using a WRN-28-10 trained on CIFAR-10/CIFAR-100, the logistic regressor was fitted on TinyImageNet (c) + Gaussian validation sets and the results are an average over all OOD test datasets. We also tested our proposed model on different TPR levels, and the results are depicted in Table 12 .

Table 12 : TNR at different TPR levels. The hyperparameters were tuned for TinyImageNet (c) and Gaussian validation sets and tested on others, and we only use the standard deviation as feature. 

:

Code for review avilable at https://www.dropbox.com/s/o652ufpznatvx8u/ ood-codes-review.zip?dl=0.

At the time of submission, Lee et al. (2018b) was still unpublished work

Baseline: github.com/hendrycks/error-detection

ODIN: github.com/ShiyuLiang/odin-pytorch

Figures and Tables:

features for each channel c as

normalized standard deviation by the BN running standard deviation σ (l) c .

-layers ensemble and final classification. Using all the features in equation 4, i.e., f = ( m(1) , s(1) , ..., m(L) , s(L) ) we fit a simple logistic regression model h(f ; θ) with parameters θ ∈ R 2L+1 . The parameters of the linear model are learned using a separate validation set formed with ID samples (positive examples) and OOD samples (negative examples).

Figure 3: Manifold visualization of the proposed features (mean and std) from a WRN-28-10 model.

Figure 5: More visualization results following the same procedure as in Figure 2 for a WRN-28-10 model.

Figure 6: More visualization results following the same procedure as in Figure 2 but for a DenseNet-BC-100-12 model.

DETECTION AT DIFFERENT TPR LEVELS

Summary of datasets. All generated samples have a size of 32 × 32.

Table content: <table><tr><td/><td/><td># samples val test</td><td>Pre-processing</td><td>Observation</td></tr><tr><td>ID</td><td>CIFAR-10 CIFAR-100</td><td colspan="2">1000 9000 -1000 9000 -</td></tr><tr><td/><td colspan="3">TinyImageNet (c) 1000 9000 Random patches</td><td>Subset of Imagenet containing 200 classes</td></tr><tr><td>OOD</td><td colspan="3">TinyImageNet (r) 1000 9000 Downsampled LSUN (c) 1000 9000 Random patches LSUN (r) 1000 9000 Downsampled</td></tr><tr><td/><td>iSUN</td><td colspan="2">1000 7925 -</td><td>Subset of SUN images</td></tr><tr><td/><td>Gaussian Noise</td><td>1000 9000</td><td>0.5 mean and unit variance clipped in [0, 1]</td><td>Channel-independent</td></tr><tr><td/><td>Uniform noise</td><td colspan="2">1000 9000 Uniform distribution in [0, 1]</td></tr></table>

Average test error rates (%) and standard deviation (in parenthesis) over 4 runs.

Table content: <table><tr><td>Architecture</td><td colspan="2">CIFAR-10 CIFAR-100</td></tr><tr><td colspan="2">DenseNet-BC-100-12 4.8 (0.2)</td><td>22.6 (0.3)</td></tr><tr><td>WRN-28-10</td><td>4.0 (0.1)</td><td>19.1 (0.3)</td></tr><tr><td colspan="3">Logistic regression: The logistic regressor is trained considering only the validation partitions for</td></tr><tr><td colspan="3">ID (positive examples) and OOD (negative examples) datasets (see Table</td></tr></table>

). Most importantly, one can see that the OOD samples are in different clusters than the ID (CIFAR-10) ones, which indicates that this feature choice is adequate for separating them.

Table content: <table><tr><td>&amp;,)$5</td></tr><tr><td>7LQ\,PDJH1HWF</td></tr><tr><td>7LQ\,PDJH1HWU</td></tr><tr><td>/681F</td></tr><tr><td>/681U</td></tr><tr><td>L681</td></tr><tr><td>8QLIRUP</td></tr><tr><td>*DXVVLDQ</td></tr></table>

. SinceLee et al. (2018b)  did not test using Wide ResNet models and the same datasets as inLiang et al. (2018);Vyas et al. (2018), here we only show the intersection between them: DenseNet BC 100-12 model, using CIFAR-10 (CIFAR-100) as in-distribution and TinyImageNet (resize) and LSUN (resize) as OOD distribution 2 . Extended results can be found in the appendix. For bothHendrycks & Gimpel (2017) andLiang et al. (

Table content: <table/>

Comparison between previous work (Section 2) and our proposed method using the TNR @ 95% TPR metric for DenseNet-BC-100-12. The displayed values (%) are the mean over 4 different training runs of the backbone models (for ODIN and our proposed model, the others results are from their respective papers).

Table content: <table><tr><td>Dataset</td><td>OOD</td><td colspan="5">Baseline ODIN Vyas et al. (2018) Lee et al. (2018b) Ours</td></tr><tr><td>CIFAR-10</td><td>TinyImageNet (r) LSUN (r)</td><td>30.3 35.5</td><td>86.5 92.7</td><td>97.1 99.2</td><td>95.3 97.5</td><td>100.0 100.0</td></tr><tr><td>CIFAR-100</td><td>TinyImageNet (r) LSUN (r)</td><td>21.0 26.7</td><td>52.9 55.7</td><td>79.5 83.8</td><td>86.6 91.2</td><td>100.0 100.0</td></tr><tr><td colspan="4">4.4 TESTING ON UNSEEN OOD DATASETS</td><td/><td/><td/></tr></table>

Comparison between ODIN and our proposed OOD detector for several setups using image classification networks. All detector parameters (and ODIN's hyperparameters) were tuned for TinyImageNet (c) and Gaussian validation sets. The results are formatted as "mean (std)".

Table content: <table><tr><td/><td>OOD dataset</td><td>TNR (95% TPR)</td><td>AUROC</td></tr><tr><td/><td/><td colspan="2">ODIN / Ours</td></tr><tr><td/><td colspan="2">TinyImageNet (c) 90.5 (2.1) / 100.0 (0.0)</td><td>98.0 (0.3) / 100.0 (0.0)</td></tr><tr><td/><td>TinyImageNet (r)</td><td>76.9 (4.7) / 96.6 (1.4)</td><td>95.8 (0.8) / 99.3 (0.3)</td></tr><tr><td/><td>LSUN (c)</td><td>82.9 (6.1) / 100.0 (0.0)</td><td>97.0 (0.7) / 100.0 (0.0)</td></tr><tr><td>DenseNet BC 100-12</td><td>LSUN (r)</td><td>86.2 (3.7) / 99.2 (0.5)</td><td>97.2 (0.6) / 99.8 (0.1)</td></tr><tr><td>CIFAR-10</td><td>iSUN</td><td>81.1 (4.6) / 98.4 (0.9)</td><td>96.4 (0.8) / 99.6 (0.2)</td></tr><tr><td/><td>Uniform</td><td colspan="2">68.7 (40.5) / 100.0 (0.0) 91.6 (9.2) / 100.0 (0.0)</td></tr><tr><td/><td>Gaussian</td><td colspan="2">75.3 (31.2) / 100.0 (0.0) 96.2 (1.8) / 100.0 (0.0)</td></tr><tr><td/><td colspan="2">TinyImageNet (c) 72.9 (4.1) / 100.0 (0.0)</td><td>95.4 (0.4) / 100.0 (0.0)</td></tr><tr><td/><td>TinyImageNet (r)</td><td>39.6 (4.5) / 95.3 (0.7)</td><td>88.3 (1.3) / 99.0 (0.2)</td></tr><tr><td/><td>LSUN (c)</td><td>54.7 (4.7) / 100.0 (0.0)</td><td>93.3 (0.5) / 100.0 (0.0)</td></tr><tr><td>DenseNet BC 100-12</td><td>LSUN (r)</td><td>42.2 (3.3) / 98.1 (0.7)</td><td>89.8 (0.8) / 99.5 (0.1)</td></tr><tr><td>CIFAR-100</td><td>iSUN</td><td>34.4 (3.6) / 96.2 (1.0)</td><td>87.5 (0.9) / 99.2 (0.2)</td></tr><tr><td/><td>Uniform</td><td>0.0 (0.0) / 100.0 (0.0)</td><td>67.3 (7.9) / 100.0 (0.0)</td></tr><tr><td/><td>Gaussian</td><td>0.0 (0.0) / 100.0 (0.0)</td><td>39.0 (15.0) / 100.0 (0.0)</td></tr><tr><td/><td>TinyImageNet (c)</td><td>91.9 (2.0) / 99.9 (0.0)</td><td>98.5 (0.3) / 100.0 (0.0)</td></tr><tr><td/><td>TinyImageNet (r)</td><td>73.4 (1.9) / 97.1 (1.3)</td><td>91.9 (0.8) / 99.4 (0.2)</td></tr><tr><td/><td>LSUN (c)</td><td>91.8 (2.3) / 100.0 (0.0)</td><td>98.4 (0.4) / 100.0 (0.0)</td></tr><tr><td>WRN 28-10</td><td>LSUN (r)</td><td>84.7 (1.8) / 99.4 (0.4)</td><td>95.8 (0.4) / 99.8 (0.1)</td></tr><tr><td>CIFAR-10</td><td>iSUN</td><td>79.8 (1.8) / 98.6 (0.9)</td><td>94.4 (0.5) / 99.6 (0.2)</td></tr><tr><td/><td>Uniform</td><td>98.9 (1.1) / 100.0 (0.0)</td><td>99.3 (0.3) / 100.0 (0.0)</td></tr><tr><td/><td>Gaussian</td><td colspan="2">100.0 (0.0) / 100.0 (0.0) 99.6 (0.3) / 100.0 (0.0)</td></tr><tr><td/><td colspan="2">TinyImageNet (c) 68.3 (3.0) / 100.0 (0.0)</td><td>94.6 (0.5) / 100.0 (0.0)</td></tr><tr><td/><td>TinyImageNet (r)</td><td>43.7 (2.3) / 96.8 (2.4)</td><td>87.7 (0.6) / 99.3 (0.5)</td></tr><tr><td/><td>LSUN (c)</td><td>38.8 (3.9) / 99.9 (0.0)</td><td>86.6 (2.3) / 99.9 (0.0)</td></tr><tr><td>WRN 28-10</td><td>LSUN (r)</td><td>43.7 (1.3) / 98.2 (1.8)</td><td>88.5 (0.2) / 99.6 (0.4)</td></tr><tr><td>CIFAR-100</td><td>iSUN</td><td>39.7 (2.0) / 97.5 (2.3)</td><td>87.2 (0.3) / 99.4 (0.4)</td></tr><tr><td/><td>Uniform</td><td>50.1 (49.0) / 99.8 (0.3)</td><td>78.5 (28.2) / 99.8 (0.3)</td></tr><tr><td/><td>Gaussian</td><td colspan="2">25.3 (42.4) / 100.0 (0.0) 70.1 (26.2) / 100.0 (0.0)</td></tr></table>

Comparison between ODIN and the proposed OOD detector. The hyperparameters were tuned for TinyImageNet (c) and Gaussian validation sets and tested on others, and we only use the standard deviation as feature.

Table content: <table><tr><td/><td>Out-of-distribution dataset</td><td>TNR (95% TPR)</td><td>AUROC</td><td>AUPR In</td><td>AUPR Out</td></tr><tr><td/><td/><td/><td colspan="2">ODIN / Ours</td><td/></tr><tr><td/><td>TinyImageNet (c)</td><td>90.5 (2.1) / 100.0 (0.0)</td><td>98.0 (0.3) / 100.0 (0.0)</td><td>98.5 (0.2) / 100.0 (0.0)</td><td>96.9 (0.6) / 100.0 (0.0)</td></tr><tr><td/><td>TinyImageNet (r)</td><td>76.9 (4.7) / 96.6 (1.4)</td><td>95.8 (0.8) / 99.3 (0.3)</td><td>96.4 (0.7) / 99.3 (0.3)</td><td>94.3 (1.2) / 99.3 (0.2)</td></tr><tr><td/><td>LSUN (c)</td><td>82.9 (6.1) / 100.0 (0.0)</td><td>97.0 (0.7) / 100.0 (0.0)</td><td>97.7 (0.5) / 100.0 (0.0)</td><td>95.7 (0.9) / 100.0 (0.0)</td></tr><tr><td>DenseNet BC 100-12</td><td>LSUN (r)</td><td>86.2 (3.7) / 99.2 (0.5)</td><td>97.2 (0.6) / 99.8 (0.1)</td><td>97.8 (0.4) / 99.8 (0.1)</td><td>95.6 (1.1) / 99.7 (0.1)</td></tr><tr><td>CIFAR-10</td><td>iSUN</td><td>81.1 (4.6) / 98.4 (0.9)</td><td>96.4 (0.8) / 99.6 (0.2)</td><td>97.3 (0.6) / 99.6 (0.1)</td><td>93.9 (1.6) / 99.5 (0.2)</td></tr><tr><td/><td>Uniform</td><td>68.7 (40.</td><td/><td/><td/></tr></table>

Peer review (iclr2019):
 The authors present a simple algorithm based on the statistics of neural activations of deep networks to detect out-of-distribution samples. The idea is to use the existing running estimate of mean and variance within BatchNorm layers to construct feature representations that are later fed into a simple linear classifier. The authors demonstrate superior performance over the previous state-of-art in the standard evaluation setting and provide fascinating insights and empirical analysis of their method. There are several aspects of this work that I admire. - The authors evaluate the generalization of their OOD detection model through evaluation against unseen OOD samples. This critical evaluation strategy is not typical in this literature and is much needed. - The organization of the material and the depth of the discussion is of high quality. They discuss and connect the previous work, they clearly explain the idea and provide empirical results to support the design decisions, and run several experiments to evaluate their method from different angles followed by interesting discussions. - The proposed method is easy to implement and has a minimal runtime complexity with no adverse effect on the underlying classifier. - The source code is already included in the submission. My only concern is that the feature pooling strategy first averages the input spatially, then across the channels. This feature size reduction is necessary because we have to ensure the following OOD classifier does not overfit in the validation stage. However, this reduction also introduces a permutation invariance in the feature space that is not desirable in OOD sample detection. I think it would make the work more valuable if the authors also take a critical look at the possible failure cases -- a short discussion of the weaknesses and assumptions. Overall, the paper is technically sound and well-organized with sufficient coverage of the previous work. A thorough series of evaluations support the claims. It is a novel combination of existing techniques. The empirical evidence is strong and insightful. Given the simplicity of the method, I would expect a quick adoption by the community. ------ Rev. In light of the rebuttal and the following discussions I have updated my rating to 7.