Detecting Out-Of-Distribution Samples Using Low-Order Deep Features Statistics

1 INTRODUCTION . In the past few years , deep neural networks ( DNNs ) ( Goodfellow et al. , 2016 ) have settled as the state-of-the art-techniques in many difficult tasks in a plurality of domains , such as image classification ( Krizhevsky et al. , 2012 ) , speech recognition ( Hinton et al. , 2012 ; Han et al. , 2018 ) , and machine translation ( Bahdanau et al. , 2014 ; van den Oord et al. , 2017 ) . This recent progress has been mainly due to their high accuracy and good generalization ability when dealing with realworld data . Unfortunately , DNNs are also highly confident when tested against unseen samples , even if the samples are vastly different from the ones employed during training ( Hendrycks & Gimpel , 2017 ) . Moreover , several works have shown that such deep networks are easily fooled by minor perturbations to the input ( Goodfellow et al. , 2014 ; Su et al. , 2017 ) . Obtaining a calibrated confidence score from a deep neural network is a problem under continuous investigation ( Hendrycks & Gimpel , 2017 ) and a major thread in artificial intelligence ( AI ) safety ( Amodei et al. , 2016 ) . In fact , knowing when the model is wrong or inaccurate has a direct impact in many production systems , such as self-driving cars , authentication and disease identification ( Akhtar & Mian , 2018 ; Goswami et al. , 2018 ) , to name a few . Guo et al . ( 2017 ) showed that despite producing significantly low classification errors , DNNs confidence scores are not faithful estimates of the true certainty . Their experiments confirmed that depth , width , weight decay , and batch normalization are the main reasons for overconfident scores . Moreover , they demonstrated that a simple and yet powerful method of temperature scaling in the softmax scores is an effective way to improve calibrate a DNNs confidence . While calibrating the classifier ’ s output to represent a faithful likelihood from the training ( in-distribution ) data has effective solutions , the problem of detecting whether or not the samples are generated from a known distribution ( out-of-distribution ) , is still an open problem ( Hendrycks & Gimpel , 2017 ) . One straightforward approach to calibrate the classifier ’ s confidence in order to detect samples whose distribution differs from the training samples distribution is to train a secondary classifier that digests both in-distribution ( ID ) and out-of-distribution ( OOD ) data so that anomalies are scored differently from ID samples , as performed in Hendrycks et al . ( 2018 ) . Re-training a network , however , can be computationally intensive and may even be intractable , since the number of OOD samples is virtually infinite . Other solutions rely on training both classification and generative neural net- works for OOD using a multi-task loss ( Lee et al. , 2018a ) , or using unsupervised fully convolutional networks as done by Sabokrou et al . ( 2016 ) to detect OOD in video samples . All these methods , however , have a major drawback : they require re-training a modified model using a different loss function possibly with additional parameters , which increases the computational burden , and demands access to the entire original ( and probably huge ) training data . In this work , we propose a novel OOD sample detection method that explores low-level statistics from feature layers . The statistics are obtained directly from the batch normalization layers ( Ioffe & Szegedy , 2015 ) , requiring no extra computations during training time , no changes to the network architecture and loss functions , nor preprocessing of the input image . During test time , the proposed method extracts statistics from selected layers and combines them into an OOD-ness score via a linear classifier . Throughout this paper , we observe that the mean and standard deviation of a given channel in a layer differ greatly between ID and OOD samples , which naturally motivates their use as features to be employed by an OOD detector . By selecting the BN layers of a network , we are able to normalize the features according to the learned batch statistics . The effectiveness of the proposed method is validated in two state-of-the-art DNN architectures ( DenseNet and ResNet ) ( Huang et al. , 2017 ; He et al. , 2015 ; 2016 ; Zagoruyko & Komodakis , 2016 ) that are trained for image classification tasks in popular datasets . The proposed approach achieves state-of-the-art performance , surpassing all competitors by a large margin in all tested scenarios , while being much more efficient . Notably , our method only requires one forward pass while Liang et al . ( 2018 ) ; Lee et al . ( 2018b ) ; Vyas et al . ( 2018 ) require two forward and one backward passes . The rest of the paper is organized as follows . Section 2 describes prior work on OOD samples detection . Section 3 introduces the proposed method , whereas Section 4 details all experiments and compares the results with state-of-the-art methods . Finally , we draw our conclusions in Section 5 . 2 PREVIOUS WORK . In this section , we describe recent prior work on OOD detection methods . A summary of all methods described can be seen in Table 1 . Hendrycks & Gimpel ( 2017 ) proposed a baseline method based on the posterior distribution ( i.e . softmax scores ) . They showed that well-trained models tend to produce higher scores for ID samples than for OOD ones . Hence their method comprises of applying a threshold on the softmaxnormalized output of a classifier . If the largest score is below the threshold , then the sample is considered OOD . Liang et al . ( 2018 ) continued the aforementioned line of work and proposed the Out-of-Distribution detector for Neural networks ( ODIN ) , which includes a temperature scaling T ∈ R+∗ to the softmax classifier as in Guo et al . ( 2017 ) . The authors in ODIN argued that a good manipulation of T eases the separation between in- and out-of-distribution samples . Allied to that , they also incorporated small perturbations to the input ( inspired by Goodfellow et al . ( 2014 ) ) whose goal is to increase the softmax score of the predicted class . Liang et al . ( 2018 ) found that this kind of perturbation has a stronger effect on ID samples than on OOD ones , increasing the separation between ID and OOD samples . ODIN outperforms the baseline method ( Hendrycks & Gimpel , 2017 ) by a fair margin ; however , it is three times slower due to the two forward and one backward passes needed to preprocess the input , while ( Hendrycks & Gimpel , 2017 ) only requires one forward pass . Vyas et al . ( 2018 ) describes a novel loss function , called margin entropy loss , over the softmax output that attempts to increase the distance between ID and OOD samples . During training , they partition the training data itself into ID and OOD by assigning samples labeled as certain classes as OOD and use the different partitions to train an ensemble of classifiers that are then used to detect OOD samples during test time . They also use the input pre-processing step proposed in Liang et al . ( 2018 ) , including temperature scaling . Lee et al . ( 2018b ) is the most recent work on OOD detection that we have knowledge of . They show that the posterior distribution defined by a generative classifier ( under Gaussian discriminant analysis ) is equivalent to that of the softmax classifier , and the generative classifier eases the separation between in- and out-of-distribution samples . The confidence score is defined using the Mahalanobis distance between the sample and the closest class-conditional Gaussian distribution . They argue that abnormal samples can be better characterized in the DNN feature space rather than the output space of softmax-based posterior distribution as done in previous work ( e.g. , ODIN ) . Samples are pre-processed similarly as done in ODIN , but the confidence score is increased instead of the softmax one . To further improve the performance , they also consider intermediate generative classifiers for all layers in the network . The final OOD sample detector is computed as an ensemble of confidence scores , chosen by training a logistic regression on validation samples . This method also shows remarkable results for detection of adversarial attacks and for incremental learning . 3 PROPOSED SOLUTION . An OOD detector should incorporate information from the training data in a natural manner , without being directly influenced by the loss function , which is intrinsically related to the task which could be well-defined for ID samples but be meaningless for most OOD samples . Moreover , if the OOD method is more dependent on the training distribution , it should be able to be applied to a wide variety networks , and not be designed specifically for a given architecture . Our method is based on a very simple observation . Input samples with different distributions generate statistically distinct feature spaces in a DNN . In other words , the deep features of an ID sample are distinct from those of an OOD one . Moreover , when using a normalization scheme , such as BN , the features are normalized by the statistics of the ID batch during training , possibly leading to feature statistics that are more similar to the batch statistics , as depicted in Figure 2 . The main problem then becomes how to summarize the feature space distribution for ID samples in a way that best discriminates between ID and OOD samples . In this work we show that using the first and second order statistics within each feature map performs remarkably well for this task . Figure 1 shows the proposed OOD detector , highlighting the feature statistics that are extracted for each channel at each BN layer . These statistics are then combined through a learned linear decision function , providing a measure of OOD-ness . The way we compute the per-channel statistics is formalized in Section 3.1 . Finally , the linear classifier used to combine the statistics from the different layers is described in Section 3.2 . 3.1 LOW-ORDER FEATURE STATISTICS . In the previous section , we motivated that characterizing the feature-space distributions might lead to a robust OOD detector . As a first approach , one could model these distributions using a nonparametric method to estimate the distribution of the features for each channel , which requires the computation of sufficient statistics using the training data or using a parametric method to fit the distribution ( Bishop , 2007 ) , which are both computationally intensive . Here , we propose to use only the mean and standard deviation computed along the spatial dimension for each channel to summarize the per-channel distribution . As it will be shown , these two statistics are sufficient to distinguish between ID and OOD . Moreover , because the mean and standard deviation of each channel are normalized by the running mean and variance computed during training by the BN layers ( Ioffe & Szegedy , 2015 ) , these statistics can be naturally combined within each layer to produce effective features for OOD detection . We describe such a procedure in what follows . Given the l-th BN layer with input tensor X ∈ RC×W×H , the output BN ( l ) c , i , j at channel c is Z ( l ) c , i , j = X ( l ) c , i , j − µ ( l ) c√ [ σ ( l ) c ] 2 + , BN ( l ) c , i , j = Z ( l ) c , i , jγ ( l ) c + β ( l ) c , ( 1 ) where γ ( l ) c > 0 and β ( l ) c are the per channel per layer learned scaling and shifting parameters , > 0 is a small constant value for numerical stability , µ ( l ) c ∈ R and [ σ ( l ) c ] 2 ∈ R+ are the mean and variance estimated through a moving average using the batch statistics during training , and Z ( l ) ∈ RC×W×H is the normalized feature tensor . It is worth noting that the statistics are calculated independently for each channel c at each layer l. In this paper , we conjecture that low-order statistics computed from either X ( l ) c , i , j or Z ( l ) c , i , j can be used to discriminate between ID and OOD samples . Given the unnormalized input X ( l ) c , i , j , we can compute the empirical mean m ( l ) c and standard deviation σ ( l ) c features for each channel c as m ( l ) c = 1 WH ∑ i , j X ( l ) c , i , j , [ s ( l ) c ] 2 = 1 WH ∑ i , j [ X ( l ) c , i , j ] 2 − [ m ( l ) c ] 2 , ( 2 ) and the features for the normalized tensor Z ( l ) c , i , j are defined as m̂ ( l ) c = 1 WH ∑ i , j Z ( l ) c , i , j = m ( l ) c − µc ( l ) σ ( l ) c [ ŝ ( l ) c ] 2 = 1 WH ∑ i , j [ Z ( l ) c , i , j ] 2 − [ m̂ ( l ) c ] 2 = [ ŝ ( l ) c σ ( l ) c ] 2 , ( 3 ) i.e. , the normalized mean feature m̂ ( l ) c represents the difference between the sample mean m ( l ) c and the BN running mean µ ( l ) c weighted by the running standard deviation σ ( l ) c , whereas ŝ ( l ) c is the normalized standard deviation by the BN running standard deviation σ ( l ) c .

Summary: The authors present a simple algorithm based on the statistics of neural activations of deep networks to detect out-of-distribution samples. The idea is to use the existing running estimate of mean and variance within BatchNorm layers to construct feature representations that are later fed into a simple linear classifier. The authors demonstrate superior performance over the previous state-of-art in the standard evaluation setting and provide fascinating insights and empirical analysis of their method.