Out-of-Distribution Detection Using Layerwise Uncertainty in Deep Neural Networks

1 INTRODUCTION . Deep neural networks ( DNNs ) have achieved high performance in many classification tasks such as image classification ( Krizhevsky et al. , 2012 ; Simonyan & Zisserman , 2014 ) , object detection ( Lin et al. , 2017 ; Redmon & Farhadi , 2018 ) , and speech recognition ( Hinton et al. , 2012 ; Hannun et al. , 2014 ) . However , DNNs tend to make high confidence predictions even for samples that are not drawn from the training distribution , i.e. , out-of-distribution ( OOD ) samples ( Hendrycks & Gimpel , 2016 ) . Such errors can be harmful to medical diagnosis and automated driving . Because it is not generally possible to control the test data distribution in real-world applications , OOD samples are inevitably included in this distribution . Therefore , detecting OOD samples is important for ensuring the safety of an artificial intelligence system ( Amodei et al. , 2016 ) . There have been many previous studies ( Hendrycks & Gimpel , 2016 ; Liang et al. , 2017 ; Lee et al. , 2017 ; DeVries & Taylor , 2018 ; Lee et al. , 2018 ; Hendrycks et al. , 2018 ) that have attempted to solve this problem by regarding samples that are difficult to classify or samples with low classification confidence as OOD examples using DNNs . Their approaches work well and they are computationally efficient . The limitation of these studies is that , when using difficult datasets or models with low classification ability , the confidence of inputs will be low , even if the inputs are in-distribution samples . Therefore , these methods incorrectly regard such in-distribution samples as OOD samples , which results in their poor detection performance ( Malinin & Gales , 2018 ) , as shown in Figure 1 . One cause of the abovementioned problem is that their approaches use only the features close to the output layer and the features are strongly related to the classification accuracy . Therefore , we use not only the features close to the output layer but also the features close to the input layer . We hypothesize that the uncertainties of the features close to the input layer are the uncertainties of the feature extraction and are effective for detecting OOD samples . For example , when using convolutional neural networks ( CNNs ) , the filters of the convolutional layer close to the input layer extract features such as edges that are useful for in-distribution classification . In other words , indistribution samples possess more features that convolutional filters react to than OOD samples . Therefore , the uncertainties of the features will be larger when the inputs are in-distribution samples . Another cause of the abovementioned problem is that their approaches disregard the uncertainty of the features close to the output layer . We hypothesize that the uncertainties of the latent features close to the output layer are the uncertainties of classification and are also effective for detecting OOD samples . For example , in-distribution samples are embedded in the feature space close to the output layer to classify samples . In contrast , OOD samples have no fixed regions for embedding . Therefore , the uncertainties of the features of OOD samples will be larger than those of in-distribution samples . Based on the hypotheses , we propose a method that extracts the Uncertainties of Features in Each Layer ( UFEL ) and combines them for detecting OOD samples . Each uncertainty is easily estimated after training the discriminative model by computing the mean and the variance of their features using a reparameterization trick such as the variational autoencoder ( Kingma & Welling , 2013 ) and variational information bottleneck ( Alemi et al. , 2016 ; 2018 ) . Our proposal is agnostic to the model architecture and can be easily combined with any regular architecture with minimum modifications . We visualize the maximum values of output probability and the combined uncertainties of the latent features in the feature space of the penultimate layer in Figure 1 . The combined uncertainties of the features discriminate the in-distribution and OOD images that are difficult to classify . For example , although the images that are surrounded by the red line are in-distribution samples , they have low maximum softmax probabilities and could be regarded as OOD samples in prior work . Meanwhile , their uncertainties are smaller than those of OOD samples and they are regarded as in-distribution samples in our method . In experiments , we validate the hypothesis demonstrating that each uncertainty is effective for detecting OOD examples . We also demonstrate that UFEL can obtain state-of-the-art performance in several datasets including CIFAR-100 , which is difficult to classify , and models including LeNet5 with low classification ability . Moreover , UFEL is robust to hyperparameters such as the number of in-distribution classes and the validation dataset . 2 RELATED WORK . Methods based on the classification confidence Hendrycks & Gimpel ( 2016 ) proposed the baseline method to detect OOD samples without the need to further re-train and change the structure of the model . They define low-maximum softmax probabilities as indicating the low confidence of in-distribution examples and detect OOD samples using the softmax outputs of a pre-trained deep classifier . Building on this work , many models have recently been proposed . Liang et al . ( 2017 ) proposed ODIN , a calibration technique that uses temperature scaling ( Guo et al. , 2017 ) in the Under review as a conference paper at ICLR 2020 ! '' # '' $ % '' $ z '' $ z '' ' Σ '' Σ '' # '' ) % '' % '' % '' ) Σ '' Σ '' z '' ) softmax function and adds small controlled perturbations to the inputs to widen the gap between indistribution and OOD features , which improves the performance of the baseline method . Likewise , Lee et al . ( 2018 ) ; DeVries & Taylor ( 2018 ) ; Lee et al . ( 2017 ) ; Hendrycks et al . ( 2018 ) also extended the baseline method . Like Hendrycks & Gimpel ( 2016 ) , we use the feature of maximum softmax probability as one of our features . Methods based on the uncertainty Malinin & Gales ( 2018 ) attempted to solve the problem of classifying in-distribution samples close to the decision boundary as OOD samples by distinguishing between data uncertainty and distributional uncertainty . Data uncertainty , or aleatoric uncertainty ( Kendall & Gal , 2017 ) , is irreducible uncertainty such as class overlap , whereas distributional uncertainty arises because of the mismatch between training and testing distributions . They argue that the value of distributional uncertainty depends on the difference in the Dirichlet distribution of the categorical parameter . Further , they estimate the parameter of the Dirichlet distribution using a DNN and train the model with in-distribution and OOD datasets . The motivation for our work is similar to that of Malinin & Gales ( 2018 ) . In our work , the distribution of the logit of the categorical parameters is modeled as a Gaussian distribution , which enables us to train the model without an OOD dataset . Furthermore , we estimate the parameters of the Gaussian distribution of latent features close to the input layer . 3 PROPOSED METHOD . In this section , we present UFEL , which extracts the uncertainties of features in each layer and combines them for detecting OOD samples . First , we use the maximum of the softmax output , as in Hendrycks & Gimpel ( 2016 ) , as one of our features . Second , we also use the distribution of the categorical parameter , as in Malinin & Gales ( 2018 ) , using the uncertainty of logits . Furthermore , we use the uncertainty of the feature extraction extracted from the latent space close to the input layer because they will not be relevant to the classification accuracy . We probabilistically model the values of these features , estimate their uncertainties , and combine them . Let x ∈ X be an input , y ∈ Y = { 1 , · · · , K } be its label , and l ∈ { 1 , · · · , L } be the index of the block layers . The objective function of normal deep classification is as follows : J = Ex , y∼p ( x , y ) [ L ( fφ ( x ) , y ) ] , ( 1 ) where p ( x , y ) is the empirical data distribution , L is a cross entropy loss function , and fφ is a DNN . We use the following notation fφ = fφL ◦ fφL−1 ◦ · · · ◦ fφ1 as shown in Figure 2 . To extract the uncertainties of features in each layer , we model the lth block layer ’ s output zl as a Gaussian whose parameters depend on the l-1th block layer ’ s output zl−1 as follows : p ( zl|zl−1 ) = N ( zl|fµφl ( z l−1 ) , fΣφl ( z l−1 ) ) , where fφl is the lth block layer , which outputs both the mean µ and covariance matrix Σ . In this paper , we use a diagonal covariance matrix to reduce the model parameters . We use the reparameterization trick ( Kingma & Welling , 2013 ) to write zl = µl + σl , where µl = fµφl ( z l−1 ) , σl = fΣφl ( z l−1 ) , and is the Gaussian noise . Then , our objective function is as follows : JU = Ex , y∼p ( x , y ) , ∼p ( ) [ L ( zL ( x , ) , y ) ] , ( 2 ) where z0 = x . Because of the reparameterization trick , the loss gradient is backpropagated directly through our model , and we can train our model like the regular classification models in Equation 1 . Next , we explain the two methods of combining the features extracted in each layer . In the first method , we sum the uncertainties of each value of the features in each layer and linearly combine them . Because the feature maps of a convolutional block layer are three dimensional , each element is computed as zlijk = µ l ijk + σ l ijk . Moreover , because the output of a fully connected layer is one dimensional , each element is formed as zli = µ l i + σ l i . We use a weighted summation of the scale of each feature and the maximum value of the softmax scores as a final feature dLR as follows : dLR ( x ) = L−1∑ l=1 λlσlsum ( x ) + λ L max ( zL ) , σlsum = { ∑ ijk σ l ijk ( if the lth block is Conv ) ∑ i σ l i ( otherwise ) ( 3 ) We choose the parameter λl by training a logistic regression ( LR ) using in-distribution and OOD validation samples . In the second method , we combine the features directly and nonlinearly using a CNN as follows : dCNN ( x ) = CNNθ ( σ 1 ( x ) , ... , σL−1 ( x ) , max ( zL ) ) . ( 4 ) We train the CNN parameter θ with in-distribution and OOD validation samples using binary crossentropy loss . The detailed structures of the CNN are given in Table A.3 . We use the values of these feature d ( x ) to test the performance of detecting OOD samples .

Summary: The paper considers the problem of out-of-distribution (OOD) sample detection while solving a classification task. The authors tackle the problem of OOD detection with exploiting uncertainty while passing a test sample through the neural network. They treat outputs of (some) layers in a NN as random Gaussian-distributed variables and measure uncertainty as variance of these Gaussians. Then when uncertainty is high, OOD is detected.