Unsupervised Out-of-Distribution Detection with Batch Normalization

1 INTRODUCTION . Modern neural network models are known to make poorly calibrated predictions ( Guo et al. , 2017 ; Kuleshov et al. , 2018 ) , and can be highly confident even for unrecognizable or irrelevant inputs ( Nguyen et al. , 2015 ; Moosavi-Dezfooli et al. , 2017 ) . This has serious implications for AI safety ( Amodei & Clark , 2016 ) in real world deployments , where a model could receive inputs that are beyond its training distribution . Detecting examples that are out of the training distribution becomes a viable solution : when encountering such samples , the model could choose to provide low confidence estimates or even abstain from making predictions ( Cortes et al. , 2017 ) . Density estimation is one approach to detecting out-of-distribution ( OoD ) samples . A likelihoodbased model is trained on the input samples ; during evaluation , samples that have low likelihoods are assumed to be out-of-distribution . For high-dimensional inputs ( such as images ) , deep generative models have been able to generate realistic samples as well as achieving good compression capabilities , which indicates high likelihoods on the training distribution ( Ballé et al. , 2016 ; Kingma & Welling , 2013 ; Kingma & Dhariwal , 2018 ; van den Oord et al. , 2016 ) ; thus , recent works have considered using deep generative models to detect out-of-distribution samples ( Li et al. , 2018 ) . However , contrary to popular belief , density estimates by deep generative models are highly inaccurate ( Nalisnick et al. , 2018 ) . For example , a Glow ( Kingma & Dhariwal , 2018 ) model trained on CIFAR10 gives higher likelihood estimates to SVHN samples than CIFAR10 ones , which makes accurate OoD detection impossible . While alternative statistics based on likelihood estimates have been proposed to alleviate this issue ( Choi et al. , 2018 ; Song et al. , 2017 ) , they are not able to detect OoD samples consistently . In this paper , we propose a new statistic to detect OoD samples using deep generative models trained with batch normalization ( Ioffe & Szegedy , 2015 ) . We argue that using batch normalization not only improves optimization ( as argued in ( Kohler et al. , 2018 ; Santurkar et al. , 2018 ) ) , but also challenges the i.i.d . assumption underlying typical likelihood-based objectives . We show in that the training objective of generative models with batch normalization can be interpreted as maximum pseudo-likelihood over a different joint distribution that does not assume data in the same batch are i.i.d . samples . Empirically , we demonstrate that over this joint distribution , the estimated likelihood of a batch of OoD samples is much lower than that of in-distribution samples . This allows us to propose a permutation test which outperforms existing methods by a significant margin without modifying how the underlying generative model is trained . In particular , we achieve near-perfect performance even on cases such as Fashion MNIST vs. KMNIST , where the likelihood distributions for single samples are nearly identical ( see Figure 1 , left ) . While generative models trained with BatchNorm might not provide state-of-the-art likelihood numbers on the test set , it remains competitive and could be more suited for the OoD detection problem ( Kingma & Dhariwal , 2018 ; Nalisnick et al. , 2018 ) . 2 BACKGROUND . Setup Let X = { xi } Ni=1 be a set of N observations ; we assume that “ in-distribution ” samples are drawn i.i.d . from some underlying distribution x ∼ p ( x ) . Our goal is to detect whether a test sample x is “ out-of-distribution ” ; we consider drawing the test samples from an unknown distribution q ( x ) , but we do not have access to samples from q ( x ) during training . Following definitions in ( Hendrycks & Gimpel , 2016 ; Liang et al. , 2017 ) , we define OoD examples as the test samples that have low densities under p ( x ) . We note that samples from q ( x ) are not necessarily all OoD ; for example , if q = p then most samples should not be classified as OoD . Detection of OoD samples is useful for applications such as anomaly detection ( Li et al. , 2018 ) and exploration in reinforcement learning ( Ostrovski et al. , 2017 ) . Likelihood-based generative models for out-of-distribution detection Likelihood-based models are trained by finding parameters θ such that the corresponding pθ ( x ) is as close as possible to the training distribution p ( x ) , in terms of Kullback-Leibler divergence ( KL ) : arg min θ DKL ( p ( x ) ‖pθ ( x ) ) = arg min θ Ep ( x ) [ log p ( x ) − log pθ ( x ) ] Finding appropriate solutions depends heavily on choosing the right family of ( parametrized ) probability distributions . In the context of images , deep neural networks have demonstrated good performance . Likelihood-based deep generative models include variational autoencoders ( Kingma & Welling , 2013 ) ( where variational lower bounds are optimized ) , autoregressive models ( van den Oord et al. , 2016 ; Salimans et al. , 2017 ) , and invertible flow models ( Dinh et al. , 2016 ; Kingma & Dhariwal , 2018 ) ( where exact likelihood is optimized ) . Given a sample x , we could attempt to test whether it is OoD or not by evaluating its likelihood − log pθ ( x ) . Intuitively , if pθ ( x ) is sufficiently similar to p ( x ) , OoD samples should have low likelihood under pθ ( x ) . However , this is not the case for some deep generative models ( Nalisnick et al. , 2018 ; Choi et al. , 2018 ) , where OoD samples are observed to have higher likelihoods than samples from the training distribution p ( x ) across several datasets and models . For example , stateof-the-art models trained on CIFAR10 ( Krizhevsky & Hinton , 2009 ) assign higher likelihood to SVHN ( Netzer et al. , 2011 ) samples than CIFAR10 samples . A similar phenomenon has also been observed by Nalisnick et al . ( 2018 ) for models trained on Fashion MNIST ( Xiao et al. , 2017 ) ( Figure 1 ( left ) ) , where images with lower variance across pixels ( such as MNIST ) are assigned higher likelihoods . This invalidates the assumption that OoD samples are assigned lower likelihoods by existing deep generative models . To alleviate this issue , Choi et al . ( 2018 ) use Watanabe Akaike Information Criterion ( WAIC ) : −WAIC ( x ) = −Eθ [ log pθ ( x ) ] + Varθ [ log pθ ( x ) ] ( 1 ) where the expectations are computed using independently trained model ensembles . This method does not assign higher negative WAIC values for OoD samples in some simple cases , but is empirically observed it to be effective for certain datasets and generative models . Likelihood-based permutation tests In the context of detecting adversarial examples , Song et al . ( 2017 ) considered using permutation tests statistics to determine whether an input x′ comes from p ( x ) or not . One such test statistic uses the rank of pθ ( x′ ) in { pθ ( x1 ) , · · · , pθ ( xN ) } , where { xi } Ni=1 is a ( training ) dataset of N samples from p ( x ) : Tperm = T ( x ′ ; x1 , . . . , xN ) , ∣∣∣∣∣ N∑ i=1 I [ pθ ( xi ) ≤ pθ ( x′ ) ] − N 2 ∣∣∣∣∣ ∈ [ 0 , N/2 + 1 ] ( 2 ) where I is the indicator function . When Tperm ( x ) is large , x is assumed to be OoD . This addresses the case where higher likelihood is observed for certain out-of-distribution samples ( such as the SVHN vs. CIFAR10 example in Nalisnick et al . ( 2018 ) ) , as both “ high-likelihood ” and “ low-likelihood ” samples will have Tperm close to N/2 ( because of the absolute value ) and be identified as OoD . 3 OUT-OF-DISTRIBUTION SAMPLES AND MODEL MIS-SPECIFICATION . Nalisnick et al . ( 2018 ) find it surprising that deep generative models assign higher likelihood to out-of-distribution samples , given their successful generalizing on a test dataset . However , we argue that this is to be expected when the unerlying generative model pθ ( x ) is mis-specified , and perhaps this is more likely than previously anticipated in the literature . We consider a simple example where a mis-specified model would overestimate the likelihood of out-of-distribution samples ( Figure 1 , right ) . Let p ( x ) be a uniform mixture of two Gaussians on R1 , N ( −1.0 , 0.01 ) and N ( 1.0 , 0.01 ) , and pθ ( x ) = N ( µ , σ2 ) be our model with parameters ( µ , σ ) . The maximum likelihood solution is ( µ ? , σ ? ) ≈ ( 0.0 , 1.0 ) , where mode-covering behavior occurs . x = 0 has the largest likelihood under pθ ( x ) , yet it is highly atypical in the original distribution . This is also a failure mode for WAIC as discussed in ( Choi et al. , 2018 ) . Given the complexity of high-dimensional image distributions , existing likelihood-based generative models are likely to be mis-specified , which invalidates the use of likelihood estimates to perform OoD detection ( White , 1982 ) . Moreover , model selection is often based on the holdout method ( Arlot et al. , 2010 ) , in which we evaluate log-likelihood over a validation set sampled from p ( x ) , but not over out-of-distribution samples . As we do not know the entropy of p ( x ) , we can never check whether DKL ( p ( x ) ‖pθ ( x ) ) ≈ 0 after training . Because of the mode-seeking nature of DKL , an alternative distribution q ( x ) 6= p ( x ) can have even lower KL-divergence with pθ ( x ) , i.e. , DKL ( q ( x ) ‖pθ ( x ) ) DKL ( p ( x ) ‖pθ ( x ) ) even if pθ ( x ) is trained on samples from p ( x ) . In Appendix B , we demonstrate that even more sophisticated tests over log-likelihood , such as likelihood-based permutation tests , can not detect out-of-distribution samples effectively . 4 DETECTING OUT-OF-DISTRIBUTION SAMPLES WITH BATCH NORMALIZATION . In the following sections , we argue that by taking advantage of batch normalization ( BatchNorm ) , outof-distribution samples can be detected with existing deep generative models . For a batch of inputs z = { zi } bi=1 of batch size b , batch normalization ( Ioffe & Szegedy , 2015 ) performs normalization over the inputs followed by a parametrized affine transformation : BatchNorm ( z ; γ , β , ) = z − E [ z ] √ Var [ z ] + · γ + β where γ , β are trainable parameters , and is a hyperparameter for numerical stability . The mean E [ z ] and variance Var [ z ] are computed over a single batch in training mode , and over the entire training set in evaluation mode1 . For deep generative models using batch normalization , existing literature evaluate the log-likelihood on some validation set in evaluation mode . The role of training mode is generally perceived to accelerate optimization ( Kohler et al. , 2018 ; Santurkar et al. , 2018 ) , similar to supervised learning . 1This is typically done by keeping an exponential moving average over the batch statistics during training .

Summary: This paper tackles the out of distribution detection problem and utilizes the property that the calculation of batch-normalization is different between training and testing for detecting out-of-distribution data with generative models. The paper first empirically demonstrates that the likelihood of out-of-distribution data has a larger difference between training mode and testing mode, then provides a possible theoretical explanation for the phenomenon. The proposed scoring function utilizes such likelihood differences in a permutation test for detecting out-of-distribution data. The evaluation is performed on two small in-distribution image datasets and four out-of-distribution datasets with three types of generative models.