Do Deep Generative Models Know What They Don't Know? 

1 INTRODUCTION . Deep learning has achieved impressive success in applications for which the goal is to model a conditional distribution p ( y|x ) , with y being a label and x the features . While the conditional model p ( y|x ) may be highly accurate on inputs x sampled from the training distribution , there are no guarantees that the model will work well on x ’ s drawn from some other distribution . For example , Louizos & Welling ( 2017 ) show that simply rotating an MNIST digit can make a neural network predict another class with high confidence ( see their Figure 1a ) . Ostensibly , one way to avoid such overconfidently wrong predictions would be to train a density model p ( x ; θ ) ( with θ denoting the parameters ) to approximate the true distribution of training inputs p∗ ( x ) and refuse to make a prediction for any x that has a sufficiently low density under p ( x ; θ ) . The intuition is that the discriminative model p ( y|x ) likely did not observe enough samples in that region to make a reliable decision for those inputs . This idea has been proposed by various papers , cf . ( Bishop , 1994 ) , and as recently as in the panel discussion at Advances in Approximate Bayesian Inference ( AABI ) 2017 ( Blei et al. , 2017 ) . Anomaly detection is just one motivating example for which we require accurate densities , and others include information regularization ( Szummer & Jaakkola , 2003 ) , open set recognition ( Herbei & Wegkamp , 2006 ) , uncertainty estimation , detecting covariate shift , active learning , model-based reinforcement learning , and transfer learning . Accordingly , these applications have lead to widespread interest in deep generative models , which take many forms such as variational auto-encoders ( VAEs ) ( Kingma & Welling , 2014 ; Rezende et al. , 2014 ) , generative adversarial networks ( GANs ) ( Goodfel∗Corresponding authors : e.nalisnick @ eng.cam.ac.uk and balajiln @ google.com . †Work done during an internship at DeepMind . low et al. , 2014 ) , auto-regressive models ( van den Oord et al. , 2016b ; a ) , and invertible latent variable models ( Tabak & Turner , 2013 ) . The last two classes—auto-regressive and invertible models—are especially attractive since they offer exact computation of the marginal likelihood , requiring no approximate inference techniques . In this paper , we investigate if modern deep generative models can be used for anomaly detection , as suggested by Bishop ( 1994 ) and the AABI pannel ( Blei et al. , 2017 ) , expecting a well-calibrated model to assign higher density to the training data than to some other data set . However , we find this to not be the case : when trained on CIFAR-10 ( Krizhevsky & Hinton , 2009 ) , VAEs , autoregressive models , and flow-based generative models all assign a higher density to SVHN ( Netzer et al. , 2011 ) than to the training data . We find this observation to be quite problematic and unintuitive since SVHN ’ s digit images are so visually distinct from the dogs , horses , trucks , boats , etc . found in CIFAR-10 . Yet this phenomenon is not restricted to CIFAR-10 vs SVHN , and we report similar findings for models trained on CelebA and ImageNet . We go on to study these curious observations in flow-based models in particular since they allow for exact marginal density calculations . When the flow is restricted to have constant volume across inputs , we show that the out-of-distribution behavior can be explained in terms of the data ’ s variance and the model ’ s curvature . To the best of our knowledge , we are the first to report these unintuitive findings for a variety of deep generative models and image data sets . Moreover , our experiments with flow-based models isolate some crucial experimental variables such as the effect of constant-volume vs non-volume-preserving transformations . Lastly , our analysis provides some simple but general expressions for quantifying the gap in the model density between two data sets . We close the paper by urging more study of the out-of-training-distribution properties of deep generative models . Understanding their behaviour in this setting is crucial for their deployment to the real world . 2 BACKGROUND . We begin by establishing notation and reviewing the necessary background material . We denote matrices with upper-case and bold letters ( e.g . X ) , vectors with lower-case and bold ( e.g . x ) , and scalars with lower-case and no bolding ( e.g . x ) . As our focus is on generative models , let the collection of all observations be denoted byX = { xn } Nn=1 with x representing a vector containing all features and , if present , labels . All N examples are assumed independently and identically drawn from some population x ∼ p∗ ( x ) ( which is unknown ) with support denoted X . We define the model density function to be p ( x ; θ ) where θ ∈ Θ are the model parameters , and let the model likelihood be denoted p ( X ; θ ) = ∏N n=1 p ( xn ; θ ) . 2.1 TRAINING NEURAL GENERATIVE MODELS . Given ( training ) data X and a model class { p ( · ; θ ) : θ ∈ Θ } , we are interested in finding the parameters θ that make the model closest to the true but unknown data distribution p∗ ( x ) . We can quantify this gap in terms of a Kullback–Leibler divergence ( KLD ) : KLD [ p∗ ( x ) ||p ( x ; θ ) ] = ∫ p∗ ( x ) log p∗ ( x ) p ( x ; θ ) dx ≈ − 1 N log p ( X ; θ ) −H [ p∗ ] ( 1 ) where the first term in the right-most expression is the average log-likelihood and the second is the entropy of the true distribution . As the latter is a fixed constant , minimizing the KLD amounts to finding the parameter settings that maximize the data ’ s log density : θ∗ = arg maxθ log p ( X ; θ ) = arg maxθ ∑N n=1 log p ( xn ; θ ) . Note that p ( xn ; θ ) alone does not have any interpretation as a probability . To extract probabilities from the model density , we need to integrate over some region Ω : P ( Ω ) = ∫ Ω p ( x ; θ ) dx . Adding noise to the data during model optimization can mock this integration step , encouraging the density model to output something nearer to probabilities ( Theis et al. , 2016 ) : log ∫ p ( xn + δ ; θ ) p ( δ ) dδ ≥ Eδ [ log p ( xn + δ ; θ ) ] ≈ log p ( xn + δ̃ ; θ ) where δ̃ is a sample from p ( δ ) . The resulting objective is a lower-bound , making it a suitable optimization target . All models in all of the experiments that we report are trained with input noise . Due to this ambiguity between densities and probabilities , we call the quantity log p ( X + ∆̃ ; θ ) a ‘ log-likelihood , ’ even ifX is drawn from a distribution unlike the training data . Regarding the choice of density model , we could choose one of the standard density functions for p ( xn ; θ ) , e.g . a Gaussian , but these may not be suitable for modeling the complex , high-dimensional data sets we often observe in the real world . Hence , we want to parametrize the model density with some high-capacity function f , which is usually chosen to be a neural network . That way the model has a somewhat compact representation and can be optimized via gradient ascent . We experiment with three variants of neural generative models : autoregressive , latent variable , and invertible . In the first class , we study the PixelCNN ( van den Oord et al. , 2016b ) , and due to space constraints , we refer the reader to van den Oord et al . ( 2016b ) for its definition . As a representative of the second class , we use a VAE ( Kingma & Welling , 2014 ; Rezende et al. , 2014 ) . See Rosca et al . ( 2018 ) for descriptions of the precise versions we use . Lastly , invertible flow-based generative models are the third class . We define them in detail below since we study them with the most depth . 2.2 GENERATIVE MODELS VIA CHANGE OF VARIABLES . The VAE and many other generative models are defined as a joint distribution between the observed and latent variables . However , another path forward is to perform a change of variables . In this case x and z are one and the same , and there is no longer any notion of a product space X × Z . Let f : X 7→ Z be a diffeomorphism from the data space X to a latent space Z . Using f then allows us to compute integrals over z as an integral over x and vice versa : ∫ z pz ( z ) dz = ∫ x pz ( f ( x ) ) ∣∣∣∣∂f∂x ∣∣∣∣ dx = ∫ x px ( x ) dx = ∫ z px ( f −1 ( z ) ) ∣∣∣∣∂f−1∂z ∣∣∣∣ dz ( 2 ) where |∂f/∂x| and |∂f−1/∂z| are known as the volume elements as they adjust for the volume change under the alternate measure . Specifically , when the change is w.r.t . coordinates , the volume element is the determinant of the diffeomorphism ’ s Jacobian matrix , which we denote as |∂f/∂x| . The change of variables formula is a powerful tool for generative modeling as it allows us to define a distribution p ( x ) entirely in terms of an auxiliary distribution p ( z ) , which we are free to choose , and f . Denote the parameters of the change of variables model as θ = { φ , ψ } with φ being the diffeomorphism ’ s parameters , i.e . f ( x ; φ ) , and ψ being the auxiliary distribution ’ s parameters , i.e . p ( z ; ψ ) . We can perform maximum likelihood estimation for the model as follows : θ∗ = arg max θ log px ( X ; θ ) = arg max φ , ψ N∑ n=1 log pz ( f ( xn ; φ ) ; ψ ) + log ∣∣∣∣∂fφ∂xn ∣∣∣∣ . ( 3 ) Optimizing ψ must be done carefully so as to not result in a trivial model . For instance , optimization could make p ( z ; ψ ) close to uniform if there are no constraints on its variance . For this reason , most implementations leave ψ as fixed ( usually a standard Gaussian ) in practice . Likewise , we assume it as fixed from here forward , thus omitting ψ from equations to reduce notational clutter . After training , samples can be drawn from the model via the inverse transform : x̃ = f−1 ( z̃ ; φ ) , z̃ ∼ p ( z ) . For the particular form of f , most work to date has constructed the bijection from affine coupling layers ( ACLs ) ( Dinh et al. , 2017 ) , which transform x by way of translation and scaling operations . Specifically , ACLs take the form : fACL ( x ; φ ) = [ exp { s ( xd : ; φs ) } x : d + t ( xd : ; φt ) , xd : ] , where denotes an element-wise product . This transformation , firstly , splits the input vector in half , i.e . x = [ x : d , xd : ] ( using Python list syntax ) . Then the second half of the vector is fed into two arbitrary neural networks ( possibly with tied parameters ) whose outputs are denoted t ( xd : ; φt ) and s ( xd : ; φs ) , with φ· being the collection of weights and biases . Finally , the output is formed by ( 1 ) scaling the first half of the input by one neural network output , i.e . exp { s ( xd : ; φs ) } x : d , ( 2 ) translating the result of the scaling operation by the second neural network output , i.e . ( · ) + t ( xd : ; φt ) , and ( 3 ) copying the second half of x forward , making it the second half of fACL ( x ; φ ) , i.e . fd : = xd : . ACLs are stacked to make rich hierarchical transforms , and the latent representation z is output from this composition , i.e . zn = f ( xn ; φ ) . A permutation operation is required between ACLs to ensure the same elements are not repeatedly used in the copy operations . We use f without subscript to denote the complete transform and overload the use of φ to denote the parameters of all constituent layers . This class of transform is known as non-volume preserving ( NVP ) ( Dinh et al. , 2017 ) since the volume element does not necessarily evaluate to one and can vary with each input x . Although non-zero , the log determinant of the Jacobian is still tractable : log |∂fφ/∂x| = ∑D j=d sj ( xd : ; φs ) . A diffeomorphic transform can also be defined with just translation operations , as was done in earlier work by Dinh et al . ( 2015 ) , and this transformation is volume preserving ( VP ) since the volume term is one and thus has no influence in the likelihood calculation . We will examine another class of flows we term constant-volume ( CV ) since the volume , while not preserved , is constant across all x . Appendix A provides additional details on implementing flow-based generative models .

Summary: This paper displays an occurrence of density models assigning higher likelihood to out-of-distribution inputs compared to the training distribution. Specifically, density models trained on CIFAR10 have higher likelihood on SVHN than CIFAR10. This is an interesting observation because the prevailing assumption is that density models can distinguish inliers from outliers. However, this phenomenon is not encountered when comparing MNIST and NotMNIST. The SVHN/CIFAR10 phenomenon has also been shown in concurrent work [1].