DO DEEP GENERATIVE MODELS KNOW WHAT THEY DON’T KNOW?
Year: 2019

Abstract:
A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data. A plethora of work has demonstrated that it is easy to find or synthesize inputs for which a neural network is highly confident yet wrong. Generative models are widely viewed to be robust to such mistaken confidence as modeling the density of the input features can be used to detect novel, out-of-distribution inputs. In this paper we challenge this assumption. We find that the density learned by flow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e. CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former. Moreover, we find evidence of this phenomenon when pairing several popular image data sets: FashionMNIST vs MNIST, CelebA vs SVHN, ImageNet vs CIFAR-10 / CIFAR-100 / SVHN. To investigate this curious behavior, we focus analysis on flow-based generative models in particular since they are trained and evaluated via the exact marginal likelihood. We find such behavior persists even when we restrict the flows to constant-volume transformations. These transformations admit some theoretical analysis, and we show that the difference in likelihoods can be explained by the location and variances of the data and the model curvature. Our results caution against using the density estimates from deep generative models to identify inputs similar to the training distribution until their behavior for out-of-distribution inputs is better understood.

heading: 1 INTRODUCTION
text: Deep learning has achieved impressive success in applications for which the goal is to model a conditional distribution p(y|x), with y being a label and x the features. While the conditional model p(y|x) may be highly accurate on inputs x sampled from the training distribution, there are no guarantees that the model will work well on x’s drawn from some other distribution. For example, Louizos & Welling (2017) show that simply rotating an MNIST digit can make a neural network predict another class with high confidence (see their Figure 1a). Ostensibly, one way to avoid such overconfidently wrong predictions would be to train a density model p(x;θ) (with θ denoting the parameters) to approximate the true distribution of training inputs p∗(x) and refuse to make a prediction for any x that has a sufficiently low density under p(x;θ). The intuition is that the discriminative model p(y|x) likely did not observe enough samples in that region to make a reliable decision for those inputs. This idea has been proposed by various papers, cf. (Bishop, 1994), and as recently as in the panel discussion at Advances in Approximate Bayesian Inference (AABI) 2017 (Blei et al., 2017).
Anomaly detection is just one motivating example for which we require accurate densities, and others include information regularization (Szummer & Jaakkola, 2003), open set recognition (Herbei & Wegkamp, 2006), uncertainty estimation, detecting covariate shift, active learning, model-based reinforcement learning, and transfer learning. Accordingly, these applications have lead to widespread interest in deep generative models, which take many forms such as variational auto-encoders (VAEs) (Kingma & Welling, 2014; Rezende et al., 2014), generative adversarial networks (GANs) (Goodfel∗Corresponding authors: e.nalisnick@eng.cam.ac.uk and balajiln@google.com. †Work done during an internship at DeepMind.
low et al., 2014), auto-regressive models (van den Oord et al., 2016b;a), and invertible latent variable models (Tabak & Turner, 2013). The last two classes—auto-regressive and invertible models—are especially attractive since they offer exact computation of the marginal likelihood, requiring no approximate inference techniques.
In this paper, we investigate if modern deep generative models can be used for anomaly detection, as suggested by Bishop (1994) and the AABI pannel (Blei et al., 2017), expecting a well-calibrated model to assign higher density to the training data than to some other data set. However, we find this to not be the case: when trained on CIFAR-10 (Krizhevsky & Hinton, 2009), VAEs, autoregressive models, and flow-based generative models all assign a higher density to SVHN (Netzer et al., 2011) than to the training data. We find this observation to be quite problematic and unintuitive since SVHN’s digit images are so visually distinct from the dogs, horses, trucks, boats, etc. found in CIFAR-10. Yet this phenomenon is not restricted to CIFAR-10 vs SVHN, and we report similar findings for models trained on CelebA and ImageNet. We go on to study these curious observations in flow-based models in particular since they allow for exact marginal density calculations. When the flow is restricted to have constant volume across inputs, we show that the out-of-distribution behavior can be explained in terms of the data’s variance and the model’s curvature.
To the best of our knowledge, we are the first to report these unintuitive findings for a variety of deep generative models and image data sets. Moreover, our experiments with flow-based models isolate some crucial experimental variables such as the effect of constant-volume vs non-volume-preserving transformations. Lastly, our analysis provides some simple but general expressions for quantifying the gap in the model density between two data sets. We close the paper by urging more study of the out-of-training-distribution properties of deep generative models. Understanding their behaviour in this setting is crucial for their deployment to the real world.

heading: 2 BACKGROUND
text: We begin by establishing notation and reviewing the necessary background material. We denote matrices with upper-case and bold letters (e.g. X), vectors with lower-case and bold (e.g. x), and scalars with lower-case and no bolding (e.g. x). As our focus is on generative models, let the collection of all observations be denoted byX = {xn}Nn=1 with x representing a vector containing all features and, if present, labels. All N examples are assumed independently and identically drawn from some population x ∼ p∗(x) (which is unknown) with support denoted X . We define the model density function to be p(x;θ) where θ ∈ Θ are the model parameters, and let the model likelihood be denoted p(X;θ) = ∏N n=1 p(xn;θ).

heading: 2.1 TRAINING NEURAL GENERATIVE MODELS
text: Given (training) data X and a model class {p(·;θ) : θ ∈ Θ}, we are interested in finding the parameters θ that make the model closest to the true but unknown data distribution p∗(x). We can quantify this gap in terms of a Kullback–Leibler divergence (KLD):
KLD[p∗(x)||p(x;θ)] = ∫ p∗(x) log p∗(x)
p(x;θ) dx ≈ − 1 N log p(X;θ)−H[p∗] (1)
where the first term in the right-most expression is the average log-likelihood and the second is the entropy of the true distribution. As the latter is a fixed constant, minimizing the KLD amounts to finding the parameter settings that maximize the data’s log density: θ∗ = arg maxθ log p(X;θ) = arg maxθ ∑N n=1 log p(xn;θ). Note that p(xn;θ) alone does not have any interpretation as a probability. To extract probabilities from the model density, we need to integrate over some region Ω: P (Ω) = ∫ Ω p(x;θ)dx. Adding noise to the data during model optimization can mock this integration step, encouraging the density model to output something nearer to probabilities (Theis et al., 2016):
log ∫ p(xn + δ;θ)p(δ) dδ ≥ Eδ [log p(xn + δ;θ)] ≈ log p(xn + δ̃;θ)
where δ̃ is a sample from p(δ). The resulting objective is a lower-bound, making it a suitable optimization target. All models in all of the experiments that we report are trained with input noise.
Due to this ambiguity between densities and probabilities, we call the quantity log p(X + ∆̃;θ) a ‘log-likelihood,’ even ifX is drawn from a distribution unlike the training data.
Regarding the choice of density model, we could choose one of the standard density functions for p(xn;θ), e.g. a Gaussian, but these may not be suitable for modeling the complex, high-dimensional data sets we often observe in the real world. Hence, we want to parametrize the model density with some high-capacity function f , which is usually chosen to be a neural network. That way the model has a somewhat compact representation and can be optimized via gradient ascent. We experiment with three variants of neural generative models: autoregressive, latent variable, and invertible. In the first class, we study the PixelCNN (van den Oord et al., 2016b), and due to space constraints, we refer the reader to van den Oord et al. (2016b) for its definition. As a representative of the second class, we use a VAE (Kingma & Welling, 2014; Rezende et al., 2014). See Rosca et al. (2018) for descriptions of the precise versions we use. Lastly, invertible flow-based generative models are the third class. We define them in detail below since we study them with the most depth.

heading: 2.2 GENERATIVE MODELS VIA CHANGE OF VARIABLES
text: The VAE and many other generative models are defined as a joint distribution between the observed and latent variables. However, another path forward is to perform a change of variables. In this case x and z are one and the same, and there is no longer any notion of a product space X × Z . Let f : X 7→ Z be a diffeomorphism from the data space X to a latent space Z . Using f then allows us to compute integrals over z as an integral over x and vice versa:∫
z
pz(z) dz = ∫ x pz(f(x)) ∣∣∣∣∂f∂x ∣∣∣∣ dx = ∫ x px(x) dx = ∫ z px(f −1(z)) ∣∣∣∣∂f−1∂z ∣∣∣∣ dz (2)
where |∂f/∂x| and |∂f−1/∂z| are known as the volume elements as they adjust for the volume change under the alternate measure. Specifically, when the change is w.r.t. coordinates, the volume element is the determinant of the diffeomorphism’s Jacobian matrix, which we denote as |∂f/∂x|. The change of variables formula is a powerful tool for generative modeling as it allows us to define a distribution p(x) entirely in terms of an auxiliary distribution p(z), which we are free to choose, and f . Denote the parameters of the change of variables model as θ = {φ,ψ} with φ being the diffeomorphism’s parameters, i.e. f(x;φ), and ψ being the auxiliary distribution’s parameters, i.e. p(z;ψ). We can perform maximum likelihood estimation for the model as follows:
θ∗ = arg max θ log px(X;θ) = arg max φ,ψ N∑ n=1 log pz(f(xn;φ);ψ) + log ∣∣∣∣∂fφ∂xn ∣∣∣∣ . (3)
Optimizing ψ must be done carefully so as to not result in a trivial model. For instance, optimization could make p(z;ψ) close to uniform if there are no constraints on its variance. For this reason, most implementations leave ψ as fixed (usually a standard Gaussian) in practice. Likewise, we assume it as fixed from here forward, thus omitting ψ from equations to reduce notational clutter. After training, samples can be drawn from the model via the inverse transform: x̃ = f−1(z̃;φ), z̃ ∼ p(z). For the particular form of f , most work to date has constructed the bijection from affine coupling layers (ACLs) (Dinh et al., 2017), which transform x by way of translation and scaling operations. Specifically, ACLs take the form: fACL(x;φ) = [exp{s(xd:;φs)} x:d + t(xd:;φt),xd:] , where denotes an element-wise product. This transformation, firstly, splits the input vector in half, i.e. x = [x:d,xd:] (using Python list syntax). Then the second half of the vector is fed into two arbitrary neural networks (possibly with tied parameters) whose outputs are denoted t(xd:;φt) and s(xd:;φs), with φ· being the collection of weights and biases. Finally, the output is formed by (1) scaling the first half of the input by one neural network output, i.e. exp{s(xd:;φs)} x:d, (2) translating the result of the scaling operation by the second neural network output, i.e. (·) + t(xd:;φt), and (3) copying the second half of x forward, making it the second half of fACL(x;φ), i.e. fd: = xd:. ACLs are stacked to make rich hierarchical transforms, and the latent representation z is output from this composition, i.e. zn = f(xn;φ). A permutation operation is required between ACLs to ensure the same elements are not repeatedly used in the copy operations. We use f without subscript to denote the complete transform and overload the use of φ to denote the parameters of all constituent layers.
This class of transform is known as non-volume preserving (NVP) (Dinh et al., 2017) since the volume element does not necessarily evaluate to one and can vary with each input x. Although
non-zero, the log determinant of the Jacobian is still tractable: log |∂fφ/∂x| = ∑D
j=d sj(xd:;φs). A diffeomorphic transform can also be defined with just translation operations, as was done in earlier work by Dinh et al. (2015), and this transformation is volume preserving (VP) since the volume term is one and thus has no influence in the likelihood calculation. We will examine another class of flows we term constant-volume (CV) since the volume, while not preserved, is constant across all x. Appendix A provides additional details on implementing flow-based generative models.

heading: 3 MOTIVATING OBSERVATIONS
text: Given the impressive advances of deep generative models, we sought to test their ability to quantify when an input comes from a different distribution than that of the training set. This calibration w.r.t. out-of-distribution data is essential for applications such as safety—if we were using the generative model to filter the inputs to a discriminative model—and for active learning. For the experiment, we trained the same Glow architecture described in Kingma & Dhariwal (2018)—except small enough that it could fit on one GPU1—on FashionMNIST and CIFAR-10. Appendix A provides additional implementation details. We then calculated the log-likelihood (higher value is better) and bits-per-dimension (BPD, lower value is better)2 of the test split of two different data sets of the same dimensionality—MNIST (28× 28) and SVHN (32× 32× 3) respectively. We expect the models to assign a lower probability to this data because they were not trained on it. Samples from the Glow models trained on each data set are shown in Figure 13 in the Appendix.
Beginning with FashionMNIST vs MNIST, the left subtable of Figure 1 shows the average BPD of the training data (FashionMNIST-Train), the in-distribution test data (FashionMNIST-Test), and the out-of-distribution data (MNIST-Test). We see a peculiar result: the MNIST split has the lowest BPD, more than one bit less than the FashionMNIST train and test sets. To check if this is due to outliers skewing the average, we report a (normalized) histogram in Figure 2 (a) of the log-likelihoods for the three splits. We see that MNIST (red bars) is clearly and systematically shifted to the RHS of the plot (highest likelihood).
Moving on to CIFAR-10 vs SVHN, the right subtable of Figure 1 again reports the BPD of the training data (CIFAR10-Train), the in-distribution test data (CIFAR10-Test), and the out-of-distribution data (SVHN-Test). We again see the phenomenon: the SVHN BPD is one bit lower than that of both in-distribution data sets. Figure 2 (b) shows a similar histogram of the log-likelihoods. Clearly the SVHN examples (red bars) have a systematically higher likelihood, and therefore the result is not caused by any outliers.
Subfigures (c) and (d) of Figure 2 show additional results for CelebA and ImageNet. When trained on CelebA, Glow assigns a higher likelihood to SVHN (red bars), a data set the model has never seen before. Similarly, when trained on ImageNet, Glow assigns a higher likelihood to the test splits of SVHN (red), CIFAR-10 (yellow), and CIFAR-100 (green). The difference is quite drastic in the case of SVHN (red) but modest for the two CIFAR splits. This phenomenon is not symmetric. CIFAR-10 does not have a higher likelihood under a Glow trained on SVHN; see Figure 6 in Appendix B
1Although we use a smaller model, it still produces good samples, which can be seen in Figure 13 of the Appendix, and competitive BPD (CIFAR-10: 3.46 for ours vs 3.35 for theirs).
2See Theis et al. (2016, Section 3.1) for the definitions of log-likelihood and bits-per-dimension.
for these results. We report results only for Glow, but we observed the same behavior for RNVP transforms (Dinh et al., 2017).
We next tested if the phenomenon occurs for other common deep generative models: PixelCNNs and VAEs. We do not include GANs in the comparison since evaluating their likelihood is an open problem. Figure 3 reports the same histograms as above for these models, showing the distribution of log p(x) evaluations for FashionMNIST vs MNIST (a, b) and CIFAR-10 vs SVHN (c, d). The training splits are again denoted with black bars, and the test splits with blue, and the out-of-distribution splits with red. The red bars are shifted to the right in all four plots, signifying the behavior exists in spite of the differences between model classes.

heading: 4 DIGGING DEEPER INTO THE FLOW-BASED MODEL
text: While we observed the out-of-distribution phenomenon for PixelCNN, VAE, and Glow, now we narrow our investigation to just the class of invertible generative models. The rationale is that they allow for better experimental control as, firstly, they can compute exact marginal likelihoods (unlike VAEs), and secondly, the transforms used in flow-based models have Jacobian constraints that simplify the analysis we present in Section 5. To further analyze the high likelihood of the out-of-distribution (non-training) samples, we next report the contributions to the likelihood of each term in the change-of-variables formula. At first this suggested the volume element was the primary cause of SVHN’s high likelihood, but further experiments with constant-volume flows show the problem exists with them as well.
Decomposing the change-of-variables objective. To further examine this curious phenomenon, we inspect the change-of-variables objective itself, investigating if one or both terms give the outof-distribution data a higher value. We report the constituent log p(z) and log |∂fφ/∂x| terms for NVP-Glow in Figure 4, showing histograms for log p(z) in subfigure (a) and for log |∂fφ/∂x| in subfigure (b). We see that p(z) behaves mostly as expected. The red bars (SVHN) are clearly shifted to the left, representing lower likelihoods under the latent distribution. Moving on to the volume element, this term seems to cause SVHN’s higher likelihood. Subfigure (b) shows that all of the
SVHN log-volume evaluations (red) are conspicuously shifted to the right—to higher values—when compared to CIFAR-10’s (blue and black). Since SVHN’s p(z) evaluations are only slightly less than CIFAR-10’s, the volume term dominates, resulting in SVHN having a higher likelihood.
Is the volume the culprit? In addition to the empirical evidence against the volume element, we notice that one of the terms in the change-of-variables objective—by rewarding the maximization of the Jacobian determinant—encourages the model to increase its sensitivity to perturbations in X . This behavior starkly contradicts a long history of derivative-based regularization penalties that reward the model for decreasing its sensitivity to input directions. For instance, Girosi et al. (1995) and Rifai et al. (2011) propose penalizing the Frobenius norm of a neural network’s Jacobian for classifiers and autoencoders respectively. See Appendix C for more analysis of the log volume element.
To experimentally control for the effect of the volume term, we trained Glow with constant-volume (CV) transformations. We modify the affine layers to use only translation operations (Dinh et al., 2015) but keep the 1× 1 convolutions as is. The log-determinant-Jacobian is then HW ∑ k log |Uk|,
where |Uk| is the determinant of the convolutional weights Uk for the kth flow. This makes the volume element constant across all inputs x, allowing us to isolate its effect while still keeping the model expressive. Subfigures (c) and (d) of Figure 4 show the results for this model, which we term CV-Glow (constant-volume Glow). Subfigure (c) shows a histogram of the log p(x) evaluations, just as shown before in Figure 2, and we see that SVHN (red) still achieves a higher likelihood (lower BPD) than the CIFAR-10 training set. Subfigure (d) shows the SVHN vs CIFAR-10 BPD over the course of training for NVP-Glow. Notice that there is no cross-over point in the curves.
Other experiments: random and constant images, ensembles. Other work on generative models (Sønderby et al., 2017; van den Oord et al., 2018) has noted that they often assign the highest likelihood to constant inputs. We also test this case, reporting the BPD in Appendix Figure 8 for NVP-Glow models. We find constant inputs have the highest likelihood for our models as well: 0.589 BPD for CIFAR-10. We also include in the table the BPD of random inputs for comparison.
We also hypothesized that averaging over the parameters may mitigate the phenomenon. While integration over the entire parameter space would be ideal, this is analytically and computationally difficult for Glow. Lakshminarayanan et al. (2017) show that deep ensembles can guard against over-confidence for anomalous inputs while being more practical to implement. We opted for this approach, training five Glow models independently and averaging their likelihoods to evaluate test data. Each model was given a different initialization of the parameters to help diversify the ensemble. Figure 9 in Appendix F reports a histogram of the log p(x) evaluations when averaging over the ensemble. We see nearly identical results: SVHN is still assigned a higher likelihood than the CIFAR-10 training data.

heading: 5 SECOND ORDER ANALYSIS
text: In this section, we aim to provide a more direct analysis of when another distribution might have higher likelihood than the one used for training. We propose analyzing the phenomenon by way of linearizing the difference in expected log-likelihoods. This approach undoubtedly gives a crude approximation, but as we show below, it agrees with and gives insight into some of the observations reported above. Consider two distributions: the training distribution x ∼ p∗ and some dissimilar distribution x ∼ q also with support on X . For a given generative model p(x;θ), the adversarial distribution q will have a higher likelihood than the training data’s if Eq[log p(x;θ)]−Ep∗ [log p(x;θ)] > 0. This expression is hard to analyze directly so we perform a second-order expansion of the log-likelihood around an interior point x0. Applying the expansion log p(x;θ) ≈ log p(x0;θ)+∇x0 log p(x0;θ)T (x−x0)+ 1 2 Tr{∇ 2 x0 log p(x0;θ)(x− x0)(x− x0)
T } to both likelihoods, taking expectations, and canceling the common terms, we have:
0 < Eq[log p(x;θ)]− Ep∗ [log p(x;θ)]
≈ ∇x0 log p(x0;θ)T (Eq[x]− Ep∗ [x]) + 1
2 Tr{∇2x0 log p(x0;θ)(Σq −Σp∗)}
(4)
where Σ = E [ (x− x0)(x− x0)T ] , the covariance matrix, and Tr{·} is the trace operation. Since the expansion is accurate only locally around x0, we next assume that Eq[x] = Ep∗ [x] = x0. While this at first glance may seem like a strong assumption, it is not too removed from practice since data is usually centered before being fed to the model. For SVHN and CIFAR-10 in particular, we find this assumption to hold; see Figure 5 (a) for the empirical means of each dimension of CIFAR-10 (green) and SVHN (orange). All of SVHN’s means fall within the empirical range of CIFAR-10’s, and the maximum difference between any dimension is less than 38 pixel values. Assuming equal means, we then have:
0 < Eq[log p(x;θ)]− Ep∗ [log p(x;θ)] ≈ 1
2 Tr{∇2x0 log p(x0;θ)(Σq −Σp∗)}
= 1
2 Tr
{[ ∇2x0 log pz(f(x0;φ)) +∇ 2 x0 log ∣∣∣∣∂fφ∂x0 ∣∣∣∣] (Σq −Σp∗)} , (5)
where the second line assumes the generative model to be flow-based.
Analysis of CV-Glow. We use the expression in Equation 5 to analyze the behavior of CV-Glow on CIFAR-10 vs SVHN, seeing if the difference in likelihoods can be explained by the model curvature
and data’s second moment. The second derivative terms simplify considerably for CV-Glow with a spherical latent density. Given a C × C kernel Uk, with k indexing the flow and C the number of input channels, the derivatives are ∂fh,w,c/∂xh,w,c = ∏ k ∑C j=1 uk,c,j , with h and w indexing the spatial height and width and j the columns of the kth flow’s 1× 1 convolutional kernel. The second derivative is then ∂2fh,w,c/∂x2h,w,c = 0, which allows us to write
Tr {[ ∇2x0 log p(x0;θ) ] (Σq −Σp∗) } = ∂2
∂z2 log p(z;ψ) C∑ c=1  K∏ k=1 C∑ j=1 uk,c,j 2∑ h,w (σ2q,h,w,c − σ2p∗,h,w,c).
The derivation is given in Appendix G. Plugging in the second derivative of the Gaussian’s log density—a common choice for the latent distribution in flow models (Dinh et al., 2017; Kingma & Dhariwal, 2018)—and the empirical variances, we have:
ESVHN[log p(x;θ)]− ECIFAR-10[log p(x;θ)]
≈ −1 2σ2ψ
[ α21(49.6− 61.9) + α22(52.7− 59.2) + α23(53.6− 68.1) ] = 1
2σ2ψ
[ α21 · 12.3 + α22 · 6.5 + α23 · 14.5 ] ≥ 0 where αc = K∏ k=1 C∑ j=1 uk,c,j
(6)
and where σ2ψ is the variance of the latent distribution. We know the final expression is greater than or equal to zero since all α2c ≥ 0. Equality is achieved only for σ2ψ →∞ or in the unusual case of at least one all-zero row in any convolutional kernel for all channels. Thus, the second-order expression does indeed predict we should see a higher likelihood for SVHN than for CIFAR-10. Moreover, we leave the CV-Glow’s parameters as constants to emphasize the expression is non-negative for any parameter setting. This finding is supported by our observations that using an ensemble of Glows resulted in an almost identical likelihood gap (Figure 9) and that the gap remained relatively constant over the course of training (Figure 4d). Furthermore, the ∂2 log p(z;ψ)/∂z2 term would be negative for any log-concave density function, meaning that changing the latent density to Laplace or logistic would not change the result.
Our conclusion is that SVHN simply "sits inside of" CIFAR-10—roughly same mean, smaller variance—resulting in its higher likelihood. This insight also holds true for the additional results presented in subfigures (c) and (d) of Figure 2. Examining Figure 5 (a) again, we see that ImageNet, the CIFARs, and SVHN all have nearly overlapping means and that ImageNet has the highest variance. Therefore we expect SVHN and the CIFARs to have a higher likelihood than ImageNet on an ImageNet-trained model, which is exactly what we observe in Figure 2 (d). Moreover, the degree of the differences in likelihoods agrees with the differences in variances. SVHN clearly has the smallest variance and the largest likelihood. In turn, we can artificially increase the likelihood of a data set by shrinking its variance. For RGB images, shrinking the variance is equivalent to ’graying’ the images, i.e. making the pixel values closer to 128. We show in Figure 5 (b) that doing
exactly this improves the likelihood of both CIFAR-10 and SVHN. Reducing the variance of the latent representations has the same effect, which is shown by Figure 12 in the Appendix.

heading: 6 RELATED WORK
text: This paper is inspired by and most related to recent work on evaluation of generative models. Worthy of foremost mention is the work of Theis et al. (2016), which showed that high likelihood is neither sufficient nor necessary for the model to produce visually satisfying samples. However, their paper does not consider out-of-distribution inputs. In this regard, there has been much work on adversarial inputs (Szegedy et al., 2014). While the term is used broadly, it commonly refers to inputs that have been imperceptibly modified so that the model can no longer provide an accurate output (a mis-classification, usually). Adversarial attacks on generative models have been studied by (at least) Tabacof et al. (2016) and Kos et al. (2018), but these methods of attack require access to the model. We, on the other hand, are interested in model calibration for any out-of-distribution set and especially for common data sets not constructed with any nefarious intentions nor for attack on a particular model. Various papers (Hendrycks & Gimpel, 2017; Lakshminarayanan et al., 2017; Liang et al., 2018) have reported that discriminative neural networks can produce overconfident predictions on out-of-distribution inputs. In a related finding, Lee et al. (2018) reported that it was much harder to recognize an input as out-of-distribution when the classifier was trained on CIFAR-10 in comparison to training on SVHN.
Testing the robustness of deep generative models to out-of-distribution inputs had not been investigated previously, to the best of our knowledge. However, there is work concurrent with ours that has tested their ability to detect anomalous inputs. Shafaei et al. (2018) and Hendrycks et al. (2019) also observe that PixelCNN++ cannot provide reliable outlier detection. Hendrycks et al. (2019) mitigate the CIFAR-10 vs SVHN issue by exposing the model to outliers during training. They do not consider flow-based models. Škvára et al. (2018) experimentally compare VAEs and GANs against k-nearest neighbors (kNNs), showing that VAEs and GANs outperform kNNs only when known outliers can be used for hyperparameter selection. In the work most similar to ours, Choi & Jang (2018) report the same CIFAR-10 vs SVHN phenomenon for Glow—independently confirming our motivating observation. As a fix, they propose training an ensemble of generative models with an adversarial objective and testing for out-of-training-distribution inputs by computing the WatanabeAkaike information criterion via the ensemble. This work is complementary to ours since they focus on providing a detection method whereas we are interested in understanding how and when the phenomenon can arise. The results we present in Equation 6 do not apply to Choi & Jang (2018)’s models since they use scaling operations in their affine coupling layers, making them NVP.

heading: 7 DISCUSSION
text: We have shown that comparing the likelihoods of deep generative models alone cannot identify the training set or inputs like it. Therefore we urge caution when using these models with outof-training-distribution inputs or in unprotected user-facing systems. Moreover, our analysis in Section 5 shows that the CIFAR-10 vs SVHN phenomenon would persist for any constant-volume Glow no matter the parameter values nor the choice of latent density (as long as it is log-concave). While we cannot conclude that this is a pathology in deep generative models, it does suggest the need for further work on generative models and their evaluation. The models we tested seem to be capturing low-level statistics rather than high-level semantics, and better inductive biases, optimization procedures, or uncertainty quantification may be necessary. Yet, deep generative models can detect out-of-distribution inputs when using alternative metrics (Choi & Jang, 2018) and modified training procedures (Hendrycks et al., 2019). The problem then may be a fundamental limitation of high-dimensional likelihoods. Until these open problems are better understood, we must temper the enthusiasm with which we preach the benefits of deep generative models.

heading: ACKNOWLEDGMENTS
text: We thank Aaron van den Oord, Danilo Rezende, Eric Jang, Florian Stimberg, Josh Dillon, Mihaela Rosca, Rui Shu, Sander Dieleman, and the anonymous reviewers for their helpful feedback and discussions.

heading: A ADDITIONAL IMPLEMENTATION DETAILS
text: A.1 FLOW-BASED MODELS
We have described the core building blocks of invertible generative models above, but there are several other architectural choices required in practice. Due to space requirements, we only describe them briefly, referring the reader to the original papers for details. In the most recent extension of this line of work, Kingma & Dhariwal (2018) propose the Glow architecture, with its foremost contribution being the use of 1× 1 convolutions in place of discrete permutation operations. Convolutions of this form can be thought of as a relaxed but generalized permutation, having all the representational power of the discrete version with the added benefit of parameters amenable to gradient-based training. As the transformation function becomes deeper, it becomes prone to the same scale pathologies as deep neural networks and therefore requires a normalization step of some form. Dinh et al. (2017) propose incorporating batch normalization and describe how to compute its contribution to the log-determinant-Jacobian term. Kingma & Dhariwal (2018) apply a similar normalization, which they call actnorm, but it uses trainable parameters instead of batch statistics. Lastly, both Dinh et al. (2017) and Kingma & Dhariwal (2018) use multi-scale architectures that factor out variables at regular intervals, copying them forward to the final latent representation. This gradually reduces the dimensionality of the transformations, improving upon computational costs.
For our MNIST experiments, we used a Glow architecture of 2 blocks of 16 affine coupling layers, squeezing the spatial dimension in between the 2 blocks. For our CIFAR experiments, we used 3 blocks of 8 affine coupling blocks, applying the multi-scale architecture between each block. For all coupling blocks, we used a 3-layer Highway network with 200 hidden units for MNIST and 400 hidden units for CIFAR. The networks we trained were shallower than those in Kingma & Dhariwal (2018). We also found that initializing the last layer of the coupling networks to 0 was sufficient to prevent scale pathologies, hence we did not use any form of normalization (batchnorm nor actnorm) for ease of initialization and training in a distributed setting. Convolution kernels were initialized with a truncated normal with variance 1/ √ D where D is fan-in size, except where zero-initialization is prescribed by Glow. All networks were trained with the RMSProp optimizer, with a learning rate of 1e− 5 for 100K steps, decaying by half at 80K and 90K steps. We used a prior with zero mean and unit variance for all experiments. We applied L2 regularization of 5e− 2 to CIFAR experiments. All experiments used batch size 32.
For CV-Glow, we used additive rather than affine coupling blocks, which removes the influence of coupling blocks on the log-det-Jacobian term. The volume change of the 1× 1 convolutions depend on its weights rather than the input, so the network has a constant volume change.
A.2 PIXELCNN
We trained a GatedPixelCNN with a categorical distribution. For FashionMNIST, we used a network with 5 gated layers with 32 features, and the final skip connection layer with size 256. We trained for 100K steps with the Adam optimizer and an initial learning rate of 1e− 4, and decaying at 80K and 90K steps. For CIFAR experiments, we used 15 dated layers with 128 features, and a skip connection size of 1024. Convolutions were initialized with Tensorflow’s variance scaling initializer with a uniform distribution and a scale of 1. We trained for 200K steps with the RMSProp optimizer with an initial learning rate of 1e− 4, decaying by 1/3 at 120K, 180K, and 195K steps. All experiments used batch size 32.
A.3 VARIATIONAL AUTO-ENCODERS
We refer to (Rosca et al., 2018, Appendix K) for additional details on the VAEs used in our CIFAR experiments. We used the CIFAR configurations without modification. For FashionMNIST, we used the encoder given in Table 4 of (Rosca et al., 2018, Appendix K, Table 4) and a decoder composed of one linear layer with 7 ∗ 7 ∗ 64 hidden units, followed by a reshape and three transposed convolutions of feature sizes 32, 32, 256 and strides 2, 2, 1. Weights were initialized with a normal distribution with variance 0.02. We trained for 200K steps using the RMSProp optimizer, with a constant learning rate of 1e− 4.

heading: B RESULTS ILLUSTRATING ASYMMETRIC BEHAVIOR

heading: C ANALYZING THE CHANGE-OF-VARIABLES FORMULA AS AN
text: OPTIMIZATION FUNCTION
Consider the intuition underlying the volume term in the change of variables objective (Equation 3). As we are maximizing the Jacobian’s determinant, it means that the model is being encouraged to maximize the ∂fj/∂xj partial derivatives. In other words, the model is rewarded for making the transformation sensitive to small changes in x. This behavior starkly contradicts a long history of derivative-based regularization penalties. Dating back at least to (Girosi et al., 1995), penalizing the Frobenius norm of a neural network’s Jacobian—which upper bounds the volume term3—has been shown to improve generalization. This agrees with intuition since we would like the model to be insensitive to small changes in the input, which are likely noise. Moreover, Bishop (1995) showed that training a network under additive Gaussian noise is equivalent to Jacobian regularization, and Rifai et al. (2011) proposed contractive autoencoders, which penalize the Jacobian-norm of the encoder. Allowing invertible generative models to maximize the Jacobian term without constraint suggests, at minimum, that these models will not learn robust representations.
Limiting Behavior. We next attempt to quantify the limiting behavior of the log volume element. Let us assume, for the purposes of a general treatment, that the bijection fφ is an L-Lipschitz function. Both terms in Equation 3 can be bounded as follows:
log p(x;θ) = log pz(f(x;φ))︸ ︷︷ ︸ O(maxz log pz(z) + log ∣∣∣∣∂fφ∂x ∣∣∣∣︸ ︷︷ ︸
O(D logL)
≤ max z log pz(z) +D logL (7)
where L is the Lipschitz constant, D the dimensionality, and O(maxz log pz(z)) an expression for the (log) mode of p(z). We will make this mode term for concrete for Gaussian distributions below. The bound on the volume term follows from Hadamard’s inequality:
log ∣∣∣∣∂fφ∂x ∣∣∣∣ ≤ log D∏
j=1
∣∣∣∣∂fφ∂x ej ∣∣∣∣ ≤ log(L |e·|)D = D logL
where ej is an eigenvector. While this expression is too general to admit any strong conclusions, we can see from it that the ‘peakedness’ of the distribution represented by the mode must keep pace with the Lipschitz constant, especially as dimensionality increases, in order for both terms to contribute equally to the objective.
We can further illuminate the connection between L and the concentration of the latent distribution through the following proposition:
3It is easy to show the upper bound via Hadamard’s inequality: det ∂f/∂x ≤ ||∂f/∂x||F .
Proposition 1. Assume x ∼ p∗ is distributed with moments E[x] = µx and V ar[x] = σ2x. Moreover, let f : X 7→ Z be L-Lipschitz and f(µx) = µz . We then have the following concentration inequality for some constant δ:
P (|f(x)− µz| ≥ δ) ≤ L2σ2x δ2 .
Proof : From the fact that f is L-Lipschitz, we know |f(x)− µz| ≤ L ∣∣x− f−1(µz)∣∣. Assuming µx = f −1(µz)), we can apply Chebyshev’s inequality to the RHS: Pr(L
∣∣x− f−1(µz)∣∣ ≥ δ) ≤ L2σ2x/δ 2. Since L ∣∣x− f−1(µz)∣∣ ≥ |f(x)− µz|, we can plug the RHS into the inequality and the bound will continue to hold.
From the inequality we can see that the latent distribution can be made more concentrated by decreasing L and/or the data’s variance σ2x. Since the latter is fixed, optimization only influences L. Yet, recall that the volume term in the change-of-variables objective rewards increasing f ’s derivatives and thus L. While we have given an upper bound and therefore cannot say that increasing L will necessarily decrease concentration in latent space, it is for certain that leaving L unconstrained does not directly pressure the f(x) evaluations to concentrate.
Previous work (Dinh et al., 2015; 2017; Kingma & Dhariwal, 2018) has almost exclusively used a factorized zero-mean Gaussian as the latent distribution, and therefore we examine this case in particular. The log-mode can be expressed as −D/2 · log 2πσ2z , making the likelihood bound
log N(f(x;φ); 0, σ2zI) + log ∣∣∣∣∂fφ∂x ∣∣∣∣ ≤ −D2 log 2πσ2z +D logL. (8) We see that both terms scale with D although in different directions, with the contribution of the z-distribution becoming more negative and the volume term’s becoming more positive. We performed a simulation to demonstrate this behavior on the two moons data set, which is shown in Figure 7 (a). We replicated the original two dimensions to create data sets of dimensionality of up to 100. The results are shown in Figure 7 (b). The empirical values of the two terms are shown by the solid lines, and indeed, we see they exhibit the expected diverging behavior as dimensionality increases.

heading: D GLOW WITH SIGMOID PARAMETRIZATION
text: Upon reading the open source implementation of Glow,4 we found that Kingma & Dhariwal (2018) in practice parametrize the scaling factor as sigmoid(s(xd:;φs)) instead of exp{s(xd:;φs)}. This
4https://github.com/openai/glow/blob/master/model.py#L376
choice allows the volume only to decrease and thus results in the volume term being bounded as (ignoring the convolutional transforms)
log ∣∣∣∣∂fφ∂x ∣∣∣∣ = F∑
f=1 df∑ j=1 log sigmoid(sf,j(xdf :;φs)) ≤ FD log 1 = 0 (9)
where f indexes the flows and df the dimensionality at flow f . Interestingly, this parametrization has a fixed upper bound of zero, removing the dependence on D found in Equation 8. We demonstrate the change in behavior introduced by the alternate parametrization via the same two moon simulation. The only difference is that the RNVP transforms use a sigmoid parametrizations for the scaling operation. See Figure 7 (c) for the results: we see that now both change-of-variable terms are oriented downward as dimensionality grows. We conjecture this parametrization helps condition the log-likelihood, limiting the volume term’s influence, when training the large models (∼ 90 flows) used by Kingma & Dhariwal (2018). However, it does not fix the out-of-distribution over-confidence we report in Section 3.

heading: E CONSTANT AND RANDOM INPUTS

heading: F ENSEMBLING GLOWS
text: The likelihood function technically measures how likely the parameters are under the data (and not how likely the data is under the model), and perhaps a better quantity would be the posterior predictive distribution p(xtest|xtrain) = 1M ∑ m p(xtest|θm) where we draw samples from posterior distribution θm ∼ p(θ|xtrain). Intutitively, it seems that such an integration would be more robust than a single maximum likelihood point estimate. As a crude approximation to Bayesian inference, we tried averaging over ensembles of generative models since Lakshminarayanan et al. (2017) showed that ensembles of discriminative models are robust to out-of-distribution inputs. We compute an “ensemble predictive distribution” as p(x) = 1M ∑ m p(x;θm), where m indexes over models. However, as Figure 9 shows, ensembles did not significantly change the relative difference between in-distribution (CIFAR-10, black and blue) and out-of-distribution (SVHN, red).

heading: G DERIVATION OF CV-GLOW’S LIKELIHOOD DIFFERENCE
text: We start with Equation 5:
1 2 Tr {[ ∇2x0 log pz(f(x0;φ)) +∇ 2 x0 log ∣∣∣∣∂fφ∂x0 ∣∣∣∣] (Σq −Σp∗)} .
The volume element for CV-Glow does not depend on x0 and therefore drops from the equation: ∇2x0 log ∣∣∣∣∂fφ∂x0 ∣∣∣∣ = ∇2x0HW∑ k log |Uk| = 0 (10)
where Uk denotes the kth 1× 1-convolution’s kernel. Moving on to the first term, the log probability under the latent distribution, we have:
∇2x0 log p(f(x0);ψ) = ∇ 2 x0 { −1 2σ2ψ ||f(x0)||22 − D 2 log 2πσ2ψ }
= ∇x0 { −1 σ2ψ (∑ d fd(x0) ) ∇x0f(x0) }
= −1 σ2ψ
[ ∇x0f(x0)(∇x0f(x0))T + (∑ d fd(x0) ) ∇2x0f(x0) ] .
(11)
Since f is comprised of translation operations and 1× 1 convolutions, its partial derivatives involve just the latter (as the former are all ones), and therefore we have the partial derivatives:
∂fh,w,c(x0)
∂xh,w,c = K∏ k=1 Ck∑ j=1 uk,c,j , ∂2fh,w,c(x0) ∂x2h,w,c = 0 (12)
where h and w index the input spatial dimensions, c the input channel dimensions, k the series of flows, and j the column dimensions of the Ck × Ck-sized convolutional kernel Uk. The diagonal elements of ∇x0f(x0)(∇x0f(x0))T are then ( ∏K k=1 ∑Ck j=1 uk,c,j)
2, and the diagonal element of ∇2x0f(x0) are all zero. Then returning to the full equation, for the constant-volume Glow model we have:
1 2 Tr {[ ∇2x0 log p(f(x0)) +∇ 2 x0 log ∣∣∣∣ ∂f∂x0 ∣∣∣∣] (Σq −Σp∗)}
= 1 2 Tr {[ ∇2x0 log p(f(x0)) ] (Σq −Σp∗) } = −1 2σ2ψ Tr {[ ∇x0f(x0)(∇x0f(x0))T + (∑ d fd(x0) ) ∇2x0f(x0) ] (Σq −Σp∗) }
= −1 2σ2ψ ∑ l,m {[ ∇x0f(x0)(∇x0f(x0))T + (∑ d fd(x0) ) ∇2x0f(x0) ] (Σq −Σp∗) } l,m .
(13)
Lastly, we assume that both Σq and Σp∗ are diagonal and thus the element-wise multiplication with ∇2x0 log p(f(x0)) collects only its diagonal elements:
−1 2σ2ψ ∑ l,m {[ ∇x0f(x0)(∇x0f(x0))T + (∑ d fd(x0) ) ∇2x0f(x0) ] (Σq −Σp∗) } l,m
= −1 2σ2ψ H∑ h W∑ w C∑ c  K∏ k=1 Ck∑ j=1 uk,c,j 2 (σ2q,h,w,c − σ2p∗,h,w,c) = −1 2σ2ψ C∑ c  K∏ k=1 Ck∑ j=1 uk,c,j 2 H∑ h W∑ w (σ2q,h,w,c − σ2p∗,h,w,c)
(14)
where we arrived at the last line by rearranging the sum to collect the shared channel terms.

heading: H HISTOGRAM OF DATA STATISTICS
text: 0.15 0.10 0.05 0.00 0.05 0.10 0.15 0.20 0
2
4
6
8
10
12
14
16 SVHN
CIFAR-10
0.2 0.4 0.6 0.8 1.0 1.2 1.4 0
2
4
6
8
10
12
14 SVHN
CIFAR-10
0.0 0.5 1.0 1.5 2.0 0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0 SVHN
CIFAR-10
(a) Latent codes for CIFAR10-SVHN trained on CIFAR-10.
0.6 0.4 0.2 0.0 0.2 0.4 0.6 0.0
0.5
1.0
1.5
2.0
2.5
3.0
MNIST FashionMNIST
0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 0
2
4
6
8
10
12
14
MNIST FashionMNIST
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
MNIST FashionMNIST
(b) Latent codes for FashionMNIST-MNIST trained on FashionMNIST.
Figure 11: Analysis of codes obtained using CV-Glow model. Histogram of means (left column)
µd = 1 N
∑N
n=1 znd, standard deviation (middle column) σd =
√
1 N−1
∑N
n=1(znd − µd)2 and norms normalized by √ D (right column) computed as |zn|√ D = √ 1 D ∑ d z 2 nd.

heading: I RESULTS ILLUSTRATING EFFECT OF GRAYING ON CODES
text: Figure 12 shows the effect of graying on codes.

heading: J SAMPLES

Review summaries:
This paper displays an occurrence of density models assigning higher likelihood to out-of-distribution inputs compared to the training distribution. Specifically, density models trained on CIFAR10 have higher likelihood on SVHN than CIFAR10. This is an interesting observation because the prevailing assumption is that density models can distinguish inliers from outliers. However, this phenomenon is not encountered when comparing MNIST and NotMNIST. The SVHN/CIFAR10 phenomenon has also been shown in concurrent work [1].