D O D EEP G ENERATIVE M ODELS K NOW W HAT T HEY D ON ’ T K NOW ?
A BSTRACT
A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data.
A plethora of work has demonstrated that it is easy to ﬁnd or synthesize inputs for which a neural network is highly conﬁdent yet wrong.
Generative models are widely viewed to be robust to such mistaken conﬁdence as modeling the density of the input features can be used to detect novel, out-of-distribution inputs.
In this paper we challenge this assumption.
We ﬁnd that the density learned by ﬂow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e. CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former.
Moreover, we ﬁnd evidence of this phenomenon when pairing several popular image data sets: FashionMNIST vs MNIST, CelebA vs SVHN, ImageNet vs CIFAR-10 / CIFAR-100 / SVHN.
To investigate this curious behavior, we focus analysis on ﬂow-based generative models in particular since they are trained and evaluated via the exact marginal likelihood.
We ﬁnd such behavior persists even when we restrict the ﬂows to constant-volume transformations.
These transformations admit some theoretical analysis, and we show that the difference in likelihoods can be explained by the location and variances of the data and the model curvature.
Our results caution against using the density estimates from deep generative models to identify inputs similar to the training distribution until their behavior for out-of-distribution inputs is better understood.
I NTRODUCTION
Deep learning has achieved impressive success in applications for which the goal is to model a conditional distribution p ( y | x ) , with y being a label and x the features.
While the conditional model p ( y | x ) may be highly accurate on inputs x sampled from the training distribution, there are no guarantees that the model will work well on x ’s drawn from some other distribution.
For example, Louizos & Welling (2017) show that simply rotating an MNIST digit can make a neural network predict another class with high conﬁdence (see their Figure 1a).
Ostensibly, one way to avoid such overconﬁdently wrong predictions would be to train a density model p ( x ; θ ) (with θ denoting the parameters) to approximate the true distribution of training inputs p ∗ ( x ) and refuse to make a prediction for any x that has a sufﬁciently low density under p ( x ; θ ) .
The intuition is that the discriminative model p ( y | x ) likely did not observe enough samples in that region to make a reliable decision for those inputs.
This idea has been proposed by various papers, cf.
(Bishop, 1994), and as recently as in the panel discussion at Advances in Approximate Bayesian Inference (AABI)(Blei et al., 2017).
Anomaly detection is just one motivating example for which we require accurate densities, and others include information regularization (Szummer & Jaakkola, 2003), open set recognition (Herbei & Wegkamp, 2006), uncertainty estimation, detecting covariate shift, active learning, model-based reinforcement learning, and transfer learning.
Accordingly, these applications have lead to widespread interest in deep generative models, which take many forms such as variational auto-encoders (VAEs) (Kingma & Welling, 2014; Rezende et al., 2014), generative adversarial networks (GANs) (Goodfel- low et al., 2014), auto-regressive models (
van den Oord et al., 2016b;a), and invertible latent variable models (Tabak & Turner, 2013).
The last two classes—auto-regressive and invertible models—are especially attractive since they offer exact computation of the marginal likelihood, requiring no approximate inference techniques.
In this paper, we investigate if modern deep generative models can be used for anomaly detection, as suggested by Bishop (1994) and the AABI pannel (Blei et al., 2017), expecting a well-calibrated model to assign higher density to the training data than to some other data set.
However, we ﬁnd this to not be the case: when trained on CIFAR-10 (Krizhevsky & Hinton, 2009), VAEs, autoregressive models, and ﬂow-based generative models all assign a higher density to SVHN (Netzer et al., 2011) than to the training data.
We ﬁnd this observation to be quite problematic and unintuitive since
SVHN’s digit images are so visually distinct from the dogs, horses, trucks, boats, etc. found in
CIFAR-10.
Yet this phenomenon is not restricted to CIFAR-10 vs SVHN, and we report similar ﬁndings for models trained on CelebA and ImageNet.
We go on to study these curious observations in ﬂow-based models in particular since they allow for exact marginal density calculations.
When the ﬂow is restricted to have constant volume across inputs, we show that the out-of-distribution behavior can be explained in terms of the data’s variance and the model’s curvature.
To the best of our knowledge, we are the ﬁrst to report these unintuitive ﬁndings for a variety of deep generative models and image data sets.
Moreover, our experiments with ﬂow-based models isolate some crucial experimental variables such as the effect of constant-volume vs non-volume-preserving transformations.
Lastly, our analysis provides some simple but general expressions for quantifying the gap in the model density between two data sets.
We close the paper by urging more study of the out-of-training-distribution properties of deep generative models. Understanding their behaviour in this setting is crucial for their deployment to the real world.
B ACKGROUND
We begin by establishing notation and reviewing the necessary background material.
We denote matrices with upper-case and bold letters (e.g. X ), vectors with lower-case and bold (e.g. x ), and scalars with lower-case and no bolding (e.g. x ).
As our focus is on generative models, let the collection of all observations be denoted by X
= { x n } Nn =1 with x representing a vector containing all features and, if present, labels.
All N examples are assumed independently and identically drawn from some population x ∼ p ∗ ( x ) (which is unknown) with support denoted X .
We deﬁne the model density function to be p ( x ; θ ) where θ ∈ Θ are the model parameters, and let the model likelihood be denoted p ( X ; θ ) =
(cid:81)
Nn =1 p ( x n ; θ ) .
2. T RAINING N EURAL G ENERATIVE M ODELS
Given (training) data X and a model class { p ( · ; θ ) : θ ∈ Θ } , we are interested in ﬁnding the parameters θ that make the model closest to the true but unknown data distribution p ∗ ( x ) .
We can quantify this gap in terms of a Kullback–Leibler divergence (KLD):
[Equation]
(1) where the ﬁrst term in the right-most expression is the average log-likelihood and the second is the entropy of the true distribution.
As the latter is a ﬁxed constant, minimizing the KLD amounts to ﬁnding the parameter settings that maximize the data’s log density: θ ∗ = arg max θ log p ( X ; θ ) = arg max θ (cid:80) Nn =1 log p ( x n ; θ ) .
Note that p ( x n ; θ ) alone does not have any interpretation as a probability.
To extract probabilities from the model density, we need to integrate over some region Ω : P ( Ω ) =
(cid:82) Ω p ( x ; θ )
d x .
Adding noise to the data during model optimization can mock this integration step, encouraging the density model to output something nearer to probabilities (Theis et al., 2016):
[Equation]
where ˜ δ is a sample from p ( δ ) .
The resulting objective is a lower-bound, making it a suitable optimization target.
All models in all of the experiments that we report are trained with input noise.
Due to this ambiguity between densities and probabilities, we call the quantity log p ( X + ˜ ∆ ; θ ) a ‘log-likelihood,’ even if X is drawn from a distribution unlike the training data.
Regarding the choice of density model, we could choose one of the standard density functions for p ( x n ; θ ) , e.g. a Gaussian, but these may not be suitable for modeling the complex, high-dimensional data sets we often observe in the real world.
Hence, we want to parametrize the model density with some high-capacity function f , which is usually chosen to be a neural network.
That way the model has a somewhat compact representation and can be optimized via gradient ascent.
We experiment with three variants of neural generative models: autoregressive, latent variable, and invertible.
In the ﬁrst class, we study the PixelCNN (van den Oord et al., 2016b), and due to space constraints, we refer the reader to van den Oord et al.
(2016b) for its deﬁnition.
As a representative of the second class, we use a VAE (Kingma & Welling, 2014; Rezende et al., 2014).
See Rosca et al.
(2018) for descriptions of the precise versions we use.
Lastly, invertible ﬂow-based generative models are the third class.
We deﬁne them in detail below since we study them with the most depth.
2. G ENERATIVE M ODELS VIA C HANGE OF V ARIABLES
The VAE and many other generative models are deﬁned as a joint distribution between the observed and latent variables.
However, another path forward is to perform a change of variables .
In this case x and z are one and the same, and there is no longer any notion of a product space
X ×
Z .
Let f : X
(cid:55)→ Z be a diffeomorphism from the data space X to a latent space Z .
Using f then allows us to compute integrals over z as an integral over x and vice versa:
[Equation]
where | ∂ f /∂ x | and | ∂ f − 1 /∂
z | are known as the volume elements as they adjust for the volume change under the alternate measure.
Speciﬁcally, when the change is w.r.t.
coordinates, the volume element is the determinant of the diffeomorphism’s Jacobian matrix, which we denote as | ∂ f /∂ x | .
The change of variables formula is a powerful tool for generative modeling as it allows us to deﬁne a distribution p ( x ) entirely in terms of an auxiliary distribution p ( z ) , which we are free to choose, and f .
Denote the parameters of the change of variables model as θ = { φ , ψ } with φ being the diffeomorphism’s parameters, i.e. f ( x ; φ ) , and ψ being the auxiliary distribution’s parameters, i.e. p ( z ; ψ ) .
We can perform maximum likelihood estimation for the model as follows:
[Equation]
Optimizing ψ must be done carefully so as to not result in a trivial model.
For instance, optimization could make p ( z ; ψ ) close to uniform if there are no constraints on its variance.
For this reason, most implementations leave ψ as ﬁxed (usually a standard Gaussian) in practice.
Likewise, we assume it as ﬁxed from here forward, thus omitting ψ from equations to reduce notational clutter.
After training, samples can be drawn from the model via the inverse transform: ˜ x
= f − 1 ( ˜ z ; φ ) , ˜ z ∼ p ( z ) .
For the particular form of f , most work to date has constructed the bijection from afﬁne coupling layers (ACLs) (Dinh et al., 2017), which transform x by way of translation and scaling operations.
Speciﬁcally, ACLs take the form: f ACL ( x ; φ ) =
[exp { s ( x d : ; φ s ) } (cid:12) x : d + t ( x d : ; φ t ) , x d : ] , where (cid:12) denotes an element-wise product.
This transformation, ﬁrstly, splits the input vector in half, i.e. x =
[ x : d , x d : ] (using Python list syntax).
Then the second half of the vector is fed into two arbitrary neural networks (possibly with tied parameters) whose outputs are denoted t ( x d : ; φ t ) and s ( x d : ; φ s ) , with φ · being the collection of weights and biases.
Finally, the output is formed by (1) scaling the ﬁrst half of the input by one neural network output, i.e. exp { s ( x d : ; φ s ) } (cid:12) x : d , (2) translating the result of the scaling operation by the second neural network output, i.e. ( · ) + t ( x d : ; φ t ) , and (3) copying the second half of x forward, making it the second half of f ACL ( x ; φ ) , i.e. f d : = x d : .
ACLs are stacked to make rich hierarchical transforms, and the latent representation z is output from this composition, i.e. z n = f ( x n ; φ ) .
A permutation operation is required between ACLs to ensure the same elements are not repeatedly used in the copy operations.
We use f without subscript to denote the complete transform and overload the use of φ to denote the parameters of all constituent layers.
This class of transform is known as non-volume preserving (NVP) (Dinh et al., 2017) since the volume element does not necessarily evaluate to one and can vary with each input x .
Although non-zero, the log determinant of the Jacobian is still tractable: log | ∂ f φ /∂ x | =
(cid:80) Dj =
d s j ( x d : ; φ s ) .
A diffeomorphic transform can also be deﬁned with just translation operations, as was done in earlier work by Dinh et al.
and this transformation is volume preserving (VP) since the volume term is one and thus has no inﬂuence in the likelihood calculation.
We will examine another class of ﬂows we term constant-volume (CV) since the volume, while not preserved, is constant across all x .
Appendix A provides additional details on implementing ﬂow-based generative models.
M OTIVATING O BSERVATIONS
Given the impressive advances of deep generative models, we sought to test their ability to quantify when an input comes from a different distribution than that of the training set.
This calibration w.r.t.
out-of-distribution data is essential for applications such as safety—if we were using the generative model to ﬁlter the inputs to a discriminative model—and for active learning.
For the experiment, we trained the same Glow architecture described in Kingma & Dhariwal (2018)—except small enough that it could ﬁt on one GPU 1 —on FashionMNIST and CIFAR-10.
Appendix A provides additional implementation details.
We then calculated the log-likelihood (higher value is better) and bits-per-dimension (BPD, lower value is better) 2 of the test split of two different data sets of the same dimensionality—MNIST ( 28 × 28) and SVHN ( 32 × 32 × 3) respectively. We expect the models to assign a lower probability to this data because they were not trained on it.
Samples from the Glow models trained on each data set are shown in Figure 13 in the Appendix.
[Table]
FashionMNIST
[Table]
[Table]
Glow Trained on MNIST MNIST
[Table] [Table]
[Table] Set
[Table]
Glow Trained on CIFAR-
CIFAR10-Train 3.
CIFAR10-Test 3.
SVHN-Test 2.
Glow Trained on SVHN
SVHN-Test 2.
Beginning with FashionMNIST vs MNIST, the left subtable of Figure 1 shows the average BPD of the training data (FashionMNIST-Train), the in-distribution test data (FashionMNIST-Test), and the out-of-distribution data (MNIST-Test).
We see a peculiar result: the MNIST split has the lowest BPD, more than one bit less than the FashionMNIST train and test sets.
To check if this is due to outliers skewing the average, we report a (normalized) histogram in Figure 2 (a) of the log-likelihoods for the three splits.
We see that MNIST (red bars) is clearly and systematically shifted to the RHS of the plot (highest likelihood).
Moving on to CIFAR-10 vs SVHN, the right subtable of Figure 1 again reports the BPD of the training data (CIFAR10-Train), the in-distribution test data (CIFAR10-Test), and the out-of-distribution data (SVHN-Test).
We again see the phenomenon: the SVHN BPD is one bit lower than that of both in-distribution data sets.
Figure 2 (b) shows a similar histogram of the log-likelihoods.
Clearly the
SVHN examples (red bars) have a systematically higher likelihood, and therefore the result is not caused by any outliers.
Subﬁgures (c) and (d) of Figure 2 show additional results for CelebA and ImageNet.
When trained on CelebA, Glow assigns a higher likelihood to SVHN (red bars), a data set the model has never seen before.
Similarly, when trained on ImageNet, Glow assigns a higher likelihood to the test splits of SVHN (red), CIFAR-10 (yellow), and CIFAR-100 (green).
The difference is quite drastic in the case of SVHN (red) but modest for the two CIFAR splits.
This phenomenon is not symmetric.
CIFARdoes not have a higher likelihood under a Glow trained on SVHN; see Figure 6 in Appendix B [Figure]
for these results.
We report results only for Glow, but we observed the same behavior for RNVP transforms (Dinh et al., 2017).
We next tested if the phenomenon occurs for other common deep generative models: PixelCNNs and VAEs.
We do not include GANs in the comparison since evaluating their likelihood is an open problem.
Figure 3 reports the same histograms as above for these models, showing the distribution of log p ( x ) evaluations for FashionMNIST vs MNIST (a, b) and CIFAR-10 vs SVHN (c, d).
The training splits are again denoted with black bars, and the test splits with blue, and the out-of-distribution splits with red.
The red bars are shifted to the right in all four plots, signifying the behavior exists in spite of the differences between model classes.
D IGGING D EEPER INTO THE F LOW -B ASED M ODEL
While we observed the out-of-distribution phenomenon for PixelCNN, VAE, and Glow, now we narrow our investigation to just the class of invertible generative models.
The rationale is that they allow for better experimental control as, ﬁrstly, they can compute exact marginal likelihoods (unlike VAEs), and secondly, the transforms used in ﬂow-based models have Jacobian constraints that simplify the analysis we present in Section 5.
To further analyze the high likelihood of the out-of-distribution (non-training) samples, we next report the contributions to the likelihood of each term in the change-of-variables formula.
At ﬁrst this suggested the volume element was the primary cause of SVHN’s high likelihood, but further experiments with constant-volume ﬂows show the problem exists with them as well.
Decomposing the change-of-variables objective.
To further examine this curious phenomenon, we inspect the change-of-variables objective itself, investigating if one or both terms give the outof-distribution data a higher value.
We report the constituent log p ( z ) and log | ∂ f φ /∂ x | terms for
NVP-Glow in Figure 4, showing histograms for log p ( z ) in subﬁgure (a) and for log | ∂ f φ /∂ x | in subﬁgure (b).
We see that p ( z ) behaves mostly as expected.
The red bars (SVHN) are clearly shifted to the left, representing lower likelihoods under the latent distribution.
Moving on to the volume element, this term seems to cause SVHN’s higher likelihood. Subﬁgure (b) shows that all of the [Figure] [Figure]
SVHN log-volume evaluations (red) are conspicuously shifted to the right—to higher values—when compared to
(blue black). Since SVHN’s p ( z ) evaluations are only slightly less than CIFAR-10’s, the volume term dominates, resulting in SVHN having a higher likelihood. Is the volume the culprit?
In addition to the empirical evidence against the volume element, we notice that one of the terms in the change-of-variables objective—by rewarding the maximization of the Jacobian determinant—encourages the model to increase its sensitivity to perturbations in X .
This behavior starkly contradicts a long history of derivative-based regularization penalties that reward the model for decreasing its sensitivity to input directions.
For instance, Girosi et al.
and
Rifai et al.
(2011) propose penalizing the Frobenius norm of a neural network’s Jacobian for classiﬁers and autoencoders respectively.
See Appendix C for more analysis of the log volume element.
To experimentally control for the effect of the volume term, we trained Glow with constant-volume (CV) transformations.
We modify the afﬁne layers to use only translation operations (Dinh et al., 2015) but keep the 1 × 1 convolutions as is.
The log-determinant-Jacobian is then HW (cid:80) k log | U k
| , where | U k | is the determinant of the convolutional weights U k for the k th ﬂow.
This makes the volume element constant across all inputs x , allowing us to isolate its effect while still keeping the model expressive.
Subﬁgures (c) and (d) of Figure 4 show the results for this model, which we term CV-Glow (constant-volume Glow).
Subﬁgure (c) shows a histogram of the log p ( x ) evaluations, just as shown before in Figure 2, and we see that SVHN (red) still achieves a higher likelihood (lower
BPD) than the CIFAR-10 training set.
Subﬁgure (d) shows the SVHN vs CIFAR-10 BPD over the course of training for NVP-Glow.
Notice that there is no cross-over point in the curves.
Other experiments: random and constant images, ensembles.
Other work on generative models (Sønderby et al., 2017; van den Oord et al., 2018) has noted that they often assign the highest likelihood to constant inputs.
We also test this case, reporting the BPD in Appendix Figure 8 for NVP-Glow models.
We ﬁnd constant inputs have the highest likelihood for our models as well: 0.589 BPD for
CIFAR-10.
We also include in the table the BPD of random inputs for comparison.
We also hypothesized that averaging over the parameters may mitigate the phenomenon.
While integration over the entire parameter space would be ideal, this is analytically and computationally difﬁcult for Glow.
Lakshminarayanan et al.
(2017) show that deep ensembles can guard against over-conﬁdence for anomalous inputs while being more practical to implement.
We opted for this approach, training ﬁve Glow models independently and averaging their likelihoods to evaluate test data.
Each model was given a different initialization of the parameters to help diversify the ensemble.
Figure 9 in Appendix F reports a histogram of the log p ( x ) evaluations when averaging over the ensemble.
We see nearly identical results: SVHN is still assigned a higher likelihood than the
CIFAR-10 training data.
S ECOND O RDER A NALYSIS
In this section, we aim to provide a more direct analysis of when another distribution might have higher likelihood than the one used for training.
We propose analyzing the phenomenon by way of linearizing the difference in expected log-likelihoods.
This approach undoubtedly gives a crude approximation, but as we show below, it agrees with and gives insight into some of the observations reported above.
Consider two distributions: the training distribution x ∼ p ∗ and some dissimilar distribution x ∼ q also with support on X .
For a given generative model p ( x ; θ ) , the adversarial distribution q will have a higher likelihood than the training data’s if E q [log p ( x ; θ )] − E p ∗ [log p ( x ; θ )] > 0 .
This expression is hard to analyze directly so we perform a second-order expansion of the log-likelihood around an interior point x 0 .
Applying the expansion log p ( x ; θ )
≈ log p ( x 0 ; θ )+ ∇ x 0 log p ( x 0 ; θ ) T ( x − x 0 )
+ 12 Tr {∇ 2 x 0 log p ( x 0 ; θ )( x − x 0 )( x − x 0 ) T } to both likelihoods, taking expectations, and canceling the common terms, we have: [Equation]
[Equation]
where Σ = E (cid:2) ( x − x 0 )( x − x 0 ) T (cid:3) , the covariance matrix, and Tr {·} is the trace operation.
Since the expansion is accurate only locally around x 0 , we next assume that E q [ x ] = E p ∗ [ x ] = x 0 .
While this at ﬁrst glance may seem like a strong assumption, it is not too removed from practice since data is usually centered before being fed to the model.
For SVHN and CIFAR-10 in particular, we ﬁnd this assumption to hold; see Figure 5 (a) for the empirical means of each dimension of CIFAR- (green) and SVHN (orange).
All of SVHN’s means fall within the empirical range of CIFAR-10’s, and the maximum difference between any dimension is less than 38 pixel values.
Assuming equal means, we then have:
[Equation]
where the second line assumes the generative model to be ﬂow-based.
Analysis of CV-Glow.
We use the expression in Equation 5 to analyze the behavior of CV-Glow on
CIFAR-10 vs SVHN, seeing if the difference in likelihoods can be explained by the model curvature and data’s second moment.
The second derivative terms simplify considerably for CV-Glow with a spherical latent density.
Given a C × C kernel U k , with k indexing the ﬂow and C the number of input channels, the derivatives are ∂f h,w,c /∂x h,w,c
= (cid:81) k
(cid:80) Cj =
1 u k,c,j , with h and w indexing the spatial height and width and j the columns of the k th ﬂow’s 1 × 1 convolutional kernel.
The second derivative is then ∂ 2 f h,w,c /∂x 2 h,w,c = 0 , which allows us to write
[Equation]
The derivation is given in Appendix G. Plugging in the second derivative of the Gaussian’s log density—a common choice for the latent distribution in ﬂow models (Dinh et al., 2017; Kingma &
Dhariwal, 2018)—and the empirical variances, we have:
[Equation]
k =
C (cid:88)
j = u k,c,j (6) and where σ 2 ψ is the variance of the latent distribution.
We know the ﬁnal expression is greater than or equal to zero since all α 2 c ≥ 0 .
Equality is achieved only for σ 2 ψ → ∞ or in the unusual case of at least one all-zero row in any convolutional kernel for all channels.
Thus, the second-order expression does indeed predict we should see a higher likelihood for SVHN than for CIFAR-10.
Moreover, we leave the CV-Glow’s parameters as constants to emphasize the expression is non-negative for any parameter setting .
This ﬁnding is supported by our observations that using an ensemble of Glows resulted in an almost identical likelihood gap (Figure 9) and that the gap remained relatively constant over the course of training (Figure 4d).
Furthermore, the ∂ 2 log p ( z ; ψ ) /∂z 2 term would be negative for any log-concave density function, meaning that changing the latent density to Laplace or logistic would not change the result.
[Figure] Our conclusion is that SVHN simply "sits inside of" CIFAR-10—roughly same mean, smaller variance—resulting in its higher likelihood.
This insight also holds true for the additional results presented in subﬁgures (c) and (d) of Figure 2.
Examining Figure 5 (a) again, we see that ImageNet, the CIFARs, and SVHN all have nearly overlapping means and that ImageNet has the highest variance.
Therefore we expect SVHN and the CIFARs to have a higher likelihood than ImageNet on an ImageNet-trained model, which is exactly what we observe in Figure 2 (d).
Moreover, the degree of the differences in likelihoods agrees with the differences in variances.
SVHN clearly has the smallest variance and the largest likelihood.
In turn, we can artiﬁcially increase the likelihood of a data set by shrinking its variance.
For RGB images, shrinking the variance is equivalent to ’graying’ the images, i.e. making the pixel values closer to 128.
We show in Figure 5 (b) that doing exactly this improves the likelihood of both CIFAR-10 and SVHN.
Reducing the variance of the latent representations has the same effect, which is shown by Figure 12 in the Appendix.
R ELATED W ORK
This paper is inspired by and most related to recent work on evaluation of generative models.
Worthy of foremost mention is the work of Theis et al.
(2016), which showed that high likelihood is neither sufﬁcient nor necessary for the model to produce visually satisfying samples.
However, their paper does not consider out-of-distribution inputs.
In this regard, there has been much work on adversarial inputs (Szegedy et al., 2014).
While the term is used broadly, it commonly refers to inputs that have been imperceptibly modiﬁed so that the model can no longer provide an accurate output (a mis-classiﬁcation, usually).
Adversarial attacks on generative models have been studied by (at least)
Tabacof et al.
(2016) and Kos et al.
, but these methods of attack require access to the model.
We, on the other hand, are interested in model calibration for any out-of-distribution set and especially for common data sets not constructed with any nefarious intentions nor for attack on a particular model.
Various papers (Hendrycks & Gimpel, 2017; Lakshminarayanan et al., 2017; Liang et al., 2018) have reported that discriminative neural networks can produce overconﬁdent predictions on out-of-distribution inputs. In a related ﬁnding, Lee et al. (2018) reported that it was much harder to recognize an input as out-of-distribution when the classiﬁer was trained on CIFAR-10 in comparison to training on SVHN.
Testing the robustness of deep generative models to out-of-distribution inputs had not been investigated previously, to the best of our knowledge.
However, there is work concurrent with ours that has tested their ability to detect anomalous inputs.
Shafaei et al.
and Hendrycks et al.
(2019) also observe that PixelCNN++ cannot provide reliable outlier detection.
Hendrycks et al.
(2019) mitigate the CIFAR-10 vs SVHN issue by exposing the model to outliers during training.
They do not consider ﬂow-based models.
Škvára et al.
(2018) experimentally compare VAEs and GANs against k-nearest neighbors (kNNs), showing that VAEs and GANs outperform kNNs only when known outliers can be used for hyperparameter selection.
In the work most similar to ours, Choi & Jang (2018) report the same CIFAR-10 vs SVHN phenomenon for Glow—independently conﬁrming our motivating observation.
As a ﬁx, they propose training an ensemble of generative models with an adversarial objective and testing for out-of-training-distribution inputs by computing the WatanabeAkaike information criterion via the ensemble.
This work is complementary to ours since they focus on providing a detection method whereas we are interested in understanding how and when the phenomenon can arise.
The results we present in Equation 6 do not apply to Choi & Jang (2018)’s models since they use scaling operations in their afﬁne coupling layers, making them NVP.
D ISCUSSION
We have shown that comparing the likelihoods of deep generative models alone cannot identify the training set or inputs like it. Therefore we urge caution when using these models with outof-training-distribution inputs or in unprotected user-facing systems.
Moreover, our analysis in Section 5 shows that the CIFAR-10 vs SVHN phenomenon would persist for any constant-volume Glow no matter the parameter values nor the choice of latent density (as long as it is log-concave).
While we cannot conclude that this is a pathology in deep generative models, it does suggest the need for further work on generative models and their evaluation. The models we tested seem to be capturing low-level statistics rather than high-level semantics, and better inductive biases, optimization procedures, or uncertainty quantiﬁcation may be necessary.
Yet, deep generative models can detect out-of-distribution inputs when using alternative metrics (Choi & Jang, 2018) and modiﬁed training procedures (Hendrycks et al., 2019).
The problem then may be a fundamental limitation of high-dimensional likelihoods.
Until these open problems are better understood, we must temper the enthusiasm with which we preach the beneﬁts of deep generative models. A CKNOWLEDGMENTS We thank Aaron van den Oord, Danilo Rezende, Eric Jang, Florian Stimberg, Josh Dillon, Mihaela
Rosca, Rui Shu, Sander Dieleman, and the anonymous reviewers for their helpful feedback and discussions.
A A DDITIONAL I MPLEMENTATION D ETAILS A. F LOW -B ASED M ODELS
We have described the core building blocks of invertible generative models above, but there are several other architectural choices required in practice.
Due to space requirements, we only describe them brieﬂy, referring the reader to the original papers for details.
In the most recent extension of this line of work, Kingma & Dhariwal (2018) propose the Glow architecture, with its foremost contribution being the use of 1 × 1 convolutions in place of discrete permutation operations.
Convolutions of this form can be thought of as a relaxed but generalized permutation, having all the representational power of the discrete version with the added beneﬁt of parameters amenable to gradient-based training.
As the transformation function becomes deeper, it becomes prone to the same scale pathologies as deep neural networks and therefore requires a normalization step of some form.
Dinh et al.
(2017) propose incorporating batch normalization and describe how to compute its contribution to the log-determinant-Jacobian term.
Kingma & Dhariwal (2018) apply a similar normalization, which they call actnorm , but it uses trainable parameters instead of batch statistics.
Lastly, both Dinh et al.
(2017) and Kingma & Dhariwal (2018) use multi-scale architectures that factor out variables at regular intervals, copying them forward to the ﬁnal latent representation.
This gradually reduces the dimensionality of the transformations, improving upon computational costs.
For our MNIST experiments, we used a Glow architecture of 2 blocks of 16 afﬁne coupling layers, squeezing the spatial dimension in between the 2 blocks. For our CIFAR experiments, we usedblocks of 8 afﬁne coupling blocks, applying the multi-scale architecture between each block. For all coupling blocks, we used a 3-layer Highway network with 200 hidden units for MNIST andhidden units for CIFAR. The networks we trained were shallower than those in Kingma & Dhariwal (2018).
We also found that initializing the last layer of the coupling networks to 0 was sufﬁcient to prevent scale pathologies, hence we did not use any form of normalization (batchnorm nor actnorm) for ease of initialization and training in a distributed setting.
Convolution kernels were initialized with a truncated normal with variance 1 / √ D where D is fan-in size, except where zero-initialization is prescribed by Glow .
All networks were trained with the RMSProp optimizer, with a learning rate of 1 e − 5 for 100K steps, decaying by half at 80K and 90K steps.
We used a prior with zero mean and unit variance for all experiments.
We applied L2 regularization of 5 e − 2 to CIFAR experiments.
All experiments used batch size 32.
For CV-Glow, we used additive rather than afﬁne coupling blocks, which removes the inﬂuence of coupling blocks on the log-det-Jacobian term.
The volume change of the 1 × 1 convolutions depend on its weights rather than the input, so the network has a constant volume change.
A. P IXEL CNN
We trained a GatedPixelCNN with a categorical distribution.
For FashionMNIST, we used a network with 5 gated layers with 32 features, and the ﬁnal skip connection layer with size 256.
We trained for 100K steps with the Adam optimizer and an initial learning rate of 1 e − 4 , and decaying at 80K and 90K steps.
For CIFAR experiments, we used 15 dated layers with 128 features, and a skip connection size of 1024.
Convolutions were initialized with Tensorﬂow’s variance scaling initializer with a uniform distribution and a scale of 1.
We trained for 200K steps with the RMSProp optimizer with an initial learning rate of 1 e − 4 , decaying by 1 / 3 at 120K, 180K, and 195K steps.
All experiments used batch size 32.
A. V ARIATIONAL A UTO -E NCODERS
We refer to (Rosca et al., 2018, Appendix K) for additional details on the VAEs used in our CIFAR experiments.
We used the CIFAR conﬁgurations without modiﬁcation.
For FashionMNIST, we used the encoder given in Table 4 of (Rosca et al., 2018, Appendix K, Table 4) and a decoder composed of one linear layer with 7 ∗ 7 ∗ 64 hidden units, followed by a reshape and three transposed convolutions of feature sizes 32, 32, 256 and strides 2, 2, 1.
Weights were initialized with a normal distribution with variance 0.02.
We trained for 200K steps using the RMSProp optimizer, with a constant learning rate of 1 e − 4 .
B R ESULTS ILLUSTRATING ASYMMETRIC BEHAVIOR
[Figure]
Note that the model trained on SVHN (MNIST) is able to assign lower likelihood to CIFAR- (FashionMNIST), which illustrates the asymmetry compared to Figure 2.
C A NALYZING THE C HANGE - OF -V ARIABLES F ORMULA AS AN O PTIMIZATION F UNCTION
Consider the intuition underlying the volume term in the change of variables objective (Equation 3).
As we are maximizing the Jacobian’s determinant, it means that the model is being encouraged to maximize the ∂ f j /∂x j partial derivatives.
In other words, the model is rewarded for making the transformation sensitive to small changes in x .
This behavior starkly contradicts a long history of derivative-based regularization penalties.
Dating back at least to (Girosi et al., 1995), penalizing the Frobenius norm of a neural network’s Jacobian—which upper bounds the volume term 3 —has been shown to improve generalization.
This agrees with intuition since we would like the model to be insensitive to small changes in the input, which are likely noise.
Moreover, Bishop (1995) showed that training a network under additive Gaussian noise is equivalent to Jacobian regularization, and Rifai et al.
(2011) proposed contractive autoencoders , which penalize the Jacobian-norm of the encoder.
Allowing invertible generative models to maximize the Jacobian term without constraint suggests, at minimum, that these models will not learn robust representations.
Limiting Behavior.
We next attempt to quantify the limiting behavior of the log volume element.
Let us assume, for the purposes of a general treatment, that the bijection f φ is an L -Lipschitz function.
Both terms in Equation 3 can be bounded as follows:
[Equation]
where L is the Lipschitz constant, D the dimensionality, and O (max z log p z
( z )) an expression for the (log) mode of p ( z ) .
We will make this mode term for concrete for Gaussian distributions below.
The bound on the volume term follows from Hadamard’s inequality:
[Equation]
where e j is an eigenvector.
While this expression is too general to admit any strong conclusions, we can see from it that the ‘peakedness’ of the distribution represented by the mode must keep pace with the Lipschitz constant, especially as dimensionality increases, in order for both terms to contribute equally to the objective.
We can further illuminate the connection between L and the concentration of the latent distribution through the following proposition:
Proposition 1.
Assume x ∼ p ∗ is distributed with moments E [ x ] = µ x and V ar [ x ] =
σ 2 x .
Moreover, let f : X
(cid:55)→ Z be L -Lipschitz and f ( µ x ) = µ z .
We then have the following concentration inequality for some constant δ :
[Equation]
Proof : From the fact that f is L -Lipschitz, we know | f ( x ) − µ z | ≤ L (cid:12)(cid:12) x − f − 1 ( µ z ) (cid:12)(cid:12) .
Assuming µ x = f − 1 ( µ z )) , we can apply Chebyshev’s inequality to the RHS:
Pr ( L (cid:12)(cid:12) x − f − 1 ( µ z ) (cid:12)(cid:12) ≥ δ ) ≤
L 2 σ 2 x /δ 2 .
Since L (cid:12)(cid:12) x − f − 1 ( µ z ) (cid:12)(cid:12) ≥ | f ( x ) − µ z | , we can plug the RHS into the inequality and the bound will continue to hold.
From the inequality we can see that the latent distribution can be made more concentrated by decreasing L and/or the data’s variance σ 2 x .
Since the latter is ﬁxed, optimization only inﬂuences L .
Yet, recall that the volume term in the change-of-variables objective rewards increasing f ’s derivatives and thus L .
While we have given an upper bound and therefore cannot say that increasing L will necessarily decrease concentration in latent space, it is for certain that leaving L unconstrained does not directly pressure the f ( x ) evaluations to concentrate.
Previous work (Dinh et al., 2015; 2017; Kingma & Dhariwal, 2018) has almost exclusively used a factorized zero-mean Gaussian as the latent distribution, and therefore we examine this case in particular.
The log-mode can be expressed as − D/ 2 · log 2 πσ 2 z , making the likelihood bound log N ( f ( x ;
[Equation]
We see that both terms scale with D although in different directions, with the contribution of the z -distribution becoming more negative and the volume term’s becoming more positive.
We performed a simulation to demonstrate this behavior on the two moons data set, which is shown in Figure 7 (a).
We replicated the original two dimensions to create data sets of dimensionality of up to 100 .
The results are shown in Figure 7 (b).
The empirical values of the two terms are shown by the solid lines, and indeed, we see they exhibit the expected diverging behavior as dimensionality increases. [Figure]
D G LOW WITH S IGMOID P ARAMETRIZATION
Upon reading the open source implementation of Glow, 4 we found that Kingma & Dhariwal (2018) in practice parametrize the scaling factor as sigmoid ( s ( x d : ; φ s )) instead of exp { s ( x d : ; φ s ) } .
This choice allows the volume only to decrease and thus results in the volume term being bounded as (ignoring the convolutional transforms)
[Equation]
where f indexes the ﬂows and d f the dimensionality at ﬂow f .
Interestingly, this parametrization has a ﬁxed upper bound of zero, removing the dependence on D found in Equation 8.
We demonstrate the change in behavior introduced by the alternate parametrization via the same two moon simulation.
The only difference is that the RNVP transforms use a sigmoid parametrizations for the scaling operation.
See Figure 7 (c) for the results: we see that now both change-of-variable terms are oriented downward as dimensionality grows.
We conjecture this parametrization helps condition the log-likelihood, limiting the volume term’s inﬂuence, when training the large models ( ∼ 90 ﬂows) used by Kingma & Dhariwal (2018).
However, it does not ﬁx the out-of-distribution over-conﬁdence we report in Section 3.
E C ONSTANT AND R ANDOM I NPUTS
[Figure]
Random
[Figure]
[Figure]
Constant (0)
[Figure]
Random
[Figure]
[Figure]
F E NSEMBLING G LOWS
The likelihood function technically measures how likely the parameters are under the data (and not how likely the data is under the model), and perhaps a better quantity would be the posterior predictive distribution p ( x test | x train )
= 1 M (cid:80) m p ( x test | θ m ) where we draw samples from posterior distribution θ m ∼ p ( θ | x train ) .
Intutitively, it seems that such an integration would be more robust than a single maximum likelihood point estimate.
As a crude approximation to Bayesian inference, we tried averaging over ensembles of generative models since Lakshminarayanan et al.
(2017) showed that ensembles of discriminative models are robust to out-of-distribution inputs.
We compute an “ensemble predictive distribution” as p ( x ) = 1 M (cid:80) m p ( x ; θ m ) , where m indexes over models.
However, as Figure 9 shows, ensembles did not signiﬁcantly change the relative difference between in-distribution (CIFAR-10, black and blue) and out-of-distribution (SVHN, red).
G D ERIVATION OF CV-G LOW ’ S L IKELIHOOD D IFFERENCE
We start with Equation 5: 2 Tr (cid:26)(cid:20)
[Equation]
The volume element for CV-Glow does not depend on x 0 and therefore drops from the equation:
[Equation]
[Equation]
[Figure]
where U k denotes the k th 1 × 1 -convolution’s kernel.
Moving on to the ﬁrst term, the log probability under the latent distribution, we have:
[Equation]
Since f is comprised of translation operations and 1 × 1 convolutions, its partial derivatives involve just the latter (as the former are all ones)
[Equation]
and therefore we have the partial derivatives:
[Equation]
where h and w index the input spatial dimensions, c the input channel dimensions, k the series of ﬂows, and j the column dimensions of the C k × C k -sized convolutional kernel U k .
The diagonal elements of ∇ x 0 f ( x 0 )( ∇ x 0 f ( x 0 ))
T are then ( (cid:81)
Kk =1 (cid:80) C k j =1 u k,c,j ) 2 , and the diagonal element of ∇ 2 x 0 f ( x 0 ) are all zero.
Then returning to the full equation, for the constant-volume Glow model we have:
[Equation]
Lastly, we assume that both Σ q and Σ p ∗ are diagonal and thus the element-wise multiplication with ∇ 2 x 0 log p ( f ( x 0 )) collects only its diagonal elements:
[Equation]
where we arrived at the last line by rearranging the sum to collect the shared channel terms.
H H ISTOGRAM OF DATA STATISTICS
[Figure]
0-1 scale by diving by 256.
See Figure 5a for results on datasets of 32 × 32 × 3 images: SVHN, CIFAR-10, CIFAR-100, CelebA and ImageNet.
[Figure]
Figure 12 shows the effect of graying on codes.
[Figure] [Figure]