D O D EEP G ENERATIVE M ODELS K NOW
W HAT T HEY D ON ’ T K NOW ?
Anonymous authors
Paper under double-blind review
A BSTRACT
A neural network deployed in the wild may be asked to make predictions for inputs that were drawn from a different distribution than that of the training data.
A plethora of work has demonstrated that it is easy to ﬁnd or synthesize inputs for which a neural network is highly conﬁdent yet wrong.
Generative models are widely viewed to be robust to such mistaken conﬁdence as modeling the density of the input features can be used to detect novel, out-of-distribution inputs.
In this paper we challenge this assumption.
We ﬁnd that the density learned by ﬂow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e. CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former.
Moreover, we ﬁnd evidence of this phenomenon when pairing several popular image data sets: FashionMNIST vs MNIST, CelebA vs SVHN, ImageNet vs CIFAR-10 / CIFAR-100 / SVHN.
To investigate this curious behavior, we focus analysis on ﬂow-based generative models in particular since they are trained and evaluated via the exact marginal likelihood.
We ﬁnd such behavior persists even when we restrict the ﬂow models to constant-volume transformations.
These transformations admit some theoretical analysis, and we show that the difference in likelihoods can be explained by the location and variances of the data and the model curvature.
Our results caution against using the density estimates from deep generative models to identify inputs similar to the training distribution until their behavior for out-of-distribution inputs is better understood.
I NTRODUCTION
Deep learning has achieved impressive success in applications for which the goal is to model a conditional distribution p ( y | x ) , with y being a label and x the features.
While the conditional model p ( y | x ) may be highly accurate on inputs x sampled from the training distribution, there are no guarantees that the model will work well on x ’s drawn from some other distribution.
For example, Louizos & Welling (2017) show that simply rotating an MNIST digit can make a neural network predict another class with high conﬁdence (see their Figure 1a).
Ostensibly, one way to avoid such overconﬁdently wrong predictions would be to train a density model p ( x ; θ ) (with θ denoting the parameters) to approximate the true distribution of training inputs p ∗ ( x ) and refuse to make a prediction for any x that has a sufﬁciently low density under p ( x ; θ ) .
The intuition is that the discriminative model p ( y | x ) likely did not observe enough samples in that region to make a reliable decision for those inputs.
This idea has been proposed by various papers, cf.
(Bishop, 1994), and as recently as in the panel discussion at Advances in Approximate Bayesian Inference (AABI)(Blei et al., 2017).
Anomaly detection is just one motivating example for which we require accurate densities, and others include information regularization (Szummer & Jaakkola, 2003), open set recognition (Herbei & Wegkamp, 2006), uncertainty estimation, detecting covariate shift, active learning, model-based reinforcement learning, and transfer learning.
Accordingly, these applications have lead to widespread interest in deep generative models, which take many forms such as variational auto-encoders (VAEs) (Kingma & Welling, 2014; Rezende et al., 2014), generative adversarial networks (GANs) (Goodfellow et al., 2014), auto-regressive models ( van den Oord et al., 2016b;a), and invertible latent variable models (Tabak & Turner, 2013).
The last two classes—auto-regressive and invertible models—are especially attractive since they offer exact computation of the marginal likelihood, requiring no approximate inference techniques.
In this paper, we investigate if modern deep generative models can be used for anomaly detection, as suggested by Bishop (1994) and the AABI pannel (Blei et al., 2017), expecting a well-calibrated model to assign higher density to the training data than to some other data set.
However, we ﬁnd this to not be the case: when trained on CIFAR-10 (Krizhevsky & Hinton, 2009), VAEs, autoregressive models, and ﬂow-based generative models all assign a higher density to SVHN (Netzer et al., 2011) than to the training data.
We ﬁnd this observation to be quite problematic and unintuitive since
SVHN’s digit images are so visually distinct from the dogs, horses, trucks, boats, etc. found in
CIFAR-10.
Yet, this phenomenon is not restricted to CIFAR-10 vs SVHN, and we report similar ﬁndings for models trained on CelebA and ImageNet as well. We go on to study these curious observations in ﬂow-based models in particular since they allow for exact marginal density calculations.
Initial experiments suggested the log-determinant-Jacobian term may contribute to the out-of-distribution set’s high density, but we ﬁnd that the phenomenon also holds for constant-volume ﬂows.
We then describe a series of analytic analyses that show this behavior can be explained in terms of the variances of the input distributions and the model curvature.
To the best of our knowledge, we are the ﬁrst to report these unintuitive ﬁndings for a variety of deep generative models and image data sets.
Moreover, our experiments with ﬂow-based models isolate some crucial experimental variables such as the effect of constant-volume vs non-volume-preserving transformations.
Lastly, our analysis provides some simple but general expressions for quantifying the gap in the model density between two data sets.
We close the paper by urging more study of the out-of-training-distribution properties of deep generative models, as understanding their behaviour in this setting is crucial for their deployment to the real world.
B ACKGROUND
We begin by establishing notation and reviewing the necessary background material.
We denote matrices with upper-case and bold letters (e.g. X ), vectors with lower-case and bold (e.g. x ), and scalars with lower-case and no bolding (e.g. x ).
As our focus is on generative models, let the collection of all observations be denoted by X
= { x n } Nn =1 with x representing a vector containing all features and, if present, labels.
All N examples are assumed independently and identically drawn from some population x ∼ p ∗ ( x ) (which is unknown) with support denoted X .
We deﬁne the model density function to be p ( x ; θ ) where θ ∈ Θ are the model parameters, and let the model likelihood be denoted p ( X ; θ ) =
(cid:81)
Nn =1 p ( x n ; θ ) .
2. T RAINING N EURAL G ENERATIVE M ODELS
Given observed (training) data X and a model class { p ( · ; θ ) : θ ∈ Θ } , we are interested in ﬁnding the parameters θ that make the model closest to the true but unknown data distribution p ∗ ( x ) .
We can quantify this gap in terms of a Kullback–Leibler divergence (KLD):
[Equation]
where the ﬁrst term in the right-most expression is the average log-likelihood and the second is the entropy of the true distribution.
As the latter is a ﬁxed constant, minimizing the KLD amounts to ﬁnding the parameter settings that maximize the data’s log density: θ ∗ = arg max θ log p ( X ; θ ) = arg max θ (cid:80) Nn =1 log p ( x n ; θ ) .
Note that p ( x n ; θ ) alone does not have any interpretation as a probability.
To extract probabilities from the model density, we need to integrate over some region Ω : P ( Ω ) =
(cid:82) Ω p ( x ; θ )
d x .
Adding noise to the data during model optimization can mock this integration step, encouraging the density model to output something nearer to probabilities (Theis et al., 2016):
[Equation]
where ˜ δ is a sample from p ( δ ) .
The resulting objective is a lower-bound, making it a suitable optimization target.
All models in all the experiments we report are trained with input noise.
Due to this ambiguity between densities and probabilities, henceforth we call the quantity log p ( X + ˜ ∆ ; θ ) a ‘log-likelihood,’ even if X is drawn from a distribution unlike the training data.
Regarding the choice of density model, we could choose one of the standard density functions for p ( x n ; θ ) , e.g. a Gaussian, but these may not be suitable for modeling the complex, high-dimensional data sets we often observe in the real world.
Hence, we want to parametrize the model density with some high-capacity function f , which is usually chosen to be a neural network.
That way, the model has a somewhat compact representation and can be optimized via gradient ascent.
We experiment with three variants of neural generative models: autoregressive, latent variable, and invertible.
In the ﬁrst class, we study the PixelCNN (van den Oord et al., 2016b), and due to space constraints, we refer the reader to van den Oord et al.
(2016b) for its deﬁnition.
As a representative of the second class, we use a VAE (Kingma & Welling, 2014; Rezende et al., 2014).
See Rosca et al.
(2018) for descriptions of the precise versions we use.
And lastly, we have invertible ﬂow-based generative models as the third class.
We deﬁne them in detail below since we study them with the most depth.
2. G ENERATIVE M ODELS VIA C HANGE OF V ARIABLES
The VAE and many other generative models are deﬁned as a joint distribution between the observed and latent variables.
However, another path forward is to perform a change of variables .
In this case x and z are one and the same, and there is no longer any notion of a product space
X ×
Z .
Let f : X
(cid:55)→ Z be a diffeomorphism from the data space X to a latent space Z .
Using f then allows us to compute integrals over z as an integral over x and vice versa:
[Equation]
where | ∂ f /∂ x | and | ∂ f − 1 /∂
z | are known as the volume elements as they adjust for the volume change under the alternate measure.
Speciﬁcally, when the change is w.r.t.
coordinates, the volume element is the determinant of the diffeomorphism’s Jacobian matrix, which we denote as | ∂ f /∂ x | .
The change of variables formula is a powerful tool for generative modeling as it allows us to deﬁne a distribution p ( x ) entirely in terms of an auxiliary distribution p ( z ) , which we are free to choose, and f .
Denote the parameters of the change of variables model as θ = { φ , ψ } with φ being the diffeomorphism’s parameters, i.e. f ( x ; φ ) , and ψ being the auxiliary distribution’s parameters, i.e. p ( z ; ψ ) .
We can perform maximum likelihood estimation for the model as follows:
[Equation]
Yet, optimizing ψ must be done carefully so as to not result in a trivial model.
For instance, optimization could make p ( z ; ψ ) close to uniform if there are no constraints on its variance.
For this reason, most implementations leave ψ as ﬁxed (usually a standard Gaussian) in practice.
Likewise, we assume it as ﬁxed from here forward, thus omitting ψ from equations to reduce notational clutter.
After training, samples can be drawn from the model via the inverse transform: ˜ x = f − 1 (˜ z ; φ ) , ˜ z ∼ p ( z ) .
For the particular form of f , most work to date has constructed the bijection from afﬁne coupling layers (ACLs) (Dinh et al., 2017), which transform x by way of translation and scaling operations.
Speciﬁcally, ACLs take the form: f ACL ( x ; φ ) =
[exp { s ( x d : ; φ s ) } (cid:12) x : d + t ( x d : ; φ t ) , x d : ] , where (cid:12) denotes an element-wise product.
This transformation, ﬁrstly, splits the input vector in half, i.e. x =
[ x : d , x d : ] (using Python list syntax).
Then the second half of the vector is fed into two arbitrary neural networks (possibly with tied parameters) whose outputs are denoted t ( x d : ; φ t ) and s ( x d : ; φ s ) , with φ · being the collection of weights and biases.
Finally, the output is formed by (1) scaling the ﬁrst half of the input by one neural network output, i.e. exp { s ( x d : ; φ s ) } (cid:12) x : d , (2) translating the result of the scaling operation by the second neural network output, i.e. ( · ) + t ( x d : ; φ t ) , and (3) copying the second half of x forward, making it the second half of f ACL ( x ; φ ) , i.e. f d : = x d : .
ACLs are stacked to make rich hierarchical transforms, and the latent representation z is output from this composition, i.e. z n = f ( x n ; φ ) .
A permutation operation is required between ACLs to ensure the same elements are not repeatedly used in the copy operations.
We use f without subscript to denote the complete transform and overload the use of φ to denote the parameters of all constituent layers.
This class of transform is known as non-volume preserving (NVP) (Dinh et al., 2017) since the volume element does not necessarily evaluate to one and can vary with each input x .
Although non-zero, the log determinant of the Jacobian is still tractable: log | ∂ f φ /∂ x | =
(cid:80) Dj =
d s j ( x d : ; φ s ) .
A diffeomorphic transform can also be deﬁned with just translation operations, as was done in earlier work by Dinh et al.
and this transformation is volume preserving (VP) since the volume term is one and thus has no inﬂuence in the likelihood calculation.
We will examine another class of ﬂows we term constant-volume (CV) since the volume, while not preserved, is constant across all x .
Appendix A provides additional details on implementing ﬂow-based generative models.
M OTIVATING O BSERVATIONS
Given the impressive advances of deep generative models, we sought to test their ability to quantify when an input comes from a different distribution than that of the training set.
This calibration w.r.t.
out-of-distribution data is essential for applications such as safety—if we were using the generative model to ﬁlter the inputs to a discriminative model—and for active learning.
For the experiment, we trained the same Glow architecture described in Kingma & Dhariwal (2018)—except small enough that it could ﬁt on one GPU 1 —on FashionMNIST and CIFAR-10.
We then calculated the log-likelihood (higher value is better) and bits-per-dimension (BPD, lower value is better) 2 of the test split of two different data sets of the same dimensionality—MNIST ( 28 × 28) and SVHN ( 32 × 32 × 3) respectively—expecting the models to give a lower probability to this data because they were not trained on it.
Samples from the Glow models trained on each data set are shown in
Figure 13 in the Appendix.
Data Set
Avg.
Bits Per Dimension
Glow Trained on FashionMNIST
FashionMNIST-Train 2.
FashionMNIST-Test 2. MNIST-Test 1.
Glow Trained on MNIST
MNIST-Test 1.
Data Set
Avg.
Bits Per Dimension
Glow Trained on CIFAR
CIFAR10-Train 3.
CIFAR10-Test 3.
SVHN-Test 2.
Glow Trained on SVHN
SVHN-Test 2.
, 2018) on MNIST, FashionMNIST, SVHN, CIFAR-10.
Beginning with FashionMNIST vs MNIST, the left subtable of Figure 1 shows the average BPD of the training data (FashionMNIST-Train), the in-distribution test data (FashionMNIST-Test), and the out-of-distribution data (MNIST-Test).
We see a peculiar result: the MNIST split has the lowest BPD, more than one bit less than the FashionMNIST train and test sets.
To check if this is due to outliers skewing the average, we report a (normalized) histogram in Figure 2 (a) of the log-likelihoods for the three splits.
We see that MNIST (red bars) is clearly and systematically shifted to the RHS of the plot (highest likelihood).
Moving on to CIFAR-10 vs SVHN, the right subtable of Figure 1 again reports the BPD of the training data (CIFAR10-Train), the in-distribution test data (CIFAR10-Test), and the out-of-distribution data (SVHN-Test).
We again see the phenomenon: the SVHN BPD is one bit lower than that of both in-distribution data sets.
Figure 2 (b) shows a similar histogram of the log-likelihoods.
Clearly, the SVHN examples (red bars) have higher likelihood across the board, and the result is therefore not caused by any outliers.
Subﬁgures (c) and (d) of Figure 2 show additional results for CelebA and ImageNet.
When trained on CelebA, Glow assigns a higher likelihood to SVHN (red bars), a data set the model has never seen before.
Similarly, when trained on ImageNet, Glow assigns a higher likelihood to the test splits of SVHN (red), CIFAR-10 (yellow), and CIFAR-100 (green).
The difference is quite drastic in the case of SVHN (red) but modest for the two CIFAR splits. This phenomenon is not symmetric. For instance, CIFAR-10 does not have a higher likelihood for a Glow trained on SVHN; see Figure 6 in Appendix B for these results.
We report results only for Glow, but we observe the same behavior for RNVP transforms as deﬁned by Dinh et al.
[Figure] We next tested if the phenomenon occurs for the other common deep generative models: PixelCNN and VAE.
We do not include GANs in the comparison since evaluating their likelihood is an open problem.
Figure 3 reports the same histograms as above for these models, showing the distribution of log p ( x ) evaluations for CIFAR-10’s train (black) and test (blue) splits and SVHN
’s test (red) split.
Since in all plots the red bars are shifted to the right much as they were before—albeit to varying degrees [Figure] with perhaps PixelCNN having the smallest gap—we see that, indeed, this inability to detect inputs unlike the training data persists for these other model classes.
SVHN images continue to have higher likelihood than CIFAR-10 training images
[Figure]
D IGGING D EEPER INTO THE F LOW -B ASED M ODEL
While we observed the CIFAR-10 vs SVHN phenomenon for the PixelCNN, VAE, and Glow, we now narrow our investigation to just the class of invertible generative models.
The rationale is that they allow for better experimental control as, ﬁrstly, they can compute exact marginal likelihoods, unlike the VAE, and secondly, the transforms used in ﬂow-based models have Jacobian constraints that simplify the analysis we present in Section 5.
To further analyze the high likelihood of the out-of-distribution (non-training) samples, we next report the contributions to the likelihood of each term in the change-of-variables formula.
At ﬁrst, this suggested the volume element was the primary [Figure] cause of SVHN’s high likelihood, but further experiments with constant-volume ﬂows show the problem exists with them as well.
Decomposing the change-of-variables objective.
To further examine this curious phenomenon we inspect the change-of-variables objective itself, investigating if one or both terms give the outof-distribution data a higher value.
We report the constituent log p ( z ) and log | ∂ f φ /∂ x | terms for
NVP-Glow in Figure 4, showing histograms for log p ( z ) in subﬁgure (a) and for log | ∂ f φ /∂ x | in subﬁgure (b).
We see that p ( z ) behaves mostly as expected.
The cause likelihood b
Is the volume the culprit?
In addition to the empirical evidence against the volume element, we notice that the change-of-variables objective—by rewarding the maximization of the Jacobian determinant—encourages the model to increase its sensitivity to perturbations in X .
This behavior starkly contradicts a long history of derivative-based regularization penalties that reward the model for decreasing its sensitivity to input directions.
For instance, Girosi et al.
and
Rifai et al.
(2011) propose penalizing the Frobenius norm of a neural network’s Jacobian for classiﬁers and autoencoders respectively. See Appendix C for more analysis of the log volume element.
To experimentally control for the effect of the volume term, we trained Glow with constant-volume (CV) transformations.
We modify the afﬁne layers to use only translation operations (Dinh et al., 2015) but keep the 1 × 1 convolutions as is.
The log-determinant-Jacobian is then HW (cid:80) k log | U k
| , where | U k | is the determinant of the convolutional weights U k for the k th ﬂow.
This makes the volume element constant across all inputs x , allowing us to isolate its effect.
Subﬁgures (c) and (d) of
Figure 4 show the results for this model, which we term CV-Glow (constant-volume Glow).
Subﬁgure (c) shows a histogram of the log p ( x ) evaluations, just as shown before in Figure 2, and we see that
SVHN (red) still achieves a higher likelihood (lower BPD) than the CIFAR-10 training set.
Subﬁgure (d) shows the SVHN vs CIFAR-10 BPD over the course of training CV-Glow. Notice that there is no cross-over point in the curves.
Other experiments: random and constant images, ensembles.
Other work on generative models (Sønderby et al., 2017; van den Oord et al., 2018) has noted that they often assign the highest likelihood to constant inputs.
We also test this case, reporting the BPD in Appendix Figure 8 for NVP-Glow models.
We ﬁnd constant inputs have the highest likelihood for our models as well: 0.589 BPD for CIFAR-10.
We also include in the table the BPD of random inputs for comparison.
We also hypothesized that averaging over the parameters may mitigate the phenomenon.
While integration over the entire parameter space would be ideal, this is analytically and computationally difﬁcult for Glow. Lakshminarayanan et al. (2017) show that discrete ensembles can guard against over-conﬁdence for anomalous inputs while being more practical to implement. We opted for this approach, training ﬁve Glow models independently and averaging their likelihoods to evaluate test data.
Each model was given a different initialization of the parameters to help diversify the ensemble.
Appendix log p x ) evaluations when averaging over the ensemble. We see nearly identical results: SVHN is still assigned a higher likelihood than the CIFAR
S ECOND O RDER A NALYSIS
In this section, we aim to provide a more direct analysis of when another distribution might have higher likelihood than the one used for training.
We propose analyzing the phenomenon by way of linearizing the difference in expected log-likelihoods.
Consider two distributions: the training distribution x ∼ p ∗ and some dissimilar distribution x ∼ q also with support on X .
For a given generative model p ( x ; θ ) , the adversarial distribution q will have a higher likelihood than the training data’s if E q [log p ( x ; θ )] − E p ∗ [log p ( x ; θ )] > 0 .
This expression is hard to analyze directly so we perform a second-order expansion of the log-likelihood around an interior point x 0 .
Applying the expansion log p ( x ; θ )
≈ log p ( x 0 ; θ ) + ∇ x 0 log p ( x 0 ; θ ) T ( x − x 0 )
+ 12 Tr {∇ 2 x 0 log p ( x 0 ; θ )( x − x 0 )( x − x 0 ) T } to both likelihoods, taking expectations, and canceling the common terms, we have:
[Equation]
(4) where Σ = E (cid:2) ( x − x 0 )( x − x 0 ) T (cid:3) , the covariance matrix, and Tr {·} is the trace operation.
Since the expansion is accurate only locally around x 0 , we next assume that E q [ x ] = E p ∗ [ x ] = x 0 .
While this at ﬁrst glance may seem like a strong assumption, it is not too removed from practice since data is usually centered before being fed to the model.
For SVHN and CIFAR-10 in particular, we ﬁnd this assumption to hold; see Figure 5 (a) for the empirical means of each dimension of CIFAR- (green) and SVHN (orange).
All of SVHN’s means fall within the empirical range of CIFAR-10’s, and the maximum difference between any dimension is less than 38 pixel values.
Assuming equal means, we then have:
[Equation]
where the second line assumes the generative model to be ﬂow-based.
Analysis of CV-Glow.
We use the expression in Equation 5 to analyze the behavior of CV-Glow on CIFAR-10 vs SVHN, seeing if the difference in likelihoods can be explained by the model curvature and data’s second moment.
The second derivative terms simplify considerably for CV-Glow with a spherical latent density.
Given a C × C kernel U k , with k indexing the ﬂow and C the number of input channels, the derivatives are ∂f h,w,c /∂x h,w,c
= (cid:81) k
(cid:80) Cj =
1 u k,c,j , with h and w indexing the spatial height and width and j the columns of the k th ﬂow’s 1 × 1 convolutional kernel.
The second derivative is then ∂ 2 f h,w,c /∂x 2 h,w,c = 0 , which allows us to write
[Equation]
The derivation is given in Appendix G. Plugging in the second derivative of the Gaussian’s log density—a common choice for the latent distribution in ﬂow models, following (Dinh et al., 2015; 2017; Kingma & Dhariwal, 2018)—and the empirical variances, we have:
[Equation]
(cid:2)
[Equation] [Equation] [Equation]
[Equation]
α 22 (52 .
7 − 59 . 2)
+ α 23 (53 . 6 − 68 . 1) (cid:3) = 2 σ 2 ψ (cid:2)
α 21 · 12 . 3 + α 22 · 6 . 5 + α 23 · 14 . 5 (cid:3) ≥ 0 where α c = K (cid:89) k =
C (cid:88)
j = u k,c,j (6) and where σ 2 ψ is the variance of the latent distribution.
We know the ﬁnal expression is greater than or equal to zero since all α 2 c ≥ 0 . Equality is achieved only for σ 2 ψ → ∞ or in the unusual case of at least one all-zero row in any convolutional kernel for all channels.
Thus, the second-order expression we derived does indeed predict we should see a higher likelihood for SVHN than for CIFAR-10.
Moreover, we leave the CV-Glow’s parameters as constants to emphasize the expression is non-negative for any parameter setting of the CV-Glow model.
This supports our observation that an ensemble of Glows resulted an almost identical likelihood gap (Figure 9) and that the gap remained relatively constant over the course of training (Figure 4 b).
Furthermore, the ∂ 2 log p ( z ; ψ ) /∂z 2 term would be negative for any log-concave density function, meaning that changing the latent density to Laplace or logistic would not change the result.
Our conclusion is that SVHN simply "sits inside of" CIFAR-10—roughly same mean, smaller variance—resulting in its higher likelihood.
This insight also holds true for the additional results presented in subﬁgures (c) and (d) for Figure 2.
Examining Figure 5 (a) again, we see that ImageNet, the CIFARs, and SVHN all have nearly overlapping means and that ImageNet has the highest variance.
Therefore, we expect SVHN and the CIFARs to have a higher likelihood than ImageNet on an ImageNet-trained model, which is exactly what we observe in Figure 2 (d).
Moreover, the degree of the differences in likelihoods agrees with the differences in variances.
SVHN clearly has the smallest variance and the largest likelihood.
In turn, this means that we can artiﬁcially increase the likelihood of a distribution by shrinking its variance.
For RGB images, shrinking the variance is equivalent to ’graying’ them, i.e. making the pixel values closer to 128.
We show in Figure 5 (b) that doing exactly this improves the likelihood of both CIFAR-10 and SVHN.
Reducing the variance of the latent representations has the same effect, which is shown by Figure 12 in the Appendix
[Figure]
R ELATED W ORK
This paper is inspired by and most related to recent work on evaluation of generative models.
Worthy of foremost mention is the work of Theis et al.
(2016), which showed that high likelihood is neither sufﬁcient nor necessary for the model to produce visually satisfying samples.
However, their paper does not consider out-of-distribution inputs.
In this regard, there has been much work on adversarial inputs (Szegedy et al., 2014).
While the term is used broadly, it commonly refers to inputs that have been imperceptibly modiﬁed so that the model no longer can provide an accurate output (a mis-classiﬁcation, usually).
Adversarial attacks on generative models have been studied by (at least)
Tabacof et al.
(2016) and Kos et al.
, but these methods of attack require access to the model.
We, on the other hand, are interested in model calibration for any out-of-distribution set and especially for common data sets not constructed with any nefarious intentions nor for attack on a particular model.
Various papers (Hendrycks & Gimpel, 2017; Lakshminarayanan et al., 2017; Liang et al., 2018) have reported that discriminative neural networks can produce overconﬁdent predictions on out-of-distribution inputs, but out-of-distribution robustness of deep generative models has not been investigated previously, to the best of our knowledge.
There is work concurrent with ours that has tested the anomaly detection abilities of deep generative models. Shafaei et al.
(2018) observe that PixelCNN++ cannot provide reliable outlier detection. They do not consider ﬂow-based models. Škvára et al.
(2018) experimentally compare VAEs and
GANs against k-nearest neighbors (kNNs), showing that VAEs and GANs outperform kNNs only when known outliers can be used to select their hyperparameters. In the work most similar to ours, Choi & Jang (2018) report the same CIFAR-10 vs SVHN phenomenon for Glow—independently conﬁrming our motivating observation. As a ﬁx, they propose training an ensemble of generative models with an adversarial objective and testing for out-of-training-distribution inputs by computing the Watanabe-Akaike information criterion via the ensemble.
Their work is complementary to ours since they focus on providing a detection metric whereas we are interested in understanding how and when the phenomenon can arise.
The results we present in Equation 6 do not apply to Choi & Jang (2018)’s models since they use afﬁne coupling layers in their Glow, making it NVP.
D ISCUSSION
The impressively sharp samples produced by Glow (Kingma & Dhariwal, 2018) and its precursor
RNVP ﬂow (Dinh et al., 2017), in addition to their ability to compute exact marginal likelihoods, makes invertible generative models attractive to study and deploy.
However, we urge caution when using deep generative models with out-of-training-distribution inputs as we have shown that comparing likelihoods alone cannot identify the training set or inputs like it.
Moreover, our analysis in Section 5 shows that the SVHN vs CIFAR-10 problem we report would persist for any constantvolume ﬂow no matter the parameter settings nor the choice of latent density (as long as it is log-concave).
The models seem to capture low-level statistics rather than high-level semantics. While we cannot conclude that this is necessarily a pathology in deep generative models, it does suggest they need to be further improved.
It could be a problem that plagues any generative model, no matter how high its capacity.
In turn, we must then temper the enthusiasm with which we preach the beneﬁts of generative models until their sensitivity to out-of-distribution inputs is better understood.
A M ORE I MPLEMENTATION D ETAILS FOR F LOW -B ASED M ODELS
We have described the core building blocks of invertible generative models above, but there are several other architectural features required in practice.
Although, due to space requirements, we only describe them brieﬂy, referring the reader to the original papers for details.
In the most recent extension of this line of work, Kingma & Dhariwal (2018) propose the Glow architecture, with its foremost contribution being the use of 1 × 1 convolutions in place of discrete permutation operations.
Convolutions of this form can be thought of as a relaxed but generalized permutation, having all the representational power of the discrete version with the added beneﬁt of parameters amenable to gradient-based training.
As the transformation function becomes deeper, it becomes prone to the same scale pathologies as deep neural networks and therefore requires a normalization step of some form.
Dinh et al.
(2017) propose incorporating batch normalization and describe how to compute its contribution to the log-determinant-Jacobian term.
Kingma & Dhariwal (2018) apply a similar normalization, which they call actnorm , but it uses trainable parameters instead of batch statistics.
Lastly, both Dinh et al.
(2017) and Kingma & Dhariwal (2018) use multi-scale architectures that factor out variables at regular intervals, copying them forward to the ﬁnal latent representation.
This gradually reduces the dimensionality of the transformations, improving upon computational costs.
B R ESULTS ILLUSTRATING ASYMMETRIC BEHAVIOR
[Figure]
Note that the model trained on SVHN (MNIST) is able to assign lower likelihood to CIFAR (NotMNIST), which illustrates the asymmetry compared to Figure 2.
C A NALYZING THE C HANGE - OF -V ARIABLES F ORMULA AS AN O PTIMIZATION F UNCTION
Consider the intuition underlying the volume term in the change of variables objective (Equation 3).
As we are maximizing the Jacobian’s determinant, it means that the model is being encouraged to maximize the ∂ f j /∂x j partial derivatives.
In other words, the model is rewarded for making the transformation sensitive to small changes in x .
This behavior starkly contradicts a long history of derivative-based regularization penalties.
Dating back at least to (Girosi et al., 1995), penalizing the Frobenius norm of a neural network’s Jacobian—which upper bounds the volume term 3 —has been shown to improve generalization.
This agrees with intuition since we would like the model to be insensitive to small changes in the input, which are likely noise.
Moreover, Bishop (1995) showed that training a network under additive Gaussian noise is equivalent to Jacobian regularization, and Rifai et al.
(2011) proposed contractive autoencoders , which penalize the Jacobian-norm of the encoder.
Allowing invertible generative models to maximize the Jacobian term without constraint suggests, at minimum, that these models will not learn robust representations.
Limiting Behavior.
We next attempt to quantify the limiting behavior of the log volume element.
Let us assume, for the purposes of a general treatment, that the bijection f φ is an L -Lipschitz function.
Both terms in Equation 3 can be bounded as follows:
[Equation]
where L is the Lipschitz constant, D the dimensionality, and O (max z log p z
( z )) an expression for the (log) mode of p ( z ) .
We will make this mode term for concrete for Gaussian distributions below.
The bound on the volume term follows from Hadamard’s inequality:
[Equation]
where e j is an eigenvector.
While this expression is too general to admit any strong conclusions, we can see from it that the ‘peakedness’ of the distribution represented by the mode must keep pace with the Lipschitz constant, especially as dimensionality increases, in order for both terms to contribute equally to the objective.
We can further illuminate the connection between L and the concentration of the latent distribution through the following proposition:
Proposition 1.
Assume x ∼ p ∗ is distributed with moments E [ x ] = µ x and V ar [ x ] =
σ 2 x .
Moreover, let f : X
(cid:55)→ Z be L -Lipschitz and f ( µ x ) = µ z .
We then have the following concentration inequality for some constant δ :
[Equation]
Proof : From the fact that f is L -Lipschitz, we know | f ( x ) − µ z | ≤ L (cid:12)(cid:12) x − f − 1 ( µ z ) (cid:12)(cid:12) .
Assuming µ x = f − 1 ( µ z )) , we can apply Chebyshev’s inequality to the RHS:
Pr ( L (cid:12)(cid:12) x − f − 1 ( µ z ) (cid:12)(cid:12) ≥ δ ) ≤
L 2 σ 2 x /δ 2 .
Since L (cid:12)(cid:12) x − f − 1 ( µ z ) (cid:12)(cid:12) ≥ | f ( x ) − µ z | , we can plug the RHS into the inequality and the bound will continue to hold.
From the inequality we can see that the latent distribution can be made more concentrated by decreasing L and/or the data’s variance σ 2 x .
Since the latter is ﬁxed, optimization only inﬂuences L .
Yet, recall that the volume term in the change-of-variables objective rewards increasing f ’s derivatives and thus L .
While we have given an upper bound and therefore cannot say that increasing L will necessarily decrease concentration in latent space, it is for certain that leaving L unconstrained does not directly pressure the f ( x ) evaluations to concentrate.
Previous work (Dinh et al., 2015; 2017; Kingma & Dhariwal, 2018) has almost exclusively used a factorized zero-mean Gaussian as the latent distribution, and therefore we examine this case in particular.
The log-mode can be expressed as − D/ 2 · log 2 πσ 2 z , making the likelihood bound
[Equation]
We see that both terms scale with D although in different directions, with the contribution of the z -distribution becoming more negative and the volume term’s becoming more positive.
We performed a simulation to demonstrate this behavior on the two moons data set, which is shown in Figure 7 (a).
We replicated the original two dimensions to create data sets of dimensionality of up to 100 .
The results are shown in Figure 7 (b).
The empirical values of the two terms are shown by the solid lines, and indeed, we see they exhibit the expected diverging behavior as dimensionality increases.
D G LOW WITH S IGMOID P ARAMETRIZATION
Upon reading the open source implementation of Glow, 4 we found that Kingma & Dhariwal (2018) in practice parametrize the scaling factor as sigmoid ( s ( x d : ; φ s )) instead of exp { s ( x d : ; φ s ) } . This [Figure]
choice allows the volume only to decrease and thus results in the volume term being bounded as (ignoring the convolutional transforms)
[Equation]
where f indexes the ﬂows and d f the dimensionality at ﬂow f .
Interestingly, this parametrization has a ﬁxed upper bound of zero, removing the dependence on D found in Equation 8.
We demonstrate the change in behavior introduced by the alternate parametrization via the same two moon simulation. The only difference is that the RNVP transforms use a sigmoid parametrizations for the scaling operation.
See Figure 7 (c) for the results: we see that now both change-of-variable terms are oriented downward as dimensionality grows.
We conjecture this parametrization helps condition the log-likelihood, limiting the volume term’s inﬂuence, when training the large models ( ∼ 90 ﬂows) used by Kingma & Dhariwal (2018).
However, it does not ﬁx the out-of-distribution over-conﬁdence we report in Section 3.
E C ONSTANT AND R ANDOM I NPUTS
Data Set
Avg.
Bits Per Dimension
Glow Trained on NotMNIST
Random 9. Constant (0) 0. Data Set Avg.
Bits Per Dimension
Glow Trained on CIFAR
[Figure]
F E NSEMBLING G LOWS
The likelihood function technically measures how likely the parameters are under the data (and not how likely the data is under the model), and perhaps a better quantity would be the posterior predictive distribution p ( x test | x train )
= 1 M (cid:80) m p ( x test | θ m ) where we draw samples from posterior distribution θ m ∼ p ( θ | x train ) .
Intutitively, it seems that such an integration would be more robust than a single maximum likelihood point estimate.
As a crude approximation to Bayesian inference, we tried averaging over ensembles of generative models since Lakshminarayanan et al.
(2017) showed that ensembles of discriminative models are robust to out-of-distribution inputs.
We compute an “ensemble predictive distribution” as p ( x ) = 1 M (cid:80) m p ( x ; θ m ) , where m indexes over models.
However, as Figure 9 shows, ensembles did not signiﬁcantly change the relative difference between in-distribution (CIFAR-10, black and blue) and out-of-distribution (SVHN, red).
[Figure]
G D ERIVATION OF CV-G LOW ’ S L IKELIHOOD D IFFERENCE
We start with Equation 5:
[Equation]
The volume element for CV-Glow does not depend on x 0 and therefore drops from the equation:
[Equation]
where U k denotes the k th 1 × 1 -convolution’s kernel. Moving on to the ﬁrst term
[Equation]
the log probability under the latent distribution
[Equation]
we have: [Equation] Since f is comprised of translation operations and 1 × 1 convolutions, its partial derivatives involve just the latter (as the former are all ones), and therefore we have the partial derivatives:
[Equation]
where h and w index the input spatial dimensions, c the input channel dimensions, k the series of ﬂows, and j the column dimensions of the C k × C k -sized convolutional kernel U k .
The diagonal elements of ∇ x 0 f ( x 0 )( ∇ x 0 f ( x 0 ))
T are then ( (cid:81)
Kk =1 (cid:80) C k j =1 u k,c,j ) 2 , and the diagonal element of ∇ 2 x 0 f ( x 0 ) are all zero.
Then returning to the full equation, for the constant-volume Glow model we have:
[Equation]
Lastly [Equation] we assume that both Σ q and Σ p ∗ are diagonal and thus the element-wise multiplication with ∇ 2 x 0 log p ( f ( x 0 )) collects only its diagonal elements:
[Equation]
(14) where we arrived at the last line by rearranging the sum to collect the shared channel terms.
H H ISTOGRAM OF DATA STATISTICS I R ESULTS ILLUSTRATING EFFECT OF GRAYING ON CODES
Figure 12 shows the effect of graying on codes.
Figure 14 reports results using GLOW architecture on additional dataset pairs where out-ofdistribution (OOD) dataset has higher likelihood (lower BPD) than the dataset on which the model was trained. Figure 14a shows results on FashionMNIST (Xiao et al., 2017) vs MNIST; BPD on
FashionMNIST train and test are 2.90 and 2.95 respectively, whereas the BPD on MNIST is 1.83. Figure 14b shows results on CelebA (Liu et al., 2015) vs SVHN; the model assigns much higher likelihood to SVHN (BPD 2.65) even though it was trained on CelebA (BPD 3.86 on train and 3. on test). Figure 14c shows results on ImageNet vs CIFAR-10, CIFAR-100 and SVHN; the model assigns higher likelihood to CIFAR-10 (BPD though
[Figure] [Figure] [Figure]
[Figure] [Figure] [Figure]