Title: UNSUPERVISED OUT-OF-DISTRIBUTION DETECTION WITH BATCH NORMALIZATION

Abstract: Likelihood from a generative model is a natural statistic for detecting out-ofdistribution (OoD) samples. However, generative models have been shown to assign higher likelihood to OoD samples compared to ones from the training distribution, preventing simple threshold-based detection rules. We demonstrate that OoD detection fails even when using more sophisticated statistics based on the likelihoods of individual samples. To address these issues, we propose a new method that leverages batch normalization. We argue that batch normalization for generative models challenges the traditional i.i.d. data assumption and changes the corresponding maximum likelihood objective. Based on this insight, we propose to exploit in-batch dependencies for OoD detection. Empirical results suggest that this leads to more robust detection for high-dimensional images.

1. INTRODUCTION:

Modern neural network models are known to make poorly calibrated predictions (Guo et al., 2017; Kuleshov et al., 2018) , and can be highly confident even for unrecognizable or irrelevant inputs (Nguyen et al., 2015; Moosavi-Dezfooli et al., 2017) . This has serious implications for AI safety (Amodei & Clark, 2016) in real world deployments, where a model could receive inputs that are beyond its training distribution. Detecting examples that are out of the training distribution becomes a viable solution: when encountering such samples, the model could choose to provide low confidence estimates or even abstain from making predictions (Cortes et al., 2017) .

Density estimation is one approach to detecting out-of-distribution (OoD) samples. A likelihoodbased model is trained on the input samples; during evaluation, samples that have low likelihoods are assumed to be out-of-distribution. For high-dimensional inputs (such as images), deep generative models have been able to generate realistic samples as well as achieving good compression capabilities, which indicates high likelihoods on the training distribution (Ballé et al., 2016; Kingma & Welling, 2013; Kingma & Dhariwal, 2018; van den Oord et al., 2016) ; thus, recent works have considered using deep generative models to detect out-of-distribution samples (Li et al., 2018) . However, contrary to popular belief, density estimates by deep generative models are highly inaccurate (Nalisnick et al., 2018) . For example, a Glow (Kingma & Dhariwal, 2018 ) model trained on CIFAR10 gives higher likelihood estimates to SVHN samples than CIFAR10 ones, which makes accurate OoD detection impossible. While alternative statistics based on likelihood estimates have been proposed to alleviate this issue (Choi et al., 2018; Song et al., 2017) , they are not able to detect OoD samples consistently.

In this paper, we propose a new statistic to detect OoD samples using deep generative models trained with batch normalization (Ioffe & Szegedy, 2015) . We argue that using batch normalization not only improves optimization (as argued in (Kohler et al., 2018; Santurkar et al., 2018) ), but also challenges the i.i.d. assumption underlying typical likelihood-based objectives. We show in that the training objective of generative models with batch normalization can be interpreted as maximum pseudo-likelihood over a different joint distribution that does not assume data in the same batch are i.i.d. samples. Empirically, we demonstrate that over this joint distribution, the estimated likelihood of a batch of OoD samples is much lower than that of in-distribution samples. This allows us to propose a permutation test which outperforms existing methods by a significant margin without modifying how the underlying generative model is trained. In particular, we achieve near-perfect performance even on cases such as Fashion MNIST vs. KMNIST, where the likelihood distributions for single samples are nearly identical (see Figure 1 , left). While generative models trained with BatchNorm might not provide state-of-the-art likelihood numbers on the test set, it remains competitive and could be more suited for the OoD detection problem (Kingma & Dhariwal, 2018; Nalisnick et al., 2018) .

2. BACKGROUND:

Setup Let X = {x i } N i=1 be a set of N observations; we assume that "in-distribution" samples are drawn i.i.d. from some underlying distribution x ∼ p(x). Our goal is to detect whether a test sample x is "out-of-distribution"; we consider drawing the test samples from an unknown distribution q(x), but we do not have access to samples from q(x) during training. Following definitions in (Hendrycks & Gimpel, 2016; Liang et al., 2017) , we define OoD examples as the test samples that have low densities under p(x). We note that samples from q(x) are not necessarily all OoD; for example, if q = p then most samples should not be classified as OoD. Detection of OoD samples is useful for applications such as anomaly detection (Li et al., 2018) and exploration in reinforcement learning (Ostrovski et al., 2017) .

Likelihood-based generative models for out-of-distribution detection Likelihood-based models are trained by finding parameters θ such that the corresponding p θ (x) is as close as possible to the training distribution p(x), in terms of Kullback-Leibler divergence (KL):

arg min θ D KL (p(x) p θ (x)) = arg min θ E p(x) [log p(x) -log p θ (x)]

Finding appropriate solutions depends heavily on choosing the right family of (parametrized) probability distributions. In the context of images, deep neural networks have demonstrated good performance. Likelihood-based deep generative models include variational autoencoders (Kingma & Welling, 2013) (where variational lower bounds are optimized), autoregressive models (van den Oord et al., 2016; Salimans et al., 2017) , and invertible flow models (Dinh et al., 2016; Kingma & Dhariwal, 2018) (where exact likelihood is optimized).

Given a sample x, we could attempt to test whether it is OoD or not by evaluating its likelihood -log p θ (x). Intuitively, if p θ (x) is sufficiently similar to p(x), OoD samples should have low likelihood under p θ (x). However, this is not the case for some deep generative models (Nalisnick et al., 2018; Choi et al., 2018) , where OoD samples are observed to have higher likelihoods than samples from the training distribution p(x) across several datasets and models. For example, stateof-the-art models trained on CIFAR10 (Krizhevsky & Hinton, 2009) assign higher likelihood to SVHN (Netzer et al., 2011) samples than CIFAR10 samples. A similar phenomenon has also been observed by Nalisnick et al. (2018) for models trained on Fashion MNIST (Xiao et al., 2017) (Figure 1 (left)), where images with lower variance across pixels (such as MNIST) are assigned higher likelihoods. This invalidates the assumption that OoD samples are assigned lower likelihoods by existing deep generative models.

To alleviate this issue, Choi et al. (2018) use Watanabe Akaike Information Criterion (WAIC):

EQUATION

where the expectations are computed using independently trained model ensembles. This method does not assign higher negative WAIC values for OoD samples in some simple cases, but is empirically observed it to be effective for certain datasets and generative models.

Likelihood-based permutation tests:

In the context of detecting adversarial examples, Song et al. (2017) considered using permutation tests statistics to determine whether an input x comes from p(x) or not. One such test statistic uses the rank of

EQUATION

)

where I is the indicator function. When T perm (x) is large, x is assumed to be OoD. This addresses the case where higher likelihood is observed for certain out-of-distribution samples (such as the SVHN vs. CIFAR10 example in Nalisnick et al. (2018) ), as both "high-likelihood" and "low-likelihood" samples will have T perm close to N/2 (because of the absolute value) and be identified as OoD.

3 OUT-OF-DISTRIBUTION SAMPLES AND MODEL MIS-SPECIFICATION Nalisnick et al. (2018) find it surprising that deep generative models assign higher likelihood to out-of-distribution samples, given their successful generalizing on a test dataset. However, we argue that this is to be expected when the unerlying generative model p θ (x) is mis-specified, and perhaps this is more likely than previously anticipated in the literature. We consider a simple example where a mis-specified model would overestimate the likelihood of out-of-distribution samples (Figure 1 , right). Let p(x) be a uniform mixture of two Gaussians on R1 , N (-1.0, 0.01) and N (1.0, 0.01), and p θ (x) = N (µ, σ 2 ) be our model with parameters (µ, σ). The maximum likelihood solution is (µ , σ ) ≈ (0.0, 1.0), where mode-covering behavior occurs. x = 0 has the largest likelihood under p θ (x), yet it is highly atypical in the original distribution. This is also a failure mode for WAIC as discussed in (Choi et al., 2018) .

Given the complexity of high-dimensional image distributions, existing likelihood-based generative models are likely to be mis-specified, which invalidates the use of likelihood estimates to perform OoD detection (White, 1982) . Moreover, model selection is often based on the holdout method (Arlot et al., 2010) , in which we evaluate log-likelihood over a validation set sampled from p(x), but not over out-of-distribution samples. As we do not know the entropy of p(x), we can never check whether D KL (p(x) p θ (x)) ≈ 0 after training. Because of the mode-seeking nature of D KL , an alternative distribution q(x) = p(x) can have even lower KL-divergence with p θ (x), i.e., D KL (q(x) p θ (x)) D KL (p(x) p θ (x)) even if p θ (x) is trained on samples from p(x). In Appendix B, we demonstrate that even more sophisticated tests over log-likelihood, such as likelihood-based permutation tests, cannot detect out-of-distribution samples effectively.

4. DETECTING OUT-OF-DISTRIBUTION SAMPLES WITH BATCH:



NORMALIZATION:

In the following sections, we argue that by taking advantage of batch normalization (BatchNorm), outof-distribution samples can be detected with existing deep generative models. For a batch of inputs z = {z i } b i=1 of batch size b, batch normalization (Ioffe & Szegedy, 2015) performs normalization over the inputs followed by a parametrized affine transformation:

BatchNorm(z; γ, β, ) = z -E[z] Var[z] + • γ + β

where γ, β are trainable parameters, and is a hyperparameter for numerical stability. The mean E[z] and variance Var[z] are computed over a single batch in training mode, and over the entire training set in evaluation mode 1 . For deep generative models using batch normalization, existing literature evaluate the log-likelihood on some validation set in evaluation mode. The role of training mode is generally perceived to accelerate optimization (Kohler et al., 2018; Santurkar et al., 2018) , similar to supervised learning. In Table 1 we found that in evaluation mode MNIST samples have smaller BPD than Fashion MNIST for models trained on Fashion MNIST; SVHN samples have smaller BPD than CIFAR10 for models trained on CIFAR10, which agrees with the observations in (Nalisnick et al., 2018) . However, as we change from evaluation mode to training mode, the log-likelihood of the in-distribution samples does not change significantly, while that of OoD samples plummets (see ∆ in Table 1 ). In Appendix C.3, we demonstrate that this sudden decrease in log-likelihood not only happens for samples from other datasets, but also adversarial samples that are able to fool likelihood models and likelihood-based permutation tests. This empirically suggests that simply by using training mode during evaluation, we are able to consistently detect a batch of out-of-distribution samples. We further note that no special modifications is made over the training procedure, and no additional OoD datasets are required.

-

4 -2 0 2 4 z2 ≈ (x1 -E(x1))/√ V(x1) 0.0 0.1 0.2 0.3 0.4 0.5 Density (unormalized) pf(x) p(x) q(x) (eval) q(x) (train)

Figure 2 : Example demonstrating how BatchNorm mitigates high-likelihood in OoD distributions. p(x), p f (x) and q(x) denote the original, model and OoD distributions. Distribution of z 2 are different for q(x) under the two modes of BatchNorm, leading to different likelihood results.

4.2. EXPLAINING THE EFFECTIVENESS OF BATCH NORMALIZATION:

Why is batch normalization effective at detecting OoD batches while other likelihood-based generative models (e.g., Glow trained with ActNorm) fail? First we provide an example to illustrate why training mode and evaluation mode provide different likelihood estimates for OoD distributions. Then we argue that with training mode, the learned generative model no longer assumes samples are i.i.d., which differs from models without BatchNorm.

4.2.1. EXAMPLE OF BATCHNORM MITIGATING MODEL MIS-SPECIFICATION:

In Figure 2 , we illustrate a case where BatchNorm could be useful for OoD detection. Suppose we try to learn some distribution p(x) with a 2-d flow model with one coupling layer (Dinh et al., 2016) :

f : (x 1 , x 2 ) → (z 1 , z 2 ) where z 1 = x 1 , z 2 = x 2 + (x 1 -E(x 1 ))/ V (x 1 ) • γ + β with learnable parameters γ, β; E(x 1 ), V (x 1

) are the empirical mean and variance of a batch of x 1 in training mode, and the mean and variance of p(x 1 ) in evaluation mode. To simplify our analysis, We assume that the training distribution p(x 1 , x 2 ) = p(x 1 )p(x 2 ) where p(x 1 ) is a mixture of two Gaussians (N (-1, σ 2 ) and N (1, σ 2 ) with σ → 0) and p(x 2 ) = δ 0 (x 2 ) (Dirac delta) and batch sizes are infinite; hence

z 2 ≈ (x 1 -E(x 1 ))/ V (x 1 ) • γ + β.

Since the Jacobian of f is triangular and has a determinant of 1, the likelihood of the flow model is simply

EQUATION

)); we assume the prior p(z 1 , z 2 ) is standard Gaussian N (0, I). The optimal parameters are then γ ≈ 14 and β = 0. Now let us consider q(x 1 , x 2 ) = q(x 1 )p(x 2 ) where q(x 1 ) = Uniform(-0.5, 0.5). In evaluation mode,

z 2 ≈ x 1 and E q [log p f (x)] ≈ -1.92; in training mode, z 2 ≈ √ 12x 1 , which decreases the log-likelihood to E q [log p f (x)] ≈ -2.38.

We include a more detailed analysis in Appendix D. In this case, BatchNorm manages to learn a global property of the input distribution (variance), and use this to counteract major distributional shifts (e.g. the proposed uniform distribution q(x)). The effect leads to lower log-likelihoods in training mode compared to that in evaluation mode.

4.2.2. REVISITING THE I.I.D. ASSUMPTIONS FOR BATCH-NORMALIZED GENERATIVE MODELS:

While it is difficult to analytically characterize the change of likelihood estimates in training mode for high-dimensional images and deep neural networks, we provide a probabilistic interpretation as to why training mode exhibits different behaviors than evaluation mode.

Training mode breaks the i.i.d. data assumption In training mode, the output for a particular sample x j is affected by other samples in the same batch (denoted as x -j ), so they are not treated as i.i.d for the generative models in training mode. This suggest that using BatchNorm when training generative models not only changes the optimization landscape but also modifies the objective: it is no longer 1 N N i=1 log p θ (x i ) where each sample is assumed to be i.i.d.

Pseudo-likelihood perspective of training mode How should we interpret the training objective with batch normalization? For each sample x j , its corresponding "likelihood" objective depends on the other samples from the same batch; we denote this as θ (x j ; x -j ). First, we show that the integral of exp( θ (x j ; x -j )) over any particular sample (x j ) when we fix the other samples in the batch (x -j ) is one; this allows us to treat exp( ) as a probability density function5 . Proposition 1. Let b ∈ N, b > 1 be the batch size, and for all possible batch of samples of size (b -1), denoted as x -j , if θ (x j ; x -j ) is the training objective over x j for a likelihood-based generative model with batch normalization, then:

xj exp( θ (x j ; x -j ))dx j = 1.

where we consider likelihood-based generative models that sample either via some parametrized distribution (such as VAE and PixelCNN) or via parametrized invertible transformations constructed via affine coupling layers (such as RealNVP).

We defer the proofs in Appendix A.1. Moreover, if we interpret θ (x j ; x -j ) as the conditional loglikelihood under a certain joint generative model pθ over an entire batch, our existing "log-likelihood" objective is a surrogate to the log-likelihood objective over the joint distribution pθ (x 1 , x 2 , . . . , x b ). Proposition 2. There exists a joint distribution pθ (x 1 , x 2 , . . . , x b ) such that for all j, pθ (x j |x -j ) → θ (x j ; x -j ) as b → ∞. Then, the objective for one batch {x j } b j=1 in training mode

EQUATION

is the pseudo-log-likelihood for the joint distribution pθ (x 1 , x 2 , . . . , x b ) as b → ∞.

Training mode versus evaluation mode In evaluation mode, the samples are treated as i.i.d., since the batch statistics is fixed and samples within the same batch do not affect each other. The objective for one batch {x j } b j=1 evaluated in evaluation mode is simply:

EQUATION

It is evident that L train and L eval are not the same objective due to the differences in i.i.d. assumptions for samples within the same batch. L train depends on the input batch statistics, whereas L eval does not.

If we consider OoD batches that exhibit major distributional shifts, then it is likely that L train differs from L eval . From the empirical evidence in Table 1 , OoD samples have much lower BPD on training mode than on evaluation mode, whereas the same does not happen for in-distribution samples.

5. A PERMUTATION TEST BASED ON BATCH NORMALIZATION:

Assume that during testing we have access to some "test dataset" X, and our goal is to identify for each test sample x ∈ X whether it is OoD or not. Based on the observations over OoD batches, we propose a permutation test for each test sample x.

Interpolation between training and evaluation First, we propose a procedure that interpolates between training mode and evaluation mode, generalizing the settings in Section 4. We randomly obtain a batch of samples Xb1 from X with size b 1 and a batch of samples X b0 from p(x) with size b 0 (e.g., from the training dataset X used to fit the generative model). For each xj ∈ Xb1 indexed by j, we compute its log-likelihood under training mode by mixing Xb1 and X b0 in the same batch:

EQUATION

Let b = b 1 + b 0 be the batch size and let r = b 1 /b be the ratio of the test samples in the entire batch. We note that the above procedure interpolates evaluation with training mode and evaluation with evaluation mode. As r → 0, the effect on the batch statistics is dominated by the training samples X b0 , so Equation 5 converges to evaluation mode; as r → 1, the effect on the batch statistics is dominated by the test samples Xb1 , so Equation 5 converges to training mode. As we increase r from 0 to 1, we are moving from evaluation mode to training mode.

The observations in Section 4 suggest that log pθ ( xj |x -j ) should be relatively stable if q(x) is close to p(x) and increase significantly if q(x) is far from p(x), which is indeed the case empirically. We consider evaluating the average log-likelihood on the test samples when the ratio r is varied, and show the results on several datasets and models in Figure 3 . As we increase the ratio of test samples, the corresponding log-likelihood decreases dramatically for OoD samples, but is relatively stable for in-distribution samples.

Permutation test with BatchNorm:

We proceed to propose a permutation test statistics based on training mode evaluation. For a test sample x, a fixed batch size b and fixed ratio r, we select (rb -1) samples from q(x) and (1 -r)b samples from p(x). The expected conditional likelihood of x is used as the test statistic:

S b,r (x ) = E p,q log pθ ( x|X (1-r)b , Xrb-1 )

where X (1-r)b ∼ p(x), Xrb-1 ∼ q(x) (6) Figure 3 : Average BPD (-log pθ (x j |x -j )) for test samples with varying ratios of test samples in the batch. The BPD of in-distribution samples do not increase as the ratio increase, yet that of OoD samples increase significantly. This justifies the use of ∆ b,r1,r2 (x ) for OoD detection.

In practice, we use monte carlo estimates for the expectation in S b,r (x ). We select two different ratios r 1 , r 2 ∈ (0, 1) where r 1 < r 2 , and compute the differences in S b,r (x ):

EQUATION

We proceed to use the rank of ∆ b,r1,r2 ( x) in the training set as our test statistic:

EQUATION

For in-distribution samples, we expect ∆ b,r1,r2 to be small across all choices of r 1 and r 2 ; for out-of-distribution samples, ∆ b,r1,r2 should be large if (r 2 -r 1 ) is large. Therefore, if q(x) = p(x), then the statistics computed over the x ∼ q(x) should be approximately uniformly distributed over [0, N ], whereas if q(x) = p(x), then the statistics computed over out-of-distribution samples would be concentrated around N .

6. EXPERIMENTS:

We verify the effectiveness of our proposed test statistic on several datasets and models, including RealNVP, VAE and PixelCNN++, and compare against several baselines including log-likelihood, permutation test and WAIC. First, we train a model pθ (x) on a dataset p(x); then we proceed to obtain the test statistic for each sample in q(x), where q(x) are several different datasets. For each q(x) = p(x), we consider a binary classification problem, where the prediction is the test statistic for each sample x, and the label is whether x is out-of-distribution (label 1) or not (label 0); larger prediction values indicate x is more likely to be out-of-distribution. We evaluate the area under the ROC curve (AUC) and average precision (AP) for each binary classification task, following the procedure in (Hendrycks & Gimpel, 2016) . We select r 1 = 0.1 and r 2 = 0.9 for our proposed test statistic, and the batch size B = 64.

The OoD detection results are shown in Table 2 . As expected, using log p θ (x) results in poor AUC and AP in cases where the OoD samples have higher log-likelihoods; T perm outperforms log p θ (x) in cases where we also consider high log-likelihood samples as OoD; WAIC achieves higher AUC / AP than log p θ (x), yet its improvement is inconsistent across different models (RealNVP improvements are lower than VAE on Fashion MNIST). Our proposed statistic is able to detect all the out-ofdistribution samples by achieving near-optimal AUC / AP in most cases; this is most notable on CIFAR vs. ImageNet and Fashion MNIST vs. KMNIST as the sample have very similar likelihood distributions in evaluation mode. In Appendix C.4, we show that our method works well even with r 1 → 0 and r 2 = 0.15; therefore, the method is not very sensitive to the r 1 , r 2 hyperparameters. This 

7. RELATED WORK:

Task-dependent OoD detection Out-of-distribution detection is crucial to applications such as anomaly detection (Pidhorskyi et al., 2018; Hendrycks & Gimpel, 2016; Vyas et al., 2018) , adversarial defense (Song et al., 2017; 2018) , and novelty detection for exploration (Marsland, 2003; Bellemare et al., 2016; Fu et al., 2017) . In the context of supervised learning, OoD detection methods (Hendrycks & Gimpel, 2016; Liang et al., 2017; DeVries & Taylor, 2018; Gal & Ghahramani, 2016; Lakshminarayanan et al., 2017) are applied to prevent poorly-calibrated neural networks (Guo et al., 2017) from making high-confidence predictions on nonsensical inputs (Szegedy et al., 2013; Goodfellow et al., 2014) ; these methods are task dependent and are not suitable for task-independent cases, such as exploration over novel states (Fu et al., 2017; Ostrovski et al., 2017) .

Generative models for task-independent OoD detection For high-dimensional inputs such as images, deep generative models are widely applied for density estimation (van den Oord et al., 2016; Dinh et al., 2016; Kingma & Dhariwal, 2018) , and thus are naturally considered for task-independent OoD detection (Chalapathy et al., 2018; Xu et al., 2018; Kliger & Fleishman, 2018; Ostrovski et al., 2017; Li et al., 2018) . However, recent empirical evidence suggest that likelihood estimates by popular deep generative models are not reliable enough for OoD detection, even against samples that are not adversarial by construction (Nalisnick et al., 2018; Hendrycks et al., 2018) . To address these issues, Škvára et al. propose tuning the hyperparameters of VAEs with additional OoD data. Choi et al. (Choi et al., 2018) address this empirically by assuming that OoD samples have higher variance likelihood estimates under different independently trained models. Ren et al. consider learning a separate density model for potentially confounding background statistics.

Statistical tests for OoD detection OoD detection can be posed as a hypothesis test where the null hypothesis assumes the data is OoD. One could also consider a more specific goodness-of-fit (GoF) test which determines whether a dataset is drawn from a specific distribution. Nalisnick et al. (2019) considers Kernelized Stein discrepancy (Chwialkowski et al., 2016; Liu et al., 2016) and Maximum-mean discrepancy (Gretton et al., 2012) as two GoF tests that scales to generative models. They propose to compare the empirical cross-entropy of the samples with entropy of the dataset, which is related to the permutation test proposed in Song et al. (2017) . Our permutation test can also be cast as a GoF test with r → 1, where all test samples in the same batch with training mode, but we can also provide individual OoD predictions via different r values.

8. CONCLUSION:

In this work, we revisit the i.i.d. assumptions in generative models trained with batch normalization, which results in an alternative interpretation to the training objective. The difference between the training objective and the evaluation objective explains the intriguing observation that likelihood estimates over OoD examples in training mode are much lower than that in evaluation mode. This allows us to develop a permutation test based on batch normalization, with which we can reliably detect OoD examples even for difficult cases such as Fashion MNIST vs. KMNIST.

We argue that batch normalization is merely one approach to introducing non-independence between observation variables in deep generative models. Neural processes (Garnelo et al., 2018) , for example, considers generative models conditioned on certain observation contexts, which may differ significantly between regular and OoD batches. It would be interesting to investigate if such models could be utilized for more reliable out-of-distribution detection methods for high-dimensional data.

A PROOFS:

A.1 PROOF OF PROPOSITION 1

Proposition 1. Let b ∈ N, b > 1 be the batch size, and for all possible batch of samples of size (b -1), denoted as x -j , if θ (x j ; x -j ) is the training objective over x j for a likelihood-based generative model with batch normalization, then:

xj exp( θ (x j ; x -j ))dx j = 1.

where we consider likelihood-based generative models that sample either via some parametrized distribution (such as VAE and PixelCNN) or via parametrized invertible transformations constructed via affine coupling layers (such as RealNVP).

Proof. Note that in training mode, we can write down the batch norm function explicitly:

BatchNorm(z; γ, β, ) = z -1 b i=1 z i 1 b-1 i=1 (z i -1 b i=1 z) 2 + • γ + β if we fix z -i , γ, β and , then BatchNorm(z; γ, β, ) is a deterministic function over z i . The corresponding neural network g θ (x i ; x -i ) is also a deterministic function over x i (given x -i ).

We summarize likelihood-based generative models into two types:

• In the first type, the neural network produces the parameters for a certain tractable distribution, which is used to compute the likelihood (or its lower bound). This includes VAE and autoregressive models. Since the distribution over x i is parametrized by the deterministic function g θ , so the likelihood is naturally normalized.

• In the second type, RealNVP models are composed by affine coupling layers (Dinh et al., 2014) , which contains transformations (x, y) → (x , y ) where

EQUATION

where s(•) and t(•) are neural networks (potentially with batch normalization layers). This has the reverse

EQUATION

Since BatchNorm is a deterministic function over z i when z -i is fixed, the corresponding s(•) and t(•) functions also have the same property. Therefore, the corresponding RealNVP model over x i with fixed x -i is also invertible.

As g θ is a deterministic function over x i , the corresponding conditional likelihood is also normalized.

This completes the argument for likelihood-based generative models.

A.2 PROOF OF PROPOSITION 2:

Proposition 2. There exists a joint distribution pθ (x 1 , x 2 , . . . , x b ) such that for all j, pθ (x j |x -j ) → θ (x j ; x -j ) as b → ∞. Then, the objective for one batch {x j } b j=1 in training mode

EQUATION

is the pseudo-log-likelihood for the joint distribution pθ (x 1 , x 2 , . . . , x b ) as b → ∞.

Proof. We treat each sample in the batch as a separate random variable, and consider the corresponding graphical model where batch normalization is involved. Since training mode aggregates the statistics of all the samples, all the random variables are connected to each other.

First, we need to show that for large enough b, there exists a joint distribution pθ (x 1 , x 2 , . . . , x b ) that is compatible with all the conditional distributions we defined (Arnold & Press, S. James, 1989) . From Theorem 4.1 in (Arnold & Press, S. James, 1989), we need to show that there exists a function

f such that pθ (x i |x -i ) pθ (x j |x -j ) = f (x -j ) f (x -i ) (11) Let f (x -j ) = E xj [ k =j log pθ (x k |x -k )]

be the sum of conditional log-likelihood of x k under the expectation of some distribution over x j . For large enough b, the value of x j will not affect the batch statistics, so f (x -j ) = k =j log pθ (x k |x -k ), so the condition for compatibility in Equation 11holds, and there exists a compatible joint distribution, which we denote as pθ (x 1 , x 2 , . . . , x b ).

Next, we verify the conditions under which the product of the conditional likelihoods is a pseudolikelihood of pθ (x 1 , x 2 , . . . , x b ). The conditional likelihood for any sample x j is independent of x -j conditioned on its neighbors (which are x -j ), so

L train ({x j } b j=1 ; θ) = b j=1 log pθ (x j |x -j )

is the pseudo-log-likelihood objective (Besag, 1975) for {x j } b j=1 , which approximates the loglikelihood of the joint distribution.

B FOOLING LIKELIHOOD-BASED PERMUTATION TESTS:

As suggested by empirical evidence, we assume that model mis-specification always occurs and that our likelihood estimates cannot be used naively for detecting out-of-distribution samples due to the mode seeking nature of D KL . Under this assumption, likelihood-based permutation tests may seems more effective than simple thresholding rules for detecting out-of-distribution samples, as it treats both high-likelihood and low-likelihood samples as OoD. In this section, however, we show that likelihood-based permutation test can also be easily fooled, making them infeasible for OoD detection in a straightforward manner.

We consider fooling a model p θ (x) using samples generated from another model q φ (x) (trained on another dataset), where we modify the "temperature" T of the generative model (Kingma & Dhariwal, 2018) (see Appendix C.2 for details). Then we gather the likelihood estimates for samples in p(x) and q(x) and obtain the area under the ROC curve (AUC) using T perm as prediction values to classify if x is OoD. We consider a RealNVP (Dinh et al., 2016) model trained on CIFAR and a PixelCNN++ (Salimans et al., 2017) model trained on Fashion MNIST. We use a RealNVP trained on SVHN, and a PixelCNN++ trained on MNIST to fool the two p θ (x) models respectively. For RealNVP we set T = 1.10 and for PixelCNN++ we set T = 1.17.

In Figure 4 , we show that it is possible to produce a distribution q φ (x) with generative models such that its likelihood distribution evaluated by p θ (x) lies around the median likelihood in p(x). This results in low AUC and thereby confusing the test based on T perm . We show some out-of-distribution sample with similar bits-per-dimension (BPD) to training samples in Figure 5 , where the generated samples look drastically different from training samples visually, even though they have similar log-likelihoods. We note that flipping the classifier predictions (i.e. using -T perm ) will not address this issue, since we would then treat low likelihood samples as more likely to be in-distribution.

C ADDITIONAL EXPERIMENTAL DETAILS:

C.1 MODEL ARCHITECTURES For RealNVP (Dinh et al., 2016) , we consider downscale once, with 4 residual blocks for each affine coupling layer and 32 channels for each convolutional layer, except for CIFAR where we have 8 residual blocks with 64 channels for each convolutional layer. PixelCNN++ samples (MNIST, T = 1.17) For VAE (Kingma & Welling, 2013; van den Berg et al., 2018) , our inference network has 6 convolutional layers followed by a fully connected layer, and the generator network has 8 deconvolutional layers. We add batch normalization to all the convolutional layers and deconvolutional layers, except for the last two deconvolutional layers. We detail the convolutional layer hyperparameters in Table 3 . For PixelCNN++, we made two simple modifications to default hyperparameters (Salimans et al., 2017) . We reduce the number of filters in each ResNet block from 160 to 80, and we use batch normalization after convolution as opposed to adding weight normalization over convolution.

All the models are trained with default optimizer hyperparameters with a batch size of 64. We use a batch size of 64 also in evaluation mode.

C.2 EXPERIMENTAL PROCEDURES FOR SECTION B:

We consider controlling the entropy of the generated samples by controlling a temperature hyperparameter T . In RealNVP, this is realized by multiplying the latent variable by a factor of T , which is equivalent to sampling from N (0, T 2 I). In PixelCNN++, this is realized by dividing each pre-softmax scalar output by T ; larger T values would lead to higher entropy samples.

C.3 LIKELIHOOD DIFFERENCES BETWEEN TRAINING MODE AND TESTING MODE ON GENERATED DATA:

We consider measuring the likelihood differences between training mode and testing mode using a RealNVP trained on CIFAR. The samples are generated from a RealNVP trained on SVHN, with several temperatures T ∈ {0.7, 1.0, 1.3}. Similar to the observation with SVHN samples, there is a large gap between training mode and evaluation mode evaluations. We include additional results with alternative values of r 1 and r 2 . If our method is able to achieve high performance with small r 2 , this suggests that we can detect the OoD examples realiably even as they occupy a small portion within the batch. We consider r 1 → 0 (which is evaluation mode) and r 2 ∈ {0.15, 0.3, 0.5, 0.9}; we show our results in Table 5 .

C.5 HISTOGRAMS OF LOG-LIKELIHOODS:

We further show the histograms of log-likelihoods of several test datasets when the model is a RealNVP trained on CIFAR-10 datasets in Figure 6 ; the mean BPD is computed with evaluation mode.

D DETAILS FOR THE DERIVATION IN SECTION 4.2.1:

Suppose we try to learn some distribution p(x) with a 2-d flow model with one coupling layer (Dinh et al., 2016) : f : (x 1 , x 2 ) → (z 1 , z 2 ) where

z 1 = x 1 (12) z 2 = x 2 + (x 1 -E(x 1 ))/ V (x 1 ) • γ + β (13)

Under review as a conference paper at ICLR 2020 For p(x 1 , x 2 ) in evaluation mode, we have E p [log p f (x 1 , x 2 )] ≈ E p [log ϕ(x 1 ) + log ϕ((x 1 -E(x 1 ))/ V (x 1 )] (16) = E p [log ϕ(x 1 ) + log ϕ(x 1 )] ≈ -2.84.

(17)

For q(x 1 , x 2 ) in evaluation mode, we have E q [log p f (x 1 , x 2 )] ≈ E q [log ϕ(x 1 ) + log ϕ(x 1 )] ≈ -1.92.

(18)

For q(x 1 , x 2 ) in training mode, E(x 1 ) = 0, V (x 1 ) = 1/ √ 12, so we have:

E q [log p f (x 1 , x 2 )] = E q [log ϕ(x 1 ) + log ϕ(

EQUATION

:

This is typically done by keeping an exponential moving average over the batch statistics during training.

BPD is defined by the negative log-likelihood divided by the number of dimensions(Theis et al., 2015); if the likelihood is measured in nats, then an additional division by ln 2 is needed.

We use an implementation where we replace weight normalization with batch normalization for PixelCNN++, see https://github.com/pclucas14/pixel-cnn-pp.

By maximum likelihood, γ ≈ arg mins -s 2 /2 + log s = 1.

For VAE, we assume that the inference distribution is always optimal, i.e., there is no gap between the evidence lower bound and the log-likelihood, so we can treat the objective as exact maximum likelihood.

Figures and Tables:

Figure 1: (Left) Density estimation with a RealNVP model trained on Fashion MNIST. The model assigns similar / higher likelihoods to several OoD datasets. (Right) Model mis-specification can result in OoD samples having higher log-likelihoods.

Figure4: ROC curves for using the p-values for likelihood-based permutation tests. We assign positive labels to samples in q(x) and negative labels to samples in p(x). While such tests can detect OoD samples from other datasets, they could be confused by samples from another generative model.

3.932 3.962 3.028 3.245 3.500 3.969 3.968 3.274 CIFAR test samples 3.525 3.712 3.464 3.293 3.253 3.159 3.632 4.063 RealNVP samples (SVHN, T = 1.10)

Figure 5: Samples and their BPD evaluated under p θ (x). (Top) p θ (x) is a RealNVP trained on CIFAR. (Bottom) p θ (x) is a PixelCNN++ trained on Fashion MNIST.

Log-likelihood (measured in bits per dimension) calculated with RealNVP, VAE, Pixel-CNN++ models on MNIST, Fashion MNIST, CIFAR10, and SVHN test sets. We train RealNVP and VAE on FashionMNIST, and train RealNVP and PixelCNN on CIFAR10. We report likelihood results with training mode, evaluation mode, and their difference (∆).

Table content: <table><tr><td colspan="3">(a) RealNVP (trained on Fashion MNIST)</td><td colspan="2">(b) RealNVP (trained on CIFAR10)</td></tr><tr><td>Evaluated on</td><td>BN Mode BPD</td><td>∆</td><td colspan="2">Evaluated on BN Mode BPD</td><td>∆</td></tr><tr><td>Fashion MNIST</td><td colspan="2">evaluation 2.92 0.02 training 2.94</td><td>CIFAR10</td><td colspan="2">evaluation 3.48 0.03 training 3.51</td></tr><tr><td>MNIST</td><td colspan="2">evaluation 1.74 7.73 training 9.47</td><td>SVHN</td><td colspan="2">evaluation 2.44 8.56 training 11.10</td></tr><tr><td colspan="2">(c) VAE (trained on Fashion MNIST)</td><td/><td colspan="2">(d) PixelCNN++ (trained on CIFAR10)</td></tr><tr><td>Evaluated on</td><td>BN Mode BPD</td><td>∆</td><td colspan="2">Evaluated on BN Mode BPD</td><td>∆</td></tr><tr><td>Fashion MNIST</td><td colspan="2">evaluation 3.19 0.01 training 3.20</td><td>CIFAR10</td><td colspan="2">evaluation 3.21 0.12 training 3.33</td></tr><tr><td>MNIST</td><td colspan="2">evaluation 1.97 6.53 training 8.50</td><td>SVHN</td><td colspan="2">evaluation 2.16 1.88 training 4.04</td></tr><tr><td colspan="4">4.1 TRAINING MODE DECREASES LIKELIHOOD OF OOD</td><td/></tr></table>

Out-of-distribution classification evaluated with AUC (left) and Average Precision (right). Higher is better. Rotation denotes q(x) uses images in p(x) yet each image is rotated randomly by d ∈ (90, 270) degrees.

Table content: <table><tr><td>p(x)</td><td>Model</td><td>q(x)</td><td>log p θ (x)</td><td>T perm</td><td>WAIC</td><td>Ours</td></tr><tr><td/><td/><td>Rotation</td><td colspan="4">0.76 / 0.78 0.64 / 0.68 0.99 / 0.99 0.99 / 0.99</td></tr><tr><td>Fashion MNIST</td><td>RealNVP</td><td>MNIST Omniglot</td><td colspan="4">0.10 / 0.32 0.78 / 0.71 0.24 / 0.38 1.00 / 1.00 0.05 / 0.06 0.86 / 0.80 0.97 / 0.95 1.00 / 1.00</td></tr><tr><td/><td/><td>KMNIST</td><td colspan="4">0.47 / 0.46 0.45 / 0.45 0.63 / 0.65 1.00 / 1.00</td></tr><tr><td/><td/><td>Rotation</td><td colspan="4">0.73 / 0.72 0.61 / 0.64 0.94 / 0.95 0.97 / 0.98</td></tr><tr><td>Fashion MNIST</td><td>VAE</td><td>MNIST Omniglot</td><td colspan="4">0.13 / 0.33 0.73 / 0.68 0.56 / 0.64 1.00 / 1.00 0.00 / 0.06 0.99 / 0.96 0.90 / 0.83 1.00 / 1.00</td></tr><tr><td/><td/><td>KMNIST</td><td colspan="4">0.55 / 0.54 0.50 / 0.50 0.84 / 0.87 1.00 / 1.00</td></tr><tr><td/><td/><td>Rotation</td><td colspan="4">0.87 / 0.87 0.79 / 0.79 0.99 / 0.98 1.00 / 1.00</td></tr><tr><td>CIFAR10</td><td>RealNVP</td><td colspan="5">SVHN ImageNet 0.51 / 0.52 0.50 / 0.51 0.58 / 0.59 0.98 / 0.97 0.07 / 0.52 0.86 / 0.82 0.16 / 0.55 1.00 / 1.00</td></tr><tr><td/><td/><td>LSUN</td><td colspan="4">0.70 / 0.39 0.58 / 0.56 0.60 / 0.28 0.99 / 0.98</td></tr><tr><td/><td/><td colspan="5">CIFAR100 0.52 / 0.54 0.55 / 0.55 0.53 / 0.54 0.71 / 0.75</td></tr><tr><td/><td/><td>Rotation</td><td colspan="4">0.77 / 0.75 0.67 / 0.63 0.90 / 0.85 0.99 / 0.99</td></tr><tr><td colspan="2">CIFAR10 PixelCNN++</td><td colspan="5">SVHN ImageNet 0.51 / 0.51 0.49 / 0.50 0.66 / 0.69 0.89 / 0.87 0.10 / 0.32 0.86 / 0.77 0.09 / 0.53 0.99 / 0.99</td></tr><tr><td/><td/><td>LSUN</td><td colspan="4">0.72 / 0.69 0.60 / 0.58 0.78 / 0.74 0.98 / 0.97</td></tr></table>

Architecture for VAE convolutional layers. k denotes image width divided by 4, c denotes the number of channels of the image (1 for gray, 3 for colored).

Table content: <table><tr><td/><td colspan="2">Inference network</td><td/><td/></tr><tr><td colspan="5">Layer Channels Kernel Stride Padding</td></tr><tr><td>1</td><td>32c</td><td>5</td><td>1</td><td>2</td></tr><tr><td>2</td><td>32c</td><td>5</td><td>2</td><td>2</td></tr><tr><td>3</td><td>32c</td><td>5</td><td>1</td><td>2</td></tr><tr><td>4</td><td>64c</td><td>5</td><td>2</td><td>2</td></tr><tr><td>5</td><td>64c</td><td>5</td><td>1</td><td>2</td></tr><tr><td>6</td><td>256</td><td>k</td><td>1</td><td>0</td></tr><tr><td/><td colspan="3">Generator network</td><td/></tr><tr><td colspan="5">Layer Channels Kernel Stride Padding</td></tr><tr><td>1</td><td>64c</td><td>5</td><td>1</td><td>2</td></tr><tr><td>2</td><td>64c</td><td>5</td><td>2</td><td>2</td></tr><tr><td>3</td><td>64c</td><td>5</td><td>1</td><td>2</td></tr><tr><td>4</td><td>32c</td><td>5</td><td>2</td><td>2</td></tr><tr><td>5</td><td>32c</td><td>5</td><td>1</td><td>2</td></tr><tr><td>6</td><td>32c</td><td>5</td><td>1</td><td>2</td></tr><tr><td>7</td><td>256</td><td>5</td><td>1</td><td>2</td></tr><tr><td>8</td><td>256c</td><td>1</td><td>1</td><td>0</td></tr></table>

Log-likelihood (measured in bits per dimension) calculated with a RealNVP trained on CIFAR and evaluated on generated samples from a RealNVP trained on SVHN. We report likelihood results with training mode, evaluation mode, and their difference (∆). OUT-OF-DISTRIBUTION DETECTION WITH ALTERNATIVE VALUES OF r 1 AND r 2

Table content: <table><tr><td>Dataset</td><td>Mode</td><td>BPD</td><td>∆</td></tr><tr><td>CIFAR</td><td colspan="2">evaluation 3.48 training 3.51</td><td>0.03</td></tr><tr><td>SVHN</td><td colspan="2">evaluation 2.44 training 11.10</td><td>8.56</td></tr><tr><td>RealNVP (SVHN, T = 1.0)</td><td colspan="2">evaluation 2.88 training 328</td><td>325</td></tr><tr><td>RealNVP (SVHN, T = 0.7)</td><td colspan="2">evaluation 1.51 training 317</td><td/></tr><tr><td>RealNVP (SVHN, T = 1.3)</td><td colspan="3">evaluation 5.43 10.57 training 16.0</td></tr><tr><td>C.4</td><td/><td/><td/></tr></table>

Peer review (iclr2020):
The paper makes the observation that likelihood models trained with batch norm assign much lower likelihoods to *training batches* of OoD data (batch norm statistics computed over over minibatch) than evaluation batches of OoD data (batch norm statistics over entire training set). One issue with comparing this method to most other OoD detection works is that it considers OoD detection on *batches* of (all OoD data) or (all in-distribution data). As soon as the problem is changed to *classify between OoD batches* and not single samples, there are a large number of possible statistical tests one can perform to perform OoD (T-test between likelihoods of each batch) and the problem becomes *much* easier. In some ways, this makes things more well-defined (hard to compare distributions when one of them is just a single sample from an arbitrary distribution). However, that brings me to a big concern I have with the evaluation protocol. The batch size used for train/evaluation is rather large (64, this detail is hidden in the Appendix and I would have appreciated the number put in the main experiments section). If you take a likelihood model and evaluate on 64 samples from SVHN, you are all but guaranteed to sample a sample with *exceedingly* low likelihood, which dominates the mean statistic, making it possible to separate SVHN batch from CIFAR10 batches. I suspect that OoD datasets have plenty of these *extremely low likelihood* examples that will drag the mean likelihood down a lot. This is consistent with your batch normalization experiments: in training mode, the likelihood is computed from mean activations over a batch of OoD samples, several of which probably contribute to the low likelihoods. In evaluation mode, likelihoods for each OoD sample are evaluated independently, which results in a similar observation to prior work showing that CIFAR10 likelihoods are inaccurate for SVHN. In other words, I think there is a mistake made here: it is the phenomenon that *batch likelihoods*, not *batch norm*, that is responsible for this method working well. One experiment that is missing from your paper (and would prove my hypothesis wrong) would be if you adapted the OoD criteria to compare the *mean* likelihoods in the evaluation mode, and show that for OoD datasets, the difference between batches still remains small. Nits: - *...such as learning a mixture of Gaussians*, I believe this toy example was on univariate gaussians, not mixtures. - Choi et al. 2018 and should also be included in the citation that *CIFAR10 gives higher likelihood estimates to SVHN than CIFAR10 ones* (this was a concurrent discovery between the two papers) - Choi et al. 2018 is not the right citation for *we evaluate the area under the ROC curve (AUC) and average precision (AP)* for each binary classification task, a more appropriate one would be Hendryks and Gimpel 2017. - No doubt the authors realized already but Page 7 has some *related work still needs some work, but there....* which should be deleted. - It took me awhile to find the batch size used in training and evaluation mode (64), which was on page 16.