Year: 2018

Abstract:
Deep generative models are capable of learning probability distributions over large, high-dimensional datasets such as images, video and natural language. Generative models trained on samples from p(x) ought to assign low likelihoods to out-of-distribution (OoD) samples from q(x), making them suitable for anomaly detection applications. We show that in practice, likelihood models are themselves susceptible to OoD errors, and even assign large likelihoods to images from other natural datasets. To mitigate these issues, we propose Generative Ensembles, a model-independent technique for OoD detection that combines density-based anomaly detection with uncertainty estimation. Our method outperforms the Outof-DIstribution detector for Neural networks (ODIN) and Variational Information Bottleneck (VIB) baselines on image datasets, and achieves comparable performance to a classification model on the Kaggle Credit Fraud dataset.

heading: 1 INTRODUCTION
text: Knowing when a machine learning (ML) model is qualified to make predictions on an input is critical to safe deployment of ML technology in the real world. When training and test distributions differ, neural networks may provide – with high confidence – arbitrary predictions on inputs that they are unaccustomed to seeing. To mitigate these Out-of-Distribution (OoD) errors, we require methods to determine whether a given input is sampled from a different stochastic generator than the one used to train the model.
OoD detection techniques have broad applications beyond safe deployment of ML technology. As datasets for ML grow ever larger and trend towards automated data collection, we require scalable methods for identifying outliers and quantifying noise before we can attempt to train models on that data. Identifying anomalies in data is a crucial feature of many data-driven applications, such as credit fraud detection and monitoring patient data in medical settings.
Generative modeling algorithms have improved dramatically in recent years, and are capable of learning probabilistic models over large, high-dimensional datasets such as images, video, and natural language (Vaswani et al., 2017; Wang et al., 2018). A generative model pθ(x), parameterized by random variable θ and trained to approximate data distribution p(x), ought to assign low likelihoods to samples from any distribution q(x) that differs from p(x). Density estimation does not presuppose a specific “alternate” distribution at training time, making it an attractive alternative to classification-based anomaly detection methods.
In this work, we apply several classes of generative models to OoD detection problems and demonstrate a significant shortcoming to high-dimensional density estimation models: the anomaly detection model itself may be mispecified. Explicit likelihood models can, in practice, realize high likelihoods to adversarial examples, random noise, and even other natural image datasets. We also illustrate how GAN discriminators presuppose a particular OoD distribution, which makes them particularly fragile at OoD classification. We propose Generative Ensembles, which combine density estimation with uncertainty estimation to detect OoD in a robust manner. Generative Ensembles are model-independent and are trained independently of the task-specific ML model of interest. Our method outperforms task-specific OoD baselines on the majority of evaluated OoD tasks and demonstrate competitive results with discriminative classification approaches on the Kaggle Credit Fraud dataset.

heading: 2 GENERATIVE ENSEMBLES
text: We consider several classes of generative modeling techniques in our experiments. Autoregressive Models and Normalizing Flows (NF) are fully-observed likelihood models that construct a tractable log-likelihood approximation to the data-generating density p(x) (Uria et al., 2016; Dinh et al., 2014; Rezende & Mohamed, 2015). Variational Autoencoders (VAE) are latent variable models that maximize a variational lower bound on log density (Kingma & Welling, 2013; Rezende et al., 2014). Finally, Generative Adversarial Networks (GAN) are implicit density models that minimize a divergence metric between p(x) and generative distribution qθ(x) (Goodfellow et al., 2014). We refer to a GAN’s generative distribution as qθ(x) (in lieu of pθ(x)) because from the GAN discriminator’s point of view, the outputs of the generator are OoD and depend on θ.
Although log p(x) and its lower bounds are proper scoring methods (Lakshminarayanan et al., 2017), we approximate them in practice with continuous-valued neural network function approximators log pθ(x). Neural networks have non-smooth predictive distributions, which makes them susceptible to malformed inputs that exploit idiosyncratic computation within the model (Szegedy et al., 2013).
Likelihood function approximators are no exception. When judging natural images, we assume an OoD input x ∼ q(x) should remain OoD within some LP -norm, and yet a Fast Gradient Sign Method (FGSM) attack (Goodfellow et al., 2015) on the predictive distribution can realize extremely high likelihood predictions (Nguyen et al., 2015). Conversely, a FGSM attack in the reverse direction on an in-distribution sample x ∼ p(x) creates a perceptually identical input with low likelihood predictions (Kos et al., 2018). To make matters worse, we show in Figure 1 that likelihood models can be fooled by OoD samples that are not even adversarial by construction, such as SVHN test images on a likelihood model trained on CIFAR-10. Concurrent work by Nalisnick et al. (2018) also show this phenomena and present additional analyses on why generative models systematically assign higher likelihoods to SVHN.
Generative Ensembles detect OoD examples by combining a density evaluation model with predictive uncertainty estimation on the density model via ensemble variance. Following the results of Lakshminarayanan et al. (2017), we elect to use independently trained ensembles instead of a Bayesian Dropout approximation (Gal & Ghahramani, 2016). For generative models that admit exact likelihoods (or variational approximations), the ensemble can be used to implement the Watanabe-Akaike Information Criterion (WAIC), which consists of a density estimation score with a Bayesian correction term for model bias (Watanabe, 2010):
WAIC(x) = Eθ[log pθ(x)]−Varθ [log pθ(x)] (1)

heading: 2.1 OOD DETECTION WITH GAN DISCRIMINATORS
text: We describe how to construct Generative Ensembles based on implicit density models such as GANs, and highlight the importance of OoD detection approaches that do not presuppose a specific OoD distribution. A discriminative model tasked with classifying between p(x) and q(x) is fragile to inputs that lie in neither distribution. Figure 2b illustrates a simple 2D density modeling task where individual GAN discriminators – when trained to convergence – learn a discriminative boundary that does not adequately capture p(x).
However, unlike discriminative anomaly classifiers on a static datasets, which model p(x)/q(x), the likelihood ratio p(x)/qθ(x) implicitly assumed by a GAN discriminator is uniquely randomized by GAN training dynamics on θ. By training an ensemble of GANs we can estimate the posterior distribution over model decision boundaries p(x)/qθ(x), or equivalently, the posterior distribution over alternate distributions qθ(x). In other words, we can use uncertainty estimation on randomly sampled discriminators to de-correlate the OoD classification errors made by a single discriminator (Figure 2c).

heading: 3 RELATED WORK
text: We can categorize existing OoD detection techniques in Table 1 using two criteria: (1) Does it assume a specific anomaly distribution? (2) Is the technique specific to the model, or does it only depend on the inputs to the model?
A common approach to OoD detection (a.k.a. anomaly detection) is to label a dataset of anomalous data and train a binary classifier on that label. Alternatively, a classification task model may be augmented with a “None of the above” class. The classifier then learns a decision boundary (likelihood ratio) between p(x) and q(x). However, the discriminative approach to anomaly detection requires the anomaly distribution to be specified at training time; this is a severe flaw when anomalous data is rare (e.g. medical seizures) or non-stationary (e.g. generated by an adversary).

heading: 3.1 UNCERTAINTY ESTIMATION
text: OoD detection is closely related to the problem of uncertainty estimation, whose goal is to yield calibrated confidence measures for a model’s predictive distribution pθ(y|x). Well-calibrated uncertainty estimation integrates several forms of uncertainty into pθ(y|x): model mispecification un-
certainty (OoD detection of invalid inputs), aleatoric uncertainty (irreducible input noise for valid inputs), and epistemic uncertainty (unknown model parameters for valid inputs). In this paper, we study OoD detection in isolation; instead of considering whether pθ(y|x) should be trusted for a given x, we are trying to determine whether x should be fed into pθ(y|x) at all. Predictive uncertainty estimation is a model-dependent OoD technique because it depends on taskspecific information (such as labels and task model architecture) in order to yield an integrated estimate of uncertainty. ODIN (Liang et al., 2017), MC Dropout (Gal & Ghahramani, 2016) and DeepEnsemble (Lakshminarayanan et al., 2017) model a calibrated predictive distribution for a classification task. Variational information bottleneck (VIB) (Alemi et al., 2018b) performs divergence estimation in latent space to detect OoD, but is technically a model-dependent technique because the latent code is trained jointly with the downstream classification task.
One limitation of model-dependent OoD techniques is that they may discard information about p(x) in learning the task-specific loss function pθ(y|x). Consider a contrived binary classification model on images that learns to solve the task perfectly by discarding all information except the contents of the first pixel (no other information is preserved in the features). Subsequently, the model yields confident predictions on any distribution that happens to preserve identical first-pixel statistics. In contrast, density estimation in data space x considers the structure of the entire input manifold, without bias towards a particular downstream task or task-specific compression.
In our work we estimate predictive uncertainty of the scoring model itself. Unlike predictive uncertainty methods applied to the task model’s predictions, Generative Ensembles do not require task-specific labels to train. Furthermore, model-independent OoD detection aids interpretation of predictive uncertainty by isolating the uncertainty component arising from OoD inputs.

heading: 3.2 ADVERSARIAL DEFENSE
text: Song et al. (2017) make the observation that adversarial examples designed to fool a downstream task have low likelihood under an independent generative model. They propose a “data purification” pipeline where inputs are first modified via gradient ascent on model likelihood, before passing it to the unmodified classifier. Their evaluations are restricted to Lp-norm attacks on in-distribution inputs to the task model, and do not take into account that the generative model itself may be susceptible to OoD errors. In fact, a preprocessing step with gradient ascent on model likelihood has the exact opposite of the desired effect when the input is OoD to begin with.
Our work considers adversarial defense in a broader OoD context. Although adversarial attacks literature typically considers small Lp-norm modifications to input (demonstrating the alarming sensitivity of neural networks), there is no such restriction in practice to the degree with which an input can be perturbed in a test setting. Adversarial defense is nothing more than making ML models robust to OoD inputs; whether they come from an attacker or not is irrelevant. We evaluate our methods on simple OoD transformations (flipping images), common ML datasets, and the adversaraial setting where a worst-case input is created from a single model in the ensemble.
He et al. (2017) demonstrate that ensembling adversarial defenses does not completely mitigate local sensitivity of neural networks. It is certainly plausible that sufficient search over a Generative
Ensemble’s predictions can find OoD inputs with both low variance and high likelihood. The focus of our work is to measure the extent to which uncertainty estimation improves robustness to model mispecification error, not to present a provably secure system. Having said that, model-independent OoD detection is easy to obfuscate in a practical ML security setting since the user only has access to the task model. Furthermore, a Generative Ensemble’s WAIC estimate can be made more robust by sampling additional models from the posterior over model parameters.

heading: 4 EXPERIMENTAL RESULTS
text: Following the experiments proposed by Liang et al. (2017) and Alemi et al. (2018b), we train OoD models on MNIST, Fashion MNIST, CIFAR-10 datasets, and evaluate anomaly detection on test samples from other datasets. In line with the aforementioned works, we measure anomaly detection capability based on AUROC over several quantities shown in Table 2. Our proposed quantities include single Wasserstein GAN (WGAN) discriminators (Arjovsky et al., 2017) with fine-tuning (D), ensemble variance of discriminators (Var(D)), likelihood models (log pθ(x)), and WAIC estimated using an ensemble of likelihood models. We follow the protocol as suggested by Lakshminarayanan et al. (2017) to use 5 independent models with different parameter initializations, trained on the full training set (no bootstrap). For likelihood estimators based on variational autoencoders (VAE), we also evaluate the rate term DKL(qθ(z|x)‖p(z)), which corresponds to information loss between the latent inference distribution and prior.
For MNIST and Fashion MNIST datasets, we use a VAE to predict a 16-sample Importance Weighted AutoEncoder (IWAE) bound. We extend the VAE example code1 from Tensorflow Probability (Dillon et al., 2017) to use a Masked Autoregressive Flow prior (Papamakarios et al., 2017), and train the model for 5k steps. Additional architectural details are found in Appendix B.
Our WGAN model’s generator and discriminator share the same architecture with the VAE decoder and encoder, respectively. The discriminator has an additional linear projection layer to its prediction of the Wasserstein metric. To ensure D represents a meaningful discriminative boundary between the two distributions, we freeze the generator and fine-tune the discriminator for an additional 4k steps on stationary p(x) and qθ(x). We also include Gaussian noise adversarially perturbed by FGSM on a single model (Adversarial).
For CIFAR-10 WGAN experiments, we change the first filter size in the discriminator from 7 to 8. For log-likelihood estimation, we train a vanilla GLOW model (Kingma & Dhariwal, 2018) for 250k steps, as we require a more powerful generative model to obtain good results.
The baseline methods are model-dependent and learn from the joint distribution of images and labels, while our methods use only images. For the VIB baseline, we use the rate term as the threshold variable. The experiments in Alemi et al. (2018b) make use of (28, 28, 5) “location-aware” features concatenated to the model inputs, to assist in distinguishing spatial inhomogeneities in the data. In this work we train vanilla generative models with no special modifications, so for fair comparison we also train VIB without location-aware features. For CIFAR-10 experiments, we train VIB for 26 epochs and converge at 75.7% classification accuracy on the test set. All other experimental parameters for VIB are identical to those in Alemi et al. (2018b).
Despite being trained on strictly less data (no labels), our methods – in particular Generative Ensembles – outperform ODIN and VIB on most OoD tasks. The VAE rate term appears to be quite effective, outperforming likelihood and WAIC estimation in data space. It is robust to adversarial inputs on the same model, because the FGSM perturbation primarily minimizes the (larger) distortion component of the approximate likelihood. The performance of VAE rate versus VIB rate also suggests that latent codes learned from generative objectives are more useful for OoD detection that latent codes learned via a classification-specific objective.

heading: 4.1 FAILURE ANALYSIS
text: In this section we discuss the experiments in which Generative Ensembles performed poorly, and suggest simple fixes to address these issues.
1https://github.com/tensorflow/probability/blob/master/tensorflow_ probability/examples/vae.py
In an earlier draft of this work, a VAE trained on Fashion MNIST performed poorly on all OoD datasets when using log pθ(x) and WAIC metrics. This was surprising, since the same metrics performed well when the same VAE architecture was trained on MNIST. To explain this phenomenon, we show in Figure 3 inputs and VAE-decoded outputs from Fashion MNIST and MNIST test sets. Fashion MNIST images are reconstructed properly, while MNIST images are are barely recognizable after decoding.
A VAEs training objective can be interpreted as the sum of a pixel-wise autoencoding loss (distortion) and a “semantic” loss (rate). Even though Fashion MNIST appears to be better reconstructed in a semantic sense, the distortion values between the FMNIST and MNIST test datasets are numerically quite similar, as shown in Figure 3. Distortion terms make up the bulk of the IWAE predictions in our models, thus explaining why log pθ(x) was not very discriminative when classifying OoD MNIST examples.
Higgins et al. (2016) propose β-VAE, a simple modification to the standard VAE objective: p(x|z) + β ·DKL(qθ(z|x)‖p(z)). β controls the relative balance between rate and distortion terms during training. Setting β < 1 is a commonly prescribed fix for encouraging VAEs to approach the “autoencoding limit” and avoid posterior collapse (Alemi et al., 2018a). At test time, this results in higher-fidelity autoencoding at the expense of higher rates, which seems to be a more useful signal for identifying outliers than the total pixel distortion (also suggested by Table 2, column 7). Re-training the ensemble with β = .1 encourages a higher distortion penalty during training, and thereby fixes the OoD detection model.

heading: 4.2 CREDIT CARD ANOMALY DETECTION
text: We consider the problem of detecting fraudulent credit card transactions from the Kaggle Credit Fraud Challenge (Dal Pozzolo et al., 2015). A conventional approach to fraud detection is to include a small fraction of fraudulent transactions in the training set, and then learn a discriminative classifier. Instead, we treat fraud detection as an anomaly detection problem where a generative model only sees normal credit card transactions at training time. This is motivated by realistic test scenarios, where an adversary is hardly restricted to generating data identically distributed to the training set.
We compare single likelihood models (16-sample IWAE) and Generative Ensembles (ensemble variance of IWAE) to a binary classifier baseline that has access to a training set of fraudulent transactions in Table 3. The classifier baseline is a fully-connected network with 2 hidden ReLU layers of 512 units, and is trained using a weighted sigmoid cross entropy loss (positive weight=580) with Dropout and RMSProp (α = 1e−5). The VAE encoder and decoder are fully connected networks with single hidden layers (32 and 30 units, respectively) and trained using Adam (α = 1e−3). Unsurprisingly, the classifier baseline performs best because fraudulent test samples are distributed identically to fraudulent training samples. Even so, the single-model density estimation and Generative Ensemble achieve reasonable results.

heading: 5 DISCUSSION AND FUTURE WORK
text: OoD detection is a critical piece of infrastructure for ML applications where the test data distribution is not known at training time. We present Generative Ensembles, a simple yet powerful technique for model-independent OoD detection that improves density models with uncertainty estimation.
An important future direction of research is that of scalability: learning good generative models of semantically rich, high-dimensional inputs (e.g. video) is an active research area in its own right. An open question is whether an ensemble of weak generative models (where each model may not necessarily generate high-quality samples) can still yield density and uncertainty predictions useful enough for OoD detection. Preliminary evidence on CIFAR-10 are promising; although the ensemble average on the test set is ∼ 3.5 bits/dim and samples from the prior do not resemble any recognizable objects, the ensemble still performs well at OoD detection. In future work we will explore other methods of de-correlating samples from the posterior over model parameters, as well as combining independent scores (D, Rate, logθ p(x), WAIC) into a more powerful OoD model.

heading: A TERMINOLOGY AND ABBREVIATIONS
text: p(x) Training data distribution q(x) OoD data distribution pθ(x) Learned approximation of true distribution with parameters θ. May be implicitly specified, i.e. a fully-observed density model qθ(x) Learned approximation of OoD distribution with parameters θ.
May be implicitly specified, i.e. via a GAN discriminator that learns p(x)/qθ(x)
OoD Input Out-of-Distribution Input. Invalid input to a ML model Anomaly Synonym with OoD Input Epistemic Uncertainty Variance in a model’s predictive distribution arising from ignorance of true model parameters for a given input Aleatoric Uncertainty Variance in a model’s predictive distribution arising from inherent, irreducible noise in the inputs Predictive Uncertainty Variance of a model’s predictive distribution, which takes into
account all of the above
MNIST Dataset of handwritten digits (size: 28x28) FashionMNIST Dataset of clothing thumbnails (size: 28x28) CIFAR-10 Dataset of color images (size: 32x32x3)
GAN Generative Adversarial Network. See Goodfellow et al. (2014) FSGM Fast Sign Gradient Method WGAN Wasserstein GAN. See Arjovsky et al. (2017) VAE Variational Autoencoder. See Kingma & Welling (2013); Rezende et al. (2014) Rate DKL(qθ(z|x)‖p(z)) term in the VAE objective. Information loss between encoder distribution and prior over latent code IWAE Importance Weighted Autoencoder GLOW A generative model based on normalizing flows. See Kingma
& Dhariwal (2018)
ODIN Out-of-DIstribution detector for Neural networks. See Liang et al. (2017) VIB Variational Information Bottleneck. See Alemi et al. (2018b) WAIC Watanabe-Akaike Information Criterion. See Watanabe (2010)
AUROC Area Under ROC Curve FPR@95%TPR False Positives at 95% True Positives AP Average Precision
B VAE ARCHITECTURAL DETAILS
We use a flexible learned prior pθ(z) in our VAE experiments, but did not observe a significant performance difference compared to the default mixture prior in the base VAE code sample. We use an alternating chain of 6 MAF bijectors and 6 random permutation bijectors. Each MAF bijector uses TensorFlow Probability’s default implementation with the following parameter:
s h i f t a n d l o g s c a l e f n = t f b . m a s k e d a u t o r e g r e s s i v e d e f a u l t t e m p l a t e ( h i d d e n l a y e r s =(512 , 5 1 2 ) )
Models are trained with Adam (α = 1e−3) with cosine decay on learning rate.

Review summaries:
The authors present an OOD detection scheme with an ensemble of generative models. When the exact likelihood is available from the generative model, the authors approximate the WAIC score. For GAN models, the authors compute the variance over the discriminators for any given input. They show that this method outperforms ODIN and VIB on image datasets and also achieves comparable performance on Kaggle Credit Fraud dataset.