Deep Variational Semi-Supervised Novelty Detection

1 INTRODUCTION . Anomaly detection ( AD ) – the task of identifying samples that are abnormal with respect to some normal data – has important applications in domains ranging from health-care , to security , and robotics ( Pimentel et al. , 2014 ) . In its common formulation , training data is provided only for normal samples , while at test time , anomalous samples need to be detected . In the probabilistic AD approach , a model of the normal data distribution is learned , and the likelihood of a test sample under this model is thresholded for classification as normal or not . Recently , deep generative models such as variational autoencoders ( VAEs , Kingma & Welling 2013 ) and generative adversarial networks ( Goodfellow et al. , 2014 ) have shown promise for learning data distributions in AD ( An & Cho , 2015 ; Suh et al. , 2016 ; Schlegl et al. , 2017 ; Wang et al. , 2017 ) . Here , we consider the setting of semi-supervised AD ( SSAD ) , where in addition to the normal samples , a small sample of labeled anomalies is provided ( Görnitz et al. , 2013 ) . Most importantly , this set is too small to represent the range of possible anomalies , making classification methods ( either supervised or semi-supervised ) unsuitable . Instead , most approaches are based on ‘ fixing ’ an unsupervised AD method to correctly classify the labeled anomalies , while still maintaining AD capabilities for unseen outliers ( e.g. , Görnitz et al. , 2013 ; Muñoz-Marı́ et al. , 2010 ; Ruff et al. , 2019 ) . In this work , we present a variational approach for learning data distributions in the SSAD problem setting . We base our method on the VAE , and modify the training objective to account for the labeled outlier data . We propose two formulations for this problem . The first maximizes the log-likelihood of normal samples , while minimizing the log-likelihood of outliers , and we effectively optimize this objective by combining the standard evidence lower bound ( ELBO ) with the χ upper bound ( CUBO , Dieng et al . 2017 ) . The second method is based on separating the VAE prior between normal and outlier samples . Effectively , both methods have a similar intuitive interpretation : they modify the VAE encoder to push outlier samples away from the prior distribution ( see Figure 1 ) . Importantly , our method does not place any restriction on the VAE architecture , and can be used to modify any VAE to account for outliers . As such , it can be used for general types of data . We evaluate our methods in the comprehensive SSAD test-suite of Ruff et al . ( 2019 ) , which includes both image data and low-dimensional data sets from astronomy , medicine , and other domains , and report a marked improvement in performance compared to both shallow and deep methods . In addition , we demonstrate the flexibility of our method by modifying a conditional VAE used for generating sampling distributions for robotic motion planning ( Ichter et al. , 2018 ) to not generate way points that collide with obstacles . 2 RELATED WORK . Anomaly detection , a.k.a . outlier detection or novelty detection , is an active field of research . Shallow methods such as one-class SVM ( OC-SVM , Schölkopf et al . 2001 ) or support vector data description ( SVDD , Tax & Duin 2004 ) have shown great success , but rely on hand-crafted features . Recently , there is growing interest in deep learning methods for AD ( Chalapathy & Chawla , 2019 ) . Most studies on semi-supervised AD also require hand designed features . Muñoz-Marı́ et al . ( 2010 ) proposed S2OC-SVM , a modification of OC-SVM that introduces labeled and unlabeled samples , Görnitz et al . ( 2013 ) proposed an approach based on SVDD , and Blanchard et al . ( 2010 ) base their approach on statistical testing . Deep SSAD has been studied recently in specific contexts such as videos ( Kiran et al. , 2018 ) , network intrusion detection ( Min et al. , 2018 ) , or specific neural network architectures ( Ergen et al. , 2017 ) . The most relevant prior work to our study is the recently proposed deep semi-supervised anomaly detection ( Deep SAD ) approach of Ruff et al . ( 2019 ) . Deep SAD is a general method based on deep SVDD ( Ruff et al. , 2018 ) , which learns a neural-network mapping of the input that minimizes the volume of data around a predetermined point . While it has been shown to work on a variety of domains , Deep SAD must place restrictions on the network architecture , such as no bias terms ( in all layers ) and no bounded activation functions , to prevent degeneration of the minimization problem to a trivial solution . The methods we propose here do not place any restriction on the network architecture , and can be combined with any VAE model . In addition , we show improved performance in almost all domains compared to Deep SAD ’ s state-of-the-art results . While our focus here is SSAD , for completeness we review recent work on deep AD . Erfani et al . ( 2016 ) ; Andrews et al . ( 2016 ) ; Cao et al . ( 2016 ) ; Chen et al . ( 2017 ) follow a hybrid approach where deep unsupervised learning is used to learn features , which are then used within a shallow AD method . Specifically for image domains , Golan & El-Yaniv ( 2018 ) learn features using a selfsupervised paradigm – by applying geometric transformations to the image and learning to classify which transformation was applied . Lee et al . ( 2018 ) learn a distance metric in feature space , for networks pre-trained on image classification . Both of these methods display outstanding AD performance , but are limited to image domains , while our approach does not require particular data or pre-trained features . Several studies explored using deep generative models such as GANs and VAEs for AD ( e.g. , An & Cho , 2015 ; Suh et al. , 2016 ; Schlegl et al. , 2017 ; Wang et al. , 2017 ) , and more recently , deep energy-based models ( Zhai et al. , 2016 ; Song & Ou , 2018 ) have reported outstanding results . Our work extends the VAE approach to the SSAD setting . Our results show that even a 1 % fraction of labelled anomalies can improve over the state-of-the-art AD scores of deep energy based models , demonstrating the importance of the SSAD setting . 3 BACKGROUND . We consider deep unsupervised learning under the standard variational inference setting ( Kingma & Welling , 2013 ) . Given some data x , one aims to fit the parameters θ of a latent variable model pθ ( x ) = Ep ( z ) [ pθ ( x|z ) ] , where the prior p ( z ) is known . For general models , the typical maximumlikelihood objective maxθ log pθ ( x ) is intractable due to the marginalization over z , and can be approximated using the following variational inference methods . 3.1 EVIDENCE LOWER BOUND ( ELBO ) . The evidence lower bound ( ELBO ) states that for some approximate posterior distribution q ( z|x ) : log pθ ( x ) ≥ Eq ( z|x ) [ log pθ ( x|z ) ] −DKL ( q ( z|x ) ‖p ( z ) ) , where the Kullback-Leibler divergence is DKL ( q ( z|x ) ‖p ( z ) ) = Eq ( z ) [ log q ( z|x ) p ( z ) ] . In the variational autoencoder ( VAE , Kingma & Welling 2013 ) , the approximate posterior is represented as qφ ( z|x ) = N ( µφ ( x ) , Σφ ( x ) ) for some neural network with parameters φ , the prior is p ( z ) = N ( µ0 , Σ0 ) , and the ELBO can be maximized using the reparameterization trick . Since the resulting model resembles an autoencoder , the approximate posterior qφ ( z|x ) is also known as the encoder , while pθ ( x|z ) is termed the decoder . 3.2 χ UPPER BOUND ( CUBO ) Recently , Dieng et al . ( 2017 ) derived variational upper bounds to the data log-likelihood . The χndivergence is given by Dχn ( p‖q ) = Eq ( z ; θ ) [ ( p ( z|X ) q ( z ; θ ) ) n − 1 ] . For n > 1 , Dieng et al . ( 2017 ) propose the following χ upper bound ( CUBO ) : log pθ ( x ) ≤ 1 n logEq ( z|x ) [ ( pθ ( x|z ) p ( z ) q ( z|x ) ) n ] . = CUBOn ( q ( z|x ) ) . For a family of approximate posteriors qφ ( z|x ) , one can minimize the CUBO using Monte Carlo ( MC ) estimation . However , MC gives a lower bound to CUBO and its gradients are biased . As an alternative , Dieng et al . ( 2017 ) proposed the following optimization objective : L = exp { n · CUBOn ( qφ ( z|x ) ) } . By monotonicity of exp , this objective reaches the same optima as CUBOn ( qφ ( z|x ) ) . Moreover , this produces an unbiased estimate , and the number of samples only affects the variance of the gradients . 4 DEEP VARIATIONAL SEMI-SUPERVISED ANOMALY DETECTION . In the anomaly detection problem , the goal is to detect whether a sample x was generated from some normal distribution pnormal ( x ) , or not – making it an anomaly . In semi-supervised anomaly detection ( SSAD ) , we are given Nnormal samples from pnormal ( X ) , which we denote Xnormal . In addition , we are given Noutlier examples of anomalous data , denoted by Xoutlier , and we assume that Noutlier Nnormal . In particular , Xoutlier does not cover the range of possible anomalies , and thus classification methods ( neither supervised nor semi-supervised ) are not applicable . Our approach for SSAD is to approximate pnormal ( x ) using a deep latent variable model pθ ( x ) , and to decide whether a sample is anomalous or not based on thresholding its predicted likelihood . In the following , we propose two variational methods for learning pθ ( x ) . The first method , which we term max-min likelihood VAE ( MML-VAE ) , maximizes the likelihood of normal samples while minimizing the likelihood of outliers . The second method , which we term dual-prior VAE ( DPVAE ) , assumes different priors for normal and outlier samples . 4.1 MAX-MIN LIKELIHOOD VAE . In this approach , we seek to find model parameters based on the following objective : max θ log pθ ( Xnormal ) − γ log pθ ( Xoutlier ) , ( 1 ) where γ ≥ 0 is a weighting term . Note that , in the absence of outlier data or when γ = 0 , Equation 1 is just maximum likelihood estimation . For γ > 0 , however , we take into account the knowledge that outliers should not be assigned high probability . We model the data distribution using a latent variable model pθ ( x ) = Ep ( z ) [ pθ ( x|z ) ] , where the prior p ( z ) is known . To optimize the objective in Equation 1 effectively , we propose the following variational lower bound : log pθ ( Xnormal ) − γ log pθ ( Xoutlier ) ≥ ELBOQ1 ( Xnormal ) − γCUBOQ2 ( Xoutlier ) , ( 2 ) where Q1 ( z|x ) , Q2 ( z|x ) are the variational auxiliary distributions . In principle , the objective in Equation 2 can be optimized using the methods of Kingma & Welling ( 2013 ) and Dieng et al . ( 2017 ) , which would effectively require training two encoders : Q1 and Q2 , and one decoder pθ ( x|z ) ,1 separately on the two datasets . However , it is well-known that training deep generative models requires abundant data . Thus , there is little hope for learning an informative Q2 using the small datasetXoutlier . To account for this , our main idea is to use the same variational distributionQ ( z|x ) for both loss terms , which effectively relaxes the lower bound as follows : log pθ ( Xnormal ) − γ log pθ ( Xoutlier ) ≥ ELBOQ ( Xnormal ) − γCUBOQ ( Xoutlier ) . ( 3 ) In other words , we use the same encoder for both normal and anomalous data . Finally , the loss function of the MML-VAE is : L = γCUBOQ ( Xoutlier ) − ELBOQ ( Xnormal ) . To gain intuition about the effect of the CUBO term in the loss function , it is instructive to assume that pθ ( x|z ) is fixed . In this case , maximizing the lower bound only affects the variational distribution q ( z|x ) . Note that the ELBO term seeks to minimize the KL distance between q ( z|Xnormal ) and p ( z|Xnormal ) , which ‘ pushes ’ q ( z|Xnormal ) toward high-likelihood regions of p ( z ) , while the CUBO term is proportional to 1n logEq ( z|x ) [ ( p ( z|Xoutlier ) q ( z|Xoutlier ) ) n ] , and thus seeks to maximize the χn distance between q ( z|Xoutlier ) and p ( z|Xoutlier ) . Now , assume that for some outlier , p ( z|Xoutlier ) falls within a high-likelihood region of p ( z ) . In this case , the CUBO term will ‘ push ’ q ( z|Xoutlier ) away from p ( z ) . Thus , intuitively , the CUBO term seeks to separate the latent distributions for normal samples , which will concentrate on high-likelihood regions of p ( z ) , and outlier samples . For the common choice of Gaussian prior and approximate posterior distributions ( Kingma & Welling , 2013 ) , the CUBO component of the loss function is given by : LCUBO = exp { −2 · LR + log |Σq|+ µTq Σ−1q µq+ logEq [ exp { −zT z + zTΣ−1q z − 2zTΣ−1q µq } ] } . Where LR = − log pθ ( x|z ) is the reconstruction error . The expectation is approximated with MC . In our experiments , we found that computing exp ( logE [ exp ( · ) ] ) is numerically more stable , as we can employ the log-sum-exp trick . In practice , we found that updating only the encoder according to the CUBO loss ( i.e. , ignoring the LR term ) results in better performance and stabler training . The reason for this is that CUBO seeks to maximize the reconstruction error , which does not produce informative updates for the decoder . Since this decoder is shared with the ELBO term , the CUBO update can decrease performance for normal samples as well . Complete algorithm details and derivation of the CUBO objective are provided Appendix A.4 .

Summary: This paper presents two novel VAE-based methods for the (more general) semi-supervised anomaly detection (SSAD) setting where one has also access to some labeled anomalous samples in addition to mostly normal data. The first method, Max-Min Likelihood VAE (MML-VAE), extends the standard VAE objective that maximizes the log-likelihood for normal data by an additional term that in contrast minimizes the log-likelihood for labeled anomalies. To optimize the MML objective, the paper proposes to minimize the sum of the standard (negative) ELBO for normal samples and the so-called CUBO, which is a variational upper bound on the data log-likelihood, for anomalous samples. The second method, Dual Prior VAE (DP-VAE), modifies the standard VAE by introducing a second separate prior for the anomalous data, which is also Gaussian but has different mean. The DP-VAE objective then is defined as the sum of the two respective ELBOs which is optimized over shared encoder and decoder networks (with the adjustment that the outlier ELBO only updates the encoder). The anomaly score for both models then is defined as the (negative) ELBO of a test sample. Finally, the paper presents quite extensive experimental results on the benchmarks from Ruff et al. [2], CatsVsDogs, and an application of robotic motion planning which indicate a slight advantage of the proposed methods.