Understanding the Effect of Bias in Deep Anomaly Detection

1 INTRODUCTION . Anomaly detection ( Chandola et al. , 2009 ; Pimentel et al. , 2014 ) trains a formal model to identify unexpected or anomalous instances in incoming data , whose behaviors differ from normal instances . It is particularly useful for detecting problematic events such as digital fraud , structural defects , and system malfunctions . Building accurate anomaly detection models is a well-known challenge in machine learning , due to the scarcity of labeled anomaly data . The classical and most common approach is to train anomaly detection models using only normal data1 , i.e. , first train a model using a corpus of normal data to capture normal behaviors , then configure the model to flag instances with large deviations as anomalies . Researchers have also developed deep learning methods to better capture the complex structure in the data ( Ruff et al . ( 2018 ) ; Wang et al . ( 2019a ) ; Zhou & Paffenroth ( 2017 ) ) . Following the terminology introduced by Chandola et al . ( 2009 ) , we refer to these models as semi-supervised anomaly detection . Recently , a new line of anomaly detection models proposes to leverage available labeled anomalies during model training , i.e. , train an anomaly detection model using both normal data and additional labeled anomaly samples as they become available ( Ruff et al . ( 2020b ) ; Yamanaka et al . ( 2019 ) ; Ruff et al . ( 2020a ) ; Hendrycks et al . ( 2019a ) ) . Existing works show that these new models achieve considerable performance improvements beyond the models trained using only normal data . We hereby refer to these models as deep supervised2 anomaly detection ( Chandola et al. , 2009 ) . When exploring these models , we found that when the labeled anomalies ( used to train the model ) do not align with the target distribution , they could introduce harmful bias to the trained model . Specifically , when comparing the performance of a supervised anomaly detector to its semi-supervised 1Existing literature has used different terms to describe this type of models : some using semi-supervised anomaly detection ( Chandola et al. , 2009 ) and others using unsupervised anomaly detection ( Ruff et al. , 2018 ) . 2Some works termed these models as semi-supervised anomaly detection ( Ruff et al. , 2020b ; Yamanaka et al. , 2019 ; Ruff et al. , 2020a ; Hendrycks et al. , 2019a ) while others termed them as supervised anomaly detection ( Chandola et al. , 2009 ) . version , the performance difference varies significantly across test anomaly data , some better and some worse . That is , using labeled anomalies during model training does not always improve model performance ; instead , it may introduce large variance ( or bias ) in anomaly detection outcomes . In this paper , we aim to understand the effect of a biased training set on deep anomaly detection models . We formally state the anomaly detection problem , focusing on the anomaly detector ’ s recall at a given false positive rate as the main performance metric . We factor the contribution of the labeled anomalies by the detector ’ s anomaly scoring function , and show that different types of labeled anomalies produce different anomaly scoring functions . Next , given any two different anomaly scoring functions , we formally define their difference in performance as the relative scoring bias of the anomaly detectors . Our novel notion of scoring bias for anomaly detection aligns with the notion of bias in the classical supervised learning setting , with the key difference being the different performance metric—we target recall at a given false positive rate , the metric used by real-world anomaly detection tasks ( Li et al. , 2019 ; Liu et al. , 2018 ) . Along this line , we establish the first finite sample rates for estimating the relative scoring bias for deep anomaly detection . We empirically validate our assumptions and theoretical results on both synthetic and three real-world datasets ( Fashion-MNIST , Statlog ( Landsat Satellite ) , and Cellular Spectrum Misuse ( Li et al. , 2019 ) ) . Furthermore , we provide an empirical study on how a biased training anomaly set affects the anomaly score function and therefore the resulting detection performance . We consider the above three real-world datasets and six deep-learning based anomaly detection models . Our study demonstrates scenarios in which the biased anomaly set can be useful or problematic , and provides a solid benchmark for future research . In this paper , we introduce a formal analysis on the effect of a biased training set on deep anomaly detection . Our main contributions are the following : • We discover the issue of large performance variance in deep anomaly detectors , caused by the use of the biased anomaly set as training data . • We model the effect of biased training as relative scoring bias , and establish the first finite sample rates for estimating the relative scoring bias of the trained models . • We conduct empirical experiments to verify and characterize the impact of the relative scoring bias on six popular anomaly detection models , and three real-world datasets . To the best of our knowledge , our work is the first to formally study the effect of a biased anomaly training set on deep anomaly detection . Our results show both significant positive and negative impacts of these biases , and suggest that model trainers must treat anomalies with additional care . We believe this leads to new opportunities for improving deep anomaly detectors and deserves more attention from the research community . 2 RELATED WORK . Anomaly Detection Models . While the literature on anomaly detection models is extensive , the most relevant to our work are deep learning based models . Following the terminology used by Chandola et al . ( 2009 ) , we consider two types of models : • Semi-supervised anomaly detection refers to models trained on only normal data , e.g. , Ruff et al . ( 2018 ) ; Sakurada & Yairi ( 2014 ) ; Zhou & Paffenroth ( 2017 ) ; • Supervised anomaly detection refers to models trained on normal data and a small set of labeled anomalies , e.g. , Pang et al . ( 2019 ) ; Daniel et al . ( 2019 ) ; Yamanaka et al . ( 2019 ) ; Ruff et al . ( 2020a ; b ) . One can also categorize models by their architecture : hypersphere ( Ruff et al. , 2018 ; 2020a ; b ) and autoencoder ( or reconstruction ) based models ( Zhou & Paffenroth , 2017 ; Yamanaka et al. , 2019 ) . Another line of recent work proposes to use synthetic or auxiliary anomalies to train anomaly detection models ( Golan & El-Yaniv ( 2018 ) ; Hendrycks et al . ( 2019c ) ; Lee et al . ( 2018 ) ; Hendrycks et al . ( 2019b ) ) , “ forcing ” the model to learn a more compact representation of the normal data . While the existing work has shown empirically that the choice of abnormal data in training can help detect some unseen abnormal distributions , it does not offer any theoretical explanation for the phe- nomenon , nor does it consider the counter-cases when additional abnormal data in training hurt the detection performance . Bias in Anomaly Detection . To the best of our knowledge , we are the first to identify the presence of bias caused by an additional labeled anomaly set in deep anomaly detection models , especially when there exists a mismatch between the anomalies present in training and those encountered in testing ( as shown in Section 5 ) . Existing work has explored the presence of bias in semi-supervised anomaly detection models when there exists defective normal data in training , like outliers and simple-to-reconstruct examples ( Tong et al. , 2019 ) , or examples with background noise ( Liu & Ma , 2019 ) . There is also literature on the bias-variance tradeoff for ensembles of semi-supervised anomaly detection models ( Aggarwal & Sathe , 2015 ; Rayana et al. , 2016 ) . But little or no work has been done on the bias of anomaly detection in the supervised setting ( i.e. , models trained on both normal data and some labeled anomalies ) . Finally , another line of work in transfer learning has identified the value of additional labeled data in training ( Kpotufe & Martinet , 2018 ; Hanneke & Kpotufe , 2019 ) and the performance bias on target data by transferring knowledge from a less related source ( Wang et al. , 2019b ; Wu et al. , 2020 ) . Yet most work only considered the cases of classification models . PAC guarantees for Anomaly Detection . Despite significant progress on developing theoretical guarantees for classification tasks ( Valiant ( 1984 ) ; Kearns et al . ( 1994 ) ) , little has been done for anomaly detection tasks . Siddiqui et al . ( 2016 ) first establishes a PAC framework for anomaly detection models using the notion of pattern space ; however , it is hard to apply such pattern spaces to deep learning models with complex latent spaces . Liu et al . ( 2018 ) proposes a model-agnostic approach to provide the PAC guarantee for anomaly detection performance , by analyzing the convergence for the cumulative distribution of anomaly scores . We follow the basic setting from this line of work to address the convergence of the relative scoring bias . In contrast to prior work , our proof relies on a novel adaption of the key theoretical tool from Massart ( 1990 ) , which allows us to extend our theory to characterize the notion of scoring bias as defined in Section 3.2 . 3 PROBLEM FORMULATION . We now formally state the anomaly detection problem . Consider a model class Θ for anomaly detection , and a ( labeled ) training set D sampled from a mixture distribution D over the normal and anomalous instances . In the context of anomaly detection , a model θ maps each input instance x to a continuous output , which corresponds to anomaly score sθ ( x ) . The model further uses a threshold τθ on the score function to produce a binary label for input x . For a given threshold value τθ , we can define the False Positive Rate ( FPR ) of the model θ on the input data distribution as FPR ( sθ , τθ ) = P [ sθ ( x ) > τθ | y = 0 ] , and the True Positive Rate ( TPR , a.k.a . Recall ) as TPR ( sθ , τθ ) = P [ sθ ( x ) > τθ | y = 1 ] . The FPR and TPR are competing objectives—therefore , a key challenge for anomaly detection algorithms is to identify a configuration of the score , threshold pair ( sθ , τθ ) that strikes a balance between the two performance metrics . W.l.o.g.3 , in this paper we focus on the following scenario , where the objective is to maximize TPR subject to achieving a target FPR . Formally , let q be the target FPR ; we define the optimal anomaly detector as4 ( s∗θ , τ∗θ ) ∈ arg max ( sθ , τθ ) : θ∈Θ TPR ( sθ , τθ ) s.t . FPR ( sθ , τθ ) ≤ q ( 3.1 ) 3.1 A GENERAL ANOMALY DETECTION FRAMEWORK . Note that the performance metric ( namely TPR ) in Problem 3.1 is statistics that depends on the entire predictive distribution , and can not be easily evaluated on any single data point . Therefore , rather than directly solving Problem 3.1 , practical anomaly detection algorithms ( such as OCSVM ( Schölkopf et al. , 1999 ) , Deep SAD ( Ruff et al. , 2020b ) , etc ) often rely on a two-stage process : ( 1 ) 3Our results can be easily extended to the setting where the goal is to minimize FPR subject to a given TPR . 4This formulation aligns with many contemporary works in deep anomaly detection . For example , Li et al . ( 2019 ) show that in real-world anomaly detection problems , it is desirable to detect anomalies with a prefixed low false alarm rate ; Liu et al . ( 2018 ) formulate the anomaly detection in a similar way , where the goal is to minimize FPR for a fixed TPR . learning the score function sθ from training data via a surrogate loss , and ( 2 ) given sθ from the previous step , computing the threshold function τθ on the training data . Formally , given a model class Θ , a training set D , a loss function ` , and a target FPR q , a two-staged anomaly detection algorithm outputs { ŝθ ∈ arg minsθ : θ∈Θ ` ( sθ , D ) τ̂θ ∈ arg maxτθ : θ∈Θ TPR ( ŝθ , τθ ) s.t . FPR ( ŝθ , τθ ) ≤ q ( 3.2 ) Note that the first part of Equation 3.2 amounts to solving a supervised learning problem . Here , the loss function ` could be instantiated into latent-space-based losses ( e.g. , Deep SAD ) , marginbased losses ( e.g. , OCSVM ) , or reconstruction-based losses ( e.g. , ABC ( Yamanaka et al. , 2019 ) ) ; therefore , many contemporary anomaly detection models fall into this framework . To set the threshold τ̂θ , we consider using the distribution of the anomaly scores ŝθ ( · ) from a labeled validation set Dval ∼ D. Let Dval : = Dval0 ∪ Dvala where Dval0 and Dvala denote the subset of normal data and the subset of abnormal data of Dval . Denote the empirical CDFs for anomaly scores assigned to x in Dval0 and D val a as F̂0 and F̂a , respectively . Then , given a target FPR value q , following a similar argument as Liu et al . ( 2018 ) , one can compute the threshold as τ̂θ = max { u ∈ R : F̂0 ( u ) ≤ q } . The steps for solving the second part of Equation 3.2 is summarized in Algorithm 1 . Algorithm 1 : Computing the anomaly detection threshold for Problem 3.2 Data : A validation dataset Dval and a scoring function s ( · ) . Result : A score threshold achieving a target FPR and the corresponding recall on Dval . 1 Get anomaly score s ( x ) for each x in Dval . 2 Compute empirical CDF F̂0 ( x ) and F̂a ( x ) for anomaly scores of x in Dval0 and Dvala . 3 Output detection threshold τ̂ = max { u ∈ R : F̂0 ( u ) ≤ q } . 4 Output TPR ( recall ) on Dvala as r̂ = 1− F̂a ( τ̂ ) .

Summary: This paper studies the potential bias in deep semi-supervised anomaly detection. The bias is evaluated in terms of TPR rate given a fixed FPR rate. It uses the anomaly scores output by unsupervised anomaly detectors as a benchmark to examine the relative scoring bias in deep semi-supervised anomaly detectors. It further studies the finite sample rate for this type of scoring bias. This type of bias is verified using some synthetic and real-world datasets. The empirical results also show the potential impact of this bias on several anomaly detectors.