C LASSIFICATION -B ASED A NOMALY D ETECTION FOR G ENERAL D ATA
A BSTRACT
Anomaly detection, ﬁnding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artiﬁcial intelligence.
Recently, classiﬁcation-based methods were shown to achieve superior results on this task.
In this work we present a unifying view and propose an open-set method, GOAD, to relax current generalization assumptions.
Furthermore, we extend the applicability of transformation-based methods to non-image data using random afﬁne transformations.
Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types.
The strong performance of our method is extensively validated on multiple datasets from different domains
I NTRODUCTION
Detecting anomalies in perceived data is a key ability for humans and for artiﬁcial intelligence.
Humans often detect anomalies to give early indications of danger or to discover unique opportunities.
Anomaly detection systems are being used by artiﬁcial intelligence to discover credit card fraud, for detecting cyber intrusion, alert predictive maintenance of industrial equipment and for discovering attractive stock market opportunities.
The typical anomaly detection setting is a one class classiﬁcation task, where the objective is to classify data as normal or anomalous.
The importance of the task stems from being able to raise an alarm when detecting a different pattern from those seen in the past, therefore triggering further inspection.
This is fundamentally different from supervised learning tasks, in which examples of all data classes are observed.
There are different possible scenarios for anomaly detection methods.
In supervised anomaly detection, we are given training examples of normal and anomalous patterns.
This scenario can be quite well speciﬁed, however obtaining such supervision may not be possible.
For example in cyber security settings, we will not have supervised examples of new, unknown computer viruses making supervised training difﬁcult.
On the other extreme, fully unsupervised anomaly detection, obtains a stream of data containing normal and anomalous patterns and attempts to detect the anomalous data.
In this work we deal with the semi-supervised scenario.
In this setting, we have a training set of normal examples (which contains no anomalies).
After training the anomaly detector, we detect anomalies in the test data, containing both normal and anomalous examples.
This supervision is easy to obtain in many practical settings and is less difﬁcult than the fully-unsupervised case.
Many anomaly detection methods have been proposed over the last few decades.
They can be broadly classiﬁed into reconstruction and statistically based methods.
Recently, deep learning methods based on classiﬁcation have achieved superior results.
Most semi-supervised classiﬁcationbased methods attempt to solve anomaly detection directly, despite only having normal training data.
One example is: Deep-SVDD (Ruff et al., 2018) - one-class classiﬁcation using a learned deep space.
Another type of classiﬁcation-based methods is self-supervised i.e. methods that solve one or more classiﬁcation-based auxiliary tasks on the normal training data, and this is shown to be useful for solving anomaly detection, the task of interest e.g. (Golan & El-Yaniv, 2018).
Self-supervised classiﬁcation-based methods have been proposed with the object of image anomaly detection, but we show that by generalizing the class of transformations they can apply to all data types.
In this paper, we introduce a novel technique, GOAD, for anomaly detection which uniﬁes current state-of-the-art methods that use normal training data only and are based on classiﬁcation.
Our method ﬁrst transforms the data into M subspaces, and learns a feature space such that inter-class separation is larger than intra-class separation.
For the learned features, the distance from the cluster center is correlated with the likelihood of anomaly.
We use this criterion to determine if a new data point is normal or anomalous.
We also generalize the class of transformation functions to include afﬁne transformation which allows our method to generalize to non-image data.
This is signiﬁcant as tabular data is probably the most important for applications of anomaly detection.
Our method is evaluated on anomaly detection on image and tabular datasets (cyber security and medical) and is shown to signiﬁcantly improve over the state-of-the-art.
1. P REVIOUS W ORKS
Anomaly detection methods can be generally divided into the following categories:
Reconstruction Methods: Some of the most common anomaly detection methods are reconstructionbased.
The general idea behind such methods is that every normal sample should be reconstructed accurately using a limited set of basis functions, whereas anomalous data should suffer from larger reconstruction costs.
The choice of features, basis and loss functions differentiates between the different methods.
Some of the earliest methods use: nearest neighbors (Eskin et al., 2002), low-rank
PCA (Jolliffe, 2011; Cand`es et al., 2011) or K-means (Hartigan & Wong, 1979) as the reconstruction basis.
Most recently, neural networks were used (Sakurada & Yairi, 2014; Xia et al., 2015) for learning deep basis functions for reconstruction.
Another set of recent methods (Schlegl et al., 2017; Deecke et al., 2018) use GANs to learn a reconstruction basis function.
GANs suffer from mode-collapse and are difﬁcult to invert, which limits the performance of such methods.
Distributional Methods: Another set of commonly used methods are distribution-based.
The main theme in such methods is to model the distribution of normal data.
The expectation is that anomalous test data will have low likelihood under the probabilistic model while normal data will have higher likelihoods.
Methods differ in the features used to describe the data and the probabilistic model used to estimate the normal distribution.
Some early methods used Gaussian or Gaussian mixture models.
Such models will only work if the data under the selected feature space satisﬁes the probabilistic assumptions implicied by the model.
Another set of methods used non-parametric density estimate methods such as kernel density estimate (Parzen, 1962).
Recently, deep learning methods (autoencoders or variational autoencoders) were used to learn deep features which are sometimes easier to model than raw features (Yang et al., 2017).
DAGMM introduced by Zong et al.
(2018) learn the probabilistic model jointly with the deep features therefore shaping the features space to better conform with the probabilistic assumption.
Classiﬁcation-Based Methods: Another paradigm for anomaly detection is separation between space regions containing normal data from all other regions.
An example of such approach is One-Class
SVM (Scholkopf et al., 2000), which trains a classiﬁer to perform this separation.
Learning a good feature space for performing such separation is performed both by the classic kernel methods as well as by the recent deep learning approach (Ruff et al., 2018).
One of the main challenges in unsupervised (or semi-supervised) learning is providing an objective for learning features that are relevant to the task of interest.
One method for learning good representations in a self-supervised way is by training a neural network to solve an auxiliary task for which obtaining data is free or at least very inexpensive.
Auxiliary tasks for learning high-quality image features include: video frame prediction (Mathieu et al., 2016), image colorization (Zhang et al., 2016; Larsson et al., 2016), puzzle solving (Noroozi & Favaro, 2016) - predicting the correct order of random permuted image patches.
Recently, Gidaris et al.
(2018) used a set of image processing transformations (rotation by 0 , 90 , 180 , 270 degrees around the image axis, and predicted the true image orientation has been used to learn high-quality image features.
Golan & El-Yaniv (2018), have used similar image-processing task prediction for detecting anomalies in images.
This method has shown good performance on detecting images from anomalous classes.
In this work, we overcome some of the limitations of previous classiﬁcation-based methods and extend their applicability of self-supervised methods to general data types.
We also show that our method is more robust to adversarial attacks.
C LASSIFICATION -B ASED A NOMALY D ETECTION
Classiﬁcation-based methods have dominated supervised anomaly detection.
In this section we will analyse semi-supervised classiﬁcation-based methods:
Let us assume all data lies in space R L (where L is the data dimension).
Normal data lie in subspace
X ⊂ R L .
We assume that all anomalies lie outside X .
To detect anomalies, we would therefore like to build a classiﬁer C ,
such that C ( x ) = 1 if x ∈ X and C ( x ) = 0 if x ∈ R L \ X .
One-class classiﬁcation methods attempt to learn C directly as P ( x ∈ X ) .
Classical approaches have learned a classiﬁer either in input space or in a kernel space.
Recently, Deep-SVDD (Ruff et al., 2018) learned end-to-end to i) transform the data to an isotropic feature space f ( x ) ii) ﬁt the minimal hypersphere of radius R and center c 0 around the features of the normal training data.
Test data is classiﬁed as anomalous if the following normality score is positive: (cid:107) f ( x ) − c 0 (cid:107) 2 − R 2 .
Learning an effective feature space is not a simple task, as the trivial solution of f ( x ) =
∀ x results in the smallest hypersphere, various tricks are used to avoid this possibility.
Geometric-transformation classiﬁcation (GEOM), proposed by Golan & El-Yaniv (2018)
ﬁrst transforms the normal data subspace X into M subspaces X 1 ..X M .
This is done by transforming each image x ∈ X using M different geometric transformations (rotation, reﬂection, translation) into
T ( x, 1) ..
T ( x, M ) .
Although these transformations are image speciﬁc, we will later extend the class of transformations to all afﬁne transformations making this applicable to non-image data.
They set an auxiliary task of learning a classiﬁer able to predict the transformation label m given transformed data point T ( x, m ) .
As the training set consists of normal data only, each sample is x ∈ X and the transformed sample is in ∪ m X m .
The method attempts to estimate the following conditional probability:
[Equation]
Where the second equality follows by design of the training set, and where every training sample is transformed exactly once by each transformation leading to equal priors.
For anomalous data x ∈ R L \ X , by construction of the subspace, if the transformations T are oneto-one, it follows that the transformed sample does not fall in the appropriate subspace: T ( x, m ) ∈
R L \ X m .
GEOM uses P ( m | T ( x, m )) as a score for determining if x is anomalous i.e. that x ∈
R L \ X .
GEOM gives samples with low probabilities P ( m | T ( x, m )) high anomaly scores.
A signiﬁcant issue with this methodology, is that the learned classiﬁer P ( m (cid:48) | T ( x, m )) is only valid for samples x ∈ X which were found in the training set.
For x ∈ R L \ X we should in fact have
P ( T ( x, m ) ∈
X m (cid:48) )
=
for all m = 1 ..M (as the transformed x is not in any of the subsets).
This makes the anomaly score P ( m (cid:48) | T ( x, m )) have very high variance for anomalies.
One way to overcome this issue is by using examples of anomalies x a and training P ( m | T ( x, m ))
=
M on anomalous data.
This corresponds to the supervised scenario and was recently introduced as
Outlier Exposure (Hendrycks et al., 2018).
Although getting such supervision is possible for some image tasks (where large external datasets can be used) this is not possible in the general case e.g. for tabular data which exhibits much more variation between datasets.
D ISTANCE -B ASED M ULTIPLE T RANSFORMATION C LASSIFICATION
We propose a novel method to overcome the generalization issues highlighted in the previous section by using ideas from open-set classiﬁcation (Bendale & Boult, 2016).
Our approach uniﬁes one-class and transformation-based classiﬁcation methods.
Similarly to GEOM, we transform X to X 1 ..X M .
We learn a feature extractor f ( x ) using a neural network, which maps the original input data into a feature representation.
Similarly to deep OC methods, we model each subspace X m mapped to the feature space { f ( x ) | x ∈ X m } as a sphere with center c m .
The probability of data point x after transformation m is parameterized by P ( T ( x, m ) ∈ X (cid:48) m )
= 1 Z e − ( f ( T ( x,m ))
− c (cid:48) m ) 2 .
The classiﬁer predicting transformation m given a transformed point is therefore:
[Equation]
The centers c m are given by the average feature over the training set for every transformation i.e. c m = 1 N (cid:80) x ∈ X f ( T ( x, m )) .
One option is to directly learn f by optimizing cross-entropy between
P
[Equation]
m
[Equation]
| T ( x, m )) and the correct label on the normal training set.
In practice we obtained better results by training f using the center triplet loss (He et al., 2018), which learns supervised clusters with low intra-class variation, and high-inter-class variation by optimizing the following loss function (where s is a margin regularizing the distance between clusters):
[Equation]
Having learned a feature space in which the different transformation subspaces are well separated, we use the probability in Eq.
2 as a normality score.
However, for data far away from the normal distributions, the distances from the means will be large.
A small difference in distance will make the classiﬁer unreasonably certain of a particular transformation.
To add a general prior for uncertainty far from the training set, we add a small regularizing constant (cid:15) to the probability of each transformation.
This ensures equal probabilities for uncertain regions:
[Equation]
At test time we transform each sample by the M transformations.
By assuming independence between transformations, the probability that x is normal (i.e. x ∈ X ) is the product of the probabilities that all transformed samples are in their respective subspace.
For log-probabilities the total score is given by:
[Equation]
The score computes the degree of anomaly of each sample.
Higher scores indicate a more anomalous sample.
Algorithm
Input: Normal training data , x 2 ...x N
Transformations T ( , 1) , T ( , 2) ...
T ( , M )
Output: Feature extractor f , centers c 1 , c 2 ...
c M
T ( x i , 1) , T ( x i , 2) ...
T ( x i , M ) ← x i // Transform each sample by all transformations 1 to M
Find f, c 1 , c 2 ...
c M that optimize the triplet loss in Eq.
Algorithm
Output
T ( x, 1) , T ( x, 2) ...
T ( x, M ) ← x // Transform test sample by all transformations 1 to M
P ( m | T ( x, m ))
← f ( T ( x, m )) , c 1 , c 2 ...
c M // Likelihood of predicting the correct transformation (Eq. 4)
Score ( x ) ← P (1 | T ( x, 1)) , P (2 | T ( x, 2)) ...
P ( M | T ( x, M )) //
Aggregate probabilities to compute anomaly score (Eq. 5)
P ARAMETERIZING THE S ET OF T RANSFORMATIONS
Geometric transformations have been used previously for unsupervised feature learning by Gidaris et al.
(2018) as well as by GEOM (Golan & El-Yaniv, 2018) for classiﬁcation-based anomaly detection.
This set of transformations is hand-crafted to work well with convolutional neural networks
(CNNs) which greatly beneﬁt from preserving neighborhood between pixels.
This is however not a requirement for fully-connected networks.
Anomaly detection often deals with non-image datasets e.g. tabular data.
Tabular data is very commonly used on the internet e.g. for cyber security or online advertising.
Such data consists of both discrete and continuous attributes with no particular neighborhoods or order.
The data is onedimensional and rotations do not naturally generalize to it.
To allow transformation-based methods to work on general data types, we therefore need to extend the class of transformations.
We propose to generalize the set of transformations to the class of afﬁne transformations (where we have a total of M transformations):
[Equation]
It is easy to verify that all geometric transformations in Golan & El-Yaniv (2018) (rotation by a multiple of 90 degrees, ﬂips and translations) are a special case of this class ( x in this case is the set of image pixels written as a vector).
The afﬁne class is however much more general than mere permutations, and allows for dimensionality reduction, non-distance preservation and random transformation by sampling W , b from a random distribution.
Apart from reduced variance across different dataset types where no apriori knowledge on the correct transformation classes exists, random transformations are important for avoiding adversarial examples.
Assume an adversary wishes to change the label of a particular sample from anomalous to normal or vice versa.
This is the same as requiring that ˜ P ( m (cid:48) | T ( x, m )) has low or high probability for m (cid:48)
= m .
If T is chosen deterministically, the adversary may create adversarial examples against the known class of transformations (even if the exact network parameters are unknown).
Conversely, if T is unknown, the adversary must create adversarial examples that generalize across different transformations, which reduces the effectiveness of the attack.
To summarize, generalizing the set of transformations to the afﬁne class allows us to: generalize to non-image data, use an unlimited number of transformations and choose transformations randomly which reduces variance and defends against adversarial examples.
E XPERIMENTS
We perform experiments to validate the effectiveness of our distance-based approach and the performance of the general class of transformations we introduced for non-image data.
5. I MAGE E XPERIMENTS
Cifar10: To evaluate the performance of our method, we perform experiments on the Cifar dataset.
We use the same architecture and parameter choices of Golan & El-Yaniv (2018), with our distance-based approach.
We use the standard protocol of training on all training images of a single digit and testing on all test images.
Results are reported in terms of AUC.
In our method, we used a margin of s = 0 .
1 (we also run GOAD with s = 1 , shown in the appendix).
Similarly to He et al.
(2018), to stabilize training, we added a softmax + cross entropy loss, as well as Lnorm regularization for the extracted features f ( x ) .
We compare our method with the deep oneclass method of Ruff et al.
(2018) as well as Golan & El-Yaniv (2018) without and with Dirichlet weighting.
We believe the correct comparison is without Dirichlet post-processing, as we also do not use it in our method.
Our distance based approach outperforms the SOTA approach by Golan & El-Yaniv (2018), both with and without Dirichlet (which seems to improve performance on a few classes).
This gives evidence for the importance of considering the generalization behavior outside the normal region used in training.
Note that we used the same geometric transformations as Golan & El-Yaniv (2018).
Random afﬁne matrices did not perform competitively as they are not pixel order preserving, this information is effectively used by CNNs and removing this information hurts performance.
This is a special property of CNN architectures and image/time series data.
As a rule of thumb, fully-connected networks are not pixel order preserving and can fully utilize random afﬁne matrices.
[Table] [Table] FasionMNIST: In Tab. 2, we present a comparison between our method (GOAD) and the strongest baseline methods (Deep SVDD and GEOM) on the FashionMNIST dataset.
We used exactly the same setting as Golan & El-Yaniv (2018). GOAD was run with s = 1 .
OCSVM and GEOM with Dirichlet were copied from their paper.
We run their method without Dirichlet and presented it in the table (we veriﬁed the implementation by running their code with Dirichlet and replicated the numbers in the paper). It appears that GEOM is quite dependent on Dirichlet for this dataset, whereas we do not use it at all. GOAD outperforms all the baseline methods. Adversarial Robustness: Let us assume an attack model where the attacker knows the architecture and the normal training data and is trying to minimally modify anomalies to look normal.
We examine the merits of two settings i) the adversary knows the transformations used (non-random) ii) the adversary uses another set of transformations.
To measure the beneﬁt of the randomized transformations, we train three networks A, B, C. Networks A and B use exactly the same transformations but random parameter initialization prior to training.
Network C is trained using other randomly selected transformations.
The adversary creates adversarial examples using PGD (Madry et al., 2017) based on network A (making anomalies appear like normal data).
On Cifar10, we randomly selected 8 transformations from the full set of 72 for A and B , another randomly selected 8 transformations are used for C .
We measure the increase of false classiﬁcation rate on the adversarial examples using the three networks.
The average increase in performance of classifying transformation correctly on anomalies (causing lower anomaly scores) on the original network A was 12 .
8% , the transfer performance for B causes an increase by 5 . 0% on network B which shared the same set of transformation, and 3% on network C that used other rotations.
This shows the beneﬁts of using random transformations.
[Table]
[Table]
T ABULAR D ATA E XPERIMENTS
Datasets: We evaluate on small-scale medical datasets Arrhythmia, Thyroid as well as large-scale cyber intrusion detection datasets KDD and KDDRev.
Our conﬁguration follows that of Zong et al.
Categorical attributes are encoded as one-hot vectors.
For completeness the datasets are described in the appendix A.2.
We train all compared methods on 50% of the normal data.
The methods are evaluated on 50% of the normal data as well as all the anomalies.
Baseline methods: The baseline methods evaluated are: One-Class SVM (OC-SVM, Scholkopf et al.
(2000)), End-to-End Autoencoder (E2E-AE), Local Outlier Factor (LOF, Breunig et al.
We also evaluated deep distributional method DAGMM (Zong et al., 2018), choosing their strongest variant. To compare against ensemble methods e.g. Chen et al.
, we implemented the Feature Bagging Autoencoder (FB-AE) with autoencoders as the base classiﬁer, feature bagging as the source of randomization, and average reconstruction error as the anomaly score.
OC-SVM, E2E-AE and DAGMM results are directly taken from those reported by Zong et al.
LOF and FB-AE were computed by us.
Implementation of GOAD: We randomly sampled transformation matrices using the normal distribution for each element.
Each matrix has dimensionality L × r , where L is the data dimension and r is a reduced dimension.
For Arryhthmia and Thyroid we used r = 32 , for KDD and KDDrev we used r = 128 and r = 64 respectively, the latter due to high memory requirements.
We usedtasks for all datasets apart from KDD ( 64 ) due to high memory requirements.
We set the bias term to 0 .
For C we used fully-connected hidden layers and leaky-ReLU activations ( 8 hidden nodes for the small datasets, 128 and 32 for KDDRev and KDD).
We optimized using ADAM with a learning rate of 0 . 001 .
Similarly to He et al. (2018), to stabilize the triplet center loss training, we added a softmax + cross entropy loss. We repeated the large-scale experiments 5 times, and the small scale GOAD experiments 500 times (due to the high variance).
We report the mean and standard deviation ( σ ).
Following the protocol in Zong et al.
(2018), the decision threshold value is chosen to result in the correct number of anomalies e.g. if the test set contains N a anomalies, the threshold is selected so that the highest N a scoring examples are classiﬁed as anomalies.
True positives and negatives are evaluated in the usual way.
Some experiments copied from other papers did not measure standard variation and we kept the relevant cell blank.
Results
Arrhythmia: The Arrhythmia dataset was the smallest examined.
A quantitative comparison on this dataset can be seen in Tab. 3.
OC-SVM and DAGMM performed reasonably well.
Our method is comparable to FB-AE.
A linear classiﬁer C performed better than deeper networks (which suffered from overﬁtting).
Early stopping after a single epoch generated the best results.
Thyroid: Thyroid is a small dataset, with a low anomaly to normal ratio and low feature dimensionality.
A quantitative comparison on this dataset can be seen in Tab. 3.
Most baselines performed about equally well, probably due to the low dimensionality.
On this dataset, we also found that early stopping after a single epoch gave the best results.
The best results on this dataset, were obtained with a linear classiﬁer.
Our method is comparable to FB-AE and beat all other baselines by a wide margin.
KDDCUP99:
The UCI KDD 10% dataset is the largest dataset examined.
A quantitative comparison on this dataset can be seen in Tab. 3.
The strongest baselines are FB-AE and DAGMM.
Our method signiﬁcantly outperformed all baselines.
We found that large datasets have different dynamics from very small datasets.
On this dataset, deep networks performed the best.
We also, did not need early stopping.
The results are reported after 25 epochs.
KDD-Rev: The KDD-Rev dataset is a large dataset, but smaller than KDDCUP99 dataset.
A quantitative comparison on this dataset can be seen in Tab. 3.
Similarly to KDDCUP99, the best baselines are FB-AE and DAGMM, where FB-AE signiﬁcantly outperforms DAGMM.
Our method signiﬁcantly outperformed all baselines.
Due to the size of the dataset, we did not need early stopping.
The results are reported after 25 epochs.
Adversarial Robustness: Due to the large number of transformations and relatively small networks, adversarial examples are less of a problem for tabular data.
PGD generally failed to obtain adversarial examples on these datasets.
On KDD, transformation classiﬁcation accuracy on anomalies was increased by 3 .
7% for the network the adversarial examples were trained on, 1 . 3% when transferring to the network with the same transformation and only 0 .
2% on the network with other randomly selected transformations.
This again shows increased adversarial robustness due to random transformations.
Further Analysis
Contaminated Data: This paper deals with the semi-supervised scenario i.e. when the training dataset contains only normal data.
In some scenarios, such data might not be available but instead we might have a training dataset that contains a small percentage of anomalies.
To evaluate the robustness of our method to this unsupervised scenario, we analysed the KDDCUP99 dataset, when
X % of the training data is anomalous.
To prepare the data, we used the same normal training data as before and added further anomalous examples.
The test data consists of the same proportions as before.
The results are shown in Fig.
Our method signiﬁcantly outperforms DAGMM for all impurity values, and degrades more graceful than the baseline.
This attests to the effectiveness of our approach.
Results for the other datasets are presented in Fig.
3, showing similar robustness to contamination.
Number of Tasks: One of the advantages of GOAD, is the ability to generate any number of tasks.
We present the anomaly detection performance on the KDD-Rev dataset with different numbers of tasks in Fig.
We note that a small number of tasks (less than 16 ) leads to poor results.
Fromtasks, the accuracy remains stable.
We found that on the smaller datasets (Thyroid, Arrhythmia) using a larger number of transformations continued to reduce F 1 score variance between differently initialized runs (Fig. 2).
D ISCUSSION
Openset vs. Softmax: The openset-based classiﬁcation presented by GOAD resulted in performance improvement over the closed-set softmax approach on Cifar10 and FasionMNIST.
In our experiments, it has also improved performance in KDDRev.
Arrhythmia and Thyroid were comparable.
As a negative result, performance of softmax was better on KDD ( F 1 = 0 . 99 ).
Choosing the margin parameter s : GOAD is not particularly sensitive to the choice of margin parameter s , although choosing s that is too small might cause some instability.
We used a ﬁxed value of s = 1 in our experiments, and recommend this value as a starting point.
Other transformations: GOAD can also work with other types of transformations such as rotations or permutations for tabular data.
In our experiments, we observed that these transformation types perform comparably but a little worse than afﬁne transformations.
Unsupervised training: Although most of our results are semi-supervised i.e. assume that no anomalies exist in the training set, we presented results showing that our method is more robust than strong baselines to a small percentage of anomalies in the training set.
We further presented results in other datasets showing that our method degrades gracefully with a small amount of contamination.
Our method might therefore be considered in the unsupervised settings.
Deep vs. shallow classiﬁers: Our experiments show that for large datasets deep networks are beneﬁcial (particularly for the full KDDCUP99), but are not needed for smaller datasets (indicating that deep learning has not beneﬁted the smaller datasets).
For performance critical operations, our approach may be used in a linear setting.
This may also aid future theoretical analysis of our method.
C ONCLUSION
In this paper, we presented a method for detecting anomalies for general data.
This was achieved by training a classiﬁer on a set of random auxiliary tasks.
Our method does not require knowledge of the data domain, and we are able to generate an arbitrary number of random tasks.
Our method signiﬁcantly improve over the state-of-the art
A A PPENDIX A. I MAGE E XPERIMENTS
Sensitive to margin s : We run Cifar10 experiments with s = 0 .
1 and s = 1 and presented the results in Fig.
The results were not affected much by the margin parameter.
This is in-line with the rest of our empirical observations that GOAD is not very sensitive to the margin parameter.
[Table]
T ABULAR D ATASETS
Following the evaluation protocol of Zong et al.
[Table]
4 datasets are used in this comparison:
Arrhythmia: A cardiology dataset from the UCI repository (Asuncion & Newman, 2007) containing attributes related to the diagnosis of cardiac arrhythmia in patients.
The datasets consists ofclasses: class 1 are normal patients, 2 - 15 contain different arrhythmia conditions [Table] and class 16 contains undiagnosed cases. Following the protocol established by ODDS (Rayana, 2016), the smallest classes: 3 , 4 , 5 , 7 , 8 , 9 , 14 , 15 are taken to be anomalous and the rest normal.
Also following ODDS, the categorical attributes are dropped, the ﬁnal attributes
total
Thyroid: A medical dataset from the UCI repository (Asuncion & Newman, 2007), containing attributes related to whether a patient is hyperthyroid.
Following ODDS (Rayana, 2016), from theclasses of the dataset, we designate hyperfunction as the anomalous class and the rest as normal.
Also following ODDS only the 6 continuous attributes are used.
KDD
The KDD Intrusion Detection dataset was created by an extensive simulation of a US Air
Force LAN network. The dataset consists of the normal and 4 simulated attack types: denial of service, unauthorized access from a remote machine, unauthorized access from local superuser and probing. The dataset consists of around 5 million TCP connection records. Following the evaluation protocol in Zong et al.
(2018), we use the UCI KDD 10% dataset, which is a subsampled version of the original dataset.
The dataset contains 41 different attributes.
34 are continuous and 7 are categorical.
Following Zong et al.
(2018), we encode the categorical attributes using 1 -hot encoding.
Following Zong et al.
(2018), we evaluate two different settings for the KDD dataset:
KDDCUP99: In this conﬁguration we use the entire UCI 10% dataset. As the non-attack class consists of only 20% of the dataset, it is treated as the anomaly in this case, while attacks are treated as normal. KDDCUP99-Rev: To better correspond to the actual use-case, in which the non-attack scenario is normal and attacks are anomalous, Zong et al.
(2018) also evaluate on the reverse conﬁguration, in which the attack data is sub-sampled to consist of 25% of the number of non-attack samples.
The attack data is in this case designated as anomalous (the reverse of the KDDCUP99 dataset). In all the above datasets, the methods are trained on 50% of the normal data.
The methods are evaluated on 50% of the normal data as well as all the anomalies.
A. N UMBER OF TASKS
We provide plots of the number of auxiliary tasks vs. the anomaly detection accuracy (measured by
F 1 ) for all datasets.
The results are presented in Fig.
Performance increases rapidly up to a certain number of tasks (around 16).
Afterwards more tasks reduce the variance of F 1 scores between runs.
[Figure]
A. C ONTAMINATION EXPERIMENTS
We conduct contamination experiments for 3 datasets.
Thyroid was omitted due to not having a sufﬁcient number of anomalies.
The protocol is different than that of KDDRev as we do not have unused anomalies for contamination.
Instead, we split the anomalies into train and test.
Train anomalies are used for contamination, test anomalies are used for evaluation.
As DAGMM did not present results for the other datasets, we only present GOAD.
GOAD was reasonably robust to contamination on KDD, KDDRev and Arrhythmia.
The results are presented in Fig.