Classification-Based Anomaly Detection for General Data
Authors: Liron Bergman
Yedid Hoshen
Venue:  (2020)
Keywords: anomaly detection

Abstract: Anomaly detection, finding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artificial intelligence. Recently, classification-based methods were shown to achieve superior results on this task. In this work, we present a unifying view and propose an open-set method, GOAD, to relax current generalization assumptions. Furthermore, we extend the applicability of transformation-based methods to non-image data using random affine transformations. Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types. The strong performance of our method is extensively validated on multiple datasets from different domains.  

Published as a conference paper at ICLR 2020
CLASSIFICATION-BASBH ANOMALY DETECTION FOR GENERAL DATA
Liron Bergman Yedid Hoshen
School of Computer Science and Engineering The Hebrew University of Jerusalem, Israel
ABSTRACT
Anomaly detection, finding patterns that substantially deviate from those seen pre- viously, is one of the fundamental problems of artificial intelligence. Recently, classification based methods were shown to achieve superior results on this task. In this work, we present a unifying view and propose an open-set method, GOAD, to relax current generalization assumptions. Furthermore, we extend the appli- cability of transformation-basse transformations. Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types. The strong performance of our method is exten- sively validated multiple datasets from different domains.
methods to non-image data using random affine
INTRODUCTION
Detecting anomalies in perceived data is a key ability for humans and for artificial intelligence. Hu- mans often detect anomalies to give early indications of danger or discover unique opportunities. Anomaly detection systems are being used by artificial intelligence to discover credit card fraud, for detecting cyber intrusion, alert predictive maintenance of industrial equipment and for discovering attractive stock market opportunities. The typical anomaly detection setting is a one class classi- fication task, where the objective is classify data as normal or anomalous. The importance of the task stems from being able to raise an alarm when detecting different pattern from those seen in the past, therefore triggering further inspection. This is fundamentally different from supervised learning tasks, in which examples of all data classes are observed.
There are different possible scenarios for anomaly detection methods. In supervised anomaly de- tection, we are given training examples of normal and anomalous patterns. This scenario can be quite well specified, however obtaining such supervision may not be possible. For example in cyber security settings, we will not have supervised examples of new, unknown computer viruses making supervised training difficult. On the other extreme, fully unsupervised anomaly detection, obtains stream of data containing normal and anomalous patterns and attempts to detect the anomalous data. In this work we deal with the semi-supervised scenario. In this setting, we have a training set of normal examples (which contains no anomalies). After training the anomaly detector, we detect anomalies in the test data, containing both normal and anomalous examples. This supervision is easy to obtain in many practical settings and is less difficult than the fully-unsupervised case.
Many anomaly detection methods have been proposed over the last few decades. They can be broadly classified into reconstruction and statistically based methods. Recently, deep learning meth- ods based on classification have achieved superior results. Most semi-supervised classification- based methods attempt to solve anomaly detection directly, despite only having normal training data. One example is: Deep-SVDD (Ruff et al. 2018) one-class classification using learned deep space. Another type of classification-bass more classification--br based auxiliary tasks on the normal training data, and this is shown to be useful for solving anomaly detection, the task of interest e.g. (Golan & El- Yaniv| 2018). Self-supervised dassification-basel methods have been proposed with the object of image anomaly detection, but we show that by generalizing the class of transformations they can apply to all data types.
methods is self-supervised i.e. methods that solve one or
In this paper, introduce novel technique, GOAD, for anomaly detection which unifies current ofate-of-thh-ann methods that use normal training data only and are based on classification. Our method first transforms the data into M subspaces, and learns feature space such that inter-class

---

Published as a conference paper at ICLR 2020
separation is larger than intra-class separation. For the learned features, the distance from the cluster center is correlated with the likelihood of anomaly. We use this criterion to determine if new data point is normal anomalous. We also generalize the class of transformation functions to include affine transformation which allows our method to generalize to non-image data. This is significant as tabular data is probably the most important for applications of anomaly detection. Our method is evaluated on anomaly detection on image and tabular datasets (cyber security and medical) and is shown to significantly improve over the state-of-the-ar
1.1 PREVIOUS WORKS
Anomaly detection methods can be generally divided into the following categories:
Reconstruction Methods: Some of the most common anomaly detection methods are reconstruction- based. The general idea behind such methods is that every normal sample should be reconstructed accurately using a limited set of basis functions, whereas anomalous data should suffer from larger reconstruction costs. The choice of features, basis and loss functions differentiates between the different methods. Some of the earliest methods use: nearest neighbors (Eskin et al. 2002), low-rank 2011) or K-means (Hartigan & Wong. 1979) as the reconstruction basis. Most recently, neural networks were used (Sakurada & Yairi 2014 Xia et al. 2015) for learning deep basis functions for reconstruction. Another set of recent methods (Schlegl et al.) 2017 Deecke et al. (2018) use GANs to learn reconstruction basis function. GANs suffer from mode-collapse and are difficult to invert, which limits the performance of such methods.
PCA (Jolliffe 2011 Candès
Distributional Methods: Another set of commonly used methods are distribution-ben based. The main theme in such methods to model the distribution of normal data. The expectation is that anomalous test data will have low likelihood under the probabilistic model while normal data will have higher likelihoods. Methods differ in the features used to describe the data and the probabilistic model used to estimate the normal distribution. Some early methods used Gaussian Gaussian mixture models. Such models will only work if the data under the selected feature space satisfies the prob- abilistic assumptions implicied by the model. Another set of methods used non- non-parametric density estimate methods such as kernel density estimate (Parzen 1962). Recently, deep learning methods (autoencoders or variational autoencoders) were used to learn deep features which are sometimes easier to model than raw features (Yang et al., 2017). DAGMM introduced by Zong et al. (2018) learn the probabilistic model jointly with the deep features therefore shaping the features space to better conform with the probabilistic assumption.
Classification- Basedd Methods: Another paradigm for anomaly detection is separation between space regions containing normal data from all other regions. An example of such approach is One-Class SVM (Scholkopf et al. 2000), which trains a classifier to perform this separation. Learning good feature space for performing such separation is performed both by the classic kernel methods as well as by the recent deep learning approach (Ruff al.] 2018). One of the main challenges in unsupervised (or s learning is providing an objective for learning features that are relevant to the task of interest. One method for learning good representations in a self-supervised way is by training neural network to solve at least very inexpensive. Auxiliary tasks for learning high-quality image features include: video frame prediction (Mathieu et al. (0166), image colorization (Zhang et al., 2016 Larsson al.. 2016), puzzle solving (Noroozi & Favaro (016)) predicting the correct order of random permuted image patches. Recently, Gidaris al. (2018) used set of image processing transformations (rotation by 0, 90, 180, 270 degrees around the image axis, and predicted the true image orientation has been used to learn high-quality image features. Golan & Yaniv (2018), have used similar image-pro task prediction for detecting anomalies in images. This method has shown good performance on detecting images from anomalous classes. In this work, we overcome some of the limitations of previous classification-bass methods and extend their applicability of self-supervised methods to general data types. We also show that our method is more robust to adversarial attacks.
auxiliary task for which obtaining data is free or
CLASSIFICATION BASED ANOMALY DETECTION
Classification based methods have dominated supervised anomaly detection. In this section we will analyse semi-supervised classification-hase methods:
2

---

Published as a conference paper at ICLR 2020
Let us assume all data lies in space R (where L is the data dimension). Normal data lie in subspace X C RR. We assume that all anomalies lie outside X. To detect anomalies, we would therefore like to build a classifier C, such that C(z) = 1 if x E X and C(x) 0 x RA
One-class classification methods attempt to learn C directly as P(r E X). Classical approaches have learned classifier either in input space in kernel space. Recently, Deep-SVDD (Ruff et al. 2018) learned end-to-end to i) transform the data to an isotropic feature space f(x) ii) fit the minimal hypersphere of radius R and center eo around the features of the normal training data. Test data is classified as anomalous if the following normality score is positive: [I coll R².
Learning an effective feature space not simple task, as the trivial solution of f(x)
results in the smallest hypersphere, various tricks are used to avoid this possibility.
Geometric-transteee transoommttooono classification (GEOM), proposed by Golan & El- Yaniv (2018) first trans- forms the normal data subspace X into M subspaces X This is done by transforming each image x E X using M different geometric transformations (rotation, reflection, translation) into T(x, 1)xT(x M). Although these transformations are image specific, we will later extend the class of transformations to all affine transformations making this applicable to non-image data. They set an auxiliary task of learning classifier able to predict the transformation label m given transformed data point Txx m). As the training set consists of normal data only, each sample is x E X and the transformed sample is in Um The method attempts to estimate the following conditional probability:
P(m'IT(x,, P(T(a, E Xmx P((m') P(T(s, ((T(a,m) m) E X(m) Σn ((T(x,m) Xm)
P(T m) E Xm))
Where the second equality follows by design of the training set, and where every training sample is transformed exactly once by each transformation leading to equal priors.
For anomalous data X by construction of the subspace, if the transformations T are to-one, it follows that the transformed sample does not fall in the appropriate subspace: T(x, m) e Rd X m- GEOM uses P(m (Tx, m)) score for determining if is anomalous i.e. that x RL X. GEOM gives samples with low probabilities [mm|T(xr m)) high anomaly scores.
A significant issue with this methodology, is that the learned classifier P(m'(TT(z, m)) is only valid for samples x E which were found in the training set. For & E RL we should in fact have for all 1..M (as the transformed makes the anomaly score P(m' [T(,, m)) have very high variance for anomalies.
P(T(a, m) E xmm)>=
any of the subsets). This
One way overcome this issue is by using examples of anomalies va and training P(m]]T(a m)) 1
M on anomalous data. This corresponds to the supervised scenario and was recently introduced as Outlier Exposure (Hendrycks al. (0188). Although getting such supervision possible for some image tasks (where large external datasets can be used) this is not possible in the general case e.g. for tabular data which exhibits much more variation between datasets.
DISTANCE-BASER MULTIPLE TRANSFORMATION CLASSIFICATION
We propose a novel method to overcome the generalization issues highlighted in the previous section by using ideas from open-set classification (Bendale & Boult 2016). Our approach unifies one-class and transformation-banl We learn a feature extractor f(x) using a neural network, which maps the original input data into feature representation. Similarly to deep OC methods, we model each subspace Xm mapped to the feature space {x))) E X as a sphere with center Cm- The probability of data point x after transformation m is parameterized by P(T(xr, m) E X²,) = ((((((((,))))d.)) The classifier
classification methods. Similarly to GEOM, we transform X to XI..XA, m-
predicting transformation m given a transformed point therefore:
C ([
P(m'IT(x, m))
(2)
Σm - -
The centers Cm are given by the average feature over the training set for every transformation i.c Gm Exex [(T(x, One option is to directly learn f by optimizing cross-entropy between
3

---

Published as a conference paper at ICLR 2020
P(m'IIT (xx, m)) and the correct label the normal training set. In practice obtained better results by training f using the center triplet loss (He et al, 2018). which learns supervised clus- ters with low intra-class variation, and high-inter-class variation by optimizing the following loss function (where is a margin regularizing the distance between clusters):
L Σ (a m)) - Cm 112 + minm' fml (T m))
Cml cm/ll" 1)
(3)
Having learned a feature space in which the different transformation subspaces are well separated, use the probability in Eq. 2 as normality score. However, for data far away from the normal distributions, the distances from the means will be large. A small difference in distance will make the classifier unreasonably certain of particular transformation. To add general prior for uncer- tainty far from the training set, we add a small regularizing constant e to the probability of each transformation. This ensures equal probabilities for uncertain regions:
c-11(T((em)))mm 112 +
P(m'IT(xx,m)) = Σn
(4)
- (
+ M E
At test time we transform each sample by the M transformations. By assuming independence be- tween transformations, the probability that x is normal (i.e. x E X) the product of the probabilities that all transformed samples are in their respective subspace. For log-probabilities the total score is given by:
Score(x)= - log P(x e xX==
=
Σ m
log P(T(z, E Xm)= = Σ log log log P(m[[(a,, (5)
m
The score computes the degree of anomaly of each sample. Higher scores indicate more anomalous sample.
Algorithm GOAD: Training Algorithm
Input:
Normal training data 21, .2...EN
Transformations (. 1), T(, 7
Output: Feature extractor f. centers C1, C2...CM
T(a,, 2I..T .) +
// Transform each sample by all transformations M
Find f, f,c C2..eM that optimize the triplet loss in Eq.
Algorithm GOAD: Evaluation Algorithm
Input: Test sample: 2, feature extractor:
centers:
C1 C2...M, transformations:
T(, 1), T,, M)
Output: Score(x)
(), M)+ +
// Transform test sample by all transformations 1 to M
P(m[T(a,m) m)) + f(T (T( (1) C2....CM // Likelihood of predicting the correct transformation (Eq.
Score(z) + P(IT((z, 1)), ((),
// Aggregate probabilities
compute anomaly score (Eq.
4 PARAMETERIZING THE SET OF TRANSFORMATIONS
Geometric transformations have been used previously for unsupervised feature learning by Gidaris iet al. (2018) as well as by GEOM (Golan & EI- Yaniv 2018) for classification based anomaly de- Tection. This set of transformations hand-crafted work well with convolutional neural networks

---

Published as a conference paper at ICLR 2020
(CNNs) which greatly benefit from preserving neighborhood between pixels. This is however not requirement for fully-connected networks.
Anomaly detection often deals with non-image datasets e.g. tabular data. Tabular data is very commonly used on the internet c.g. for cyber security or online advertising. Such data consists of order. The data is one- dimensional and rotations do not naturally generalize to it. To allow transformation-based methods work on general data types, we therefore need to extend the class of transformations.
both discrete and continuous attributes with particular neighborhoods
We propose to generalize the set of transformations to the class of affine transformations (where we have total of M transformations):
T(x, m) = Wm + bm
(6)
is easy to verify that all geometric transformations in Golan & Yaniv (2018) (rotation by a multiple of 90 degrees, flips and translations) are special case of this class (æ in this is the set of image pixels written as a vector). The affine class is however much more general than mere permutations, and allows for dimensionality reduction, non-distance preservation and random trans- formation by sampling W, from a random distribution.
Apart from reduced variance across different dataset types where apriori knowledge the cor- rect transformation classes exists, random transformations are important for avoiding adversarial examples. Assume adversary wishes to change the label of particular sample from anomalous normal vice versa. This is the same requiring that P(m' m)) has low or high proba- bility for m' = m. If I' is chosen deterministically, the adversary may create adversarial examples against the known class of transformations (even if the exact network parameters unknown). Conversely, if T is unknown, the adversary must create adversarial examples that generalize across different transformations, which reduces the effectiveness of the attack.
To summarize, generalizing the set of transformations to the affine class allows us to: generalize to non-image data, use an unlimited number transformations and choose transformations randomly which reduces variance and defends against adversarial examples.
5 EXPERIMENTS
We perform experiments to validate the effectiveness of our distance- based approach and the per- formance of the general class of transformations we introduced for non-image data.
5.1 IMAGE EXPERIMENTS
Cifar10: To evaluate the performance of our method, we perform experiments on the Cifar10 dataset. We use the same architecture and parameter choices of Golan & El- (2018), with distance-based approach. We use the standard protocol of training single digit and testing on all test images. Results are reported in terms of AUC. In our method, used margin of = 0.1 (we also run GOAD with 1, shown in the appendix). Similarly to He et al. (0188). to stabilize training, we added a softmax + cross entropy loss, as well as Le norm regularization for the extracted features f(x)). We compare our method with the deep one- class method of Ruff et al. (0018) as well as Golan & El-Yaniv (2018) without and with Dirichlet weighting. We believe the correct comparison is without Dirichlet post-processing as we also do not use it in our method. Our distance based approach outperforms the SOTA approach by Golan El- Yaniv (208)) both with and without Dirichlet (which seems to improve performance on a few classes). This gives evidence for the importance of considering the generalization behavior outside the normal region used in training. Note that we used the same geometric transformations as Golan & Ylniv (2018). Random affine matrices did not perform competitively they are not pixel order preserving, this information is effectively used by CNNs and removing this information hurts performance. This is special property of CNN architectures and image/time series data. As rule of thumb, fully-connected networks are not pixel order preserving and can fully utilize random affine matrices.
all training images of
5

---

Published as a conference paper at ICLR 2020
Table 1: Anomaly Detection Accuracy on Cifarlo (ROC-AUC %)
Class
Method
Deep-SVDD
GEOM (no Dirichlet)
GEOM (w. Dirichlet)
Ours
0
61.7 + 1.3
76.0 + 0.8
74.7 + 0.4
77.2 + 0.6
65.9 + 0.7
83.0 + 1.6
95.7 0.0
96.7 0.2
2
50.8 0.3
79.5 0.7
78.1 = 0.4
83.3 + 1.4
3
59.1
71.4
72.4 0.5
77.7 0.7
4
60.9
$
87.8 = 0.2
87.8 =
5
65.7
84.0
87.8 0.1
87.8 0.6
6
67.7 0.8
83,4 0.5
90.0 0.6
7
67.3 + 0.3
89.3 + 0.5
95.5 0.1
96.1 033
8
75.9 + 0.4
88.6 0.6
93.3 0.0
93.8 0.9
9
73.1 0.4
82.4 0.7
91.3 0.1
92.0 0.6
Average
64.8
81.6
86.0
88.2
Table 2: Anomaly Detection Accuracy on FashionMNIST (ROC-AUC %)
Class
Method
Deep-SVDD
GEOM (no Dirichlet)
GEOM (w. Dirichlet)
Ours
98.2
77.8 + 5.9
99.4 + 0.0
94.1 0.9
90.3
79.1 + 16.3
97.6 + 0.1
98.5
90.7
80.8 + 6.9
91.1 + 0.2
90.8 +
94.2
79.2 + 9.1
89.9 + 0.4
91.6 0.9
4
89.4
77.8 3.3
92.1 + 0.0
91.4 0.3
91.8
58.0 29.4
29.4
93.4 0.9
94.8
6
83.4
7
83.3
8.
98.8
87.4 11.4
98.9 0.1
97.9
8
91.9
84.6 + 5.6
90.8 + 0.1
98.9
99.0
99.5 0.0
99.2 + 0.0
99.2 03
Average
92.8
79.8
93.5
94.1
FasionMNIST: In Tab. we present a comparison between our method (GOAD) and the strongest baseline methods (Deep SVDD and GEOM) on the FashionMNIST dataset. We used exactly the same setting as Golan & El-Yaniv 2018). GOAD was run with with Dirichlet were copied from their paper. We run their method without Dirichlet and presented it in the table (we verified the implementation by running their code with Dirichlet and replicated the numbers in the paper). appears that GEOM is quite dependent on Dirichlet for this dataset, whereas do
1. OCSVM and GEOM
at all. GOAD outperforms all the baseline methods.
Adversarial Robustness: Let us assume an attack model where the attacker knows the architecture and the normal training data and is trying minimally modify anomalies to look normal. We exam- ine the merits of two settings i) the adversary knows the transformations used (non-random) ii) the adversary uses another set of transformations. To measure the benefit of the randomized transfor- mations, we train three networks A, B, C. Networks A and B use exactly the same transformations but random parameter initialization prior to training. Network is trained using other randomly se- lected transformations. The ad versary creates adversarial examples using PGD (Madry et al.) (017)) based on network (making anomalies appear like normal data). On Cifar10, we randomly selected 8 transformations from the full set of 72 for A and B, another randomly selected transformations are used for C. We measure the increase of false classification rate on the adversarial examples us- ing the three networks. The average increase in performance of classifying transformation correctly on anomalies (causing lower anomaly scores) on the original network A was 12.8%, the transfer performance for B causes increase by 5.0% network B which shared the same set of transfor- mation, and 3% on network C that used other rotations. This shows the benefits of using random transformations.

---

Published as a conference paper at ICLR 2020
Table 3: Anomaly Detection Accuracy (%)
Method
Dataset
Arrhythmia
Thyroid
KDD
KDDRev
Fj Score
Fi Score
a
Fi Score
Fi Score
a
OC-SVM
45.8
38.9
79.5
83.2
E2E-AE
45.9
11.8
0.3
74.5
LOF
50.0
0.0
52.7
0.0
83.8
5.2
81.6
3.6
DAGMM
49.8
47.8
93.7
93.8
FB-AE
51,5
1.6
75.0
0.8
92.7
0.3
95.9
0.4
GOAD(Ours)
52.0
2.3
74.5
1.1
98.4
0.2
98.9
0.3
5.2
TABULAR DATA EXPERIMENTS
Datasets: We evaluate small-scale medical datasets Arrhythmia, Thyroid as well as large-scale cyber intrusion detection datasets KDD and KDDRev. Our configuration follows that of Zong et al. 2018). Categorical attributes encoded one-hot vectors. For completeness the datasets described in the appendix A.2 We train all compared methods on 50% of the normal data. The methods are evaluated 50% of the normal data well as all the anomalies.
Baseline methods: The baseline methods evaluated are: One-Class SVM (OC-SVM, Scholloop et al. (2000)), End-to-End Autoencoder (E2E-AE), Local Outlier Factor (LOF, Breunig et (2000)). We also evaluated deep distributional method DAGMM (Zong et al. 2018), choosing their strongest variant. To compare against ensemble methods e.g. Chen al. (017)) we implemented the Fea- ture Bagging Autoencoder (FB-AE) with autoencoders as the base classifier, feature bagging as the source of randomization. and average reconstruction error the anomaly score. OC-SVM. E2E-AE and DAGMM results are directly taken from those reported by Zong et al. (2018). LOF and FB-AE were computed by us.
Implementation of GOAD: We randomly sampled transformation matrices using the normal distri- bution for each element. Each matrix has dimensionality r is reduced dimension. For Arryhthmia and Thyroid we used 32, for KDD and KDDrev we used r 128 and r = 64 respectively. the latter due to high memory requirements. We used 256 tasks for all datasets apart from KDD (64) due to high memory requirements. We set the bias term to O. For C' we used fully-connecte hidden layers and Ieaky- activations (8 hidden nodes for the small datasets, 128 and 32 for KDDRev and KDD). We optimized using ADAM with learning rate of 0.001. Similarly to He et al. (2018), to stabilize the triplet center loss training, we added a softmax cross entropy loss. We repeated the large-scale experiments times, and the small scale GOAD experiments 500 times (due to the high variance). We report the mean and standard deviation (a). Following the protocol in Zong al. (2018), the decision threshold value is chosen result in the correct number of anomalies e.g. if the test set contains N₂ anomalies, the threshold is selected so that the highest No scoring examples are classified as anomalies. True positives and negatives are evaluated in the usual way. Some experiments copied from other papers did not measure standard variation and we kept the relevant cell blank.
r, where the data dimension and
Results
Arrhythmia: The Arrhythmia dataset was the smallest examined. A quantitative comparison on this dataset can be seen in Tab. 3 OC-SVM and DAGMM performed reasonably well. Our method is comparable to FB-AE. linear classifier performed better than deeper networks (which suffered from overfitting). Early stopping after single epoch generated the best results.
Thyroid: Thyroid is small dataset, with low anomaly to normal ratio and low feature dimension- ality. A quantitative comparison this dataset can be seen in Tab. 3 Most baselines performed about equally well, probably due to the low dimensionality. On this dataset, we also found that early stopping after single epoch gave the best results. The best results on this dataset, were obtained with a linear classifier. Our method is comparable to FB-AE and beat all other basclines by a wide margin.

---

Published as a conference paper at ICLR 2020
KDOww
0.12
testrattrenn
0.00
1.2
1.5
2.0
2.5
3.5
35
4.0 45
30
225
Figure Left: Classification enror for our method and DAGMM as a function of percentage of the anomalous examples in the training set (on the KDDCUP99 dataset). Our method consistently outperforms the baseline. Right: Classification error as a function of the number of transforma- tions (on the KDDRev dataset). The error and instability decrease as function of the number of transformations. For both, lower is better.
KDDCUP99: The UCI KDD 10% dataset the largest dataset examined. A quantitative comparison on this dataset can be seen in Tab. The strongest baselines are FB-AE and DAGMM. Our method significantly outperformed all baselines. We found that large datasets have different dynamics from very small datasets. On this dataset, deep networks performed the best. We also, did not need early stopping. The results are reported after 25 epochs.
KDD-Rev: The KDD-Rev dataset a large dataset, but smaller than KDDCUP99 dataset. quanti- tative comparison on this dataset can be seen in Tab. 3 Similarly to KDDCUP99, the best baselines are FB-AE and DAGMM, where FB-AE significantly outperforms DAGMM. Our method signifi- cantly outperformed all baselines. Due to the size of the dataset, we did not need early stopping. The results are reported after 25 epochs.
Adversarial Robustness: Due to the large number of transformations and relatively small networks, adversarial examples are less of problem for tabular data. PGD generally failed to obtain adversar- ial examples these datasets. On KDD, transformation classification accuracy anomalies was increased by 3.7% for the network the adversarial examples were trained on, 1.3% when transfer- ring the network with the same transformation and only 0.2% on the network with other randomly selected transformations. This again shows increased adversarial robustness due to random transfor- mations.
Further Analysis
Contaminated Data: This paper deals with the semi-supervised scenario i.e. when the training dataset contains only normal data. In some scenarios, such data might not available but instead might have a training dataset that contains small percentage of anomalies. To evaluate the robustness of our method to this unsupervised scenario, we analysed the KDDCUP99 dataset, when X% of the training data anomalous. To prepare the data, we used the same normal training data as before and added further anomalous examples. The test data consists of the same proportions as before. The results are shown in Fig. [ Our method significantly outperforms DAGMM for all impurity values, and degrades more graceful than the baseline. This attests to the effectiveness of our approach. Results for the other datasets are presented in Fig. 3 showing similar robustness to contamination.
Number of Tasks: One of the advantages of GOAD, is the ability to generate any number of tasks. We present the anomaly detection performance on the KDD-Rev dataset with different numbers of tasks in Fig. [ We note that a small number of tasks (less than 16) leads to poor results. From 16 tasks, the accuracy remains stable. We found that the smaller datasets (Thyroid, Arrhythmia) using a larger number of transformations continued to reduce F; score variance between differently initialized runs (Fig. 2).
8

---

Published as a conference paper at ICLR 2020
6 DISCUSSION
Openset vs. Softmax: The openset-based classification presented by GOAD resulted in performance improvement over the closed-set softmax approach on Cifar10 and FasionMNIST. In our experi- ments, it has also improved performance in KDDRev. Arrhythmia and Thyroid were comparable. As negative result, performance of softmax was better on KDD (Fi 0.99).
Choosing the margin parameter GOAD is not particularly sensitive to the choice of margin parameter 8, although choosing that is too small might cause some instability. We used fixed value of s 1 in experiments, and recommend this value starting point.
Other transformations: GOAD can also work with other types transformations such as rotations or permutations for tabular data. In our experiments, observed that these transformation types perform comparably but a little worse than affine transformations.
Unsupervised training: Although most of our results are semi- sumn---seee i.c. assume that anoma- lies exist in the training set, we presented results showing that our method is more robust than strong baselines to a small percentage of anomalies in the training set. We further presented results in other datasets showing that our method degrades gracefully with a small amount contamination. Our method might therefore be considered in the unsupervised settings.
Deep vs. shallow classifiers: Our experiments show that for large datasets deep networks are ben- eficial (particularly for the full KDDCUP99), but are not needed for smaller datasets (indicating that deep learning has not benefited the smaller datasets). For performance critical operations, our approach may be used linear setting. This may also aid future theoretical analysis of our method.
7 CONCLUSION
In this paper, we presented method for detecting anomalies for general data. This was achieved by training classifier set of random auxiliary tasks. Our method does not require knowledge of the data domain, and we are able to generate an arbitrary number of random tasks. Our method significantly improve the state-of-the-art.
REFERENCES
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1563-1572, 2016.
Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. Lof: identifying density- based local outliers. In ACM sigmod record, volume 29, pp. 93-104. ACM, 2000.
Emmanuel J Candès, Xiaodong Li, Yi Ma, and John Wright. Robust principal component analysis? JACM, 2011.
Jinghui Chen, Saket Sathe. Charu Aggarwal, and Deepak Turaga. Outlier detection with autoencoder ensembles. In ICDM, 2017.
Lucas Deecke. Robert Vandermeulen, Lukas Ruff, Stephan Mandt, and Marius Kloft. Anomaly detection with generative adversarial networks. In ICLR, 2018.
Eleazar Eskin, Andrew Amold, Michael Prerau, Leonid Portnoy, and Sal Stolfo. A geometric frame- work for unsupervised anomaly detection. In Applications of data mining in computer security, pp. 77-101. Springer, 2002.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. ICLR, 2018.
Izhak Golan and Ran El- Yaniv. Deep anomaly detection using geometric transformations. In NewrIPS, 2018.

---

Published as a conference paper at ICLR 2020
John A Hartigan and Manchek A Wong. Algorithm as 136: k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 1979.
Xinwei He, Yang Zhou, Zhichao Zhou, Song Bai, and Xiang Bai. Triplet-center loss for multi- view 3d object retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1945-1954, 2018.
Dan Hendrycks, Mantas Mazeika, and Thomas G Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: 11f2::88/2.4600 2018.
lan Jolliffe. Principal component analysis. Springer, 2011.
Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Learning representations for auto- matic colorization. In ECCV, 2016.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 1rXiv:1706.0666 06083, 2017.
Michael Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction beyond mean square error. ICLR, 2016.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In ECCV, 2016.
Emanuel Parzen. On estimation of probability density function and mode. The annals of mathe- matical statistics, 1962.
Shebuti Rayana. ODDS library http://odds.cs.smt http:/dddd....ttooonoook.... 2016.
Lukas Ruff, Nico Gornitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Robert Vandermeulen, Alexan- der Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. ICML, 2018.
Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with nonlinear dimen- sionality reduction. In MLSD. ACM, 2014.
Thomas Schlegl, Philipp Seebück, Sebastian M Waldstein, Ursula Schmidt-Erfurth, and Georg Langs. Unsupervised anomaly detection with generative adversarial networks guide marker discovery. In International Conference on Information Processing in Medical Imaging, 2017.
Bernhard Scholkopf, Robert C Williamson, Alex J Smola, John Shawe-Taylor, and John C Platt. Support vector method for novelty detection. In NIPS, 2000.
Yan Xia, Xudong Cao, Fang Wen, Gang Hua, and Jian Sun. Learning discriminative reconstructions for unsupervised outlier removal. In ECCV, 2015.
Bo Yang, Xiao Fu, Nicholas D Sidiropoulos, and Mingyi Hong. Towards k-means-friendly spaces: Simultaneous deep learning and clustering. In ICML, 2017.
Richard Zhang, Phillip Isola, and Alexei Efros. Colorful image colorization. ECCV, 2016.
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Dacki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. ICLR, 2018.
A APPENDIX
A.1 IMAGE EXPERIMENTS
Sensitive to margin& We run Cifar10 experiments with 0.1 and and presented the results in Fig. 4 The results were not affected much by the margin parameter. This is in-line with the rest of empirical observations that GOAD is not very sensitive the margin parameter.
10

---

Published as a conference paper at ICLR 2020
Table Anomaly Detection Accuracy on Cifarl0 (%)
Class
Method
GEOM (w. Dirichlet)
GOAD(s = 0.1)
GOAD(L.0)
0
74.7 + 0,4
77.2 + 0.6
77.9 + 0.7
95.7 0.0
96.7 0.2
96.4 0.9
2
78.1 = 0,4
83.3 1.4
118
72.4 = 0.5
77.7 0.7
77.0 0.7
4
87.8 = 0.2
0.7
87.7
5
87.8 0.1
8.6
87.8
6
83.4 0
9.6
90.9 0.5
7
95.5 0.1 0.1
96.1
96.1 + 0.2
8
93.3 0.0
93.8
93.3 + 0.1
9
91.3 0.1
92.0 =
92.4 + 0.3
Average
86.0
88.2
88.1
A.2 TABULAR DATASETS
Following the evaluation protocol of Zong et al. (2018), 4 datasets used in this comparison:
Arrhythmia: A cardiology dataset from the UCI repository (Asuncion & Newman, 2007) contain- ing attributes related to the diagnosis of cardiac arrhythmia in patients. The datasets consists of 16 classes: class 1 are normal patients, 2-15 contain different arrhythmia conditions, and class 16 con- tains undiagnosed cases. Following the protocol established by ODDS (Rayana (016), the smallest classes: 3, 4, 5, 7, 8, 9, 14, 15 are taken to be anomalous and the rest normal. Also following ODDS, the categorical attributes are dropped, the final attributes total 274.
Thyroid: A medical dataset from the UCI repository (Asuncion & Newman, 2007 containing at- tributes related to whether a patient hyperthyroid. Following ODDS (Rayana 2016), from the classes of the dataset, we designate hyperfunction as the anomalous class and the rest as normal. Also following ODDS only the 6 continuous attributes are used.
KDD: The KDD Intrusion Detection dataset was created by an extensive simulation of a US Air Force LAN network. The dataset consists of the normal and 4 simulated attack types: denial of service, unauthorized access from remote machine, unauthorized access from local superuser and probing. The dataset consists of around million TCP connection records. Following evaluation protocol in Zong al. (2018), we use the UCI KDD 10% dataset, which is subsampled version of the original dataset. The dataset contains 41 different attributes. 34 are continuous and are categorical. Following Zong et al. (2018), encode the categorical attributes using 1-hot encoding.
Following Zong et al. (0188), we evaluate two different settings for the KDD dataset:
KDDCUP99; In this configuration use the entire UCI 10% dataset. As the non-attack class consists of only 20% of the dataset, is treated as the anomaly in this case, while attacks are treated normal.
KDDCUP99-RD Rev: To better correspond to the actual use-case, in which the non-attack scenario is normal and attacks anomalous, Zong et al. (2018) also evaluate the reverse configuration, in which the attack data is sub-sampled to consist of 25% of the number of non-attack samples. The attack data is in this case designated anomalous (the reverse of the KDDCUP99 dataset).
In all the above datasets, the methods are trained on 50% of the normal data. The methods are evaluated on 50% of the normal data well all the anomalies.
A.3 NUMBER OF TASKS
We provide plots of the number of auxiliary tasks the anomaly detection accuracy (measured by F2 ) for all datasets. The results are presented in Fig.2] Performance increases rapidly up to a certain number of tasks (around 16). Afterwards more tasks reduce the variance of F, scores between runs.

---

Published as a conference paper at ICLR 2020
0.70
Arthythrrte
0.9
0.5
0.7-
0.6-
0.5
0.50
0.45
0.40
0.2
600
800
Eransiormations
a)
b)
327
zanrhve minarttrmariise
Figure 2: Plots of the number of auxiliary tasks the anomaly detection accuracy (measured by Fi) a) Arrhythmia b) Thyroid c) KDDRev d) KDDCup99 Accuracy often increases with the number of tasks, although the rate diminishes with the number of tasks.
Figure 3: Plots of the degree of contamination vs. the anomaly detection accuracy (measured by Fi) (left) KDDRev (center) KDDCup99 (right) Arrhythmia. GOAD is generally robust to the degree of contamination.
A.4 CONTAMINATION EXPERIMENTS
We conduct contamination experiments for 3 datasets. Thyroid was omitted due to not having a sufficient number of anomalies. The protocol is different than that of KDDRev unused anomalies for contamination. Instead, we split the anomalies into train and test. Train anomalies are used for contamination, test anomalies are used for evaluation. As DAGMM did not present results for the other datasets, we only present GOAD. GOAD was reasonably robust to contamination on KDD, KDDRev and Arrhythmia. The results are presented Fig.
do have
12


Reviews:
[{"review_id": "SJx_brSjKH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "UPDATE:\nI acknowledge that I‘ve read the author responses as well as the other reviews.\n\nI appreciate the clarifications, additional experiments, and overall improvements made to the paper. I updated my score to 6 Weak Accept. \n\n\n####################\n\nThis paper proposes a deep method for anomaly detection (AD) that unifies recent deep one-class classification [6] and transformation-based classification [3, 4] approaches. The proposed method transforms the data to $M$ subspaces via $M$ random affine transformations and identifies with each such transformation a cluster centered around some centroid (set as the mean of the respectively transformed samples). The training objective of the method is defined by the triplet loss [5] which learns to separate the subspaces via maximizing the inter-class as well as minimizing the intra-class variation. The anomaly score for a sample is finally given by the sum of log-probabilities, where each transformation-/cluster-probability is derived from the distance to the cluster center. Using random affine transformations, the proposed method is applicable to general data types in contrast to previous works that only consider geometric transformations (rotation, translation, etc.) on image data [3, 4]. The paper conclusively presents experiments on CIFAR-10 and four tabular datasets (Arrhythmia, Thyroid, KDD, KDD-Rev) that indicate a superior detection performance of the proposed method over baselines and deep competitors.\n\nI think this paper is not yet ready for acceptance due to the following main reason: \n(i) The experimental evaluation needs clarification and should be extended to judge the significance of the empirical results.\n\n(i) I think the comparison with state-of-the-art deep competitors [6, 4] should consider at least another image dataset besides CIFAR-10, e.g. Fashion-MNIST or the recently published MVTec [1] for AD. On CIFAR-10, do you also consider geometric transformations however using your triplet loss or are the reported results from random affine transformations? I think reporting both would be insightful to see the difference between image-specific and random affine transformations.\nOn the tabular datasets, how do deep networks perform in contrast to the final linear classifier reported on most datasets? Especially when only using a final linear classifier, the proposed method is very similar to ensemble learning on random subspace projections. Figure 1 (right) shows an error curve that is also typical for ensemble learning (decrease in mean error and reduction in overall variance). I think this should be discussed and ensemble baselines [2] should be considered for a fair comparison. Table 2 also seems incomplete with the variances missing for some methods?\nFurther clarifications are needed. How many transformations $M$ do you consider on the specific datasets? How is hyperparameter $s$ chosen?\nFinally, I think the claim that the approach is robust against training data contamination is too early from only comparing against the DAGMM method on KDDCUP (Is Figure 1 (left) wrong labeled? As presented DAGMM shows a lower classification error).\n\nOverall, I think the paper proposes an interesting unification and generalization of existing state-of-the-art approaches [6, 4], but I think the experimental evaluation needs to be more extensive and clarified to judge the potential significance of the results. The presentation of the paper also needs some polishing as there are many typos and grammatical errors in the current manuscript (see comments below).\n\n\n####################\n*Additional Feedback*\n\n*Positive Highlights*\n1. Well motivated anomaly detection approach that unifies existing state-of-the-art deep one-class classification [6] and transformation-based classification [3, 4] approaches that indicates improved detection performance and is applicable to general types of data.\n2. The work is well placed in the literature. All relevant and recent related work is included in my view.\n\n*Ideas for Improvement*\n3. Extend and clarify the experimental evaluation as discussed in (i) to infer statistical significance of the results.\n4. I think many details from the experimental section could be moved to the Appendix leaving space for the additional experiments.\n5. Maybe add some additional tabular datasets as presented in [2, 7].\n6. Maybe clarify “Classification-based AD” vs. “Self-Supervised AD” a bit more since unfamiliar readers might be confused with supervised classification.\n7. Improve the presentation of the paper (fix typos and grammatical errors, improve legibility of plots)\n8. Some practical guidance on how to choose hyperparameter $s$ would be good. This may just be a default parameter recommendation and showing that the method is robust to changes in s with a small sensitivity analysis.\n\n*Minor comments*\n9. The set difference is denoted with a backslash not a forward slash, e.g. $R^L \\setminus X$.\n10. citet vs citep typos in the text (e.g. Section 1.1, first paragraph “ ... Sakurada & Yairi (2014); ...”)\n11. Section 1.1: “ADGMM introduced by Zong et al. (2018) ...” » “DAGMM introduced by Zong et al. (2018) ...”.\n12. Eq. (1): $T(x, \\tilde{m})$ in the first denominator as well.\n13. Section 2, 4th paragraph: $T(x, \\tilde{m}) \\in R^L \\setminus X_{\\tilde{m}}$.\n14. $m$, $\\tilde{m}$, and $m'$ are used somewhat inconsistently in the text.\n15. Section 3: “Note, that it is defined everywhere.”?\n16. Section 4: \"If $T$ is chosen deterministicaly ...\" >> \"If $T$ is chosen deterministically ...\"\n17. Section 5, first sentence: “... to validate the effectiveness our distance-based approach ...” » “... to validate the effectiveness of our distance-based approach ...”.\n18. Section 5.1: “We use the same same architecture and parameter choices of Golan & El-Yaniv (2018) ...” » “We use the same architecture and parameter choices as Golan & El-Yaniv (2018) ...”\n19. Section 5.2: “Following the evaluation protocol of Zong et al. Zong et al. (2018) ...” » “Following the evaluation protocol of Zong et al. (2018) ...”.\n20. Section 5.2: “Thyroid is a small dataset, with a low anomally to normal ratio ...” » “Thyroid is a small dataset, with a low anomaly to normal ratio ...”.\n21. Section 5.2, KDDCUP99 paragraph: “Tab. ??” reference error.\n22. Section 5.2, KDD-Rev paragraph: “Tab. ??” reference error.\n\n\n####################\n*References*\n\n[1] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger. Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9592–9600, 2019.\n[2] J. Chen, S. Sathe, C. Aggarwal, and D. Turaga. Outlier detection with autoencoder ensembles. In SDM, pages 90–98, 2017.\n[3] S. Gidaris, P. Singh, and N. Komodakis. Unsupervised representation learning by predicting image rotations. In ICLR, 2018.\n[4] I. Golan and R. El-Yaniv. Deep anomaly detection using geometric transformations. In NIPS, 2018.\n[5] X. He, Y. Zhou, Z. Zhou, S. Bai, and X. Bai. Triplet-center loss for multi-view 3d object retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1945–1954, 2018.\n[6] L. Ruff, R. A. Vandermeulen, N. Görnitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, and M. Kloft. Deep one-class classification. In International Conference on Machine Learning, pages 4393–4402, 2018.\n[7] L. Ruff, R. A. Vandermeulen, N. Görnitz, A. Binder, E. Müller, K.-R. Müller, and M. Kloft. Deep semi-supervised anomaly detection. arXiv preprint arXiv:1906.02694, 2019.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "HkgZFGqjKH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a novel approach to classification-based anomaly detection for general data. Classification-based anomaly detection uses auxiliary tasks (transformations) to train a model to extract useful features from the data. This approach is well-known in image data, where auxiliary tasks such as classification of rotated or flipped images have been demonstrated to work effectively. The paper generalizes to the task by using the affine transformation y = Wx+b. A novel distance-based classification is also devised to learn the model in such as way that it generalizes to unseen data. This is achieved by modeling the each auxiliary task subspace by a sphere and by using the distance to the center for the calculation of the loss function. The anomaly score then becomes the product of the probabilities that the transformed samples are in their respective subspaces. The paper provides comparison to SOT methods for both Cifar10 and 4 non-image datasets. The proposed method substantially outperforms SOT on all datasets. A section is devoted to explore the benefits of this approach on adversarial attacks using PGD. It is shown that random transformations (implemented with the affine transformation and a random matrix) do increase the robustness of the models by 50%. Another section is devoted to studying the effect of contamination (anomaly data in the training set). The approach is shown to degrade more gracefully than DAGMM on KDDCUP99. Finally, a section studies the effect of the number of tasks on the performance, showing that after a certain number of task (which is probably problem-dependent), the accuracy stabilizes.\n\n\nPROS:\n\n* A general and novel approach to anomaly detection with SOT results.\n\n* The method allows for any type of classifier to be used. The authors note that deep models perform well on the large datasets (KDDCUP) while shallower models are sufficient for smaller datasets.\n\n* The paper is relatively well written and easy to follow, the math is clearly laid out.\n\n\nCONS:\n\n* The lack of a pseudo-code algorithm makes it hard to understand and reproduce the method\n\n* Figure 1 (left) has inverted colors (DAGMM should be blue - higher error).\n* Figure 1 (right) - it is unclear what the scale of the x-axis is since there is only 1 label. Also the tick marks seem spaced logarithmically, which, if i understand correctly, is wrong.\n\n* The paragraph \"Number of operations\" should be renamed \"Number of tasks\" to be consistent. Also the sentence \"From 16 ...\" should be clarified, as it seems to contrast accuracy and results, which are the same entity. The concept of 'stability of results' is not explained clearly. It would suffice to say: 'From 16 tasks and larger, the accuracy remains stable'.\n\n* In section 6, the paragraph \"Generating many tasks\" should be named \"Number of tasks\", to be consistent with the corresponding paragraph in section 5.2. Also the first sentence should be: \"As illustrated in Figure 1 (right), increasing the number of tasks does result in improved performance but the trend is not linear and beyond a certain threshold, no improvements are made. And again the concept of 'stability' is somewhat misleading here. The sentence '...it mainly improves the stability of the results' is wrong. The stability is not improved, it is just that the performance trend is stable.\n\n* The study on the number of tasks should be carried on several datasets. Only one dataset is too few to make any claims on the accuracy trends as the number of task is increased.\n\n* The authors should coin an acronym to name their methods.\n\nOverall this paper provides a novel approach to classification-based semi-supervised anomaly detection of general data. The results are very encouraging, beating SOT methods by a good margin on standard benchmarks.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HJgOSSS6Fr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Review: The paper proposes a technique for anomaly detection. It presents a novel method that unifies the current classification-based approaches to overcome generalization issues and outperforms the state of the art. This work also generalizes to non-image data by extending the transformation functions to include random affine transformations. A lot of important applications of anomaly detection are based on tabular data so this is significant. The “normal” data is divided into M subspaces where there are M different transformations, the idea is to then learn a feature space using triplet loss that learns supervised clusters with low intra-class variation and high inter-class variation. A score is computed (using the probabilities based on the learnt feature space) on the test samples to obtain their degree of anomalousness. The intuition behind this self-supervised approach is that learning to discriminate between many types of geometric transformations applied to normal images can help to learn cues useful for detecting novelties. \n\nPros:\n- There is an exhaustive evaluation and comparison across different types of data with the existing methods along with the SOTA. \n- It is interesting to see how random transformations indeed helped to achieve adversarial robustness.\n- The method is generalized to work on any type of data with arbitrary number of random tasks. It can even be used in a linear setting if needed for small datasets.\n\nCons:\n- While I liked that an analysis was done to see the robustness of the method on the contaminated data, I would be interested to see a more rigorous comparison in this fully unsupervised setting. \n\n\nComments/Question:\nDoes the selection of the transformation types affect the method performance at all? \n\nIn the Results section on Page 7, there are a couple of “??” instead of table numbers.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a method that unifies classification-based approaches for outlier detection and (one-class) anomaly detection. The paper also extends the applicability to non-image data.

In the end, all the reviewers agreed that the paper makes a valuable contribution and I'm happy to recommend acceptance.
Decision: Accept (Poster)