One-class Classification Robust to Geometric Transformation
Authors: Hyunjun Ju
Dongha Lee
SeongKu Kang
Hwanjo Yu
Venue:  (2021)
Keywords: one-class classification
image classification
object classification
self-supervised learning
geometric robustness

Abstract: Recent studies on one-class classification have achieved a remarkable performance, by employing the self-supervised classifier that predicts the geometric transformation applied to in-class images. However, they cannot identify in-class images at all when the input images are geometrically-transformed (e.g., rotated images), because their classification-based in-class scores assume that input images always have a fixed viewpoint, as similar to the images used for training. Pointing out that humans can easily recognize such transformed images as the same class, in this work, we aim to propose a one-class classifier robust to geometrically-transformed inputs, named as GROC. To this end, we introduce a conformity score which indicates how strongly an input image agrees with one of the predefined in-class transformations, then utilize the conformity score with our proposed agreement measures for one-class classification. Our extensive experiments demonstrate that GROC is able to accurately distinguish in-class images from out-of-class images regardless of whether the inputs are geometrically-transformed or not, whereas the existing methods fail.

(e.g.. Under review as a conference paper at ICLR 2021
ONE-CLASS CLASSIFICATION ROBUST TO GEOMETRIC TRANSFORMATIONS
Anonymous authors
Paper under double-blind review
ABSTRACT
Recent studies on one-class classification have achieved a remarkable perfor- mance, by employing the self-supervised classifier that predicts the geometric transformation applied to in-class images. However, they cannot identify in- class images at all when the input images are ssometrically-transfooo rotated images), because their Ilassification-basee in-class scores assume that in- put images always have a fixed viewpoint, as similar to the images used for train- ing. Pointing out that humans can easily recognize such transformed images the same class, in this work, we aim to propose a one-class classifier robust to aimmetrrcalll-tranfffomr
inputs, named as GROC. To this end, we introduce conformity score which indicates how strongly an input image agrees with one of the predefined in-class transformations, then utilize the conformity score with our proposed agreement measures for one-class classification. Our extensive ex- periments demonstrate that GROC able accurately distinguish in-class im- ages from out-of-class images regardless of whether the inputs are geometrically- transformed or not, whereas the existing methods fail.
INTRODUCTION
One-class classification refers to the problem of identifying whether an input example belongs to single target class (in-class) or any of novel classes (out-of-class). The main challenge of this task that only in-class examples are available training time. Thus, by using only positive examples, model has to learn the decision boundary that distinguishes in-class examples from out-of-class examples, whose distribution is assumed be unknown practice. Early work on one-class clas- sification mainly utilized kernel-based methods (Schölkopf et al., 20000 Tax & Duin 2004) to find a hypersphere (or hyperplane) enclosing all training in-class examples, or density estimation tech- niques (Parzen 1962) to measure the likelihood of an input example.
In the era of deep learning, numerous literature have tried to employ deep neural networks to ef- fectively learn the tigh-dimensional data (e.g., images). Most of them aim to detect out-of-class examples based on density estimation, by adopting the architecture of autoencoders (Ruff et al.)
2018 Zong et al.] (018) generative adversarial networks (GANs) (Schlegl al.] [017]] Zenati et al. 2018). Nevertheless, their supervision is not useful enough to capture the semantic of high- dimensional data for target class, which eventually leads to the limited performance. Recently, there have been several attempts to make use of self-supervised learning (Golan & El-Yaniv 2018 Hendrycks al. 2019 Bergman & Hoshen 2020) for more informative supervision on the target class, and made a major breakthrough to this problem. They build a self-labeled image set by ap- plying a bunch of geometric transfornations to training images, then train a classifier to accurately predict the transformation applied to original input images. This approach achieved the state-of-the- art performance for one-class classification even without modeling the latent distribution of in-class examples for density estimation.
However, all the aforementioned methods are quite vulnerable to spatial variances within the im- ages, because they were developed based on the assumption that in-class (and out-of-class) images have a fixed viewpoint. In particular, the existing self-super methods do not work completely for the inputs with various viewpoints in that their capability of predicting the geometric transfor- mation relies on the fixed viewpoint. Note that humans usually recognize that the images of a target object with different viewpoints belong to the same class; in this sense, the one-class classifiers also

---

Under review as a conference paper at ICLR 2021
should be robust to the viewpoint of input images. In other words, we need to make geometrically- transformed in-class images not to be identified as out-of-class, from the perspective that a geometric transformation (e.g., rotation & s.y-trrnslation) does not change the semantic (i.c., object class) but the viewpoint.
The goal of our work is to propose effective strategy that can circumvent the limitation of view- point sensitivity, without compromising the performance for the images with the fixed viewpoint. We first present several evaluation settings for validating the robustness to flexible viewpoints, arti- ficially introduced by geometric transformations. Then, we describe our proposed solution, termed as GROC, which measures a conformity score indicating how confidently an imput image matches with one of the predefined (anchor) in-class transformations. In this work, we offer two measures for the conformity score, which are the inner product similarity and the conditional likelihood, and show how they can be optimized by the training in-class images. The empirical experiments on the proposed evaluation scenarios show that GROC considerably outperforms all the other competing methods in terms of the robustness to geometric transformation.
2 PRELIMINARIES
2.1 PROBLEM FORMULATION
Let X' be a set of all kinds of images, Xn X and Xout = Xin be the sets all in-class and out-of-class images, respectively. Given training in-class data X C Xin, we consider the one-class classification problem which differentiates in-class and out-of-class data. The problem aims to build classifier by using only the known in-class data for training. The classifier learns in-class score function, Sin (x) X' + R, where a higher score indicates that the input x is more likely to be in X... Based the score, the classifier determines whether the input belongs in-class not.
2.2 SELF-SUPERVISED LEARNING METHODS FOR ONE-CLASS CLASSIFICATION
Recently, the supervuue learning methods (Golan & El- Yaniy 2018 Hendrycks et al. 2019 Bergman& & Hoshen 2020) have achieved the state-of the-art performance one-class classification. For self-supervised learning, they first create self-labeled dataset and use it to train multi-class classifier. Concretely, let T (To: ... Ti,... TT- be a set of predefined (anchor) geometric is the identity mapping and each transformation T; is a com- position of multiple unit transformations (i.e., rotation & x.y-translation) The self-labeled dataset consists of transformed images and their corresponding labels.
transformations, where Tz(x)
Daelf = ((T.(x), x) E X S < <) K),
where Ti(-) is the i-th transformation operator and its label i is the transformation id of T:(-).
Using the self-labeled dataset, these methods train a softmax classifier based on a multi-class clas- sification loss (i.e., cross-entropy) for discrimination among the transformations. For one-class classification, they define an in-class score under the assumption that a well-trained classifier would better predict the transformation for the in-class images than that for the out-of-class images. In the end, the in-class score for an unseen image x is defined by the sum of softmax probabilities that its transformed images are correctly classified as their labels (Golan & El- Yaniy 2018 Bergman & Hoshen 2020).
K-1
Sin = Σ p (
where p py = ix), is the softmax probability that (x) is classified as the i-th transformation.
The state-of-the-art method based on this self-supervised approach (Hendrycks et al. 2019) sig- nificantly improves the performance by formulating the classification task mubti-label manner. Since each transformation is determined by the combination of unit transformations from three cat- e (i.e., rotation, (horizontal) x-translation, and (vertical) y y-translation), the unit transforma- tions applied to input image can be independently predicted for each category. Thus, they adopt
'They build the set of transformations T by the combination of the following unit transformations: rotation (09, 90", 180" 270"). x-translation E {-8, +8), and y-translation {-8, 0, +8).
2

---

Under review as a conference paper at ICLR 2021
softmax head for each transformation category, then train classifier to predict the degree of trans- formations within each category. The final in-class score is also replaced with the one summarizing all the softmax heads, each of which is for the unit transformation applied to the input.
3 METHOD
3.1 MOTIVATION
The underlying concept of the self-supervised methods based on transformation classification is to learn discriminative features of in-class images, in order to classify various viewpoints caused by the geometric transformations. The precondition for this approach is that the viewpoint of training images is always the same, otherwise the classifier cannot be trained due to the inconsistent super- vision. However, test time, the input images can have different viewpoints from those appearing in the training images. We remark that the images of the same object with different viewpoints belong to the same class, as usually recognized by humans. In this sense, it desired that in-class images with various viewpoints are identified as in-class, not out-of-class. That is, the robustness to geometric transformations should be considered for one-class classification.
In this respect, the existing self-supervised methods totally fail to compute the effective in- class scores for inputs with various viewpoints. We observe that they produce an undesirable in-class score especially when the input image has the same (or similar) viewpoint represented by the anchor transformations T [TT].. For ex- ample, suppose a classifier is trained on Deelf with the transformations T of clockwise rota- tions {0", 900 1800 ) Given two images of sea lions x' and x", let x' have the same viewpoint with the training images and x" have the 90 rotated viewpoint, which is equivalent to Ti (x'). As illustrated in Figure 1. the soft- max probability of each transformed image has high value for the input x', but the one for x" cannot cor-
90'
180'
270'
Tu(x²)
T3 (x')
Tx(x')
T; (x')
Transform
Softmax Prob.
x
Smx())= =
Σ ply ((T,()
Tx("")
Tx()")
Tx(x")
Tss"""
Transform
Softmax Prob.
Stn(x"))) = Y ((y==11T(0+"))
has a low value. Consequently, rectly identify x" as in-class, though it comes from the target class as well. We point out that setting the target label of each transformed im- age to the applied transformation is not valid any longer when an input viewpoint changed,
Figure I: A toy example of computing the in-class scores for two in-class (sea lion) images with dif- ferent viewpoints. Each row represents how the in-class score is calculated for given input.
A straightforward solution for this challenge is augmenting the training dataset so that it can cover various viewpoints of in-class images. Unfortunately. the data augmentation technique is not ap- plicable because it results in inconsistent supervision for the task of discriminating the viewpoints, which is the learning objective of the self- self-supe methods. On the other hand, there exist several one-class classification methods (Ruff et al.] [018 Zong et al. 2018)) that can adopt the data aug- mentation technique. However, they cannot achieve the performance as high as the self-supervised methods even in the case that all input images have a fixed viewpoint; this will be further discussed in Section 4 To sum up, we need to consider another strategy to develop a robust one-class classifier that works well even for the input images having various viewpoints.
3.2 PROPOSED SETUPS
We first propose three evaluation setups for testing the robustness to various viewpoints: 1) fixed viewpoint, 2) anchor viewpoint, and 3) random viewpoint. We artificially introduce the spatial variance (i.e., the changes of the viewpoint) in test images by using the geometric transformations. Note that th denotes the test data, which contains both in-class and out-of-class images.
Fixed viewpoint setup. In this setup, we consider only the fixed viewpoint that is used for training, as done in previous work. We do not change the viewpoint of the original test images, X Xe
3

---

Under review as a conference paper at ICLR 2021
25
25
Ourvoficlass In cless
Outohhcless in clsss
Out-of-clss In closs
5
NasGoo
Fiol
God
In-caas scores
0.6
0.6
0.8
in In-daas Scores
5.4 0.6
0.5
In-class scores
0.4
CIAARD
CIFARIOO
SVHN
(a) Fixed setup, X77
(b) Anchor setup, Aaoo
(c) Random setup, Xrr
(d) For transformed inputs
Figure 2: The one-class classification performance of the self-supervised method (Hendrycks et al. 2019). (a-c) The in-class score distributions of in-class and out-of-class test images (Dataset: CIFAR-10, 10, In-class: Horse), (d) AUROC for wwwnetrrccllyloongemmetrirciil transformed inputs.
Anchor viewpoint setup. This setup is designed for verifying the robustness to the viewpoints induced by the anchor transformations. We build a test dataset by X {T(x)]T T, x E X²*). where 7 is randomly sampled from the set of the anchor transformations T for each image x.
Random viewpoint setup. The random viewpoint setup further considers the geometric transfor- mations that are not included in the set of anchor transformations. We first define the superset of T, denoted by T including number of transformations with continuous degrees. A test dataset for this setup is built by X {T(x) T", x E )² where T is sampled for each image x.
As preliminary result, we plot the in-class score distributions for in-class and out-of-class test im- ages, computed by the state-of- oftte-of-the-ar self-supervised method (Hendrycks et al. 2019). In Figure we observe that the score distributions of in-class and out-of-class images X7e are clearly distin- guishable, which supports the great performance for one-class classification. On the contrary, the two score distributions are almost overlapping with each other in cases of X and XZe, strongly indicating that they fail figure out in-class images due to their various viewpoints.
We additionally investigate the performance drop for geometricalyyhon-ggoeeee inputs. Figure d))), is obvious that geometric transformations make the celf-supervised method totally malfunction, while non-geometric transformations (e.g., brightness, contrast, sharpness, and color temperature) hardly degrade the final performance for one-class classification.
transformed
3.3 PROPOSED STRATEGY
To deal with the viewpoint sensitivity of the self-supervised methods, note that the in-class images match better with the in-class transformations than the out-of-class images do, regardless of their viewpoints. Our proposed strategy, named as GROC, defines the in-class score by the sum of the conformity scores for K transformed images; Scon 'T) calculates how conformable an input image is the given T. Formally, it is defined by the maximum similarity between the representation of an input image and that of each anchor transformation:
Sin (x) = Σ Scon (Ti(x); where (x;; max [sim (x, Tj)]
TYET
The foremost condition for GROC is that the representations of the anchor transformations should be discriminative, so that the similarity measure can effectively capture the viewpoint of input images. Note that similarity between an image x and transformation Tj. denoted by sim(x, (x, Tÿ). can be defined in various ways. In the following subsections, conformity score, respectively modeled by inner product similarity and conditional likelihood. and present how the representations of input images and anchor transformations optimized.
offer two similarity measures for the
3.3.1 INNER PRODUCT SIMILARITY FOR CONFORMITY SCORE
To model the similarity measure for the conformity score, we use our encoder network (:: X' > Rdd which outputs the representation of an input image, and the weight matrix W E Kxdd that parameterizes the representations of K anchor transformations. We first present GROC-IP whose similarity measure is simply computed by the inner product f(x; 0) and wy:
sim (x, T3) f(x: 0) Wj-
4

---

Under review as a conference paper at ICLR 2021
Based on the inner product similarity, the encoder network needs to map all in-class images with the same viewpoint close to their corresponding transformation vector, while keeping the transformation vectors far from each other. In other words, it has to extract discriminative features for classifying the input images according to their viewpoint; this has been already achieved by the conventional softmax classifier in self-supervised manner.
Therefore, we adopt the optimization strategy provided by the existing self-supervised methods for one-class classification. After we build softmax classifier by adding the linear classification layer of weights W on the top of the encoder network ((; we train both the weight matrix and the network by using the cross-entropy loss with Deelf. In case of GROC-IP, the conformity score becomes equivalent to the maximum logit value computed by the softmax classifier.
3.3.2 CONDITIONAL LIKELIHOOD FOR CONFORMITY SCORE
Our second method, GROC-CI, defines the similarity measure by using the likelihood of an input image conditioned
each transformation. For simplicity, we assume that the conditional likelihood has the form of an isotropic Gaussian distribution, whose mean is μe; E Id and standard deviation is J E R4 for the condition (i.e., transformation T,). In short, GROC-CL models the representation of T as (Hz, oj) rather than w Similar to Section 3.3.1 all the Gaussian distributions need to be separable for discrimination among different viewpoints. Based on this assumption, the similarity between x and T3 is defined by the log- likeeiikl as follows.
(1r1s:00 -15111 203
sim (x, T3) = log [lg
+ +oo of
(5)
Note that this similarity be interpreted as the Mahalanobis distance between f(x θ) and μe, with the covariance matrix Lf The challenge here is to optimize the encoder network that its outputs follow N Nu,, I) for all in-class inputs having the viewpoint corresponding Tj. We train the parameters for the Gaussian distributions (J,, σ and the network [(; 0) by the following objective.
K-1
exp (sim (1) (x), TT) bn)
max o,
ΣΣ
Σ
sim (x), T;) log K Σk=O exp (sim (T, (x). bx)
(6)
XEXA 1=0
where by E ERi is the bias term for transformation j. As discussed (Lee et al. 2020,, this objective aims to learn the discriminative conditional likelihoods modeled by K separable Gaussian distribu- tions. To be precise, the first term enforcing that ff fT) (x)) follows the j-th Gaussian distribution, and the second term makes them distinguishable based Gaussian discriminant analysis.
EXPERIMENTS
4.1 COMPETING METHODS
In our experiments, we consider a variety of approaches to onc-class classification as the compet- ing methods. We choose One-class SVM (OCSVM) (Schölkopf al. 2000) and Deep SVDD (DSVDD) (Ruff et al. 2018) as the non-self-supervised methods. OCSVM is classical kernel- based method for ole-class classification, which finds a maximum-margh hyperplane the separates enclosing most of the training in-class examples. DSVDD, deep learning variant of OCSVM, explicitly models the latent space in which training in-class examples gather to a specific center point.
The main competitors are the self-supervised methods based on transformation classification: Ge- ometric Transformation (GT) (Golan & El- El-Yaniy, 2018) and Multi-labeled Geometric Transforma- tion (MGT) (Hendrycks et al. 2019). The details of the methods are presented in Scction 2.2 For the anchor transformations, GT adopts four transformation categories (i.e., horizontal flipping. x.y-translation, and rotation), while MGT excludes horizontal flipping from the above categories.
The last competing method is SimCLR (Chen et al. 2020) which learns the transformation invariant -nvaaiaaaitinnniiiantntt representations of input images in a self-supervised manner. Since it is optimized to maximize the agreement among the images differeenty-transfforeedd from single image, it is capable of alleviating
5

---

Under review as a conference paper at ICLR 2021
the viewpoint sensitivity to some degree. Several recent work representation learning based on this approach (Chen et al.) 2020] Grill et al. 2020 He et al., 2020) showed the remarkable performance for wide range of downstream tasks. Note that SimCLR is not originally designed for one-class classification, thus we tailor for our task. We define the final in-class score by
f(T(())))) Sin = Σ ())lll()) 1=1
k-1
For its optimization, we use the set of the anchor transformations adopted by MGT. More details of SimCLR are provided in Appendix
4.2 DATASETS
We validate the effectiveness of the proposed methods using three benchmark image datasets: CI0AR-10, CIFAR-100 (Krizhevsky et al.) 2009), and SVHN (Netzer et al. 201I). We scale the pixel values of all images to be in [-1, 1] as done in (Golan & El- Yaniv, 2018). Note that CIFAR- 100 has 20 super-classes and we use these super-classes rather than 100 full classes.
4.3 EXPERIMENTAL SETTINGS
Following the experimental setting of the previous studies (Golan & El-Yaniv) 2018 Ruff et al. 2018 Zenati et al. 2018), we employ the one-vs-all evaluation scheme. For the dataset with C classes, we generate C' one-class classification settings; the images of a target class regarded as in-class data, and the other images belonging to C classes are regarded as out-of-class data. We the Area Under the ROC curve (AUROC) an evaluation metric.
We build the set of the anchor transformations by the combination of the following unit transformations: x-translation
{-8, 0, +8), and rotation (0", 90", 1500,270% As presented in Section 3.2 we evaluate our method by using the three pro- posed setups: fixed viewpoint, anchor viewpoint, and random viewpoint. For the random viewpoint setup, we build the test set by randomly sampling the transformation degrees for the x.y-translation in the range of [-8, 8). and for the rotation in the range of [oa, 360%). In case of SVHN, we exclude the x.y-translation because it may change the semantic (i.e., class label) of the images; the label of each image is determined by the digit in the middle of the image.
{-8, 0, +8], y-translation
4.4 EXPERIMENTAL RESULTS
4.4.1 COMPARISONS WITH SELF-SUPERVISED METHODS
The experimental results of the self-supervised methods on the three setups are presented in Table I For each dataset, the original class name is replaced by its class id due to the limited space. In sum- mary. our methods effectively overcome the limitation of viewpoint sensitivity, without compromis- ing the performance in the fixed viewpoint setup. We analyze the results from various perspectives.
Fixed viewpoint setup. this setup, the state-of-the-ar competitor MGT consistently shows the best results, and our methods show the comparable results to MGT. We also observe that the per- formance of SimCLR is not as good as those of the classification-based methods. Note that the classification--aass methods aim for discrimination among the differently-trassforned images, whereas SimCLR tries to make them indistinguishable. From this observation, we can conclude that directly learning the transformation invariant visual features is less effective identify in-class and out-of-class images for one-class classification.
Anchor viewpoint setup. In case that the input images have anchor viewpoints, the classification- based methods fail to distinguish in-class images from out-of-class images; their performances are even worse than a random guess whose AUROC is 0.5. Because their capability ing the geometric transformations depends on the fixed viewpoint, they do not work at all for the input images with various viewpoints, as discussed in Section 3.1. In contrast, our methods con- siderably outperform all the competing methods while providing outstanding performances robust to the changes the viewpoint. These results show that our methods successfully identify the in- class images irrespective of their viewpoint, by the help of the conformity score that measures how
discriminat-

---

Under review as a conference paper at ICLR 2021
Table 1: The AUROC of self-supervised methods for one-class classification on the three evaluation setups. The best results are marked in bold face.
Fixed Viewpoint
Anchor Viewpoint
Rundom Viewpoint
Dataset
in-class
SimCLR GT MGT GROC-IP IP/ GROC-CL
0
0.66 / 0.78 / /.75
0,54/0.44 / 0045/ / 0,70 / 1066
0.53 /.3///4///// / 0.44 /0.3 / 0.64
0.62 0.87/ / 0.96 / 0.95 0.97
0.40/0.32 0.39 0..99/0.9 / / /0.94
0.40/ 0,39 /.40 / 0.84 0.88
0.64/ / /0.84
0.44/ / 0.45/ / 0.76 / 0.73
0.56/ 0.6//..6//00 0.45 / 0.64/ 0.65
0.54/0.77 0.80 0.77 /0.77
0.43 0.44 0.73 / 0.69
0.43 / 0.49 0.62 0.61
CIFAR-10
0.61 /.86 0.91 / 0 0.90
/ 0.42/ 0.85 / 0.83
0.49 / 0.42/ 0.72 0.71
5
0.54 /0.87/ 0.89 / 0.88/ 0.88
0.43 0,.4 /0.42 / 0.83 0,83
0.44/0.44 / 0,44 / 0.72 0.73
6
0,65 / 0.899 / 0.89 / 0,90 0.93
0.42 0.44 / 0.42 / 0.81 / 0,83
0 / 0,46 / 0.72 0.76
7
0,58/ / 0.91 / 0.96 / 0.95 0.96
0.43 / 0.32 / 0.42 / 0.91 / 0.89
0 0,40 0.77 / 0.77
8
0.52 0.88/ / 0.94 / 00.2//092/2/// 0.94
0.44 / 0.35 / 0..11 / 0.88 / 0.87
0,41 /0.66 0.39 / 0.80 0.80
9
0.51 0.84//04///// / 0.90 / / 0.93
0.41 / 0.36 0.42 0,86 0.88
0.40 0.39 / /.79 0.83
avg
0.59 0.85//////// 0.89 / 0.88/ 0.89
0.46 0.39 / 0.42 0.82 0.82
0.45 / 0.43
/..43 / 0.73 0.74
0
0.57 0.75 / 0.77 / 0.77 0.78
0.55 0.44 / 0.44 / 0.71 / 0.72
0.55 / 0.44 / 0.44 / 0.62 0.64
0,64 / 0.69 / 0.72 / 0.74 0.72
0.58 / 0,46 / 0.43 / 0,66 / 0.62
0.40 / 0,44 / 0.60 0.62
2
0.63 0071 / 0.71 / 0.75 / 0.82
0.49 / 0,44 / 0.48/ / 0.70 / 0.75
0,51 /0.48 / 0.47 / 0.65 0.70
3
0,58 0.77/ / 0.80 / 0.80/ / 0.80
0.55 / 0,40 / 0.41 / 0.72 / 0.75
0.48 / 0.47 / /0.52 / 0.65
4
0.63 0.78/ 0.80 / 0.81 0.81
0,53 0.44 / 0.46/ / 0.76 / 0.76
0.50/ / 0..43 / 0.70 / 0.72
0.59 0.66 / 0.67 0.67 0.66
0,56 /..55 / 0.49/ / 0.60 / 0.59
0 / 0.51 / 0.53 0.55
0.50/ 0.86
0.844 0.85
0.48 /0.4 0.46 / 0.77 / 0.82
0.57 0.44/046 / 0.46/ 0.59 0.58
0.67 /0.6 0.66 0.67 0.69
/ 0.47 0.60 0.62
0, 0.59 0.55
CIFAR-100 100
0.50/ 0.84/ 0.87 / 0.85/ 0.87
0.48 0.44 / 0646 0.78 / 0,80
0 O. 06 / 0.69 0.75
0,42 0.88// 0.92 / 0.91 1.91
0.41 0.39 / 00.4 / 0.89 / 0.87
0.41 0.38 / 0.39 / 0.76 0.77
10
0,60/ 0.85/ / 0.87 / 0.85/ 0.85
0.54 / 0.39/ / 0.44/ / 0.82 / 0,81
0.42/ / 0,43 / 0.73 / 0.74
11
0.53 / 0.84/ / 0,86 / 0,86/ 0.87
0.46 / 0,41 / 0.43 / 0.80/ / 0.81
0 / 0,44/ / 0.70 / 0.70
12
0.54 0.82/ / 0.84/ / 0.83 / 0.84
0.47 / 0.43 / 00.55 / 0.77 / 0.80
0.47 / 0.45 / 0.68 0.72
13
0.65 0.60 / 0.62 / 0.60/
0.63 / 0.51 / 0.49 / 0,56 / 0.55
0 / 0.47 / 0.51 / 0,52
14
0.56 0.52 / 0.93 / 0.90 0,90
/0.40 0.42/ / 0.86 0.84
0.44 / 0.43 / 0.70
15
0.71 0.72
0.55 / 0.46 / 0.65 0.65
0.5 / 0.49 / 0.60 0.61
16
0.55 / 0.80 0.80 0.76
0.52 0..4 0.73
0 / 0.66
17
0.65 0.90 0.93 0.92 0.92
0.55 / 0.36 0.41 0.88 0.84
0.52/ 0.38 0.39 0.79 0.76
18
0.89 0.90 0 0.92
0.44 0,440 0.40 0.87 00.6
0,41 0.47 / 0,42 0.75 0.76
19
0.53 0.85/ / 0.89 / 0.87 0.85
0.45 / 0.40 0.41 0.81 / 0..9
0.47 / 0.43 / 0.39 0.73
avg
0.57 0.79/ 0.81 / 0.80 0.81
0.51 / 0.43 0.44 / 0.75 / 0.75
0.50 / 0.45 / 0.45 / 0.66 0.67
0.79/ 0.71 / 0.86 / 0.92 0.92
0.79 / 0.39 0.21 / 0.92 / 0.92
0.63 0.43 / 0.44 0.78 0.80
0.68 0.72/ 0.80 0.89 0.90
0.68 / 0.35 0.20 / 0.89 / 0.89
0.52 0.42 / 0.43 0.75 0.77
0.75 0.98/ 0.96 / 0.94
0.75 / 0.30 0.23 0.94 / 0.94
0.53 / 0.40 / 0.41 / 0:75 0.76
0.63/ / 0.91 0.94 / 0.93 0.92
0.63 / 0.35 / 0.411 0.93 /0.91
0.60 0.42 0.75 0.73
SVHN
4
0,63/ 0.98 / 0.98 / 0.95 0.96
0.63 / 0.33 / 0.21 / 0.95 / 0.96
0.54/ / 0,42 / 0.71 / 0.71
5
0.86 / 0.96 / 0.95 / 0.95 00
0.86 / 0.30 0.32/ / 0.95 / 0.94
0.69 / 0.40 0.83 0.78
6
0.69 / 0.96 / 0 /
0.69 / 0.31 / 0.35 0,91 / 0.91
0.61 /.36 / 0.39 /
7
0.75 / 0.99 0.98 / 0.96
0.75 / 0.30 / 0.23
/
0,46/ / 0.43 /0.70 0.74
0.67 0.77 / 0.88 / 0.92
0.67 / 0.36 0.33 0.92
/.59/044
0.67
0.65 0.97 / 0.97 0.90
0.65 / 0.32 0.31 / 0.90 0.89
0.60/ /.34 / 0.43 0.79
avg
0.71 / 0.90 / 0.93 / 0.93 0.92 0.71 / 0.33 / 0.28 / 0.93 0.92 0.58 / 0.40/ 0.42 / 0.76 0.75
confidently an input image matches with one of the in-class transformations. Interestingly, the per- formance of SimCLR is higher than that of the classification based methods in this setup. This is because its learning objective that encourages the multiple transformations of an input image to be similar with each other makes the one-class classifier less affected by the viewpoint.
Random viewpoint setup. In the hardest setting where the input images have random viewpoints, the classification- methods cannot beat a random guessing, similarly to the anchor viewpoint setup. Our methods perform the best for all the datasets, which strongly indicates the robustness to the changes of the viewpoint. In conclusion, both of the our methods (i.e., GROC-IP and GROC- CL) are able to correctly classify the images having diverse viewpoints into in-class or out-of-class, even for the viewpoints that have not been seen during the training.
4.4.2 COMPARISONS WITH
METHODS
Figure presents the comparison results against the non-self self-supervised methods on the three eval- uation setups. Due the limited space, we report the score averaged over C in-class settings for
7

---

Under review as a conference paper at ICLR 2021
0.8
0.8
0.6
AUROC
0.6
0.6
0.4
0.4
CROCCLL
GPOC-CL
C10C
CIFARIO
CIRAR10D
SVHIN
(a) Fixed Viewpoint
(b) Anchor Viewpoint
(c) Random Viewpoint
Figure 3: Comparisons with -un--ell-uueeeeieed methods in the three evaluation setups
each dataset. We also provide the results of DSVDD that adopts the data augmentation technique, denoted by DSVDD+; the training in-class images are randomly augmented by T T For all the setups, the competing methods show poorer performances compared to our methods. Specifi- cally, the data augmentation technique results in the limited performance gains in the anchor/random setups, whereas it even brings an adverse effect in the fixed setup. This implies that the simple ap- proach is not sufficient address the viewpoint sensitivity of the one-class classifiers.
4.4.3 FURTHER ANALYSIS
We also provide in-depth analyses on the performance the self- self-supervised methods, for the SVHN dataset. In the fixed viewpoint setup, we observe that there is a distinct performance improvement of our GROC over MGT, especially for the cases of in-class 0, 1, and 8. Figure4sshows shows the in-class score distributions obtained by GROC and MGT, where the class 0 is set to in-class. Since the digit '0' has a symmetric shape. it is difficult for MGT to differentiate its transformations between the rotation 0 and 180 (or, 90 and 270"). For this reason, as illustrated in the rightmost figure, MGT outputs relatively low scores for the images having a single 'O' but high scores for the images having other digits around '0'. On the contrary, GROC produces the similar (and high) in-class scores for these images (i.e., with or without other digits), which can be separated from the scores of out-of- class images. This helps GROC less overlap the in-class scores between in-class and out-of-class images, and as a result, it leads to higher AUROC compared to MGT.
On the other hand, for the cases of in-class 6 and 9, the performance of GROC slightly degrades because the 180" rotation of the out-of-class digit '9' is more likely to be conformable to the in- class digit '6', and vice versa. Nevertheless, under the assumption that the input images have various viewpoints, it is impossible even for humans to accurately figure out whether the images that look like '6' (or '9') belong in-class or out-of-class
In-closs
In
in-llass
Outiof dass
Dendyy
Dendyy
soore
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.5
0.8
1.0
In-class score of MGT
In-class score GROC
In-class score of MGT
Figure 4: The in-class score distributions of in-class and out-of-class test images in the fixed view- point setup (Dataset: SVHN, In-class: 0).
CONCLUSION
This paper proposes a novel one-class classification method robust to geometric transformations, which effectively addresses the challenge that in-class images cannot be correctly distinguished from out-of-class images when they have various viewpoints. We first present new evaluation setups that cover the diverse viewpoints by artificially introducing the spatial variance into test images. measure how strongly input im-
Then, we define the conformity-basee in-class score age is conformable to one of the anchor transformations, whose representations are optimized to be discriminative. The extensive experiments demonstrate that the proposed GROC keeps its outstand-
8

---

Under review as a conference paper at ICLR 2021
ing performance even in the anchor/random viewpoint setups where the input images have various viewpoints, whereas the sateeoff-the-al methods perform even worse than a random guessing.
REFERENCES
Liron Bergman and Yedid Hoshen. Classification based anomaly detection for general data. arXiv preprint arXiv:2005.025:: 2020.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXivv2002.05709, 2020.
Izhak Golan and Ran El-Yaniv. Deep anomaly detection using geometric transformations. In Ad- vances Neural Information Processing Systems, pp. 9758-9766 2018.
Jean-Bas Monn-Batienn Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint 200::0006.07733. 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9729-9738, 2020.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. Advances Neural Information Processing Systems, pp. 15663-15674, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 36(11):2277-2324 1998.
Dongha Lee, Sehun Yu, and Hwanjo Yu. Multi-class data description for tion. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discov- ery & Data Mining, pp. 1362-1370, 2020.
detec-
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv: 160.003888.. 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning. 2011.
Emanuel Parzen. On estimation probability density function and mode. The annals of mathe- matical statisties, 33(3):1065-1076 1962.
Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexan- der Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference on machine learning, pp. 4393 4402, 2018.
Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Ursula Schmidt-Erfurte and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. International conference on information processing in medical imaging, pp. 146 157. Springer, 2017.
Bernhard Schölkopf. Robert C Williamson, Alex Smola, John Shawe- TTylor, and John C Platt. Support vector method for novelty detection. In Advances in neural information processing sys- tems, pp. 582-588, 2000.

---

Under review as a conference paper at ICLR 2021
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, 1lya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-1958, 2014.
David MJ Tax and Robert PW Duin. Support vector data description. Machine learning, 54(1): 45-66, 2004.
Sergey Zagoruyko and Nikos Komodakis. 100.006605 2016.
Wide residual networks.
arXiv preprint
Houssam Zenati, Manon Romain, Chuan-Sheng Foo, Bruno Lecouat, and Vijay Chandrasekhar. Adversarially learned anomaly detection. In 2018 IEEE International Conference Data Mining (ICDM), pp- 727-736. IEEE, 2018.
Bo Zong. Qi Song. Martin Rengiang Min, Wei Cheng, Cristian Lumezanu, Dacki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In Inter- national Conference on Learning Representations, 2018.
10

---

Under review as a conference paper at ICLR 2021
A SIMCLR
In the experiments, we slightly modified SimCLR (Chen et al. 2020) for the one-class classification task. The main idea of SimCLR is learning representations by maximizing the agreement among the images
from the same image via contrastive loss in the latent space. For the optimization of SimCLR, we use the set the anchor transformations that adopted by the classification-bass self-supervised method (i.e., MGT); this set is different from the one used by the original SimCLR. Let E 2 and = T (x) where T a transformation operator randomly sampled from the set of the anchor transformation T. Given batch B (x1,-- .NN} J C. Xi define B {31; X2, S2N-1, } where X2k-1 and X2k generated by applying different transformations to each image X2 in the batch. The loss function for a pair of two differently- transformed images (x2R-1, (2xk) from the input image X is defined as follows.
exp (sim (f (x 0f (xex: 0)) /T) 2N Ti (2h 1)pexp (sim (f (x 0) (8;; 0))
1(x2=-1,Xek K2k) log
where N is the number of images in batch. f-) is an encoder network including a projection layer, and sim u, v) u v u/22lll /u/22ll// lllvlle is the cosine similarity. It is worth noting that we include the pro- jection layer the encoder network in order to obtain the cransfrrattioninnariii unlike the original version of SimCLR that discards the projection layer for their downstream tasks.
representations,
In the end, the objective function of SimCLR for batch B is defined
N
LSSmCLR =
(822-11, + (X22,> 22t-
(9)
2N
B IMPLEMENTATION DETAILS
We choose a 16-4 WideResnet ZZgorrykko & Komodakis 2016) as the backbone architecture. We adopt the training strategy for lulti-label classification, proposed in (Hendrycks et al. 2019). Dur- ing the training, we use the cosine annealing for scheduled learning (Loshchilov Hutter 2016)
with initial learning rate 0.1 and Nesterov momentum. The dropout rate (Srivastava et al.] (014) is
set to 0.3. All the self-supervised methods based on transformation classification also use the same backbone architecture and training hyperparameters with ours. For DSVDD, we use LeNet (LeCun) et al. 1998) style network as described in the paper and implementation For SimCLR, we employ the ResNet18 (He al.) (016)) with the fixed T value of 0.5. For GROC-CL, the regularization coefficient for optimizing the conditional likelihoods is set to 0.0001.
?
C EXPERIMENTAL RESULTS
In T we report the full comparison results with the non-self-superviie methods for one-class classification (summarized in Figure 3)
D THEORETICAL BACKGROUNDS FOR GROC-CL
GROC-CL basically utilizes the encoder network / that induces the latent space, where the similarity between the representation of an input image and that of each anchor transformation is modeled by an isotropic Gaussian distribution (conditioned on the transformation). Formally, the similarity between x and Tí can be described as sim (x, T,) = log (x(T,) = log.N' (f (x) [p If where
p [x TT)) is the class-condie conditioniii (or transformation- cnnifformatioo-ccnditiinaall probability.
As discussed in Section 3.3 the representation of each anchor transformation should be distinguish- able from the others', which is the foremost condition for GROC, in order to effectively calculate the conformity score and identify in-class/out-of-chhe images from the score. In this sense, GROC- CL. can be understood from the perspective of Gaussian Discriminant Analysis (GDA). To this end,

---

Under review as a conference paper at ICLR 2021
optimize the encoder network by maximizing the posterior probability of transformed image (x) having the maximum similarity with the transformation j. which is denoted by p (TT)TT)(x)) For simplicity, assume that the prior probability for each class (or transformation) follows the Bernoulli distribution, i.e., p (T₃) = B BE.
p (T,)) (T,
P (T))T)(x))
Σ p (Txk) (Tx) (x)())
exp (- (203) 11f (T)(x)) Myll² log of log 53)
(10)
K-1 Σ K= exp (- (20g)) /|ʃ (T₃ - pall - log x + log 3kx
exp (sim (T) (x), T2) + by)
kk exp (sim (T) (x), T;. + by)
Note that taking the log of this equation becomes equivalent the second term in Equation (6)
In addition, we need to force the empirical class-conditional distribution to follow the isotropic Gaussian distribution and also approximate the empirical class mean to the obtained class mean μj. Thus, we minimize the Kullback-Leibler (KL) divergence between the j-th empirical class- conditional distribution Pj and the corresponding Gaussian distribution N (mg, oī).. The empirical class-conditional distribution for transformation defined as follows.
Pj
Σ
f
(1I)
in
X
where 8 (-) is the Dirac measure. Finally, the KL divergence is obtained by
KL (P (15, = /
1
12 (-1z
Me
5 (T,(xx)) log
d/2 exp
de
(2
203
xE
/
5 (z (T))
Σ 8(z (TT
dz
|
xe
xexti
(T) - 14:1 4illt)]
Σ log
(2mT²) 2/2 exp
log
20²
xexri
(T,()) Mall
=
P
Σ
20²
+ log logoy)
+ constant
Σ
sim (T), + constant.
XE
(12)
The final form of the KL divergence is derived by using the definition of the Dirac measure. After the constant term is excluded, it becomes the same with the first term in Equation (6. Minimizing the KL term for all the classes (or transformations) matches the empirical class-conditional distributions with the isotropic Gaussian distributions.
12

---

Under review as a conference paper at ICLR 2021
Table The AUROC of non-self -ew-sesuee--iieeddd methods for one-class classification on the three eval- uation setups.
Fixed Viewpoint
Anchor Viewpoint
Random Viewpoint
Dataset
in-class
OCSVM / DSVDD DSVDD+ GROC-IP/ GROC-CL
0
0.65 0/5/0..55/0000.77600 0.75
0.63 / 0.64 /..6 / 0,700/0000000000000 /0.68
0 / 0.69 /..63 / 0.64
0.41 /..44/04445////////// /0.95 /097
0.38 /..48 / / 0,91 /0.94
0.39 / 0.46// 0.49 / 0.88
0.62 0..66//085//00 /0.84
0.65 /..62 0.63 0.76/0.73 0.73
0.64/0.63 0.64 (/64/0.64) /..44 0.65
0.50 0.54/ /0.77
0.49 0,73 0.69
0.49
0.62 0.61 0.53 0.55
CIFAR-18
0.75 0.766//0.6///6
0.74 0.75 0.85 0.88
0..33 0.74 0.72 0.77
0.51 /.51 0.34 /.88
0.50/0.45 / 0.49 0,83 /0.83
/ / 0.45 / 0.72 0.73
6
0.72 0.75 /.7.
0.71 / 0.72 / 0.73 0.81 /0.83
0.68 / 0.73 / 0.72 0.76
7
0,51 0.52 0.58 0,95/0.90
0,51 / 0.53 0.91 0.89
0.50/ 0,54 / 0.77 0.77
8
0.68 0.68 / 0.41 / /
0.54 / 0.53 / 0.50 0.88/0.88 0.87
0.54 / 0.61 / 00.00 0.80
9
0.49 /0.52/ / 0.37 /..90
0.38 / 0.377 / 0.39 0.86/0.8...0 0.88
0.38 /0.40/ /
ave
0.59 /.58 0.89
0.55/0.56
0.82
4.54/0.52/0.58 0.7.///558//.3/.7.
0
(.64/064/// /05.////////0.7/////////
0.65 / / 0.58 0.71 0..72
0.64/0.49 / 0.58 0.62 0.64
0.52 /.55 / /0.74 0.72
0.51 0.55 / 0.57 0.66 0.62
0 / 0.54 / 0.60 0.62
2
0,52/ 0.56/ 0.39 0,75 0.82
0.52/0.33 / 0.54 0.70 0.75
0.58 / 0.61 /0.65 0.70
3
0,51 0.58 / 0.47 / 0,80 0.80
0.50 / 0.53 / 0.57 0.72 / 0.75
0 / 0.44 / 0.52 0.65
4
0.52 /0.58////////// 0,46 / 0.811
0.51 0.53 / 0.56 / 0.76 / 0.76
0.46 / 0.56 / 0.70 0.72
5
0.44/ /44/0.55 0.50 0.43 / 00.67/ 0.66
0.44 /0.50 / 0.47 / /0,59
0 / 0.51 / 0.53 0.55
0.52 / 0.50 /0.8
0,51 0.47 0.77 /0.82
0 / 0.40 0.59 0.58
0.59 0.63 0.62 0.67 /0.69
0.59/0.58 0.55 0.60
0.54 0.58 0.59 0.56
CIFAR-100 100
0.67 /.67 0.63 0.85 /087
0.66 0.69 0.88
/ 0.65 0.69 0.75
0.69 0.69 0.51 / 0
0.50 / 08 0.89 00.87
/ 0.47
10
0.75 0.82 0.57 / 0,85 0.85
0.62 /0.71 / 00.7 0.82//2/ /
0.61 / / 0.67 / 0.73 0.74
11
0,61 /.61 / 0.53 / 0.86 0.87
0.61 0.60 / 0.58 0.80/0.81
0.50 / 0,52 / 0.70 0.70
12
0.68 0.63 0.6S / 0.83 0.88
0.67 0.65 / 6464 / 0.77 / 0.80
0.66/ / 0.66 / 0.68 0.72
13
0.66 /.60
0.59 /.60 / 0.62 / 0,56 0.55
0.68 0.65 /0.1 0.52
14
/0.43 / 0.46 0.86
0.44/045 / / 0.70
15
0.00/0665/0.5//007/0000
0.60 0.59 /0.65
0.55 /..0
16
0.66/0.6///000/0.00000
0.65 /.58 0.60 0..6
0.53 0.59 /...0/////..9//6//////
17
0271/00.//0..700.92//0
0.61 0..48 0.53 0,88
0.57 / 0.61 /.79
18
/.5//0.50//.44/.00000 0.85
/ /..43 0.86
0.48 / 0.47 /.75
19
0.55
0.49 / /0.1
0.49 0.45 /0.3
avg
0.59 0.61
0.56 /.54 / 0.55 0.75
0.54 0.53 0.55 0.66
0.54 0.62 0.55 0.92 0.99
0.53 / 0.51 / 0.52 0,92 0.92
0.52 0.53 /.54 0.80
0.51 0.55 0.50 /..90
0,50/ 0.51 0.522 0.89 0.89
0.50 0.52 0.75 /
0.52 0.55 0.50 /0.94
0.51 / 0.50 0.50/ 0.94 0.94
0.50 0.50 0.51 00.75 0.76
0.51 0.53 /.50 0.93 0.92
0.50 / 0.49 / 0.50 0.93 /0.91
0.50 0,51 0.75 0.73
AS NEEN
4
0.50/ 0.0//.05///5
0.95 /.96
0.49 / 0.49 / 0.50 0.95 /
0.51 / 0.51 0.71 0.71
0,52/ 0..22//0.5///99/ 0.95 0094
0.51 / 0,49 / 0.51
/
/.51 / 0,51 / 0.83 0.78
6
0.51 00.6 / 0,50 /0.1 /0.91
0.49 5.50/ 0.91 /0.1
/.54 / 0.51 / 0.80 / 0.78
7
0.51 /.5100.6//052/ /..///66///////////....66
0,50 0.51 0,96
0,49 / 0.51 /.77
8
2.50/0.54/049///0 /0.9////9//
/ 0.49 / 0,92
0,49 /0.1 0.67 0.51 / / /..///////000000 /0.78
0.52/0.51 /..90000..
0.51 /..5 /.900//0/////000...
avg
0.51 / /.5.
0.50/
/ 0.52 0.76
13


Reviews:
[{"review_id": "mYofWYjpK2Q", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n**UPDATE**\n\nI acknowledge that I have read the author responses as well as the other reviews. I appreciate the improvements made and clarifications given by the authors.\n\nI would keep my score recommendation at 5 (marginally below acceptance threshold) at this point, however, mainly due to (i) a limited novelty (as partly acknowledged by the authors in the response) and (ii) a limited experimental evaluation (missing high-resolution datasets such as ImageNet or real-world AD datasets such as MVTec-AD) of the current manuscript.\n\nI encourage the authors to build on their current results and extend their work with additional datasets and an analysis of geometric shifts also at training time.\n\n#####\n\n\n**Summary**\n\nThis work considers the unsupervised anomaly detection task on image data. The paper critically remarks that existing state-of-the-art self-supervised methods [5, 6, 2], which learn one-class models through auxiliary classification based on applying a set of geometric transformations, are sensitive to changes in viewpoint due to a one-to-one identification of auxiliary labels with certain transformations, thus assuming a fixed viewpoint for inlier data. A Geometrically Robust One-class Classifier (GROC) is proposed which is a variant of previous self-supervised methods [5, 6] where at testing time, instead of the one-to-one relation between transformations and labels, an inlier conformity score is derived from the most confident in-class transformation agreement for a given input. This enables, even when there is a geometric change of viewpoint for some images at test time, to find the best matching in-class transformation, thus implicitly arranging an image with the most characteristic in-class viewpoint learned from the training data. The sum of such conformity scores obtained from applying the set of transformations to an input finally serves as an overall inlier score. Two ways to measure agreement in the output layer are suggested, dot product similarity and conditional likelihood, which show similar detection and robustness performance. Experiments using the one-vs-rest evaluation protocol on CIFAR-10, CIFAR-100, and SVHN demonstrate that GROC indeed is more robust to geometric changes, which is simulated by three experimental scenarios that cover various degrees of changes in viewpoint at testing time.\n\n\n**Pros**\n\n+ The proposed method, GROC, presents a simple and elegant tweak to make self-supervised anomaly detection methods more robust to geometric changes and shifts at testing time.\n+ The paper is technically correct and the presented experimental evaluation scientifically rigorous.\n+ The paper has a clear and easy to follow structure.\n+ The work is placed well into the existing literature.\n\n\n**Cons**\n\n- The novelty of the presented work is rather low as the non-robustness of self-supervised methods to geometric changes is not surprising, given that these models have been trained on the premise that the training data has similar viewpoints and geometry. The methodological novelty is also rather low, as GROC presents a test time score adaptation, but the training of self-supervised models is the same as in [5, 6, 2].\n- The experimental evaluation is limited to low-resolution synthetic anomaly detection settings and somewhat tailored to fit the proposed model as geometric shifts and changes are only applied to the testing data in the proposed evaluation scenarios, but GROC still relies on the criticized similar viewpoint assumption for model training.\n\n\n**Recommendation**\n\nI tend towards rejecting this paper (score: 5) as I find the novelty of the presented work to be rather low and the current experiments insufficient for evaluating model robustness to geometric transformations.\n\nThe presented method, GROC, still relies on the assumption that the viewpoint in the training data is mostly the same, given that training a model follows [5, 6, 2] and the training data remains untransformed. In my opinion, the experimental evaluation should also consider a geometric augmentation of the training data to evaluate the robustness. In fact, such training data augmentation is used with the non-self-supervised methods (DSVDD+ and SimCLR) to make them more robust to geometric variation, but this is also the harder setting to learn from. GROC should be evaluated on such a setting as well to properly assess its robustness. Variation in viewpoints in both, the training and test data, is also the more relevant and natural setting in practice. Such an analysis would be interesting and could be insightful for the community to assess the usefulness of self-supervised methods based on geometric transformation in general.\n\n\n**Additional feedback and ideas for improvement**\n\n- Include geometric variation in the training data into the experimental evaluation.\n- Include higher resolution datasets with more geometric variation (e.g., ImageNet [6, 8]) and real-world anomaly detection datasets (e.g., MVTec [3]) into the analysis.\n- Is GROC beneficial on non-image data as well, as considered by GOAD [2]?\n- Other previous work has also shown that self-supervised methods are vulnerable to non-geometric anomalies [4].\n- The one-vs-all evaluation scheme has seen some critique recently [1, 7]. It would be good to consider leave-one-out and real anomaly detection datasets (e.g., MVTec [3]) as well.\n\n\n**Minor Comments**\n\n1. Section 1 : ‘Nevertheless, their supervision is not useful enough to capture the semantic of high-dimensional data for a target class, which eventually leads to *a* limited performance.’ Citation? Also ‘supervision’ might be misleading here, as the referred to methods are unsupervised. Maybe ‘learning’?\n2. Section 1, 2nd paragraph: GOAD [2] also can be applied to general data, e.g. tabular data. Not only images.\n3. Section 2.2: ‘*For self-supervised* learning, [...]’\n4. Section 2.2: ‘Using the self-labeled dataset, *these methods* train a softmax classifier [...]’\n5. Insufficient space below Figure 1 caption.\n6. Figure 1: Should be $S_{in}(x')$ and $S_{in}(x'')$, correct?\n7. Section 3.3: ‘[...], can be defined in various *ways*.’\n8. Section 3.3.1: ‘After we build a softmax classifier by adding the linear classification layer of weights $W$ *on top* of the encoder network [...]’ There is no change in architecture to previous methods, right? [5, 6] employ linear layers before the cross-entropy as well, correct? The difference lies in taking the maximum over dot products/logit values at testing time?\n9. Section 4.1: ‘OCSVM is a classical kernel-based method for one-class classification, which finds a *maximum-margin hyperplane the separates* most of the training in-class examples.’\n10. Section 4.4.1: ‘[...], the classification-based methods cannot beat *random guessing*, [...]’\n11. Section 4.4.1: ‘In conclusion, both *of* our methods [...]’\n12. Section 5: ‘[...], whereas the state-of-the-art methods perform even worse than *random guessing*.’\n\n\n#####\n\n**References**\n\n[1] F. Ahmed and A. Courville. Detecting semantic anomalies. In AAAI, pages 3154–3162, 2020.\n\n[2] L. Bergman and Y. Hoshen. Classification-based anomaly detection for general data. In ICLR, 2020.\n\n[3] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger. Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In CVPR, pages 9592–9600, 2019.\n\n[4] P. Chong, L. Ruff, M. Kloft, and A. Binder. Simple and effective prevention of mode collapse in deep one-class classification. In IJCNN, pages 1–9. IEEE, 2020.\n\n[5] I. Golan and R. El-Yaniv. Deep anomaly detection using geometric transformations. In NeurIPS, pages 9758–9769, 2018.\n\n[6] D. Hendrycks, M. Mazeika, S. Kadavath, and D. Song. Using self-supervised learning can improve model robustness and uncertainty. In NeurIPS, pages 15637–15648, 2019.\n\n[7] L. Ruff, J. R. Kauffmann, R. A. Vandermeulen, G. Montavon, W. Samek, M. Kloft, T. G. Dietterich, and K.-R. Müller. A unifying review of deep and shallow anomaly detection. arXiv preprint arXiv:2009.11732, 2020.\n\n[8] L. Ruff, R. A. Vandermeulen, B. J. Franks, K.-R. Müller, and M. Kloft. Rethinking assumptions in deep anomaly detection. arXiv preprint arXiv:2006.00339, 2020.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "mUifIQ8ZSg", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors aim to solve the problem of one class classification using self-supervision. While, this method has been adopted previously, in this paper, the authors aim to make the one-class classification robust to rotations. \n\nThe main idea in the method is based on the idea of using anchor transformations instead of augmenting the dataset using transformed examples. The method is compared against other self-supervised one class classification methods on CIFAR-10/CIFAR-100 and SVHN datasets\n\nThe novelty in the proposed work is limited as it specifically addresses the issue of geometric transformations for one-class classification and it is a very specific case that is addressed. The techniques adopted are also fairly straightforward. In the first technique, K anchor transformations are used to obtain the prediction with the learned self-supervised method. The second technique obtains the conditional likelihood conditioned on each transformation. These techniques are not particularly novel.\n\nThe evaluation is limited as the paper argues that data-augmentation would not be applicable as it would result in inconsistent supervision. However, no actual evaluation is provided for the same. Methods like SIM-CLR advocate the use of contrastive learning with many classes. It is not evident that using the same approach for one-class classification can be done using just a single class. It would be interesting to consider additional evaluation where SIM-CLR is evaluated only with the additional one-class that is of concern. \n\n---\nI have considered the rebuttal provided. Particularly the aspect that data-augmentation would result in inconsistent supervision is an interesting point and experimental analysis of the same would be useful. However, I am not convinced that the paper provides a broad enough solution. In view of this I raise my score from 3 to 4, but maintain my view that the paper is presently not good enough for acceptance.", "extra_scores": {}, "extra_text": {}}, {"review_id": "CeQKnyJeFfF", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper considers the deep one-class classification problem. Some recent state of the art in this area is built upon self-supervised learning methods that are trained to predict the rotation applied to a training image, and then use the success of rotation prediction on test images as an outlier score. The paper observes that, while successful on standard benchmarks, this strategy is not robust to unexpected image rotations at test-time. Since, humans are (presumably) able to exhibit rotation invariance during test-time in 1-class classification, this is considered a flaw in existing methods. To rectify this flaw, the paper proposes to use an anomaly score which is the maximum over all possible rotation predictions. The results show that the proposed method outperforms prior approaches when exposed to novel rotations at test time. \n\nStrength: \n+ The paper correctly identifies a flaw in existing rotation self-supervision-based approaches to one-class deep learning. It can be seen as identifying a family of pathological examples where these methods fail.\n+ The proposed modification is a reasonable, intuitive, and effective approach to correcting this flaw, and obtaining robustness to these difficult rotated examples. \n\nWeaknesses: \n1. Niche issue: This paper is ultimately focusing on a niche issue: One-class classification => Specific rotation-prediction family of one-class methods => Robustness of this specific family of methods to novel test-time rotations. It’s not clear that this issue is of sufficiently general interest to be worth publication in ICLR. \n2. Practical relevance: For real applications, we would likely either work with data where photos do indeed come in a typical orientations, and thus existing methods would work fine. Or if not we would use a one-class method that aims for rotation-invariance, rather than rotation equivariance. The “bug” that this paper identifies is not surprising, and would not be likely to bite anyone in practice.\n3. Technical advance/novelty: The fix proposed by this model (take the max over rotation outputs, rather than look at a single output) is rather obvious and not a significant technical advance. This modification relies on the same “low-confidence in presence of novel data” assumption that is already explored in Hendrycks ICLR’17 for class-wise probabilities. \n4. Significance: The whole study is about fixing a glitch in some current high-performing self-supervision-for-1class methods. But while these methods may currently top the leaderboards for the 1class benchmarks, there are a plethora of self-supervised methods out there (jigsaw-prediction, colorisation, etc) that are yet to be adapted to 1-class learning. These may outperform the rotation-prediction methods for 1-class learning, while not being susceptible to the glitch identified here. Unless it’s clear that rotation-prediction is the final word in 1-class learning, then it may not be a significant result.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "93aYFpu7oVk", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a one-class classifier robust to geometrically-transformed inputs (GROC). A conformity score is proposed that measures how strongly an input image agrees with one of the predefined in-class transformations. Experiments show that the proposed method works well on 3 datasets for out-of-class detection and produces similar scores for in-class images under different transformations.\n\nOverall this is a well written paper. The proposed method is well motivated and results are comprehensive. Results are quite promoting compared to existing works. \n\nTechnical: \n- How the proposed method is different from SimCLR? Both try to maximize the similarity between an image under different transformations. This should be better clarified.\n- A nice thing in Bergman and Hoshen's paper is that geometric transformation is generalized to the affine class, which enable it to be applied to non-visual data, such as tabular data. In this paper, the application domain seems to be confined within images only. I wonder how the proposed method can be extended to other modalities?\n- While the paper mentions a challenge \"to optimize the encoder network so that its outputs follows\" a Gaussian distribution, it is not clear how this challenges is resolved effectively. \n\nExperiments:\nResults and analysis look good, but I do have a few concerns as follows:\n- Why the performance of SimCLR is so low compared to other methods? it is somehow unexpected given that SimCLR has shown outstanding performance on unsupervised image classification as a representation learning method. Any insights?  Also, which data augmentations are you using? From SimCLR paper, the choice of data augmentations is critical, so I wonder whether this is related to the low performance here.\n- Why the work: Bergman and Hoshenet al., Classification-Based Anomaly Detection for General Data is not included in experiments? It is closely related to this topic as it also applies geometric transformation for anomaly detection and compares with GT. Without such comparisons, the results are less convincing. Please provide more explanations. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper explores the robustness of one-class classifiers to geometric transformations at test time. The authors observe that some existing methods fail to detect novel images from the same class when they have undergone specific transformations at test time i.e. in-plane rotations. In contrast, it is suggested that humans have no difficulty in ignoring the impact of these types of transformations. To address this issue, the authors propose to take the maximum prediction over the set of rotated versions of a given test image. 

The current consensus from reviewers, and this meta-reviewer agrees with this view, is that the paper, while not without some merit, is too narrow in focus to be of general interest in its current form. The main contribution is limited to one family of transformations, and it is not immediately clear how to generalize this to others when the entire transformation space is not easily enumerated. There are also legitimate concerns regarding if the specific issue outlined is likely to be a problem in practice (see R1's comments). The authors allude to some interesting negative results related to data augmentation in their response to R4 (R2 also had questions about this). The authors should consider adding these results to a future revision of the paper as it will strengthen the central message.

In conclusion given the limited support, this AC also agrees that the paper is not yet ready for publication at ICLR. 

Decision: Reject