One-class Classification Robust to Geometric Transformation

1 INTRODUCTION . One-class classification refers to the problem of identifying whether an input example belongs to a single target class ( in-class ) or any of novel classes ( out-of-class ) . The main challenge of this task is that only in-class examples are available at training time . Thus , by using only positive examples , a model has to learn the decision boundary that distinguishes in-class examples from out-of-class examples , whose distribution is assumed to be unknown in practice . Early work on one-class classification mainly utilized kernel-based methods ( Schölkopf et al. , 2000 ; Tax & Duin , 2004 ) to find a hypersphere ( or hyperplane ) enclosing all training in-class examples , or density estimation techniques ( Parzen , 1962 ) to measure the likelihood of an input example . In the era of deep learning , numerous literature have tried to employ deep neural networks to effectively learn the high-dimensional data ( e.g. , images ) . Most of them aim to detect out-of-class examples based on density estimation , by adopting the architecture of autoencoders ( Ruff et al. , 2018 ; Zong et al. , 2018 ) or generative adversarial networks ( GANs ) ( Schlegl et al. , 2017 ; Zenati et al. , 2018 ) . Nevertheless , their supervision is not useful enough to capture the semantic of highdimensional data for a target class , which eventually leads to the limited performance . Recently , there have been several attempts to make use of self-supervised learning ( Golan & El-Yaniv , 2018 ; Hendrycks et al. , 2019 ; Bergman & Hoshen , 2020 ) for more informative supervision on the target class , and made a major breakthrough to this problem . They build a self-labeled image set by applying a bunch of geometric transformations to training images , then train a classifier to accurately predict the transformation applied to original input images . This approach achieved the state-of-theart performance for one-class classification even without modeling the latent distribution of in-class examples for density estimation . However , all the aforementioned methods are quite vulnerable to spatial variances within the images , because they were developed based on the assumption that in-class ( and out-of-class ) images have a fixed viewpoint . In particular , the existing self-supervised methods do not work completely for the inputs with various viewpoints in that their capability of predicting the geometric transformation relies on the fixed viewpoint . Note that humans usually recognize that the images of a target object with different viewpoints belong to the same class ; in this sense , the one-class classifiers also should be robust to the viewpoint of input images . In other words , we need to make geometricallytransformed in-class images not to be identified as out-of-class , from the perspective that a geometric transformation ( e.g. , rotation & x , y-translation ) does not change the semantic ( i.e. , object class ) but the viewpoint . The goal of our work is to propose an effective strategy that can circumvent the limitation of viewpoint sensitivity , without compromising the performance for the images with the fixed viewpoint . We first present several evaluation settings for validating the robustness to flexible viewpoints , artificially introduced by geometric transformations . Then , we describe our proposed solution , termed as GROC , which measures a conformity score indicating how confidently an input image matches with one of the predefined ( anchor ) in-class transformations . In this work , we offer two measures for the conformity score , which are the inner product similarity and the conditional likelihood , and show how they can be optimized by the training in-class images . The empirical experiments on the proposed evaluation scenarios show that GROC considerably outperforms all the other competing methods in terms of the robustness to geometric transformation . 2 PRELIMINARIES . 2.1 PROBLEM FORMULATION . Let X be a set of all kinds of images , Xin ⊆ X and Xout = X\Xin be the sets of all in-class and out-of-class images , respectively . Given training in-class data X trin ⊆ Xin , we consider the one-class classification problem which differentiates in-class and out-of-class data . The problem aims to build a classifier by using only the known in-class data for training . The classifier learns an in-class score function , Sin ( x ) : X → R , where a higher score indicates that the input x is more likely to be in Xin . Based on the score , the classifier determines whether the input belongs to in-class or not . 2.2 SELF-SUPERVISED LEARNING METHODS FOR ONE-CLASS CLASSIFICATION . Recently , the self-supervised learning methods ( Golan & El-Yaniv , 2018 ; Hendrycks et al. , 2019 ; Bergman & Hoshen , 2020 ) have achieved the state-of-the-art performance in one-class classification . For self-supervised learning , they first create a self-labeled dataset and use it to train a multi-class classifier . Concretely , let T = { T0 , · · · , Ti , · · · , TK−1 } be a set of predefined ( anchor ) geometric transformations , where T0 ( x ) = x is the identity mapping and each transformation Ti is a composition of multiple unit transformations ( i.e. , rotation & x , y-translation ) . The self-labeled dataset consists of transformed images and their corresponding labels . Dself = { ( Ti ( x ) , i ) |x ∈ X trin , 0 ≤ i < K } , ( 1 ) where Ti ( · ) is the i-th transformation operator and its label i is the transformation id of Ti ( · ) . Using the self-labeled dataset , these methods train a softmax classifier based on a multi-class classification loss ( i.e. , cross-entropy ) for a discrimination among the transformations . For one-class classification , they define an in-class score under the assumption that a well-trained classifier would better predict the transformation for the in-class images than that for the out-of-class images . In the end , the in-class score for an unseen image x is defined by the sum of softmax probabilities that its transformed images are correctly classified as their labels ( Golan & El-Yaniv , 2018 ; Bergman & Hoshen , 2020 ) . Sin ( x ) = K−1∑ i=0 p ( y = i|Ti ( x ) ) , ( 2 ) where p ( y = i|Ti ( x ) ) is the softmax probability that Ti ( x ) is classified as the i-th transformation . The state-of-the-art method based on this self-supervised approach ( Hendrycks et al. , 2019 ) significantly improves the performance by formulating the classification task in a multi-label manner . Since each transformation is determined by the combination of unit transformations from three categories1 , ( i.e. , rotation , ( horizontal ) x-translation , and ( vertical ) y-translation ) , the unit transformations applied to an input image can be independently predicted for each category . Thus , they adopt a 1They build the set of transformations T by the combination of the following unit transformations : rotation ∈ { 0◦ , 90◦ , 180◦ , 270◦ } , x-translation ∈ { −8 , 0 , +8 } , and y-translation ∈ { −8 , 0 , +8 } . softmax head for each transformation category , then train a classifier to predict the degree of transformations within each category . The final in-class score is also replaced with the one summarizing all the softmax heads , each of which is for the unit transformation applied to the input . 3 METHOD . 3.1 MOTIVATION . The underlying concept of the self-supervised methods based on transformation classification is to learn discriminative features of in-class images , in order to classify various viewpoints caused by the geometric transformations . The precondition for this approach is that the viewpoint of training images is always the same , otherwise the classifier can not be trained due to the inconsistent supervision . However , at test time , the input images can have different viewpoints from those appearing in the training images . We remark that the images of the same object with different viewpoints belong to the same class , as usually recognized by humans . In this sense , it is desired that in-class images with various viewpoints are identified as in-class , not out-of-class . That is , the robustness to geometric transformations should be considered for one-class classification . Sea lion , n02077923 In this respect , the existing self-supervised methods totally fail to compute the effective inclass scores for inputs with various viewpoints . We observe that they produce an undesirable in-class score especially when the input image has the same ( or similar ) viewpoint represented by the anchor transformations T \ { T0 } . For example , suppose a classifier is trained on Dself with the transformations T of clockwise rotations { 0◦ , 90◦ , 180◦ , 270◦ } . Given two images of sea lions x′ and x′′ , let x′ have the same viewpoint with the training images and x′′ have the 90◦ rotated viewpoint , which is equivalent to T1 ( x′ ) . As illustrated in Figure 1 , the softmax probability of each transformed image has a high value for the input x′ , but the one for x′′ has a low value . Consequently , it can not correctly identify x′′ as in-class , though it comes from the target class as well . We point out that setting the target label of each transformed image to the applied transformation is not valid any longer when an input viewpoint is changed . A straightforward solution for this challenge is augmenting the training dataset so that it can cover various viewpoints of in-class images . Unfortunately , the data augmentation technique is not applicable because it results in inconsistent supervision for the task of discriminating the viewpoints , which is the learning objective of the self-supervised methods . On the other hand , there exist several one-class classification methods ( Ruff et al. , 2018 ; Zong et al. , 2018 ) that can adopt the data augmentation technique . However , they can not achieve the performance as high as the self-supervised methods even in the case that all input images have a fixed viewpoint ; this will be further discussed in Section 4 . To sum up , we need to consider another strategy to develop a robust one-class classifier that works well even for the input images having various viewpoints . 3.2 PROPOSED SETUPS . We first propose three evaluation setups for testing the robustness to various viewpoints : 1 ) fixed viewpoint , 2 ) anchor viewpoint , and 3 ) random viewpoint . We artificially introduce the spatial variance ( i.e. , the changes of the viewpoint ) in test images by using the geometric transformations . Note that X te denotes the test data , which contains both in-class and out-of-class images . Fixed viewpoint setup . In this setup , we consider only the fixed viewpoint that is used for training , as done in previous work . We do not change the viewpoint of the original test images , X tefv = X te . Anchor viewpoint setup . This setup is designed for verifying the robustness to the viewpoints induced by the anchor transformations . We build a test dataset by X teav = { T ( x ) |T ∼ T , x ∈ X te } , where T is randomly sampled from the set of the anchor transformations T for each image x . Random viewpoint setup . The random viewpoint setup further considers the geometric transformations that are not included in the set of anchor transformations . We first define the superset of T , denoted by T ∗ , including a number of transformations with continuous degrees . A test dataset for this setup is built by X terv = { T ( x ) |T ∼ T ∗ , x ∈ X te } , where T is sampled for each image x . As a preliminary result , we plot the in-class score distributions for in-class and out-of-class test images , computed by the state-of-the-art self-supervised method ( Hendrycks et al. , 2019 ) . In Figure 2 , we observe that the score distributions of in-class and out-of-class images in X tefv are clearly distinguishable , which supports the great performance for one-class classification . On the contrary , the two score distributions are almost overlapping with each other in cases of X teav and X terv , strongly indicating that they fail to figure out in-class images due to their various viewpoints . We additionally investigate the performance drop for geometrically/non-geometrically transformed inputs . In Figure 2 ( d ) , it is obvious that geometric transformations make the self-supervised method totally malfunction , while non-geometric transformations ( e.g. , brightness , contrast , sharpness , and color temperature ) hardly degrade the final performance for one-class classification .

Summary: This paper considers the deep one-class classification problem. Some recent state of the art in this area is built upon self-supervised learning methods that are trained to predict the rotation applied to a training image, and then use the success of rotation prediction on test images as an outlier score. The paper observes that, while successful on standard benchmarks, this strategy is not robust to unexpected image rotations at test-time. Since, humans are (presumably) able to exhibit rotation invariance during test-time in 1-class classification, this is considered a flaw in existing methods. To rectify this flaw, the paper proposes to use an anomaly score which is the maximum over all possible rotation predictions. The results show that the proposed method outperforms prior approaches when exposed to novel rotations at test time.