One-class Classification Robust to Geometric Transformation

1 INTRODUCTION . One-class classification refers to the problem of identifying whether an input example belongs to a single target class ( in-class ) or any of novel classes ( out-of-class ) . The main challenge of this task is that only in-class examples are available at training time . Thus , by using only positive examples , a model has to learn the decision boundary that distinguishes in-class examples from out-of-class examples , whose distribution is assumed to be unknown in practice . Early work on one-class classification mainly utilized kernel-based methods ( Schölkopf et al. , 2000 ; Tax & Duin , 2004 ) to find a hypersphere ( or hyperplane ) enclosing all training in-class examples , or density estimation techniques ( Parzen , 1962 ) to measure the likelihood of an input example . In the era of deep learning , numerous literature have tried to employ deep neural networks to effectively learn the high-dimensional data ( e.g. , images ) . Most of them aim to detect out-of-class examples based on density estimation , by adopting the architecture of autoencoders ( Ruff et al. , 2018 ; Zong et al. , 2018 ) or generative adversarial networks ( GANs ) ( Schlegl et al. , 2017 ; Zenati et al. , 2018 ) . Nevertheless , their supervision is not useful enough to capture the semantic of highdimensional data for a target class , which eventually leads to the limited performance . Recently , there have been several attempts to make use of self-supervised learning ( Golan & El-Yaniv , 2018 ; Hendrycks et al. , 2019 ; Bergman & Hoshen , 2020 ) for more informative supervision on the target class , and made a major breakthrough to this problem . They build a self-labeled image set by applying a bunch of geometric transformations to training images , then train a classifier to accurately predict the transformation applied to original input images . This approach achieved the state-of-theart performance for one-class classification even without modeling the latent distribution of in-class examples for density estimation . However , all the aforementioned methods are quite vulnerable to spatial variances within the images , because they were developed based on the assumption that in-class ( and out-of-class ) images have a fixed viewpoint . In particular , the existing self-supervised methods do not work completely for the inputs with various viewpoints in that their capability of predicting the geometric transformation relies on the fixed viewpoint . Note that humans usually recognize that the images of a target object with different viewpoints belong to the same class ; in this sense , the one-class classifiers also should be robust to the viewpoint of input images . In other words , we need to make geometricallytransformed in-class images not to be identified as out-of-class , from the perspective that a geometric transformation ( e.g. , rotation & x , y-translation ) does not change the semantic ( i.e. , object class ) but the viewpoint . The goal of our work is to propose an effective strategy that can circumvent the limitation of viewpoint sensitivity , without compromising the performance for the images with the fixed viewpoint . We first present several evaluation settings for validating the robustness to flexible viewpoints , artificially introduced by geometric transformations . Then , we describe our proposed solution , termed as GROC , which measures a conformity score indicating how confidently an input image matches with one of the predefined ( anchor ) in-class transformations . In this work , we offer two measures for the conformity score , which are the inner product similarity and the conditional likelihood , and show how they can be optimized by the training in-class images . The empirical experiments on the proposed evaluation scenarios show that GROC considerably outperforms all the other competing methods in terms of the robustness to geometric transformation . 2 PRELIMINARIES . 2.1 PROBLEM FORMULATION . Let X be a set of all kinds of images , Xin ⊆ X and Xout = X\Xin be the sets of all in-class and out-of-class images , respectively . Given training in-class data X trin ⊆ Xin , we consider the one-class classification problem which differentiates in-class and out-of-class data . The problem aims to build a classifier by using only the known in-class data for training . The classifier learns an in-class score function , Sin ( x ) : X → R , where a higher score indicates that the input x is more likely to be in Xin . Based on the score , the classifier determines whether the input belongs to in-class or not . 2.2 SELF-SUPERVISED LEARNING METHODS FOR ONE-CLASS CLASSIFICATION . Recently , the self-supervised learning methods ( Golan & El-Yaniv , 2018 ; Hendrycks et al. , 2019 ; Bergman & Hoshen , 2020 ) have achieved the state-of-the-art performance in one-class classification . For self-supervised learning , they first create a self-labeled dataset and use it to train a multi-class classifier . Concretely , let T = { T0 , · · · , Ti , · · · , TK−1 } be a set of predefined ( anchor ) geometric transformations , where T0 ( x ) = x is the identity mapping and each transformation Ti is a composition of multiple unit transformations ( i.e. , rotation & x , y-translation ) . The self-labeled dataset consists of transformed images and their corresponding labels . Dself = { ( Ti ( x ) , i ) |x ∈ X trin , 0 ≤ i < K } , ( 1 ) where Ti ( · ) is the i-th transformation operator and its label i is the transformation id of Ti ( · ) . Using the self-labeled dataset , these methods train a softmax classifier based on a multi-class classification loss ( i.e. , cross-entropy ) for a discrimination among the transformations . For one-class classification , they define an in-class score under the assumption that a well-trained classifier would better predict the transformation for the in-class images than that for the out-of-class images . In the end , the in-class score for an unseen image x is defined by the sum of softmax probabilities that its transformed images are correctly classified as their labels ( Golan & El-Yaniv , 2018 ; Bergman & Hoshen , 2020 ) . Sin ( x ) = K−1∑ i=0 p ( y = i|Ti ( x ) ) , ( 2 ) where p ( y = i|Ti ( x ) ) is the softmax probability that Ti ( x ) is classified as the i-th transformation . The state-of-the-art method based on this self-supervised approach ( Hendrycks et al. , 2019 ) significantly improves the performance by formulating the classification task in a multi-label manner . Since each transformation is determined by the combination of unit transformations from three categories1 , ( i.e. , rotation , ( horizontal ) x-translation , and ( vertical ) y-translation ) , the unit transformations applied to an input image can be independently predicted for each category . Thus , they adopt a 1They build the set of transformations T by the combination of the following unit transformations : rotation ∈ { 0◦ , 90◦ , 180◦ , 270◦ } , x-translation ∈ { −8 , 0 , +8 } , and y-translation ∈ { −8 , 0 , +8 } . softmax head for each transformation category , then train a classifier to predict the degree of transformations within each category . The final in-class score is also replaced with the one summarizing all the softmax heads , each of which is for the unit transformation applied to the input . 3 METHOD . 3.1 MOTIVATION . The underlying concept of the self-supervised methods based on transformation classification is to learn discriminative features of in-class images , in order to classify various viewpoints caused by the geometric transformations . The precondition for this approach is that the viewpoint of training images is always the same , otherwise the classifier can not be trained due to the inconsistent supervision . However , at test time , the input images can have different viewpoints from those appearing in the training images . We remark that the images of the same object with different viewpoints belong to the same class , as usually recognized by humans . In this sense , it is desired that in-class images with various viewpoints are identified as in-class , not out-of-class . That is , the robustness to geometric transformations should be considered for one-class classification . Sea lion , n02077923 In this respect , the existing self-supervised methods totally fail to compute the effective inclass scores for inputs with various viewpoints . We observe that they produce an undesirable in-class score especially when the input image has the same ( or similar ) viewpoint represented by the anchor transformations T \ { T0 } . For example , suppose a classifier is trained on Dself with the transformations T of clockwise rotations { 0◦ , 90◦ , 180◦ , 270◦ } . Given two images of sea lions x′ and x′′ , let x′ have the same viewpoint with the training images and x′′ have the 90◦ rotated viewpoint , which is equivalent to T1 ( x′ ) . As illustrated in Figure 1 , the softmax probability of each transformed image has a high value for the input x′ , but the one for x′′ has a low value . Consequently , it can not correctly identify x′′ as in-class , though it comes from the target class as well . We point out that setting the target label of each transformed image to the applied transformation is not valid any longer when an input viewpoint is changed . A straightforward solution for this challenge is augmenting the training dataset so that it can cover various viewpoints of in-class images . Unfortunately , the data augmentation technique is not applicable because it results in inconsistent supervision for the task of discriminating the viewpoints , which is the learning objective of the self-supervised methods . On the other hand , there exist several one-class classification methods ( Ruff et al. , 2018 ; Zong et al. , 2018 ) that can adopt the data augmentation technique . However , they can not achieve the performance as high as the self-supervised methods even in the case that all input images have a fixed viewpoint ; this will be further discussed in Section 4 . To sum up , we need to consider another strategy to develop a robust one-class classifier that works well even for the input images having various viewpoints . 3.2 PROPOSED SETUPS . We first propose three evaluation setups for testing the robustness to various viewpoints : 1 ) fixed viewpoint , 2 ) anchor viewpoint , and 3 ) random viewpoint . We artificially introduce the spatial variance ( i.e. , the changes of the viewpoint ) in test images by using the geometric transformations . Note that X te denotes the test data , which contains both in-class and out-of-class images . Fixed viewpoint setup . In this setup , we consider only the fixed viewpoint that is used for training , as done in previous work . We do not change the viewpoint of the original test images , X tefv = X te . Anchor viewpoint setup . This setup is designed for verifying the robustness to the viewpoints induced by the anchor transformations . We build a test dataset by X teav = { T ( x ) |T ∼ T , x ∈ X te } , where T is randomly sampled from the set of the anchor transformations T for each image x . Random viewpoint setup . The random viewpoint setup further considers the geometric transformations that are not included in the set of anchor transformations . We first define the superset of T , denoted by T ∗ , including a number of transformations with continuous degrees . A test dataset for this setup is built by X terv = { T ( x ) |T ∼ T ∗ , x ∈ X te } , where T is sampled for each image x . As a preliminary result , we plot the in-class score distributions for in-class and out-of-class test images , computed by the state-of-the-art self-supervised method ( Hendrycks et al. , 2019 ) . In Figure 2 , we observe that the score distributions of in-class and out-of-class images in X tefv are clearly distinguishable , which supports the great performance for one-class classification . On the contrary , the two score distributions are almost overlapping with each other in cases of X teav and X terv , strongly indicating that they fail to figure out in-class images due to their various viewpoints . We additionally investigate the performance drop for geometrically/non-geometrically transformed inputs . In Figure 2 ( d ) , it is obvious that geometric transformations make the self-supervised method totally malfunction , while non-geometric transformations ( e.g. , brightness , contrast , sharpness , and color temperature ) hardly degrade the final performance for one-class classification .

Summary: This paper presents a one-class classifier robust to geometrically-transformed inputs (GROC). A conformity score is proposed that measures how strongly an input image agrees with one of the predefined in-class transformations. Experiments show that the proposed method works well on 3 datasets for out-of-class detection and produces similar scores for in-class images under different transformations.