Deep Repulsive Clustering of Ordered Data Based on Order-Identity Decomposition

We propose the deep repulsive clustering ( DRC ) algorithm of ordered data for effective order learning . First , we develop the order-identity decomposition ( ORID ) network to divide the information of an object instance into an order-related feature and an identity feature . Then , we group object instances into clusters according to their identity features using a repulsive term . Moreover , we estimate the rank of a test instance , by comparing it with references within the same cluster . Experimental results on facial age estimation , aesthetic score regression , and historical color image classification show that the proposed algorithm can cluster ordered data effectively and also yield excellent rank estimation performance . 1 INTRODUCTION . There are various types of ‘ ordered ’ data . For instance , in facial age estimation ( Ricanek & Tesafaye , 2006 ) , face photos are ranked according to the ages . Also , in a video-sharing platform , videos can be sorted according to the numbers of views or likes . In these ordered data , classes , representing ranks or preferences , form an ordered set ( Schröder , 2003 ) . Attempts have been made to estimate the classes of objects , including multi-class classification ( Pan et al. , 2018 ) , ordinal regression ( Frank & Hall , 2001 ) , metric regression ( Fu & Huang , 2008 ) . Recently , a new approach , called order learning ( Lim et al. , 2020 ) , was proposed to solve this problem . Order learning is based on the idea that it is easier to predict ordering relationship between objects than to estimate the absolute classes ( or ranks ) ; telling the older one between two people is easier than estimating their exact ages . Hence , in order learning , the pairwise ordering relationship is learned from training data . Then , the rank of a test object is estimated by comparing it with reference objects with known ranks . However , some objects can not be easily compared . It is less easy to tell the older one between people of different genders than between those of the same gender . Lim et al . ( 2020 ) tried to deal with this issue , by dividing an ordered dataset into disjoint chains . But , the chains were not clearly separated , and no meaningful properties were discovered from the chains . In this paper , we propose a reliable clustering algorithm , called deep repulsive clustering ( DRC ) , of ordered data based on order-identity decomposition ( ORID ) . Figure 1 shows a clustering example of ordered data . Note that some characteristics of objects , such as genders or races in age estimation , are not related to their ranks , and the ranks of objects sharing such characteristics can be compared more reliably . To discover such characteristics without any supervision , the proposed ORID network decomposes the information of an object instance into an order-related feature and an identity feature unrelated to the rank . Then , the proposed DRC clusters object instances using their identity features ; in each cluster , the instances share similar identity features . Furthermore , given a test instance , we decide its cluster based on the nearest neighbor ( NN ) rule , and compare it with reference instances within the cluster to estimate its rank . To this end , we develop a maximum a posteriori ( MAP ) estimation rule . Experimental results on ordered data for facial age estimation , aesthetic score regression ( Kong et al. , 2016 ) , and historical color image classification ( Palermo et al. , 2012 ) demonstrate that the proposed algorithm separates ordered data clearly into meaningful clusters and provides excellent rank estimation performances for unseen test instances . The contributions of this paper can be summarized as follows . • We first propose the notion of identity features of ordered data and develop the ORID network for the order-identity decomposition . • We develop the DRC algorithm to cluster data on a unit sphere effectively using a repulsive term . We also prove the local optimality of the solution . • We propose the MAP decision rule for rank estimation . The proposed algorithm provides the state-of-the-art performances for facial age estimation and aesthetic score regression . 2 RELATED WORK . 2.1 ORDER LEARNING . The notion of order learning was first proposed by Lim et al . ( 2020 ) . It aims to determine the order graph of classes and classify an object into one of the classes . In practice , it trains a pairwise comparator , which is a ternary classifier , to categorize the relationship between two objects into one of three cases : one object is bigger than , similar to , or smaller than the other . Then , it estimates the rank of a test object , by comparing it with reference objects with known ranks . However , not every pair of objects are easily comparable . Although Lim et al . ( 2020 ) attempted to group objects into clusters , in which objects could be more accurately compared , their clustering results were unreliable . Pairwise comparison has been used to estimate object ranks , because relative evaluation is easier than absolute evaluation in general . Saaty ( 1977 ) proposed the scaling method to estimate absolute priorities from relative priorities , which has been applied to various decision processes , including aesthetic score regression ( Lee & Kim , 2019 ) . Also , some learning to rank ( LTR ) algorithms are based on pairwise comparison ( Liu , 2009 ; Cohen et al. , 1998 ; Burges et al. , 2005 ; Tsai et al. , 2007 ) . Order learning attempts to combine ( possibly inconsistent ) pairwise ordering results to determine the rank of each object . Thus , it is closely related to the Cohen et al. ’ s LTR algorithm ( 1998 ) , which learns a pairwise preference function and obtains a total order of a set to maximize agreements among preference judgments of pairs of elements . Also , order learning is related to rank aggregation ( Dwork et al. , 2001 ) , in which partially ordered sets are combined into a linearly ordered set to achieve the maximum consensus among those partial sets . Rank aggregation has been studied in various fields ( Brüggemann et al. , 2004 ) . Since optimal aggregation is NP-hard , Dwork et al . ( 2001 ) proposed an approximate algorithm , called Markov chain ordering . There are many other approximate schemes , such as the local Kemenization , Borda count , and scaled footrule aggregation . 2.2 CLUSTERING . Data clustering is a fundamental problem to partition data into disjoint groups , such that elements in the same group are similar to one another but elements from different groups are dissimilar . Although various clustering algorithms have been proposed ( Hartigan & Wong , 1979 ; Ester et al. , 1996 ; Kohonen , 1990 ; Dhillon & Modha , 2001 ; Reynolds , 2009 ) , conventional algorithms often yield poor performance on high-dimensional data due to the curse of dimensionality and ineffectiveness of similarity metrics . Dimensionality reduction and feature transform methods have been studied to map raw data into a new feature space , in which they are more easily separated . Linear transforms , such as PCA ( Wold et al. , 1987 ) , and non-linear transformations , including kernel methods ( Hofmann et al. , 2008 ) and spectral clustering ( Ng et al. , 2002 ) , have been proposed . Recently , deep neural networks have been adopted effectively as feature embedding functions ( LeCun et al. , 2015 ) , and these deep-learning-based feature embedding functions have been combined with classical clustering algorithms . For instance , Caron et al . ( 2018 ) proposed a deep clustering algorithm based on k-means . It clusters features from a neural network and then trains the network using the cluster assignments as pseudo-labels . This is done iteratively . Also , Yang et al . ( 2016 ) jointly learned feature representations and clustered images , based on agglomerative clustering . Chang et al . ( 2017 ) recast the image clustering task into a binary classification problem to predict whether a pair of images belong to the same cluster or different clusters . Similarly to these algorithms , we use a neural network to determine a feature space in which clustering is done more effectively . However , we consider the clustering of ordered data , and each cluster should consists of elements , whose ranks can be compared more accurately . There are conventional approaches to use clustering ideas to aid in classification or rank estimation . For example , Yan et al . ( 2015 ) developed a hierarchical classifier , which clusters fine categories into coarse category groups and classifies an object into a fine category within its coarse category group . For extreme multiclass classification , Daumé III et al . ( 2017 ) proposed to predict a class label among candidate classes only , which are dynamically selected by the recall tree . It is however noted that the leaves of the recall tree do not partition the set of classes . Also , for age estimation , Li et al . ( 2019 ) proposed a tree-like structure , called bridge-tree , to divide data into overlapping age groups and train a local regressor for each group . The set of local regressors can be more accurate than a global regressor to deal with the entire age range . Whereas these conventional approaches group data in the label dimension to perform their tasks more effectively , the proposed algorithm cluster data in the dimension orthogonal to the label dimension . In other words , we cluster data using identity features , instead of using order features . 3 PROPOSED ALGORITHM . 3.1 PROBLEM DEFINITION . An order is a binary relation , often denoted by ≤ , on a set Θ = { θ1 , θ2 , . . . , θm } ( Schröder , 2003 ) . It should satisfy three properties of reflexivity ( θi ≤ θi for all i ) , antisymmetry ( θi ≤ θj and θj ≤ θi imply θi = θj ) , and transitivity ( θi ≤ θj and θj ≤ θk imply θi ≤ θk ) . Then , Θ is called a partially ordered set . Furthermore , if every pair of elements are comparable ( θi ≤ θj or θj ≤ θi for all i , j ) , Θ is called a chain or linearly ordered set . An order describes ranks or priorities of classes . For example , in age estimation , θi may represent the age class of i-year-olds . Then , θ14 ≤ θ49 represents that 14-year-olds are younger than 49-year-olds . As mentioned previously , it is less easy to tell the older one between people of different genders . An algorithm , hence , may compare a subject with reference subjects of the same gender only . In such a case , each age class θi represents two subclasses θfemalei and θ male i of different types , and the algorithm compares only subjects of the same type . Lim et al . ( 2020 ) assumed that subclasses of different types are incomparable and thus the set of subclasses is the union of k disjoint chains , where k is the number of types . However , in many ranking applications , objects of different types can be compared ( although less easily than those of the same type are ) . Thus , instead of assuming incomparability across chains , we assume that there is a total order on Θ = { θ1 , θ2 , . . . , θm } , in which each class θi consists of k types of subclasses , and that object instances of the same type are more easily compared than those of different types . Suppose that n training instances in X = { x1 , x2 , . . . , xn } are given . Also , suppose that there are m ranks and the ground-truth rank of each instance is known . In this sense , X contains ordered data . The problem is twofold . The first goal is to decompose the whole instances X into k disjoint clusters { Cj } kj=1 in which instances are more easily compared ; X = ⋃k j=1 Cj ( 1 ) where Ci∩Cj = ∅ for i 6= j . In other words , we aim to partition the ordered data inX into k clusters , by grouping them according to their characteristics unrelated to their ranks . These characteristics , which tend to remain the same even when an object experiences rank changes , are referred to as ‘ identity ’ features in this work . For example , in age estimation , genders or races can be identity features . However , we perform the clustering without any supervision for identity features . Notice that instances within a cluster would be compared more easily than those across clusters , since they have similar identity features . The number k of clusters is assumed to be known a priori . Impacts of k on the clustering performance are discussed in Appendix B.7 . The second goal is to assign an unseen test instance into one of the clusters and determine its rank by comparing it with reference instances within the cluster . To achieve these goals , we propose the ORID network and the DRC algorithm .

Summary: The novelty of the network structure is marginal. The decomposition way of feature is very common in computer vision. Just utilizing the latent vector of the encoder with only the comparator loss to decompose the feature into two feature types is limited. The authors should show the visual differences between these two feature types. The expression of the article is very clear, but some basic theories need not be explained in detail (Such in Section 3.4)