Order Learning and Its Application to Age Estimation
Authors: Kyungsun Lim
Nyeong-Ho Shin
Young-Yoon Lee
Chang-Su Kim
Venue:  (2020)
Keywords: Order learning
age estimation
aesthetic assessment

Abstract: We propose order learning to determine the order graph of classes, representing ranks or priorities, and classify an object instance into one of the classes. To this end, we design a pairwise comparator to categorize the relationship between two instances into one of three cases: one instance is `greater than,' `similar to,' or `smaller than' the other. Then, by comparing an input instance with reference instances and maximizing the consistency among the comparison results, the class of the input can be estimated reliably. We apply order learning to develop a facial age estimator, which provides the state-of-the-art performance. Moreover, the performance is further improved when the order graph is divided into disjoint chains using gender and ethnic group information or even in an unsupervised manner.

Published as a conference paper at ICLR 2020
ORDER LEARNING AND ITS APPLICATION TO AGE ESTIMATION
Kyungsun Lim, Nyeong-Ho Shin, Young- Young-Yoon Lee, and Chang-Su Kim School of Electrical Engineering, Korea University and Samsung Electronics Co., Ltd (kalim, nhshin, cak iakimmpi mmmmmlmml... .korea.... ac.kr, yy77lee@gma com
ABSTRACT
We propose order learning to determine the order graph of classes, representing ranks or priorities, and classify object instance into one of the classes. To this end, we design a pairwise comparator to categorize the relationship between two instances into one of three cases: one instance is "greater than,' 'similar to," or 'smaller than' the other. Then, by comparing an input instance with reference in- stances and maximizing the consistency among the comparison results, the class of the input can be estimated reliably. We apply order learning to develop a facial age estimator, which provides the state-of-the-ar performance. Moreover, the per- formance is further improved when the order graph is divided into disjoint chains using gender and ethnic group information or even in an unsupervised manner.
INTRODUCTION
To measure the quality of something, often compare it with other things of a similar kind. Before assigning 4 stars to a film, a critic would have thought, "It better than 3-star films but worse than 5-stars." This ranking through pairwise comparisons is done in various decision processes (Saaty) 1977). It is easier to tell the nearer one between two objects in a picture than to estimate the distance of each object directly (Chen et 2016 Lee& & Kim 2019a). Also, it is easy tell higher pitch between two notes, but absolute pitch is rare ability (Bachem 1955).
Ranking through comparisons has been investigated for machine learning. In learning to rank (LTR), the pairwise approach learns, between two documents, which one is more relevant to query (Liu 2009). Also, in ordinal regression (Frank & Hall, 2001 Li & Lin 2007). to predict the rank of an object, binary classifications are performed to tell whether the rank is higher than series of thresholds or not. In this paper, we propose order learning to learn ordering relationship between objects. Thus, order learning is related to LTR and ordinal regression. However, whereas LTR and ordinal regression assume that ranks form a total order (Hrbacek & Jech 1984). order learning can be used for a partial order well. Order learning is also related to metrie learning (Xing et al., 2003). While metric learning is about whether an object is 'similar to or dissimilar from" another object, order learning is about "greater than or smaller than.' Section 2 reviews this related work.
In order learning, a set of classes, θ = (01, o2.... ..)) ), is ordered, where each class 0g represents one more object instances. Between two classes 0, and 0,. there three possibilities: > 0, or 0, < 0, or neither (i.e. incomparable). These relationships are represented by the order graph. The goal of order learning to determine the order graph and then classify an instance into one of the classes in e. To achieve this, we develop a pairwise comparator that determines ordering relationship between two instances r and y into one of three categories: x 'greater than,' 'similar to,' or 'smaller than' y. Then, we use the comparator to measure an input instance against multiple reference instances known classes. Finally, we estimate the class of the input to maximize the consistency among the comparison results. It is noted that the parameter optimization of the pair- wise comparator, the selection of the references, and the discovery of the order graph are jointly performed to minimize common loss function. Section proposes this order learning.
We apply order learning to facial age estimation. Order learning matches age estimation well, since is casier to tell a younger one between two people than to estimate each person's 's age directly (Chang ct al. 2010; /Zhang et al.) 2017a).. Even when we assume that age classes are linearly ordered, the proposed age estimator performs well. The performance is further improved, when classes

---

Published as a conference paper at ICLR 2020
divided into disjoint chains in supervised manner using gender and ethnic group information or even in an unsupervised manner. Section 4 describes this age estimator and discusses its results. Finally, Section concludes this work.
2 RELATED WORK
Pairwise comparison: It is fundamental problem to estimate the priorities (or ranks) of objects through pairwise comparison. In the classic paper, [Saaty] (1977) noted that, even when direct esti- mates of certain quantities are unavailable, rough ratios between them are easily obtained in many cases. Thus, he proposed the scaling method to reconstruct absolute priorities using only relative priorities. The scaling method was applied to monocular depth estimation (Lee & Kim 2019a) and aesthetic assessment (Lee & Kim 2019b). Ranking from a pairwise comparison matrix has been studied to handle cases, in which the matrix is huge some elements are noisy (Braverman & Mos- sel. 2008: Jamieson & Nowak] 201 Negahban et al.) 2012 Wauthier et al. (013). On the other hand, the pairwise approach to LTR learns, between two documents, which one is more relevant to a query (Ciu 2009: Herbrich et al. 1999 [urges et al. 2003 Tsai et al., 2007). The proposed order learning is related to LTR, since it also predicts the order between objects. But, while LTR sorts multiple objects with unknown ranks and focuses on the sorting quality, order learning compares a single object æ with optimally selected references with known ranks to estimate the rank of x.
Ordinal regression: Ordinal regression predicts an ordinal variable (or rank) of an instance. Sup- pose that 20-year-old is misclassified as 50-year old and a 25-year old, respectively. The former error should be more penalized than the latter. Ordinal regression exploits this characteristic in the design of classifier or regressor. In Frank Hall (200T) and Li & Lin (2007). a conversion scheme proposed to transform ordinal regression problem into multiple binary classification problems. Ordinal regression based on this conversion scheme has been used in various applications, including age estimation (Chang al. 2010 2011 Niu et al. 2016 Chen et al. 2017) and monoc- ular depth estimation (Fu et al. 2018). Note that order learning is different from ordinal regression. Order learning performs pairwise comparison between objects, instead of directly estimating the rank of each object. In age estimation, ordinal regression based on the conversion scheme is con- cerned with the problem, "Is person's age bigger than a threshold 0?" for each 0. In contrast, order learning concerns "Between two people, who is older?" Conceptually, order learning is easier. Technically, if there are N ranks, the conversion scheme requires N 1 binary classifiers, but order learning needs only single ternary classifier. Moreover, whereas ordinal regression assumes that ranks form a total order, order learning can be used even the case of partial order (Hrbacek & Jech 1984).
Metric learning: A distance metric can be learned from examples of similar pairs of points and those of dissimilar pairs (Xing al., 2003). The similarity depends
application and is implic- itly defined by user-provided examples. If learned metric generalizes well to unseen data, can be used to enforce the desired similarity criterion in clustering (Xing et al. 2003), classification (Wein- berger et al. 2006), or information retrieval (McFee & Lanckriet 2010). Both metric learning and order learning learn important binary relations in mathematics: metric and order (Hrbacek & Jech, 1984). However, a metric decides whether an object x is similar to or dissimilar from another object y. whereas an order tells whether is greater than or smaller than y. Thus, a learned metric is useful for grouping similar data, whereas a learned order is suitable for processing ordered data.
Age estimation: Human ages can be estimated from facial appearance (Kwon & da Vitoria Lobo 1994). Geng et al. (2007) proposed the aging pattern subspace, and Guo et al. (2009) introduced biologically inspired features age estimation. Recently, deep learning has been adopted for age estimation. Niu et al. (216)) proposed OR-CNN for age estimation, which is an ordinal regressor using the conversion scheme. Chen et al. (017)) proposed Ranking-CNi -CNN, which is another ordinal regressor. While OR-CNN uses common feature for multiple binary classifiers, Ranking- CNN employs a separate CNN to extract a feature for each binary classifier. [Ta et al. (2018) grouped adjacent ages via the group-n encoding, determined whether a face belongs to each group, and com- bined the results to predict the age. [an et al. (2018) proposed the mean-vant variinnce loss to train a CNN classifier for age estimation. [Shen et al. (2018) proposed the deep regression forests for age esti- mation. Zhang et al. (2019) developed a compact age estimator using the two-points representation. Also, Li et al. 2019 proposed continuity-aware probabilistic network for age estimation.
2

---

Published as a conference paper at ICLR 2020
2
2
3
3
3
3
9
5
5
(a)
Figure I: Examples of order graphs, in which node precedes node m (n + m), if n divides m. For clarity, self-loops for reflexivity and edges deducible from transitivity are omitted from the graphs.
ORDER LEARNING
3.1
WHAT IS ORDER?
Let us first review mathematical definitions and concepts related to order. An order (Hrbacek & Jech, 1984. Bartle 1976,, often denoted by S. is binary relation on a set = { 02 8n that satisfies the three properties of
Reflexivity: 02 S 0; for every 0; E
Antisymmetry: If 0; S 03 and 0j S 0, then 0: == = 0j;
Transitivity: If 0; S 0g and 0z S 0s. then 0; S Oz.
In real-world problems, an order describes ranks or priorities of objects. For example, in age esti- mation, 0 $ 0, means that people in age class 0; look younger than those in fj-
We may use the symbol -,, instead of <. to denote an order on a finite set θ Then, the order can be represented by directed graph (Gross & Yellen 2006) using elements in @ nodes. If 0; - 0j. there a directed edge from node 0; to node 0j. The order graph is acyelic because of antisymmetry and transitivity. For example, for n. m N, let + m denote that m is a multiple n. Note that it is an order on any subset of N. Figure a) is the graph representing this order on (1....,9}.
Elements 0g and 0j are comparable if + 0, 0, + 0g, or incomparable otherwise. Fig- ure [a), 6 and 8 are incomparable. In age estimation, it is difficult to compare apparent ages of people in different ethnic groups of different genders.
An order on set θ is total (or linear) if all elements in θ are comparable to one another. In such a case, θ called linearly ordered set. In some real-world problems, orders are not linear. this work, subset O of θ is referred to as a chain, if O. is linearly ordered and also maximal, Žee there no proper superset of O that linearly ordered. In Figure [ [)), nodes 1, 4, and 8 form a chain. In Figure [b), the entire set is composed of three disjoint chains.
3.2 ORDER LEARNING BASICS
Let θ = (0,,02.... 6n) be an ordered set of classes, where each class 0 represents one or more object instances. For example, in age estimation, age class 11 is the of l-year-olds. The objec- tive of order learning is to determine the order graph, such as Figure I[a] or (b), and categorize an object instance into one of the classes. However, in many cases, order graphs are given explicitly or obvious from the contexts. For example. in quality assessment, there are typically five classes (poor satisfactory - good + very good excellent). forming a single chain. Also, age estimation, suppose that an algorithm first classifies person's gender into female or male and then estimates the age differently according the gender. this case, implicitly, there are separate age classes for each gender, and the age classes compose two disjoint chains similarly Figure l[b). Thus, in this subsection, we assume that the order graph is already known. Also, given an object instance, assume that the chain which the instance belongs is known. Then, attempt cat- egorize the instance into one the classes the chain. Section (.4 will propose the order learning in the case of an unknown order graph, composed of disjoint chains.
Instead of directly estimating the class of each instance, we learn pairwise ordering relationship be- tween two instances. Let O. (0, 1,..., N- be chain, where N is the number of classes. Let
3

---

Published as a conference paper at ICLR 2020
CNN feature extractor
2 y
Shared parameters (Stamese)
Temary classifier
23 1 <
CNN festure extractor
Figure 2: Illustration of the pairwise comparator, where ④ denotes concatenation.
ar and y be two instances belonging to classes in Or. Let 0(-) denote the class of an instance. Then,
and y are compared and their ordering relationship is defined according to their class difference as
I > y if ((x) - 0(y) > T, if (0(z) - 0(y) TT
(1)
y
(2)
I ェマリ if ((x) - 0(v) <-.,
(3)
where 7 is threshold. To avoid confusion, use ">_ 00, <" for the instance ordering, while '>, =, <' for the class order. In practice, the categorization in (I(3) is performed by a pairwise
comparator in Figure 2 which consists of Siamese network and ternary classifier (Lee & Kim 2019b). To train the comparator, only comparable instance pairs are employed.
We estimate the class (xx) of test instance by comparing it with reference instances Ymn 0 S m S M - 1, where M is the number of references. The references are selected from training data such that they are from the same chain as x. Gíven x and ym. the comparator provides one of three categories ">, Re <² as result. Let 0' be an estimate of the true class ((r). Then, the consistency between the comparator result and the estimate is defined as
Ocon (z. Vm: 6¹)
(4)
[x > ym] [8' (Vm)) > + [x> ymm [10 ((ym)) ST] + [ x ym] [ - ((ym) <
-r]
where [-] is the indicator function. The function deon (I, ym, 0') returns either 0 for an inconsistent case or 1 for a consistent case. For example, suppose that the pairwise comparator declares x ymu but - (ym) > r. Then, ocon (z, ym. 8¹) = 0- 1-1 + 0-0 +-+0+110 = O. Due to a possible classification error of the comparator, this inconsistency may occur even when the estimate 6' equals the true class (xr). To maximize the consistency with all references, we estimate the class of x by
M-1
OMC = -arg max
Σ
Doon (x,, Mm, 0'),
(5)
C'E
m7200
which is called the maximum consistency (MC) rule. Figure Billustrates this MC rule.
It is noted that ">, R3, <" is not an mathematical order. For example, if 0(x) + f 0(y) = 0(2) 3r. then x R y and y R but < This is impossible in an order. More precisely. due to the quantization effect of the ternary classifier in [ '>, R2, X" is quasi-transitive (Sen) 1969).
and 'x=" is symmetric but intransitive. We use this quasi-transitte relation to categorize instance into one of the classes, on which a mathematical order is well defined.
3.3
ORDER LEARNING SUPERVISED CHAINS
3.3.1
SINGLE-CHAIN HYPOTHESIS (1CH)
In the simplest case of ICH, all classes form a single chain +, = (0, 1,..,, N - 13. For example, in 1CH age estimation, people's ' ages are estimated regardless of their ethnic groups or genders.
We implement the comparator in Figure using CNNs, as described in Section 4.1 Let q (4a 4i 42 ay, be the one-hot vector, indicating the ground-truth ordering relationship between training instances v and y. Specifically, (1, 0, 0). 1,,, and (0, 0, 1) represent æ y, x y, and r x y. Also, n ay zy 22 is the corresponding softmax probability vector of the comparator. We train the comparator to minimize the comparator loss
.P P2
= -E TET VER
log P7
(6)
4

---

Published as a conference paper at ICLR 2020
=
=9
Class Plm)
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
0
2
4
10
11
12
13
14
Figure 3: Consistency computation in the MC rule: is illustrated how to compute the sum in 5 for two candidates 0' 7 and 9. Each box represents reference 1/m- There are 5 references for each class in {0. ..... 14]. Comparison results are color-coded (yellow for x > Ym. gray for I = ym. and green for
ym.). The bold black rectangle encloses the references satisfying ( - {(ym)] ) т, where t = 4. The computed consistency @con (2, ym f') in (5) is written within the box. For O' = there six inconsistent boxes. For 0' = 9, there are 24 such boxes. In this example, 0' minimizes the inconsistency, or equivalently maximizes the consistency. Therefore, (Mc(2) 7.
where T is the of all training instances and R C T is the set of reference instances. First, we initialize R = T and minimize tco via the stochastic gradient descent. Then, we reduce the reference set R by sampling references from T. Specifically, for each class in O.. we choose M/N reference images to minimize the same loss feo. where M is the number of all references and N is the number of classes. In other words, the reliability score a reference candidate is defined as
a(v) = ΣΣ ZET ==0
29 logp 4
and the M/N candidates with the highest reliability scores are selected. Next, after fixing the reference set R. the comparator trained to minimize the loss loo Then, after fixing the comparator parameters, the reference set R is updated to minimize the same loss tco. and so forth.
In the test phase, an input instance is compared with the references and its class is estimated using the MC rule in (5)
3.3.2 K-CHAIN HYPOTHESIS (KCH)
In KCH, we assume that classes form K disjoint chains, as in Figure [b). For example, in the supervised 6CH for age estimation, predict a person's 's age according to the gender in {female, male) and the ethnic group in (African, Asian, European]. Thus, there are chains total. this case, people in different chains are assumed to be incomparable for age estimation. It is supervised, since gender and ethnic group annotations are used to separate the chains. The supervised 2CH or 3CH also be implemented by dividing chains by genders only ethnic groups only.
The comparator is trained similarly to 1CH. However, in computing the comparator loss in (6), a training instance x: and reference y constrained to be from the same chain. Also, during the test, the type (or chain) of test instance should be determined. Therefore, a K-way type classifier trained, which shares the feature extractor with the comparator in Figure 2 and uses additional fully- coly-cconeettd (FC) layers. Thus, the overall loss is given by
i = feo + lyy
where feo is the comparator loss and by is the type classifier loss. The comparator and the type classifier are jointly trained to minimize this overall loss E.
During the test, given an input instance, determine its chain using the type classifier, and compare it with the references from the same chain, and then estimate its class using the MC rule in (5)
3.4 ORDER LEARNING UNSUPERVISED CHAINS
This subsection proposes algorithm separate classes into K disjoint chains when there supervision or annotation data available for the separation. First, we randomly partition the training set T into To, Ti..... Tx where T .T...UTT- and Tg 0 0 for k: of I. Then, similarly
5

---

However, as described in Algorithm| I we modify this iterative algorithm by including the mem- bership refinement step in lines 10 12. Specifically, train K-way type classifier using This refine- Published as a conference paper at ICLR 2020
Algorithm Order Learning with Unsupervised Chains
Input: 7 training set of ordinal data, K of chains, N of classes in each chain, and M of references in each chain
1: Partition 7 randomly into To...., Tk- 1 and train pairwise comparator for each chain do
b Reference Selection (Rt)
3: From Th. select M/N references y with the highest reliability scores ax
4: end for
5: repeat
6: for each instance x do
Membership Update (Tx)
Assign to Tex, where k* arg maxy Be (1x) subject to regularization constraint
8: end for
9: Fine-tune the comparator and train type classifier using Tu...., TK-1 to minimize E tco + lay Membership Refinement (Tk,
10:
for each instance do
Assign it Tel where k' is its type classification result 12: end for
13:
for each chain k do
b Reference Selection (Rx)
14:
From Tx. select M/N references with the highest reliability scores (1)(v) end for
15:
until convergence or predefined number of iterations
Output: Pairwise comparator, type classifier, reference Ro...., RK-1
6 the comparator loss leo can be written as
K-1 lco = <-=-EEEX
K Σ Σ
K
log
ax(y) = Σ
3x(x))
(9)
K= TETH yER
== MERx
TET
where R2 C To is the set of references for the kth chain, 0x (v) = =EZET E, 9 logp; is the reliability of reference y in the kth chain, and (x(x) EVER, Eg 9 log is the affinity of an instance x the references in the kth chain. Note that (2) - -E "(q""llp") where D is the Dallback-Leibler distance [Cover & Thomas, 2006). Second, after fixing the chain membership Te for each chain k; we select references y to maximize the reliability scores os (y)- These references form Rg. Third, after fixing Ro,..., RK--1, we update the chain membership To ..... TK-1. by assigning each training instance to the kth chain that maximizes the affinity score (x). The second and third steps are iteratively repeated. Both steps decrease the same loss loo in (9)
The second and third steps are analogous to the centroid rule and the nearest neighbor rule the K- means clustering (Gersho & Gray 1991), respectively. The second step determines representatives in each chain (or cluster), while the third step assigns each instance to an optimal chain according to the affinity. Furthermore, both steps decrease the same loss alternately.
To...., - Then, we accept the type classification results to refine ment is necessary because the type classifier should be used in the test phase to determine the chain of an unseen instance. Therefore, it is desirable to select the references also after refining the chain membership. Also, in line 7, we assign an instance x to maximize (x(x)) only, some classes may be assigned too few training instances, leading to data imbalance. To avoid this, we enforce the regularization constraint so that every class assigned at least a predefined number of instances. This regularized membership update is described in Appendix A
4 AGE ESTIMATION
We develop age estimator based on the proposed order learning. Order learning is suitable for age estimation, since telling the older one between two people is easier than estimating each person's 's age directly (Chang et al. 2010 Zhang et al. (017a).
4.1 IMPLEMENTATION DETAILS is less difficult to distinguish between a 5-year-old and 10-year-old than between a 65-year- old and 70-year-old. Therefore, in age estimation, replace the categorization based the

---

Published as a conference paper at ICLR 2020
Table I: A summary of the balanced dataset, formed from MORPH II, AFAD, and UTK. An element
m means that, out of m images in the original dataset, n images are sampled for the balanced dataset.
MORPH II
AFAD
UTK
Balanced
Male
Female
Male
Female
Male
Femalc
Male
Female
African
4,022
4,44G
0
2.047
1.871
6,069
6,317
36,772
5,748
0
2,319
2,209
Asian
17
5,000
1.200
6.168
6,217
153
17
100,752
63,680
1,375
1,839
European
1,852
2.602
4.457
6,339
6.039
7,992
2,002
0
3,477
4,601
arithmetic difference in 1~ 3) with that based on the geometric ratio as follows.
x > y
if log ((x:) log ((y) > Tage:
(10)
x Rd y if llog ((x) log ((v) Σ Tages
(11)
I y if log ((x)) log 0(y) < -Tges
(12)
which represent 'older,' 'similar,' and "younger.' The consistency in (4) is also modified accordingly.
There are reference images for each age class within range [15, 80) in this work (M = 330, .N- = 66). Thus, test image should be compared with 330 references. However, step approach, which does at most 130 comparisons but performs as good as the method using 330 comparisons. The two-step estimation is employed in all experiments. It described in Appendix B
develop two-
We align all facial images using Sengnne (Zhang et al. 2014) and resize them into 256 x 256 x Then, we crop resized image into 224 224 x 3. For the feature extractors in Figure we use VGG16 without the FC layers (Simonyan & Zisserman 2014). They yield 512-channel feature vectors. Then, the vectors are concatenated and input the ternary classifier, which has three FC layers, yielding 512-, 512-, and 3-channel vectors sequentially. The 3-channel vector is normalized to the softmax probabilities of the three categories '>, 25 <.' In [00-11 Fage is set to 0.1.
(2).
In KCH with K 2 2, the type (or chain) of a test image should be determined. Thus, we design type classifier, which shares the feature extractor with the comparator. Similarly to the ternary classi- fier, the type classifier uses three FC layers, yielding 512-, 512-, and K K-channel vectors sequentially. The comparator and the type classifier are jointly trained.
To initialize the feature extractors, we adopt the VGG16 parameters pre-trained on ImageNet (Deng) jet al. (009). We randomly initialize all the other layers. We update the parameters using the Adam optimizer (Kingma & Ba 2014). We set the learning rate 10 for the first 70 epochs. Then, select references for each age class. Using the selected references, we fine-tune the network with a learning rate of 10-5, We repeat the reference selection and the parameter fine-tuning up to 3 times.
In the case of unsupervised chains, we enforce the regularization constraint (line 7 in Algorithm T By default, for each age, all chains are constrained to be assigned the same number of training images. there are training images of 0-year-oldd olds, the age classes 0 in the K chains are assigned L/K K images, respectively, according the affinity scores Bx (a) by Algorithm 2 in Appendix A
4.2 DATASETS AND EVALUATION METRICS
MORPH II (Ricanek & Tesafaye 2006 is the most popular age estimation benchmark, containing about 55,000 facial images in the age range [16, 77). IMDB-WIKE (Rothe et al. 2018) is another dataset containing about 500,000 celebrity images obtained from IMDB and Wikipedia. It some- times used to pre-train age estimation networks. Optionally, we also select 1. 1,50,000 clean data from IMDB-WIKI to pre-train the proposed pairwise comparator.
Although several facial age datasets are available, most are biased to specific ethnic groups gen- ders. Data unbalance restricts the usability and degrades the generalization performance. Thus, we form a 'balanced dataset" from MORPH II, AFAD (Niu et al. 2016). and UTK (Zhang et al. 2017b). Table I shows how the balanced dataset is organized. Before sampling images from MORPH II, AFAD, and UTK, rectify inconsistent labels by following the strategy in Yip et al. (0118) For each combination gender in (female, male) and ethnic group in (African, Asian, European), we sample about 6,000 images. Also, during the sampling, we attempt to make the age distribution as
7

---

Published as a conference paper at ICLR 2020
Table 2: Performance comparison on the MORPH II dataset: * means that the networks are pre- trained on IMDB- WIKI, and the values are read from the reported CS curves or measured by experiments. The best results are boldfaced, and the second best ones are underlined.
Setting A
Setting B
Setting C (SE)
Seting D (RS)
MAE CS(%)
MAE
CS(%)
MAE
CS(%)
MAE
CS(%)
OHRank (Chang ct al.
2011
6.07
56.3
OR-CNN Niu al.
2016
3.27
73.0f
Ranking CNN (Chen et al.
2017
2.96
85.01
DMTL Han et al. 2018
3.00
85.3
DEX* Rothe ct
2018
2.68
DRFs Shen
2018)
2.91
82.9
2.98
2.17
91.3
MO-CNN ( (Tan et al.
2018
2.52
85.0
2.70
83.0
MV [Pan et al.
2018
2.80
87.0
2.41
90.0
MV*
2.79
2.16
BridgeNet* (Li al.
2019
2.38
91.0*
2.63
86.0
Proposed (ICH)
2.69
89.1
3.00
85.2
2.76
88.0
2.32
92.4
Proposed* (ICH)
2.41
91.7
2.75
88.2
2.68
88.8
2.22
93.3
uniform as possible within range [15, 80]. The balanced dataset is partitionnd into training and test subsets with ratio 8 2.
For performance assessment, we calculate the mean absolute error (MAE) (Lanitis et al. 2004) and the cumulative score (CS) (Geng al. 2006). MAE is the average absolute error between predicted and ground-truth ages. Given a tolerance level I, CS computes the percentage of test images whose absolute errors are less than or equal to Z. In this work, is fixed to 5, done in Chang et al. (2011). Han et al. (2018), and Shen et al. (2018).
4.3 EXPERIMENTAL RESULTS
Table 2 compares the proposed algorithm (ICH) with conventional algorithms on MORPH II. As evaluation protocols for MORPH II, we use four different settings, including the f- subject- exclusive (SE) and the 5-fold random split (RS) (Chang et al., 2010 Guo & Wang 2012). Ap- pendix C. describes these four settings detail and provides an extended version of Table
OHRank, OR-CNN, and Ranking-CNi -CNN are all based on ordinal regression. OHRank uses traditional features, yielding relatively poor performances, whereas OR-CNN and Banking-CNN use CNN fea- tures. DEX, DRFs, MO-CNN, MV, and BridgeNet employ VGG16 as backbone networks. Among them, MV and BridgeNet achieve the state-of-the- results, by employing the mean-variance loss and the gating networks, respectively. The proposed algorithm outperforms these algorithms in set- ting C, which the most challenging task. Furthermore, in terms of CS, the proposed algorithm yields the best performances in all four settings. These outstanding performances indicate that order learning is an effective approach to age estimation.
In Table analyze the performances of the proposed algorithm on the balanced dataset according to the number of hypothesized chains. We also implement and train the state-of-the-art MV on the balanced dataset and provide its results using supervised chains.
Let us first analyze the performances of the proposed algorithm using 'supervised' chains. The MAE and CS scores on the balanced dataset are worse than those MORPH II, since the balanced dataset contains more diverse data and thus is more challenging. By processing facial images sepa- rately according the genders (2CH), the proposed algorithm reduces MAE by 0.05 and improves CS by 0.2% in comparison with ICH. Similar improvements are obtained by 3CH or 6CH, which consider the ethnic groups only or both gender and ethnic groups, respectively. case of MV, multi-chain hypotheses sometimes degrade the performances; e.g., MV (6CH) yields a lower CS than MV (ICH). Regardless the number of chains, the proposed algorithm trains a single comparator but uses different set of references for each chain. The comparator is ternary classifier. In contrast, MV (6CH) should train six different age estimators, each of which is a 66-666 way classifier, to handle different chains. Thus, their training is more challenging than that of the single ternary classifier. Note that, for the multi-chain hypotheses, the proposed algorithm first identifies the chain of test image using the type classifiers, whose accuracies about 98%. In Table these
contrast, in the
8

---

Published as a conference paper at ICLR 2020
Table 3: Comparison of the proposed algorithm with MV on the balanced dataset. In MV and the
supervised algorithm, multi-chain hypotheses divide data by the genders and/or the ethnic groups.
MAE
CS(%)
ICH
2CH
3CH
6CH
ICH
2CH
3CH
6CH
MV (Pan et al. 2018) Proposed (supervised)
4.49
4.52
4.44
4.40
69.9
70.1
70.3
69.6
4.23
4.18
4.19
4.18
73.2
73.4
73,4
73.4
Proposed (unsupervised)
4.16
4.17
4.16
74.0
73.9
74.0
Toal imagg
Reference iagges
Male
Asian
17 19
Figure 4: Age estimation in 6CH: Only the references of ages from 15 to 50 are shown. Comparison results are color-coded. Cyan, yellow, and magenta mean that the test subject is older than (>)) similar to (Rx), and younger than (-) a reference. The age estimated correctly as 22.
type classifiers are used to obtain the results of the proposed algorithm, whereas the ground-truth gender and ethnic group of each test image used for MV.
Figure showws how to estimate an age in 6CH. this test, the subject is a 22-year-old Asian male. He is compared with the references who also Asian males. Using the comparison results, the age correctly estimated as 22 by the MC rule in (5)
Table 4 lists the MAE results for each test chain. Europeans yield poorer MAEs than Africans or Asians. However, this is not due to inherent differences between ethnic groups It is rather caused by differences in image qualities. As listed in Table more European faces are sampled from UTK. The UTK faces were crawled from the Internet and their qualities are relatively low. Also, from the cross-chain test results using 6CH, some observations can be made:
Except for the As-F test chain, the lowest MAE is achieved by the references in the same chain.
Eu-M and Eu-F mutually compatible. For Eu-M, the second best performance is obtained by the Eu-F references, and vice versa. On the other hand, some chains, such as Af-M and Eu-F, are less compatible for the purpose of the proposed age estimation.
Table also includes the performances of the proposed algorithm using 'unsupervised" chains. The unsupervised algorithm outperforms the supervised one, which indicates that the gender ethnic group is not the best information to divide data for age estimation. As in the supervised case, 2CH, 3CH, and 6CH yield similar performances, which means that two chains are enough for the balanced set. Compared with MV (ICH), the unsupervised algorithm (2CH) improves the performances significantly, by 0.33 in terms of MAE and 4.1% 1% in terms of CS.
Figure how training images are divided into two chains in the unsupervised 2CH. During the membership update, for each age, each chain is regularized to include least a certain percentage (k) of the training images. In the default mode, the two chains are assigned the same number of images with k 50%. However, Appendix C.3 shows that the performance not very sensitive to k. At 10%, MAE = 4.17 and CS = 73.7%. From Figure we observe
The division of the chains is not clearly related to genders or ethnic groups. Regardless of genders or ethnic groups, about half of the images assigned to chain and the others chain 2.
At 10%, chain 1 mostly consists middle ages, while chain 2 of 10s, 20s, 60s, and 70s.
At バ = 50%, there no such strong age-dependent tendency. But, for some combinations of gender, ethnic group, and age band, it is not equal division. For example, for Asian females, majority of 40s are assigned to chain but a majority of 50s and 60s are assigned to chain 2.
The unsupervised algorithm is designed to divide instances into multiple clusters when gender and ethnic group information unavailable. As shown in Appendix C.3] different k's yield various clustering results. Surprisingly, these different clusters still outperform the supervised algorithm.

---

Published as a conference paper at ICLR 2020
Table 4: Cross-chain tests the balanced dataset (MAEs). For example, in 6CH, when African male references are used to estimate the ages of Asian females, the resultant MAE is 3.82.
Test chain
Method
Reference chain
Af-M
At-F
As-M
As-F
Eu-M
Eu-F
ICH
All
3.87
3.82
3.98
3.79
5.21
4.69
6CH
African-Male
3.85
3.79
4.03
3.82
5.50
5.00
African-Femake
4.02
3.65
4.18
3.85
5.42
5.02
Asian-Mak
3.97
3.75
3.97
3.81
5.48
4.87
Asian-Female
4.06
3.78
4.05
3.78
5.69
4.89
European- Male
3.99
3.71
4.02
3,80
5.13
4.66
European-Feman Female
4.45
3.79
4.11
3.77
5.21
4.65
10%
K 50%
chhin mcbon
ACM
AF-F
As-M
As-F
Eu-M
Eu-F
Figure Distributions of training images in the unsupervised algorithm (2CH).
For example, are used to estimate the ages people in chain 1, the average error is 4.6 years. On the contrary, the references in chain mean that people in chain tend to look older than those in chain 2. These 'looking-older people in 20s and 30s compose the blue cluster (chain 1) together with most people in 40s and 50s in Figure In this case, 'looking-olde" people in 20s and 30s are separated from 'looking- 'looking-younger younger' ones by the unsupervised algorithm. This is more effective than the gender-base based ethnic- seree-grrup-aaasse division of the supervised algorithm. Appendix C presents more results age estimation.
10%, let consider the age band of 20s and 30s. If the references in chain 2
used for chain 2, the average error is -5.4 years. These opposite biases
CONCLUSIONS
Order learning was proposed in this work. In order learning, classes form ordered set, and each class represents object instances of the same rank. Its goal is to determine the order graph classes and classify test instance into one of the classes. To this end, designed the pairwise comparator to learn ordering relationships between instances. We then decided the class an instance by comparing it with reference instances in the same chain and maximizing the consistency among the comparison results. For age estimation, it was shown that the proposed algorithm yields the state- of-the-art performance improved when the order graph is divided into multiple disjoint chains.
in the of the single-chain hypothesis. The performance is further
In this paper, assumed that the order graph is composed of disjoint chains. However, there more complicated graphs, egg. Figure Ta), than disjoint chains. For example, it is hard to recognize an infant's sex from its facial image (Porter al.) 1984). But, after puberty, male and female take divergent paths. This can be reflected by an order graph, which consists of two chains sharing common nodes up to a certain age. is an open problem generalize order learning to find an optimal order graph, which is not restricted to disjoint chains.
ACKNOWLEDGEMENTS
This work was supported by 'The Cross-Ministry Giga KOREA Project' grant funded by the Korea government (MSIT) (No. GK19P0200, Development of 4D reconstruction and dynamic deformable action model based hyperrealistic service technology). and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. NRF-201 8R1 BRRAAB3300E A2B3003896).
10

---

Published as a conference paper at ICLR 2020
REFERENCES
A. Bachem. Absolute pitch. J. Acoust. Soc. Am., 2766):1100-1185, November 1955.
Robert G. Bartle. The Elements of Real Analysis. John Wiley & Sons, 2nd edition, 1976.
Mark Braverman and Elchanan Mossel. Noisy sorting without resampling. In Symp. Discrete Algo- rithms, pp. 268-276, 2008.
Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hul- lender. Learning to rank using gradient descent. In ICML, pp. 89-96, 2005.
Kuang-Yo Yu Chang, Chu-Song Chen, and Yi-Ping Hung. A ranking approach for human ages estima- tion based on face images. In ICPR, pp. 3396-3399, 2010.
Kuang-Yo Yu Chang, Chu-Song Chen, and Yi-Ping Hung. Ordinal hyperplanes ranker with sensi- tivities for age estimation. In CVPR, pp. 585-592, 2011.
Shixing Chen, Caojin Zhang. Ming Dong, Jialiang Le, and Mike Rao. Using Ranking-CNN for age estimation. CVPR, pp. 5183-5192, 2017.
Weifeng Chen, Zhao Fu, Dawei Yang, and Jia Deng. Single-image depth perception in the wild. In NIPS, pp. 730-738, 2016.
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley, 2006.
Jia Deng. Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, pp. 248-255 2009.
Eibe Frank and Mark Hall. A simple approach to ordinal classification. In ECML, pp. 145-156, 2001.
Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao. Deep ordinal regression network for monocular depth estimation. CVPR, pp. 2002-2011, 2018.
Xin Geng, Zhi-Hua Zhou, Yu Zhang, Gang Li, and Honghua Dai. Learning from facial aging patterns for automatic age estimation. ACM Multimedia, pp. 307-316, 2006.
Xin Geng, Zhi-Hua Zhou, and Kate Smith-Miles. Automatic age estimation based on facial aging patterns. IEEE Trans. Pattern Anal. Mach. Intell., 29(12):2234-23) 22400, December 2007.
A. Gersho and R. M. Gray. Vector Quantization and Signal Compression. Kluwer Academic Pub- lishers Norwell, 1991.
Jonathan L. Gross and Jay Yellen. Graph Theory and Its Applications. Chapman & Hall, 2nd edition, 2006.
Guodong Guo and Guowang Mu. Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In CVPR, pp. 657-664, 2011.
Guodong Guo and Xiaolong Wang. A study on human age estimation under facial expression changes. In CVPR, pp. 2547-2553, 2012.
Guodong Guo, Guowang Mu, Yun Fu, and Thomas S. Huang. Human age estimation using bio- inspired features. In CVPR, pp. 112-119, 2009.
Hu Han, Anil K. Jain, Fang Wang. Shiguang Shan, and Xilin Chen. Heterogeneous face attribute estimation: A deep multi-task learning approach. IEEE Trans. Pattern Anal. Mach. Intell., 40 (11):2597-2609 November 2018.
Ralf Herbrich, Thore Graepel, and Klaus Obermayer. Support vector learning for ordinal regression. In ICANN, pp. 97-102, 1999.
Karel Hrbacck and Thomas Jech. Introduction to Set Theory. Marcel Dekker, Inc., 2nd edition, 1984.
11

---

Published as a conference paper at ICLR 2020
Ivan Huerta, Carles Fernández, Carlos Segura, Javier Hermando and Andrea Prati. A deep analysis on age estimation. Pattern Recog. Lett., 68:239-22 229, December 2015.
Kevin G. Jamieson and Robert D. Nowak. Active ranking using pairwise comparisons. In NIPS, pp- 2240-2248, 2011.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 141v:6:412.6980 2014.
Young Ho Kwon and Niels da Vitoria Lobo. Age classification from facial images. In CVPR, pp- 762-767, 1994.
Andreas Lanitis, Chrisina Draganova, and Chris Christodoulou. Comparing different classifiers for automatic age estimation. IEEE Trans. Syst. Man, Cybern. B, Cybern., 34(1):621-62 -628, February 2004.
Jae-Han Lee and Chang-Su Kim. Monocular depth estimation using relative depth maps. In CVPR, pp. 9729-9738, 2019a.
Jun-Tae Lee and Chang-Su Kim. Image aesthetic assessment based on pairwise comparison unified approach to score regression, binary classification, and personalization. In ICCV, pp- 1191-1200, 2019b.
Ling Li and Hsuan-Tien Lin. Ordinal regression by extended binary classification. In NIPS, pp. 865-872. 2007.
Wanhua Li, Jiwen Lu, Jianjiang Feng, Chunjing Xu, Jie Zhou, and Qi Tian. BridgeNet: A continuity- aware probabilistic network for age estimation. In CVPR, pp. 1145-1154, 2019.
Tie-Yan Liu. Learning to rank for information retrieval. Foundations and Trends in Informatioon Retrieval, 3(3):225-331, 2009.
Brian McFee and Gert Lanckriet. Metric learning to rank. ICML, pp. 775-782, 2010.
Sahand Negahban, Sewoong Oh, and Devavrat Shah. Iterative ranking from pair-wise comparisons. In NIPS, pp. 2474-2482. 2012.
Zhenxing Niu, Mo Zhou, Le Wang. Xinbo Gao, and Gang Hua. Ordinal regression with multiple output CNN for age estimation. In CVPR, pp. 4920-4928, 2016.
Hongyu Pan, Hu Han, Shiguang Shan, and Xilin Chen. Mean-variance loss for deep age estimation from face. CVPR, pp. 5285-5294, 2018.
Gabriel Panis, Andreas Lanitis, Nicholas Tsapatsoulis, and Timothy F. Cootes. Overview of research on facial ageing using the FG-NET ageing database. IET Biometrics, 5(2):37-46, 2016.
Richard H. Porter, Jennifer M. Cemoch, and Rene D. Balogh. Recognition of neonates by facial- visual characteristics. Pediatrics, 74(4):501-504, October 1984.
Karl Ricanek and Tamirat Tesafaye. MORPH: A longitudinal image database of normal adult age- progression, In FGR, pp. 341 141--45,70 2006.
Rasmus Rothe, Radu Timofte, and Luc Van Gool. Deep expectation of real and apparent age from single image without facial landmarks. Int. J. Comput. Vis., 126(2):444-177, April 2018.
Thomas I.. Saaty. A scaling method for priorities hierarchical structures. J. Math. Psychol., 15 (3):234-281, June 1977.
Amartya Sen. Quasi-transitivityy rational choice and collective decisions. Rev. Econ. Stud., 36(3): 381-393, July 1969.
Wei Shen, Yilu Guo, Yan Wang, Kai Zhao, Bo Wang, and Alan Yuille. Deep regression forests for age estimation. In CVPR, pp. 2304-2313, 2018.
12

---

Published as a conference paper at ICLR 2020
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2014.
Zichang Tan, Shuai Zhou, Jun Wan, Zhen Lei, and Stan Z. Li. Age estimation based on a single network with soft softmax of aging modeling. In ACCV, 2016.
Zichang Tan, Jun Wan, Zhen Lei, Ruicong Zhi, Guodong Guo, and Stan Z. Li. Efficient group-n encoding and decoding for facial age estimation. IEEE Trans. Pattern Anal. Mach. Intell., 40(11): 2610-2623, November 2018.
Ming-Feng Tsai, Tan Liu, Tao Qin, Hsin-Hsi Chen, and Wei-Ying Ma. FRank: A ranking method with fidelity loss. In SIGIR, pp. 383-390, 2007.
Fabian L. Wauthier, Michael I. Jordan, and Nebojsa Jojic. Efficient ranking from pairwise compar- isons. In ICML, pp. 109-117,2011
Kilian Q. Weinberger, John Blitzer, and Lawrence K. Saul. Distance metric learning for large margin nearest neighbor classification. In NIPS, pp. 1473-1480, 2006.
Eric P. Xing, Andrew Y. Ng. Michael I. Jordan, and Stuart Russell. Distance metric learning with application to clustering with side-information. In NIPS, pp- 521-528, 2003.
Dong Yi, Zhen Lei, and Stan Z. Li. Age estimation by multi-scale convolutional network. In ACCV, 2014.
Benjamin Yip. Garrett Bingham, Katherine Kempfert, Jonathan Fabish, Troy Kling, Cuixian Chen, and Yishi Wang. Preliminary studies on large face database. In ICBD, pp. 2572-2579, 2018.
ByungIn Yoo, Youngjun Kwak, Youngsung Kim, Changkyu Choi, and Junmo Kim. Deep facial age estimation using conditional multitask learning with weak label expansion. IEEE Signal Process. Lett., 25(6):808 812, June 2018.
Chao Zhang, Shuaicheng Liu, Xun Xu, and Ce Zhu, C3AE: Exploring the limits of compact model for age estimation. In CVPR, pp. 12587-12596, 2019.
Jie Zhang, Shiguang Shan, Meina Kan, and Xilin Chen. Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In ECCV, pp. 1-16, 2014.
Yunxuan Zhang, Li Liu, Cheng Li, and Chen Change Loy. Quantifying facial age by posterior of age comparisons. BMVC, 2017a.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/regression by conditional adversarial autoencoder. In CVPR, pp. 5810-5818, 2017b.
13

---

Published as a conference paper at ICLR 2020
A REGULARIZED MEMBERSHIP UPDATE
During the chain membership update in Algorithm I we assign an instance J to chain & to maximize 3x (a) subject the regularization constraint. As mentioned in Section 4.1 in age estimation, this regularization is enforced for each age. Let X' denote the set of 0-year-olds for a certain 0. Also, let K {0, 1........ K 1t be the set chains. Suppose that we should assign at least a certain ..... number (L) of instances in X' to each chain. This is done by calling RegularAssignn X, L) in Algorithm [ which is a recursive function. Algorithm 2 yields the membership function c(z) as output. For example, c(x) = means that belongs to chain 1.
Algorithm 2 Regular Assign(K, X, L)
Input: K. set of chains, X' = set of instances, and L minimum number
I: for each E KC do
Initialize chains
2: X2 0
3: end for
for each
do
Ireeullar partitioning
c(z) = arg max (z(
6: X((()) Xn(es) U {
7: end for
8: Km arg keK [&4
Chain of the minimum size
1xx... then
10: return
11: else
12:
X= X -Kim
13: while |Nx... do
Increase Xkm
maxxEx Skm (x)
15:
x= (2¹)
16:
Nt.. = Xtm (a')
17:
end while RegularAssign((C- {kim }. X. L)
18:
b Recursion
19:
end if
Output: Membership function c(x)
B TWO-STEP ESTIMATION
There are 5 reference images for each age within range [15, 80] in this work. Thus, for the age estimation of test image using the MC rule in (5), the test image should be compared with M = 330 reference images. However, we reduce the number comparisons using a two-step approach. First, the test image is compared with the 35 references of ages 15, 25, 75 only, and a rough age estimate 0, is obtained using the MC rule. Second, it is compared with the 105 references of all ages within 10, 10, 0 + 10). and the final estimate 02 is obtained. Since there at least 10 common references in the first and second steps, the two-step estimation requires at most 130 comparisons.
14

---

Published as a conference paper at ICLR 2020
C MORE EXPERIMENTS
C.1 PERFORMANCE COMPARISON ON MORPH II
Four experimental settings are used for performance comparison MORPH II (Ricanck & Tesafaye 2006).
Setting A: 5,492 images of Europeans are randomly selected and then divided into training and testing sets with ratio 8:2 (Chang et al. 2011)
Setting B: About 21,000 images randomly selected, while restricting the ratio between Africans and Europeans to 1:1 and that between females and males to 1:3, They are vided into three subsets (S1, S2, S3). The training and testing are done under two sub- settings (Guo & Mu 2011).
- (BI) training on S1, testing on S2 S3
-
-B2) training on S2, testing on S1 S3
Setting (SE): The entire dataset is randomly split into five folds, subject to the con- straint that the same person's 's images should belong to only one fold, and the 5-fold cross- validation is performed.
Setting D (RS): The entire dataset is randomly split into five folds without any constraint, and the 5-fold cross-validation is performed.
Table ils an extended version Table 2 It includes the results of more conventional algorithms.
Table 5: Performance comparison on the MORPH II dataset: means that the networks are pre-
trained IMDB-WIKI, and the values read from the reported CS curves measured by experiments. The best results are boldfaced, and the second best ones are underlined.
Setting A
Setting B
Setting C (SE)
Setring D (RS)
MAE
CS(%)
MAE CS(%)
MAE
CS(%)
MAE
CS(%)
RED-SVM (Chang al.
2010
6.49
49.01
OHRank Chang al. 2011)
6.07
56.3
KPLS (Guo & Mu 201T)
4.18
CPLF YY et al. 2014)
3.63
Huerta et al. (Huerta et all
2015
3.88
OR-CNN (Nu et al.
2016
3.27
73.01
Tan et al.
[an et al. 2016
3.03
Ranking CNN (Chen et
2017
2.96
85.0
DMTL (Han et al.
2018
3.00
85.3
DEX (Rothe et al.
2018
3,25
DEX*
2.68
CMT Yoo al. 2018
2.91
DRFs [Shen et al 2018
2.91
82.9
2.98
2.17
91.3
MO-CNN Tan et 2018
2.93
83.0
2.86
82.0
MO-CNN*
2.52
85.0
2.70
83.0
MV Pan et al. 2018
2.80
87.0
2.41
90.0
MV*
2.79
2.16
C3AE (Zhang
2019)
2.78
C3AF#
2.75
BridgeNet* (Li et al.
2019
2.38
91.0
2.63
86.0
Proposed (ICH)
2.69
89.1
3.00
85.2
2.76
2.32
92.4
Proposed* (ICH)
2.41
91.7
2.75
88.2
2.68
88.8
2.22
93.3
15

---

Figure 6: MAEs of the proposed algorithm (1CH) and MV on FG-NET in terms of test age. Published as a conference paper at ICLR 2020
C.2 GENERALIZATION PERFORMANCE OF COMPARATOR ON FG-NET
We assess the proposed age estimator (ICH) on the FG-NET database (Panis et al., 2016). FG-NET relatively small dataset, composed of 1,002 facial images of 82 subjects. Ages range from 0 to 69. For FG-NET, the leave one person out (LOPO) approach is often used for evaluation. other words, perform
on each subject, an estimator is trained using the remaining 81 subjects. Then, the results are averaged over all 82 subjects.
In order to assess the generalization performance, we do not retrain the comparator on the FG-NET data. Instead, we fix the comparator trained on the balanced dataset and just select references from the remaining subjects' faces each LOPO test. For the comparator, the arithmetic scheme in is tested as well as the default geometric scheme in (10
For comparison, MV (Pnn et al. 2018) is tested, but it trained for each LOPO test.
summarizss the comparison results. MV provides better average performances on the entire age range [0, 69] than the proposed algorithm does. This is because the balanced dataset does not include subjects of ages between 0 and 14. If we reduce the test age range to [15, 69), the proposed algorithm outperforms MV, even though the comparator not retrained. These results indicate that the comparator generalizes well to unseen data, long the training images cover desired age range. Also, note that the geometric scheme provides better performances than the arithmetic scheme.
Table 6: Performance comparison on FG-NET. The average performances over test ages within ranges [0, 69) and [15, 69] are reported, respectively.
to 69
to 69
MAE
CS(%)
MAE
CS(%)
MV
3.98
79.5
6.00
63.7
Proposed (ICH, Geometric Tage 0.15)
8.04
41.4
4.90
64.3
Proposed (ICH, Arithmetic r = 9.26
33.1
5.32
64.1
Figure 6 compares MAEs according to test age. Again, within the covered range [15, 69), the proposed algorithm significantly outperforms MV especially when test subjects are older than 45.
MV
1CH Geometric
40
1CH Arthmotic
30
MAE
20
10
0
0
10
20
30
40
50
60
70
Age
16

---

Published as a conference paper at ICLR 2020
C.3 PERFORMANCE ACCORDING TO
Table 7: MAE and CS performances of the unsupervised algorithm (2CH) on the balanced dataset, according to the minimum percentage (x) constraint during the regularized membership update. The performances are not very sensitive to s. The best performances are achieved the default mode, Le. 50%.
(%)
10
20
30
40
50
MAE
4.17
4.18
4.17
4.16
4.16
CS (%)
73.7
73.6
73.6
73.7
74.0
20%
chain mchaiin
Af-M
Af-F
As-M
As-F
Eu-M
Eu-F
Age
== 30%
Af-M
AF-F
As-M
As-F
Eu-M
Eu-F
305
Age
40%
Af-M
Af-F
As-M
As-F
Eu-M
Eu-F
100
20p
30x
4Cs
506
60
Age
Figure 7: Distributions of training images in the unsupervised algorithm (2CH) at = 20%, 30%, and 40%. From Figures 5 and we see that stronger age-dependent tendencies are observed, as gets smaller.
17

---

Published as a conference paper at ICLR 2020
C.4 PERFORMANCE ACCORDING TO THRESHOLDS AND Tage
The ordering relationship between two instances can be categorized via the arithmetic scheme in (T~ using threshold the geometric scheme in (0 using threshold Tage- Ta- ble lists the performances of the proposed algorithm (ICH) according to these thresholds. We see that the geometric scheme outperforms the arithmetic scheme in general. The best performance is achieved with Tage = 0.1, which is used in all experiments in the main paper. Note that the scores are poorer than those in Table 3 since the comparator trained for smaller number of epochs to facilitate this test. At Tage 0.1, two teenagers are declared to be not "similar to' each other if their age difference larger than about I. Also, two forties are not 'similar" the age difference is larger than about 5.
Table 8: The performances of the proposed algorithm (ICH) on the balanced dataset according to thresholds and Tage.
r for arithmetic scheme
Tage for geometric scheme
0
0.15
0.20
0.25
MAE
4.42
4.36
4.33
4.32
4.33
4.38
4.31
4.32
4.41
4.41
CS(%)
71.0
71.7
72.2
72.2
72.5
71.4
72.8
72.2
71.7
C.5 PERFORMANCE ACCORDING TO NUMBER OF REFERENCES
Table 9: The performances of the proposed algorithm (supervised) on the balanced dataset according the number of references for each age class (M/N). In general, the performances get better with more references. However, the performances are not very sensitive to M/N. They saturate when M/N 2 Therefore, we M/N
this work.
ICH
2CH1
3CH
6CH
Average
M/N
MAE
CS(%)
MAE
CS(%)
MAE
CS(%)
MAE
CS(%)
MAE
CS(%)
1
4.321
72.43
4.180
72.98
4.199
73.20
4.168
73.76
4.217
73.09
2
4.318
72.43
4.182
73.00
4.200
73.23
4.170
73.64
4.218
73.08
3
4.313
72.61
4.175
73.04
4.200
73.29
4.170
73.68
4.214
73.16
4
4.311
72.58
4.178
72.96
4.197
73.24
4.176
73.62
4.215
73.10
5 4.309
72.61
4.177
73.02
4.197
73.27
4.168
73.72
4.213
73.16
6
4.308
72.66
4.178
73.01
4.195
73.20
4.170
73.76
4.213
73.16
7
4.308
72.70
4.179
73.00
4.196
73.24
4.167
73.81
4.213
73.19
8 4.306
72.69
4.178
72.94
4.196
73.31
4.172
73.71
4.213
73.16
9 4.305
72.63
4.180
73.04
4.194
73.36
4.172
73.72
4.213
73.19
10
4.305
72.65
4.180
73.07
4.193
73.35
4.173
73.75
4.213
73.21
C.6 REFERENCE IMAGES
Figure shows all references in the supervised 6CH.
18

---

Published as a conference paper at ICLR 2020
Figure 8: All reference images in the supervised 6CH. For some ages in certain chains, the balanced dataset includes less than 5 faces. In such cases, there are less than 5 references.
24
62 6E 67 38
6
02
R H
M
imigre
IT
is
Male Eumgenn
Male
Pazzale
Altion
Male
Patralo
Asion
19

---

Published as a conference paper at ICLR 2020
C.7
AGE ESTIMATION EXAMPLES
20
27 NM
MENIS M/M
22
M125
42 1REM
3AAM (55
(a) Success cases
AUF
27A6H
AT
29 A&M
25 ASF
AcT
2 DM4 BI MM
23 2or 18 B
35 Om 12 E
02 RE MT
WE B3 EPP 33 MD Br 38 Aor 16 E
tu-Mi
(b) Failure cases
Figure 9: Age estimation results of the proposed algorithm (supervised 6CH). For each face, the timated label is provided together with the ground-truth in parentheses. In (a), the ages are estimated correctly. In the last row, third column, the ethnic group misclassified. This happens rarely. In (b), failure cases are provided. These are hard examples due to various challenging factors, such as low quality photographs and occlusion by hairs, hats, hands, and stickers.
20


Reviews:
[{"review_id": "HygmIfiAKB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents an order learning method and applies it to age estimation from facial images. It designs a pairwise comparator that categorizes ordering relationship between two instances into ternary classes of greater than, similar, and smaller than. Instead of directly estimating the class of each instance, it learns pairwise ordering relationship between two instances.\n\nFor age estimation from facial images, it uses a Siamese network as a feature extractor of the pairwise comparator, concatenates feature maps of dual inputs (test instance and one of the multiple reference instances that are selected from the training data) and applies it to a ternary classifier. Given the softmax probability scores of the classifier results, it estimates the final class as the one that maximizes a consistency rule computed over indicator functions. The comparator is trained to minimize a comparator loss computed over the softmax probability scores.\n\nThe paper also provides an extended version for multiple disjoint chains, where each chain may correspond to a higher-level attribute class, for example, gender or ethnic group. When there is no supervision available, it randomly partitions the training set.  and iteratively updates reliability scores and chain memberships. \n\nNovelty-wise, I consider that the proposed solution to be satisfactorily innovative and on a different vein than the existing methods. \n\nOne concern about the method is that it imposes the geometric ratio (log distance) between the class distances in age estimation, considering the difference between 5 and 10-year-old instances is easier to detect than 65 to 70-year-old instances as stated in the paper. However, as far as it is understood from the provided discussion compared SOTA methods are not retrained or fine-tuned in the same manner. This raises the question of whether the slight performance improvement is a result of the geometric ratio, in particular when computing the cumulative score CS.   \n\nAnother concern is that the performance is not stellar as the presented method underperforms in comparison to DRF (Shen et al. 2018) on MORPH II. The presentation of the results for FG-Net is neither complete nor included in the main paper, and results for CLAP2016 are missing. Discussion of the results Table 8 does not seem fair as it is not clear whether MV is retrained keeping the geometric ratio in mind. It might be the case that even for 15-69, MV might attain lover MAEs. \n\nThe paper also misses the latest SOTA, for instance, BridgeNet [1] is mentioned in a sentence, yet its results are omitted from the Tables. BridgeNet, according to its results, is on par with the proposed method. Also, the numbers reported in this paper and the numbers in [1] have a discrepancy, which causes confusion.\n\n[1] Wanhua Li et al. Bridgenet: A continuity aware probabilistic network for age estimation, CVPR 2019.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "H1xU9KYDoB", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper departs from traditional age estimation methods by proposing an ordering/ranking way. A testing face is compared to those anchor faces from each age category to estimate the right category that the testing face belongs to. It is novel to use the MC rule to drive the order learning. What more promising is the unsupervised chains, which could automatically search for a more optimal multi-chain division scheme than the pre-defined data division. The paper is well written.\n\nI have a few concerns as the following:\n1.\tThe geometric ratio (Eq. 10) further enlarges the distance between younger faces and shortens the distance between older faces. This may improve the prediction of younger ages. However, this makes the prediction on older ages more confusion, i.e., too robust to distinguish adjacent categories. The results shown in Table 6 show a better number for Geometric, but it is tested on FG-NET which presents unbalanced age distribution (ages up to 40 years are the most populated). Therefore, the improvement may be caused by the data distribution that is more suitable for geometric ratio, i.e., more younger faces and less older faces. Table 8 also provides such comparison, but please clarify the testing dataset. \n2.\tIn Algorithm 1, are you using a pre-trained comparator? If not, the T_k obtained from line 3 is still close to random. Such random initialization works for multiple chains with small mutual shift (the distance of references between chains). As indicated in Table 3, more chains will not significantly affect performance, which implies a relatively small mutual shift. However, if encounter a large shift, e.g., other properties would significantly affect the target property that you want to classify (ranking soccer teams where gender makes difference), will the proposed unsupervised learning converge? \n3.\tIt would be great if the authors could provide more insight to the 2CH unsupervised results. How the two chains are divided by analyzing the trained model? What is the key to divide the data? Maybe, it is like a black-box that is hard to explain but this will not affect the merit of this paper.\n ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "Byg6NpMCKH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method for learning partial orders based on learning fuzzy pairwise comparisons (smaller/greater/approximatively equal to), and the retaining of a set of representants in each chain in order to allow consistent result by consistency maximization. The method is applied to the problem of estimating the age based on faces. Extensions proposed are the learning of multiple disjoint chains on this dataset, manually partitioned or learned through an iterative assignment algorithm that has similarities with a soft expectation-maximization principle. Extensive experiments are done on the age estimation problem with comparison to exisiting approaches, and an application to aesthetic assessment is proposed in appendix.\n\nThe paper is very well written and contains very extensive and relevant experiments. I only have minor concerns:\n\n1) I couldn't find the explanation for the underlined numbers in Table 2.\n2) It would be good to include the details of Appendix C in the text, as is, it looks like an artificial space saving trick.\n3) typo in Section 3.2: \"we assume that the chain, to which the instance belong, is known\" -> \"we assume that the chain to which the instance belongs is known\"", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses a promising method for order learning and applies the new ideas of multiple-chain learning and anchor selection to age estimation and aesthetic regression. The decision regarding instance class is made by comparing it with anchor instances in the same chain and maximizing the consistency among the comparison results. In a multi-chain setting, each chain may correspond to a higher-level attribute class, for example, gender or ethnic group. Supervised and unsupervised learning of multiple ordered chains is proposed.  As rightly acknowledged by R4: “What more promising is the unsupervised chains, which could automatically search for a more optimal multi-chain division scheme than the pre-defined data division.”
All three reviewers and AC agree that the proposed approach is interesting and shows promising results. There are several potential weaknesses and suggestions to further strengthen this work:
(1) more quantitative results are needed for assessing the benefits of this approach (R3, R4) -- see R3’s request to complete the results for FG-Net, to include the results for CLAP2016 and a comparison with the SOTA method BridgeNet. Pleased to report that the authors have revised the manuscript and have included performance of the arithmetic scheme as well as the geometric scheme for FG-Net. Also the authors have provided some initial evaluations of BridgeNet and promised to report the final results as well as the results for CLAP2016 in the final version. 
(2) R3 and R4 have expressed concerns regarding using the geometric ratio of the class distances in age estimation and that the improvement may be caused by the data distribution that favours it (R4) or because the baseline methods are not fine-tuned in the same manner (R3). The authors have partially addressed this concern in the rebuttal. 
There is a large body of work in computer vision that is focused on relative comparison of samples based on attributes (e.g. age) that is not clearly articulated in the discussions / baseline comparisons (1CH) -- see the seminal work [Relative attributes by Parikh and Grauman, ICCV2011] and the follow up works. 
Considering the author response, the AC decided that the most crucial concerns have been addressed in the revision and that the paper could be accepted, but the authors are strongly urged to include additional results that were promised in the rebuttal for the final revision. 

Decision: Accept (Poster)