Original: O RDER L EARNING AND I TS A PPLICATION TO A GE E STIMATION
Revised: O RDER L EARNING AND I TS A PPLICATION TO A GE E STIMATION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We propose order learning to determine the order graph of classes, representing ranks or priorities, and classify an object instance into one of the classes.
Revised: We propose order learning to determine the order graph of classes, representing ranks or priorities and classify an object instance into one of the classes.

Original: To this end, we design a pairwise comparator to categorize the relationship between two instances into one of three cases: one instance is ‘greater than,’ ‘similar to,’ or ‘smaller than’ the other.
Revised: To this end, we design a pairwise comparator to categorize the relationship between two instances into one of three cases: one instance is ‘greater than,’ ‘similar to,’ or ‘smaller than’ the other.

Original: Then, by comparing an input instance with reference instances and maximizing the consistency among the comparison results, the class of the input can be estimated reliably.
Revised: Then, by comparing an input instance with reference instances and maximizing the consistency among the comparison results, the class of the input can be estimated reliably.

Original: We apply order learning to develop a facial age estimator, which provides the state-of-the-art performance.
Revised: We apply order learning to develop a facial age estimator, which provides the state-of-the-art performance.

Original: Moreover, the performance is further improved when the order graph is divided into disjoint chains using gender and ethnic group information or even in an unsupervised manner.
Revised: Moreover, the performance is further improved when the order graph is divided into disjoint chains using gender and ethnic group information or even in an unsupervised manner

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: To measure the quality of something, we often compare it with other things of a similar kind.
Revised: To measure the quality of something, we often compare it with other things of a similar kind.

Original: Before assigning 4 stars to a ﬁlm, a critic would have thought, “It is better than 3-star ﬁlms but worse than 5-stars.”
Revised: Before assigning 4 stars to a ﬁlm, a critic would have thought, “It is better than 3-star ﬁlms but worse than 5-stars.”

Original: This ranking through pairwise comparisons is done in various decision processes (Saaty, 1977).
Revised: This ranking through pairwise comparisons is done in various decision processes (Saaty, 1977).

Original: In a sports league, teams compete against each other and their ranks are determined according to their numbers of wins.

Original: It is easier to tell the nearer one between two objects in a picture than to estimate the distance of each object directly (Chen et al., 2016; Lee & Kim, 2019a).
Revised: It is easier to tell the nearer one between two objects in a picture than to estimate the distance of each object directly (Chen et al., 2016; Lee & Kim, 2019a).

Original: Also, it is easy to tell a higher pitch between two notes, but absolute pitch is a rare ability (Bachem, 1955).
Revised: Also, it is easy to tell a higher pitch between two notes, but absolute pitch is a rare ability (Bachem, 1955).

Original: Ranking through comparisons has been investigated for machine learning.
Revised: Ranking through comparisons has been investigated for machine learning.

Original: In learning to rank (LTR), the pairwise approach learns, between two documents, which one is more relevant to a query (Liu, 2009).
Revised: In learning to rank (LTR), the pairwise approach learns, between two documents, which one is more relevant to a query (Liu, 2009).

Original: Also, in ordinal regression (Frank & Hall, 2001; Li & Lin, 2007), to predict the rank of an object, binary classiﬁcations are performed to tell whether the rank is higher than a series of thresholds or not.
Revised: Also, in ordinal regression (Frank & Hall, 2001; Li & Lin, 2007), to predict the rank of an object, binary classiﬁcations are performed to tell whether the rank is higher than a series of thresholds or not.

Original: In this paper, we propose order learning to learn ordering relationship between objects.
Revised: In this paper, we propose order learning to learn ordering relationship between objects.

Original: Thus, order learning is related to LTR and ordinal regression.
Revised: Thus, order learning is related to LTR and ordinal regression.

Original: However, whereas LTR and ordinal regression assume that ranks form a total order (Hrbacek & Jech, 1984), order learning can be used for a partial order as well.
Revised: However, whereas LTR and ordinal regression assume that ranks form a total order (Hrbacek & Jech, 1984), order learning can be used for a partial order as well.

Original: Order learning is also related to metric learning (Xing et al., 2002).
Revised: Order learning is also related to metric learning (Xing et al., 2002).

Original: While metric learning is about whether an object is ‘similar to or dissimilar from’ another object, order learning is about ‘greater than or smaller than.’
Revised: While metric learning is about whether an object is ‘similar to or dissimilar from’ another object, order learning is about ‘greater than or smaller than.’

Original: Section 2 reviews this related work brieﬂy.
Revised: Section 2 reviews this related work.

Original: In order learning, a set of classes, Θ = { θ 1 , θ 2 , · · · , θ n } , is ordered, where each class θ i represents one or more object instances.
Revised: In order learning, a set of classes, Θ = { θ 1 , θ 2 , · · · , θ n } , is ordered, where each class θ i represents one or more object instances.

Original: Between two classes θ i and θ j , there are three possibilities: θ i > θ j or θ i < θ j or neither ( i.e. incomparable).
Revised: Between two classes θ i and θ j , there are three possibilities: θ i > θ j or θ i < θ j or neither ( i.e. incomparable).

Original: These relationships are represented by the order graph.
Revised: These relationships are represented by the order graph.

Original: The goal of order learning is to determine the order graph and then classify an instance into one of the classes in Θ .
Revised: The goal of order learning is to determine the order graph and then classify an instance into one of the classes in Θ .

Original: To achieve this, we develop a pairwise comparator that determines ordering relationship between two instances x and y into one of three categories: x is ‘greater than,’ ‘similar to,’ or ‘smaller than’ y .
Revised: To achieve this, we develop a pairwise comparator that determines ordering relationship between two instances x and y into one of three categories: x is ‘greater than,’ ‘similar to,’ or ‘smaller than’ y .

Original: Then, we use the comparator to measure an input instance against multiple reference instances in known classes.
Revised: Then, we use the comparator to measure an input instance against multiple reference instances in known classes.

Original: Finally, we estimate the class of the input to maximize the consistency among the comparison results.
Revised: Finally, we estimate the class of the input to maximize the consistency among the comparison results.

Original: It is noted that the parameter optimization of the pairwise comparator, the selection of the references, and the discovery of the order graph are jointly performed to minimize a common loss function.
Revised: It is noted that the parameter optimization of the pairwise comparator, the selection of the references, and the discovery of the order graph are jointly performed to minimize a common loss function.

Original: Section 3 proposes this order learning.
Revised: Section 3 proposes this order learning.

Original: We apply order learning to facial age estimation.
Revised: We apply order learning to facial age estimation.

Original: Order learning matches age estimation well, since it is easier to tell a younger one between two people than to estimate each person’s age directly (Chang et al., 2010; Zhang et al., 2017a).
Revised: Order learning matches age estimation well, since it is easier to tell a younger one between two people than to estimate each person’s age directly (Chang et al., 2010; Zhang et al., 2017a).

Original: Even when we assume that age classes are linearly ordered, the proposed age estimator outperforms conventional algorithms.
Revised: Even when we assume that age classes are linearly ordered, the proposed age estimator performs well.

Original: The performance is further improved, when classes are divided into disjoint chains in a supervised manner using gender and ethnic group information or even in an unsupervised manner.
Revised: The performance is further improved, when classes are divided into disjoint chains in a supervised manner using gender and ethnic group information or even in an unsupervised manner.

Original: Section 4 describes this age estimator and discusses its results.
Revised: Section 4 describes this age estimator and discusses its results.

Original: Finally, Section 5 concludes this work.
Revised: Finally, Section 5 concludes this work.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Pairwise comparison: It is a fundamental problem to estimate the priorities (or ranks) of objects through pairwise comparison.
Revised: Pairwise comparison: It is a fundamental problem to estimate the priorities (or ranks) of objects through pairwise comparison.

Original: In the classic paper, Saaty (1977) noted that, even when direct estimates of certain quantities are unavailable, rough ratios between them are easily obtained in many cases.
Revised: In the classic paper, Saaty (1977) noted that, even when direct estimates of certain quantities are unavailable, rough ratios between them are easily obtained in many cases.

Original: Thus, he proposed the scaling method to reconstruct absolute priorities using only relative priorities.
Revised: Thus, he proposed the scaling method to reconstruct absolute priorities using only relative priorities.

Original: The scaling method was applied to monocular depth estimation (Lee & Kim, 2019a) and aesthetic assessment (Lee & Kim, 2019b).
Revised: The scaling method was applied to monocular depth estimation (Lee & Kim, 2019a) and aesthetic assessment (Lee & Kim, 2019b).

Original: Ranking from a pairwise comparison matrix has been studied to handle cases, in which the matrix istoo huge or some elements are noisy (Braverman & 
Revised: Ranking from a pairwise comparison matrix has been studied to handle cases, in which the matrix is huge or some elements are noisy (Braverman & Mossel, 2008; Jamieson & Nowak, 2011;

Original: Mossel, 2008; Jamieson & Nowak, 2011; Negahban et al., 2012; Wauthier et al., 2013).
Revised: Negahban et al., 2012; Wauthier et al., 2013).

Original: On the other hand, the pairwise approach to LTR learns, between two documents, which one is more relevant to a query (Liu, 2009; Herbrich et al., 1999; Burges et al., 2005; Tsai et al., 2007).
Revised: On the other hand, the pairwise approach to LTR learns, between two documents, which one is more relevant to a query (Liu, 2009; Herbrich et al., 1999; Burges et al., 2005; Tsai et al., 2007).

Original: The proposed order learning is related to LTR, since it also predicts the order between objects.
Revised: The proposed order learning is related to LTR, since it also predicts the order between objects.

Original: However, while LTR sorts multiple objects with unknown ranks and focuses on the sorting quality, order learning compares a single object x with optimally selected references with known ranks to estimate the rank of x .
Revised: But, while LTR sorts multiple objects with unknown ranks and focuses on the sorting quality, order learning compares a single object x with optimally selected references with known ranks to estimate the rank of x .

Original: Ordinal regression: Ordinal regression predicts an ordinal variable (or rank) of an instance.
Revised: Ordinal regression: Ordinal regression predicts an ordinal variable (or rank) of an instance.

Original: Suppose that a 20-year-old is misclassiﬁed as a 50-year old and a 25-year old, respectively.
Revised: Suppose that a 20-year-old is misclassiﬁed as a 50-year old and a 25-year old, respectively.

Original: The former error should be more penalized than the latter.
Revised: The former error should be more penalized than the latter.

Original: Ordinal regression exploits this characteristic in the design of a classiﬁer or a regressor.
Revised: Ordinal regression exploits this characteristic in the design of a classiﬁer or a regressor.

Original: In Frank & Hall (2001) and Li & Lin (2007), a conversion scheme was proposed to transform an ordinal regression problem into multiple binary classiﬁcation problems.
Revised: In Frank & Hall (2001) and Li & Lin (2007), a conversion scheme was proposed to transform an ordinal regression problem into multiple binary classiﬁcation problems.

Original: Ordinal regression based on this conversion scheme has been used in various applications, including age estimation (Chang et al., 2010; 2011;
Revised: Ordinal regression based on this conversion scheme has been used in various applications, including age estimation (Chang et al., 2010; 2011;

Original: Niu et al., 2016; Chen et al., 2017) and monocular depth estimation (Fu et al., 2018).
Revised: Niu et al., 2016; Chen et al., 2017) and monocular depth estimation (Fu et al., 2018).

Original: Note that order learning is different from ordinal regression.
Revised: Note that order learning is different from ordinal regression.

Original: Order learning performs pairwise comparison between objects, instead of directly estimating the rank of each object.
Revised: Order learning performs pairwise comparison between objects, instead of directly estimating the rank of each object.

Original: In age estimation, ordinal regression based on the conversion scheme is concerned with the problem, “Is a person’s age bigger than a threshold θ ?”
Revised: In age estimation, ordinal regression based on the conversion scheme is concerned with the problem, “Is a person’s age bigger than a threshold θ ?”

Original: for each θ .
Revised: for each θ .

Original: In contrast, order learning concerns “Between two people, who is older?”
Revised: In contrast, order learning concerns “Between two people, who is older?”

Original: Conceptually, order learning is easier.
Revised: Conceptually, order learning is easier.

Original: Technically, if there are N ranks, the conversion scheme requires N − 1 binary classiﬁers, but order learning needs only a single ternary classiﬁer.
Revised: Technically, if there are N ranks, the conversion scheme requires N − 1 binary classiﬁers, but order learning needs only a single ternary classiﬁer.

Original: Moreover, whereas ordinal regression assumes that ranks form a total order, order learning can be used even in the case of a partial order (Hrbacek &
Revised: Moreover, whereas ordinal regression assumes that ranks form a total order, order learning can be used even in the case of a partial order (Hrbacek &

Original: Jech, 1984).
Revised: Jech, 1984).

Original: Metric learning: A distance metric can be learned from examples of similar pairs of points and those of dissimilar pairs (Xing et al., 2002).
Revised: Metric learning: A distance metric can be learned from examples of similar pairs of points and those of dissimilar pairs (Xing et al., 2002).

Original: The similarity depends on an application and is implicitly deﬁned by user-provided examples.
Revised: The similarity depends on an application and is implicitly deﬁned by user-provided examples.

Original: If a learned metric generalizes well to unseen data, it can be used to enforce the desired similarity criterion in clustering (Xing et al., 2002), classiﬁcation (Weinberger et al., 2006), or information retrieval (McFee & Lanckriet, 2010).
Revised: If a learned metric generalizes well to unseen data, it can be used to enforce the desired similarity criterion in clustering (Xing et al., 2002), classiﬁcation (Weinberger et al., 2006), or information retrieval (McFee & Lanckriet, 2010).

Original: Both metric learning and order learning learn important binary relations in mathematics: metric and order (Hrbacek & Jech, 1984).
Revised: Both metric learning and order learning learn important binary relations in mathematics: metric and order (Hrbacek & Jech, 1984).

Original: However, a metric decides whether an object x is similar to or dissimilar from another object y , whereas an order tells whether x is greater than or smaller than y .
Revised: However, a metric decides whether an object x is similar to or dissimilar from another object y , whereas an order tells whether x is greater than or smaller than y .

Original: Thus, a learned metric is useful for grouping similar data, whereas a learned order is suitable for processing ordered data.
Revised: Thus, a learned metric is useful for grouping similar data, whereas a learned order is suitable for processing ordered data.

Original: Age estimation: Human ages can be estimated from facial appearance (Kwon & da Vitoria Lobo, 1994).
Revised: Age estimation: Human ages can be estimated from facial appearance (Kwon & da Vitoria Lobo, 1994).

Original: Geng et al.
Revised: Geng et al.

Original: (2004) proposed the aging pattern subspace, and Guo et al.
Revised: (2004) proposed the aging pattern subspace, and Guo et al.

Original: (2009) introduced biologically inspired features to age estimation.
Revised: (2009) introduced biologically inspired features to age estimation.

Original: Recently, deep learning has been adopted for age estimation.
Revised: Recently, deep learning has been adopted for age estimation.

Original: Niu et al.
Revised: Niu et al.

Original: (2016) proposed OR-CNN for age estimation, which is an ordinal regressor using the conversion scheme.
Revised: (2016) proposed OR-CNN for age estimation, which is an ordinal regressor using the conversion scheme.

Original: Chen et al.
Revised: Chen et al.

Original: (2017) proposed Ranking-CNN, which is another ordinal regressor.
Revised: (2017) proposed Ranking-CNN, which is another ordinal regressor.

Original: While OR-CNN uses a common feature for multiple binary classiﬁers, Ranking-CNN employs a separate CNN to extract a feature for each binary classiﬁer.
Revised: While OR-CNN uses a common feature for multiple binary classiﬁers, Ranking-CNN employs a separate CNN to extract a feature for each binary classiﬁer.

Original: Tan et al.
Revised: Tan et al.

Original: (2018) grouped adjacent ages via the group-n encoding, determined whether a face belongs to each group, and combined the results to predict the age.
Revised: (2018) grouped adjacent ages via the group-n encoding, determined whether a face belongs to each group, and combined the results to predict the age.

Original: Pan et al.
Revised: Pan et al.

Original: (2018) proposed the mean-variance loss to train a CNN classiﬁer for age estimation.
Revised: (2018) proposed the mean-variance loss to train a CNN classiﬁer for age estimation.

Original: Shen et al.
Revised: Shen et al.

Original: (2018) proposed the deep regression forests for age estimation.
Revised: (2018) proposed the deep regression forests for age estimation.

Original: Zhang et al.
Revised: Zhang et al.

Original: (2019) developed a compact age estimator using the two-points representation.
Revised: (2019) developed a compact age estimator using the two-points representation.

Original: Also, Li et al.
Revised: Also, Li et al.

Original: (2019) proposed a continuity-aware probabilistic network for age estimation.
Revised: (2019) proposed a continuity-aware probabilistic network for age estimation.

Original: 3       (a) 222 333 555(b)
Revised: 3       (a) 222 333 555(b)

Original:  O RDER LEARNING 3. W HAT IS ORDER ?
Revised:  O RDER LEARNING 3. W HAT IS ORDER ?

Original: We propose order learning.

Original: To this end, we ﬁrst review mathematical deﬁnitions and concepts related to order.
Revised: Let us ﬁrst review mathematical deﬁnitions and concepts related to order.

Original: An order (Hrbacek & Jech, 1984; Bartle, 1976), often denoted by ≤ , is a binary relation on a set Θ = { θ 1 , θ 2 , · · · , θ n } that satisﬁes the three properties of
Revised: An order (Hrbacek & Jech, 1984; Bartle, 1976), often denoted by ≤ , is a binary relation on a set Θ = { θ 1 , θ 2 , · · · , θ n } that satisﬁes the three properties of

Original: • Reﬂexivity: θ i ≤ θ i for every θ i ∈ Θ ; • Antisymmetry: If θ i ≤ θ j and θ j ≤ θ i , then θ i
Revised: • Reﬂexivity: θ i ≤ θ i for every θ i ∈ Θ ; • Antisymmetry: If θ i ≤ θ j and θ j ≤ θ i , then θ i

Original: =
Revised: =

Original: θ j ; • Transitivity:
Revised: θ j ; • Transitivity:

Original: If θ i ≤ θ j and θ j ≤ θ k , then θ i ≤ θ k .
Revised: If θ i ≤ θ j and θ j ≤ θ k , then θ i ≤ θ k .

Original: In real-world problems, an order describes ranks or priorities of objects.
Revised: In real-world problems, an order describes ranks or priorities of objects.

Original: For example, in age estimation, θ i ≤ θ j means that people in age class θ i look younger than those in θ j .
Revised: For example, in age estimation, θ i ≤ θ j means that people in age class θ i look younger than those in θ j .

Original: We may use the symbol → , instead of ≤ , to denote an order on a ﬁnite set Θ .
Revised: We may use the symbol → , instead of ≤ , to denote an order on a ﬁnite set Θ .

Original: Then, the order can be represented by a directed graph (Gross & Yellen, 2006) using elements in Θ as nodes.
Revised: Then, the order can be represented by a directed graph (Gross & Yellen, 2006) using elements in Θ as nodes.

Original: If θ i → θ j , there is a directed edge from node θ i to node θ j .
Revised: If θ i → θ j , there is a directed edge from node θ i to node θ j .

Original: The order graph is acyclic because of antisymmetry and transitivity.
Revised: The order graph is acyclic because of antisymmetry and transitivity.

Original: For example, for n, m ∈ N , let n → m denote that m is a multiple of n .
Revised: For example, for n, m ∈ N , let n → m denote that m is a multiple of n .

Original: Note that it is an order on any subset of N .
Revised: Note that it is an order on any subset of N .

Original: Figure 1(a) is the graph representing this order on { 1 , . .
Revised: Figure 1(a) is the graph representing this order on { 1 , . .

Original: Elements θ i and θ j are comparable if θ i
Revised: Elements θ i and θ j are comparable if θ i

Original: → θ j or θ j
Revised: → θ j or θ j

Original: → θ i , or incomparable otherwise.
Revised: → θ i , or incomparable otherwise.

Original: In Figure 1(a), 6 and 8 are incomparable.
Revised: In Figure 1(a), 6 and 8 are incomparable.

Original: In age estimation, it is difﬁcult to compare apparent ages of people in different ethnic groups or of different genders.
Revised: In age estimation, it is difﬁcult to compare apparent ages of people in different ethnic groups or of different genders.

Original: An order on a set Θ is total (or linear ) if all elements in Θ are comparable to one another.
Revised: An order on a set Θ is total (or linear ) if all elements in Θ are comparable to one another.

Original: In such a case, Θ is called a linearly ordered set.
Revised: In such a case, Θ is called a linearly ordered set.

Original: In some real-world problems, orders are not linear.
Revised: In some real-world problems, orders are not linear.

Original: In this work, a subset Θ c of Θ is referred to as a chain , if Θ c is linearly ordered and also maximal, i.e. there is no proper superset of Θ c that is linearly ordered.
Revised: In this work, a subset Θ c of Θ is referred to as a chain , if Θ c is linearly ordered and also maximal, i.e. there is no proper superset of Θ c that is linearly ordered.

Original: In Figure 1(a), nodes 1, 2, 4, and 8 form a chain.
Revised: In Figure 1(a), nodes 1, 2, 4, and 8 form a chain.

Original: In Figure 1(b), the entire set is composed of three disjoint chains.
Revised: In Figure 1(b), the entire set is composed of three disjoint chains.

Original: 3. O RDER LEARNING – B ASICS
Revised: 3. O RDER LEARNING – B ASICS

Original: Let Θ = { θ 1 , θ 2 , · · · , θ n } be an ordered set of classes, where each class θ i represents one or more object instances.
Revised: Let Θ = { θ 1 , θ 2 , · · · , θ n } be an ordered set of classes, where each class θ i represents one or more object instances.

Original: For example, in age estimation, age class 11 is the set of 11-year-olds.
Revised: For example, in age estimation, age class 11 is the set of 11-year-olds.

Original: The objective of order learning is to determine the order graph, such as Figure 1(a) or (b), and categorize an object instance into one of the classes.
Revised: The objective of order learning is to determine the order graph, such as Figure 1(a) or (b), and categorize an object instance into one of the classes.

Original: However, in many cases, order graphs are given explicitly or obvious from the contexts.
Revised: However, in many cases, order graphs are given explicitly or obvious from the contexts.

Original: For example, in quality assessment, there are typically ﬁve classes (poor → satisfactory → good → very good → excellent), forming a single chain.
Revised: For example, in quality assessment, there are typically ﬁve classes (poor → satisfactory → good → very good → excellent), forming a single chain.

Original: Also, in age estimation, suppose that an algorithm ﬁrst classiﬁes a person’s gender into female or male and then estimates the age differently according to the gender.
Revised: Also, in age estimation, suppose that an algorithm ﬁrst classiﬁes a person’s gender into female or male and then estimates the age differently according to the gender.

Original: In this case, implicitly, there are separate age classes for each gender, and the age classes compose two disjoint chains similarly to Figure 1(b).
Revised: In this case, implicitly, there are separate age classes for each gender, and the age classes compose two disjoint chains similarly to Figure 1(b).

Original: Thus, in this subsection, we assume that the order graph is already known.
Revised: Thus, in this subsection, we assume that the order graph is already known.

Original: Also, given an object instance, we assume that the chain to which the instance belongs is known.
Revised: Also, given an object instance, we assume that the chain to which the instance belongs is known.

Original: Then, we attempt to categorize the instance into one of the classes in the chain.
Revised: Then, we attempt to categorize the instance into one of the classes in the chain.

Original: Section 3.4 will propose the order learning in the case of an unknown order graph, composed of disjoint chains.
Revised: Section 3.4 will propose the order learning in the case of an unknown order graph, composed of disjoint chains.

Original: Instead of directly estimating the class of each instance, we learn pairwise ordering relationship between two instances.
Revised: Instead of directly estimating the class of each instance, we learn pairwise ordering relationship between two instances.

Original: Let Θ c
Revised: Let Θ c

Original: = { 0 , 1 , . .
Revised: = { 0 , 1 , . .

Original: . , N − 1 } be a chain, where N is the number of classes. 
Revised: . , N − 1 } be a chain, where N is the number of classes. Let

Original: [Figure]
Revised: [Figure]

Original: Without loss of generality, we use consecutive nonnegative integers to denote the classes.

Original: Let x and y be two instances belonging to classes in Θ c .
Revised: x and y be two instances belonging to classes in Θ c .

Original: Let θ ( · ) denote the class of an instance.
Revised: Let θ ( · ) denote the class of an instance.

Original: Then, x and y are compared and their ordering relationship is deﬁned according to their class difference as
Revised: Then, x and y are compared and their ordering relationship is deﬁned according to their class difference as

Original: [Equation]
Revised: [Equation]

Original: where τ is a threshold.
Revised: where τ is a threshold.

Original: To avoid confusion, we use ‘ (cid:31) , ≈ , ≺ ’ for the instance ordering, while ‘ > , = , < ’ for the class order.
Revised: To avoid confusion, we use ‘ (cid:31) , ≈ , ≺ ’ for the instance ordering, while ‘ > , = , < ’ for the class order.

Original: In practice, the categorization in (1) ∼ (3) is performed by a pairwise comparator in Figure 2, which consists of a Siamese network and a ternary classiﬁer (Lee & Kim, 2019b).
Revised: In practice, the categorization in (1) ∼ (3) is performed by a pairwise comparator in Figure 2, which consists of a Siamese network and a ternary classiﬁer (Lee & Kim, 2019b).

Original: To train the comparator, only comparable instance pairs are employed.
Revised: To train the comparator, only comparable instance pairs are employed.

Original: We estimate the class θ ( x ) of a test instance x by comparing it with reference instances y m , 0 ≤ m ≤ M − 1 , where M is the number of references.
Revised: We estimate the class θ ( x ) of a test instance x by comparing it with reference instances y m , 0 ≤ m ≤ M − 1 , where M is the number of references.

Original: The references are selected from training data such that they are from the same chain as x .
Revised: The references are selected from training data such that they are from the same chain as x .

Original: Given x and y m , the comparator provides one of three categories ‘ (cid:31) , ≈ , ≺ ’ as a result.
Revised: Given x and y m , the comparator provides one of three categories ‘ (cid:31) , ≈ , ≺ ’ as a result.

Original: Let θ (cid:48) be an estimate of the true class θ ( x ) .
Revised: Let θ (cid:48) be an estimate of the true class θ ( x ) .

Original: Then, the consistency between the comparator result and the estimate is deﬁned as
Revised: Then, the consistency between the comparator result and the estimate is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where [ · ] is the indicator function.
Revised: where [ · ] is the indicator function.

Original: The function φ con ( x, y m , θ (cid:48) ) returns either 0 for an inconsistent case or 1 for a consistent case.
Revised: The function φ con ( x, y m , θ (cid:48) ) returns either 0 for an inconsistent case or 1 for a consistent case.

Original: For example, suppose that the pairwise comparator declares x ≺ y m but θ (cid:48) − θ ( y m ) >
Revised: For example, suppose that the pairwise comparator declares x ≺ y m but θ (cid:48) − θ ( y m ) >

Original: τ .
Revised: τ .

Original: Then, φ con ( x, y m , θ (cid:48) )
Revised: Then, φ con ( x, y m , θ (cid:48) )

Original: = 0 · 1+0 · 0+1 · 0 = 0 .
Revised: = 0 · 1+0 · 0+1 · 0 = 0 .

Original: Due to a possible classiﬁcation error of the comparator, this inconsistency may occur even when the estimate θ (cid:48) equals the true class θ ( x ) .
Revised: Due to a possible classiﬁcation error of the comparator, this inconsistency may occur even when the estimate θ (cid:48) equals the true class θ ( x ) .

Original: To maximize the consistency with all references, we estimate the class of x by
Revised: To maximize the consistency with all references, we estimate the class of x by

Original: [Equation]
Revised: [Equation]

Original: which is called the maximum consistency (MC) rule.
Revised: which is called the maximum consistency (MC) rule.

Original: Figure 3 illustrates how the consistency is computed in the MC rule.
Revised: Figure 3 illustrates this MC rule.

Original: It is noted that ‘ (cid:31) , ≈ , ≺ ’ is not an mathematical order.
Revised: It is noted that ‘ (cid:31) , ≈ , ≺ ’ is not an mathematical order.

Original: For example, if θ ( x ) + 34 τ = θ ( y )
Revised: For example, if θ ( x ) + 34 τ = θ ( y )

Original: =
Revised: =

Original: θ ( z ) − 34 τ , then x ≈ y and y ≈ z but x ≺ z .
Revised: θ ( z ) − 34 τ , then x ≈ y and y ≈ z but x ≺ z .

Original: This is impossible in an order.
Revised: This is impossible in an order.

Original: More precisely, due to the quantization effect of the ternary classiﬁer in (1) ∼ (3), ‘ (cid:31) , ≈ , ≺ ’ is quasi-transitive (Sen, 1969), and ‘ ≈ ’ is symmetric but intransitive.
Revised: More precisely, due to the quantization effect of the ternary classiﬁer in (1) ∼ (3), ‘ (cid:31) , ≈ , ≺ ’ is quasi-transitive (Sen, 1969), and ‘ ≈ ’ is symmetric but intransitive.

Original: We use this quasi-transitive relation to categorize an instance into one of the classes, on which a mathematical order is deﬁned.
Revised: We use this quasi-transitive relation to categorize an instance into one of the classes, on which a mathematical order is deﬁned.

Original: 3. O RDER LEARNING – S UPERVISED 
Revised: 3. O RDER LEARNING – S UPERVISED CHAINS 3.3. S INGLE - CHAIN HYPOTHESIS (1CH)

Original: 3.3. S INGLE - CHAIN HYPOTHESIS (1CH)

Original: In the simplest case of 1CH, all classes form a single chain Θ c = { 0 , 1 , . . .
Revised: In the simplest case of 1CH, all classes form a single chain Θ c = { 0 , 1 , . . .

Original: , N − 1 } .
Revised: , N − 1 } .

Original: For example, in 1CH age estimation, people’s ages are estimated regardless of their ethnic groups or genders.
Revised: For example, in 1CH age estimation, people’s ages are estimated regardless of their ethnic groups or genders.

Original: We implement the comparator in Figure 2 using CNNs, as described in Section 4.1.
Revised: We implement the comparator in Figure 2 using CNNs, as described in Section 4.1.

Original: Let q xy
Revised: Let q xy

Original: =
Revised: =

Original: ( q xy 0 , q xy 1 , q xy 2 ) be the one-hot vector, indicating the ground-truth ordering relationship between training instances x and y .
Revised: ( q xy 0 , q xy 1 , q xy 2 ) be the one-hot vector, indicating the ground-truth ordering relationship between training instances x and y .

Original: Speciﬁcally, (1 , 0 , 0) , (0 , 1 , 0) , and (0 , 0 , 1) represent x (cid:31) y , x ≈ y ,[Figure] and x ≺ y .
Revised: Speciﬁcally, (1 , 0 , 0) , (0 , 1 , 0) , and (0 , 0 , 1) represent x (cid:31) y , x ≈ y , and x ≺ y .

Original: Also, p xy =
Revised: Also, p xy =

Original: ( p xy 0 , p xy 1 , p xy 2 ) is the corresponding softmax probability vector of the comparator.
Revised: ( p xy 0 , p xy 1 , p xy 2 ) is the corresponding softmax probability vector of the comparator.

Original: We train the comparator to minimize the comparator loss,
Revised: We train the comparator to minimize the comparator loss,

Original: [Equation]
Revised: [Equation]

Revised: [Figure]

Original: where T is the set of all training instances and R ⊂ T is the set of reference instances.
Revised: where T is the set of all training instances and R ⊂ T is the set of reference instances.

Original: First, we initialize R = T and minimize (cid:96) co via the stochastic gradient descent.
Revised: First, we initialize R = T and minimize (cid:96) co via the stochastic gradient descent.

Original: Then, we reduce the reference set R by sampling references from T .
Revised: Then, we reduce the reference set R by sampling references from T .

Original: Speciﬁcally, for each class in Θ c , we choose M/N reference images to minimize the same loss (cid:96) co , where M is the number of all references and N is the number of classes.
Revised: Speciﬁcally, for each class in Θ c , we choose M/N reference images to minimize the same loss (cid:96) co , where M is the number of all references and N is the number of classes.

Original: In other words, the reliability score of a reference candidate y is deﬁned as
Revised: In other words, the reliability score of a reference candidate y is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: and the M/N candidates with the highest reliability scores are selected.
Revised: and the M/N candidates with the highest reliability scores are selected.

Original: Next, after ﬁxing the reference set R , the comparator is trained to minimize the loss (cid:96) co .
Revised: Next, after ﬁxing the reference set R , the comparator is trained to minimize the loss (cid:96) co .

Original: Then, after ﬁxing the comparator parameters, the reference set R is updated to minimize the same loss (cid:96) co , and so forth.
Revised: Then, after ﬁxing the comparator parameters, the reference set R is updated to minimize the same loss (cid:96) co , and so forth.

Original: In the test phase, an input instance is compared with the M references and its class is estimated using the MC rule in (5).
Revised: In the test phase, an input instance is compared with the M references and its class is estimated using the MC rule in (5).

Original: 3.3. K - CHAIN HYPOTHESIS ( K CH)
Revised: 3.3. K - CHAIN HYPOTHESIS ( K CH)

Original: In K CH, we assume that classes form K disjoint chains, as in Figure 1(b).
Revised: In K CH, we assume that classes form K disjoint chains, as in Figure 1(b).

Original: For example, in the supervised 6CH for age estimation, we predict a person’s age according to the gender in { female, male } and the ethnic group in { African, Asian, European } .
Revised: For example, in the supervised 6CH for age estimation, we predict a person’s age according to the gender in { female, male } and the ethnic group in { African, Asian, European } .

Original: Thus, there are 6 chains in total.
Revised: Thus, there are 6 chains in total.

Original: In this case, people in different chains are assumed to be incomparable forthe purpose of age estimation.
Revised: In this case, people in different chains are assumed to be incomparable for age estimation.

Original: It is supervised, since gender and ethnic group annotations are used to separate the chains.
Revised: It is supervised, since gender and ethnic group annotations are used to separate the chains.

Original: The supervised 2CH or 3CH also can be implemented by dividing chains by genders only or ethnic groups only.
Revised: The supervised 2CH or 3CH also can be implemented by dividing chains by genders only or ethnic groups only.

Original: The comparator is trained similarly to 1CH.
Revised: The comparator is trained similarly to 1CH.

Original: However, in computing the comparator loss in (6), a training instance x and a reference y are constrained to be from the same chain.
Revised: However, in computing the comparator loss in (6), a training instance x and a reference y are constrained to be from the same chain.

Original: Also, during the test, the type (or chain) of a test instance should be determined.
Revised: Also, during the test, the type (or chain) of a test instance should be determined.

Original: Therefore, a K -way type classiﬁer is trained, which shares the feature extractor with the comparator in Figure 2 and uses additional fully-connected (FC) layers.
Revised: Therefore, a K -way type classiﬁer is trained, which shares the feature extractor with the comparator in Figure 2 and uses additional fully-connected (FC) layers.

Original: Thus, the overall loss is given by
Revised: Thus, the overall loss is given by

Original: [Equation]
Revised: [Equation]

Original: where (cid:96) co is the comparator loss and (cid:96) ty is the type classiﬁer loss.
Revised: where (cid:96) co is the comparator loss and (cid:96) ty is the type classiﬁer loss.

Original: The comparator and the type classiﬁer are jointly trained to minimize this overall loss (cid:96) .
Revised: The comparator and the type classiﬁer are jointly trained to minimize this overall loss (cid:96) .

Original: During the test, given an input instance, we determine its chain using the type classiﬁer, and compare it with the references from the same chain, and then estimate its class using the MC rule in (5).
Revised: During the test, given an input instance, we determine its chain using the type classiﬁer, and compare it with the references from the same chain, and then estimate its class using the MC rule in (5).

Original: 3. O RDER LEARNING – U NSUPERVISED CHAINS
Revised: 3. O RDER LEARNING – U NSUPERVISED CHAINS

Original: This subsection proposes an algorithm to separate classes into K disjoint chains when there are no supervision or annotation data available for the separation.
Revised: This subsection proposes an algorithm to separate classes into K disjoint chains when there are no supervision or annotation data available for the separation.

Original: First, we randomly partition the training set T into T 0 , T 1 , . . .
Revised: First, we randomly partition the training set T into T 0 , T 1 , . . .

Original: , T K − 1 , where T = T 0 ∪ . . .
Revised: , T K − 1 , where T = T 0 ∪ . . .

Original: ∪ T K −and T k ∩ T l = ∅ for k (cid:54) =
Revised: ∪T K − 1 and T k ∩T l = ∅ for k (cid:54) =

Original: l .
Revised: l .

Original: Then, similarly  to (6), the comparator loss (cid:96) co can be written as 
Revised: Then, similarly R to (6), the comparator loss (cid:96) co can be written as (cid:96) co = − K − 1 (cid:88) k = (cid:88) x

Original: [Equation]
Revised: ∈T k (cid:88) y ∈R k 2 (cid:88) j = q xyj log p xyj = −

Revised: K − 1 (cid:88) k

Revised: = (cid:88)

Revised: y ∈R k α k

Revised: ( y )

Original:  where R k ⊂ T k is the set of references for the k th chain, α k ( y )
Revised: = − K − 1 (cid:88) k = (cid:88) x ∈T k β k ( x ) (9) where R k ⊂ T k is the set of references for the k th chain, α k ( y )

Original: = (cid:80) x
Revised: = (cid:80) x

Original: ∈T k (cid:80) j q xyj log p xyj is the reliability of a reference y in the k th chain, and β k ( x ) = (cid:80) y ∈R k (cid:80) j q xyj log p xyj is the afﬁnity of an instance x to the references in the k th chain.
Revised: ∈T k (cid:80) j q xyj log p xyj is the reliability of a reference y in the k th chain, and β k ( x ) = (cid:80) y ∈R k (cid:80) j q xyj log p xyj is the afﬁnity of an instance x to the references in the k th chain.

Original: Note that β k ( x ) = − (cid:80)
Revised: Note that β k ( x ) = − (cid:80)

Original: y ∈R k D ( q xy (cid:107) p xy ) , where D is the Kullback-Leibler distance (Cover & Thomas, 2006).
Revised: y ∈R k D ( q xy (cid:107) p xy ) , where D is the Kullback-Leibler distance (Cover & Thomas, 2006).

Original: Second, after ﬁxing the chain membership T k for each chain k , we select references y to maximize the reliability scores α k ( y ) .
Revised: Second, after ﬁxing the chain membership T k for each chain k , we select references y to maximize the reliability scores α k ( y ) .

Original: These references form R k .
Revised: These references form R k .

Original: Third, after ﬁxing R 0 , . .
Revised: Third, after ﬁxing R 0 , . .

Original: , R K − 1 , we update the chain membership
Revised: , R K − 1 , we update the chain membership

Original: T 0 , . .
Revised: T 0 , . .

Original: , T K − 1 , by assigning each training instance x to the k th chain that maximizes the afﬁnity score β k ( x ) .
Revised: , T K − 1 , by assigning each training instance x to the k th chain that maximizes the afﬁnity score β k ( x ) .

Original: The second and third steps are iteratively repeated.
Revised: The second and third steps are iteratively repeated.

Original: Both steps decrease the same loss (cid:96) co in (9).
Revised: Both steps decrease the same loss (cid:96) co in (9).

Original: The second and third steps are analogous to the centroid rule and the nearest neighbor rule in the Kmeans clustering (Gersho & Gray, 1991), respectively.
Revised: The second and third steps are analogous to the centroid rule and the nearest neighbor rule in the Kmeans clustering (Gersho & Gray, 1991), respectively.

Original: The second step determines representatives in each chain (or cluster), while the third step assigns each instance to an optimal chain according to the afﬁnity.
Revised: The second step determines representatives in each chain (or cluster), while the third step assigns each instance to an optimal chain according to the afﬁnity.

Original: Furthermore, both steps decrease the same loss alternately.
Revised: Furthermore, both steps decrease the same loss alternately.

Original: However, as described in Algorithm 1 , we modify this iterative algorithm by including the membership reﬁnement step in lines 10 ∼ 12 .
Revised: However, as described in Algorithm 1 , we modify this iterative algorithm by including the membership reﬁnement step in lines 10 ∼ 12 .

Original: Speciﬁcally, we train a K -way type classiﬁer using
Revised: Speciﬁcally, we train a K -way type classiﬁer using

Original: T 0 , . .
Revised: T 0 , . .

Original: , T K − 1 .
Revised: , T K − 1 .

Original: Then, we accept the type classiﬁcation results to reﬁne T 0 , . .
Revised: Then, we accept the type classiﬁcation results to reﬁne T 0 , . .

Original: , T K − 1 .
Revised: , T K − 1 .

Original: This reﬁnement is necessary because the type classiﬁer should be used in the test phase to determine the chain of an unseen instance.
Revised: This reﬁnement is necessary because the type classiﬁer should be used in the test phase to determine the chain of an unseen instance.

Original: Therefore, it is desirable to select the references also after reﬁning the chain membership.
Revised: Therefore, it is desirable to select the references also after reﬁning the chain membership.

Original: Also, in line 7, if we assign an instance x to maximize β k ( x ) only, some classes may be assigned too few training instances, leading to data imbalance.
Revised: Also, in line 7, if we assign an instance x to maximize β k ( x ) only, some classes may be assigned too few training instances, leading to data imbalance.

Original: To avoid this, we enforce the regularization constraint so that every class is assigned at least a predeﬁned number of instances.
Revised: To avoid this, we enforce the regularization constraint so that every class is assigned at least a predeﬁned number of instances.

Original: This regularized membership update is described in Appendix B.
Revised: This regularized membership update is described in Appendix A.

Original:  A GE ESTIMATION
Revised:  A GE ESTIMATION

Original: We develop an age estimator based on the proposed order learning.
Revised: We develop an age estimator based on the proposed order learning.

Original: Order learning is suitable for age estimation, since telling the older one between two people is easier than estimating each person’s age directly (Chang et al., 2010; Zhang et al., 2017a).
Revised: Order learning is suitable for age estimation, since telling the older one between two people is easier than estimating each person’s age directly (Chang et al., 2010; Zhang et al., 2017a).

Original: 4. I MPLEMENTATION DETAILS
Revised: 4. I MPLEMENTATION DETAILS

Original: It is much easier to distinguish between a 5-year-old and a 10-year-old than between a 65-yearold and a 70-year-old.
Revised: It is much easier to distinguish between a 5-year-old and a 10-year-old than between a 65-yearold and a 70-year-old.

Original: Therefore, in age estimation, we replace the categorization based on the  arithmetic difference in (1) ∼ (3) with that based on the geometric ratio as follows.
Revised: Therefore, in age estimation, we replace the categorization based on the [Table] arithmetic difference in (1) ∼ (3) with that based on the geometric ratio as follows.

Original: [Equation]
Revised: [Equation]

Original: which represent ‘older,’ ‘similar,’ and ‘younger.’
Revised: which represent ‘older,’ ‘similar,’ and ‘younger.’

Original: The consistency in (4) is also modiﬁed accordingly.
Revised: The consistency in (4) is also modiﬁed accordingly.

Original: There are 5 reference images for each age class within range [15 , 80] in this work ( M = 330 , N = 66 ).
Revised: There are 5 reference images for each age class within range [15 , 80] in this work ( M = 330 , N = 66 ).

Original: Thus, a test image should be compared with 330 references.
Revised: Thus, a test image should be compared with 330 references.

Original: However, we develop a twostep approach, which does at most 130 comparisons but performs as good as the method usingcomparisons.
Revised: However, we develop a twostep approach, which does at most 130 comparisons but performs as good as the method usingcomparisons.

Original: The two-step estimation is employed in all experiments.
Revised: The two-step estimation is employed in all experiments.

Original: It is described in Appendix C.
Revised: It is described in Appendix B.

Original: We align all facial images using SeetaFaceEngine (Zhang et al., 2014) and resize them into 256 × 256 × 3 .
Revised: We align all facial images using SeetaFaceEngine (Zhang et al., 2014) and resize them into 256 × 256 × 3 .

Original: Then, we crop a resized image into 224 × 224 × 3 for age estimation.
Revised: Then, we crop a resized image into 224 × 224 × 3 .

Original: For thetwin feature extractors in Figure 2, we use VGG16 without the FC layers (Simonyan & Zisserman, 2015).
Revised: For the feature extractors in Figure 2, we use VGG16 without the FC layers (Simonyan & Zisserman, 2015).

Original: Each feature extractor yields a 512-channel feature vector.
Revised: They yield 512-channel feature vectors.

Original: Then, thetwo feature vectors are concatenated andthen input to the ternary classiﬁer, which has three FC layers, yielding 512-, 512-, and 3-channel vectors sequentially.
Revised: Then, the vectors are concatenated and input to the ternary classiﬁer, which has three FC layers, yielding 512-, 512-, and 3-channel vectors sequentially.

Original: Thelast 3-channel vector is normalized to the softmax probabilities of the three categories ‘ (cid:31) , ≈ , ≺ .’
Revised: The 3-channel vector is normalized to the softmax probabilities of the three categories ‘ (cid:31) , ≈ , ≺ .’

Original: In (10) ∼ (12), τ age is set to 0.1.
Revised: In (10) ∼ (12), τ age is set to 0.1.

Original: In K CH with K ≥ 2 , the type (or chain) of a test image should be determined.
Revised: In K CH with K ≥ 2 , the type (or chain) of a test image should be determined.

Original: Thus, we design a type classiﬁer, which shares the feature extractor with the comparator.
Revised: Thus, we design a type classiﬁer, which shares the feature extractor with the comparator.

Original: Similarly to the ternary classiﬁer, the type classiﬁer uses three FC layers, yielding 512-, 512-, and K -channel vectors sequentially.
Revised: Similarly to the ternary classiﬁer, the type classiﬁer uses three FC layers, yielding 512-, 512-, and K -channel vectors sequentially.

Original: The comparator and the type classiﬁer are jointly trained.
Revised: The comparator and the type classiﬁer are jointly trained.

Original: To initialize the feature extractors, we adopt the VGG16 parameters pre-trained on ImageNet (Deng et al., 2009).
Revised: To initialize the feature extractors, we adopt the VGG16 parameters pre-trained on ImageNet (Deng et al., 2009).

Original: We randomly initialize all the other layers.
Revised: We randomly initialize all the other layers.

Original: We update the parameters using the Adam optimizer (Kingma & Ba, 2014).
Revised: We update the parameters using the Adam optimizer (Kingma & Ba, 2014).

Original: We set the learning rate to 10 − 4 for the ﬁrst 70 epochs.
Revised: We set the learning rate to 10 − 4 for the ﬁrst 70 epochs.

Original: Then, we select 5 references for each age class.
Revised: Then, we select 5 references for each age class.

Original: Using the selected references, we ﬁne-tune the network with a learning rate of 10 − 5 .
Revised: Using the selected references, we ﬁne-tune the network with a learning rate of 10 − 5 .

Original: We repeat the reference selection and the parameter ﬁne-tuning up to 3 times.
Revised: We repeat the reference selection and the parameter ﬁne-tuning up to 3 times.

Original: In the case of unsupervised chains, we enforce the regularization constraint (line 7 in Algorithm 1 ).
Revised: In the case of unsupervised chains, we enforce the regularization constraint (line 7 in Algorithm 1 ).

Original: By default, for each age, all chains are constrained to be assigned the same number of training images.
Revised: By default, for each age, all chains are constrained to be assigned the same number of training images.

Original: If there are L training images of θ -year-olds, the age classes θ in the K chains are assigned
Revised: If there are L training images of θ -year-olds, the age classes θ in the K chains are assigned

Original: L/K images, respectively, according to the afﬁnity scores β k ( x ) by Algorithm 2 in Appendix B.
Revised: L/K images, respectively, according to the afﬁnity scores β k ( x ) by Algorithm 2 in Appendix A.

Original:  D ATASETS AND EVALUATION METRICS
Revised: 4. D ATASETS AND EVALUATION METRICS

Original: MORPH II (Ricanek & Tesafaye, 2006) is the most popular age estimation benchmark, containing about 55,000 facial images in the age range [16 , 77] .
Revised: MORPH II (Ricanek & Tesafaye, 2006) is the most popular age estimation benchmark, containing about 55,000 facial images in the age range [16 , 77] .

Original: As evaluation protocols for MORPH II, we use  the 5-fold subject-exclusive (SE) and the 5-fold random split (RS) with the cross-validation (Chang et al., 2010; Guo & Wang, 2012), which are widely used.
Revised: As evaluation protocols for MORPH II, we use four different settings, including the 5-fold subject-exclusive (SE) and the 5-fold random split (RS) with the cross-validation (Chang et al., 2010; Guo & Wang, 2012).

Revised: Appendix C describes these settings in detail.

Revised: IMDB-WIKI (Ricanek & Tesafaye, 2006) is a dataset containing about 500,000 celebrity images obtained from IMDB and Wikipedia.

Revised: It is often used to pretrain age estimation networks.

Revised: Optionally, we select 150,000 clean data from IMDB-WIKI to pre-train the pairwise comparator in this work.

Original: Although several facial age datasets are available, most are biased to speciﬁc ethnic groups or genders.
Revised: Although several facial age datasets are available, most are biased to speciﬁc ethnic groups or genders.

Original: Data unbalance restricts the usability and degrades the generalization performance.
Revised: Data unbalance restricts the usability and degrades the generalization performance.

Original: Thus, we form a ‘balanced dataset’ from MORPH II, AFAD (Niu et al., 2016), and UTK (Zhang et al., 2017b).
Revised: Thus, we form a ‘balanced dataset’ from MORPH II, AFAD (Niu et al., 2016), and UTK (Zhang et al., 2017b).

Original: Table 1 shows how the balanced dataset is organized.
Revised: Table 1 shows how the balanced dataset is organized.

Original: Before sampling images from MORPH II,
Revised: Before sampling images from MORPH II,

Revised: [Table]

Original: AFAD, and UTK, we rectify inconsistent labels by following the strategy in Yip et al.
Revised: AFAD, and UTK, we rectify inconsistent labels by following the strategy in Yip et al.

Original: For each combination of gender in { female, male } and ethnic group in { African, Asian, European } , we sample about 6,000 images.
Revised: For each combination of gender in { female, male } and ethnic group in { African, Asian, European } , we sample about 6,000 images.

Original: Also, during the sampling, we attempt to make the age distribution as uniform as possible within range [15 , 80] .
Revised: Also, during the sampling, we attempt to make the age distribution as uniform as possible within range [15 , 80] .

Original: The balanced dataset is partitioned into training and test subsets with ratio 8 : 2 .
Revised: The balanced dataset is partitioned into training and test subsets with ratio 8 : 2 .

Original: [Table] [Table] For performance assessment, we calculate the mean absolute error (MAE) (Lanitis et al., 2004) and the cumulative score (CS) (Geng et al., 2006).
Revised: For performance assessment, we calculate the mean absolute error (MAE) (Lanitis et al., 2004) and the cumulative score (CS) (Geng et al., 2006).

Original: MAE is the average absolute error between predicted and ground-truth ages.
Revised: MAE is the average absolute error between predicted and ground-truth ages.

Original: Given a tolerance level l , CS computes the percentage of test images whose absolute errors are less than or equal to l .
Revised: Given a tolerance level l , CS computes the percentage of test images whose absolute errors are less than or equal to l .

Original: In this work, l is ﬁxed to 5, as done in Chang et al.
Revised: In this work, l is ﬁxed to 5, as done in Chang et al.

Original: Han et al.
Revised: Han et al.

Original: (2018), and Shen et al.
Revised: (2018), and Shen et al.

Original: 4. E XPERIMENTAL RESULTS
Revised: 4. E XPERIMENTAL RESULTS

Original: Table 2 compares the proposed algorithm (1CH) with conventional algorithms on MORPH II.
Revised: Table 2 compares the proposed algorithm (1CH) with conventional algorithms on MORPH II.

Original: REDSVM, OHRank, OR-CNN, and Ranking-CNN are all based on ordinal regression.
Revised: OHRank, OR-CNN, and Ranking-CNN are all based on ordinal regression.

Original: RED-SVM and

Original: OHRank use traditional features, yielding relatively poor performances.
Revised: OHRank uses traditional features, yielding relatively poor performances.

Original: The other algorithms use CNN features.
Revised: OR-CNN and Ranking-CNN use CNN features.

Original: DEX,  MO-CNN,CMT, MV, and DRFs employ VGG16 as backbone networks.
Revised: DEX, DRFs, MO-CNN, MV, and BridgeNet employ VGG16 as backbone networks.

Original: Among them, MV and DRFs achieve the state-of-the-art results, by employing the mean-variance loss and the deep regression forests, respectively.
Revised: Among them, MV and BridgeNet achieve the state-of-the-art results, by employing the mean-variance loss and the gating networks, respectively.

Original: The proposed algorithm outperforms these algorithms, except for MAE in  the RS protocol, while using the same VGG16 backbone.
Revised: The proposed algorithm outperforms these algorithms in setting C, which is the most challenging task. Furthermore, in terms of CS, the proposed algorithm yields the best performances in all four settings.

Original: Its outstanding performances indicate that order learning is an effective approach to age estimation.
Revised: These outstanding performances indicate that order learning is an effective approach to age estimation.

Original: In Table 3, we analyze the performances of the proposed algorithm on the balanced dataset according to the number of hypothesized chains.
Revised: In Table 3, we analyze the performances of the proposed algorithm on the balanced dataset according to the number of hypothesized chains.

Original: We also implement and train the state-of-the-art MV on the balanced dataset and provide its results using supervised chains.
Revised: We also implement and train the state-of-the-art MV on the balanced dataset and provide its results using supervised chains.

Original: Let us ﬁrst analyze the performances of the proposed algorithm using ‘supervised’ chains.
Revised: Let us ﬁrst analyze the performances of the proposed algorithm using ‘supervised’ chains.

Original: The
Revised: The

Original: MAE and CS scores on the balanced dataset are worse than those on MORPH II, since the balanced dataset contains more diverse data and thus is more challenging.
Revised: MAE and CS scores on the balanced dataset are worse than those on MORPH II, since the balanced dataset contains more diverse data and thus is more challenging.

Original: By processing facial images separately according to the genders (2CH), the proposed algorithm reduces MAE by 0.05 and improves
Revised: By processing facial images separately according to the genders (2CH), the proposed algorithm reduces MAE by 0.05 and improves

Original: CS by 0.2% in comparison with 1CH.
Revised: CS by 0.2% in comparison with 1CH.

Original: Similar improvements are obtained by 3CH or 6CH, which consider the ethnic groups only or both gender and ethnic groups, respectively.
Revised: Similar improvements are obtained by 3CH or 6CH, which consider the ethnic groups only or both gender and ethnic groups, respectively.

Original: In contrast, in the case of MV, multi-chain hypotheses sometimes degrade the performances; e.g. , MV (6CH) yields a lower CS than MV (1CH).
Revised: In contrast, in the case of MV, multi-chain hypotheses sometimes degrade the performances; e.g. , MV (6CH) yields a lower CS than MV (1CH).

Original: Regardless of the number of chains, the proposed algorithm trains a single comparator but uses a different set of references for each chain.
Revised: Regardless of the number of chains, the proposed algorithm trains a single comparator but uses a different set of references for each chain.

Original: The comparator is a ternary classiﬁer.
Revised: The comparator is a ternary classiﬁer.

Original: In contrast, MV (6CH) should train six different age estimators, each of which is a 66-way[Table] [Figure] classiﬁer, to handle different chains.
Revised: In contrast, MV (6CH) should train six different age estimators, each of which is a 66-way classiﬁer, to handle different chains.

Original: Thus, their training is more challenging than that of the single ternary classiﬁer.
Revised: Thus, their training is more challenging than that of the single ternary classiﬁer.

Original: Note that, for the multi-chain hypotheses, the proposed algorithm ﬁrst identiﬁes the chain of a test image using the type classiﬁers, whose accuracies are about 98%.
Revised: Note that, for the multi-chain hypotheses, the proposed algorithm ﬁrst identiﬁes the chain of a test image using the type classiﬁers, whose accuracies are about 98%.

Original: In Table 3, these type classiﬁers are used to obtain the results of the proposed algorithm, whereas the ground-truth gender and ethnic group of each test image are used for MV.
Revised: In Table 3, these [Table] [Figure]

Original: Figure 4 shows how to estimate an age in 6CH.
Revised: Figure 4 shows how to estimate an age in 6CH.

Original: In this test, the subject is a 22-year-old Asian male.
Revised: In this test, the subject is a 22-year-old Asian male.

Original: He is compared with the references who are also Asian males.
Revised: He is compared with the references who are also Asian males.

Original: Using the comparison results, the age is correctly estimated as 22 by the MC rule in (5).
Revised: Using the comparison results, the age is correctly estimated as 22 by the MC rule in (5).

Original: Table 4 lists the MAE results for each test chain.
Revised: Table 4 lists the MAE results for each test chain.

Original: Europeans yield poorer MAEs than Africans or
Revised: Europeans yield poorer MAEs than Africans or

Original: Asians.
Revised: Asians.

Original: However, this is not due to inherent differences between ethnic groups.
Revised: However, this is not due to inherent differences between ethnic groups.

Original: It is rather caused by differences in image qualities.
Revised: It is rather caused by differences in image qualities.

Original: As listed in Table 1, more European faces are sampled from UTK.
Revised: As listed in Table 1, more European faces are sampled from UTK.

Original: The UTK faces were crawled from the Internet and their qualities are relatively low.
Revised: The UTK faces were crawled from the Internet and their qualities are relatively low.

Original: Also, from the cross-chain test results using 6CH, some observations can be made:
Revised: Also, from the cross-chain test results using 6CH, some observations can be made:

Original: • Except for the As-F test chain, the lowest MAE is achieved by the references in the same chain.
Revised: • Except for the As-F test chain, the lowest MAE is achieved by the references in the same chain.

Original: • Eu-M and Eu-F are mutually compatible.
Revised: • Eu-M and Eu-F are mutually compatible.

Original: For Eu-M, the second best performance is obtained by the Eu-F references, and vice versa.
Revised: For Eu-M, the second best performance is obtained by the Eu-F references, and vice versa.

Original: On the other hand, some chains, such as Af-M and Eu-F, are less compatible for the purpose of the proposed age estimation.
Revised: On the other hand, some chains, such as Af-M and Eu-F, are less compatible for the purpose of the proposed age estimation.

Original: Table 3 also includes the performances of the proposed algorithm using ‘unsupervised’ chains.
Revised: Table 3 also includes the performances of the proposed algorithm using ‘unsupervised’ chains.

Original: The unsupervised algorithm outperforms the supervised one, which indicates that the gender or ethnic group information is not the best way to divide data for age estimation.
Revised: The unsupervised algorithm outperforms the supervised one, which indicates that the gender or ethnic group information is not the best way to divide data for age estimation.

Original: As in the supervised case, 2CH, 3CH, and 6CH yield similar performances, which means that two chains are enough for the balanced set.
Revised: As in the supervised case, 2CH, 3CH, and 6CH yield similar performances, which means that two chains are enough for the balanced set.

Original: Compared with MV (1CH), the unsupervised algorithm (2CH) improves the performances signiﬁcantly, by 0.33 in terms of MAE and 4.1% in terms of CS.
Revised: Compared with MV (1CH), the unsupervised algorithm (2CH) improves the performances signiﬁcantly, by 0.33 in terms of MAE and 4.1% in terms of CS.

Original: Figure 5 shows how training images are divided into two chains in the unsupervised 2CH.
Revised: Figure 5 shows how training images are divided into two chains in the unsupervised 2CH.

Original: During the membership update, for each age, each chain is regularized to include at least a certain percentage ( κ ) of the training images.
Revised: During the membership update, for each age, each chain is regularized to include at least a certain percentage ( κ ) of the training images.

Original: In the default mode, the two chains are assigned the same number of images with κ = 50% .
Revised: In the default mode, the two chains are assigned the same number of images with κ = 50% .

Original: However, Appendix D.2 shows that the performance is not very sensitive to κ .
Revised: However, Appendix D.2 shows that the performance is not very sensitive to κ .

Original: At κ = 10% , MAE = 4.17 and CS = 73.7%.
Revised: At κ = 10% , MAE = 4.17 and CS = 73.7%.

Original: From Figure 5, we observe
Revised: From Figure 5, we observe

Original: • The division of the chains is not clearly related to genders or ethnic groups.
Revised: • The division of the chains is not clearly related to genders or ethnic groups.

Original: Regardless of genders or ethnic groups, about half of the images are assigned to chain 1 and the others to chain 2.
Revised: Regardless of genders or ethnic groups, about half of the images are assigned to chain 1 and the others to chain 2.

Original: • At κ = 10% , chain 1 mostly consists of middle ages, while chain 2 of 10s, 20s, 60s, and 70s.
Revised: • At κ = 10% , chain 1 mostly consists of middle ages, while chain 2 of 10s, 20s, 60s, and 70s.

Original: • At κ = 50% , there is no such strong age-dependent tendency.
Revised: • At κ = 50% , there is no such strong age-dependent tendency.

Original: But, for some combinations of gender, ethnic group, and age band, it is not equal division.
Revised: But, for some combinations of gender, ethnic group, and age band, it is not equal division.

Original: For example, for Asian females, a majority of 40s are assigned to chain 1 but a majority of 50s and 60s are assigned to chain2. [Table] [Figure]
Revised: For example, for Asian females, a majority of 40s are assigned to chain 1 but a majority of 50s and 60s are assigned to chain

Original: The unsupervised algorithm is designed to divide instances into multiple clusters when gender and ethnic group information is unavailable.
Revised: The unsupervised algorithm is designed to divide instances into multiple clusters when gender and ethnic group information is unavailable.

Original: As shown in Appendix D.2, different κ ’s yield various clustering results.
Revised: As shown in Appendix D.2, different κ ’s yield various clustering results.

Original: Surprisingly, these different clusters still outperform the supervised algorithm.
Revised: Surprisingly, these different clusters still outperform the supervised algorithm.

Original:  For example, at κ = 10% , let us consider the age band of 20s and 30s.
Revised: [Table] [Figure] For example, at κ = 10% , let us consider the age band of 20s and 30s.

Original: If the references in chainare used to estimate the ages of people in chain 1, the average error is 4 . 6 years.
Revised: If the references in chainare used to estimate the ages of people in chain 1, the average error is 4 . 6 years.

Original: On the contrary, if the references in chain 1 are used for chain 2, the average error is − 5 .
Revised: On the contrary, if the references in chain 1 are used for chain 2, the average error is − 5 .

Original: 4 years.
Revised: 4 years.

Original: These opposite biases mean that people in chain 1 tend to look older than those in chain 2.
Revised: These opposite biases mean that people in chain 1 tend to look older than those in chain 2.

Original: These ‘looking-older’ people in 20s and 30s compose the blue cluster (chain 1) together with most people in 40s and 50s in Figure 5.
Revised: These ‘looking-older’ people in 20s and 30s compose the blue cluster (chain 1) together with most people in 40s and 50s in Figure 5.

Original: In this case, ‘looking-older’ people in 20s and 30s are separated from ‘looking-younger’ ones by the unsupervised algorithm.
Revised: In this case, ‘looking-older’ people in 20s and 30s are separated from ‘looking-younger’ ones by the unsupervised algorithm.

Original: This is more effective than the gender-based or ethnic-group-based division of the supervised algorithm.
Revised: This is more effective than the gender-based or ethnic-group-based division of the supervised algorithm.

Original: Appendix D presents more results on age estimation.
Revised: Appendix D presents more results on age estimation.

Original: The proposed order learning is applicable to other regression problems, as well as age estimation.

Original: Image aesthetic assessment is a challenging task due to its subjectivity (Deng et al., 2017).

Original: We develop an aesthetic score regressor based on order learning.

Original: Appendix A shows that the aesthetic regressor also yields excellent performance.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: Order learning was proposed in this work.
Revised: Order learning was proposed in this work.

Original: In order learning, classes form an ordered set, and each class represents object instances of the same rank.
Revised: In order learning, classes form an ordered set, and each class represents object instances of the same rank.

Original: Its goal is to determine the order graph of classes and classify a test instance into one of the classes.
Revised: Its goal is to determine the order graph of classes and classify a test instance into one of the classes.

Original: To this end, we designed the pairwise comparator to learn ordering relationship between instances.
Revised: To this end, we designed the pairwise comparator to learn ordering relationship between instances.

Original: We then decided the class of an instance by comparing it with reference instances in the same chain and maximizing the consistency among the comparison results.
Revised: We then decided the class of an instance by comparing it with reference instances in the same chain and maximizing the consistency among the comparison results.

Original: For age estimation, it was shown that the proposed algorithm yields the stateof-the-art performance even in the case of the single-chain hypothesis.
Revised: For age estimation, it was shown that the proposed algorithm yields the stateof-the-art performance even in the case of the single-chain hypothesis.

Original: The performance is further improved when the order graph is divided into multiple disjoint chains.
Revised: The performance is further improved when the order graph is divided into multiple disjoint chains.

Original: Furthermore, the proposed order learning can be applied successfully to image aesthetic assessment as well.

Original: In this paper, we assumed that the order graph is composed of disjoint chains.
Revised: In this paper, we assumed that the order graph is composed of disjoint chains.

Original: However, there are more complicated graphs, e.g. Figure 1(a), than disjoint chains.
Revised: However, there are more complicated graphs, e.g. Figure 1(a), than disjoint chains.

Original: For example, it is hard to recognize an infant’s sex from its facial image (Porter et al., 1984).
Revised: For example, it is hard to recognize an infant’s sex from its facial image (Porter et al., 1984).

Original: But, after puberty, male and female take divergent paths.
Revised: But, after puberty, male and female take divergent paths.

Original: This can be reﬂected by an order graph, which consists of two chains sharing common nodes up to a certain age.
Revised: This can be reﬂected by an order graph, which consists of two chains sharing common nodes up to a certain age.

Original: It is an open problem to generalize order learning to ﬁnd an optimal order graph, which is not restricted to disjoint chains.
Revised: It is an open problem to generalize order learning to ﬁnd an optimal order graph, which is not restricted to disjoint chains.

Original: A A ESTHETIC ASSESSMENT
Revised: A CKNOWLEDGEMENTS

Original: Recently, many attempts have been made for image aesthetic assessment (Deng et al., 2017).

Original: However, aesthetic assessment is challenging due to its inherent subjectiveness.

Original: Fewer algorithms have been proposed for aesthetic score regression than for binary aesthetic classiﬁcation, which is relatively easy. Table 5 compares the proposed algorithm with Reg-Net (Kong et al., 2016) and the aesthetic scaling method (ASM) (Lee & Kim, 2019b) on the aesthetics and attribute database (AADB) (Kong et al., 2016). In AADB, there are 8,500 training, 500 validation, and 1,000 testing images. Each image in
Revised: This work was supported by ‘The Cross-Ministry Giga KOREA Project’ grant funded by the Korea government (MSIT) (No. GK18P0200, Development of 4D reconstruction and dynamic deformable action model based hyperrealistic service technology), and by the National

Original: AADB is annotated with an aesthetic score and eleven attributes, such as rule-of-thirds and vividcolor.

Original: Aesthetic scores range from 0 to 1, with 1 representing the highest quality.

Original: [Table]

Original: The Spearman’s ρ is the correlation coefﬁcient between the ranks, obtained from ground-truth scores and regressed scores, respectively.

Original: Also, the mean of absolute score differences (MASD) computes the differences between ground-truth and regressed scores directly and averages them.

Original: ASM is the state-of-the-art algorithm, which also uses pairwise comparisons.

Original: However, it selects references heuristically and combines the comparison results using the Saaty’s scaling method (Saaty, 1977).

Original: We implement an aesthetic score regressor based on order learning.

Original: As in ASM, the proposed aggressor adopts ResNet-50 (He et al., 2016) for feature extraction.

Original: In this test, 1CH is implemented.

Original: We select M = 112 references to minimize the comparator loss in (6).

Original: In other words, the reference selection and the parameter optimization are performed jointly.

Original: We see that the proposed regressor provides better performances than the state-of-the-art ASM in terms of both ρ and MASD.

Original: B R EGULARIZED MEMBERSHIP UPDATE
Revised: A R EGULARIZED MEMBERSHIP UPDATE

Original: During the chain membership update in Algorithm 1 , we assign an instance x to chain k to maximize β k ( x ) subject to the regularization constraint.
Revised: During the chain membership update in Algorithm 1 , we assign an instance x to chain k to maximize β k ( x ) subject to the regularization constraint.

Original: As mentioned in Section 4.1, in age estimation, this regularization is enforced for each age.
Revised: As mentioned in Section 4.1, in age estimation, this regularization is enforced for each age.

Original: Let X denote the set of θ -year-olds for a certain θ .
Revised: Let X denote the set of θ -year-olds for a certain θ .

Original: Also, let K
Revised: Also, let K

Original: = { 0 , 1 , . .
Revised: = { 0 , 1 , . .

Original: , K − 1 } be the set of chains.
Revised: , K − 1 } be the set of chains.

Original: Suppose that we should assign at least a certain number ( L ) of instances in X to each chain.
Revised: Suppose that we should assign at least a certain number ( L ) of instances in X to each chain.

Original: This is done by calling RegularAssign( K , X , L ) in
Revised: This is done by calling RegularAssign( K , X , L ) in

Original: Algorithm 2 , which is a recursive function.
Revised: Algorithm 2 , which is a recursive function.

Original: Algorithm 2 yields the membership function c ( x ) as output.
Revised: Algorithm 2 yields the membership function c ( x ) as output.

Original: For example, c ( x ) = 1 means that x belongs to chain 1.
Revised: For example, c ( x ) = 1 means that x belongs to chain 1.

Original: C T WO - STEP ESTIMATION
Revised: B T WO - STEP ESTIMATION

Original: There are 5 reference images for each age within range [15 , 80] in this work.
Revised: There are 5 reference images for each age within range [15 , 80] in this work.

Original: Thus, for the age estimation of a test image using the MC rule in (5), the test image should be compared with M =reference images.
Revised: Thus, for the age estimation of a test image using the MC rule in (5), the test image should be compared with M =reference images.

Original: However, we reduce the number of comparisons using a two-step approach.
Revised: However, we reduce the number of comparisons using a two-step approach.

Original: First, the test image is compared with the 35 references of ages 15 , 25 , . . .
Revised: First, the test image is compared with the 35 references of ages 15 , 25 , . . .

Original: , 75 only, and a rough age estimate
Revised: , 75 only, and a rough age estimate

Original: ˆ θ 1 is obtained using the MC rule.
Revised: ˆ θ 1 is obtained using the MC rule.

Original: Second, it is compared with the 105 references of all ages within [ˆ θ 1 − 10 , ˆ θ 1 + 10] , and the ﬁnal estimate ˆ θ 2 is obtained.
Revised: Second, it is compared with the 105 references of all ages within [ˆ θ 1 − 10 , ˆ θ 1 + 10] , and the ﬁnal estimate ˆ θ 2 is obtained.

Original: Since there are at least 10 common references in the ﬁrst and second steps, the two-step estimation requires at most 130 comparisons.
Revised: Since there are at least 10 common references in the ﬁrst and second steps, the two-step estimation requires at most 130 comparisons.

Original: D A GE ESTIMATION – M ORE EXPERIMENTS D. G ENERALIZATION PERFORMANCE OF COMPARATOR ON FG-NET
Revised: C F OUR EXPERIMENTAL SETTINGS ON MORPH II

Revised: • Setting A: 5,492 images of Europeans are randomly selected and then divided into training and testing sets with ratio 8:2 (Chang et al., 2011). • Setting B: About 21,000 images are randomly selected, while restricting the ratio between Africans and Europeans to 1:1 and that between females and males to 1:3. They are divided into three subsets (S1, S2, S3).

Revised: The training and testing are done under two sub-settings (Guo & Mu, 2011).

Revised: – (B1) training on S1, testing on S2 + S – (B2) training on S2, testing on S1 + S3.

Revised: • Setting C:

Revised: The entire dataset is randomly split into ﬁve folds, and the 5-fold cross-validation is performed.

Revised: • Setting D: It is the same as setting D, except that the same person’s images are constrained to belong to only one fold.

Revised: D M ORE EXPERIMENTS D. G ENERALIZATION PERFORMANCE OF COMPARATOR ON FG-NET

Original: We assess the proposed age estimator (1CH) on the FG-NET database (Panis et al., 2016).
Revised: We assess the proposed age estimator (1CH) on the FG-NET database (Panis et al., 2016).

Original: FG-NET is a relatively small dataset, composed of 1,002 facial images of 82 subjects.
Revised: FG-NET is a relatively small dataset, composed of 1,002 facial images of 82 subjects.

Original: Ages range from 0 to 69.
Revised: Ages range from 0 to 69.

Original: For FG-NET, the leave one person out (LOPO) approach is often used for evaluation.
Revised: For FG-NET, the leave one person out (LOPO) approach is often used for evaluation.

Original: In other words, to perform tests on each subject, an estimator is trained using the remaining 81 subjects.
Revised: In other words, to perform tests on each subject, an estimator is trained using the remaining 81 subjects.

Original: Then, the results are averaged over all 82 subjects.
Revised: Then, the results are averaged over all 82 subjects.

Original: In order to assess the generalization performance, we do not retrain the comparator on the FG-NET data.
Revised: In order to assess the generalization performance, we do not retrain the comparator on the FG-NET data.

Original: Instead, we ﬁx the comparator trained on the balanced dataset and just select references from the remaining subjects’ faces in each LOPO test.
Revised: Instead, we ﬁx the comparator trained on the balanced dataset and just select references from the remaining subjects’ faces in each LOPO test.

Original: For the comparator, the arithmetic scheme in (1) ∼ (3) is tested as well as the default geometric scheme in (10) ∼ (12).
Revised: For the comparator, the arithmetic scheme in (1) ∼ (3) is tested as well as the default geometric scheme in (10) ∼ (12).

Original: For comparison, MV (Pan et al., 2018) is tested, but it is trained for each LOPO test.
Revised: For comparison, MV (Pan et al., 2018) is tested, but it is trained for each LOPO test.

Original: Table 6 summarizes the comparison results.
Revised: Table 5 summarizes the comparison results.

Original: MV provides better average performances on the entire age range [0 , 69] than the proposed algorithm does.
Revised: MV provides better average performances on the entire age range [0 , 69] than the proposed algorithm does.

Original: This is because the balanced dataset does not include subjects of ages between 0 and 14.
Revised: This is because the balanced dataset does not include subjects of ages between 0 and 14.

Original: If we reduce the test age range to [15 , 69] , the proposed algorithm outperforms MV, even though the comparator is not retrained.
Revised: If we reduce the test age range to [15 , 69] , the proposed algorithm outperforms MV, even though the comparator is not retrained.

Original: These results indicate that the comparator generalizes well to unseen data, as long as the training images cover a desired age range.
Revised: These results indicate that the comparator generalizes well to unseen data, as long as the training images cover a desired age range.

Original: Also, note that the geometric scheme provides better performances than the arithmetic scheme.
Revised: Also, note that the geometric scheme provides better performances than the arithmetic scheme.

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: 64.3 Proposed (1CH, Arithmetic τ = 7 ) 9.

Original: Figure 6 compares MAEs according to a test age.
Revised: Figure 6 compares MAEs according to a test age.

Original: Again, within the covered range [15 , 69] , the proposed algorithm signiﬁcantly outperforms MV especially when test subjects are older than 45.
Revised: Again, within the covered range [15 , 69] , the proposed algorithm signiﬁcantly outperforms MV especially when test subjects are older than 45.

Original: [Figure]
Revised: [Figure]

Original: D. P ERFORMANCE ACCORDING TO κ
Revised: D. P ERFORMANCE ACCORDING TO κ

Original: [Figure]
Revised: [Figure]

Original: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: D. P ERFORMANCE ACCORDING TO THRESHOLDS τ AND τ age
Revised: D. P ERFORMANCE ACCORDING TO THRESHOLDS τ AND τ age

Original: The ordering relationship between two instances can be categorized via the arithmetic scheme in (1) ∼ (3) using a threshold τ or the geometric scheme in (10) ∼ (12) using a threshold τ age .
Revised: The ordering relationship between two instances can be categorized via the arithmetic scheme in (1) ∼ (3) using a threshold τ or the geometric scheme in (10) ∼ (12) using a threshold τ age .

Original: Table 8 lists the performances of the proposed algorithm (1CH) according to these thresholds.
Revised: Table 7 lists the performances of the proposed algorithm (1CH) according to these thresholds.

Original: We see that the geometric scheme outperforms the arithmetic scheme in general.
Revised: We see that the geometric scheme outperforms the arithmetic scheme in general.

Original: The best performance is achieved with τ age = 0 .
Revised: The best performance is achieved with τ age = 0 .

Original: 1 , which is used in all experiments in the main paper.
Revised: 1 , which is used in all experiments in the main paper.

Original: Note that the scores are poorer than those in Table 3, since the comparator is trained for a smaller number of epochs to facilitate this test.
Revised: Note that the scores are poorer than those in Table 3, since the comparator is trained for a smaller number of epochs to facilitate this test.

Original: At τ age = 0 .
Revised: At τ age = 0 .

Original: 1 , two teenagers are declared to be not ‘similar to’ each other if their age difference is larger than about 1.
Revised: 1 , two teenagers are declared to be not ‘similar to’ each other if their age difference is larger than about 1.

Original: Also, two forties are not ‘similar’ if the age difference is larger than about
Revised: Also, two forties are not ‘similar’ if the age difference is larger than about

Original: [Table] [Table]
Revised: [Table] [Table]

Original: D.

Original: P ERFORMANCE ACCORDING TO NUMBER OF REFERENCES
Revised: P ERFORMANCE ACCORDING TO NUMBER OF REFERENCES

Original: [Table]
Revised: ( M N )

Revised: In performances get better more references

Revised: However, the are not very sensitive to M/N .

Revised: They saturate

Revised: M

Revised: N ≥ 5 .

Revised: Therefore, we set M/N = 5 in this work

Revised: [Table]

Original: D. R EFERENCE IMAGES
Revised: D. R EFERENCE IMAGES

Original: Figure 8 shows all references in the supervised 6CH.
Revised: Figure 8 shows all references in the supervised 6CH.

Original: M a l e A fr i [Table]
Revised: [Table] [Table] [Table]

Original: n
Revised: [Table] [Table] [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Original: [Table]
Revised: [Table]

Original: F e m a l e A
Revised: e m a l e A

Original: fr i ca n
Revised: fr i [Table] n

Original: M a l e A s i a n
Revised: M a l e A s i a n

Original: F e m a l e A s i a n
Revised: F e m a l e A s i a n

Original: M a l e E u r op ea n
Revised: M a l e E u r op ea n

Original: F e m a l e E u r op ea n
Revised: F e m a l e E u r op ea n

Original: F i gu r e: A ll r e f e r e n ce
Revised: F i gu r e: A ll r e f e r e n ce

Original: i m a
Revised: i m a

Original: g e s i n t h e s up e r
Revised: g e s i n t h e s up e r

Original: v i s e d 6 C H .
Revised: v i s e d 6 C H .

Original: F o r s o m e
Revised: F o r s o m e

Original: a g e s i n ce r t
Revised: a g e s i n ce r t

Original: a
Revised: a

Original: i n c h a i n s , t h e b a l a n ce d d a t a s e t i n c l ud e l e ss t h a n 5 f ace s .
Revised: i n c h a i n s , t h e b a l a n ce d d a t a s e t i n c l ud e l e ss t h a n 5 f ace s .

Original: I n s u c h ca s e s , t h e r e a r e l e ss t h a n 5 r e f e r e n ce s .
Revised: I n s u c h ca s e s , t h e r e a r e l e ss t h a n 5 r e f e r e n ce s .

Original: D. A GE ESTIMATION EXAMPLES
Revised: D. A GE ESTIMATION EXAMPLES

Original: [Table] [Figure] 
Revised: [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]