Original: Geometric Order Learning for Rank Estimation
Revised: Geometric Order Learning for Rank Estimation

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: A novel approach to rank estimation, called geometric order learning (GOL), isproposed in this paper.
Revised: A novel approach to rank estimation, called geometric order learning (GOL), isproposed in this paper.

Original: First, we construct an embedding space, in which the direction and distance between objects represent order and metric relations betweentheir ranks, by enforcing two geometric constraints: the order constraint compelsobjects to be sorted according to their ranks, while the metric constraint makesthe distance between objects reﬂect their rank difference.
Revised: First, we construct an embedding space, in which the direction and distance between objects represent order and metric relations betweentheir ranks, by enforcing two geometric constraints: the order constraint compelsobjects to be sorted according to their ranks, while the metric constraint makesthe distance between objects reﬂect their rank difference.

Original: Then, we perform thesimple k nearest neighbor ( k -NN) search in the embedding space to estimate therank of a test object.
Revised: Then, we perform thesimple k nearest neighbor ( k -NN) search in the embedding space to estimate therank of a test object.

Original: Moreover, to assess the quality of embedding spaces forrank estimation, we propose a metric called discriminative ratio for ranking (DRR).
Revised: Moreover, to assess the quality of embedding spaces forrank estimation, we propose a metric called discriminative ratio for ranking (DRR).

Original: Extensive experiments on facial age estimation, historical color image (HCI) classiﬁcation and aesthetic score regression demonstrate that GOL constructs effectiveembedding spaces and thus yields excellent rank estimation performances.
Revised: Extensive experiments on facial age estimation, historical color image (HCI) classiﬁcation and aesthetic score regression demonstrate that GOL constructs effectiveembedding spaces and thus yields excellent rank estimation performances.

Original:  Introduction
Revised:  Introduction

Original: In rank estimation, we estimate the rank (or ordered class) of an object.
Revised: In rank estimation, we estimate the rank (or ordered class) of an object.

Original: It is different from ordinaryclassiﬁcation in that its classes are arranged in a natural order.
Revised: It is different from ordinaryclassiﬁcation in that its classes are arranged in a natural order.

Original: For example, in movie rating, classescan be ordered from ‘outstanding’ to ‘very good,’ ‘satisfactory,’ ‘unsatisfactory,’ and ‘poor.’
Revised: For example, in movie rating, classescan be ordered from ‘outstanding’ to ‘very good,’ ‘satisfactory,’ ‘unsatisfactory,’ and ‘poor.’

Original: Rankestimation is a fundamental problem and, e.g. , used for various computer vision tasks including facialage estimation (Shin et al., 2022), aesthetic quality assessment (Schifanella et al., 2015), and HCIclassiﬁcation (Palermo et al., 2012).
Revised: Rankestimation is a fundamental problem and, e.g. , used for various computer vision tasks including facialage estimation (Shin et al., 2022), aesthetic quality assessment (Schifanella et al., 2015), and HCIclassiﬁcation (Palermo et al., 2012).

Original: For rank estimation, many techniques (Li et al., 2021; Liu et al., 2018) adopt the ordinal regressionframework, which employs a classiﬁer or a regressor to predict the rank of an object directly.
Revised: For rank estimation, many techniques (Li et al., 2021; Liu et al., 2018) adopt the ordinal regressionframework, which employs a classiﬁer or a regressor to predict the rank of an object directly.

Original: However,they may fail to yield reliable estimates, for there is no clear distinction between ranks in many cases.
Revised: However,they may fail to yield reliable estimates, for there is no clear distinction between ranks in many cases.

Original: For instance, in facial age estimation, the aging process — causing variations in facial shapes, sizes,and texture — has large individual differences due to factors such as genes, diet, and lifestyle.
Revised: For instance, in facial age estimation, the aging process — causing variations in facial shapes, sizes,and texture — has large individual differences due to factors such as genes, diet, and lifestyle.

Original: To address this issue, comparison-based algorithms (Lim et al., 2020; Lee & Kim, 2021; Li et al.
Revised: To address this issue, comparison-based algorithms (Lim et al., 2020; Lee & Kim, 2021; Li et al.

Original: ,2014; Nguyen et al., 2018) have been proposed.
Revised: ,2014; Nguyen et al., 2018) have been proposed.

Original: Instead of predicting the rank directly, they learn abinary relation between objects, such as order or metric (Hrbacek & Jech, 1984).
Revised: Instead of predicting the rank directly, they learn abinary relation between objects, such as order or metric (Hrbacek & Jech, 1984).

Original: These relationsprovide useful information for rank estimation: an order indicates the relative priority between objectsx and y , while a metric informs of the distance between them.
Revised: These relationsprovide useful information for rank estimation: an order indicates the relative priority between objectsx and y , while a metric informs of the distance between them.

Original: In order learning (Lim et al., 2020; Lee & Kim, 2021), a comparator is learned to classify therelationship between x and y into one of three cases: x is ‘greater than,’ ‘similar to,’ or ‘smaller than’y .
Revised: In order learning (Lim et al., 2020; Lee & Kim, 2021), a comparator is learned to classify therelationship between x and y into one of three cases: x is ‘greater than,’ ‘similar to,’ or ‘smaller than’y .

Original: Then, the rank of a test object is estimated by comparing it with multiple reference objects withknown ranks.
Revised: Then, the rank of a test object is estimated by comparing it with multiple reference objects withknown ranks.

Original: This approach is based on the idea that it is easier to predict ordering relationshipsbetween objects than to estimate the absolute ranks; telling the older one between two people is easierthan estimating their exact ages.
Revised: This approach is based on the idea that it is easier to predict ordering relationshipsbetween objects than to estimate the absolute ranks; telling the older one between two people is easierthan estimating their exact ages.

Original: However, order learning disregards how much x is different from y .
Revised: However, order learning disregards how much x is different from y .

Original: In other words, it ignores metric information.
Revised: In other words, it ignores metric information.

Original: [Figure]
Revised: [Figure]

Original: Figures 1 (a) and (b) compare embedding spaces, obtained by order learning and metric learning,respectively.
Revised: Figures 1 (a) and (b) compare embedding spaces, obtained by order learning and metric learning,respectively.

Original: Order learning sorts instances according to ranks in general, but instances in each classare scattered in the embedding space.
Revised: Order learning sorts instances according to ranks in general, but instances in each classare scattered in the embedding space.

Original: In contrast, metric learning reduces within-class scattering butdoes not sort instances properly.
Revised: In contrast, metric learning reduces within-class scattering butdoes not sort instances properly.

Original: For example, both blue and orange ranks are adjacent to the yellowone, although they should be arranged in the order of blue, orange, and yellow.
Revised: For example, both blue and orange ranks are adjacent to the yellowone, although they should be arranged in the order of blue, orange, and yellow.

Revised: Unlike ordinaryclassiﬁcation, different errors have different severities in rank estimation: misclassifying an object inthe blue rank as a yellow one is severer than mistaking it for an orange one.

Original: This indicates thatboth order learning and metric learning construct suboptimal embedding spaces.
Revised: Thus, ordering in theembedding space is important to avoid such severe errors.

Original: We propose a novel algorithm, GOL, to estimate the rank of an object reliably by exploiting bothorder and metric relations.
Revised: We propose a novel algorithm, GOL, to estimate the rank of an object reliably by exploiting bothorder and metric relations.

Original: To this end, we construct an embedding space, in which the directionand distance between objects represent the order and metric relations between their ranks.
Revised: To this end, we construct an embedding space, in which the directionand distance between objects represent the order and metric relations between their ranks.

Original: For theconstruction, we formulate two geometric constraints in the embedding space: 1) the order constraintenforces the feature vectors of instances to be arranged according to their ranks, and 2) the metricconstraint makes the distance between instances reﬂect their rank difference.
Revised: For theconstruction, we formulate two geometric constraints in the embedding space: 1) the order constraintenforces the feature vectors of instances to be arranged according to their ranks, and 2) the metricconstraint makes the distance between instances reﬂect their rank difference.

Original: To satisfy these twoconstraints simultaneously, we introduce reference points that guide the region of each rank in theembedding space.
Revised: To satisfy these twoconstraints simultaneously, we introduce reference points that guide the region of each rank in theembedding space.

Original: Then, we use the simple k -NN rule in the embedding space to estimate the rank ofa test instance.
Revised: Then, we use the simple k -NN rule in the embedding space to estimate the rank ofa test instance.

Original: Extensive experiments show that GOL constructs high-quality embedding spaces andthus provides excellent rank estimation performances.
Revised: Extensive experiments show that GOL constructs high-quality embedding spaces andthus provides excellent rank estimation performances.

Original: The contributions of this paper can be summarized as follows.
Revised: The contributions of this paper can be summarized as follows.

Original: • To the best of our knowledge, GOL is the ﬁrst attempt to design an embedding space inwhich the direction and distance between objects represent their order and metric relations.
Revised: • To the best of our knowledge, GOL is the ﬁrst attempt to design an embedding space inwhich the direction and distance between objects represent their order and metric relations.•

Original: • We introduce a novel metric, called DRR, to assess the quality of embedding spaces for rankestimation.
Revised: We introduce a novel metric, called DRR, to assess the quality of embedding spaces for rankestimation.

Original: Then, it is shown that GOL effectively sorts and separates instances accordingto their ranks in an embedding space, as illustrated in Figure 1 (c).
Revised: Then, it is shown that GOL effectively sorts and separates instances accordingto their ranks in an embedding space, as illustrated in Figure 1 (c).

Original: • The proposed GOL algorithm provides state-of-the-art performances on various benchmark datasets for facial age estimation, HCI classiﬁcation, and aesthetic score regression.
Revised: • The proposed GOL algorithm provides state-of-the-art performances on various benchmark datasets for facial age estimation, HCI classiﬁcation, and aesthetic score regression.

Original: Speciﬁcally, GOL performs the best in 20 out of 25 benchmark tests.
Revised: Speciﬁcally, GOL performs the best in 20 out of 25 benchmark tests.

Original:  Related Work
Revised:  Related Work

Original: Ordinal regression: Many ordinal regression methods have been developed to estimate the rank ofan object directly using classiﬁers or regressors.
Revised: Ordinal regression: Many ordinal regression methods have been developed to estimate the rank ofan object directly using classiﬁers or regressors.

Original: For example, Rothe et al.
Revised: For example, Rothe et al.

Original: (2015) employed tens ofclassiﬁers to yield the average of their predictions as output.
Revised: (2015) employed tens ofclassiﬁers to yield the average of their predictions as output.

Original: Yi et al.
Revised: Yi et al.

Original: (2014) developed a regressorto estimate the rank of an image using multi-scale patches.
Revised: (2014) developed a regressorto estimate the rank of an image using multi-scale patches.

Original: Also, Frank & Hall (2001) employedmultiple binary classiﬁers, each of which tells whether the rank of an object is higher than a series ofthresholds or not.
Revised: Also, Frank & Hall (2001) employedmultiple binary classiﬁers, each of which tells whether the rank of an object is higher than a series ofthresholds or not.

Original: However, such direct estimation of ranks is challenging even for human beings ingeneral; e.g. , humans usually predict only a rough range of another one’s age with limited conﬁdence.
Revised: However, such direct estimation of ranks is challenging even for human beings in general; e.g. , humans usually predict only a rough range of another one’s age with limited conﬁdence.

Original: Thus, Diaz & Marathe (2019) trained a regressor using soft ordinal labels to alleviate penalties on close predictions.
Revised: Thus, Diaz & Marathe (2019) trained a regressor using soft ordinal labels to alleviate penalties onclose predictions.

Original: Furthermore, Li et al.
Revised: Furthermore, Li et al.

Original: (2021) and Li et al.
Revised: (2021) and Li et al.

Original: (2022) modeled the uncertainty of eachprediction as a Gaussian distribution.
Revised: (2022) modeled the uncertainty of eachprediction as a Gaussian distribution.

Original: In contrast to these methods, the proposed algorithm providesmore accurate rank estimates, although it uses only a single encoder network with the simple k -NNrule and makes no complicated probabilistic assumptions.
Revised: In contrast to these methods, the proposed algorithm providesmore accurate rank estimates, although it uses only a single encoder network with the simple k -NNrule and makes no complicated probabilistic assumptions.

Original: Order learning:  Lim et al.
Revised: Order learning: Rank-ordered (or ordinal) descriptors have been used in computer vision. Forexample, Toews & Wells (2009) proposed SIFT-rank descriptors for correspondence matching, whichwere also used to model anatomical development in MRI data (Toews et al., 2012). Recently, withthe great success of deep learning, Lim et al.

Original: ﬁrst proposed the notion of order learning, which learns orderingrelationships between objects and determines the rank of an unseen object.
Revised: ﬁrst proposed the notion of order learning,which learns ordering relationships between objects and determines the rank of an unseen object bycomparing it with references with known ranks.

Original: It trains a pairwisecomparator to tell whether one object is greater than, similar to, or smaller than the other object.

Original: Then, it predicts the rank of a test object by comparing it with references with known ranks.

Original: It yieldspromising results because relative assessment is easier than absolute assessment in general.
Revised: It yields promising results because relative assessmentis easier than absolute assessment in general.

Original: Lee & Kim (2021) improved the performance of order learning by ﬁnding more reliable references.
Revised: Lee & Kim (2021) improved the performance of orderlearning by ﬁnding more reliable references.

Original: Theydecomposed object information into an order-related feature and an identity feature and showedthat objects with similar identity features can be compared more reliably.
Revised: They decomposed object information into an orderrelated feature and an identity feature and showed that objects with similar identity features can becompared more reliably.

Original: Also, Shin et al.
Revised: Also, Shin et al.

Original: (2022)extended the classiﬁcation approach in (Lim et al., 2020; Lee & Kim, 2021) to regression-basedorder learning.
Revised: (2022) extended the classiﬁcation approach in (Lim et al. ,2020; Lee & Kim, 2021) to regression-based one.

Original: However, these order learning methods require a signiﬁcant computational cost toﬁnd reliable references from an entire training set.
Revised: These order learning methods, however, require asigniﬁcant computational cost to ﬁnd reliable references from an entire training set.

Original: Moreover, for rank estimation, they should docomparisons with many references with different ranks because they consider only relative prioritiesbetween objects.
Revised: Moreover, forrank estimation, they should do comparisons with many references with different ranks because theyconsider only relative priorities between objects.

Original: On the contrary, the proposed algorithm simply carries out the k -NN search to yieldoutstanding rank estimation performances.
Revised: On the contrary, the proposed algorithm simplycarries out the k -NN search to yield outstanding rank estimation results.

Original: Metric learning: Metric learning aims to construct an embedding space in which the distancebetween objects reﬂects their semantic difference.
Revised: Metric learning: Metric learning aims to construct an embedding space in which the distancebetween objects reﬂects their semantic difference.

Original: Most metric learning algorithms (Schroff et al.
Revised: Most metric learning algorithms (Schroff et al.

Original: ,2015; Deng et al., 2019) are for ordinary classiﬁcation, clustering, or image retrieval tasks.
Revised: ,2015; Deng et al., 2019) are for ordinary classiﬁcation, clustering, or image retrieval tasks.

Original: Therefore,they enforce an object to be located near other objects in the same class in the embedding space butfar from objects in different classes.
Revised: Therefore,they enforce an object to be located near other objects in the same class in the embedding space butfar from objects in different classes.

Original: However, in rank estimation, this approach may be suboptimalbecause it does not consider ordinal relationships among classes.
Revised: However, in rank estimation, this approach may be suboptimalbecause it does not consider ordinal relationships among classes.

Original: For example, in movie rating, itdoes not discriminate the class difference between ‘outstanding’ and ‘very good’ from that between‘outstanding’ and ‘poor.’
Revised: For example, in movie rating, itdoes not discriminate the class difference between ‘outstanding’ and ‘very good’ from that between‘outstanding’ and ‘poor.’

Original: To alleviate this problem, Xiao et al.
Revised: To alleviate this problem, Xiao et al.

Original: (2009) designed a metric, calledlabeled distance, to measure semantic similarities between objects and attempted to preserve localsemantic structures in the feature space.
Revised: (2009) designed a metric, calledlabeled distance, to measure semantic similarities between objects and attempted to preserve localsemantic structures in the feature space.

Original: Moreover, to preserve the ordinal relationships, Li et al.(2012) developed a metric learning algorithm to make the distances between objects proportionalto their rank differences.
Revised: Moreover, to preserve the ordinal relationships, Li et al.(2012) developed a metric learning algorithm to make the distances between objects proportionalto their rank differences.

Original: Also, Tian et al.
Revised: Also, Tian et al.

Original: (2016) employed a series of margins to explicitly imposedifferent embedding distances according to rank differences.
Revised: (2016) employed a series of margins to explicitly imposedifferent embedding distances according to rank differences.

Original: Suárez et al.
Revised: Suárez et al.

Original: (2021) attempted to makethe embedding distances between pairs of objects sorted according to their rank differences.
Revised: (2021) attempted to makethe embedding distances between pairs of objects sorted according to their rank differences.

Original:  Proposed Algorithm
Revised:  Proposed Algorithm

Original: 3.1 Preliminary – Order and Metric
Revised: 3.1 Preliminary – Order and Metric

Original: Mathematically, both order and metric are binary relations (Hrbacek & Jech, 1984).
Revised: Mathematically, both order and metric are binary relations (Hrbacek & Jech, 1984).

Original: An order(Schröder, 2003), denoted by ≤ , on a setΘ = { θ 0 , θ 1 , . . , θ M − 1 } should satisfy the properties of • Reﬂexivity: θ i ≤ θ i for all i , • Antisymmetry:
Revised: An order, denotedby ≤ , on a set should satisfy the properties of reﬂexivity, antisymmetry, and transitivity (Schröder,2003).

Original: θ i ≤ θ j and θ j ≤ θ i imply θ i = θ j , • Transitivity: θ i ≤ θ j and θ j ≤ θ k imply θ i ≤ θ k . On the other hand, a metric(Rudin, 1991) is a distance function d satisfying 
Revised: On the other hand, a metric is a distance function d satisfying nonnegativity, commutativity,and triangle inequality (Rudin, 1991).

Original: [-• Nonnegativity: d ( θ i , θ j ) ≥ 0 for all i, j , and d ( θ i , θ j ) = 
-]
Revised: [-• Nonnegativity: d ( θ i , θ j ) ≥ 0 for all i, j , and d ( θ i , θ j ) = 
-]

Original: if and only if θ i = θ j ,

Original: • Commutativity: d ( θ i , θ j ) =

Original: d ( θ j , θ i ) for all i, j ,

Original: • Triangle inequality: d ( θ i , θ k )

Original: ≤ d ( θ i , θ j ) + d ( θ j , θ k ) for all i, j, k .

Original: In rank estimation, an order describes the priorities of ranks or classes in the set Θ = { θ 0 , . . .
Revised: In rank estimation, an order describes the priorities of ranks or classes in the set Θ = { θ 0 , . . .

Original: , θ M − 1 } ,where each rank represents one or more object instances.
Revised: , θ M − 1 } ,where each rank represents one or more object instances.

Original: For example, in age estimation, θ imay represent i -year-olds, and θ 17 < θ 32 indicates that 17-year-olds are younger than 32-yearolds.
Revised: For example, in age estimation, θ imay represent i -year-olds, and θ 17 < θ 32 indicates that 17-year-olds are younger than 32-yearolds.

Original: Let θ ( · ) be the rank function, and let x and y be instances.
Revised: Let θ ( · ) be the rank function, and let x and y be instances.

Original: Then, θ ( x ) = θ 17 means thatperson x is 17-year-old.
Revised: Then, θ ( x ) = θ 17 means thatperson x is 17-year-old.

Original: Also, a metric describes the difference between ranks in Θ .
Revised: Also, a metric describes the difference between ranks in Θ .

Original: For example,d ( θ ( x ) , θ ( y ))
Revised: For example,d ( θ ( x ) , θ ( y ))

Original: = 15 means that two people x and y are 15 years apart.
Revised: = 15 means that two people x and y are 15 years apart.

Revised: 3.2 Embedding Space Construction

Revised: Given an object instance x , the objective is to estimate its rank θ ( x ) .

Revised: In such rank estimation, an orderand a metric convey complementary information: the order provides directional information betweenranks, while the metric does length (or magnitude of difference) information.

Revised: In age estimation, letus consider three age ranks θ 2 , θ 17 , and θ 32 .

Original: {+Since θ 2 < θ 17 < θ 32 , the order informs that θ 2 and θ 
+}
Revised: {+Since θ 2 < θ 17 < θ 32 , the order informs that θ 2 and θ 
+}

Original: Instances 𝑥 𝑦 ℎ ℎ
Revised: Instances 𝑥 𝑦 ℎ ℎ

Original: Encoder ℎ 𝑦 ℎ 𝑥
Revised: Encoder ℎ 𝑦 ℎ 𝑥

Original: Order constraint
Revised: Order constraint

Original: Metric constraint
Revised: Metric constraint

Original: AttractionRepulsion
Revised: AttractionRepulsion

Original: Reference points
Revised: Reference points

Original: Embedding space
Revised: Embedding space

Original: Instance embeddings
Revised: Instance embeddings

Original: Forward direction Backward direction
Revised: Forward direction Backward direction

Original: Direction vector
Revised: Direction vector

Original: Rank4 3 2ℎ 𝑦 𝑟 𝜃(𝑥)− 𝑟 𝜃(𝑦)+ 𝑟 𝜃(𝑥) 𝑟 𝜃(𝑦)
Revised: Rank4 3 2ℎ 𝑦 𝑟 𝜃(𝑥)− 𝑟 𝜃(𝑦)+ 𝑟 𝜃(𝑥) 𝑟 𝜃(𝑦)

Original: ℎ 𝑦 ℎ 𝑥
Revised: ℎ 𝑦 ℎ 𝑥

Original: 𝑣 f 𝑣 b 𝑟 𝜃(𝑥) 𝑟 𝜃(𝑥)− 𝑟 𝜃(𝑦)ℎ 𝑥
Revised: 𝑣 f 𝑣 b 𝑟 𝜃(𝑥) 𝑟 𝜃(𝑥)− 𝑟 𝜃(𝑦)

Original: 3.2 Embedding Space Construction

Original: [Figure]

Original: Given an object instance x , the objective is to estimate its rank θ ( x ) . In such rank estimation, an orderand a metric convey complementary information: the order provides directional information betweenranks, while the metric does length (or magnitude of difference) information. In age estimation, letus consider three age ranks θ 2 , θ 17 , and θ 32 . Since θ 2 < θ 17 < θ 32 , the order informs that θ 2 and θ 32are at the opposite sides with respect to θ 17 .
Revised: ℎ 𝑥 are at the opposite sides with respect to θ 17 .

Original: On the other hand, since d ( θ 2 , θ 17 ) =
Revised: On the other hand, since d ( θ 2 , θ 17 ) =

Original: d ( θ 32 , θ 17 ) = 15 ,the metric indicates how far θ 2 and θ 32 are from θ 17 .
Revised: d ( θ 32 , θ 17 ) = 15 ,the metric indicates how far θ 2 and θ 32 are from θ 17 .

Original: In this case, both lengths are identically 15.
Revised: In this case, both lengths are identically 15.

Original: For rank estimation, the pairwise comparison methods (Lim et al., 2020; Lee & Kim, 2021; Nguyenet al., 2018) attempt to learn these relations.
Revised: For rank estimation, the pairwise comparison methods (Lim et al., 2020; Lee & Kim, 2021; Nguyenet al., 2018) attempt to learn these relations.

Original: However, Lim et al.
Revised: However, Lim et al.

Original: and Lee & Kim (2021)exploit the order relation only, whereas Nguyen et al.
Revised: and Lee & Kim (2021)exploit the order relation only, whereas Nguyen et al.

Original: (2018) use the metric relation only to train theirneural networks.
Revised: (2018) use the metric relation only to train theirneural networks.

Original: Thus, the conventional methods may yield sub-optimal results.
Revised: Thus, the conventional methods may yield sub-optimal results.

Original: To learn both order and metric relations, we propose a geometric approach called GOL.
Revised: To learn both order and metric relations, we propose a geometric approach called GOL.

Original: It containstwo types of geometric constraints that enforce directional (order) and distance (metric) relationshipsbetween object instances according to their ranks in an embedding space.
Revised: It containstwo types of geometric constraints that enforce directional (order) and distance (metric) relationshipsbetween object instances according to their ranks in an embedding space.

Original: Speciﬁcally, the orderconstraint sorts instances directionally according to the ranks, while the metric constraint separatestwo instances farther if their rank difference is larger.
Revised: Speciﬁcally, the orderconstraint sorts instances directionally according to the ranks, while the metric constraint separatestwo instances farther if their rank difference is larger.

Original: Figure 2 is an overview of GOL.
Revised: Figure 2 is an overview of GOL.

Original: Order constraint: Suppose that there are M ranks in a training set X .
Revised: Order constraint: Suppose that there are M ranks in a training set X .

Original: Without loss of generality,the ranks are assumed to be consecutive integers in Θ = { 0 , 1 , . . .
Revised: Without loss of generality,the ranks are assumed to be consecutive integers in Θ = { 0 , 1 , . . .

Original: , M − 1 } .
Revised: , M − 1 } .

Original: In Figure 2, an encoderh maps each instance x ∈ X into a feature vector
Revised: In Figure 2, an encoderh maps each instance x ∈ X into a feature vector

Original: h x = h
Revised: h x = h

Original: ( x ) in an embedding space.
Revised: ( x ) in an embedding space.

Original: As h , weadopt VGG16 (Simonyan & Zisserman, 2015) without fully connected layers.
Revised: As h , weadopt VGG16 (Simonyan & Zisserman, 2015) without fully connected layers.

Original: The output of the lastpooling layer is normalized so that h tx h x = 1 .
Revised: The output of the lastpooling layer is normalized so that h tx h x = 1 .

Original: Thus, the embedding space is a unit hypersphere.
Revised: Thus, the embedding space is a unit hypersphere.

Original: The order constraint encourages instances to be sorted according to their ordering relationships.

Original: Asin Lim et al.
Revised: As in Lim et al.

Original: and Lee & Kim (2021), we classify the ordering between two instances x and yin X into three categories:
Revised: and Lee & Kim (2021), we classify the ordering between two instances x andy in X into three categories:

Original: [Equation]
Revised: [Equation]

Original: where τ is a threshold.
Revised: where τ is a threshold.

Original: For instance ordering, notations ‘ ≺ , ≈ , (cid:31) ’ are used instead of ‘ <, = , > .’
Revised: For instance ordering, notations ‘ ≺ , ≈ , (cid:31) ’ are used instead of ‘ <, = , > .’

Revised: The order constraint encourages instances to be sorted according to their ordering relationships.

Original: For two instances x and y with ordering x ≺ y , the vector from h x to h y should be aligned with thedirection of the rank increment in the embedding space.
Revised: In other words, for two instances x and y with ordering x ≺ y , the vector from h x to h y shouldbe aligned with the direction of the rank increment in the embedding space.

Original: To model such rank directions, we introduce M reference points, r 0 , r 1 , . .
Revised: To model such rankdirections, we introduce M reference points, r 0 , r 1 , . .

Original: , r M − 1 , which are learnable parameters and guide the positions of the M ranks in the embedding space.
Revised: , r M − 1 , which are learnable parametersguiding the positions of the M ranks in the embedding space.

Revised: These reference points are randomlyinitialized by the Glorot normal method (Glorot & Bengio, 2010) and jointly optimized with encoderparameters during training.

Original: Let us deﬁne the direction vector v ( r, s ) from point r to point s onthe unit hypersphere as
Revised: Let us deﬁne the direction vector v ( r, s ) from point r to point s on the unit hypersphere as

Original: [Equation]
Revised: [Equation]

Original: Then, v ( r i , r j ) is called the rank direction from rank i to rank j .
Revised: Then, v ( r i , r j ) is called the rank direction from rank i to rank j .

Original: Also, the rank direction v ( r i , r j ) isforward if i < j , and backward if i > j .
Revised: Also, the rank direction v ( r i , r j ) isforward if i < j , and backward if i > j .

Original: Note that forward directions may differ from one another,for they may represent different physical changes.
Revised: Note that forward directions may differ from one another,for they may represent different physical changes.

Original: For example, in age estimation, visual variationsfrom 0-years-old to 5-years-old are mainly due to craniofacial development, whereas those from45-years-old to 50-years-old are due to skin aging (Geng et al., 2007).
Revised: For example, in age estimation, visual variationsfrom 0-years-old to 5-years-old are mainly due to craniofacial development, whereas those from45-years-old to 50-years-old are due to skin aging (Geng et al., 2007).

Original: If x ≺ y , we determine the forward and backward rank directions, respectively, by
Revised: If x ≺ y , we determine the forward and backward rank directions, respectively, by

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Then, the encoder is trained so that the embedded features h x and h y satisfy the order constraint:
Revised: Then, the encoder is trained so that the embedded features h x and h y satisfy the order constraint:

Original: [Equation]
Revised: [Equation]

Original: In other words, the direction vector v ( h x , h y ) should be aligned more with the forward direction v fthan with the backward direction v b .
Revised: In other words, the direction vector v ( h x , h y ) should be aligned more with the forward direction v fthan with the backward direction v b .

Original: To enforce the order constraint in (5), we compute the softmax probability p xy
Revised: To enforce the order constraint in (5), we compute the softmax probability p xy

Original: =
Revised: =

Original: [ p xy f , p xy b ] t , wherep xy f = e v t f v ( h x ,h y ) /
Revised: [ p xy f , p xy b ] t , wherep xy f = e v t f v ( h x ,h y ) /

Original: ( e v t
Revised: ( e v t

Original: f
Revised: f

Original: v ( h x ,h y )
Revised: v ( h x ,h y )

Original: + e v t b v ( h x ,h y ) ) and p xy b = 1 − p xy f .
Revised: + e v t b v ( h x ,h y ) ) and p xy b = 1 − p xy f .

Original: We then deﬁne the order loss
Revised: We then deﬁne the order loss

Original: L order as the cross entropy between p xy and q xy =
Revised: L order as the cross entropy between p xy and q xy =

Original: [ q xy f , q xy b ] t = [1 , 0] t , given by
Revised: [ q xy f , q xy b ] t = [1 , 0] t , given by

Original: [Equation]
Revised: [Equation]

Original: The order loss for case x (cid:31) y is formulated similarly in a symmetric manner.
Revised: The order loss for case x (cid:31) y is formulated similarly in a symmetric manner.

Original: Metric constraint: Next, we formulate a metric constraint to make the distance between instancesin the embedding space reﬂect their rank difference.
Revised: Metric constraint: Next, we formulate a metric constraint to make the distance between instancesin the embedding space reﬂect their rank difference.

Original: Speciﬁcally, it is desirable that
Revised: Speciﬁcally, it is desirable that

Original: [Equation]
Revised: [Equation]

Original: where d e is the Euclidean distance in the embedding space, and γ is a margin.
Revised: where d e is the Euclidean distance in the embedding space, and γ is a margin.

Original: Note that if | θ ( x ) −θ ( y ) | > τ , either x ≺ y or x (cid:31) y in (1).
Revised: Note that if | θ ( x ) −θ ( y ) | > τ , either x ≺ y or x (cid:31) y in (1).

Original: Hence, the metric constraint in (7) is equivalent to
Revised: Hence, the metric constraint in (7) is equivalent to

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: From the triangle inequality, we have | d e ( r i , h x ) − d e ( r i , h y ) | ≤ d e ( h x , h y ) for every referencepoint r i .
Revised: From the triangle inequality, we have | d e ( r i , h x ) − d e ( r i , h y ) | ≤ d e ( h x , h y ) for every referencepoint r i .

Original: So, if x ≈ y , we also have | d e ( r i , h x ) − d e ( r i , h y ) | ≤ γ .
Revised: So, if x ≈ y , we also have | d e ( r i , h x ) − d e ( r i , h y ) | ≤ γ .

Original: To encourage this inequality, wedeﬁne a loss L x ≈ y as
Revised: To encourage this inequality, wedeﬁne a loss L x ≈ y as

Original: [Equation]
Revised: [Equation]

Original: On the contrary, if x ≺ y , it should be that d e ( h x , h y ) > γ .
Revised: On the contrary, if x ≺ y , it should be that d e ( h x , h y ) > γ .

Original: Thus, we deﬁne another loss
Revised: Thus, we deﬁne another loss

Revised: L x ≺

Original: [Equation]
Revised: [Equation]

Original: To minimize the ﬁrst sum, d e ( r i , h x ) should be reduced, while d e ( r i , h y ) should be increased.
Revised: To minimize the ﬁrst sum, d e ( r i , h x ) should be reduced, while d e ( r i , h y ) should be increased.

Original: Thus,reference points r i , 0 ≤ i ≤ θ ( x ) , are trained to attract h x and repel h y , as illustrated in Figure 2.
Revised: Thus,reference points r i , 0 ≤ i ≤ θ ( x ) , are trained to attract h x and repel h y , as illustrated in Figure 2.

Original: The second sum in (10) is similar.
Revised: The second sum in (10) is similar.

Original: We do not use reference points r l , θ ( x ) <
Revised: We do not use reference points r l , θ ( x ) <

Original: l < θ ( y ) , which tendto be between h x and h y and unhelpful for guiding them.
Revised: l < θ ( y ) , which tendto be between h x and h y and unhelpful for guiding them.

Original: The derivation of L x ≺ y in (10) from themetric constraint in (7) is provided in Appendix B. Also, L x (cid:31) y is formulated symmetrically.
Revised: The derivation of L x ≺ y in (10) from themetric constraint in (7) is provided in Appendix B. Also, L x (cid:31) y is formulated symmetrically.

Original: Then, we deﬁne the metric loss L metric as
Revised: Then, we deﬁne the metric loss L metric as

Original: [Equation]
Revised: [Equation]

Original: where [ · ] is the indicator function.
Revised: where [ · ] is the indicator function.

Original: Note that, in (9) or (10), we make h x and h y attract or repeleach other indirectly through reference points r i .
Revised: Note that, in (9) or (10), we make h x and h y attract or repeleach other indirectly through reference points r i .

Original: This is because the direct attraction or repulsionusing d e ( h x , h y ) may cause h x and h y to move in arbitrary directions.
Revised: This is because the direct attraction or repulsionusing d e ( h x , h y ) may cause h x and h y to move in arbitrary directions.

Original: Because the reference pointsare trained also with the order loss in (6), they provide proper directional guidance of attraction orrepulsion in (9) or (10).
Revised: Because the reference pointsare trained also with the order loss in (6), they provide proper directional guidance of attraction orrepulsion in (9) or (10).

Original: Experimental analysis on L x ≺ y in (10) is in Appendix D.6.
Revised: Experimental analysis on L x ≺ y in (10) is in Appendix D.6.

Original: Loss function:
Revised: Loss function:

Original: In addition to the losses for the order and metric constraints, we employ the centerloss (Nguyen et al., 2018), which aims at locating each reference point r
Revised: In addition to the losses for the order and metric constraints, we employ the centerloss (Nguyen et al., 2018), which aims at locating each reference point r

Original: i at the center of all instanceswith rank i .
Revised: i at the center of all instanceswith rank i .

Original: It is given by

Original: [Equation]
Revised: [Equation]

Original: Finally, we use the overall loss function to optimize the encoder parameters and the reference pointsr i , which is given by
Revised: Finally, we use the overall loss function to optimize the encoder parameters and the reference pointsr i , which is given by

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Equation]

Original: It is worth pointing out that reference points play essential roles in all three loss terms in (13).
Revised: It is worth pointing out that reference points play essential roles in all three loss terms in (13).

Original: Theyare used to deﬁne forward and backward rank directions in L order , so they help to sort instancesdirectionally according to the ranks.
Revised: Theyare used to deﬁne forward and backward rank directions in L order , so they help to sort instancesdirectionally according to the ranks.

Original: The reference points themselves are also sorted, since instancesare clustered around the reference points because of L center .
Revised: The reference points themselves are also sorted since instancesare clustered around the reference points because of L center .

Original: Also, as mentioned before, the referencepoints provide proper directional guidance of attraction or repulsion in L metric to satisfy the metricconstraint.
Revised: Also, as mentioned before, the referencepoints provide proper directional guidance of attraction or repulsion in L metric to satisfy the metricconstraint.

Original: In other words, GOL uses the reference points to satisfy both order and metric constraintssimultaneously and thus construct a well-arranged, well-clustered embedding space. 
Revised: In other words, GOL uses the reference points to satisfy both order and metric constraintssimultaneously and thus constructs a well-arranged, well-clustered embedding space. [Table] [Figure]

Original: 3.3 k -NN Rank Estimation
Revised: 3.3 k -NN Rank Estimation

Original: For rank estimation, we use the simple k -NN rule.
Revised: For rank estimation, we use the simple k -NN rule.

Original: Given a test instance x , in the embedding space,we ﬁnd a set N of its k NNs among all training instances in X .
Revised: Given a test instance x , in the embedding space,we ﬁnd a set N of its k NNs among all training instances in X .

Original: Then, the rank of x is estimated by
Revised: Then, the rank of x is estimated by

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original:  Experimental Results
Revised:  Experimental Results

Original: We conduct experiments on embedding space construction and rank estimation.
Revised: We conduct experiments on embedding space construction and rank estimation.

Original: Implementationdetails and more results are available in Appendices C and D, respectively.
Revised: Implementationdetails and more results are available in Appendices C and D, respectively.

Original: 4.1 Implementation
Revised: 4.1 Implementation

Original: We initialize an encoder h with VGG16 pre-trained on ILSVRC2012 (Deng et al., 2009) and referencepoints with the Glorot normal method (Glorot & Bengio, 2010).
Revised: We initialize an encoder h with VGG16 pre-trained on ILSVRC2012 (Deng et al., 2009) and referencepoints with the Glorot normal method (Glorot & Bengio, 2010).

Original: The Adam optimizer (Kingma &
Revised: The Adam optimizer (Kingma &

Original: Ba, 2015) is used with a batch size of 32 and a weight decay of 5 × 10 − 4 , and the initial learningrates for the encoder and the reference points are set to 10 − 4 and 10 − 3 , respectively.
Revised: Ba, 2015) is used with a batch size of 32 and a weight decay of 5 × 10 − 4 , and the initial learningrates for the encoder and the reference points are set to 10 − 4 and 10 − 3 , respectively.

Original: We perform thescheduled learning according to cosine annealing cycles (Huang et al., 2017).
Revised: We perform thescheduled learning according to cosine annealing cycles (Huang et al., 2017).

Original: For data augmentation,we do random horizontal ﬂips only.
Revised: For data augmentation,we do random horizontal ﬂips only.

Original: 4.2 Embedding Spaces
Revised: 4.2 Embedding Spaces

Original: The proposed GOL algorithm attempts to design an embedding space in which both order and metricconstraints are satisﬁed: instances should be well sorted according to their ranks and well separatedif they have big rank differences.
Revised: The proposed GOL algorithm attempts to design an embedding space in which both order and metricconstraints are satisﬁed: instances should be well sorted according to their ranks and well separatedif they have big rank differences.

Original: To measure the quality of an embedding space for rank estimation, we may adopt the between-classvariance to within-class variance (B2W) criterion in Fisher’s linear discriminant (Duda et al., 2006),B2W
Revised: To measure the quality of an embedding space for rank estimation, we may adopt the between-classvariance to within-class variance (B2W) criterion in Fisher’s linear discriminant (Duda et al., 2006),

Original: [Equation]
Revised: [Equation]

Original: (cid:0) [Equation] [Table] where X i = { x ∈ X | θ ( x ) = i } 
Revised: (15) where X i = { x ∈ X | θ ( x ) = i } , c i = (cid:80) x ∈X i h x / |X

Original: [Table]

Original: c i = (cid:80) x ∈X i h x / |X i | is the centroid for X i 
Revised: i | is the centroid for X i , and c = (cid:80)

Original: [Table] and c = (cid:80) x ∈X h x / |X|is the centroid for all instances in X .
Revised: x ∈X h x / |X|is the centroid for all instances in X .

Original: A high B2W score indicates that the rank sets X i , 0 ≤ i ≤ M − 1 ,are well separated from one another, while instances in each rank set are compactly distributed.
Revised: A high B2W score indicates that the rank sets X i , 0 ≤ i ≤ M − 1 ,are well separated from one another, while instances in each rank set are compactly distributed.

Original: However, B2W does not consider the ordinal relationships of ranks in Θ, since it was formulated forordinary classiﬁcation.
Revised: However, B2W does not consider the ordinal relationships of ranks in Θ since it was formulated forordinary classiﬁcation.

Original: Therefore, to assess embedding spaces for rank estimation, we propose thediscriminative ratio for ranking (DRR), given by
Revised: Therefore, to assess embedding spaces for rank estimation, we propose the [Table] discriminative ratio for ranking (DRR), given by

Original: [Equation]
Revised: [Equation]

Original: which is the ratio of the average pairwise centroid distance to the average pairwise instance distancein each rank set.
Revised: which is the ratio of the average pairwise centroid distance to the average pairwise instance distancein each rank set.

Original: Note that, in the numerator, ordinal weights ( j − i ) β are used to emphasize thedifference between a pair of rank sets with a large rank difference.
Revised: Note that, in the numerator, ordinal weights ( j − i ) β are used to emphasize thedifference between a pair of rank sets with a large rank difference.

Original: Here, β is a nonnegative parameterfor controlling the level of emphasis.
Revised: Here, β is a nonnegative parameterfor controlling the level of emphasis.

Original: A high DRR score is obtained when instances are well sortedand well separated according to their ranks in the embedding space.
Revised: A high DRR score is obtained when instances are well sortedand well separated according to their ranks in the embedding space.

Original: Table 1 compares the B2W and DRR scores on the MORPH II (Ricanek & Tesafaye, 2006),
Revised: Table 1 compares the B2W and DRR scores on the MORPH II (Ricanek & Tesafaye, 2006),

Original: CACD (Chen et al., 2015), and Adience (Levi & Hassner, 2015) train data.
Revised: CACD (Chen et al., 2015), and Adience (Levi & Hassner, 2015) train data.

Revised: To compare the embeddingspaces as fairly as possible, we use the same encoder backbone of VGG16 for all algorithms.

Original: GOL signiﬁcantlyoutperforms all conventional algorithms in all tests.
Revised: GOLsigniﬁcantly outperforms all conventional algorithms in all tests.

Original: In GOL, instances within each rank set X i arelocated compactly around reference point r i by L center in (12), while those in different rank sets arewell separated by L metric in (11).
Revised: In GOL, instances within each rankset X i are located compactly around reference point r i by L center in (12), while those in different ranksets are well separated by L metric in (11).

Original: Hence, GOL yields outstanding B2W scores.
Revised: Hence, GOL yields outstanding B2W scores.

Original: Moreover, GOLexcels with a larger score gap in DRR 1 . 0 than in DRR 0 . 5 , which means that it arranges the rank setseffectively in the embedding space to reﬂect their ordinal relationships.
Revised: Moreover, GOL excels with a larger score gap in DRR 1 . 0 than in DRR 0 . 5 , which means that it arranges the ranksets effectively in the embedding space to reﬂect their ordinal relationships using L order in (6).

Original: Figure 3 visualizes the embedding spaces on Adience.
Revised: Figure 3 visualizes the embedding spaces on Adience.

Original: As in Figure 1, we add a fully connected layerwith 3 output neurons to each encoder for the visualization.
Revised: As in Figure 1, we add a fully connected layerwith 3 output neurons to each encoder for the visualization.

Original: In (a) and (b), instances are sorted butscattered over the spaces, as the order learning algorithms (Lim et al., 2020; Shin et al., 2022) ignorethe metric relation of ranks.
Revised: In (a) and (b), instances are sorted butscattered over the spaces, as the order learning algorithms (Lim et al., 2020; Shin et al., 2022) ignorethe metric relation of ranks.

Original: In (c), instances in each rank set are well clustered, but the rank sets arenot sorted because ML (Schroff et al., 2015) neglects the order relation.
Revised: In (c), instances in each rank set are well clustered, but the rank sets arenot sorted because ML (Schroff et al., 2015) neglects the order relation.

Original: In (d), MV (Pan et al., 2018),which is an ordinal regressor, exhibits large within-class scattering, as well as large between-classscattering.
Revised: In (d), MV (Pan et al., 2018),which is an ordinal regressor, exhibits large within-class scattering, as well as large between-classscattering.

Original: In contrast, in (e), GOL constructs a well-sorted, well-clustered embedding space.
Revised: In contrast, in (e), GOL constructs a well-sorted, well-clustered embedding space.

Original: 4.3 Rank Estimation
Revised: 4.3 Rank Estimation

Original: Since GOL constructs high-quality embedding spaces, it provides excellent rank estimation performances even with the simple k -NN rule.
Revised: Since GOL constructs high-quality embedding spaces, it provides excellent rank estimation performances even with the simple k -NN rule.

Original: Facial age estimation: We use four datasets of MORPH II (Ricanek & Tesafaye, 2006), CACD(Chen et al., 2015), UTK (Zhang et al., 2017), and Adience (Levi & Hassner, 2015), as detailed in
Revised: Facial age estimation: We use four datasets of MORPH II (Ricanek & Tesafaye, 2006), CACD(Chen et al., 2015), UTK (Zhang et al., 2017), and Adience (Levi & Hassner, 2015), as detailed in

Original: Appendix D.3.
Revised: Appendix D.3.

Original: In Table 2, we compare the performances in the four evaluation settings of MORPH II, which isone of the most popular datasets in age estimation.
Revised: In Table 2, we compare the performances in the four evaluation settings of MORPH II, which is oneof the most popular datasets in age estimation.

Original: We use the mean absolute error (MAE) andcumulative score (CS) metrics.
Revised: We use the mean absolute error (MAE) and cumulativescore (CS) metrics.

Original: MAE is the average absolute error between estimated and ground-truthages ( i.e. ranks), and CS computes the percentage of images whose absolute errors are less than orequal to a tolerance level l = 5 .
Revised: MAE is the average absolute error between estimated and ground-truth ages ( i.e.ranks), and CS computes the percentage of images whose absolute errors are less than or equal toa tolerance level l = 5 .

Revised: Note that all algorithms in Table 2 use VGG16 as the encoder backbones,except for C3AE employing a shallow CNN.

Original: As for Shin et al.
Revised: As for Shin et al.

Original: (2022), the scores of the global ρ -regressor are

Original: [Table] [Table] [Table] compared, since their local scheme employs as many as six independent VGG16 encoders recursively.
Revised: (2022), the scores of the globalρ -regressor are compared since their local scheme employs as many as six independent VGG16encoders recursively.

Original: GOL performs the best in 5 out of 8 tests, including setting C, which is the most challenging task.
Revised: GOL performs the best in 5 out of 8 tests, including setting C, which is themost challenging task.

Original: Furthermore, unlike most conventional algorithms, GOL does not exploit the IMDB-WIKI pretrainingto boost the performances.
Revised: Furthermore, unlike most conventional algorithms, GOL does not exploit the [Table] [Table] [Table]

Revised: IMDB-WIKI pretraining to boost performances.

Original: Even without the pretraining, i.e. by employing only MORPH II trainingdata, GOL yields outstanding results in Table 2.
Revised: Even without the pretraining, i.e. by employing only MORPH II training data , GOL yields outstanding results in Table 2.

Original: Table 3 compares the results on CACD, which is a bigger dataset containing over 100,000 naturalface shots in diverse environments.
Revised: Table 3 compares the results on CACD, which is a bigger dataset containing over 100,000 naturalface shots in diverse environments.

Original: GOL outperforms the second-best methods with meaningful gapsof 0.12 and 0.17 in the train and validation settings, respectively, which indicates that it can copewith large and diverse data effectively as well.
Revised: GOL outperforms the second-best methods with meaningful gapsof 0.12 and 0.17 in the train and validation settings, respectively, which indicates that it can copewith large and diverse data effectively as well.

Original: Table 3 also compares the results on UTK.
Revised: Table 3 also compares the results on UTK.

Original: Note that Gustafsson et al.
Revised: Note that Gustafsson et al.

Original: (2020) and Berg et al.
Revised: (2020) and Berg et al.

Original: (2021)employ the deep ResNet50 network (He et al., 2016) as their encoders, and MWR-G (Shin et al.
Revised: (2021)employ the deep ResNet50 network (He et al., 2016) as their encoders, and MWR-G (Shin et al.

Original: ,2022) predicts ranks using a complicated, recursive regressor.
Revised: ,2022) predicts ranks using a complicated, recursive regressor.

Original: In contrast, GOL uses the shallower
Revised: In contrast, GOL uses the shallower

Original: VGG16 encoder and performs rank estimation based on the simple k -NN search.
Revised: VGG16 encoder and performs rank estimation based on the simple k -NN search.

Original: Nevertheless, ityields an excellent result, for its geometric constraints help to construct an effective embedding space.
Revised: Nevertheless, ityields an excellent result, for its geometric constraints help to construct an effective embedding space.

Original: Table 4 shows the results on Adience, where each image is labeled as one of 8 age groups.
Revised: Table 4 shows the results on Adience, where each image is labeled as one of 8 age groups.

Original: Comparedwith the second-best MWR-G, GOL improves the accuracy by 0.3% and reduces MAE by 0.03.
Revised: Comparedwith the second-best MWR-G, GOL improves the accuracy by 0.3% and reduces MAE by 0.03.

Original: HCI classiﬁcation: The HCI dataset (Palermo et al., 2012) is used for estimating the shootingdecade of a photograph.
Revised: HCI classiﬁcation: The HCI dataset (Palermo et al., 2012) is used for estimating the shootingdecade of a photograph.

Original: It contains 1,325 images from ﬁve decades 1930s ∼ 1970s.
Revised: It contains 1,325 images from ﬁve decades 1930s ∼ 1970s.

Original: Table 4 liststhe results on HCI.
Revised: Table 4 liststhe results on HCI.

Original: GOL yields the best performances as well, by improving the accuracy by 1.5%and reducing MAE by 0.05 as compared with the second-best methods.
Revised: GOL yields the best performances as well, by improving the accuracy by 1.5%and reducing MAE by 0.05 as compared with the second-best methods.

Original: This means that GOL yieldsreliable results even for a small dataset, which may be unfavorable for the k -NN estimation.
Revised: This means that GOL yieldsreliable results even for a small dataset, which may be unfavorable for the k -NN estimation.

Original: Aesthetic score regression: The aesthetics dataset (Schifanella et al., 2015) contains 15,687 imagesin four categories.
Revised: Aesthetic score regression: The aesthetics dataset (Schifanella et al., 2015) contains 15,687 imagesin four categories.

Original: Each image is annotated with a 5-scale aesthetic score.
Revised: Each image is annotated with a 5-scale aesthetic score.

Original: In Table 5, we follow theexperimental setting in (Liu et al., 2018).
Revised: In Table 5, we follow theexperimental setting in (Liu et al., 2018).

Original: It is challenging to estimate aesthetic scores reliably due tothe subjectivity and ambiguity of aesthetic criteria, but GOL performs the best in 8 out of 10 tests.
Revised: It is challenging to estimate aesthetic scores reliably due tothe subjectivity and ambiguity of aesthetic criteria, but GOL performs the best in 8 out of 10 tests.

Original: Compared with the state-of-the-art POE (Li et al., 2021), GOL improves the accuracy by 0.3% andreduces MAE by 0.1 overall.
Revised: Compared with the state-of-the-art POE (Li et al., 2021), GOL improves the accuracy by 0.3% andreduces MAE by 0.1 overall.

Original: 4.4 Analysis
Revised: 4.4 Analysis

Original: Ablation study: Table 6 compares ablated methods for the loss function in (13).
Revised: Ablation study: Table 6 compares ablated methods for the loss function in (13).

Original: Method Iemploys L center only.
Revised: Method Iemploys L center only.

Original: In II and III, L metric and L order are excluded, respectively.
Revised: In II and III, L metric and L order are excluded, respectively.

Original: Compared with IV(GOL), method I degrades the performances severely, since the center loss alone cannot constructa meaningful embedding space, as discussed in Appendix D.6.
Revised: Compared with IV(GOL), method I degrades the performances severely, since the center loss alone cannot constructa meaningful embedding space, as discussed in Appendix D.6.

Original: Also, from II and IV, we see thatthe metric constraint signiﬁcantly improves the performances by separating instances with different [Figure] [Table]
Revised: Also, from II and IV, we see thatthe metric constraint signiﬁcantly improves the performances by separating instances with differentranks from each other.

Original: [Table] ranks from each other. From III and IV, we see that the order constraint also improves the results byarranging instances directionally according to  ranks.
Revised: From III and IV, we see that the order constraint also improves the resultsby arranging instances directionally according to their ranks.

Original: To summarize, both order and metric constraintsimprove the results and are complementary to each other.
Revised: To summarize, both order and metricconstraints improve the results and are complementary to each other.

Original:  Comparison with order learning: The conventional order learning algorithms (Lim et al., 2020;
Revised: [Figure] [Table] [Table] Comparison with order learning: The conventional order learning algorithms (Lim et al., 2020;

Original: Lee & Kim, 2021; Shin et al., 2022) provide competitive ranking performances, but they requirerelatively heavy computations.
Revised: Lee & Kim, 2021; Shin et al., 2022) provide competitive ranking performances, but they requirerelatively heavy computations.

Original: Table 7 lists the testing times.
Revised: Table 7 lists the testing times.

Original: To estimate the rank of a test instance,
Revised: To estimate the rank of a test instance,

Original: OL(Lim et al., 2020) uses ﬁve references for each of the M ranks; e.g. , 300 comparisons shouldbe made if M = 60 as in typical age estimation.
Revised: OL uses ﬁve references for each of the M ranks; e.g. , 300 comparisons should be made if M = 60 asin typical age estimation.

Revised: Moreover, OL and DRC-ORID need additional processes, such as MAPestimation, to estimate a rank based on ordering relationships.

Original: Similarly, MWR-G (Shin et al., 2022) compares atest instance with multiple reference pairs recursively.
Revised: Also, MWR-G estimates the rank bycomparing a test instance with multiple reference pairs recursively.

Original: In contrast, GOL performs the simple k -NNestimation and thus is about 160 and 23 times faster than OL and MWR-G, respectively.
Revised: In contrast, GOL simply performsthe k -NN estimation, which can be done efﬁciently in a parallel manner, and thus is about 160 and 23times faster than OL and MWR-G, respectively.

Original: Furthermore, the order learning algorithms should select references among a training set X througha complicated optimization process; e.g. , OL compares all possible pairs of training instances with
Revised: Furthermore, the order learning algorithms should select references among a training set X througha complicated optimization process; e.g. , OL compares all possible pairs of training instances with

Original: O ( |X| 2 ) complexity to select the most reliable references, and DRC-ORID (Lee & Kim, 2021)performs joint network training and clustering for reference selection.
Revised: O ( |X| 2 ) complexity to select the most reliable references, and DRC-ORID (Lee & Kim, 2021)performs joint network training and clustering for reference selection.

Original: On the contrary, GOL requiresno such process.
Revised: On the contrary, GOL requiresno such process.

Original: Also, in Table 7, the proposed algorithm demands the fewest parameters because itdirectly estimates the rank of an instance in the embedding space through the k -NN search, whereasthe order learning algorithms should adopt comparators or regressors tailored for each task.
Revised: Also, in Table 7, the proposed algorithm demands the fewest parameters because itdirectly estimates the rank of an instance in the embedding space through the k -NN search, whereasthe order learning algorithms should adopt comparators or regressors tailored for each task.

Original: Embedding space transition: Figure 4 visualizes the transition of an embedding space for Adience.
Revised: Embedding space transition: Figure 4 visualizes the transition of an embedding space for Adience.

Original: As the GOL training step goes on, we see that instances are gradually sorted and separated accordingto their ranks to satisfy the geometric constraints.
Revised: As the GOL training step goes on, we see that instances are gradually sorted and separated accordingto their ranks to satisfy the geometric constraints.

Original: After the convergence, for each rank, the referencepoint r i and the centroid c i are close to each other, indicating that instances are clustered around theirreference points.
Revised: After the convergence, for each rank, the referencepoint r i and the centroid c i are close to each other, indicating that instances are clustered around theirreference points.

Original: Moreover, those reference points are well sorted as well.
Revised: Moreover, those reference points are well sorted as well.

Original:  Conclusions
Revised:  Conclusions

Original: The GOL algorithm for rank estimation was proposed in this work.
Revised: The GOL algorithm for rank estimation was proposed in this work.

Original: First, we construct an embeddingspace based on the two geometric constraints, which enforce the direction and distance betweeninstances to represent the order and metric relations between their ranks.
Revised: First, we construct an embeddingspace based on the two geometric constraints, which enforce the direction and distance betweeninstances to represent the order and metric relations between their ranks.

Original: Then, we perform thesimple k -NN search in the embedding space for rank estimation.
Revised: Then, we perform thesimple k -NN search in the embedding space for rank estimation.

Original: Extensive experiments on variousrank estimation tasks demonstrated that GOL constructs high-quality embedding spaces and yieldsexcellent rank estimation performances.
Revised: Extensive experiments on variousrank estimation tasks demonstrated that GOL constructs high-quality embedding spaces and yieldsexcellent rank estimation results.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See failure cases in Appendix D.6.
Revised: [Yes] See failure cases in Appendix D.6.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] SeeAppendix A.
Revised: [Yes] SeeAppendix A.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] See Section 4.
Revised: [Yes] See Section 4.

Original: and Appendix C.
Revised: and Appendix C.

Original: (b) Did you specify all the training details (e
Revised: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how they were chosen)?
Revised: g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] See Appendix D.
Revised: [Yes] See Appendix D.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi- ments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [No] We will release the source codes.
Revised: [No] We will release the source codes.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Appendix C.
Revised: [Yes] See Appendix C.

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See Appendix D.3.
Revised: [Yes] See Appendix D.3.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [Yes] See Appendix D.
Revised: [Yes] See Appendix D.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Broader Impacts
Revised: A Broader Impacts

Original: Recently, ethical concerns about the fairness of deep-learning-based systems have been raised(Castelvecchi, 2020; Roussi, 2020; Noorden, 2020).
Revised: Recently, ethical concerns about the fairness of deep-learning-based systems have been raised(Castelvecchi, 2020; Roussi, 2020; Noorden, 2020).

Original: Especially, due to the intrinsic imbalance offacial datasets (Ricanek & Tesafaye, 2006; Zhang et al., 2017; Niu et al., 2016), most deep learningmethods on facial analysis (Wen et al., 2020; Or-El et al., 2020) have unwanted gender or racialbias.
Revised: Especially, due to the intrinsic imbalance offacial datasets (Ricanek & Tesafaye, 2006; Zhang et al., 2017; Niu et al., 2016), most deep learningmethods on facial analysis (Wen et al., 2020; Or-El et al., 2020) have unwanted gender or racialbias.

Original: The proposed algorithm is not free from this bias either when trained on such datasets.
Revised: The proposed algorithm is not free from this bias either when trained on such datasets.

Original: Hence,the bias should be resolved before any practical usage.
Revised: Hence,the bias should be resolved before any practical usage.

Original: Also, even though the proposed algorithmdiscovers some subclasses by ranking instances, these results should never be misinterpreted in sucha way as to encourage any kind of discrimination.
Revised: Also, even though the proposed algorithmdiscovers some subclasses by ranking instances, these results should never be misinterpreted in sucha way as to encourage any kind of discrimination.

Original: We recommend using the proposed algorithm forresearch only.
Revised: We recommend using the proposed algorithm forresearch only.

Original: B Derivation of L x ≺ y in (10)
Revised: B Derivation of L x ≺ y in (10)

Original: From the metric constraint in (7), if x ≺ y , it should be that d e ( h x , h y ) > γ , or equivalently
Revised: From the metric constraint in (7), if x ≺ y , it should be that d e ( h x , h y ) > γ , or equivalently

Original: [Equation]
Revised: [Equation]

Original: Note that reference points and instances are roughly sorted according to their corresponding ranks inthe embedding space by the order constraint in (5).
Revised: Note that reference points and instances are roughly sorted according to their corresponding ranks inthe embedding space by the order constraint in (5).

Original: Therefore, for x ≺ y , h x tends to be closer toeach reference point r
Revised: Therefore, for x ≺ y , h x tends to be closer toeach reference point r

Original: i , 0 ≤ i ≤ θ ( x ) , than h y is.
Revised: i , 0 ≤ i ≤ θ ( x ) , than h y is.

Original: In other words, x ≺ y , it is likely that
Revised: In other words, x ≺ y , it is likely that

Original: [Equation]
Revised: [Equation]

Original: However, to satisfy the constraint in (17), we impose a stricter upper bound on the differenced e ( r i , h x ) − d e ( r i , h y ) as follows.
Revised: However, to satisfy the constraint in (17), we impose a stricter upper bound on the differenced e ( r i , h x ) − d e ( r i , h y ) as follows.

Original: From the triangle inequality, we have
Revised: From the triangle inequality, we have

Original: [Equation]
Revised: [Equation]

Original: (19) which is equivalent to
Revised: (19) which is equivalent to

Original: [Equation]
Revised: [Equation]

Original: Therefore, if
Revised: Therefore, if

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: the constraint in (17) is satisﬁed.
Revised: the constraint in (17) is satisﬁed.

Original: This desirable condition in (21) is formulated into the ﬁrst sum in
Revised: This desirable condition in (21) is formulated into the ﬁrst sum in

Original: L x ≺ y in (10).
Revised: L x ≺ y in (10).

Original: Similarly, for each reference point r j , θ ( y ) ≤ j ≤ M − 1 , it is likely that
Revised: Similarly, for each reference point r j , θ ( y ) ≤ [Equation] ≤ M − 1 , it is likely that

Original: [Equation]
Revised: [Equation]

Original: However, to satisfy the metric constraint, it is desirable that
Revised: However, to satisfy the metric constraint, it is desirable that

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: which is formulated into the second sum in L x ≺ y in (10).
Revised: which is formulated into the second sum in L x ≺ y in (10).

Original: C Implementation Details
Revised: C Implementation Details

Original: C.1 Network Architecture
Revised: C.1 Network Architecture

Original: The structure of the encoder h is detailed in Table 8, where ‘ k h × k w - s - c Conv’ denotes the 2Dconvolution with kernel size k h × k w , stride s
Revised: The structure of the encoder h is detailed in Table 8, where ‘ k h × k w - s - c Conv’ denotes the 2Dconvolution with kernel size k h × k w , stride s

Original: [Table]
Revised: [Table]

Original: and c output channels.
Revised: and c output channels.

Original: Similarly, ‘ k h × k w - s MaxPool’and ‘ k h × k w - s AvgPool’ represent the 2D max pooling and 2D average pooling with a k h × k w kernelat stride s , respectively.
Revised: Similarly, ‘ k h × k w - s MaxPool’and ‘ k h × k w - s AvgPool’ represent the 2D max pooling and 2D average pooling with a k h × k w kernelat stride s , respectively.

Original: Also, BN means batch normalization (Ioffe & Szegedy, 2015).
Revised: Also, BN means batch normalization (Ioffe & Szegedy, 2015).

Original: The encoderis based on the VGG16 network and takes a 224 × 224 × 3 image as input
Revised: The encoderis based on the VGG16 network and takes a 224 × 224 × 3 image as input.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: C.2 Embedding Space Visualization
Revised: .2 Embedding Space Visualization

Original: In Figure 1, ResNet18 is employed as an encoder, but the output dimension of its last fully connectedlayer is reduced to 3 for the visualization.
Revised: In Figure 1, ResNet18 is employed as an encoder, but the output dimension of its last fully connectedlayer is reduced to 3 for the visualization.

Original: The encoder takes a 28 × 28 × 3 image as input.
Revised: The encoder takes a 28 × 28 × 3 image as input.

Original: We usethe same training setting in Section 4.1.
Revised: We usethe same training setting in Section 4.1.

Original: Similarly, for the other 3D embedding space visualizations, including Figures 3 and 4, we modify theencoder in Table 8 to append a fully connected layer with output dimension 3.
Revised: Similarly, for the other 3D embedding space visualizations, including Figures 3 and 4, we modify theencoder in Table 8 to append a fully connected layer with output dimension 3.

Original: We train the modiﬁedencoder using the same setting in Section 4.1.
Revised: We train the modiﬁedencoder using the same setting in Section 4.1.

Original: C.3 Computing Environment
Revised: C.3 Computing Environment

Original: We conduct all experiments using PyTorch (Paszke et al., 2019) and an NVIDIA GeForce RTX [-
-]
Revised: We do all experiments using PyTorch (Paszke et al., 2019) and an NVIDIA GeForce RTX [-
-]3090 GPU.

Original: GPU.
Revised: We train the encoder until the loss converges. Table 9 lists the training epochs and time for eachdataset. [Table]

Original: D More Experimental Results
Revised: D More Experimental ResultsD.1 Hyper-Parameters

Original: D.1 Hyper-parameters

Original: The proposed GOL algorithm has three hyper-parameters; τ in (1), γ in (7), and k in (14).
Revised: The proposed GOL algorithm has three hyper-parameters; τ in (1), γ in (7), and k in (14).

Original: Unlessspeciﬁed otherwise, we set τ = 0 and γ = 0 .
Revised: Unlessspeciﬁed otherwise, we set τ = 0 and γ = 0 .

Original: Also, we ﬁx k = 50 and k = 16 for facialage estimation tasks and the others, respectively.
Revised: Also, we ﬁx k = 50 and k = 16 for facialage estimation tasks and the others, respectively.

Original: Let us describe how the hyper-parameters affectperformances.
Revised: Let us describe how the hyper-parameters affectperformances.

Original: Hyper-parameter γ : Table 9 compares the MAE scores at different γ ’s on the MORPH II, CACD,and UTK datasets.
Revised: Hyper-parameter γ : Table 10 compares the MAE scores at different γ ’s on the MORPH II, CACD,and UTK datasets.

Original: In this test, τ = 0 and k = 50 .
Revised: In this test, τ = 0 and k = 50 .

Original: Note that γ is a margin to impose the minimumdistance between instances with a rank difference larger than τ in the embedding space.
Revised: Note that γ is a margin to impose the minimumdistance between instances with a rank difference larger than τ in the embedding space.

Original: We see thatγ = 0 .
Revised: We see thatγ = 0 .

Original: 05 performs well in general, so we set γ = 0 .
Revised: 05 performs well in general, so we set γ = 0 .

Original: as the default option.
Revised: as the default option.

Original: Only for UTK, we setγ = 0 .
Revised: Only for UTK, we setγ = 0 .

Original: 25 , even though GOL outperforms the state-of-the-art MWR-G (4.49) at the other γ levels.
Revised: 25 , even though GOL outperforms the state-of-the-art MWR-G (4.49) at the other γ levels aswell.

Original: [Table]
Revised: [Table]

Original: Hyper-parameter τ : Table 10 compares the MAE scores at different τ ’s onthe MORPH II, CACD,and UTK datasets.
Revised: Hyper-parameter τ : Table 11 compares the MAE scores at different τ ’s on MORPH II, CACD,and UTK.

Original: In this test, γ = 0 . 05 and k = 50 .
Revised: In this test, γ = 0 . 05 and k = 50 .

Original: We see that τ = 0 provides decent results ingeneral.
Revised: We see that τ = 0 provides decent results in general.

Original: We hence set the default τ = 0 .
Revised: We hence set the default τ = 0 .

Original: [Table]
Revised: [Table]

Original: parameter k : Table 11 compares the MAE scores according to k .
Revised: -parameter k : Table 12 compares the MAE scores according to k .

Original: In each age estimationtask, there are more than 40 ranks, but there are no clear distinctive characteristics for each rank.
Revised: In each age estimationtask, there are more than 40 ranks, but there are no clear distinctive characteristics for each rank.

Original: Therefore, we set a relatively large k = 50 as the default option.
Revised: Therefore, we set a relatively large k = 50 as the default option.

Original: However, note that the results arenot very sensitive as long as k ≥ 18 .
Revised: However, note that the results arenot very sensitive as long as k ≥ 18 .

Original: [Table]
Revised: [Table]

Original: Next, Table 12 compares the MAE scores on different folds of the HCI dataset according to k .
Revised: Next, Table 13 compares the MAE scores on different folds of the HCI dataset according to k .

Original: The
Revised: The

Original: HCI and aesthetics datasets contain only ﬁve ranks, respectively.
Revised: HCI and aesthetics datasets contain only ﬁve ranks, respectively.

Original: Thus, we use a relatively smallk = 16 for these tasks.
Revised: Thus, we use a relatively smallk = 16 for these tasks.

Original: D.2 Embedding Spaces
Revised: D.2 Embedding Spaces

Original: Table 13 compares the B2W and DRR scores on the test data of MORPH II, CACD, and Adience.
Revised: Table 14 compares the B2W and DRR scores on the test data of MORPH II, CACD, and Adience.

Original: GOL performs the best in all tests, as well as for the training data in Table 1.
Revised: GOL performs the best in all tests, as well as for the training data in Table 1.

Original: These results indicate that
Revised: These results indicate that

Original: GOL can arrange the rank sets effectively in the embedding space to reﬂect their ordinal relationshipseven for unseen test instances.
Revised: GOL can arrange the rank sets effectively in the embedding space to reﬂect their ordinal relationshipseven for unseen test instances.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: D.3 Rank Estimation – Facial Age Estimation
Revised: D.3 Rank Estimation – Facial Age Estimation

Original: Datasets: We provide facial age estimation performances of GOL on four datasets: MORPH II(Ricanek & Tesafaye, 2006), CACD (Chen et al., 2015), UTK (Zhang et al., 2017), and Adience (Levi& Hassner, 2015).
Revised: Datasets: We provide facial age estimation results of GOL on four datasets: MORPH II (Ricanek & Tesafaye, 2006), CACD (Chen et al., 2015), UTK (Zhang et al., 2017), and Adience (Levi & Hassner,2015).

Original: MORPH II has the Institutional Review Board approval.
Revised: MORPH II has the Institutional Review Board approval.

Original: The other datasets weremade for academic research purposes only.
Revised: The other datasets were made foracademic research purposes only.

Original: Any images getting deletion requests from the originalowners will be discarded from the datasets.
Revised: Any images getting deletion requests from the original ownerswill be discarded from the datasets.

Original: There are no name labels, except for CACD containingcelebrity names.
Revised: There are no name labels, except for CACD containing celebritynames.

Original: We exploit the datasets only for the performance assessment of GOL.
Revised: We exploit the datasets only for the performance assessment of GOL.

Original: MORPH II (Ricanek & Tesafaye, 2006) is a popular dataset for facial age estimation, containingabout 55,000 facial images of 13,617 subjects in the age range [16, 77].
Revised: MORPH II (Ricanek & Tesafaye, 2006) is a popular dataset for facial age estimation, containingabout 55,000 facial images of 13,617 subjects in the age range [16, 77].

Original: In each image, the genderand race labels are annotated.
Revised: In each image, the genderand race labels are annotated.

Original: Based on these labels, various evaluation protocols have been proposed.
Revised: Based on these labels, various evaluation protocols have been proposed.

Original: We employ the four evaluation settings A, B, C, and D (Lim et al., 2020; Lee & Kim, 2021; Shinet al., 2022).
Revised: We employ the four evaluation settings A, B, C, and D (Lim et al., 2020; Lee & Kim, 2021; Shinet al., 2022).

Original: • Setting A: 5,492 Caucasian images are randomly sampled and divided into training and
Revised: • Setting A: 5,492 Caucasian images are randomly sampled and divided into training and

Original: testing sets with a ratio of 8:2.
Revised: testing sets with a ratio of 8:2.

Original: • Setting B: About 21K images of Caucasians and Africans are randomly chosen so that the
Revised: • Setting B: About 21K images of Caucasians and Africans are randomly chosen so that the

Original: ratio between Caucasians and Africans is 1:1 and that between females and males is 1:3.
Revised: ratio between Caucasians and Africans is 1:1 and that between females and males is 1:3.

Original: Then, it is divided into three subsets (S1, S2, S3).
Revised: Then, it is divided into three subsets (S1, S2, S3).

Original: The training and testing are repeated
Revised: The training and testing are repeated

Original: twice — 1) training on S1, testing on S2+S3, and 2) training on S2, testing on S1+S3.
Revised: twice — 1) training on S1, testing on S2+S3, and 2) training on S2, testing on S1+S3.

Original: • Setting C:
Revised: • Setting C:

Original: The entire dataset is randomly split into ﬁve folds, subject to the constraint that
Revised: The entire dataset is randomly split into ﬁve folds, subject to the constraint that

Original: the same person should belong to only one fold, and the 5-fold cross-validation is performed.
Revised: the same person should belong to only one fold, and the 5-fold cross-validation is performed.

Original: • Setting D: The whole dataset is randomly divided into ﬁve folds without any constraint, andthe 5-fold cross-validation is performed.
Revised: • Setting D: The whole dataset is randomly divided into ﬁve folds without any constraint, andthe 5-fold cross-validation is performed.

Original: CACD (Chen et al., 2015) provides 160k images from 2,000 celebrities.
Revised: CACD (Chen et al., 2015) provides 160k images from 2,000 celebrities.

Original: It is split into three subsetsby celebrities: 1,800 for training, 80 for validation, and 120 for testing.
Revised: It is split into three subsetsby celebrities: 1,800 for training, 80 for validation, and 120 for testing.

Original: We provide two results bytraining on the train set and on the validation set, respectively, as done in (Rothe et al., 2018; Shenet al., 2018; Shin et al., 2022).
Revised: We provide two results bytraining on the train set and on the validation set, respectively, as done in (Rothe et al., 2018; Shenet al., 2018; Shin et al., 2022).

Original: The age range is [14, 62].
Revised: The age range is [14, 62].

Original: UTK (Zhang et al., 2017) consists of 20,000 facial images in a wide age range [0, 116].
Revised: UTK (Zhang et al., 2017) consists of 20,000 facial images in a wide age range [0, 116].

Original: We adopt theevaluation protocol in (Gustafsson et al., 2020; Berg et al., 2021).
Revised: We adopt theevaluation protocol in (Gustafsson et al., 2020; Berg et al., 2021).

Original: Adience (Levi & Hassner, 2015) is used for age group estimation.
Revised: Adience (Levi & Hassner, 2015) is used for age group estimation.

Original: There are 26,580 facial imagesfrom 2,284 subjects, which are grouped into 8 ordinal classes: 0-2, 4-6, 8-13, 15-20, 25-32, 38-43, 4853, and over 60-year-olds.
Revised: There are 26,580 facial imagesfrom 2,284 subjects, which are grouped into 8 ordinal classes: 0-2, 4-6, 8-13, 15-20, 25-32, 38-43, 4853, and over 60-year-olds.

Original: We employ the 5-fold subject-exclusive (SE) cross-validation evaluationsetting in (Liu et al., 2018, 2019;
Revised: We employ the 5-fold subject-exclusive (SE) cross-validation evaluationsetting in (Liu et al., 2018, 2019;

Original: Diaz & Marathe, 2019; Li et al., 2021).
Revised: Diaz & Marathe, 2019; Li et al., 2021).

Original: More comparison: Table 14 provides comparison results with more conventional algorithms on
Revised: More comparison: Table 15 provides comparison results with more conventional algorithms on

Original: MORPH II.
Revised: MORPH II.

Original: It is an extended version of Table 2.
Revised: It is an extended version of Table 2.

Original: [Table]
Revised: [Table]

Original: D.4 Rank Estimation – HCI Classiﬁcation
Revised: D.4 Rank Estimation – HCI Classiﬁcation

Original: Table 15 is an extended version of the HCI results in Table
Revised: Table 16 is an extended version of the HCI results in Table

Original: [Table] [Table]
Revised: [Table] [Table]

Original: D.5 Rank Estimation – Aesthetic Score Regression
Revised: D.5 Rank Estimation – Aesthetic Score Regression

Original: The aesthetics dataset (Schifanella et al., 2015) provides 15,687 image URLs on Flickr, where 13,929images are available but the others are lost.
Revised: The aesthetics dataset (Schifanella et al., 2015) provides 15,687 image URLs on Flickr, where 13,929images are available but the others are lost.

Original: Each image is annotated with a 5-scale aesthetic score.
Revised: Each image is annotated with a 5-scale aesthetic score.

Original: We adopt the 5-fold cross-validation.
Revised: We adopt the 5-fold cross-validation.

Original: For training, we set the learning rate of all parameters, includingreference points, to 10 − 6 .
Revised: For training, we set the learning rate of all parameters, includingreference points, to 10 − 6 .

Original: The other settings are the same as in Section 4.1.
Revised: The other settings are the same as in Section 4.1.

Original: Table 16 is an extendedversion of Table 5.
Revised: Table 17 is an extendedversion of Table 5.

Original: [Table]
Revised: [Table]

Original: D.6 Analysis
Revised: D.6 Analysis

Original: [Table]

Original: Ablation study: Figure 5 shows the embedding spaces for the ablated methods in Table 6.
Revised: Ablation study: Figure 5 shows the embedding spaces for the ablated methods in Table 6.

Original: The toprow visualizes the reduced 3D embedding spaces, as described in Appendix C.2, while the bottomrow does the original 512D embedding spaces via t-SNE (Maaten & Hinton, 2008).
Revised: The toprow visualizes the reduced 3D embedding spaces, as described in Appendix C.2, while the bottomrow does the original 512D embedding spaces via t-SNE (Maaten & Hinton, 2008).

Original: In (a), only
Revised: In (a), only

Original: [Table]

Original: L center is used, so the 3D embedding space almost collapses to a single point.
Revised: L center is used, so the 3D embedding space almost collapses to a single point.

Original: It is because a trivialsolution to minimize L center can be obtained by merging all instances and all reference points into asingle point.
Revised: It is because a trivialsolution to minimize L center can be obtained by merging all instances and all reference points into asingle point.

Original: In (b), instances are directionally aligned, but adjacent rank sets overlap one another.
Revised: In (b), instances are directionally aligned, but adjacent rank sets overlap one another.

Original: In (c) and (d), the 3D embedding spaces seem similar.
Revised: In (c) and (d), the 3D embedding spaces seem similar.

Original: However, GOL yields more clearly orderedinstances in the t-SNE visualization of the original embedding space than ‘ L center + L metric ’ does.
Revised: However, GOL yields more clearly orderedinstances in the t-SNE visualization of the original embedding space than ‘ L center + L metric ’ does.

Original: [Figure]
Revised: [Figure]

Original: More visualizations: Figure 6 visualizes the embedding spaces of the proposed algorithm on the
Revised: More visualizations: Figure 6 visualizes the embedding spaces of the proposed algorithm on the

Original: HCI and aesthetics datasets.
Revised: HCI and aesthetics datasets.

Original: For each dataset, the right subﬁgure is the t-SNE visualization (Maaten& Hinton, 2008).
Revised: For each dataset, the right subﬁgure is the t-SNE visualization (Maaten& Hinton, 2008).

Original: [Figure]
Revised: [Figure]

Original: Young, median, and old instances: Figure 7 shows sampled instances from each rank set X i of the
Revised: Young, median, and old instances: Figure 7 shows sampled instances from each rank set X i of the

Original: Adience dataset.
Revised: Adience dataset.

Original: Speciﬁcally, Figures 7 (a) and (c) list the instances in each X i that are the farthestfrom the reference point
Revised: Speciﬁcally, Figures 7 (a) and (c) list the instances in each X i that are the farthestfrom the reference point

Original: r i in the backward and forward directions, respectively.
Revised: r i in the backward and forward directions, respectively.

Original: Also, Figures 7 (b)shows the closest instance to r i among all instances in X i .
Revised: Also, Figures 7 (b)shows the closest instance to r i among all instances in X i .

Original: In Adience, each rank set
Revised: In Adience, each rank set

Original: X i consists ofinstances in an age range, marked at the top of the ﬁgure.
Revised: X i consists ofinstances in an age range, marked at the top of the ﬁgure.

Original: Thus, from each rank set, young, median,and old instances tend to be selected in (a), (b), and (c), respectively, which means that the instancesare well sorted in the embedding space although their exact ages are not annotated.
Revised: Thus, from each rank set, young, median,and old instances tend to be selected in (a), (b), and (c), respectively, which means that the instancesare well sorted in the embedding space although their exact ages are not annotated.

Original: [Figure]
Revised: [Figure]

Original: Alternatives to L x ≺ y : Table 17 compares the rank estimation results of alternative loss terms for
Revised: Alternatives to L x ≺ y : Table 18 compares the rank estimation results of alternative loss terms for

Original: L x ≺ y in (10).
Revised: L x ≺ y in (10).

Original: Method I directly increases d e ( h x , h y ) to make it larger than γ .
Revised: Method I directly increases d e ( h x , h y ) to make it larger than γ .

Original: Method II uses onlytwo reference points r θ ( x ) and r θ ( y ) .
Revised: Method II uses onlytwo reference points r θ ( x ) and r θ ( y ) .

Original: In other words, unlike method III ( i.e. GOL), it does not employreference points r i , 0 ≤ i < θ ( x ) , and θ ( y ) <
Revised: In other words, unlike method III ( i.e. GOL), it does not employreference points r i , 0 ≤ i < θ ( x ) , and θ ( y ) <

Original: r j ≤ M − 1 .
Revised: r j ≤ M − 1 .

Original: For each method, L x (cid:31) y is also modiﬁedaccordingly.
Revised: For each method, L x (cid:31) y is also modiﬁedaccordingly.

Original: Method I degrades the performances signiﬁcantly, as h x and h y move in arbitrary directions.
Revised: Method I degrades the performances signiﬁcantly, as h x and h y move in arbitrary directions.

Original: Also,
Revised: Also,

Original: GOL performs better than II because attraction and repulsion with many reference points facilitatethe positioning of instances in the embedding space, as well as the reference points themselves.
Revised: GOL performs better than II because attraction and repulsion with many reference points facilitatethe positioning of instances in the embedding space, as well as the reference points themselves [Table]

Revised: [Table]

Revised: Testing time: Table 19 lists the testing time of GOL according to the number of samples for thek -NN search on the MORPH II and CACD datasets.

Revised: Note that the distances to all samples can becomputed efﬁciently in a parallel manner.

Revised: Therefore, the proposed algorithm performs fast even with44,000 samples.

Original: [Table]
Revised: [Table]

Original: Success and failure cases: Figure 8{+
+} shows some success and failure cases of GOL on the CACDdataset.
Revised: Limitations: Figure {+
+} shows some success and failure cases of GOL on the CACD dataset. In

Original: In (a), the ages of test instances are predicted accurately with absolute errors less than 4.
Revised: Figure 8 (a), the ages of test instances are predicted accurately with absolute errors less than 4.

Original: In(b), the failure cases are presented, in which various factors such as poor illumination, overexposure,and low-quality photographs make it hard to predict ages precisely.
Revised: However, in Figure 8 (b), GOL yields poor estimates on some hard cases, in which various factors,such as poor illumination, overexposure, and low-quality photographs, hinder accurate estimation.

Revised: Also, since GOL predicts a rank based on the k -NN search, it suffers when there are insufﬁcienttraining instances.

Revised: For example, GOL yields relatively poor results on MORPH II setting B, whichconsists of 7,000 training samples and 14,000 test samples.

Revised: Similarly, GOL tends to yield lessaccurate estimates on minority classes, such as toddlers and elders, since most age estimation datasetscontain fewer instances in such classes.

Revised: These limitations of GOL might be alleviated by generatingpseudo-references in the embedding space, which we leave as future work in this paper.

Original: Test
Revised: Test

Original: Nearest neighbors 22 (22) 23   22  32 (32) 33   26  51 (54) 50   51  38 (39) 38   40  39 (40) 35   40  ( [Figure]
Revised: Nearest neighbors 22 (22) 23   22  32 (32) 33   26  51 (54) 50   51  38 (39) 38   40  39 (40) 35   40  (a) 26 (57) 30   23  28 (55) 30   28  45 (16) 46   46   40   32 33 35 (60) 48   44 45 45 (16)

Original: Test
Revised: Test

Original: Nearest neighbors (

Original: [Figure]
Revised: [Figure]