Original: Geometric Order Learning for Rank Estimation
Revised: Geometric Order Learning for Rank Estimation

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: A novel approach to rank estimation, called geometric order learning (GOL), isproposed in this paper.
Revised: A novel approach to rank estimation, called geometric order learning (GOL), is proposed in this paper.

Original: First, we construct an embedding space, in which the direction and distance between objects represent order and metric relations betweentheir ranks, by enforcing two geometric constraints: the order constraint compelsobjects to be sorted according to their ranks, while the metric constraint makesthe distance between objects reﬂect their rank difference.
Revised: First, we construct an embedding space, in which the direction and distance between objects represent order and metric relations between their ranks, by enforcing two geometric constraints: the order constraint compels objects to be sorted according to their ranks, while the metric constraint makes the distance between objects reﬂect their rank difference.

Original: Then, we perform thesimple k nearest neighbor ( k -NN) search in the embedding space to estimate therank of a test object.
Revised: Then, we perform the simple k nearest neighbor ( k -NN) search in the embedding space to estimate the rank of a test object.

Original: Moreover, to assess the quality of embedding spaces forrank estimation, we propose a metric called discriminative ratio for ranking (DRR).
Revised: Moreover, to assess the quality of embedding spaces for rank estimation, we propose a metric called discriminative ratio for ranking (DRR).

Original: Extensive experiments on facial age estimation, historical color image (HCI) classiﬁcation and aesthetic score regression demonstrate that GOL constructs effectiveembedding spaces and thus yields excellent rank estimation performances.
Revised: Extensive experiments on facial age estimation, historical color image (HCI) classiﬁcation, and aesthetic score regression demonstrate that GOL constructs effective embedding spaces and thus yields excellent rank estimation performances.

Original:  Introduction
Revised:  Introduction

Original: In rank estimation, we estimate the rank (or ordered class) of an object.
Revised: In rank estimation, we estimate the rank (or ordered class) of an object.

Original: It is different from ordinaryclassiﬁcation in that its classes are arranged in a natural order.
Revised: It is different from ordinary classiﬁcation in that its classes are arranged in a natural order.

Original: For example, in movie rating, classescan be ordered from ‘outstanding’ to ‘very good,’ ‘satisfactory,’ ‘unsatisfactory,’ and ‘poor.’
Revised: For example, in movie rating, classes can be ordered from ‘outstanding’ to ‘very good,’ ‘satisfactory,’ ‘unsatisfactory,’ and ‘poor.’

Original: Rankestimation is a fundamental problem and, e.g. , used for various computer vision tasks including facialage estimation (Shin et al., 2022), aesthetic quality assessment (Schifanella et al., 2015), and HCIclassiﬁcation (Palermo et al., 2012).
Revised: Rank estimation is a fundamental problem and, e.g. , used for various computer vision tasks including facial age estimation (Shin et al., 2022), aesthetic quality assessment (Schifanella et al., 2015) and HCI classiﬁcation (Palermo et al., 2012).

Original: For rank estimation, many techniques (Li et al., 2021; Liu et al., 2018) adopt the ordinal regressionframework, which employs a classiﬁer or a regressor to predict the rank of an object directly.
Revised: For rank estimation, many techniques (Li et al., 2021; Liu et al., 2018) adopt the ordinal regression framework, which employs a classiﬁer or a regressor to predict the rank of an object directly.

Original: However,they may fail to yield reliable estimates, for there is no clear distinction between ranks in many cases.
Revised: However, they may fail to yield reliable estimates, for there is no clear distinction between ranks in many cases.

Original: For instance, in facial age estimation, the aging process — causing variations in facial shapes, sizes,and texture — has large individual differences due to factors such as genes, diet, and lifestyle.
Revised: For instance, in facial age estimation, the aging process — causing variations in facial shapes, sizes, and texture — has large individual differences due to factors such as genes, diet, and lifestyle.

Original: To address this issue, comparison-based algorithms (Lim et al., 2020; Lee & Kim, 2021; Li et al. ,2014; Nguyen et al., 2018) have been proposed.
Revised: To address this issue, comparison-based algorithms (Lim et al., 2020; Lee & Kim, 2021; Li et al., 2014; Nguyen et al., 2018) have been proposed.

Original: Instead of predicting the rank directly, they learn abinary relation between objects, such as order or metric (Hrbacek & Jech, 1984).
Revised: Instead of predicting the rank directly, they learn a binary relation between objects, such as order or metric (Hrbacek & Jech, 1984).

Original: These relationsprovide useful information for rank estimation: an order indicates the relative priority between objectsx and y , while a metric informs of the distance between them.
Revised: These relations provide useful information for rank estimation: an order indicates the relative priority between objects x and y , while a metric informs of the distance between them.

Original: In order learning (Lim et al., 2020; Lee & Kim, 2021), a comparator is learned to classify therelationship between x and y into one of three cases: x is ‘greater than,’ ‘similar to,’ or ‘smaller than’y .
Revised: In order learning (Lim et al., 2020; Lee & Kim, 2021), a comparator is learned to classify the relationship between x and y into one of three cases: x is ‘greater than,’ ‘similar to,’ or ‘smaller than’ y .

Original: Then, the rank of a test object is estimated by comparing it with multiple reference objects withknown ranks.
Revised: Then, the rank of a test object is estimated by comparing it with multiple reference objects with known ranks.

Original: This approach is based on the idea that it is easier to predict ordering relationshipsbetween objects than to estimate the absolute ranks; telling the older one between two people is easierthan estimating their exact ages.
Revised: This approach is based on the idea that it is easier to predict ordering relationships between objects than to estimate the absolute ranks; telling the older one between two people is easier than estimating their exact ages.

Original: However, order learning disregards how much x is different from y .
Revised: However, order learning disregards how much x is different from y .

Original: In other words, it ignores metric information.
Revised: In other words, it ignores metric information.

Original: [Figure]
Revised: [Figure]

Original: Figures 1 (a) and (b) compare embedding spaces, obtained by order learning and metric learning,respectively.
Revised: Figures 1 (a) and (b) compare embedding spaces, obtained by order learning and metric learning, respectively.

Original: Order learning sorts instances according to ranks in general, but instances in each classare scattered in the embedding space.
Revised: Order learning sorts instances according to ranks in general, but instances in each class are scattered in the embedding space.

Original: In contrast, metric learning reduces within-class scattering butdoes not sort instances properly.
Revised: In contrast, metric learning reduces within-class scattering but does not sort instances properly.

Original: For example, both blue and orange ranks are adjacent to the yellowone, although they should be arranged in the order of blue, orange, and yellow.
Revised: For example, both blue and orange ranks are adjacent to the yellow one, although they should be arranged in the order of blue, orange, and yellow.

Original: Unlike ordinaryclassiﬁcation, different errors have different severities in rank estimation: misclassifying an object inthe blue rank as a yellow one is severer than mistaking it for an orange one.
Revised: Unlike ordinary classiﬁcation, different errors have different severities in rank estimation: misclassifying an object in the blue rank as a yellow one is severer than mistaking it for an orange one.

Original: Thus, ordering in theembedding space is important to avoid such severe errors.
Revised: Thus, ordering in the embedding space is important to avoid such severe errors.

Original: We propose a novel algorithm, GOL, to estimate the rank of an object reliably by exploiting bothorder and metric relations.
Revised: We propose a novel algorithm, GOL, to estimate the rank of an object reliably by exploiting both order and metric relations.

Original: To this end, we construct an embedding space, in which the directionand distance between objects represent the order and metric relations between their ranks.
Revised: To this end, we construct an embedding space, in which the direction and distance between objects represent the order and metric relations between their ranks.

Original: For theconstruction, we formulate two geometric constraints in the embedding space: 1) the order constraintenforces the feature vectors of instances to be arranged according to their ranks, and 2) the metricconstraint makes the distance between instances reﬂect their rank difference.
Revised: For the construction, we formulate two geometric constraints in the embedding space: 1) the order constraint enforces the feature vectors of instances to be arranged according to their ranks, and 2) the metric constraint makes the distance between instances reﬂect their rank difference.

Original: To satisfy these twoconstraints simultaneously, we introduce reference points that guide the region of each rank in theembedding space.
Revised: To satisfy these two constraints simultaneously, we introduce reference points that guide the region of each rank in the embedding space.

Original: Then, we use the simple k -NN rule in the embedding space to estimate the rank ofa test instance.
Revised: Then, we use the simple k -NN rule in the embedding space to estimate the rank of a test instance.

Original: Extensive experiments show that GOL constructs high-quality embedding spaces andthus provides excellent rank estimation performances.
Revised: Extensive experiments show that GOL constructs high-quality embedding spaces and thus provides excellent rank estimation performances.

Original: The contributions of this paper can be summarized as follows.
Revised: The contributions of this paper can be summarized as follows.

Original: • To the best of our knowledge, GOL is the ﬁrst attempt to design an embedding space inwhich the direction and distance between objects represent their order and metric relations.•
Revised: • To the best of our knowledge, GOL is the ﬁrst attempt to design an embedding space in which the direction and distance between objects represent their order and metric relations.

Original:  We introduce a novel metric, called DRR, to assess the quality of embedding spaces for rankestimation.
Revised: • We introduce a novel metric, called DRR, to assess the quality of embedding spaces for rank estimation.

Original: Then, it is shown that GOL effectively sorts and separates instances accordingto their ranks in an embedding space, as illustrated in Figure 1 (c).
Revised: Then, it is shown that GOL effectively sorts and separates instances according to their ranks in an embedding space, as illustrated in Figure 1 (c).

Original: • The proposed GOL algorithm provides state-of-the-art performances on various benchmark datasets for facial age estimation, HCI classiﬁcation, and aesthetic score regression.
Revised: • The proposed GOL algorithm provides state-of-the-art performances on various benchmark datasets for facial age estimation, HCI classiﬁcation, and aesthetic score regression.

Original: Speciﬁcally, GOL performs the best in 20 out of 25 benchmark tests.
Revised: Speciﬁcally, GOL performs the best in 20 out of 25 benchmark tests.

Original:  Related Work
Revised:  Related Work

Original: Ordinal regression: Many ordinal regression methods have been developed to estimate the rank ofan object directly using classiﬁers or regressors.
Revised: Ordinal regression: Many ordinal regression methods have been developed to estimate the rank of an object directly using classiﬁers or regressors.

Original: For example, Rothe et al.
Revised: For example, Rothe et al.

Original: (2015) employed tens ofclassiﬁers to yield the average of their predictions as output.
Revised: (2015) employed tens of classiﬁers to yield the average of their predictions as output.

Original: Yi et al.
Revised: Yi et al.

Original: (2014) developed a regressorto estimate the rank of an image using multi-scale patches.
Revised: (2014) developed a regressor to estimate the rank of an image using multi-scale patches.

Original: Also, Frank & Hall (2001) employedmultiple binary classiﬁers, each of which tells whether the rank of an object is higher than a series ofthresholds or not.
Revised: Also, Frank & Hall (2001) employed multiple binary classiﬁers, each of which tells whether the rank of an object is higher than a series of thresholds or not.

Original: However, such direct estimation of ranks is challenging even for human beings in general; e.g. , humans usually predict only a rough range of another one’s age with limited conﬁdence.
Revised: However, such direct estimation of ranks is challenging even for human beings in general; e.g. , humans usually predict only a rough range of another one’s age with limited conﬁdence.

Original: Thus, Diaz & Marathe (2019) trained a regressor using soft ordinal labels to alleviate penalties onclose predictions.
Revised: Thus, Diaz & Marathe (2019) trained a regressor using soft ordinal labels to alleviate penalties on close predictions.

Original: Furthermore, Li et al.
Revised: Furthermore, Li et al.

Original: (2021) and Li et al.
Revised: (2021) and Li et al.

Original: (2022) modeled the uncertainty of eachprediction as a Gaussian distribution.
Revised: (2022) modeled the uncertainty of each prediction as a Gaussian distribution.

Original: In contrast to these methods, the proposed algorithm providesmore accurate rank estimates, although it uses only a single encoder network with the simple k -NNrule and makes no complicated probabilistic assumptions.
Revised: In contrast to these methods, the proposed algorithm provides more accurate rank estimates, although it uses only a single encoder network with the simple k -NN rule and makes no complicated probabilistic assumptions.

Original: Order learning:Rank-ordered (or ordinal) descriptors have been used in computer vision. Forexample, Toews & Wells (2009) proposed SIFT-rank descriptors for correspondence matching, whichwere also used to model anatomical development in MRI data (Toews et al., 2012). Recently, withthe great success of deep learning, Lim et al.
Revised: Order learning: Lim et al.

Original: ﬁrst proposed the notion of order learning,which learns ordering relationships between objects and determines the rank of an unseen object bycomparing it with references with known ranks.
Revised: ﬁrst proposed the notion of order learning, which learns ordering relationships between objects and determines the rank of an unseen object by comparing it with references with known ranks.

Original: It yields promising results because relative assessmentis easier than absolute assessment in general.
Revised: It yields promising results because relative assessment is easier than absolute assessment in general.

Original: Lee & Kim (2021) improved the performance of orderlearning by ﬁnding more reliable references.
Revised: Lee & Kim (2021) improved the performance of order learning by ﬁnding more reliable references.

Original: They decomposed object information into an orderrelated feature and an identity feature and showed that objects with similar identity features can becompared more reliably.
Revised: They decomposed object information into an order-related feature and an identity feature and showed that objects with similar identity features can be compared more reliably.

Original: Also, Shin et al.
Revised: Also, Shin et al.

Original: (2022) extended the classiﬁcation approach in (Lim et al.
Revised: (2022) extended the classiﬁcation approach in (Lim et al., 2020;

Original: ,2020; Lee & Kim, 2021) to regression-based one.
Revised: Lee & Kim, 2021) to regression-based one.

Original: These order learning methods, however, require asigniﬁcant computational cost to ﬁnd reliable references from an entire training set.
Revised: These order learning methods, however, require a signiﬁcant computational cost to ﬁnd reliable references from an entire training set.

Original: Moreover, forrank estimation, they should do comparisons with many references with different ranks because theyconsider only relative priorities between objects.
Revised: Moreover, for rank estimation, they should do comparisons with many references with different ranks because they consider only relative priorities between objects.

Original: On the contrary, the proposed algorithm simplycarries out the k -NN search to yield outstanding rank estimation results.
Revised: On the contrary, the proposed algorithm simply carries out the k -NN search to yield outstanding rank estimation results.

Original: Metric learning: Metric learning aims to construct an embedding space in which the distancebetween objects reﬂects their semantic difference.
Revised: Metric learning: Metric learning aims to construct an embedding space in which the distance between objects reﬂects their semantic difference.

Original: Most metric learning algorithms (Schroff et al. ,2015; Deng et al., 2019) are for ordinary classiﬁcation, clustering, or image retrieval tasks.
Revised: Most metric learning algorithms (Schroff et al., 2015; Deng et al., 2019) are for ordinary classiﬁcation, clustering, or image retrieval tasks.

Original: Therefore,they enforce an object to be located near other objects in the same class in the embedding space butfar from objects in different classes.
Revised: Therefore, they enforce an object to be located near other objects in the same class in the embedding space but far from objects in different classes.

Original: However, in rank estimation, this approach may be suboptimalbecause it does not consider ordinal relationships among classes.
Revised: However, in rank estimation, this approach may be suboptimal because it does not consider ordinal relationships among classes.

Original: For example, in movie rating, itdoes not discriminate the class difference between ‘outstanding’ and ‘very good’ from that between‘outstanding’ and ‘poor.’
Revised: For example, in movie rating, it does not discriminate the class difference between ‘outstanding’ and ‘very good’ from that between ‘outstanding’ and ‘poor.’

Original: To alleviate this problem, Xiao et al.
Revised: To alleviate this problem, Xiao et al.

Original: (2009) designed a metric, calledlabeled distance, to measure semantic similarities between objects and attempted to preserve localsemantic structures in the feature space.
Revised: (2009) designed a metric, called labeled distance, to measure semantic similarities between objects and attempted to preserve local semantic structures in the feature space.

Original: Moreover, to preserve the ordinal relationships, Li et al.(2012) developed a metric learning algorithm to make the distances between objects proportionalto their rank differences.
Revised: Moreover, to preserve the ordinal relationships, Li et al. (2012) developed a metric learning algorithm to make the distances between objects proportional to their rank differences.

Original: Also, Tian et al.
Revised: Also, Tian et al.

Original: (2016) employed a series of margins to explicitly imposedifferent embedding distances according to rank differences.
Revised: (2016) employed a series of margins to explicitly impose different embedding distances according to rank differences.

Original: Suárez et al.
Revised: Suárez et al.

Original: (2021) attempted to makethe embedding distances between pairs of objects sorted according to their rank differences.
Revised: (2021) attempted to make the embedding distances between pairs of objects sorted according to their rank differences.

Original:  Proposed Algorithm 
Revised:  Proposed Algorithm 3.1 Preliminary – Order and Metric

Original: 3.1 Preliminary – Order and Metric

Original: Mathematically, both order and metric are binary relations (Hrbacek & Jech, 1984).
Revised: Mathematically, both order and metric are binary relations (Hrbacek & Jech, 1984).

Original: An order, denotedby ≤ , on a set  should satisfy the properties of reﬂexivity, antisymmetry, and transitivity (Schröder,2003).
Revised: An order (Schröder, 2003), denoted by ≤ , on a set Θ = { θ 0 , θ 1 , . . . , θ M − 1 } should satisfy the properties of • Reﬂexivity: θ i ≤ θ i for all i , • Antisymmetry: θ i ≤ θ j and θ j ≤ θ i imply θ i = θ j , • Transitivity: θ i ≤ θ j and θ j ≤ θ k imply θ i ≤ θ k .

Original: On the other hand, a metric  is a distance function d satisfying nonnegativity, commutativity,and triangle inequality (Rudin, 1991).{+• Nonnegativity: d ( θ i , θ j ) ≥ 0 for all i, j , and d ( θ i , θ j ) = 
 if and only if θ i = θ j , • Commutativity: d ( θ i , θ j ) =+}
Revised: On the other hand, a metric (Rudin, 1991) is a distance function d satisfying {+• Nonnegativity: d ( θ i , θ j ) ≥ 0 for all i, j , and d ( θ i , θ j ) = 
 if and only if θ i = θ j , • Commutativity: d ( θ i , θ j ) =+}

Revised: d ( θ j , θ i ) for all i, j , • Triangle inequality: d ( θ i , θ k ) ≤ d ( θ i , θ j ) + d ( θ j , θ k ) for all i, j, k .

Original: In rank estimation, an order describes the priorities of ranks or classes in the set Θ = { θ 0 , . . .
Revised: In rank estimation, an order describes the priorities of ranks or classes in the set Θ = { θ 0 , . . .

Original: , θ M − 1 } ,where each rank represents one or more object instances.
Revised: , θ M − 1 } , where each rank represents one or more object instances.

Original: For example, in age estimation, θ imay represent i -year-olds, and θ 17 < θ 32 indicates that 17-year-olds are younger than 32-yearolds.
Revised: For example, in age estimation, θ i may represent i -year-olds, and θ 17 < θ 32 indicates that 17-year-olds are younger than 32-yearolds.

Original: Let θ ( · ) be the rank function, and let x and y be instances.
Revised: Let θ ( · ) be the rank function, and let x and y be instances.

Original: Then, θ ( x ) = θ 17 means thatperson x is 17-year-old.
Revised: Then, θ ( x ) = θ 17 means that person x is 17-year-old.

Original: Also, a metric describes the difference between ranks in Θ .
Revised: Also, a metric describes the difference between ranks in Θ .

Original: For example,d ( θ ( x ) , θ ( y ))
Revised: For example, d ( θ ( x ) , θ ( y ))

Original: = 15 means that two people x and y are 15 years apart.
Revised: = 15 means that two people x and y are 15 years apart.

Original: 3.2 Embedding Space Construction

Original: Given an object instance x , the objective is to estimate its rank θ ( x ) .

Original: In such rank estimation, an orderand a metric convey complementary information: the order provides directional information betweenranks, while the metric does length (or magnitude of difference) information.

Original: In age estimation, letus consider three age ranks θ 2 , θ 17 , and θ 32 .

Original: [-Since θ 2 < θ 17 < θ 32 , the order informs that θ 2 and θ 
-]
Revised: [-Since θ 2 < θ 17 < θ 32 , the order informs that θ 2 and θ 
-]

Original: Instances 𝑥 𝑦 ℎ ℎ
Revised: Instances 𝑥 𝑦 ℎ ℎ

Original: Encoder ℎ 𝑦 ℎ 𝑥
Revised: Encoder ℎ 𝑦 ℎ 𝑥

Original: Order constraint
Revised: Order constraint

Original: Metric constraint
Revised: Metric constraint

Original: AttractionRepulsion
Revised: AttractionRepulsion

Original: Reference points
Revised: Reference points

Original: Embedding space
Revised: Embedding space

Original: Instance embeddings
Revised: Instance embeddings

Original: Forward direction Backward direction
Revised: Forward direction Backward direction

Original: Direction vector
Revised: Direction vector

Original: Rank4 3 2ℎ 𝑦 𝑟 𝜃(𝑥)− 𝑟 𝜃(𝑦)+ 𝑟 𝜃(𝑥) 𝑟 𝜃(𝑦)
Revised: Rank4 3 2ℎ 𝑦 𝑟 𝜃(𝑥)− 𝑟 𝜃(𝑦)+ 𝑟 𝜃(𝑥) 𝑟 𝜃(𝑦)

Original: ℎ 𝑦 ℎ 𝑥
Revised: ℎ 𝑦 ℎ 𝑥

Original: 𝑣 f 𝑣 b 𝑟 𝜃(𝑥) 𝑟 𝜃(𝑥)− 𝑟 𝜃(𝑦) 
Revised: 𝑣 f 𝑣 b 𝑟 𝜃(𝑥) 𝑟 𝜃(𝑥)− 𝑟 𝜃(𝑦) ℎ 𝑥

Revised: 3.2 Embedding Space Construction

Original: ℎ 𝑥 are at the opposite sides with respect to θ 17 .
Revised: Given an object instance x , the objective is to estimate its rank θ ( x ) . In such rank estimation, an order and a metric convey complementary information: the order provides directional information between ranks, while the metric does length (or magnitude of difference) information. In age estimation, let us consider three age ranks θ 2 , θ 17 , and θ 32 . Since θ 2 < θ 17 < θ 32 , the order informs that θ 2 and θare at the opposite sides with respect to θ 17 .

Original: On the other hand, since d ( θ 2 , θ 17 ) =
Revised: On the other hand, since d ( θ 2 , θ 17 ) =

Original: d ( θ 32 , θ 17 ) = 15 ,the metric indicates how far θ 2 and θ 32 are from θ 17 .
Revised: d ( θ 32 , θ 17 ) = 15 , the metric indicates how far θ 2 and θ 32 are from θ 17 .

Original: In this case, both lengths are identically 15.
Revised: In this case, both lengths are identically 15.

Original: For rank estimation, the pairwise comparison methods (Lim et al., 2020; Lee & Kim, 2021; Nguyenet al., 2018) attempt to learn these relations.
Revised: For rank estimation, the pairwise comparison methods (Lim et al., 2020; Lee & Kim, 2021; Nguyen et al., 2018) attempt to learn these relations.

Original: However, Lim et al.
Revised: However, Lim et al.

Original: and Lee & Kim (2021)exploit the order relation only, whereas Nguyen et al.
Revised: and Lee & Kim (2021) exploit the order relation only, whereas Nguyen et al.

Original: (2018) use the metric relation only to train theirneural networks.
Revised: (2018) use the metric relation only to train their neural networks.

Original: Thus, the conventional methods may yield sub-optimal results.
Revised: Thus, the conventional methods may yield sub-optimal results.

Original: To learn both order and metric relations, we propose a geometric approach called GOL.
Revised: To learn both order and metric relations, we propose a geometric approach called GOL.

Original: It containstwo types of geometric constraints that enforce directional (order) and distance (metric) relationshipsbetween object instances according to their ranks in an embedding space.
Revised: It contains two types of geometric constraints that enforce directional (order) and distance (metric) relationships between object instances according to their ranks in an embedding space.

Original: Speciﬁcally, the orderconstraint sorts instances directionally according to the ranks, while the metric constraint separatestwo instances farther if their rank difference is larger.
Revised: Speciﬁcally, the order constraint sorts instances directionally according to the ranks, while the metric constraint separates two instances farther if their rank difference is larger.

Original: Figure 2 is an overview of GOL.
Revised: Figure 2 is an overview of GOL.

Original: Order constraint: Suppose that there are M ranks in a training set X .
Revised: Order constraint: Suppose that there are M ranks in a training set X .

Original: Without loss of generality,the ranks are assumed to be consecutive integers in Θ = { 0 , 1 , . . .
Revised: Without loss of generality, the ranks are assumed to be consecutive integers in Θ = { 0 , 1 , . . .

Original: , M − 1 } .
Revised: , M − 1 } .

Original: In Figure 2, an encoderh maps each instance x ∈ X into a feature vector
Revised: In Figure 2, an encoder h maps each instance x ∈ X into a feature vector

Original: h x = h
Revised: h x = h

Original: ( x ) in an embedding space.
Revised: ( x ) in an embedding space.

Original: As h , weadopt VGG16 (Simonyan & Zisserman, 2015) without fully connected layers.
Revised: As h , we adopt VGG16 (Simonyan & Zisserman, 2015) without fully connected layers.

Original: The output of the lastpooling layer is normalized so that h tx h x = 1 .
Revised: The output of the last pooling layer is normalized so that h tx h x = 1 .

Original: Thus, the embedding space is a unit hypersphere.
Revised: Thus, the embedding space is a unit hypersphere.

Original: As in Lim et al.
Revised: As in Lim et al.

Original: and Lee & Kim (2021), we classify the ordering between two instances x andy in X into three categories:
Revised: and Lee & Kim (2021), we classify the ordering between two instances x and y in X into three categories:

Original: [Equation]
Revised: [Equation]

Original: where τ is a threshold.
Revised: where τ is a threshold.

Original: For instance ordering, notations ‘ ≺ , ≈ , (cid:31) ’ are used instead of ‘ <, = , > .’
Revised: For instance ordering, notations ‘ ≺ , ≈ , (cid:31) ’ are used instead of ‘ <, = , > .’

Original: The order constraint encourages instances to be sorted according to their ordering relationships.
Revised: The order constraint encourages instances to be sorted according to their ordering relationships.

Original: In other words, for two instances x and y with ordering x ≺ y , the vector from h x to h y shouldbe aligned with the direction of the rank increment in the embedding space.
Revised: In other words, for two instances x and y with ordering x ≺ y , the vector from h x to h y should be aligned with the direction of the rank increment in the embedding space.

Original: To model such rankdirections, we introduce M reference points, r 0 , r 1 , . .
Revised: To model such rank directions, we introduce M reference points, r 0 , r 1 , . .

Original: , r M − 1 , which are learnable parametersguiding the positions of the M ranks in the embedding space.
Revised: , r M − 1 , which are learnable parameters guiding the positions of the M ranks in the embedding space.

Original: These reference points are randomlyinitialized by the Glorot normal method (Glorot & Bengio, 2010) and jointly optimized with encoderparameters during training.
Revised: These reference points are randomly initialized by the Glorot normal method (Glorot & Bengio, 2010) and jointly optimized with encoder parameters during training.

Original: Let us deﬁne the direction vector v ( r, s ) from point r to point s on the unit hypersphere as
Revised: Let us deﬁne the direction vector v ( r, s ) from point r to point s on the unit hypersphere as

Original: [Equation]
Revised: [Equation]

Original: Then, v ( r i , r j ) is called the rank direction from rank i to rank j .
Revised: Then, v ( r i , r j ) is called the rank direction from rank i to rank j .

Original: Also, the rank direction v ( r i , r j ) isforward if i < j , and backward if i > j .
Revised: Also, the rank direction v ( r i , r j ) is forward if i < j , and backward if i > j .

Original: Note that forward directions may differ from one another,for they may represent different physical changes.
Revised: Note that forward directions may differ from one another, for they may represent different physical changes.

Original: For example, in age estimation, visual variationsfrom 0-years-old to 5-years-old are mainly due to craniofacial development, whereas those from45-years-old to 50-years-old are due to skin aging (Geng et al., 2007).
Revised: For example, in age estimation, visual variations from 0-years-old to 5-years-old are mainly due to craniofacial development, whereas those from 45-years-old to 50-years-old are due to skin aging (Geng et al., 2007).

Original: If x ≺ y , we determine the forward and backward rank directions, respectively, by
Revised: If x ≺ y , we determine the forward and backward rank directions, respectively, by

Original: [Equation]
Revised: [Equation]

Original: Then, the encoder is trained so that the embedded features h x and h y satisfy the order constraint:
Revised: Then, the encoder is trained so that the embedded features h x and h y satisfy the order constraint:

Original: [Equation]
Revised: [Equation]

Original: In other words, the direction vector v ( h x , h y ) should be aligned more with the forward direction v fthan with the backward direction v b .
Revised: In other words, the direction vector v ( h x , h y ) should be aligned more with the forward direction v f than with the backward direction v b .

Original: To enforce the order constraint in (5), we compute the softmax probability p xy
Revised: To enforce the order constraint in (5), we compute the softmax probability p xy

Original: =
Revised: =

Original: [ p xy f , p xy b ] t , wherep xy f = e v t f v ( h x ,h y ) /
Revised: [ p xy f , p xy b ] t , where p xy f = e v t f v ( h x ,h y ) /

Original: ( e v t
Revised: ( e v t

Original: f
Revised: f

Original: v ( h x ,h y )
Revised: v ( h x ,h y )

Original: + e v t b v ( h x ,h y ) ) and p xy b = 1 − p xy f .
Revised: + e v t b v ( h x ,h y ) ) and p xy b = 1 − p xy f .

Original: We then deﬁne the order loss
Revised: We then deﬁne the order loss

Original: L order as the cross entropy between p xy and q xy =
Revised: L order as the cross entropy between p xy and q xy =

Original: [ q xy f , q xy b ] t = [1 , 0] t , given by
Revised: [ q xy f , q xy b ] t = [1 , 0] t , given by

Original: [Equation]
Revised: [Equation]

Original: The order loss for case x (cid:31) y is formulated similarly in a symmetric manner.
Revised: The order loss for case x (cid:31) y is formulated similarly in a symmetric manner.

Original: Metric constraint: Next, we formulate a metric constraint to make the distance between instancesin the embedding space reﬂect their rank difference.
Revised: Metric constraint: Next, we formulate a metric constraint to make the distance between instances in the embedding space reﬂect their rank difference.

Original: Speciﬁcally, it is desirable that
Revised: Speciﬁcally, it is desirable that

Original: [Equation]
Revised: [Equation]

Original: where d e is the Euclidean distance in the embedding space, and γ is a margin.
Revised: where d e is the Euclidean distance in the embedding space, and γ is a margin.

Original: Note that if | θ ( x ) −θ ( y ) | > τ , either x ≺ y or x (cid:31) y in (1).
Revised: Note that if | θ ( x ) − θ ( y ) | > τ , either x ≺ y or x (cid:31) y in (1).

Original: Hence, the metric constraint in (7) is equivalent to
Revised: Hence, the metric constraint in (7) is equivalent to

Original: [Equation]
Revised: [Equation]

Original: From the triangle inequality, we have | d e ( r i , h x ) − d e ( r i , h y ) | ≤ d e ( h x , h y ) for every referencepoint r i .
Revised: From the triangle inequality, we have | d e ( r i , h x ) − d e ( r i , h y ) | ≤ d e ( h x , h y ) for every reference point r i .

Original: So, if x ≈ y , we also have | d e ( r i , h x ) − d e ( r i , h y ) | ≤ γ .
Revised: So, if x ≈ y , we also have | d e ( r i , h x ) − d e ( r i , h y ) | ≤ γ .

Original: To encourage this inequality, wedeﬁne a loss L x ≈ y as
Revised: To encourage this inequality, we deﬁne a loss L x ≈ y as

Original: [Equation]
Revised: [Equation]

Original: On the contrary, if x ≺ y , it should be that d e ( h x , h y ) > γ .
Revised: On the contrary, if x ≺ y , it should be that d e ( h x , h y ) > γ .

Original: Thus, we deﬁne another loss
Revised: Thus, we deﬁne another loss

Original: L x ≺

Original: [Equation]
Revised: [Equation]

Original: To minimize the ﬁrst sum, d e ( r i , h x ) should be reduced, while d e ( r i , h y ) should be increased.
Revised: To minimize the ﬁrst sum, d e ( r i , h x ) should be reduced, while d e ( r i , h y ) should be increased.

Original: Thus,reference points r i , 0 ≤ i ≤ θ ( x ) , are trained to attract h x and repel h y , as illustrated in Figure 2.
Revised: Thus, reference points r i , 0 ≤ i ≤ θ ( x ) , are trained to attract h x and repel h y , as illustrated in Figure 2.

Original: The second sum in (10) is similar.
Revised: The second sum in (10) is similar.

Original: We do not use reference points r l , θ ( x ) <
Revised: We do not use reference points r l , θ ( x ) <

Original: l < θ ( y ) , which tendto be between h x and h y and unhelpful for guiding them.
Revised: l < θ ( y ) , which tend to be between h x and h y and unhelpful for guiding them.

Original: The derivation of L x ≺ y in (10) from themetric constraint in (7) is provided in Appendix B. Also, L x (cid:31) y is formulated symmetrically.
Revised: The derivation of L x ≺ y in (10) from the metric constraint in (7) is provided in Appendix B. Also, L x (cid:31) y is formulated symmetrically.

Original: Then, we deﬁne the metric loss L metric as
Revised: Then, we deﬁne the metric loss L metric as

Original: [Equation]
Revised: [Equation]

Original: where [ · ] is the indicator function.
Revised: where [ · ] is the indicator function.

Original: Note that, in (9) or (10), we make h x and h y attract or repeleach other indirectly through reference points r i .
Revised: Note that, in (9) or (10), we make h x and h y attract or repel each other indirectly through reference points r i .

Original: This is because the direct attraction or repulsionusing d e ( h x , h y ) may cause h x and h y to move in arbitrary directions.
Revised: This is because the direct attraction or repulsion using d e ( h x , h y ) may cause h x and h y to move in arbitrary directions.

Original: Because the reference pointsare trained also with the order loss in (6), they provide proper directional guidance of attraction orrepulsion in (9) or (10).
Revised: Because the reference points are trained also with the order loss in (6), they provide proper directional guidance of attraction or repulsion in (9) or (10).

Original: Experimental analysis on L x ≺ y in (10) is in Appendix D.6.
Revised: Experimental analysis on L x ≺ y in (10) is in Section 4.4.

Original: Loss function:
Revised: Loss function:

Original: In addition to the losses for the order and metric constraints, we employ the centerloss (Nguyen et al., 2018), which aims at locating each reference point r
Revised: In addition to the losses for the order and metric constraints, we employ the center loss (Nguyen et al., 2018), which aims at locating each reference point r

Original: i at the center of all instanceswith rank i .
Revised: i at the center of all instances with rank i .

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original: Finally, we use the overall loss function to optimize the encoder parameters and the reference pointsr i , which is given by[Equation]
Revised: Finally [Table] we use the overall loss function to optimize the encoder parameters and the reference points r i , which is given by

Original: [Equation]
Revised: [Equation]

Original: It is worth pointing out that reference points play essential roles in all three loss terms in (13).
Revised: It is worth pointing out that reference points play essential roles in all three loss terms in (13).

Original: Theyare used to deﬁne forward and backward rank directions in L order , so they help to sort instancesdirectionally according to the ranks.
Revised: They are used to deﬁne forward and backward rank directions in L order , so they help to sort instances directionally according to the ranks.

Original: The reference points themselves are also sorted since instancesare clustered around the reference points because of L center .
Revised: The reference points themselves are also sorted since instances are clustered around the reference points because of L center .

Original: Also, as mentioned before, the referencepoints provide proper directional guidance of attraction or repulsion in L metric to satisfy the metricconstraint.
Revised: Also, as mentioned before, the reference points provide proper directional guidance of attraction or repulsion in L metric to satisfy the metric constraint.

Original: In other words, GOL uses the reference points to satisfy both order and metric constraintssimultaneously and thus constructs a well-arranged, well-clustered embedding space.[Table] [Figure]
Revised: In other words, GOL uses the reference points to satisfy both order and metric constraints simultaneously and thus constructs a well-arranged, well-clustered embedding space.

Original: 3.3 k -NN Rank Estimation
Revised: 3.3 k -NN Rank Estimation

Original: For rank estimation, we use the simple k -NN rule.
Revised: For rank estimation, we use the simple k -NN rule.

Original: Given a test instance x , in the embedding space,we ﬁnd a set N of its k NNs among all training instances in X .
Revised: Given a test instance x , in the embedding space, we ﬁnd a set N of its k NNs among all training instances in X .

Original: Then, the rank of x is estimated by
Revised: Then, the rank of x is estimated by

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original:  Experimental Results
Revised:  Experimental Results

Original: We conduct experiments on embedding space construction and rank estimation.
Revised: We conduct experiments on embedding space construction and rank estimation.

Original: Implementationdetails and more results are available in Appendices C and D, respectively.
Revised: Implementation details and more results are available in Appendices C and D, respectively.

Original: 4.1 Implementation
Revised: 4.1 Implementation

Original: We initialize an encoder h with VGG16 pre-trained on ILSVRC2012 (Deng et al., 2009) and referencepoints with the Glorot normal method (Glorot & Bengio, 2010).
Revised: We initialize an encoder h with VGG16 pre-trained on ILSVRC2012 (Deng et al., 2009) and reference points with the Glorot normal method (Glorot & Bengio, 2010).

Original: The Adam optimizer (Kingma &
Revised: The Adam optimizer (Kingma &

Original: Ba, 2015) is used with a batch size of 32 and a weight decay of 5 × 10 − 4 , and the initial learningrates for the encoder and the reference points are set to 10 − 4 and 10 − 3 , respectively.
Revised: Ba, 2015) is used with a batch size of 32 and a weight decay of 5 × 10 − 4 , and the initial learning rates for the encoder and the reference points are set to 10 − 4 and 10 − 3 , respectively.

Original: We perform thescheduled learning according to cosine annealing cycles (Huang et al., 2017).
Revised: We perform the scheduled learning according to cosine annealing cycles (Huang et al., 2017).

Original: For data augmentation,we do random horizontal ﬂips only.
Revised: For data augmentation, we do random horizontal ﬂips only.

Original: 4.2 Embedding Spaces
Revised: 4.2 Embedding Spaces

Original: The proposed GOL algorithm attempts to design an embedding space in which both order and metricconstraints are satisﬁed: instances should be well sorted according to their ranks and well separatedif they have big rank differences. 
Revised: The proposed GOL algorithm attempts to design an embedding space in which both order and metric constraints are satisﬁed: instances should be well sorted according to their ranks and well separated if they have big rank differences. [Table] To measure the quality of an embedding space for rank estimation [Table]

Original: To measure the quality of an embedding space for rank estimation, we may adopt the between-classvariance to within-class variance (B2W) criterion in Fisher’s linear discriminant (Duda et al., 2006), 
Revised: we may adopt the between-class variance to within-class variance (B2W) criterion in Fisher’s linear discriminant (Duda et al., 2006), B2W = (cid:0) (cid:88) [Table] |X i | d 2e ( c i , c ) (cid:1)

Original: [Equation]
Revised: [Equation]

Original: (15) where X i = { x ∈ X | θ ( x ) = i } , c i = (cid:80) x ∈X i h x / |X i | is the centroid for X i , and c = (cid:80) x ∈X h x / |X|is the centroid for all instances in X .
Revised: where X i = { x ∈ X | θ ( x ) = i } , c i = (cid:80) x ∈X i h x / |X i | is the centroid for X i , and c = (cid:80) x ∈X h x / |X| is the centroid for all instances in X .

Original: A high B2W score indicates that the rank sets X i , 0 ≤ i ≤ M − 1 ,are well separated from one another, while instances in each rank set are compactly distributed.
Revised: A high B2W score indicates that the rank sets X i , 0 ≤ i ≤ M − 1 , are well separated from one another, while instances in each rank set are compactly distributed.

Original: However, B2W does not consider the ordinal relationships of ranks in Θ since it was formulated forordinary classiﬁcation.
Revised: However, B2W does not consider the ordinal relationships of ranks in Θ since it was formulated for ordinary classiﬁcation.

Original: Therefore, to assess embedding spaces for rank estimation, we propose the[Table] discriminative ratio for ranking (DRR), given by
Revised: Therefore, to assess embedding spaces for rank estimation, we propose the discriminative ratio for ranking (DRR), given by

Original: [Equation]
Revised: [Equation]

Original:  which is the ratio of the average pairwise centroid distance to the average pairwise instance distancein each rank set.
Revised: 16) which is the ratio of the average pairwise centroid distance to the average pairwise instance distance in each rank set.

Original: Note that, in the numerator, ordinal weights ( j − i ) β are used to emphasize thedifference between a pair of rank sets with a large rank difference.
Revised: Note that, in the numerator, ordinal weights ( j − i ) β are used to emphasize the difference between a pair of rank sets with a large rank difference.

Original: Here, β is a nonnegative parameterfor controlling the level of emphasis.
Revised: Here, β is a nonnegative parameter for controlling the level of emphasis.

Original: A high DRR score is obtained when instances are well sortedand well separated according to their ranks in the embedding space.
Revised: A high DRR score is obtained when instances are well sorted and well separated according to their ranks in the embedding space.

Original: Table 1 compares the B2W and DRR scores on the MORPH II (Ricanek & Tesafaye, 2006),
Revised: Table 1 compares the B2W and DRR scores on the MORPH II (Ricanek & Tesafaye, 2006),

Original: CACD (Chen et al., 2015), and Adience (Levi & Hassner, 2015) train data.
Revised: CACD (Chen et al., 2015), and Adience (Levi & Hassner, 2015) train data.

Original: To compare the embeddingspaces as fairly as possible, we use the same encoder backbone of VGG16 for all algorithms.
Revised: To compare the embedding spaces as fairly as possible, we use the same encoder backbone of VGG16 for all algorithms.

Original: GOLsigniﬁcantly outperforms all conventional algorithms in all tests.
Revised: GOL signiﬁcantly outperforms all conventional algorithms in all tests.

Original: In GOL, instances within each rankset X i are located compactly around reference point r i by L center in (12), while those in different ranksets are well separated by L metric in (11).
Revised: In GOL, instances within each rank set X i are located compactly around reference point r i by L center in (12), while those in different rank sets are well separated by L metric in (11).

Original: Hence, GOL yields outstanding B2W scores.
Revised: Hence, GOL yields outstanding B2W scores.

Original: Moreover,
Revised: Moreover,

Original: GOL excels with a larger score gap in DRR 1 . 0 than in DRR 0 .
Revised: GOL excels with a larger score gap in DRR 1 . 0 than in DRR 0 .

Original: 5 , which means that it arranges the ranksets effectively in the embedding space to reﬂect their ordinal relationships using L order in (6).
Revised: 5 , which means that it arranges the rank sets effectively in the embedding space to reﬂect their ordinal relationships using L order in (6).

Original: Figure 3 visualizes the embedding spaces on Adience.
Revised: Figure 3 visualizes the embedding spaces on Adience.

Original: As in Figure 1, we add a fully connected layerwith 3 output neurons to each encoder for the visualization.
Revised: As in Figure 1, we add a fully connected layer with 3 output neurons to each encoder for the visualization.

Original: In (a) and (b), instances are sorted butscattered over the spaces, as the order learning algorithms (Lim et al., 2020; Shin et al., 2022) ignorethe metric relation of ranks.
Revised: In (a) and (b), instances are sorted but scattered over the spaces, as the order learning algorithms (Lim et al., 2020; Shin et al., 2022) ignore the metric relation of ranks.

Original: In (c), instances in each rank set are well clustered, but the rank sets arenot sorted because ML (Schroff et al., 2015) neglects the order relation.
Revised: In (c), instances in each rank set are well clustered, but the rank sets are not sorted because ML (Schroff et al., 2015) neglects the order relation.

Original: In (d), MV (Pan et al., 2018),which is an ordinal regressor, exhibits large within-class scattering, as well as large between-classscattering.
Revised: In (d), MV (Pan et al., 2018), which is an ordinal regressor, exhibits large within-class scattering, as well as large between-class scattering.

Original: In contrast, in (e), GOL constructs a well-sorted, well-clustered embedding space.
Revised: In contrast, in (e), GOL constructs a well-sorted, well-clustered embedding space.

Original: 4.3 Rank Estimation
Revised: 4.3 Rank Estimation

Original: Since GOL constructs high-quality embedding spaces, it provides excellent rank estimation performances even with the simple k -NN rule.
Revised: Since GOL constructs high-quality embedding spaces, it provides excellent rank estimation performances even with the simple k -NN rule.

Original: Facial age estimation: We use four datasets of MORPH II (Ricanek & Tesafaye, 2006), CACD(Chen et al., 2015), UTK (Zhang et al., 2017), and Adience (Levi & Hassner, 2015), as detailed in
Revised: Facial age estimation: We use four datasets of MORPH II (Ricanek & Tesafaye, 2006), CACD (Chen et al., 2015), UTK (Zhang et al., 2017), and Adience (Levi & Hassner, 2015), as detailed in

Original: Appendix D.3.
Revised: Appendix D.3.

Original:  In Table 2, we compare the performances in the four evaluation settings of MORPH II, which is oneof the most popular datasets in age estimation.
Revised: [Table] [Table] [Table] In Table 2, we compare the performances in the four evaluation settings of MORPH II, which is one of the most popular datasets in age estimation.

Original: We use the mean absolute error (MAE) and cumulativescore (CS) metrics.
Revised: We use the mean absolute error (MAE) and cumulative score (CS) metrics.

Original: MAE is the average absolute error between estimated and ground-truth ages ( i.e.ranks), and CS computes the percentage of images whose absolute errors are less than or equal toa tolerance level l = 5 .
Revised: MAE is the average absolute error between estimated and ground-truth ages ( i.e. ranks), and CS computes the percentage of images whose absolute errors are less than or equal to a tolerance level l = 5 .

Original: Note that all algorithms in Table 2 use VGG16 as the encoder backbones,except for C3AE employing a shallow CNN.
Revised: Note that all algorithms in Table 2 use VGG16 as the encoder backbones, except for C3AE employing a shallow CNN.

Original: As for Shin et al.
Revised: As for Shin et al.

Original: (2022), the scores of the globalρ -regressor are compared since their local scheme employs as many as six independent VGG16encoders recursively.
Revised: (2022), the scores of the global ρ -regressor are compared since their local scheme employs as many as six independent VGG encoders recursively.

Original: GOL performs the best in 5 out of 8 tests, including setting C, which is themost challenging task.
Revised: GOL performs the best in 5 out of 8 tests, including setting C, which is the most challenging task.

Original: Furthermore, unlike most conventional algorithms, GOL does not exploit the[Table] [Table] [Table]
Revised: Furthermore, unlike most conventional algorithms, GOL does not exploit the

Original: IMDB-WIKI pretraining to boost performances.
Revised: IMDB-WIKI pretraining to boost performances.

Original: Even without the pretraining, i.e. by employing only
Revised: Even without the pretraining, i.e. by employing only

Original: MORPH II training data
Revised: MORPH II training data

Original: , GOL yields outstanding results in Table 2.
Revised: , GOL yields outstanding results in Table 2.

Original: Table 3 compares the results on CACD, which is a bigger dataset containing over 100,000 naturalface shots in diverse environments.
Revised: Table 3 compares the results on CACD, which is a bigger dataset containing over 100,000 natural face shots in diverse environments.

Original: GOL outperforms the second-best methods with meaningful gapsof 0.12 and 0.17 in the train and validation settings, respectively, which indicates that it can copewith large and diverse data effectively as well.
Revised: GOL outperforms the second-best methods with meaningful gaps of 0.12 and 0.17 in the train and validation settings, respectively, which indicates that it can cope with large and diverse data effectively as well.

Original: Table 3 also compares the results on UTK.
Revised: Table 3 also compares the results on UTK.

Original: Note that Gustafsson et al.
Revised: Note that Gustafsson et al.

Original: (2020) and Berg et al.
Revised: (2020) and Berg et al.

Original: (2021)employ the deep ResNet50 network (He et al., 2016) as their encoders, and MWR-G (Shin et al. ,2022) predicts ranks using a complicated, recursive regressor.
Revised: (2021) employ the deep ResNet50 network (He et al., 2016) as their encoders, and MWR-G (Shin et al., 2022) predicts ranks using a complicated, recursive regressor.

Original: In contrast, GOL uses the shallower
Revised: In contrast, GOL uses the shallower

Original: VGG16 encoder and performs rank estimation based on the simple k -NN search.
Revised: VGG16 encoder and performs rank estimation based on the simple k -NN search.

Original: Nevertheless, ityields an excellent result, for its geometric constraints help to construct an effective embedding space.
Revised: Nevertheless, it yields an excellent result, for its geometric constraints help to construct an effective embedding space.

Original: Table 4 shows the results on Adience, where each image is labeled as one of 8 age groups.
Revised: Table 4 shows the results on Adience, where each image is labeled as one of 8 age groups.

Original: Comparedwith the second-best MWR-G, GOL improves the accuracy by 0.3% and reduces MAE by 0.03.
Revised: Compared with the second-best MWR-G, GOL improves the accuracy by 0.3% and reduces MAE by 0.03.

Original: HCI classiﬁcation: The HCI dataset (Palermo et al., 2012) is used for estimating the shootingdecade of a photograph.
Revised: HCI classiﬁcation: The HCI dataset (Palermo et al., 2012) is used for estimating the shooting decade of a photograph.

Original: It contains 1,325 images from ﬁve decades 1930s ∼ 1970s.
Revised: It contains 1,325 images from ﬁve decades 1930s ∼ 1970s.

Original: Table 4 liststhe results on HCI.
Revised: Table 4 lists the results on HCI.

Original: GOL yields the best performances as well, by improving the accuracy by 1.5%and reducing MAE by 0.05 as compared with the second-best methods.
Revised: GOL yields the best performances as well, by improving the accuracy by 1.5% and reducing MAE by 0.05 as compared with the second-best methods.

Original: This means that GOL yieldsreliable results even for a small dataset, which may be unfavorable for the k -NN estimation.
Revised: This means that GOL yields reliable results even for a small dataset, which may be unfavorable for the k -NN estimation.

Original: Aesthetic score regression: The aesthetics dataset (Schifanella et al., 2015) contains 15,687 imagesin four categories.
Revised: Aesthetic score regression: The aesthetics dataset (Schifanella et al., 2015) contains 15,687 images in four categories.

Original: Each image is annotated with a 5-scale aesthetic score.
Revised: Each image is annotated with a 5-scale aesthetic score.

Original: In Table 5, we follow theexperimental setting in (Liu et al., 2018).
Revised: In Table 5, we follow the experimental setting in (Liu et al., 2018).

Original: It is challenging to estimate aesthetic scores reliably due tothe subjectivity and ambiguity of aesthetic criteria, but GOL performs the best in 8 out of 10 tests.
Revised: It is challenging to estimate aesthetic scores reliably due to the subjectivity and ambiguity of aesthetic criteria, but GOL performs the best in 8 out of 10 tests.

Original: Compared with the state-of-the-art POE (Li et al., 2021), GOL improves the accuracy by 0.3% andreduces MAE by 0.1 overall. 
Revised: Compared with the state-of-the-art POE (Li et al., 2021), GOL improves the accuracy by 0.3% and reduces MAE by 0.1 overall. [Table] [Figure] [Table]

Original: 4.4 Analysis
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: 4.4 Analysis

Original: Ablation study: Table 6 compares ablated methods for the loss function in (13).
Revised: Ablation study: Table 6 compares ablated methods for the loss function in (13).

Original: Method Iemploys L center only.
Revised: Method I employs L center only.

Original: In II and III, L metric and L order are excluded, respectively.
Revised: In II and III, L metric and L order are excluded, respectively.

Original: Compared with IV(GOL), method I degrades the performances severely, since the center loss alone cannot constructa meaningful embedding space, as discussed in Appendix D.6.
Revised: Compared with IV (GOL), method I degrades the performances severely, since the center loss alone cannot construct a meaningful embedding space; a trivial solution to minimize L center can be obtained by merging all instances and all reference points into a single point.

Original: Also, from II and IV, we see thatthe metric constraint signiﬁcantly improves the performances by separating instances with differentranks from each other.
Revised: Also, from II and IV, we see that the metric constraint signiﬁcantly improves the performances by separating instances with different ranks from each other.

Original: From III and IV, we see that the order constraint also improves the resultsby arranging instances directionally according to their ranks.
Revised: From III and IV, we see that the order constraint also improves the results by arranging instances directionally according to their ranks.

Original: To summarize, both order and metricconstraints improve the results and are complementary to each other.
Revised: To summarize, both order and metric constraints improve the results and are complementary to each other.

Revised: Figure 4 (a) ∼ (d) show the embedding spaces for the ablated methods I ∼ IV in Table 6, respectively.

Revised: The top row visualizes the reduced 3D embedding spaces, as described in Appendix C.2, while the bottom row does the original 512D embedding spaces via t-SNE (Maaten & Hinton, 2008).

Revised: In (a), only L center is used, so the 3D embedding space almost collapses to a single point.

Revised: In (b), instances are directionally aligned, but adjacent rank sets overlap one another.

Original: [Figure]
Revised: In (c) and (d), the 3D embedding spaces seem similar. However, GOL yields more clearly ordered instances in the t-SNE visualization of the original embedding space than ‘ L center + L metric ’ does. Alternatives to L x ≺ y : Table 7 compares alternative loss terms for L x ≺ y in (10). Method I directly increases d e ( h x , h y ) to make it larger than γ .

Original: [Table]
Revised: Method II uses only two reference points r θ ( x ) and r θ ( y ) . In other words, unlike method III ( i.e. GOL), it does not employ reference points r i , 0 ≤ i < θ ( x ) , and θ ( y ) < r j ≤ M − 1 . For each method, L x (cid:31) y is also modiﬁed accordingly.

Revised: Method I degrades

Revised: [Figure]

Original: [Table]
Revised: [Table]

Revised: the performances signiﬁcantly, as h x and h y move in arbitrary directions.

Revised: Also, GOL performs better than II because attraction and repulsion with many reference points facilitate the positioning of instances in the embedding space, as well as the reference points themselves.

Revised: Embedding space transition: Figure 5 visualizes the transition of an embedding space for Adience.

Revised: As the GOL training step goes on, we see that instances are gradually sorted and separated according to their ranks to satisfy the geometric constraints.

Revised: After the convergence, for each rank, the reference point r i and the centroid c i are close to each other, indicating that instances are clustered around their reference points.

Revised: Moreover, those reference points are well sorted as well.

Original: Comparison with order learning: The conventional order learning algorithms (Lim et al., 2020;
Revised: Comparison with order learning: The conventional order learning algorithms (Lim et al., 2020;

Original: Lee & Kim, 2021; Shin et al., 2022) provide competitive ranking performances, but they requirerelatively heavy computations.
Revised: Lee & Kim, 2021; Shin et al., 2022) provide competitive ranking performances, but they require relatively heavy computations.

Original: Table 7 lists the testing times.
Revised: Table 8 lists the testing times.

Original: To estimate the rank of a test instance,
Revised: To estimate the rank of a test instance,

Original: OL uses ﬁve references for each of the M ranks; e.g. , 300 comparisons should be made if M = 60 asin typical age estimation.
Revised: OL uses ﬁve references for each of the M ranks; e.g. , 300 comparisons should be made if M = 60 as in typical age estimation.

Original: Moreover, OL and DRC-ORID need additional processes, such as MAPestimation, to estimate a rank based on ordering relationships.
Revised: Moreover, OL and DRC-ORID need additional processes, such as MAP estimation, to estimate a rank based on ordering relationships.

Original: Also, MWR-G estimates the rank bycomparing a test instance with multiple reference pairs recursively.
Revised: Also, MWR-G estimates the rank by comparing a test instance with multiple reference pairs recursively.

Original: In contrast, GOL simply performsthe k -NN estimation, which can be done efﬁciently in a parallel manner, and thus is about 160 and 23times faster than OL and MWR-G, respectively.
Revised: In contrast, GOL simply performs the k -NN estimation, which can be done efﬁciently in a parallel manner, and thus is about 160 andtimes faster than OL and MWR-G, respectively.

Original: Furthermore, the order learning algorithms should select references among a training set X througha complicated optimization process; e.g. , OL compares all possible pairs of training instances with
Revised: Furthermore, the order learning algorithms should select references among a training set X through a complicated optimization process; e.g. , OL compares all possible pairs of training instances with

Original: O ( |X| 2 ) complexity to select the most reliable references, and DRC-ORID (Lee & Kim, 2021)performs joint network training and clustering for reference selection.
Revised: O ( |X| 2 ) complexity to select the most reliable references, and DRC-ORID (Lee & Kim, 2021) performs joint network training and clustering for reference selection.

Original: On the contrary, GOL requiresno such process.
Revised: On the contrary, GOL requires no such process.

Original: Also, in Table 7, the proposed algorithm demands the fewest parameters because itdirectly estimates the rank of an instance in the embedding space through the k -NN search, whereasthe order learning algorithms should adopt comparators or regressors tailored for each task.
Revised: Also, in Table 8, the proposed algorithm demands the fewest parameters because it directly estimates the rank of an instance in the embedding space through the k -NN search, whereas the order learning algorithms should adopt comparators or regressors tailored for each task.

Original: Embedding space transition: Figure 4 visualizes the transition of an embedding space for Adience.

Original: As the GOL training step goes on, we see that instances are gradually sorted and separated accordingto their ranks to satisfy the geometric constraints.

Original: After the convergence, for each rank, the referencepoint r i and the centroid c i are close to each other, indicating that instances are clustered around theirreference points.

Original: Moreover, those reference points are well sorted as well.

Original:  Conclusions
Revised:  Conclusions

Original: The GOL algorithm for rank estimation was proposed in this work.
Revised: The GOL algorithm for rank estimation was proposed in this work.

Original: First, we construct an embeddingspace based on the two geometric constraints, which enforce the direction and distance betweeninstances to represent the order and metric relations between their ranks.
Revised: First, we construct an embedding space based on the two geometric constraints, which enforce the direction and distance between instances to represent the order and metric relations between their ranks.

Original: Then, we perform thesimple k -NN search in the embedding space for rank estimation.
Revised: Then, we perform the simple k -NN search in the embedding space for rank estimation.

Original: Extensive experiments on variousrank estimation tasks demonstrated that GOL constructs high-quality embedding spaces and yieldsexcellent rank estimation results.Checklist
Revised: Extensive experiments on various rank estimation tasks demonstrated that GOL constructs high-quality embedding spaces and yields excellent rank estimation results.

Original: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?

Original: [Yes]

Original: (b) Did you describe the limitations of your work?

Original: [Yes] See failure cases in Appendix D.6.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] SeeAppendix A.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [Yes] See Section 4.

Original: and Appendix C.

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] See Appendix D.

Original: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?

Original: [No] We will release the source codes.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Appendix C.

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]

Original: (b) Did you mention the license of the assets?

Original: [Yes] See Appendix D.3.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating? [N/A]
Revised: Acknowledgments

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content? [Yes] See Appendix D. If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: This work was conducted by Center for Applied Research in Artiﬁcial Intelligence(CARAI) grant funded by Defense Acquisition Program Administration(DAPA) and Agency for Defense Development(ADD)

Original: applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
Revised: (UD190031RD).

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]

Original: A Broader Impacts
Revised: A Broader Impacts

Original: Recently, ethical concerns about the fairness of deep-learning-based systems have been raised(Castelvecchi, 2020; Roussi, 2020; Noorden, 2020).
Revised: Recently, ethical concerns about the fairness of deep-learning-based systems have been raised (Castelvecchi, 2020; Roussi, 2020; Noorden, 2020).

Original: Especially, due to the intrinsic imbalance offacial datasets (Ricanek & Tesafaye, 2006; Zhang et al., 2017; Niu et al., 2016), most deep learningmethods on facial analysis (Wen et al., 2020; Or-El et al., 2020) have unwanted gender or racialbias.
Revised: Especially, due to the intrinsic imbalance of facial datasets (Ricanek & Tesafaye, 2006; Zhang et al., 2017; Niu et al., 2016), most deep learning methods on facial analysis (Wen et al., 2020; Or-El et al., 2020) have unwanted gender or racial bias.

Original: The proposed algorithm is not free from this bias either when trained on such datasets.
Revised: The proposed algorithm is not free from this bias either when trained on such datasets.

Original: Hence,the bias should be resolved before any practical usage.
Revised: Hence, the bias should be resolved before any practical usage.

Original: Also, even though the proposed algorithmdiscovers some subclasses by ranking instances, these results should never be misinterpreted in sucha way as to encourage any kind of discrimination.
Revised: Also, even though the proposed algorithm discovers some subclasses by ranking instances, these results should never be misinterpreted in such a way as to encourage any kind of discrimination.

Original: We recommend using the proposed algorithm forresearch only.
Revised: We recommend using the proposed algorithm for research only.

Original: B Derivation of L x ≺ y in (10)
Revised: B Derivation of L x ≺ y in (10)

Original: From the metric constraint in (7), if x ≺ y , it should be that d e ( h x , h y ) > γ , or equivalently
Revised: From the metric constraint in (7), if x ≺ y , it should be that d e ( h x , h y ) > γ , or equivalently

Original: [Equation]
Revised: [Equation]

Original: Note that reference points and instances are roughly sorted according to their corresponding ranks inthe embedding space by the order constraint in (5).
Revised: Note that reference points and instances are roughly sorted according to their corresponding ranks in the embedding space by the order constraint in (5).

Original: Therefore, for x ≺ y , h x tends to be closer toeach reference point r
Revised: Therefore, for x ≺ y , h x tends to be closer to each reference point r

Original: i , 0 ≤ i ≤ θ ( x ) , than h y is.
Revised: i , 0 ≤ i ≤ θ ( x ) , than h y is.

Original: In other words, x ≺ y , it is likely that
Revised: In other words, x ≺ y , it is likely that

Original: [Equation]
Revised: [Equation]

Original: However, to satisfy the constraint in (17), we impose a stricter upper bound on the differenced e ( r i , h x ) − d e ( r i , h y ) as follows.
Revised: However, to satisfy the constraint in (17), we impose a stricter upper bound on the difference d e ( r i , h x ) − d e ( r i , h y ) as follows.

Original: From the triangle inequality, we have
Revised: From the triangle inequality, we have

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: (19) which is equivalent to
Revised: which is equivalent to

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Therefore, if
Revised: Therefore, if

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: the constraint in (17) is satisﬁed.
Revised: the constraint in (17) is satisﬁed.

Original: This desirable condition in (21) is formulated into the ﬁrst sum in
Revised: This desirable condition in (21) is formulated into the ﬁrst sum in

Original: L x ≺ y in (10).
Revised: L x ≺ y in (10).

Original: Similarly, for each reference point r j , θ ( y ) ≤ [Equation] ≤ M − 1 , it is likely that
Revised: Similarly, for each reference point r j , θ ( y ) ≤ j ≤ M − 1 , it is likely that

Original: [Equation]
Revised: [Equation]

Original: However, to satisfy the metric constraint, it is desirable that
Revised: However, to satisfy the metric constraint, it is desirable that

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: which is formulated into the second sum in L x ≺ y in (10).
Revised: which is formulated into the second sum in L x ≺ y in (10).

Original: C Implementation Details 
Revised: C Implementation Details C.1 Network Architecture

Original: C.1 Network Architecture

Original: The structure of the encoder h is detailed in Table 8, where ‘ k h × k w - s - c Conv’ denotes the 2Dconvolution with kernel size k h × k w , stride s
Revised: The structure of the encoder h is detailed in Table 9, where ‘ k h × k w - s - c Conv’ denotes the 2D convolution with kernel size k h × k w , stride s

Original: [Table]
Revised: [Table]

Original: and c output channels.
Revised: and c output channels.

Original: Similarly, ‘ k h × k w - s MaxPool’and ‘ k h × k w - s AvgPool’ represent the 2D max pooling and 2D average pooling with a k h × k w kernelat stride s , respectively.
Revised: Similarly, ‘ k h × k w - s MaxPool’ and ‘ k h × k w - s AvgPool’ represent the 2D max pooling and 2D average pooling with a k h × k w kernel at stride s , respectively.

Original: Also, BN means batch normalization (Ioffe & Szegedy, 2015).
Revised: Also, BN means batch normalization (Ioffe & Szegedy, 2015).

Original: The encoderis based on the VGG16 network and takes a 224 × 224 × 3 image as input.
Revised: The encoder is based on the VGG16 network and takes a 224 × 224 × 3 image as input

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: .2 Embedding Space Visualization
Revised: C.2 Embedding Space Visualization

Original: In Figure 1, ResNet18 is employed as an encoder, but the output dimension of its last fully connectedlayer is reduced to 3 for the visualization.
Revised: In Figure 1, ResNet18 is employed as an encoder, but the output dimension of its last fully connected layer is reduced to 3 for the visualization.

Original: The encoder takes a 28 × 28 × 3 image as input.
Revised: The encoder takes a 28 × 28 × 3 image as input.

Original: We usethe same training setting in Section 4.1.
Revised: We use the same training setting in Section 4.1.

Original: Similarly, for the other 3D embedding space visualizations, including Figures 3 and 4, we modify theencoder in Table 8 to append a fully connected layer with output dimension 3.
Revised: Similarly, for the other 3D embedding space visualizations, including Figures 3 and 5, we modify the encoder in Table 9 to append a fully connected layer with output dimension 3.

Original: We train the modiﬁedencoder using the same setting in Section 4.1.
Revised: We train the modiﬁed encoder using the same setting in Section 4.1.

Original: C.3 Computing Environment
Revised: C.3 Computing Environment

Original: We do all experiments using PyTorch (Paszke et al., 2019) and an NVIDIA GeForce RTX 3090 GPU.
Revised: We do all experiments using PyTorch (Paszke et al., 2019) and an NVIDIA GeForce RTX 3090 GPU.

Original: C.4 Training Time
Revised: C.4 Training Time

Original: We train the encoder until the loss converges.
Revised: We train the encoder until the loss converges.

Original: Table 9 lists the training epochs and time for eachdataset.
Revised: Table 10 lists the training epochs and time for each dataset.

Original: [Table]
Revised: [Table]

Original: D More Experimental ResultsD.1 Hyper-Parameters
Revised: D More Experimental Results D.1 Hyper-Parameters

Original: The proposed GOL algorithm has three hyper-parameters; τ in (1), γ in (7), and k in (14).
Revised: The proposed GOL algorithm has three hyper-parameters; τ in (1), γ in (7), and k in (14).

Original: Unlessspeciﬁed otherwise, we set τ = 0 and γ = 0 .
Revised: Unless speciﬁed otherwise, we set τ = 0 and γ = 0 .

Original: Also, we ﬁx k = 50 and k = 16 for facialage estimation tasks and the others, respectively.
Revised: Also, we ﬁx k = 50 and k = 16 for facial age estimation tasks and the others, respectively.

Original: Let us describe how the hyper-parameters affectperformances.
Revised: Let us describe how the hyper-parameters affect performances.

Original: Hyper-parameter γ : Table 10 compares the MAE scores at different γ ’s on the MORPH II, CACD,and UTK datasets.
Revised: Hyper-parameter γ : Table 11 compares the MAE scores at different γ ’s on the MORPH II, CACD, and UTK datasets.

Original: In this test, τ = 0 and k = 50 .
Revised: In this test, τ = 0 and k = 50 .

Original: Note that γ is a margin to impose the minimumdistance between instances with a rank difference larger than τ in the embedding space.
Revised: Note that γ is a margin to impose the minimum distance between instances with a rank difference larger than τ in the embedding space.

Original: We see thatγ = 0 .
Revised: We see that γ = 0 .

Original: 05 performs well in general, so we set γ = 0 .
Revised: 05 performs well in general, so we set γ = 0 .

Original: as the default option.
Revised: as the default option.

Original: Only for UTK, we setγ = 0 .
Revised: Only for UTK, we set γ = 0 .

Original: 25 , even though GOL outperforms the state-of-the-art MWR-G (4.49) at the other γ levels aswell.
Revised: 25 , even though GOL outperforms the state-of-the-art MWR-G (4.49) at the other γ levels as well.

Original: [Table]
Revised: [Table]

Original: Hyper-parameter τ : Table 11 compares the MAE scores at different τ ’s on MORPH II, CACD,and UTK.
Revised: Hyper-parameter τ : Table 12 compares the MAE scores at different τ ’s on MORPH II, CACD, and UTK.

Original: In this test, γ = 0 . 05 and k = 50 .
Revised: In this test, γ = 0 . 05 and k = 50 .

Original: We see that τ = 0 provides decent results in general.
Revised: We see that τ = 0 provides decent results in general.

Original: We hence set the default τ = 0 .
Revised: We hence set the default τ = 0 .

Original: [Table]
Revised: [Table]

Original: -parameter k : Table 12 compares the MAE scores according to k .
Revised: Hyper-parameter k : Table 13 compares the MAE scores according to k .

Original: In each age estimationtask, there are more than 40 ranks, but there are no clear distinctive characteristics for each rank.
Revised: In each age estimation task, there are more than 40 ranks, but there are no clear distinctive characteristics for each rank.

Original: Therefore, we set a relatively large k = 50 as the default option.
Revised: Therefore, we set a relatively large k = 50 as the default option.

Original: However, note that the results arenot very sensitive as long as k ≥ 18 .
Revised: However, note that the results are not very sensitive as long as k ≥ 18 .

Original: [Table]
Revised: [Table]

Original: Next, Table 13 compares the MAE scores on different folds of the HCI dataset according to k .
Revised: Next, Table 14 compares the MAE scores on different folds of the HCI dataset according to k .

Original: The
Revised: The

Original: HCI and aesthetics datasets contain only ﬁve ranks, respectively.
Revised: HCI and aesthetics datasets contain only ﬁve ranks, respectively.

Original: Thus, we use a relatively smallk = 16 for these tasks.
Revised: Thus, we use a relatively small k = 16 for these tasks.

Original: D.2 Embedding Spaces
Revised: D.2 Embedding Spaces

Original: Table 14 compares the B2W and DRR scores on the test data of MORPH II, CACD, and Adience.
Revised: Table 15 compares the B2W and DRR scores on the test data of MORPH II, CACD, and Adience.

Original: GOL performs the best in all tests, as well as for the training data in Table 1.
Revised: GOL performs the best in all tests, as well as for the training data in Table 1.

Original: These results indicate that
Revised: These results indicate that

Original: GOL can arrange the rank sets effectively in the embedding space to reﬂect their ordinal relationshipseven for unseen test instances.
Revised: GOL can arrange the rank sets effectively in the embedding space to reﬂect their ordinal relationships even for unseen test instances.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: D.3 Rank Estimation – Facial Age Estimation
Revised: D.3 Rank Estimation – Facial Age Estimation

Original: Datasets: We provide facial age estimation results of GOL on four datasets: MORPH II (Ricanek &
Revised: Datasets: We provide facial age estimation results of GOL on four datasets: MORPH II (Ricanek &

Original: Tesafaye, 2006), CACD (Chen et al., 2015), UTK (Zhang et al., 2017), and Adience (Levi & Hassner,2015).
Revised: Tesafaye, 2006), CACD (Chen et al., 2015), UTK (Zhang et al., 2017), and Adience (Levi & Hassner, 2015).

Original: MORPH II has the Institutional Review Board approval.
Revised: MORPH II has the Institutional Review Board approval.

Original: The other datasets were made foracademic research purposes only.
Revised: The other datasets were made for academic research purposes only.

Original: Any images getting deletion requests from the original ownerswill be discarded from the datasets.
Revised: Any images getting deletion requests from the original owners will be discarded from the datasets.

Original: There are no name labels, except for CACD containing celebritynames.
Revised: There are no name labels, except for CACD containing celebrity names.

Original: We exploit the datasets only for the performance assessment of GOL.
Revised: We exploit the datasets only for the performance assessment of GOL.

Original: MORPH II (Ricanek & Tesafaye, 2006) is a popular dataset for facial age estimation, containingabout 55,000 facial images of 13,617 subjects in the age range [16, 77].
Revised: MORPH II (Ricanek & Tesafaye, 2006) is a popular dataset for facial age estimation, containing about 55,000 facial images of 13,617 subjects in the age range [16, 77].

Original: In each image, the genderand race labels are annotated.
Revised: In each image, the gender and race labels are annotated.

Original: Based on these labels, various evaluation protocols have been proposed.
Revised: Based on these labels, various evaluation protocols have been proposed.

Original: We employ the four evaluation settings A, B, C, and D (Lim et al., 2020; Lee & Kim, 2021; Shinet al., 2022).
Revised: We employ the four evaluation settings A, B, C, and D (Lim et al., 2020; Lee & Kim, 2021; Shin et al., 2022).

Original: • Setting A: 5,492 Caucasian images are randomly sampled and divided into training and testing sets with a ratio of 8:2.
Revised: • Setting A: 5,492 Caucasian images are randomly sampled and divided into training and testing sets with a ratio of 8:2.

Original: • Setting B: About 21K images of Caucasians and Africans are randomly chosen so that the ratio between Caucasians and Africans is 1:1 and that between females and males is 1:3.
Revised: • Setting B: About 21K images of Caucasians and Africans are randomly chosen so that the ratio between Caucasians and Africans is 1:1 and that between females and males is 1:3.

Original: Then, it is divided into three subsets (S1, S2, S3).
Revised: Then, it is divided into three subsets (S1, S2, S3).

Original: The training and testing are repeated twice — 1) training on S1, testing on S2+S3, and 2) training on S2, testing on S1+S3.
Revised: The training and testing are repeated twice — 1) training on S1, testing on S2+S3, and 2) training on S2, testing on S1+S3.

Original: • Setting C:
Revised: • Setting C:

Original: The entire dataset is randomly split into ﬁve folds, subject to the constraint that the same person should belong to only one fold, and the 5-fold cross-validation is performed.
Revised: The entire dataset is randomly split into ﬁve folds, subject to the constraint that the same person should belong to only one fold, and the 5-fold cross-validation is performed.

Original: • Setting D: The whole dataset is randomly divided into ﬁve folds without any constraint, andthe 5-fold cross-validation is performed.
Revised: • Setting D: The whole dataset is randomly divided into ﬁve folds without any constraint, and the 5-fold cross-validation is performed.

Original: CACD (Chen et al., 2015) provides 160k images from 2,000 celebrities.
Revised: CACD (Chen et al., 2015) provides 160k images from 2,000 celebrities.

Original: It is split into three subsetsby celebrities: 1,800 for training, 80 for validation, and 120 for testing.
Revised: It is split into three subsets by celebrities: 1,800 for training, 80 for validation, and 120 for testing.

Original: We provide two results bytraining on the train set and on the validation set, respectively, as done in (Rothe et al., 2018; Shenet al., 2018; Shin et al., 2022).
Revised: We provide two results by training on the train set and on the validation set, respectively, as done in (Rothe et al., 2018; Shen et al., 2018; Shin et al., 2022).

Original: The age range is [14, 62].
Revised: The age range is [14, 62].

Original: UTK (Zhang et al., 2017) consists of 20,000 facial images in a wide age range [0, 116].
Revised: UTK (Zhang et al., 2017) consists of 20,000 facial images in a wide age range [0, 116].

Original: We adopt theevaluation protocol in (Gustafsson et al., 2020; Berg et al., 2021).
Revised: We adopt the evaluation protocol in (Gustafsson et al., 2020; Berg et al., 2021).

Original: Adience (Levi & Hassner, 2015) is used for age group estimation.
Revised: Adience (Levi & Hassner, 2015) is used for age group estimation.

Original: There are 26,580 facial imagesfrom 2,284 subjects, which are grouped into 8 ordinal classes: 0-2, 4-6, 8-13, 15-20, 25-32, 38-43, 4853, and over 60-year-olds.
Revised: There are 26,580 facial images from 2,284 subjects, which are grouped into 8 ordinal classes: 0-2, 4-6, 8-13, 15-20, 25-32, 38-43, 4853, and over 60-year-olds.

Original: We employ the 5-fold subject-exclusive (SE) cross-validation evaluationsetting in (Liu et al., 2018, 2019;
Revised: We employ the 5-fold subject-exclusive (SE) cross-validation evaluation setting in (Liu et al., 2018, 2019;

Original: Diaz & Marathe, 2019; Li et al., 2021).
Revised: Diaz & Marathe, 2019; Li et al., 2021).

Original: More comparison: Table 15 provides comparison results with more conventional algorithms on
Revised: More comparison: Table 16 provides comparison results with more conventional algorithms on

Original: MORPH II.
Revised: MORPH II.

Original: It is an extended version of Table 2.
Revised: It is an extended version of Table 2.

Original: [Table]
Revised: [Table]

Original: D.4 Rank Estimation – HCI Classiﬁcation
Revised: D.4 Rank Estimation – HCI Classiﬁcation

Original: Table 16 is an extended version of the HCI results in Table
Revised: Table 17 is an extended version of the HCI results in Table

Original: [Table] [Table]
Revised: [Table] [Table]

Original: D.5 Rank Estimation – Aesthetic Score Regression
Revised: D.5 Rank Estimation – Aesthetic Score Regression

Original: The aesthetics dataset (Schifanella et al., 2015) provides 15,687 image URLs on Flickr, where 13,929images are available but the others are lost.
Revised: The aesthetics dataset (Schifanella et al., 2015) provides 15,687 image URLs on Flickr, where 13, images are available but the others are lost.

Original: Each image is annotated with a 5-scale aesthetic score.
Revised: Each image is annotated with a 5-scale aesthetic score.

Original: We adopt the 5-fold cross-validation.
Revised: We adopt the 5-fold cross-validation.

Original: For training, we set the learning rate of all parameters, includingreference points, to 10 − 6 .
Revised: For training, we set the learning rate of all parameters, including reference points, to 10 − 6 .

Original: The other settings are the same as in Section 4.1.
Revised: The other settings are the same as in Section 4.1.

Original: Table 17 is an extendedversion of Table 5.
Revised: Table 18 is an extended version of Table 5.

Original: [Table]
Revised: [Table]

Original: D.6 Analysis
Revised: D.6 Analysis

Original: Ablation study: Figure 5 shows the embedding spaces for the ablated methods in Table 6.

Original: The toprow visualizes the reduced 3D embedding spaces, as described in Appendix C.2, while the bottomrow does the original 512D embedding spaces via t-SNE (Maaten & Hinton, 2008).

Original: In (a), only

Original: L center is used, so the 3D embedding space almost collapses to a single point.

Original: It is because a trivialsolution to minimize L center can be obtained by merging all instances and all reference points into asingle point.

Original: In (b), instances are directionally aligned, but adjacent rank sets overlap one another.

Original: In (c) and (d), the 3D embedding spaces seem similar.

Original: However, GOL yields more clearly orderedinstances in the t-SNE visualization of the original embedding space than ‘ L center + L metric ’ does.

Original: [Figure] More visualizations: Figure 6 visualizes the embedding spaces of the proposed algorithm on the
Revised: More visualizations: Figure 6 visualizes the embedding spaces of the proposed algorithm on the

Original: HCI and aesthetics datasets.
Revised: HCI and aesthetics datasets.

Original: For each dataset, the  right subﬁgure is the t-SNE visualization (Maaten& Hinton, 2008).
Revised: For each dataset, the left subﬁgure is reduced 3D embedding space and the right subﬁgure is the t-SNE visualization (Maaten & Hinton, 2008) of original 512D embedding space. [Figure]

Original: [Figure]
Revised: [Figure]

Original: Young, median, and old instances: Figure 7 shows sampled instances from each rank set X i of the
Revised: Young, median, and old instances: Figure 7 shows sampled instances from each rank set X i of the

Original: Adience dataset.
Revised: Adience dataset.

Original: Speciﬁcally, Figures 7 (a) and (c) list the instances in each X i that are the farthestfrom the reference point
Revised: Speciﬁcally, Figures 7 (a) and (c) list the instances in each X i that are the farthest from the reference point

Original: r i in the backward and forward directions, respectively.
Revised: r i in the backward and forward directions, respectively.

Original: Also, Figures 7 (b)shows the closest instance to r i among all instances in X i .
Revised: Also, Figures 7 (b) shows the closest instance to r i among all instances in X i .

Original: In Adience, each rank set
Revised: In Adience, each rank set

Original: X i consists ofinstances in an age range, marked at the top of the ﬁgure.
Revised: X i consists of instances in an age range, marked at the top of the ﬁgure.

Original: Thus, from each rank set, young, median,and old instances tend to be selected in (a), (b), and (c), respectively, which means that the instancesare well sorted in the embedding space although their exact ages are not annotated.
Revised: Thus, from each rank set, young, median, and old instances tend to be selected in (a), (b), and (c), respectively, which means that the instances are well sorted in the embedding space although their exact ages are not annotated.

Original: [Figure]
Revised: [Figure]

Original: Alternatives to L x ≺ y : Table 18 compares the rank estimation results of alternative loss terms for

Original: L x ≺ y in (10).

Original: Method I directly increases d e ( h x , h y ) to make it larger than γ .

Original: Method II uses onlytwo reference points r θ ( x ) and r θ ( y ) .

Original: In other words, unlike method III ( i.e. GOL), it does not employreference points r i , 0 ≤ i < θ ( x ) , and θ ( y ) <

Original: r j ≤ M − 1 .

Original: For each method, L x (cid:31) y is also modiﬁedaccordingly.

Original: Method I degrades the performances signiﬁcantly, as h x and h y move in arbitrary directions.

Original: Also,

Original: GOL performs better than II because attraction and repulsion with many reference points facilitatethe positioning of instances in the embedding space, as well as the reference points themselves

Original: [Table] [Table] Testing time: Table 19 lists the testing time of GOL according to the number of samples for thek -NN search on the MORPH II and CACD datasets.
Revised: Testing time: Table 19 lists the testing time of GOL according to the number of samples for the k -NN search on the MORPH II and CACD datasets.

Original: Note that the distances to all samples can becomputed efﬁciently in a parallel manner.
Revised: Note that the distances to all samples can be computed efﬁciently in a parallel manner.

Original: Therefore, the proposed algorithm performs fast even with44,000 samples.
Revised: Therefore, the proposed algorithm performs fast even with 44,000 samples.

Original: [Table]
Revised: [Table]

Original: Limitations: Figure 

Revised: Limitations: Figure

Original: shows some success and failure cases of GOL on the CACD dataset.
Revised: shows some success and failure cases of GOL on the CACD dataset.

Original: In
Revised: In

Original: Figure 8 (a), the ages of test instances are predicted accurately with absolute errors less than 4.
Revised: Figure 8 (a), the ages of test instances are predicted accurately with absolute errors less than 4.

Original: However, in Figure 8 (b), GOL yields poor estimates on some hard cases, in which various factors,such as poor illumination, overexposure, and low-quality photographs, hinder accurate estimation.
Revised: However, in Figure 8 (b), GOL yields poor estimates on some hard cases, in which various factors, such as poor illumination, overexposure, and low-quality photographs, hinder accurate estimation.

Original: Also, since GOL predicts a rank based on the k -NN search, it suffers when there are insufﬁcienttraining instances.
Revised: Also, since GOL predicts a rank based on the k -NN search, it suffers when there are insufﬁcient training instances.

Original: For example, GOL yields relatively poor results on MORPH II setting B, whichconsists of 7,000 training samples and 14,000 test samples.
Revised: For example, GOL yields relatively poor results on MORPH II setting B, which consists of 7,000 training samples and 14,000 test samples.

Original: Similarly, GOL tends to yield lessaccurate estimates on minority classes, such as toddlers and elders, since most age estimation datasetscontain fewer instances in such classes.
Revised: Similarly, GOL tends to yield less accurate estimates on minority classes, such as toddlers and elders, since most age estimation datasets contain fewer instances in such classes.

Original: These limitations of GOL might be alleviated by generatingpseudo-references in the embedding space, which we leave as future work in this paper.
Revised: These limitations of GOL might be alleviated by generating pseudo-references in the embedding space, which we leave as future work in this paper.

Original: Test
Revised: Test

Original: Nearest neighbors 22 (22) 23   22  32 (32) 33   26  51 (54) 50   51  38 (39) 38   40  39 (40) 35   40  (a) 26 (57) 30   23  28 (55) 30   28  45 (16) 46   46   40   32 33 35 (60) 48   44 45 45 (16)
Revised: Nearest neighbors 22 (22) 23   22  32 (32) 33   26  51 (54) 50   51  38 (39) 38   40  39 (40) 35   40  (a) 26 (57) 30   23  28 (55) 30   28  45 (16) 46   46   40   32 33 35 (60) 48   44 45 45 (16)

Original: Test
Revised: Test

Original: [Figure]
Revised: [Figure]