Original: O PTIMAL T RANSPORT FOR L ONG -T AILED R ECOGNITION WITH L EARNABLE C OST M ATRIX
Revised: O PTIMAL T RANSPORT FOR L ONG -T AILED R ECOGNITION WITH L EARNABLE C OST M ATRIX

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: It is attracting attention to the long-tailed recognition problem, a burning issue that has become very popular recently.
Revised: It is attracting attention to the long-tailed recognition problem, a burning issue that has become very popular recently.

Original: Distinctive from conventional recognition is that it posits that the allocation of the training set is supremely distorted.
Revised: Distinctive from conventional recognition is that it posits that the allocation of the training set is supremely distorted.

Original: Predictably, it will pose challenges to the generalisation behaviour of the model.
Revised: Predictably, it will pose challenges to the generalisation behaviour of the model.

Original: Approaches to these challenges revolve into two groups: ﬁrstly, training-aware methods, with the aim of enhancing the generalisability of the model by exploiting its potential in the training period; and secondly, post-hoc correction, liberally coupled with trainingaware methods, which is intended to reﬁne the predictions to the extent possible in the post-processing stage, offering the advantages of simplicity and effectiveness.
Revised: Approaches to these challenges revolve into two groups: ﬁrstly, training-aware methods, with the aim of enhancing the generalisability of the model by exploiting its potential in the training period; and secondly, post-hoc correction, liberally coupled with trainingaware methods, which is intended to reﬁne the predictions to the extent possible in the post-processing stage, offering the advantages of simplicity and effectiveness.

Original: This paper introduces an alternative direction to do the post-hoc correction, which goes beyond the statistical methods.
Revised: This paper introduces an alternative direction to do the post-hoc correction, which goes beyond the statistical methods.

Original: Mathematically, we approach this issue from the perspective of optimal transport (OT), yet, choosing the exact cost matrix when applying OT is challenging and requires expert knowledge of various tasks.
Revised: Mathematically, we approach this issue from the perspective of optimal transport (OT), yet, choosing the exact cost matrix when applying OT is challenging and requires expert knowledge of various tasks.

Original: To overcome this limitation, we propose to employ linear mapping to learn the cost matrix without necessary conﬁgurations adaptively.
Revised: To overcome this limitation, we propose to employ linear mapping to learn the cost matrix without necessary conﬁgurations adaptively.

Original: Testing our methods in practice, along with high efﬁciency and excellent performance, our method surpasses all previous methods and has the best performance to date.
Revised: Testing our methods in practice, along with high efﬁciency and excellent performance, our method surpasses all previous methods and has the best performance to date.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Classiﬁcation problems in the real world are generally challenged by the long-tailed label distribution, i.e., having a small number of samples for a majority of labels, and a dominant number of samples for a minority of labels (Van Horn & Perona, 2017; Buda et al., 2018; Liu et al., 2019) on the training data set.
Revised: Classiﬁcation problems in the real world are generally challenged by the long-tailed label distribution, i.e., having a small number of samples for a majority of labels, and a dominant number of samples for a minority of labels (Van Horn & Perona, 2017; Buda et al., 2018; Liu et al., 2019).

Original: It is also known as imbalanced recognition, which has been widely studied in the past decades (Cardie & Howe, 1997; Chawla et al., 2002; Qiao & Liu, 2009; Cui et al., 2019).
Revised: It is also known as imbalanced recognition, which has been widely studied in the past decades (Cardie & Howe, 1997; Chawla et al., 2002; Qiao & Liu, 2009; Cui et al., 2019).

Original: These distribution biases pose a signiﬁcant challenge to predictive modeling; conceivably, models often suffer from poor generalisation and undesirable estimation bias (Cao et al., 2019; Kang et al., 2019;
Revised: These distribution biases pose a signiﬁcant challenge to predictive modeling; conceivably, models often suffer from poor generalisation and undesirable estimation bias (Cao et al., 2019; Kang et al., 2019;

Original: Zhou et al., 2020).
Revised: Zhou et al., 2020).

Original: Recently, a renewed interest in the problem of long-tail recognition has emerged following the context of neural networks, as numerous publications in the literature endeavour to resolve the problem albeit in different ways including decouple (Kang et al., 2019), meta-learning (Ren et al., 2020;
Revised: Recently, a renewed interest in the problem of long-tail recognition has emerged following the context of neural networks, as numerous publications in the literature endeavour to resolve the problem albeit in different ways including decouple (Kang et al., 2019), meta-learning (Ren et al., 2020;

Original: Wang et al., 2020a; Li et al., 2021), post-hoc correction (Tang et al., 2020;
Revised: Wang et al., 2020a; Li et al., 2021), post-hoc correction (Tang et al., 2020;

Original: Hong et al., 2020), etc (Liu et al., 2019; Cao et al., 2019; Tang et al., 2020).
Revised: Hong et al., 2020), etc (Liu et al., 2019; Cao et al., 2019; Tang et al., 2020).

Original: One of the representative methods of post-hoc correction, Logit Adjustment Menon et al.
Revised: One of the representative methods of post-hoc correction, Logit Adjustment Menon et al.

Original: (2020), provides a statistical correction to the prediction, receiving widespread attention for its simplicity and validity, but the downside is that its optimisation is conducted on individual samples, without any indication that rectiﬁed marginal distribution satisﬁes the desired distribution we are striving for.
Revised: (2020), provides a statistical correction to the prediction, receiving widespread attention for its simplicity and validity. But the downside is that its optimisation is conducted on individual samples, without any indication that rectiﬁed marginal distribution satisﬁes the desired distribution we are striving for.

Original: Figuring out how to solve the above problem, our explicit modeling of the problem mathematically turns into an equational constraint, meanwhile to minimise the difference between reﬁned distribution and the original one, this minimisation is motivated upon the inner-product similarity.
Revised: Figuring out exact ﬂaws of Logit Adjustment, our explicit modeling of the problem mathematically turns into an equational constraint, meanwhile to minimise the difference between reﬁned distribution and the original one, this minimisation is motivated upon the inner-product similarity.

Original: A little further, the resulting problem can be linked to OT.
Revised: A little further, the resulting problem can be linked to OT.

Original: Drawing on this linkage, we develop it further by proposing a linear mapping to automatically learn cost matrix, thereby circumventing the requirement for expert knowledge to conﬁgure this matrix.
Revised: Drawing on this linkage, we develop it further by proposing a linear mapping to automatically learn cost matrix, thereby circumventing the requirement for expert knowledge to conﬁgure this matrix.

Original: In short, we contribute as follows:
Revised: In short, we contribute as follows:

Original: • We propose an alternative direction based on convex optimisation to do post-hoc correction, which goes beyond previous direction from the statistical view.
Revised: • We propose an alternative direction based on convex optimisation to do post-hoc correction, which goes beyond previous direction from the statistical view.

Original: • Imposing marginal distributions to align ideal marginal distributions, we derive an optimisation problem tied to the OT that is solved using Sinkhorn, a highly efﬁcient algorithm.
Revised: • Imposing marginal distributions to align ideal marginal distributions, we derive an optimisation problem tied to the OT that is solved using Sinkhorn, a highly efﬁcient algorithm.

Original: More further, for better learning of the cost matrix, we present a linear mapping enabling very elegant learning with one-layer neural network.
Revised: More further, for better learning of the cost matrix, we present a linear mapping enabling very elegant learning with one-layer neural network.

Original: • The experimental evidence shows the high efﬁciency of the algorithm, and the best performance achieved on three benchmarks, also supports our claim that addressing the post-hoc problem via OT is helpful and effective.
Revised: • The experimental evidence shows the high efﬁciency of the algorithm, and the best performance achieved on three benchmarks. It also supports our claim that addressing the post-hoc problem via OT is helpful and effective.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: In this section, we begin by giving a notational deﬁnition, followed by an introduction to the longtailed recognition problem, and ﬁnally, we brieﬂy review the OT, and give a quick recap of Logit
Revised: In this section, we begin by giving a notational deﬁnition, followed by an introduction to the longtailed recognition problem, and ﬁnally, we brieﬂy review the OT, and give a quick recap of Logit

Original: Adjustment Menon et al.
Revised: Adjustment Menon et al.

Original: Notations : In what follows, for two matrices X , Y ∈ R N × K , we denote (cid:104) X , Y (cid:105) =
Revised: Notations : In what follows, for two matrices X , Y ∈ R N × K , we denote (cid:104) X , Y (cid:105) =

Original: (cid:80) Nn =1 (cid:80) 
Revised: (cid:80) N n =1 (cid:80) K k

Original: Kk =1 X
Revised: =1 X

Original: nk Y nk as the Frobenius dot-product.
Revised: nk Y nk as the Frobenius dot-product.

Original: δ ( · ) stands for the Dirac function, p ( · ) represents the probability distribution.
Revised: δ ( · ) stands for the Dirac function, p ( · ) represents the probability distribution.

Original: U ( r , c ) = { P ∈ R N × K + | P 1 K = r , P (cid:124) 1 N = c } , where 1 N and 1 K are N -dimension and K -dimension vector whose elements are all 1.
Revised: U ( r , c ) = { P ∈ R N × K + | P 1 K = r , P (cid:124) 1 N = c } , where 1 N and 1 K are N -dimension and K -dimension vector whose elements are all 1.

Original: r and c refer to the vectors of size N and K respectively, U ( r , c ) include all matrices with row and column sums r and c respectively
Revised: r and c refer to the vectors of size N and K respectively, U ( r , c ) include all matrices with row and column sums r and c respectively.

Original: 2. P ROBLEM F ORMULATION
Revised: 2. P ROBLEM F ORMULATION

Original: Having a collection of training samples { ( x sn , y sn ) } N s n =1 , validation samples { ( x vn , y vn ) } N v n =1 and test samples { ( x tn , y n ) t } N t n =1 for classiﬁcation with K labels and input x ∈ R d , long-tailed recognition assumes that the class-prior distribution for training data p ( y s ) is different from that for validation data p ( y v ) and test data p ( y t ) .
Revised: Having a collection of training samples { ( x sn , y sn ) } N s n =1 , validation samples { ( x vn , y vn ) } N v n =1 and test samples { ( x tn , y n ) t } N t n =1 for classiﬁcation with K labels and input x ∈ R d , long-tailed recognition assumes that the class-prior distribution for training data p ( y s ) is different from that for validation data p ( y v ) and test data p ( y t ) .

Original: Speciﬁcally, long-tailed recognition means the distribution p ( y s ) is highly skewed,  some classes have the dominant number of samples, while tailed labels own a very small number of samples .
Revised: Speciﬁcally, long-tailed recognition means the distribution p ( y s ) is highly skewed, that is, some classes have the dominant number of samples, while tailed labels own a very small number of samples .

Original: We can use imbalance ratio to measure the skewness in training data set, which can be deﬁned as R = N smax N smin , where N smax and N smin denote the largest and smallest number of samples in the training data set, respectively.
Revised: We can use imbalance ratio to measure the skewness in training data set, which can be deﬁned as R = N smax N smin , where N smax and N smin denote the largest and smallest number of samples in the training data set, respectively.

Original: In this paper, we assume that the marginal distribution of the test set is known, we consider it as an implicit prior knowledge to be applied .
Revised: In this paper, we assume that the marginal distribution of the test set is known, we consider it as an implicit prior knowledge to be applied .

Original: Stepping back, even if we do not know the marginal distribution of the test dataset in advance.
Revised: Stepping back, even if we do not know the marginal distribution of the test dataset in advance.

Original: There are still ways to estimate the marginal distribution of the test dataset relatively precisely, such as methods in Hendrycks et al.
Revised: There are still ways to estimate the marginal distribution of the test dataset relatively precisely, such as methods in Hendrycks et al.

Original: ; Azizzadenesheli et al.
Revised: ; Azizzadenesheli et al.

Original: Obviously, most models trained on imbalanced training data set would suffer from extremely limited generalisation ability.
Revised: Obviously, most models trained on imbalanced training data set would suffer from extremely limited generalisation ability.

Original: Hence the ultimate goal is to learn a model that minimises the empirical risk, which can be formulated as:
Revised: Hence the ultimate goal is to learn a model that minimises the empirical risk, which can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where Φ( x sn ) ∈ R K denotes logits with associated sample, Φ( · ) :
Revised: where Φ( x sn ) ∈ R K denotes logits with associated sample, Φ( · ) :

Original: R d → R K represents the mapping via neural networks, L stands for the loss function, typically cross entropy loss function for classiﬁcation problem.
Revised: R d → R K represents the mapping via neural networks, L stands for the loss function, typically cross entropy loss function for classiﬁcation problem.

Original: 2. R EMINDERS ON O PTIMAL T RANSPORT
Revised: 2. R EMINDERS ON O PTIMAL T RANSPORT

Original: OT is used to calculate the cost of transporting one probability measure to another.
Revised: OT is used to calculate the cost of transporting one probability measure to another.

Original: We next present a brief introduction to OT with the aim of helping us better view the long-tailed problem from an OT perspective.
Revised: We next present a brief introduction to OT with the aim of helping us better view the long-tailed problem from an OT perspective.

Original: Consider probability measures r and c of the two random variables X and Y respectively, when the cost function C ( X, Y ) :
Revised: Consider probability measures r and c of the two random variables X and Y respectively, when the cost function C ( X, Y ) :

Original: X × Y → R + is deﬁned to denote the cost of transporting X to Y , the OT distance can be deﬁned as :
Revised: X × Y → R + is deﬁned to denote the cost of transporting X to Y , the OT distance can be deﬁned as :

Original: [Equation]
Revised: [Equation]

Original: where Π ( r, c ) = (cid:110)(cid:82) Y π ( x
Revised: where Π ( r, c ) = (cid:110)(cid:82) Y π ( x

Original: , y ) d y = r, (cid:82)
Revised: , y ) d y = r, (cid:82)

Original: X π ( x , y )
Revised: X π ( x , y )

Original: d x = c (cid:111) is the joint probability measure with r and c .
Revised: d x = c (cid:111) is the joint probability measure with r and c .

Original: When we extend the above continuous case to the discrete situation, we consider following discrete distributions:
Revised: When we extend the above continuous case to the discrete situation, we consider following discrete distributions:

Original: [Equation]
Revised: [Equation]

Original: where p i ( x i ) and p i ( y j ) represent the probability mass to the sample x i and y j respectively.
Revised: where p i ( x i ) and p i ( y j ) represent the probability mass to the sample x i and y j respectively.

Original: In this context, OT distance can be expressed as:
Revised: In this context, OT distance can be expressed as:

Original: [Equation]
Revised: [Equation]

Original: where M stands for the cost matrix constructed by M ij =
Revised: where M stands for the cost matrix constructed by M ij =

Original: C ( x i , y j ) .
Revised: C ( x i , y j ) .

Original: The goal of OT is to ﬁnd a transportation matrix P that minimizes the distance d M ( r , c )
Revised: The goal of OT is to ﬁnd a transportation matrix P that minimizes the distance d M ( r , c )

Original: As we can see, OT is a distance measure between two probability distributions under some cost matrix (Villani, 2008).
Revised: As we can see, OT is a distance measure between two probability distributions under some cost matrix (Villani, 2008).

Original: However, when we use network simplex or interior point methods to solve the above optimization problem, it often comes at the cost of heavy computational demands.
Revised: However, when we use network simplex or interior point methods to solve the above optimisation problem, it often comes at the cost of heavy computational demands.

Original: To tackle this issue, OT with entropy constraint is proposed to allow the optimisation at small computational cost in sufﬁcient smoothness (Cuturi, 2013).
Revised: To tackle this issue, OT with entropy constraint is proposed to allow the optimisation at small computational cost in sufﬁcient smoothness (Cuturi, 2013).

Original: By adding a Lagrangian multiplier to the entropy constraint, the new formulation can be deﬁned as follows:
Revised: By adding a Lagrangian multiplier to the entropy constraint, the new formulation can be deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: where λ ∈ [0 , + ∞ ] ,
Revised: where λ ∈ [0 , + ∞ ] ,

Original: [Equation]
Revised: [Equation]

Original:  ) is also known as dual-Sinkhorn divergence, besides, it can be calculated with matrix scaling algorithms for cheaper computational demand.
Revised: c ) is also known as dual-Sinkhorn divergence, besides, it can be calculated with matrix scaling algorithms for cheaper computational demand.

Original: The following lemma guarantees the convergence and uniqueness of the solution of the algorithm.
Revised: The following lemma guarantees the convergence and uniqueness of the solution of the algorithm.

Original: Lemma 1 For λ > 0 , the solution P λ is unique and has the form P λ = diag ( u ) K diag ( v ) , where u and v are two non-negative vectors uniquely deﬁned up to a multiplicative factor and K = e − M /λ is the element-wise exponential of − M /λ .
Revised: Lemma 1 For λ > 0 , the solution P λ is unique and has the form P λ = diag ( u ) K diag ( v ) , where u and v are two non-negative vectors uniquely deﬁned up to a multiplicative factor and K = e − M /λ is the element-wise exponential of − M /λ .

Original: The above lemma states the uniqueness of P λ
Revised: The above lemma states the uniqueness of P λ

Original: (Sinkhorn, 1974), and P λ can be efﬁciently computed via Sinkhorn’s ﬁxed point iteration u , v ← r ./
Revised: (Sinkhorn, 1974), and P λ can be efﬁciently computed via Sinkhorn’s ﬁxed point iteration u , v ← r ./

Original: Kv , c ./ K (cid:124) u .
Revised: Kv , c ./ K (cid:124) u .

Original: 2. A Q UICK R ECAP OF L OGIT A DJUSTMENT
Revised: 2. A Q UICK R ECAP OF L OGIT A DJUSTMENT

Original: We give a brief introduction to Logit Adjustment (Menon et al., 2020; Hong et al., 2020).
Revised: We give a brief introduction to Logit Adjustment (Menon et al., 2020; Hong et al., 2020).

Original: For the model Φ( · ) , it is trained by the standard cross-entropy loss function on imbalanced training data set, and evaluated on test data.
Revised: For the model Φ( · ) , it is trained by the standard cross-entropy loss function on imbalanced training data set, and evaluated on test data.

Original: In this algorithm, the test logit is adjusted as follows:
Revised: In this algorithm, the test logit is adjusted as follows:

Original: [Equation]
Revised: [Equation]

Original: This simple procedure is derived from the Bayes optimal rule.
Revised: This simple procedure is derived from the Bayes optimal rule.

Original: It is apparent that Logit Adjustment involves a post hoc correction on an individual sample, which does not necessarily guarantee that the marginal distribution of the whole dataset matches the desired distribution.
Revised: It is apparent that Logit Adjustment involves a post hoc correction on an individual sample, which does not necessarily guarantee that the marginal distribution of the whole dataset matches the desired distribution.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: The ﬁrst part of this section explores post-hoc correction from an OT perspective, proceeds to the automatic learning of the cost matrix via linear mapping.
Revised: The ﬁrst part of this section explores post-hoc correction from an OT perspective, proceeds to the automatic learning of the cost matrix via linear mapping.

Original: Lastly, we demonstrate how it can be achieved simply with one-layer neural network.
Revised: Lastly, we demonstrate how it can be achieved simply with one-layer neural network.

Original: 3. P OST - HOC C ORRECTION F ORMALISED FROM A N OT P ERSPECTIVE
Revised: 3. P OST - HOC C ORRECTION F ORMALISED FROM A N OT P ERSPECTIVE

Original: Since Logit Adjustment applies adjustment at the individual sample level.
Revised: Since Logit Adjustment applies adjustment at the individual sample level.

Original: It doesn’t assure that the marginal distribution of the overall data set fulﬁls our desired distribution.
Revised: It doesn’t assure that the marginal distribution of the overall data set fulﬁls our desired distribution.

Original: In this respect, we clearly put the constraint into an equation:
Revised: In this respect, we clearly put the constraint into an equation:

Original: [Equation]
Revised: [Equation]

Original: where Y ∈ R N × K indicates the reﬁned prediction value in matrix form, µ represents the expected distribution on the test set.
Revised: where Y ∈ R N × K indicates the reﬁned prediction value in matrix form, µ represents the expected distribution on the test set.

Original: Alternatively, it is desirable to preserve another characteristic of Y , namely, remaining almost as similar to the original prediction as possible.
Revised: Alternatively, it is desirable to preserve another characteristic of Y , namely, remaining almost as similar to the original prediction as possible.

Original: We consider inner-product based similarity to measure the similarity, which is a straightforward yet useful similarity measure.
Revised: We consider inner-product based similarity to measure this, which is a straightforward yet useful similarity measure.

Original: In mathematical terms, this can be expressed as follows:
Revised: In mathematical terms, this can be expressed as follows:

Original: [Equation]
Revised: [Equation]

Original: where ˆ Z represents the original prediction in matrix form, C ( · ) denotes toperform some transformation to ˆ Z , it can be some simple function, like Logarithmic function log( z ) , exponential function z α .
Revised: where ˆ Z represents the original prediction in matrix form, C ( · ) denotes to some transformation to ˆ Z , it can be some simple function, like Logarithmic function log( z ) , exponential function z α .

Original: Here we select − log( · ) as the cost function.
Revised: Here we select − log( · ) as the cost function.

Original: This choice was driven by the requirement that the cost matrix must be positive deﬁnite, whereas the transformation of the original prediction by − log( · ) satisﬁes this condition, in addition, as log likelihood represents the local probability density of the associated samples, it can also be used to substitute ˆ Z for the similarity approximation.
Revised: This choice was driven by the requirement that the cost matrix must be positive deﬁnite, whereas the transformation of the original prediction by − log( · ) satisﬁes this condition. In addition, as log likelihood represents the local probability density of the associated samples, it can also be used to substitute ˆ Z for the similarity approximation.

Original: In brief, the resulting numerical form can be put in formal terms as follows:
Revised: In brief, the resulting numerical form can be put in formal terms as follows:

Revised: [Equation]

Revised: subject

Original: [Equation]
Revised: [Equation]

Original: Extra constraint on Y are imposed simply cos the tuned estimation has to fulﬁl the basic probabilistic requirement that its sum is one.
Revised: Extra constraint on Y is imposed simply cos the tuned estimation has to fulﬁl the basic probabilistic requirement that its sum is one.

Original: Comparing Eq.
Revised: Comparing Eq.

Original: (9-10) with Eq.
Revised: (9-10) with Eq.

Original: (4), we can see that if we substitute
Revised: (4), we can see that if we substitute

Original: P with Y  and substitute r and c with 1 N and µ respectively, we ﬁnd that the above optimization problem is actually a special case of OT.
Revised: P with Y , and substitute r and c with 1 N and µ respectively, we ﬁnd that the above optimisation problem is actually a special case of OT.

Original: In preliminaries, the entropy regularised OT  is introduced (EOT), by adding entropy regularisation to
Revised: In preliminaries, the entropy regularised OT (EOT) is introduced. By adding entropy regularisation to

Original: OT, the given equation can be solved efﬁciently by Sinkhorn algorithm.
Revised: OT, the given equation can be solved efﬁciently by Sinkhorn algorithm.

Original: Speciﬁcally, the optimisation problem is:
Revised: Speciﬁcally, the optimisation problem is:

Original: [Equation]
Revised: [Equation]

Original: 11) subject to
Revised: (11) subject to

Revised: Y

Original: [Equation]
Revised: [Equation]

Original: The associated algorithmic ﬂow  is outlined in detail in Algorithm 1.
Revised: The associated algorithmic ﬂow for solving Eq. (11) is outlined in detail in Algorithm 1.

Original: Algorithm 1:
Revised: Algorithm 1:

Original: Solve OT-related algorithm efﬁciently in the post-hoc correction via Sinkhorn
Revised: Solve OT-related algorithm efﬁciently in the post-hoc correction via Sinkhorn

Original: Algorithm.
Revised: Algorithm.

Revised: iteration

Revised: current

Original: Input: Cost matrix M , trade-off parameter λ , max number of iterations N T , iteration number t , error threshold (cid:15) , current error δ , two probability vectors r  and c , |·| denotes the vector norm.
Revised: column sums r 1 N and = µ , |· denotes the vector norm

Original: Result: Reﬁned predictions Y1 Initialise
Revised: Result: Reﬁned predictions Y

Original: 2 while δ Assigning λ with 1, it is observed that we equate our objective function to the KL divergence, thus illustrating the extensive nature of our approach.
Revised: Assigning λ with 1, it is observed that we equate our objective function to the KL divergence, thus illustrating the extensive nature of our approach.

Original: DARP (Kim et al., 2020a) has previously applied it to long-tailed semi-supervised classiﬁcation.
Revised: DARP (Kim et al., 2020a) has previously applied it to long-tailed semi-supervised classiﬁcation.

Original: Remark We would like to illustrate the non-applicable scenarios of our method.
Revised: Remark We would like to illustrate the non-applicable scenarios of our method.

Original: Firstly, our method requires a large number of samples for evaluation.
Revised: Firstly, our method requires a large number of samples for evaluation.

Original: This is because if the batch size is small, we can not guarantee that the desired marginal distribution can be satisﬁed within the batch.
Revised: This is because if the batch size is small, we can not guarantee that the desired marginal distribution can be satisﬁed within the batch.

Original: In some online scenarios, the sample-wise correction method is more suitable.
Revised: In some online scenarios, the sample-wise correction method is more suitable.

Original: In addition, our method assumes that the marginal distribution is already known, and in the paper we assume in particular that it is consistent with a uniform distribution.
Revised: In addition, our method assumes that the marginal distribution is already known. In this paper, we assume in particular that it is consistent with a uniform distribution.

Original: [Figure]
Revised: [Figure]

Original: 3. C OST F UNCTION L EARNING VIA L INEAR M APPING
Revised: 3. C OST F UNCTION L EARNING VIA L INEAR M APPING

Original: Simple functions is likely to be sub-optimal for the real data sets; this suggests the design of a better cost function to better ﬁt the long-tailed recognition problem.
Revised: Simple functions are likely to be sub-optimal for the real data sets; this suggests the design of a better cost function to better ﬁt the long-tailed recognition problem.

Original: However, manually designed cost functions require expert knowledge in different domains.
Revised: However, manually designed cost functions require expert knowledge in different domains.

Original: Thus, we propose to use a linear mapping to automatically learn the cost function, which relieves the need to conﬁguration .
Revised: Thus, we propose to use a linear mapping to automatically learn the cost function, which relieves the need to conﬁguration .

Original: More speciﬁcally, for predictions ˜ Z generated by Softmax operation via leveraging linear transformation matrix W , W ∈ R K × K is learned so that the following objective function is minimised.
Revised: More speciﬁcally, for predictions ˜ Z generated by Softmax operation via leveraging linear transformation matrix W , W ∈ R K × K is learned so that the following objective function is minimised.

Original: [Equation]
Revised: [Equation]

Original: The resulting formula can be illustrated using a simple one-layer network of weight parameter W , together with an error function in Eq.
Revised: The resulting formula can be illustrated using a simple one-layer network of weight parameter W , together with an error function in Eq.

Original: By normalising each row of W , each element can be understood as the probability value of confusing two classes, and somehow as a transition matrix between classes .

Original: We initialise W with an identity matrix and use small learning rate to learn W .
Revised: We initialise W with an identity matrix and use small learning rate to learn W .

Original: One could also absorb the Logit Adjustment term − log p ( y s ) as the ﬁxed bias parameter into the network.
Revised: One could also absorb the Logit Adjustment term − log p ( y s ) as the ﬁxed bias parameter into the network.

Original: Motivated by this description, we can use a general gradient descent algorithm, such as SGD, to optimise the error function.
Revised: Motivated by this description, we can use a general gradient descent algorithm, such as SGD, to optimise the error function.

Original: Take into account the implementation in practice, the direct calculation of Eq.
Revised: Taking into account the implementation in practice, the direct calculation of Eq.

Original: (13) can be done using the Sinkhorn iterations (Cuturi, 2013; Frogner et al., 2015; Peyré et al., 2019) in mini-batch training efﬁciently.
Revised: (12) can be done using the Sinkhorn iterations (Cuturi, 2013; Frogner et al., 2015; Peyré et al., 2019) in mini-batch training efﬁciently.

Original: Besides, We term the proposed method as OTLM ( O ptimal T ransport via L inear M apping ), Figure 1 illustrates the overview of OTLM.
Revised: Besides, We term the proposed method as OTLM ( O ptimal T ransport via L inear M apping ), Figure 1 illustrates the overview of OTLM.

Original: Compute the gradients w.r.t W
Revised: Compute the gradients w.r.t W

Original: With the optimisation in Algorithm 1, it was conducted on the overall data set with a comparatively large number of data.
Revised: With the optimisation in Algorithm 1, it was conducted on the overall data set with a comparatively large number of data.

Original: Quite a different scenario now, as minibatch training is more favoured when it comes to neural networks.
Revised: Quite a different scenario now, as minibatch training is more favoured when it comes to neural networks.

Original: For this reason, the optimisation workﬂow in Algorithm 1, as pointed by Peyré et al.
Revised: For this reason, the optimisation workﬂow in Algorithm 1, as pointed by Peyré et al.

Original: (2019); Viehmann (2019), will have a problem of batch stablisation.
Revised: (2019); Viehmann (2019), will have a problem of batch stablisation.

Original: To this end, we perform logarithmic transformation to step 3-4 in Algorithm 1.
Revised: To this end, we perform logarithmic transformation to step 3-4 in Algorithm 1.

Original: [Equation]
Revised: [Equation]

Original: with log-sum-exp operation logsumexp ( x ) = log( (cid:80) i exp x i ) .
Revised: with log-sum-exp operation logsumexp ( x ) = log( (cid:80) i exp x i ) .

Original: Assuming convergence is achieved in the Sinkhorn’s loops, it is not necessary for us manually to compute the derivative of the loss function in Eq.
Revised: Assuming convergence is achieved in the Sinkhorn’s loops, it is not necessary for us manually to compute the derivative of the loss function in Eq.

Original: (13) w.r.t W .
Revised: (12) w.r.t W .

Original: As a matter of fact, as long as we make sure that the entire optimisation process is differentiable, modern deep learning libraries can then automatically implement end-toend derivations, as it allows us to differentiate between the results of Sinkhorn’s loops as a mere composition of elementary operations.
Revised: As a matter of fact, as long as we make sure that the entire optimisation process is differentiable, modern deep learning libraries can then automatically implement end-toend derivations, as it allows us to differentiate between the results of Sinkhorn’s loops as a mere composition of elementary operations.

Original: Performance degradation caused by imbalanced distributions can be addressed with meta-learning based methods (Ren et al., 2020; Wang et al., 2020a; Li et al., 2021).
Revised: Performance degradation caused by imbalanced distributions can be addressed with meta-learning based methods (Ren et al., 2020; Wang et al., 2020a; Li et al., 2021).

Original: Recent works have illustrated neu- ral networks can learn more meaningful representations from the validation data set { ( x vn , y vn ) } N v n =1 .
Revised: Recent works have illustrated neural networks can learn more meaningful representations from the validation data set { ( x vn , y vn ) } N v n =1 .

Original: Inspired by the methods based on meta learning,Here we also take a validation data set to optimise the parameter W , but with the still signiﬁcant difference in that we require no labelling information, thus avoiding a large expanse of labeling samples .
Revised: Inspired by the methods based on meta learning, we also take a validation data set to optimise the parameter W , but with the still signiﬁcant difference in that we require no labelling information, thus avoiding a large expanse of labeling samples .

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we ﬁrst conduct experiments comparing our approach versus extant post-hoc correction methods on three data sets, including CIFAR-100-LT (Cao et al., 2019), ImageNet-LT (Liu et al., 2019), and iNaturalist (Van Horn et al., 2018) with varying backbones.
Revised: In this section, we ﬁrst conduct experiments comparing our approach versus extant post-hoc correction methods on three data sets, including CIFAR-100-LT (Cao et al., 2019), ImageNet-LT (Liu et al., 2019), and iNaturalist (Van Horn et al., 2018) with varying backbones.

Original: Finally, we empirically make a comparison of our algorithm with alternative cutting-edge long-tailed recognition methods.
Revised: Finally, we empirically make a comparison of our algorithm with alternative cutting-edge long-tailed recognition methods.

Original: Observe that, for one thing, a plausible coupling of our algorithm to any training-aware long-tailed recognition method exists .
Revised: Observe that, for one thing, a plausible coupling of our algorithm to any training-aware long-tailed recognition method exists .

Original: As an illustration of the potency and strong generalisation of our approach, we conducted post-hoc correction of the pseudo-predictions of both methods, which are fairly typical for the training stage of all data sets: The ﬁrst is based on the cross entropy (CE) , and the other is the recently proposed RIDE (Wang et al., 2020b), which is based on multi experts.
Revised: As an illustration of the potency and strong generalisation of our approach, we conducted post-hoc correction of the pseudo-predictions of both methods, which are fairly typical for the training stage of all data sets: The ﬁrst is based on the cross entropy (CE) , and the other is the recently proposed RIDE (Wang et al., 2020b), which is based on multi experts.

Revised: We conducted experiments in the appendix for semi-supervised learning, to some extent to imitate the online situation.

Original: 4. D ATA S ETS AND B ASELINES
Revised: 4. D ATA S ETS AND B ASELINES

Original: The baseline methods and data set are brieﬂy described here, with implementation details placed in the appendix.
Revised: The baseline methods and data set are brieﬂy described here, with implementation details placed in the appendix.

Original: In our experiments, we assume that the marginal distribution of the test set is uniform due to the characteristics of the three data sets.
Revised: In our experiments, we assume that the marginal distribution of the test set is uniform due to the characteristics of the three data sets.

Original: Baselines : We compare our methods with (i) post-hoc correction methods including Logit Adjustment (Menon et al., 2020), DARP (Kim et al., 2020a), (ii) state-of-the-art methods including Focal
Revised: Baselines : We compare our methods with (i) post-hoc correction methods including Logit Adjustment (Menon et al., 2020), DARP (Kim et al., 2020a), (ii) state-of-the-art methods including Focal

Original: Loss (Lin et al., 2017), LDAM (Cao et al., 2019), BBN (Zhou et al., 2020), Balanced Softmax (Ren et al., 2020), Causal Norm (Tang et al., 2020), LADE (Hong et al., 2020), M2m (Kim et al., 2020b),
Revised: Loss (Lin et al., 2017), LDAM (Cao et al., 2019), BBN (Zhou et al., 2020), Balanced Softmax (Ren et al., 2020), Causal Norm (Tang et al., 2020), LADE (Hong et al., 2020), M2m (Kim et al., 2020b),

Original: Decouple (Kang et al., 2019), LFME (Xiang et al., 2020), RIDE (Wang et al., 2020b)
Revised: Decouple (Kang et al., 2019), LFME (Xiang et al., 2020), RIDE (Wang et al., 2020b)

Original: Long-tailed data set: We take experiments on three data sets including CIFAR-100-LT, ImageNetLT, and iNaturalist.
Revised: Long-tailed data set: We take experiments on three data sets including CIFAR-100-LT, ImageNetLT, and iNaturalist.

Original: We build the imbalanced version of CIFAR-100 by downsampling samples per class following the proﬁle in Liu et al.
Revised: We build the imbalanced version of CIFAR-100 by downsampling samples per class following the proﬁle in Liu et al.

Original: ; Kang et al.
Revised: ; Kang et al.

Original: (2019) with imbalanced ratio 10, 50, and 100.
Revised: (2019) with imbalanced ratios 10, 50, and 100.

Original: For all the benchmarks, we evaluate the performance on the test data set using a model trained on the training data set, and report the results using top-1 accuracy.
Revised: For all the benchmarks, we evaluate the performance on the test data set using a model trained on the training data set, and report the results using top-1 accuracy.

Original: 4. M AIN R ESULTS
Revised: 4. M AIN R ESULTS

Original: Table 1 illustrates the results on the CIFAR-100-LT data set.
Revised: Table 1 illustrates the results on the CIFAR-100-LT data set.

Original: For CE-based training-aware methods, it ﬁrst shows that Logit Adjustment is indeed a simple yet effective method for post-hoc correction.
Revised: For CE-based training-aware methods, it ﬁrst shows that Logit Adjustment is indeed a simple yet effective method for post-hoc correction.

Original: Remarkably, it outperforms the baseline methods by 2.6%, 4.1%, and 4.1% under the imbalance ratio of 10, 50, and 100 respectively. With a quick look at the results based on the OT approach, the superior results stress the advantages of our approach over the DARP and Logit adjustments. OT algorithm can further improve the performance by 0.62%, 0.43%, 0.70%, for OTLM, it outperforms
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: CE 61.
Revised: CE 62.

Original: DARP+CE 61.
Revised: [Table] [Table] [Table] [Table]

Original: OT+CE 62.25 +0. 50.05 +0. 45.81 +0. 51.87 +0.
Revised: [Table]

Original: 71.20 +1.
Revised: [Table]

Original: OTLM+CE 62.18 +0.
Revised: OTLM [Table] CE 62.

Original: 50.35 +0.
Revised: [Table]

Original: 46.10 +0.
Revised: [Table]

Original: 52.35 +0.
Revised: [Table] [Table] [Table] [Table]

Original: 72.30 +2.
Revised: [Table]

Original: RIDE - - 50.
Revised: RIDE - - 50.

Original: OT+RIDE - - 50.82 +0.
Revised: [Table]

Original: 58.45 +0.
Revised: [Table]

Original: 75.85 +2.
Revised: [Table]

Original: OTLM+RIDE - - 51.43 +1.
Revised: [Table] [Table] [Table] [Table]

Original: 58.71 +1.
Revised: [Table]

Original: 76.12 +3.
Revised: 76.12 +3.

Revised: Remarkably, it outperforms the baseline methods by 2.6%, 4.1%, and 4.1% under the imbalance ratio of 10, 50, and 100 respectively.

Revised: With a quick look at the results based on the OT approach, the superior results stress the advantages of our approach over the DARP and Logit adjustments.

Revised: OT algorithm can further improve the performance by 0.62%, 0.43%, 0.70%, for OTLM, it outperforms

Original: Logit Adjustment by 0.55%, 0.73%, 0.99% under the three imbalance ratios on CIFAR-100-LT data set.
Revised: Logit Adjustment by 0.55%, 0.73%, 0.99% under the three imbalance ratios on CIFAR-100-LT data set.

Original: For RIDE-based training-aware methods, there is also a 0.6%  improvement in the accuracy of our method.
Revised: For RIDE-based training-aware methods, there is also a 0.6% and 1.23% improvement in the accuracy of our method.

Original: Table 1 also provides the results on ImageNet-LT and iNaturalist data sets.
Revised: Table 1 also provides the results on ImageNet-LT and iNaturalist data sets.

Original: The fact again attracts our attention that for CE-based training-aware methods, OT, which is based on convex optimisation, consistently outperform Logit Adjustment by a large margin (1.76%) on iNaturalist data set.
Revised: The fact again attracts our attention that for CE-based training-aware methods, OT, which is based on convex optimisation, consistently outperforms Logit Adjustment by a large margin (1.76%) on iNaturalist data set.

Original: The accuracy boosting indicates the great potential of methods based on optimisation for post-hoc correction.
Revised: The accuracy boosting indicates the great potential of methods based on optimisation for post-hoc correction.

Original: Besides, not surprisingly, OTLM can further enhance the prediction accuracy, it outperforms
Revised: Besides, not surprisingly, OTLM can further enhance the prediction accuracy, it outperforms

Original: Logit Adjustment by 0.73% and 2.86% on ImageNet-LT and iNaturalist respectively.
Revised: Logit Adjustment by 0.73% and 2.86% on ImageNet-LT and iNaturalist respectively.

Original: As for RIDE, the gain in accuracy is also signiﬁcant, about 3% on iNaturalist.
Revised: As for RIDE, the gain in accuracy is also signiﬁcant, about 3% on iNaturalist.

Original: 4. C OMPARISON WITH THE S TATE - OF - THE - ART M ETHODS
Revised: 4. C OMPARISON WITH THE S TATE - OF - THE - ART M ETHODS

Original: Armed with a demonstration of the validity of OT and OTLM, we switched to a comparison of our performance with existing methods that achieved the most advanced results on three benchmarks.
Revised: Armed with a demonstration of the validity of OT and OTLM, we switched to a comparison of our performance with existing methods that achieved the most advanced results on three benchmarks.

Original: As shown in Table 2, in particular, on ImageNet-LT and iNaturalist data sets, the experimental results are remarkable and impressive, our method outperforms RIDE by 1.0% and 3.0%, respectively.
Revised: As shown in Table 2, in particular, on ImageNet-LT and iNaturalist data sets, the experimental results are remarkable and impressive, our method outperforms RIDE by 1.0% and 3.0%, respectively.

Original: Since iNaturalist is a very difﬁcult and ﬁne-grained data set consisting of 8,142 categories.
Revised: Since iNaturalist is a very difﬁcult and ﬁne-grained data set consisting of 8,142 categories.

Original: These huge performance gains come at a fraction of the cost , as we show in the next subsection.
Revised: These huge performance gains come at a fraction of the cost , as we show in the following subsection.

Original: [Table]
Revised: [Table]

Original: iNaturalist

Original: [Table]

Original: − − 
Revised: − − −

Original: [Table]

Original: [Table]
Revised: [Table]

Original: − − 
Revised: − − −

Original: [Table]

Original: − − − 43.

Original: [Table]
Revised: [Table]

Original: − − −41.
Revised: − − −

Original: [Table]
Revised: [Table]

Original: − − −47.
Revised: − − −

Original: [Table]
Revised: [Table]

Original: − − − 49. [Table]
Revised: − − − − [Table]

Original: −− − 49.
Revised: −

Original: LWS − − − 49. LDAM − − − − 64. LDAM+DRW 58.
Revised: [Table]

Original: −68.
Revised: −

Original: [Table]
Revised: [Table]

Original: −66.
Revised: −

Original: [Table]
Revised: [Table]

Original: M2m 58.

Original: − 43.
Revised: − [Table]

Original: − −
Revised: − −

Original: [Table]
Revised: [Table]

Original: − − 43.

Original: − −
Revised: − −

Original: Balanced Softmax 61.
Revised: [Table]

Original: − −
Revised: − −

Original: LADE 61.
Revised: [Table]

Original: RIDE (
Revised: − −

Original: [Table]
Revised: [Table]

Original: experts) − −50.
Revised: − −

Original: Logit Adjustment 61. DARP 61.
Revised: [Table]

Original: OT+CE 62.3 +0.
Revised: CE 62.3 +0.

Original: OTLM+CE 62.
Revised: OTLM+CE 62.

Original: 50.4 +0.
Revised: 50.4 +0.

Original: OT+RIDE (6 experts) − − 50.8 +0.
Revised: OT+RIDE (6 experts) − − 50.8 +0.

Original: 58.5 +1.
Revised: 58.5 +1.

Original: 75.9 +3.
Revised: 75.9 +3.

Original: OTLM+RIDE (6 experts) − − 51.
Revised: OTLM+RIDE (6 experts) − − 51.4 +1.

Original: [Table]
Revised: 58.7 +1. 76.1 +3.

Original: 4. C OMPUTATION C OST OF OT AND OTLM
Revised: 4. C OMPUTATION C OST OF OT AND OTLM

Original: As we have highlighted, the additional computational cost of OT and OTLM is particularly small, compared to that of training-aware methods.
Revised: As we have highlighted, the additional computational cost of OT and OTLM is particularly small, compared to that of training-aware methods.

Original: In Table 3, exact time of the evaluations on the ImageNetLT and iNaturalist data sets is provided.
Revised: In Table 3, exact time of the evaluations on the ImageNet- [Table] LT and iNaturalist data sets is provided.

Original: Please note, the times here are measured from the start of each method to the ﬁnal best performance.
Revised: Please note, the time here are measured from the start of each method to the ﬁnal best performance.

Original: For the comparison between OT and OTLM, because
Revised: For the comparison between OT and OTLM, because

Original: [Table]

Original: OTLM runs on the GPU, one optimization iteration of OTLM is in fact much faster than OT.
Revised: OTLM runs on the GPU, one optimisation iteration of OTLM is in fact much faster than OT.

Original: Except for OTLM, which was run on an NVidia card (V100), the results come from a 28-core machine (2.20 Ghz Xeon).
Revised: Except for OTLM, which was run on an NVidia card (V100), the results come from a 28-core machine (2.20 Ghz Xeon).

Original: Firstly, we can observe that all post-hoc correction methods here are not quite time-consuming .
Revised: Firstly, we can observe that all post-hoc correction methods here are not quite time-consuming .

Original: For comparison, ImageNet-LT and iNaturalist can be trained on 4 NVidia cards (V100) for extremely long periods of time.
Revised: For comparison, ImageNet-LT and iNaturalist can be trained on 4 NVidia cards (V100) for extremely long periods of time.

Original: Using ResNet-50-32x4d on the iNaturalist training data set, for example, with a batch size of 256, the amount of training time (in seconds) to performiteration is approximately 850.
Revised: Using ResNeXt-50-32x4d on the iNaturalist training data set, for example, with a batch size of 256, the amount of training time (in seconds) to performiteration is approximately 850.

Original: Also, if one differs the performance of each method, we gather that
Revised: Also, if one differs the performance of each method, we gather that

Original: OT and OTLM provide the best balance of performance and running costs to match .
Revised: OT and OTLM provide the best balance of performance and running costs to match .

Original: 4. C ONVERGENCE B EHAVIOR OF OTLM
Revised: 4. C ONVERGENCE B EHAVIOR OF OTLM

Original: To verify that our adopted OTLM algorithm is really efﬁcient.
Revised: To verify that our adopted OTLM algorithm is really efﬁcient.

Original: With ResNet-50 as the backbone, we plotted curves depicting the decreasing absolute error σ with the number of iterations t on the ImageNet-LT validation data set.
Revised: With ResNeXt-50-32x4d as the backbone, we plotted curves depicting the decreasing absolute error σ with the number of iterations t on the ImageNet-LT validation data set.

Original: Image on the left in Figure 2 gives us a visualisation of the optimisation process.
Revised: Image on the left in Figure 2 gives us a visualisation of the optimisation process.

Original: It is found that the absolute error σ converges at around 60 iterations.
Revised: It is found that the absolute error σ converges at around 60 iterations.

Original: In fact, we discovered that 10 or even fewer iterations were enough to yield satisfactory performance.
Revised: In fact, we discovered that 10 or even fewer iterations were enough to yield satisfactory performance.

Original: backbone to obtain the original predictions Z .

Original: The right image demonstrates the block in W , we trained on CIFAR-100 to produce the original predictions with OTLM to minimise the difference between the ideal and actual distributions.

Original: For ease of display, the values of the diagonal elements are relatively large and we will replace these values with zeros.

Original: 4. D ETAILED A NALYSIS ON D ISTRIBUTION M ATCHING
Revised: 4. D ETAILED A NALYSIS ON D ISTRIBUTION M ATCHING

Original: Upon distribution ﬁtting, preferably the estimation bias is lessened, which implies that the KL divergence distance between the reﬁned prediction and the target distribution is decreased.
Revised: Upon distribution ﬁtting preferably the estimation bias is lessened, which implies that the KL divergence distance between the reﬁned prediction and the target distribution is decreased.

Original: Tabledisplays the KL distances on the ImageNet-LT and iNaturalist data sets, and we can ﬁnd that OT possesses a smaller KL distance than Logit Adjustment, which suggests that OT decreases the estimation bias more than Logit Adjustment by a much larger margin.
Revised: Tabledisplays the KL distances on the ImageNet-LT and iNaturalist data sets and we can ﬁnd that OT possesses a smaller KL distance than Logit Adjustment which suggests that OT decreases the estimation bias more than Logit Adjustment by a much larger margin [Table]

Original: 4. V ISUALISATION ON W EIGHT P ARAMETER IN OTLM
Revised: V ISUALISATION ON W EIGHT P ARAMETER IN OTLM

Original: As a natural consequence, one wonders about the exact structure of this weight matrix W , and no better way to present its results than through visualisation.
Revised: As a natural consequence, one wonders about the exact structure of this weight matrix W , and no better way to present its results than through visualisation.

Original: Towards this end, we applied OTLM[Table] to reﬁne the original predictions on CIFAR-100 data set with an imbalanced ratio R = 100 and
Revised: Towards this end, we applied OTLM to reﬁne the original predictions on CIFAR-100 data set with an imbalanced ratio R = 100 and

Original: ResNet-32 backbone, and trained a one-layer model until it converged to extract the value of W , where the shape of is 100 × 100 .
Revised: ResNet-32 backbone, and trained a one-layer model until it converged to extract the value of W , where the shape of is 100 × 100 .

Original: We visualised a sub-block of size 20 × 20 , the illustration is presented on the right image in Figure 2.
Revised: We visualised a sub-block of size 20 × 20 , the illustration is presented on the right image in Figure 2.

Original: The very physical meaning of the elements in W can be interpreted as the probability of transition between classes.

Original: The visualised matrix is very dense, this non-sparsity suggests that the OTLM manages to correct the predictions of the model in the training stage, whilst minimising the distance between the desired and the actual distribution.
Revised: The visualised matrix is very dense, this non-sparsity suggests that the OTLM manages to correct the predictions of the model in the training stage, whilst minimising the distance between the desired and the actual distribution.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Long-tailed recognition:
Revised: Long-tailed recognition:

Original: The great majority of current methods applied to long-tailed recognition can be divided into either training-aware and post-hoc correction methods.
Revised: The great majority of current methods applied to long-tailed recognition can be divided into either training-aware and post-hoc correction methods.

Original: The training-aware methods aim to improve the generalisation ability of models trained on imbalanced training data set.
Revised: The training-aware methods aim to improve the generalisation ability of models trained on imbalanced training data set.

Original: Among them, re-sampling and re-weighting are two major methods.
Revised: Among them, re-sampling and re-weighting are two major methods.

Original: Re-sampling controls the class numbers by the means of over-sampling (Chawla et al., 2002) and under-sampling (Barandela et al., 2004; Drummond et al., 2003).
Revised: Re-sampling controls the class numbers by the means of over-sampling (Chawla et al., 2002) and under-sampling (Barandela et al., 2004; Drummond et al., 2003).

Original: The re-weighting approaches achieve balanced learning by giving smaller weights to majority classes and larger weights to minority classes (Tang et al., 2008; Zadrozny et al., 2003; Lin et al., 2017; Lee et al., 2017;
Revised: The re-weighting approaches achieve balanced learning by giving smaller weights to majority classes and larger weights to minority classes (Tang et al., 2008; Zadrozny et al., 2003; Lin et al., 2017; Lee et al., 2017;

Original: Cui et al., 2019; Khan et al., 2019).
Revised: Cui et al., 2019; Khan et al., 2019).

Original: Other approaches have also been proposed to resolve the imbalanced problem.
Revised: Other approaches have also been proposed to resolve the imbalanced problem.

Original: For example, Cao et al.
Revised: For example, Cao et al.

Original: introduces a theoretically-principled loss function motivated by minimizing the generalisation error bound (Cao et al., 2019), Kang et al.
Revised: introduces a theoretically-principled loss function motivated by minimizing the generalisation error bound (Cao et al., 2019), Kang et al.

Original: proposes to decouple the learning phases into representation learning and classiﬁer tuning (Kang et al., 2019).
Revised: proposes to decouple the learning phases into representation learning and classiﬁer tuning (Kang et al., 2019).

Original: Apart from training-aware methods, post-hoc correction also offers us an avenue for long-tailed recognition problem, it can be applied to the normalisation of classiﬁer weights (Kang et al., 2019; Zhang et al., 2019; Kim & Kim, 2020) or logits based on label frequency (Provost, 2000;
Revised: Apart from training-aware methods, post-hoc correction also offers us an avenue for long-tailed recognition problem, it can be applied to the normalisation of classiﬁer weights (Kang et al., 2019; Zhang et al., 2019; Kim & Kim, 2020) or logits based on label frequency (Provost, 2000;

Original: Collell et al., 2016).
Revised: Collell et al., 2016).

Original: Among them, Logit Adjustment (Menon et al., 2020) is a simple yet powerful methods to use.
Revised: Among them, Logit Adjustment (Menon et al., 2020) is a simple yet powerful method to use.

Original: Compared with Logit Adjustment, our approach perform the post-hoc correction from the optimisation view.
Revised: Compared with Logit Adjustment, our approach performs the post-hoc correction from the optimisation view.

Original: Optimal transport: OT distances is a important family of distances for probability measures (Villani, 2008; Santambrogio, 2015).
Revised: Optimal transport: OT distance is a important family of distances for probability measures (Villani, 2008; Santambrogio, 2015).

Original: It has been extensively applied in different ﬁelds in machine learning.
Revised: It has been extensively applied in different ﬁelds in machine learning.

Original: Courty et al.
Revised: Courty et al.

Original: (Courty et al., 2016) proposes a regularised unsupervised optimal transportation model to perform the domain adaption.
Revised: (Courty et al., 2016) proposes a regularised unsupervised optimal transportation model to perform the domain adaption.

Original: Other applications include 3D shape matching (Su et al., 2015), generative model (Arjovsky et al., 2017; Salimans et al., 2018; Bunne et al., 2019; Deshpande et al., 2019; Bhagoji et al., 2019), graph matching (Xu et al., 2019a;b), model designs Kandasamy et al.
Revised: Other applications include 3D shape matching (Su et al., 2015), generative model (Arjovsky et al., 2017; Salimans et al., 2018; Bunne et al., 2019; Deshpande et al., 2019; Bhagoji et al., 2019), graph matching (Xu et al., 2019a;b), model designs Kandasamy et al.

Original: ; Chizat & Bach (2018).
Revised: ; Chizat & Bach (2018).

Original: To the best of our knowledge, we are the ﬁrst to perform the post-hoc correction from the OT perspective.
Revised: To the best of our knowledge, we are the ﬁrst to perform the post-hoc correction from the OT perspective.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we present two techniques for post-hoc corrections from an optimisation perspective.
Revised: In this paper, we present two techniques for post-hoc corrections from an optimisation perspective.

Original: Our approach reviews the basic mathematical formulation of the distribution alignment problem and relates the resulting formulation to OT, which we ﬁnd, moreover, offers greater possibilities and ﬂexibility to solve the problem.
Revised: Our approach reviews the basic mathematical formulation of the distribution alignment problem and relates the resulting formulation to OT, which we ﬁnd, moreover, offers greater possibilities and ﬂexibility to solve the problem.

Original: In order to avoid manual conﬁguration of the cost matrix, we propose to learn the cost matrix automatically by linear mapping, which leads to further performance improvements.
Revised: In order to avoid manual conﬁguration of the cost matrix, we propose to learn the cost matrix automatically by linear mapping, which leads to further performance improvements.

Original: The experimental results conﬁrm the superiority of our proposed approach.
Revised: The experimental results conﬁrm the superiority of our proposed approach.

Original: I believe our work can open up more possibilities and imagination 
Revised: We believe our work can open up more possibilities and imagination for post-hoc corrections.

Revised: There are potential challenges that remain to be addressed.

Revised: These include, for example, how to apply OT to correction of predictions under unknown marginal distribution.

Revised: Meanwhile, an extension of our approach to other applications where label shift is involved is considered possible.

Original: A O PTIMAL T RANSPORT ON L ONG - TAILED S EMI - SUPERVISED C LASSIFICATION
Revised: A O PTIMAL T RANSPORT ON L ONG - TAILED S EMI - SUPERVISED C LASSIFICATION

Original: Together, with the extension of our approach, we adapt it to semi-supervised long-tailed classiﬁcation tasks.
Revised: Together, with the extension of our approach, we adapt it to semi-supervised long-tailed classiﬁcation tasks.

Original: We bring our approach together with the advanced semi-supervised learning methods
Revised: We bring our approach together with the advanced semi-supervised learning methods

Original: FixMatch Sohn et al.
Revised: FixMatch Sohn et al.

Original: Adhering to the conﬁguration in DARP Kim et al.
Revised: Adhering to the conﬁguration in DARP Kim et al.

Original: (2020a), we also create synthetically long-tailed variants of CIFAR-100 with imbalanced ratio R = 20 .
Revised: (2020a), we also create synthetically long-tailed variants of CIFAR-100 with imbalanced ratio R = 20 .

Original: The sample size of the unlabelled data is twice as large as the labelled data, with the same proportion of imbalance between them.
Revised: The sample size of the unlabeled data is twice as large as the labelled data, with the same proportion of imbalance between them.

Original: To assess the performance, we report two popular metrics: balanced accuracy (bACC)
Revised: To assess the performance, we report two popular metrics: balanced accuracy (bACC)

Original: Huang et al.
Revised: Huang et al.

Original: and geometric mean scores (GM) Branco et al.
Revised: and geometric mean scores (GM) Branco et al.

Original: Experimentally, with the exception of an OT to align the marginal distribution of the unlabelled data, all settings are consistent with DARP, and we set λ to 0.005.
Revised: Experimentally, with the exception of an OT to align the marginal distribution of the unlabelled data, all settings are consistent with DARP, and we set λ to 0.005.

Original: The results are reported in Table 5.
Revised: The results are reported in Table 5.

Original: It can be seen that our approach also achieves much better performance on semi-supervised learning tasks.
Revised: It can be seen that our approach also achieves much better performance on semi-supervised learning tasks.

Revised: It can somehow mimic the online scenarios.

Revised: And the advantages of OT coping with such cases can be demonstrated.

Original: [Table]
Revised: [Table]

Original: B P ARAMETER S TUDY OF λ
Revised: B P ARAMETER S TUDY OF λ

Original: We conduct an additional experiment on iNaturalist data set to study the effect of λ on the ﬁnal performance.
Revised: We conduct an additional experiment on iNaturalist data set to study the effect of λ on the ﬁnal performance.

Original: The value of λ is selected from { 0 . 01 , 05 , 0 . 1 , 1 . 0 . 2 . 0 , 5 . 0 } , all other experimental settings are the same, including various hyperparameters, network structure, etc.
Revised: The value of λ is selected from { 0 . 01 , 05 , 0 . 1 , 1 . 0 . 2 . 0 , 5 . 0 } , all other experimental settings are the same, including various hyperparameters, network structure, etc.

Original: As illustrated in
Revised: As illustrated in

Original: Figure 3, we can ﬁnd that OT achieves the best performance when λ
Revised: Figure 3, we can ﬁnd that OT achieves the best performance when λ

Original: = 1 .
Revised: = 1 .

Original: C I MPLEMENTATION D ETAILS
Revised: C I MPLEMENTATION D ETAILS

Original: The speciﬁc implementation details for each data set under the different methods are described below.
Revised: The speciﬁc implementation details for each data set under the different methods are described below.

Original: C. I MPLEMENTATION D ETAILS OF C ROSS E NTROPY
Revised: C. I MPLEMENTATION D ETAILS OF C ROSS E NTROPY

Original: CIFAR-100-LT : During the training stage, following the recipe of (Tang et al., 2020; Hong et al., 2020), we apply SGD with batch size 256 and weight decay 0.0005 to train a ResNet-32 (He et al., 2016) model for 200 epochs, we employ the linear warm-up learning rate schedule for the ﬁrst ﬁve epochs.
Revised: CIFAR-100-LT : During the training stage, following the recipe of (Tang et al., 2020; Hong et al., 2020), we apply SGD with batch size 256 and weight decay 0.0005 to train a ResNet-32 (He et al., 2016) model for 200 epochs, we employ the linear warm-up learning rate schedule for the ﬁrst ﬁve epochs.

Original: We also set the base learning rate to 0.2 and reduce it at epoch 120 and 160 by a factor of 
Revised: We also set the base learning rate to 0.2 and reduce it at epoch 120 and 160 by a factor of 100.

Original: During the post-hoc stage, we use Adam with batch size 10k and weight decay 0.001 to train the single layered feed-forward network for 20 iterations.
Revised: During the post-hoc stage, we use Adam with batch size 10k and weight decay 0.001 to train the one-layer feed-forward network for 20 iterations.

Original: We set λ to 0.1, T to 200, ∆ to 0.001.
Revised: We set λ to 0.1, T to 200, ∆ to 0.001.

Original: The learning rate is constant and set to 0.001.
Revised: The learning rate is constant and set to 0.001.

Original: ImageNet-LT : During the training stage, following the implementation of (Hong et al., 2020), we train the ResNeXt-50-32x4d (Xie et al., 2017) for 90 epochs and perform a cosine learning rate scheme with an initial learning rate of 0.05.
Revised: ImageNet-LT : During the training stage, following the implementation of (Hong et al., 2020), we train the ResNeXt-50-32x4d (Xie et al., 2017) for 90 epochs and perform a cosine learning rate scheme with an initial learning rate of 0.05.

Original: SGD is also employed to optimise the neural network with batch size 256, weight decay 0.0005, and momentum 0.9.
Revised: SGD is also employed to optimise the neural network with batch size 256, weight decay 0.0005, and momentum 0.9.

Original: During the post-hoc stage, we use
Revised: During the post-hoc stage, we use

Original: SGD with batch size 10k, weight decay 0.001 to train the single layered feed-forward network for 100 iterations.
Revised: SGD with batch size 10k, weight decay 0.001 to train the one-layer feed-forward network foriterations.

Original: The constant learning rate is 0.2.
Revised: The constant learning rate is 0.2.

Original: We set λ to 1.2, T to 7, ∆ to 0.001.
Revised: We set λ to 1.2, T to 7, ∆ to 0.001.

Original: iNaturalist :
Revised: iNaturalist :

Original: During the training stage, ResNet-50 (He et al., 2016) is chosen as the backbone network.
Revised: During the training stage, ResNet-50 (He et al., 2016) is chosen as the backbone network.

Original: We use SGD with momentum 0.9, batch size 256 to train the network.
Revised: We use SGD with momentum 0.9, batch size 256 to train the network.

Original: We utilise the cosine learning rate schedule gradually decaying from 0.1 to 0.
Revised: We utilise the cosine learning rate schedule gradually decaying from 0.1 to 0.

Original: During the post-hoc stage, we use SGD with batch size 23,826, weight decay 0.001, λ to 1.0, ∆ to 0.001 and T to 10.
Revised: During the post-hoc stage, we use SGD with batch size 23,826, weight decay 0.001, λ to 1.0, ∆ to 0.001 and T to 10.

Original: The constant learning rate is set to 0.05.
Revised: The constant learning rate is set to 0.05.

Original: As iNaturalist has no test data set, we directly report the performance on validation data set.
Revised: As iNaturalist has no test data set, we directly report the performance on validation data set.

Original: C. I MPLEMENTATION D ETAILS OF RIDE
Revised: C. I MPLEMENTATION D ETAILS OF RIDE

Original: The training details we use in RIDE share the same with the original paper, there are 6 experts in our
Revised: The training details we use in RIDE share the same as the original paper, there are 6 experts in our

Original: RIDE on all three data sets.
Revised: RIDE on all three data sets.

Original: CIFAR-100-LT We experimented with R = 100 and ResNet-32 as the base network.
Revised: CIFAR-100-LT We experimented with R = 100 and ResNet-32 as the base network.

Original: SGD is utilised as our solver with momentum 0.9, batch size 128, and epoch 200.
Revised: SGD is utilised as our solver with momentum 0.9, batch size 128, and epoch 200.

Original: The learning rate is initialised as 0.1 and decayed by 0.01 at epoch 120 and 160 respectively.
Revised: The learning rate is initialised as 0.1 and decayed by 0.01 at epoch 120 and 160 respectively.

Original: ImageNet-LT We take experiments with Re as backbone with 100 epochs.
Revised: ImageNet-LT We take experiments with ResNeXt-50 as backbone with 100 epochs.

Original: We set the initial learning rate to 0.1 and reduce it by 10 at epoch 60 and 80.
Revised: We set the initial learning rate to 0.1 and reduce it by 10 at epoch 60 and 80.

Original: SGD with batch size 256 and momentum 0.9 is also adopted as the optimisation solver here.
Revised: SGD with batch size 256 and momentum 0.9 is also adopted as the optimisation solver here.

Original: iNaturalist We conduct experiments with ResNet-50 as backbone.
Revised: iNaturalist We conduct experiments with ResNet-50 as backbone.

Original: The training details are the same as ImageNet-LT except from batch size 512.
Revised: The training details are the same as ImageNet-LT except from batch size 512.