Original: O PTIMAL T RANSPORT FOR L ONG -T AILED R ECOGNITION WITH L EARNABLE C OST M ATRIX
Revised: O PTIMAL T RANSPORT FOR L ONG -T AILED R ECOGNITION WITH L EARNABLE C OST M ATRIX

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: It is attracting attention to the long-tailed recognition problem, a burning issue that has become very popular recently.
Revised: It is attracting attention to the long-tailed recognition problem, a burning issue that has become very popular recently.

Original: Distinctive from conventional recognition is that it posits that the allocation of the training set is supremely distorted, with a balanced validation and test set.
Revised: Distinctive from conventional recognition is that it posits that the allocation of the training set is supremely distorted, with a balanced validation and test set.

Original: Predictably, there are severe challengesposed to the generalisation behaviour of the model, due to the distribution shift between the training and test sets.
Revised: Predictably, it will pose challenges to the generalisation behaviour of the model.

Original: Approaches to the challenge revolve into two groups: ﬁrstly, training-aware methods, with the aim of enhancing the generalisability of the model by exploiting its potential in the training period; and secondly, post-hoc correction, liberally coupled with training-aware methods, which is intended to reﬁne the predictions to the extent possible in the post-processing stage, offering the advantages of simplicity and effectiveness.
Revised: Approaches to these challenges revolve into two groups: ﬁrstly, training-aware methods, with the aim of enhancing the generalisability of the model by exploiting its potential in the training period; and secondly, post-hoc correction, liberally coupled with training-aware methods, which is intended to reﬁne the predictions to the extent possible in the post-processing stage, offering the advantages of simplicity and effectiveness.

Original: In this paper, we introduce an alternative direction to do the post-hoc correction, which goes beyond the statistical methods.
Revised: This paper introduces an alternative direction to do the post-hoc correction, which goes beyond the statistical methods.

Original: Mathematically, we approach this issue from the perspective of optimal transport (OT), yet, choosing the exact cost matrix when applying OT is challenging and requires expert knowledge of various tasks.
Revised: Mathematically, we approach this issue from the perspective of optimal transport (OT), yet, choosing the exact cost matrix when applying OT is challenging and requires expert knowledge of various tasks.

Original: To overcome this limitation, we propose to employ linear mapping toadaptively learn the cost matrix without necessary conﬁgurations.
Revised: To overcome this limitation, we propose to employ linear mapping to learn the cost matrix without necessary conﬁgurations adaptively.

Original: Testing our methods in practice,our experiments reveals that, along with high efﬁciency and excellent performance, our method surpasses all previous methods and has the best performance to date.
Revised: Testing our methods in practice, along with high efﬁciency and excellent performance, our method surpasses all previous methods and has the best performance to date.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Classiﬁcation problems in the real world are generally challenged by the long-tailed label distribution, i.e., having a small number of samples for a majority of labels, and a dominant number of samples for a minority of labels (Van Horn & Perona, 2017; Buda et al., 2018; Liu et al., 2019) on the training data set, vice versa, with validation and test data being evenly distributed.
Revised: Classiﬁcation problems in the real world are generally challenged by the long-tailed label distribution, i.e., having a small number of samples for a majority of labels, and a dominant number of samples for a minority of labels (Van Horn & Perona, 2017; Buda et al., 2018; Liu et al., 2019) on the training data set, vice versa, with validation and test data being evenly distributed.

Original: It is also known as imbalanced recognition, which has been widely studied in the past decades (Cardie & Howe, 1997; Chawla et al., 2002; Qiao & Liu, 2009;Cui et al., 2019).
Revised: It is also known as imbalanced recognition, which has been widely studied in the past decades (Cardie & Howe, 1997; Chawla et al., 2002; Qiao & Liu, 2009;

Original: Inasmuch on the assumption that the same number of examples are available for each category, the majority of machine learning algorithms for classiﬁcation are structured around the hypothesis.
Revised: Cui et al., 2019).

Original: Therefore, the distribution biases poses a signiﬁcant challenge to predictive modelng. When validated on test data sets, models often suffer from poor generalisation and undesirable estimation bias (Cao et al., 2019;
Revised: These distribution biases pose a signiﬁcant challenge to predictive modeling; conceivably, models often suffer from poor generalisation and undesirable estimation bias (Cao et al., 2019;

Original: Kang et al., 2019;
Revised: Kang et al., 2019;

Original: Zhou et al., 2020), especially for the minority class.
Revised: Zhou et al., 2020).

Original: Of late, a renewed interest in the problem of long-tail recognition has emerged following the context of neural networks, as numerous publications in the literature endeavour to resolve the problem albeit in different ways including decouple (Kang et al., 2019), meta-learning (Ren et al., 2020;
Revised: Recently, a renewed interest in the problem of long-tail recognition has emerged following the context of neural networks, as numerous publications in the literature endeavour to resolve the problem albeit in different ways including decouple (Kang et al., 2019), meta-learning (Ren et al., 2020;

Original: Wang et al., 2020a; Li et al., 2021), post-hoc correction (Tang et al., 2020;
Revised: Wang et al., 2020a; Li et al., 2021), post-hoc correction (Tang et al., 2020;

Original: Hong et al., 2020), etc (Liu et al., 2019; Cao et al., 2019; Tang et al., 2020).
Revised: Hong et al., 2020), etc (Liu et al., 2019; Cao et al., 2019; Tang et al., 2020).

Original: One of themore representative methods of post-hoc correction, Logit Adjustment Menon et al.
Revised: One of the representative methods of post-hoc correction, Logit Adjustment Menon et al.

Original: (2020), provides a statistical correction to the prediction, receiving widespread attention for its simplicity and validity, but the downside is that its optimisation is conducted on individual samples, without any indication that rectiﬁed marginal distribution satisﬁes the desired distribution we are striving for.
Revised: (2020), provides a statistical correction to the prediction, receiving widespread attention for its simplicity and validity, but the downside is that its optimisation is conducted on individual samples, without any indication that rectiﬁed marginal distribution satisﬁes the desired distribution we are striving for.

Original: Figuring out how to solve the above problem, our explicit modelling of the problem mathematically turns into an equational constraint, meanwhile to minimise the difference between reﬁned distribution and the original one, this minimisation is motivated upon the inner product similarity.
Revised: Figuring out how to solve the above problem, our explicit modeling of the problem mathematically turns into an equational constraint, meanwhile to minimise the difference between reﬁned distribution and the original one, this minimisation is motivated upon the inner-product similarity.

Original: A little further, the resulting problem can be linked to OT.
Revised: A little further, the resulting problem can be linked to OT.

Original: Drawing on this linkage, we develop it further by proposing a linear mapping to automatically learn cost matrix, thereby circumventing the requirement for expert knowledge to conﬁgure this matrix, simultaneously enabling automatic adaption to various tasks.
Revised: Drawing on this linkage, we develop it further by proposing a linear mapping to automatically learn cost matrix, thereby circumventing the requirement for expert knowledge to conﬁgure this matrix.

Original: In short, we contribute as follows:
Revised: In short, we contribute as follows:

Original: • We propose an alternative direction based on convex optimisation to do post-hoc correction, which goes beyond previous direction from the statistical view.
Revised: • We propose an alternative direction based on convex optimisation to do post-hoc correction, which goes beyond previous direction from the statistical view.

Original: • Imposing marginal distributions to align ideal marginal distributions, we derive an optimisation problem tied to the OT that is solved using Sinkhorn, a highly efﬁcient algorithm.
Revised: • Imposing marginal distributions to align ideal marginal distributions, we derive an optimisation problem tied to the OT that is solved using Sinkhorn, a highly efﬁcient algorithm.

Original: More further, for better learning of the cost matrix, we present a linear mapping enabling very elegant learning with one-layer neural network.
Revised: More further, for better learning of the cost matrix, we present a linear mapping enabling very elegant learning with one-layer neural network.

Original: • The experimental evidence shows the high efﬁciency of the algorithm, and the best performance achieved on three benchmarks, also supports our claim that addressing the post-hoc problem via OT is helpful and effective.
Revised: • The experimental evidence shows the high efﬁciency of the algorithm, and the best performance achieved on three benchmarks, also supports our claim that addressing the post-hoc problem via OT is helpful and effective.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: In this section, we begin by giving a notational deﬁnition, followed by an introduction to the longtailed recognition problem, and ﬁnally, we brieﬂy review the OT.
Revised: In this section, we begin by giving a notational deﬁnition, followed by an introduction to the longtailed recognition problem, and ﬁnally, we brieﬂy review the OT, and give a quick recap of Logit

Revised: Adjustment Menon et al.

Original: Notations : In what follows, for two matrices X , Y ∈ R N × K , we denote (cid:104) X , Y (cid:105) =
Revised: Notations : In what follows, for two matrices X , Y ∈ R N × K , we denote (cid:104) X , Y (cid:105) =

Original: (cid:80) Nn =1 (cid:80)
Revised: (cid:80) Nn =1 (cid:80)

Original: Kk =1 X
Revised: Kk =1 X

Original: nk Y nk as the Frobenius dot-product.
Revised: nk Y nk as the Frobenius dot-product.

Original: δ ( · ) stands for the Dirac function, p ( · ) represents the probability distribution.
Revised: δ ( · ) stands for the Dirac function, p ( · ) represents the probability distribution.

Original: U ( r , c ) = { P ∈ R N × K + | P 1 K = r , P (cid:124) 1 N = c } , where 1 N and 1 K are N -dimension and K -dimension vector whose elements are all 1.
Revised: U ( r , c ) = { P ∈ R N × K + | P 1 K = r , P (cid:124) 1 N = c } , where 1 N and 1 K are N -dimension and K -dimension vector whose elements are all 1.

Revised: r and c refer to the vectors of size N and K respectively, U ( r , c ) include all matrices with row and column sums r and c respectively

Original: 2. P ROBLEM F ORMULATION
Revised: 2. P ROBLEM F ORMULATION

Original: Having a collection of training samples { ( x sn , y sn ) } N s n =1 , validation samples { ( x vn , y vn ) } N v n =1 and test samples { ( x tn , y n ) t } N t n =1 for classiﬁcation with K labels and input x ∈ R d , long-tailed recognition assumes that the class-prior distribution for training data p ( y s ) is different from that for validation data p ( y v ) and test data p ( y t ) .
Revised: Having a collection of training samples { ( x sn , y sn ) } N s n =1 , validation samples { ( x vn , y vn ) } N v n =1 and test samples { ( x tn , y n ) t } N t n =1 for classiﬁcation with K labels and input x ∈ R d , long-tailed recognition assumes that the class-prior distribution for training data p ( y s ) is different from that for validation data p ( y v ) and test data p ( y t ) .

Original: Speciﬁcally, long-tailed recognition means the distribution p ( y s ) is highly skewed, some classes have the dominant number of samples, while tailed labels own a very small number of samples.
Revised: Speciﬁcally, long-tailed recognition means the distribution p ( y s ) is highly skewed, some classes have the dominant number of samples, while tailed labels own a very small number of samples.

Original: In Contrary to this, the validation and test data sets is uniformly balanced .
Revised: In contrary to this, the validation and test data sets is uniformly balanced .

Original: We can use imbalance ratio to measure the skewness in training data set, which can be deﬁned as
Revised: We can use imbalance ratio to measure the skewness in training data set, which can be deﬁned as

Original: R = N smax N smin , where N smax and N smin denote the largest and smallest number of samples in the training data set, respectively.
Revised: R = N smax N smin , where N smax and N smin denote the largest and smallest number of samples in the training data set, respectively.

Original: Consider that the uniformity of the distribution on the validation and test data sets, it could be treated as an implicit prior knowledge to be applied, exactly as the previous approach did, such as Balanced Softmax (Ren et al., 2020), Logit Adjustment (Menon et al., 2020).
Revised: Consider that the uniformity of the distribution on the validation and test data sets, it could be treated as an implicit prior knowledge to be applied , exactly as the previous approach did, such as Balanced Softmax (Ren et al., 2020), Logit Adjustment (Menon et al., 2020).

Revised: Stepping back, even if we do not know the marginal distribution of the test dataset in advance.

Revised: There are still ways to estimate the marginal distribution of the test dataset relatively precisely, such as methods in Hendrycks et al.

Original: Under these circumstances, obviously, most models trained on imbalanced training data set would suffer from extremely limited generalisation ability.
Revised: ; Azizzadenesheli et al. Obviously, most models trained on imbalanced training data set would suffer from extremely limited generalisation ability.

Original: Hence the ultimate goal is to learn a model that minimises the empirical risk, which can be formulated as:
Revised: Hence the ultimate goal is to learn a model that minimises the empirical risk, which can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where Φ( x sn ) ∈ R K denotes logits with associated sample, Φ( · ) :
Revised: where Φ( x sn ) ∈ R K denotes logits with associated sample, Φ( · ) :

Original: R d → R K represents the mapping via neural networks, L stands for the loss function, typically cross entropy loss function for classiﬁcation problem.
Revised: R d → R K represents the mapping via neural networks, L stands for the loss function, typically cross entropy loss function for classiﬁcation problem.

Original: Next, for convenience, we will drop the superscript s, v, t .

Original: 2. R EMINDERS ON O PTIMAL T RANSPORT
Revised: 2. R EMINDERS ON O PTIMAL T RANSPORT

Original: Measuring the difference between two distributions remains a crucial step in machine learning. In the regime of little prior knowledge about the distributions, numerous distance functions with minimal limitations or dependency including KL divergences, total variation (Devroye et al., 2018) have been proposed to play this role. However, these distance functions may poorly capture the structure information, in contrast, OT sometimes can learn more powerful geometry for distribution comparison, even outperforms in some practical applications.
Revised: OT is used to calculate the cost of transporting one probability measure to another.

Original: We next present some necessary deﬁnitions to help us better view the long-tailed problem from an OT perspective.
Revised: We next present a brief introduction to OT with the aim of helping us better view the long-tailed problem from an OT perspective.

Original: Deﬁnition 1 (Kantorovich Formulation) For two separable distribution space such that any distance measure on X (or Y ) is a Radon measure. For probability measures ron X and c on Y , we denote C ( x , y ) :
Revised: Consider probability measures r and c of the two random variables X and Y respectively, when the cost function C ( X, Y ) :

Original: X × Y → R + as the cost matrix. Kantorovich formulation seeks to ask for a probabilistic coupling π ∈ P ( X × Y ) by solving optimal transport:
Revised: X × Y → R + is deﬁned to denote the cost of transporting X to Y , the OT distance can be deﬁned as :

Original: [Equation]
Revised: [Equation]

Original: where Π ( r , c ) = (cid:110)(cid:82) Y π ( x
Revised: where Π ( r, c ) = (cid:110)(cid:82) Y π ( x

Original: , y ) d y = r , (cid:82)
Revised: , y ) d y = r, (cid:82)

Original: X π ( x , y )
Revised: X π ( x , y )

Original: d x = c (cid:111) 
Revised: d x = c (cid:111) is the joint probability measure with r and c .

Original: Deﬁnition 2 (Optimal Transport) Consider two discrete distributions r
Revised: When we extend the above continuous case to the discrete situation, we consider following discrete distributions: [Equation]

Original: = (cid:80) Nn =1 p  ( x n ) δ ( x n ) , c = (cid:80) Kk =1 p  ( y k ) δ ( y k ) , given a cost matrix M ∈ R N × K + , the cost transportation from r to c can be quantiﬁed as (cid:104) P , M (cid:105) , and optimal transport seeks toﬁnd the minimum optimal amount of mass, which can be deﬁned as:
Revised: where p i ( x i ) and p i ( y j ) represent the probability mass to the sample x i and y j respectively. In this context, OT distance can be expressed as:

Original: [Equation]
Revised: [Equation]

Revised: where M stands for the cost matrix constructed by M ij =

Original:  As we can see, OT is a distance measure between two probability distributions under some cost matrix (Villani, 2008).It is also called Earth Move distance or Wasserstein distance for some special cases.
Revised: C ( x i , y j ) . The goal of OT is to ﬁnd a transportation matrix P that minimizes the distance d M ( r , c ) As we can see, OT is a distance measure between two probability distributions under some cost matrix (Villani, 2008).

Original: Despite their great power and appealing theoretical properties, it often comes at the cost of heavy computational demands.
Revised: However, when we use network simplex or interior point methods to solve the above optimization problem, it often comes at the cost of heavy computational demands.

Original: To tackle this problem, OT with entropy constraint is proposed to allow the optimisation at small computational cost in sufﬁcient smoothness (Cuturi, 2013).
Revised: To tackle this issue, OT with entropy constraint is proposed to allow the optimisation at small computational cost in sufﬁcient smoothness (Cuturi, 2013).

Original: By adding a Lagrangian multiplier to the entropy constraint, the new formulation can be deﬁned as follows:
Revised: By adding a Lagrangian multiplier to the entropy constraint, the new formulation can be deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: P λ , M (cid:105) where P λ = arg min
Revised: where P λ = arg min

Original: P ∈ U ( r , c ) (cid:104) P , M (cid:105) − λh ( P ) , (4) where λ ∈ [0 , + ∞ ] , h ( P ) = 
Revised: [Equation] where λ ∈ [0 , + ∞ ] , h ( P ) = − (cid:80)

Original: − (cid:80) Nn =1 (cid:80)
Revised: Nn =1 (cid:80)

Original: Kk =1 P nk log P
Revised: Kk =1 P nk log P

Original: nk , d λ M ( r , c ) is also known as dual-Sinkhorn divergence, besides, it can be calculated with matrix scaling algorithms for cheaper computational demand than d M ( r , c ) with linear programming.
Revised: nk , d λ M ( r , c ) is also known as dual-Sinkhorn divergence, besides, it can be calculated with matrix scaling algorithms for cheaper computational demand.

Original: The entropy regularisation can enforce a simple structure on the optimal regularised transport to compute d λ M ( r , c ) .
Revised: The following lemma guarantees the convergence and uniqueness of the solution of the algorithm.

Original: Lemma 1 For λ > 0 , the solution P λ is unique and has the form P λ = diag ( u ) K diag ( v ) , where u and v are two non-negative vectors uniquely deﬁned up to a multiplicative factor and K = e − M /λ is the element-wise exponential of − M /λ .
Revised: Lemma 1 For λ > 0 , the solution P λ is unique and has the form P λ = diag ( u ) K diag ( v ) , where u and v are two non-negative vectors uniquely deﬁned up to a multiplicative factor and K = e − M /λ is the element-wise exponential of − M /λ .

Original: The above lemma states the uniqueness of P λ
Revised: The above lemma states the uniqueness of P λ

Original: (Sinkhorn, 1974), and P λ can be efﬁciently computed via Sinkhorn’s ﬁxed point iteration u , v ← r ./
Revised: (Sinkhorn, 1974), and P λ can be efﬁciently computed via Sinkhorn’s ﬁxed point iteration u , v ← r ./

Original: Mv , c ./ M (cid:124) u .
Revised: Kv , c ./ K (cid:124) u .

Revised: 2. A Q UICK R ECAP OF L OGIT A DJUSTMENT

Revised: We give a brief introduction to Logit Adjustment (Menon et al., 2020; Hong et al., 2020).

Revised: For the model Φ( · ) , it is trained by the standard cross-entropy loss function on imbalanced training data set, and evaluated on test data.

Original: M ETHODOLOGY
Revised: In this algorithm, the test logit is adjusted as follows: [Equation] This simple procedure is derived from the Bayes optimal rule. It is apparent that Logit Adjustment involves a post hoc correction on an individual sample, which does not necessarily guarantee that the marginal distribution of the whole dataset matches the desired distribution.

Revised: In addition, Logit

Revised: Adjustment assumes that the marginal distribution on the test set is known .

Revised:  M ETHODOLOGY

Original: The ﬁrst part of this section explores post-hoc correction from an OT perspective, proceeds to the automatic learning of the cost matrix via linear mapping.
Revised: The ﬁrst part of this section explores post-hoc correction from an OT perspective, proceeds to the automatic learning of the cost matrix via linear mapping.

Original: Lastly, we demonstrate how it can be achieved simply  one-layer neural network.
Revised: Lastly, we demonstrate how it can be achieved simply with one-layer neural network.

Original: 3. P OST - HOC C ORRECTION F ORMALISED FROM A N OT P ERSPECTIVE
Revised: 3. P OST - HOC C ORRECTION F ORMALISED FROM A N OT P ERSPECTIVE

Original: We have stated already: statistical post-hoc correction methods, typically Logit Adjustment, apply adjustment at the individual sample level.
Revised: Since Logit Adjustment applies adjustment at the individual sample level.

Original: This does not, after all, assure that the marginal distribution of the overall data set fulﬁls our desired distribution.
Revised: It doesn’t assure that the marginal distribution of the overall data set fulﬁls our desired distribution.

Original: In this respect, we clearly put the constraint into an equation:
Revised: In this respect, we clearly put the constraint into an equation:

Original: [Equation]
Revised: [Equation]

Original: where Y ∈ R N × K indicates the reﬁned prediction value in matrix form, c gets redeﬁned here with the meaning of the expected distribution on the test set.
Revised: where Y ∈ R N × K indicates the reﬁned prediction value in matrix form, µ represents the expected distribution on the test set.

Original: Alternatively, it is desirable to preserve another characteristic of Y , namely remaining almost as similar to the original prediction as possible.
Revised: Alternatively, it is desirable to preserve another characteristic of Y , namely, remaining almost as similar to the original prediction as possible.

Original: We consider inner-product based similarity to measure the similarity, which is a straightforward yet useful similarity measure.
Revised: We consider inner-product based similarity to measure the similarity, which is a straightforward yet useful similarity measure.

Original: In mathematical terms, this can be expressed as follows:
Revised: In mathematical terms, this can be expressed as follows:

Original: [Equation]
Revised: [Equation]

Original: where ˆ Z represents the original prediction in matrix form. C ( · ) may be some simple function, like Logarithmic function log( z ) , exponential function z α .
Revised: where ˆ Z represents the original prediction in matrix form, C ( · ) denotes to perform some transformation to ˆ Z , it can be some simple function, like Logarithmic function log( z ) , exponential function z α .

Original: Here we select − log( · ) as the cost function.
Revised: Here we select − log( · ) as the cost function.

Original: This choice was driven by the requirement that the cost matrix must be positive deﬁnite, whereas the transformation of the original prediction by − log( · ) satisﬁes this condition, in addition, as log likelihood represents the local probability density of the associated samples, it can also be used to substitute ˆ Z for the similarity approximation.
Revised: This choice was driven by the requirement that the cost matrix must be positive deﬁnite, whereas the transformation of the original prediction by − log( · ) satisﬁes this condition, in addition, as log likelihood represents the local probability density of the associated samples, it can also be used to substitute ˆ Z for the similarity approximation.

Original: In brief, the resulting numerical form can be put in formal terms as follows:
Revised: In brief, the resulting numerical form can be put in formal terms as follows:

Original: [Equation]
Revised: [Equation]

Original:  Extra constraint on Y are imposed simply cos the tuned estimation has to fulﬁl the basic probabilistic requirement that its sum is one. 
Revised: subject to [Equation] Extra constraint on Y are imposed simply cos the tuned estimation has to fulﬁl the basic probabilistic requirement that its sum is one. Comparing Eq.

Revised: (9-10) with Eq.

Revised: (4), we can see that if we substitute

Revised: P with Y and substitute r and c with 1 N and µ respectively, we ﬁnd that the above optimization problem is actually a special case of OT.

Original: In preliminaries, the entropy regularised OT is introduced (EOT), by adding entropy regularisation to
Revised: In preliminaries, the entropy regularised OT is introduced (EOT), by adding entropy regularisation to

Original: OT, the given equation can be solved efﬁciently by Sinkhorn algorithm.
Revised: OT, the given equation can be solved efﬁciently by Sinkhorn algorithm.

Original: Speciﬁcally, the optimisation problem is:
Revised: Speciﬁcally, the optimisation problem is:

Original: [Equation]
Revised: [Equation]

Original: The associated algorithmic ﬂow is outlined in detail in Algorithm 1.
Revised: The associated algorithmic ﬂow is outlined in detail in Algorithm 1.

Original: Algorithm 1:
Revised: Algorithm 1:

Original: Solve OT-related algorithm efﬁciently in the post-hoc correction via Sinkhorn
Revised: Solve OT-related algorithm efﬁciently in the post-hoc correction via Sinkhorn

Original: Algorithm.
Revised: Algorithm.

Revised: Input

Revised: -off parameter λ

Original: Input: Cost matrix M = − log ˆ Z , trade-off parameter λ , max number of iterations NT , iteration numbert , error threshold (cid:15) , current error δ, two probability vectors r and c , |·
Revised: number of iterations N iteration number current error δ two probability vectors r and c , |·

Original: | denotes the vector norm.
Revised: denotes the vector norm.

Original: Result: Reﬁned predictions Y
Revised: Result: Reﬁned predictions Y

Original:  Assigning λ with 1, it is observed that we equate our objective function to the KL divergence, thus illustrating the extensive nature of ourapproach, where the KL divergence can be regarded as a special case of our approach.
Revised: T Assigning λ with 1, it is observed that we equate our objective function to the KL divergence, thus illustrating the extensive nature of our approach.

Original: DARP (Kim et al., 2020a) has previously applied it to  semi-supervised classiﬁcation.
Revised: DARP (Kim et al., 2020a) has previously applied it to long-tailed semi-supervised classiﬁcation.

Revised: In addition, our method is better suited to situations where there is a large number of sample predictions to adjust, which is very common in the real world.

Revised: In the case of a very small number of samples, Logit Adjustment is more appropriate to be adopted.

Revised: [Figure]

Original: 3. C OST F UNCTION L EARNING VIA L INEAR M APPING
Revised: 3. C OST F UNCTION L EARNING VIA L INEAR M APPING

Original: Simple functionsas cost matrix are naive, as the cost function is likely to be sub-optimal for the real data sets, this suggests the design of a better cost function to better ﬁt the long-tailed recognition problem.
Revised: Simple functions is likely to be sub-optimal for the real data sets; this suggests the design of a better cost function to better ﬁt the long-tailed recognition problem.

Original: Despite this, manually designed cost functions require expert knowledge in different domains.
Revised: However, manually designed cost functions require expert knowledge in different domains.

Original: Thus, we propose to use a linear mapping to automatically learn the cost function, which relieves the need to conﬁguration.
Revised: Thus, we propose to use a linear mapping to automatically learn the cost function, which relieves the need to conﬁguration .

Original: [Figure] More speciﬁcally, for predictions ˜ Z generated by Softmax operation via leveraging linear transformation matrix W , W ∈ R K × K is learned so that the following objective function is minimised. 
Revised: More speciﬁcally, for predictions ˜ Z generated by Softmax operation via leveraging linear transformation matrix W , W ∈ R K × K is learned so that the following objective function is minimised. [Equation]

Original: [Equation]
Revised: [Equation]

Original: The resulting formula can be illustrated using a simple feed-forward layered network of weight parameter W , together with an error function in Eq.
Revised: The resulting formula can be illustrated using a simple one-layer network of weight parameter W , together with an error function in Eq.

Original: By normalising each row of W , each element can be understood as the probability value of confusing two classes, and somehow as a transition matrix between classes .
Revised: By normalising each row of W , each element can be understood as the probability value of confusing two classes, and somehow as a transition matrix between classes .

Original: We initialise W with an identity matrix and use small learning rate to learn W .
Revised: We initialise W with an identity matrix and use small learning rate to learn W .

Original: One could also absorb the Logit Adjustment term − log p ( y s ) + log p ( y  ) as the ﬁxed bias parameter into the network.
Revised: One could also absorb the Logit Adjustment term − log p ( y s ) + log p ( y t ) as the ﬁxed bias parameter into the network.

Original: Motivated by this description, we can use a general gradient descent algorithm, such as SGD, to optimise the error function.
Revised: Motivated by this description, we can use a general gradient descent algorithm, such as SGD, to optimise the error function.

Original: Take into account the implementation in practice, the direct calculation of Eq.
Revised: Take into account the implementation in practice, the direct calculation of Eq.

Original: (11) can be done using the Sinkhorn iterations (Cuturi, 2013; Frogner et al., 2015; Peyré et al., 2019) in mini-batch training efﬁciently.
Revised: (13) can be done using the Sinkhorn iterations (Cuturi, 2013; Frogner et al., 2015; Peyré et al., 2019) in mini-batch training efﬁciently.

Original: Besides, We term the proposed method as OTLM ( O ptimal T ransport via L inear M apping ), Figure 1 illustrates the overview of OTLM.
Revised: Besides, We term the proposed method as OTLM ( O ptimal T ransport via L inear M apping ), Figure 1 illustrates the overview of OTLM.

Original: Compute the gradients w.r.t W
Revised: Compute the gradients w.r.t W

Original: With the optimisation in Algorithm 1, it was conducted on the overall data set with a comparatively large number of data.
Revised: With the optimisation in Algorithm 1, it was conducted on the overall data set with a comparatively large number of data.

Original: Quite a different scenario now, as mini-batch training is more favoured when it comes to neural networks, since limited by the memory size of the GPU.
Revised: Quite a different scenario now, as minibatch training is more favoured when it comes to neural networks.

Original: For this reason, the optimisation workﬂow in Algorithm 1, as pointed by Peyré et al.
Revised: For this reason, the optimisation workﬂow in Algorithm 1, as pointed by Peyré et al.

Original: (2019); Viehmann (2019), will have a problem of batch stablisation.
Revised: (2019); Viehmann (2019), will have a problem of batch stablisation.

Original: To this end, we perform logarithmic transformation to step 3-4 in Algorithm 1.
Revised: To this end, we perform logarithmic transformation to step 3-4 in Algorithm 1.

Original: [Equation]
Revised: [Equation]

Original:  with log-sum-exp operation logsumexp  x  = log( (cid:80) i exp x i ) .
Revised: (15) with log-sum-exp operation logsumexp ( x ) = log( (cid:80) i exp x i ) .

Original: Assuming convergence is achieved in the Sinkhorn’s loops, it is not necessary for us manually to compute the derivative of the loss function in Eq.
Revised: Assuming convergence is achieved in the Sinkhorn’s loops, it is not necessary for us manually to compute the derivative of the loss function in Eq.

Original: (11) w.r.t W .
Revised: (13) w.r.t W .

Original: As a matter of fact, as long as we make sure that the entire optimisation process is differentiable. It may be tempting to fall back on the automatic differentiation machine available from modern libraries; as it allows use to differentiate between the results of Sinkhorn’s loops as a mere composition of elementary operations.
Revised: As a matter of fact, as long as we make sure that the entire optimisation process is differentiable, modern deep learning libraries can then automatically implement end-toend derivations, as it allows us to differentiate between the results of Sinkhorn’s loops as a mere composition of elementary operations.

Original: For speciﬁc implementation, we document the pseudo-code in the appendix.

Original: Performance degradation caused by imbalanced distributions can be addressed with meta-learning based methods (Ren et al., 2020; Wang et al., 2020a; Li et al., 2021).
Revised: Performance degradation caused by imbalanced distributions can be addressed with meta-learning based methods (Ren et al., 2020; Wang et al., 2020a; Li et al., 2021).

Original: Recent works have illustrated neural networks can learn more meaningful representations from the validation data set { ( x vn , y vn ) } N v n =1 .
Revised: Recent works have illustrated neural networks can learn more meaningful representations from the validation data set { ( x vn , y vn ) } N v n =1 .

Original: Inspired by the methods based on meta learning, Here we also take a validation data set to optimise the parameter W , but with the still signiﬁcant difference in that we require no labelling information, thus avoiding a large expanse of labeling samples.
Revised: Inspired by the methods based on meta learning, Here we also take a validation data set to optimise the parameter W , but with the still signiﬁcant difference in that we require no labelling information, thus avoiding a large expanse of labeling samples .

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we ﬁrst conduct experiments comparing our approach versus extant post-hoc correction methods on three data sets, including CIFAR-100-LT (Cao et al., 2019), ImageNet-LT (Liu et al., 2019), and iNaturalist (Van Horn et al., 2018) with varying backbones.
Revised: In this section, we ﬁrst conduct experiments comparing our approach versus extant post-hoc correction methods on three data sets, including CIFAR-100-LT (Cao et al., 2019), ImageNet-LT (Liu et al., 2019), and iNaturalist (Van Horn et al., 2018) with varying backbones.

Original: It is further demonstrated that OTLM delivers signiﬁcant further performance improvements.

Original: Finally, we empirically make a comparison of our algorithm with alternative cutting-edge long-tailed recognition methods, and the results suggest that equivalent or even better results are achieved by our approach.
Revised: Finally, we empirically make a comparison of our algorithm with alternative cutting-edge long-tailed recognition methods.

Original: Observe that, for one thing, a plausible coupling of our algorithm to any training-aware long-tailed recognition method exists.
Revised: Observe that, for one thing, a plausible coupling of our algorithm to any training-aware long-tailed recognition method exists .

Original: As an illustration of the potency and strong generalisation of our approach, we conducted post-hoc correction of the pseudo-predictions of both methods, which are fairly typical for the training stage of all data sets: The ﬁrst is based on the cross entropy (CE) , and the other is the recently proposed RIDE (Wang et al., 2020b), which is based on multi experts.
Revised: As an illustration of the potency and strong generalisation of our approach, we conducted post-hoc correction of the pseudo-predictions of both methods, which are fairly typical for the training stage of all data sets: The ﬁrst is based on the cross entropy (CE) , and the other is the recently proposed RIDE (Wang et al., 2020b), which is based on multi experts.

Original: 4. D ATA S ETS AND B ASELINES
Revised: 4. D ATA S ETS AND B ASELINES

Original: The baseline methods and data set are brieﬂy described here, with implementation details placed in the appendix.
Revised: The baseline methods and data set are brieﬂy described here, with implementation details placed in the appendix.

Original: Baselines : We compare our methods with (i) post-hoc correction methods including Logit Adjustment (Menon et al., 2020), DARP (Kim et al., 2020a), (ii) state-of-the-art methods including Focal
Revised: Baselines : We compare our methods with (i) post-hoc correction methods including Logit Adjustment (Menon et al., 2020), DARP (Kim et al., 2020a), (ii) state-of-the-art methods including Focal

Original: Loss (Lin et al., 2017), LDAM (Cao et al., 2019), BBN (Zhou et al., 2020), Balanced Softmax (Ren et al., 2020), Causal Norm (Tang et al., 2020), LADE (Hong et al., 2020), M2m (Kim et al., 2020b),
Revised: Loss (Lin et al., 2017), LDAM (Cao et al., 2019), BBN (Zhou et al., 2020), Balanced Softmax (Ren et al., 2020), Causal Norm (Tang et al., 2020), LADE (Hong et al., 2020), M2m (Kim et al., 2020b),

Original: Decouple (Kang et al., 2019), LFME (Xiang et al., 2020), RIDE (Wang et al., 2020b)
Revised: Decouple (Kang et al., 2019), LFME (Xiang et al., 2020), RIDE (Wang et al., 2020b)

Original: Long-tailed data set: We take experiments on three data sets including CIFAR-100-LT, ImageNetLT, and iNaturelist.
Revised: Long-tailed data set: We take experiments on three data sets including CIFAR-100-LT, ImageNetLT, and iNaturelist.

Original: We build the imbalance version of CIFAR-100 by downsampling samples per class following the proﬁle in Liu et al.
Revised: We build the imbalanced version of CIFAR-100 by downsampling samples per class following the proﬁle in Liu et al.

Original: ; Kang et al.
Revised: ; Kang et al.

Original: (2019) with imbalanced ratio 10, 50, and 100.
Revised: (2019) with imbalanced ratio 10, 50, and 100.

Original: For all the benchmarks, we evaluate the performance on the test data set using a model trained on the training data set, and report the results using top-1 accuracy. 
Revised: For all the benchmarks, we evaluate the performance on the test data set using a model trained on the training data set, and report the results using top-1 accuracy. [Table] CE 62. [Table]

Original: 4. M AIN R ESULTS
Revised: [Table] [Table] [Table] [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: 4. M AIN R ESULTS

Original: Table 1 illustrates the results on the CIFAR-100-LT data set.
Revised: Table 1 illustrates the results on the CIFAR-100-LT data set.

Original: For CE-based training-aware methods, it ﬁrst shows that Logit Adjustment is indeed a simple yet effective method for post-hoc correction.
Revised: For CE-based training-aware methods, it ﬁrst shows that Logit Adjustment is indeed a simple yet effective method for post-hoc correction.

Original: Remarkably, it outperforms the baseline methods by 2.6%, 4.1%, and 4.1% under the imbalance ratio of 10, 50, and 100 respectively.
Revised: Remarkably, it outperforms the baseline methods by 2.6%, 4.1%, and 4.1% under the imbalance ratio of 10, 50, and 100 respectively.

Original: With a quick look at the results based on the OT approach, the superior results stress the advantages of our approach over the DARP and Logit adjustments.OT
Revised: With a quick look at the results based on the OT approach, the superior results stress the advantages of our approach over the DARP and Logit adjustments.

Original: [Table]

Original:  algorithm can further improve the performance by 0.62%, 0.43%, 0.70%, for OTLM, it outperforms
Revised: OT algorithm can further improve the performance by 0.62%, 0.43%, 0.70%, for OTLM, it outperforms

Original: Logit Adjustment by 0.55%, 0.73%, 0.99% under the three imbalance ratios on CIFAR-100-LT data set.
Revised: Logit Adjustment by 0.55%, 0.73%, 0.99% under the three imbalance ratios on CIFAR-100-LT data set.

Original: For RIDE-based training-aware methods, there is also a 0.6% improvement in the accuracy of our method.
Revised: For RIDE-based training-aware methods, there is also a 0.6% improvement in the accuracy of our method.

Original: Table 1 also provides the results on ImageNet-LT and iNaturelist data sets.
Revised: Table 1 also provides the results on ImageNet-LT and iNaturelist data sets.

Original: The fact again attracts our attention that for CE-based training-aware methods, OT, which is based on convex optimisation, consistently outperform Logit Adjustment by a large margin (1.76%) on iNaturelist data set.
Revised: The fact again attracts our attention that for CE-based training-aware methods, OT, which is based on convex optimisation, consistently outperform Logit Adjustment by a large margin (1.76%) on iNaturelist data set.

Original: The accuracy boosting indicates the great potential of methods based on optimisation for post-hoc correction.
Revised: The accuracy boosting indicates the great potential of methods based on optimisation for post-hoc correction.

Original: Besides, not surprisingly, OTLM can further enhance the prediction accuracy, it outperforms
Revised: Besides, not surprisingly, OTLM can further enhance the prediction accuracy, it outperforms

Original: Logit Adjustment by 0.73% and 2.86% on ImageNet-LT and iNaturelist respectively.
Revised: Logit Adjustment by 0.73% and 2.86% on ImageNet-LT and iNaturelist respectively.

Original: As for RIDE, the gain in accuracy is also signiﬁcant, about 3%.
Revised: As for RIDE, the gain in accuracy is also signiﬁcant, about 3% on iNaturelist.

Original: 4. C OMPARISON WITH THE S TATE - OF - THE - ART M ETHODS
Revised: 4. C OMPARISON WITH THE S TATE - OF - THE - ART M ETHODS

Original: Armed with a demonstration of the validity of OT and OTLM, we switched to a comparison of our performance with existing methods that achieved the most advanced results on three benchmarks.
Revised: Armed with a demonstration of the validity of OT and OTLM, we switched to a comparison of our performance with existing methods that achieved the most advanced results on three benchmarks.

Original: As shown in Table 2, in particular, on ImageNet-LT and iNaturelist data sets, the experimental results are remarkable and impressive, our method outperformsby RIDE by 1.0% and 3.0%, respectively.
Revised: As shown in Table 2, in particular, on ImageNet-LT and iNaturelist data sets, the experimental results are remarkable and impressive, our method outperforms RIDE by 1.0% and 3.0%, respectively.

Original: Since iNaturalist is a very difﬁcult and ﬁne-grained data set consisting of 8,142 categories.
Revised: Since iNaturalist is a very difﬁcult and ﬁne-grained data set consisting of 8,142 categories.

Original: These huge performance gains come at a fraction of the cost, as we show in the next subsection.
Revised: These huge performance gains come at a fraction of the cost , as we show in the next subsection.

Original: [Table]

Original: iNaturelist

Original: Imabalced Ratio   − −

Original: Cross Entropy (CE) 59.

Original: Focal Loss 55.

Original: − −

Original: CB-Focal − − − 43.

Original: OLTR − − − 41.

Original: NCM − − − 47.

Original: norm − − − 49.

Original: cRT − − − 49.

Original: LWS − − − 49.

Original: LDAM − − − − 64.

Original: LDAM+DRW 58.

Original: − 68.

Original: BBN 59.

Original: − 66.

Original: Causal Norm 59.

Original: M2m 58.

Original: − 43.

Original: − −

Original: LFME − − 43.

Original: − −

Original: Balanced Softmax 61.

Original: − −

Original: LADE 61.

Original: RIDE (6 experts) − − 50.

Original: Logit Adjustment 61.

Original: DAPR 61.

Original: OT+CE 62.3 +0.

Original: OTLM+CE 62.

Original: 50.4 +0.

Original: OT+RIDE (6 experts) − − 50.8 +0.

Original: [Table]

Original: 4. C OMPUTATION C OST OF OT AND OTLM
Revised: 4. C OMPUTATION C OST OF OT AND OTLM

Original: As we have highlighted, the additional computational cost of OT and OTLM is particularly small, compared to that of training-aware methods.
Revised: As we have highlighted, the additional computational cost of OT and OTLM is particularly small, compared to that of training-aware methods.

Original: In Table 3, exact time of the evaluations on the ImageNetLT and iNaturelist data sets is provided.
Revised: In Table 3, exact time of the evaluations on the ImageNetLT and iNaturelist data sets is provided.

Original: Except for OTLM, which was run on an NVidia card (V100),[Table] the results come from a 28-core machine (2.20 Ghz Xeon).
Revised: Except for OTLM, which was run on an NVidia card (V100), the results come from a 28-core machine (2.20 Ghz Xeon).

Original: Firstly, we can observe that all post-hoc correction methods here are not quite time-consuming.
Revised: Firstly, we can observe that all post-hoc correction methods here are not quite time-consuming .

Original: For comparison, ImageNet-LT and iNaturelist can be trained on 4 NVidia cards (V100) for extremely long periods of time.
Revised: For comparison, ImageNet-LT and iNaturelist can be trained on 4 NVidia cards (V100) for extremely long periods of time.

Original: Using ResNet-50-32x4d on the iNaturelist training data , for example, with a batch size of 256, the amount of training time (in seconds) to perform 1 iteration is approximately 850.
Revised: Using ResNet-50-32x4d on the iNaturelist training data set, for example, with a batch size of 256, the amount of training time (in seconds) to perform 1 iteration is approximately 850.

Original: Also, if one differs the performance of each method, we gather that OT and OTLM provide the best balance of performance and running costs to match.
Revised: Also, if one differs the performance of each method, we gather that OT and OTLM provide the best balance of performance and running costs to match .

Original: results indicate that OT and OTLM are an efﬁcient post- correction methods.

Original: 4. C ONVERGENCE B EHAVIOR OF OTLM
Revised: 4. C ONVERGENCE B EHAVIOR OF OTLM

Original: To verify that our adopted OTLM algorithm is really efﬁcient in resolving the optimisation issue.
Revised: To verify that our adopted OTLM algorithm is really efﬁcient.

Original: With ResNet-50 as the backbone, we plotted curves depicting the decreasing absolute error σ with the number of iterations t on the ImageNet-LT validation data set.
Revised: With ResNet-50 as the backbone, we plotted curves depicting the decreasing absolute error σ with the number of iterations t on the ImageNet-LT validation data set.

Original: Image on the left in Figure 2 gives us a visualisation of the optimisation process.
Revised: Image on the left in Figure 2 gives us a visualisation of the optimisation process.

Original: It is found that the absolute error σ converges at around 60 iterations.
Revised: It is found that the absolute error σ converges at around 60 iterations.

Original: In fact, we discovered that 10 or even fewer iterations were enough to yield satisfactory performance.
Revised: In fact, we discovered that 10 or even fewer iterations were enough to yield satisfactory performance.

Original: OTLM to minimise the difference between the ideal and actual distributions.

Original: For ease of display, the values of the diagonal elements are relatively large and we will replace these values with zeros.

Original: 4. D ETAILED A NALYSIS ON D ISTRIBUTION M ATCHING
Revised: 4. D ETAILED A NALYSIS ON D ISTRIBUTION M ATCHING

Original: Upon distribution ﬁtting, preferably the estimation bias is lessened, which implies that the KL divergence distance between the reﬁned prediction and the target distribution is decreased.
Revised: Upon distribution ﬁtting, preferably the estimation bias is lessened, which implies that the KL divergence distance between the reﬁned prediction and the target distribution is decreased.

Original: Tabledisplays the KL distances on the ImageNet-LT and iNaturelist data sets, and we can ﬁnd that OT possesses a smaller KL distance than Logit Adjustment, which suggests that OT decreases the estimation bias more than Logit Adjustment by a much larger margin. 
Revised: Tabledisplays the KL distances on the ImageNet-LT and iNaturelist data sets, and we can ﬁnd that OT possesses a smaller KL distance than Logit Adjustment, which suggests that OT decreases the estimation bias more than Logit Adjustment by a much larger margin. [Table]

Original: [Table]
Revised: [Table]

Original:  V ISUALISATION ON W EIGHT P ARAMETER IN OTLM
Revised: 4. V ISUALISATION ON W EIGHT P ARAMETER IN OTLM

Revised: As a natural consequence

Original: As a natural consequence, one wonders about the exact structure of this weight matrix W , and no better way to present its results than through visualisation.
Revised: [Table] one wonders about the exact structure of this weight matrix W [Table] and no better way to present its results than through visualisation.

Original: Towards this end, we applied OTLM to reﬁne the original predictions on CIFAR-100 data set with an imbalanced ratio R = 100 and
Revised: Towards this end [Table] we applied OTLM to reﬁne the original predictions on CIFAR-100 data set with an imbalanced ratio R = 100 and

Original: ResNet-32 backbone, and trained a one-layer model until it converged to extract the value of W , where the shape of is 100 × 100 .
Revised: ResNet-32 backbone [Table] and trained a one-layer model until it converged to extract the value of W , where the shape of is 100 × 100 .

Original: We visualised a sub-block of size 20 × 20 , the illustration is presented on the right image in Figure 2.
Revised: We visualised a sub-block of size 20 × 20 , the illustration is presented on the right image in Figure 2.

Original: The very physical meaning of the elements in W can be  interpreted as the probability of transition between classes.
Revised: The very physical meaning of the elements in W can be [Table] interpreted as the probability of transition between classes.

Original: The visualised matrix is very dense, this non-sparsity suggests that the OTLM manages to correct the predictions of the model in the training stage, whilst minimising the distance between the desired and the actual distribution.
Revised: The visualised matrix is very dense, this non-sparsity suggests that the OTLM manages to correct the predictions of the model in the training stage, whilst minimising the distance between the desired and the actual distribution.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Long-tailed recognition:
Revised: Long-tailed recognition:

Original: The great majority of current methods applied to long-tailed recognition can be divided into either training-aware and post-hoc correction methods.
Revised: The great majority of current methods applied to long-tailed recognition can be divided into either training-aware and post-hoc correction methods.

Original: The training-aware methods aim to improve the generalisation ability of models trained on imbalanced training data set.
Revised: The training-aware methods aim to improve the generalisation ability of models trained on imbalanced training data set.

Original: Among them, re-sampling and re-weighting are two major methods.
Revised: Among them, re-sampling and re-weighting are two major methods.

Original: Re-sampling controls the class numbers by the means of over-sampling (Chawla et al., 2002) and under-sampling (Barandela et al., 2004; Drummond et al., 2003).
Revised: Re-sampling controls the class numbers by the means of over-sampling (Chawla et al., 2002) and under-sampling (Barandela et al., 2004; Drummond et al., 2003).

Original: The re-weighting approaches achieve balanced learning by giving smaller weights to majority classes and larger weights to minority classes (Tang et al., 2008; Zadrozny et al., 2003; Lin et al., 2017; Lee et al., 2017;
Revised: The re-weighting approaches achieve balanced learning by giving smaller weights to majority classes and larger weights to minority classes (Tang et al., 2008; Zadrozny et al., 2003; Lin et al., 2017; Lee et al., 2017;

Original: Cui et al., 2019; Khan et al., 2019).
Revised: Cui et al., 2019; Khan et al., 2019).

Original: Other approaches have also been proposed to resolve the imbalanced problem.
Revised: Other approaches have also been proposed to resolve the imbalanced problem.

Original: For example, Cao et al.
Revised: For example, Cao et al.

Original: introduces a theoretically-principled loss function motivated by minimizing the generalisation error bound (Cao et al., 2019), Kang et al.
Revised: introduces a theoretically-principled loss function motivated by minimizing the generalisation error bound (Cao et al., 2019), Kang et al.

Original: proposes to decouple the learning phases into representation learning and classiﬁer tuning (Kang et al., 2019).
Revised: proposes to decouple the learning phases into representation learning and classiﬁer tuning (Kang et al., 2019).

Original: Apart from training-aware methods, post-hoc correction also offers us an avenue for long-tailed recognition problem, it can be applied to the normalisation of classiﬁer weights (Kang et al., 2019; Zhang et al., 2019; Kim & Kim, 2020) or logits based on label frequency (Provost, 2000;
Revised: Apart from training-aware methods, post-hoc correction also offers us an avenue for long-tailed recognition problem, it can be applied to the normalisation of classiﬁer weights (Kang et al., 2019; Zhang et al., 2019; Kim & Kim, 2020) or logits based on label frequency (Provost, 2000;

Original: Collell et al., 2016).
Revised: Collell et al., 2016).

Original: Among them, Logit Adjustment (Menon et al., 2020) is a simple yet powerful methods to use.
Revised: Among them, Logit Adjustment (Menon et al., 2020) is a simple yet powerful methods to use.

Original: Compared with Logit Adjustment, our approach perform the post-hoc correction from the optimisation view.
Revised: Compared with Logit Adjustment, our approach perform the post-hoc correction from the optimisation view.

Original: Optimal transport: OT distances is a important family of distances for probability measures (Villani, 2008; Santambrogio, 2015).
Revised: Optimal transport: OT distances is a important family of distances for probability measures (Villani, 2008; Santambrogio, 2015).

Original: It has been extensively applied in different ﬁelds in machine learning.
Revised: It has been extensively applied in different ﬁelds in machine learning.

Original: Courty et al.
Revised: Courty et al.

Original: (Courty et al., 2016) proposes a regularised unsupervised optimal transportation model to perform the domain adaption.
Revised: (Courty et al., 2016) proposes a regularised unsupervised optimal transportation model to perform the domain adaption.

Original: Other applications include 3D shape matching (Su et al., 2015), generative model (Arjovsky et al., 2017; Salimans et al., 2018; Bunne et al., 2019; Deshpande et al., 2019; Bhagoji et al., 2019), graph matching (Xu et al., 2019a;b), model designs Kandasamy et al.
Revised: Other applications include 3D shape matching (Su et al., 2015), generative model (Arjovsky et al., 2017; Salimans et al., 2018; Bunne et al., 2019; Deshpande et al., 2019; Bhagoji et al., 2019), graph matching (Xu et al., 2019a;b), model designs Kandasamy et al.

Original: ; Chizat & Bach (2018).
Revised: ; Chizat & Bach (2018).

Original: To the best of our knowledge, we are the ﬁrst to perform the post-hoc correction from the OT perspective.
Revised: To the best of our knowledge, we are the ﬁrst to perform the post-hoc correction from the OT perspective.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we present two techniques for post-hoc corrections from an optimisation perspective, which go beyond previous research directions from a statistical point of view.
Revised: In this paper we present two techniques for post-hoc corrections from an optimisation perspective, which go beyond previous research directions from a statistical point of view.

Original: Our approach reviews the basic mathematical formulation of the distribution alignment problem and relates the resulting formulation to OT, which we ﬁnd, moreover, offers greater possibilities and ﬂexibility to solve the problem.
Revised: Our approach reviews the basic mathematical formulation of the distribution alignment problem and relates the resulting formulation to OT, which we ﬁnd, moreover, offers greater possibilities and ﬂexibility to solve the problem.

Original: In order to avoid manual conﬁguration of the cost matrix, we propose to learn the cost matrix automatically by linear mapping, which leads to further performance improvements.
Revised: In order to avoid manual conﬁguration of the cost matrix, we propose to learn the cost matrix automatically by linear mapping, which leads to further performance improvements.

Original: The experimental results conﬁrm the superiority of our proposed approach.
Revised: The experimental results conﬁrm the superiority of our proposed approach.

Original: I believe our work can open up more possibilities and imagination for post-hoc corrections.
Revised: I believe our work can open up more possibilities and imagination for post-hoc corrections

Original: A O PTIMAL T RANSPORT ON L ONG - TAILED S EMI - SUPERVISED C LASSIFICATION
Revised: A O PTIMAL T RANSPORT ON L ONG - TAILED S EMI - SUPERVISED C LASSIFICATION

Original: Together, with the extension of our approach, we adapt it to semi-supervised long-tailed classiﬁcation tasks.
Revised: Together, with the extension of our approach, we adapt it to semi-supervised long-tailed classiﬁcation tasks.

Original: We bring our approach together with the advanced semi-supervised learning methods
Revised: We bring our approach together with the advanced semi-supervised learning methods

Original: FixMatch Sohn et al.
Revised: FixMatch Sohn et al.

Original: Adhering to the conﬁguration in DARP Kim et al.
Revised: Adhering to the conﬁguration in DARP Kim et al.

Original: (2020a), we also create synthetically long-tailed variants of CIFAR-100 with imbalanced ratio R = 20 .
Revised: (2020a), we also create synthetically long-tailed variants of CIFAR-100 with imbalanced ratio R = 20 .

Original: The sample size of the unlabelled data is twice as large as the labelled data, with the same proportion of imbalance between them.
Revised: The sample size of the unlabelled data is twice as large as the labelled data, with the same proportion of imbalance between them.

Original: To assess the performance, we report two popular metrics: balanced accuracy (bACC)
Revised: To assess the performance, we report two popular metrics: balanced accuracy (bACC)

Original: Huang et al.
Revised: Huang et al.

Original: and geometric mean scores (GM) Branco et al.
Revised: and geometric mean scores (GM) Branco et al.

Original: Experimentally, with the exception of an OT to align the marginal distribution of the unlabelled data, all settings are consistent with DARP, and we set λ to 0.005.
Revised: Experimentally, with the exception of an OT to align the marginal distribution of the unlabelled data, all settings are consistent with DARP, and we set λ to 0.005.

Original: The results are reported in Table 5.
Revised: The results are reported in Table 5.

Original: It can be seen that our approach also achieves much better performance on semi-supervised learning tasks.
Revised: It can be seen that our approach also achieves much better performance on semi-supervised learning tasks.

Original: [Table]
Revised: [Table]

Revised: B P ARAMETER S TUDY OF λ

Revised: We conduct an additional experiment on iNaturelist data set to study the effect of λ on the ﬁnal performance.

Revised: The value of λ is selected from { 0 . 01 , 05 , 0 . 1 , 1 . 0 . 2 . 0 , 5 . 0 } , all other experimental settings are the same, including various hyperparameters, network structure, etc.

Revised: As illustrated in

Revised: Figure 3, we can ﬁnd that OT achieves the best performance when λ

Revised: = 1 .

Original: B I MPLEMENTATION D ETAILS
Revised: C I MPLEMENTATION D ETAILS

Original: The speciﬁc implementation details for each data set under the different methods are described below.
Revised: The speciﬁc implementation details for each data set under the different methods are described below.

Original: B. I MPLEMENTATION D ETAILS OF C ROSS E NTROPY
Revised: C. I MPLEMENTATION D ETAILS OF C ROSS E NTROPY

Original: CIFAR-100-LT : During the training stage, following the recipe of (Tang et al., 2020; Hong et al., 2020), we apply SGD with batch size 256 and weight decay 0.0005 to train a ResNet-32 (He et al., 2016) model for 200 epochs, we employ the linear warm-up learning rate schedule for the ﬁrst ﬁve epochs.
Revised: CIFAR-100-LT : During the training stage, following the recipe of (Tang et al., 2020; Hong et al., 2020), we apply SGD with batch size 256 and weight decay 0.0005 to train a ResNet-32 (He et al., 2016) model for 200 epochs, we employ the linear warm-up learning rate schedule for the ﬁrst ﬁve epochs.

Original: We also set the base learning rate to 0.2 and reduce it at epoch 120 and 160 by a factor of 100.
Revised: We also set the base learning rate to 0.2 and reduce it at epoch 120 and 160 by a factor of 100.

Original: During the post-hoc stage, we use Adam with batch size 10k and weight decay 0.001 to train the single layered feed-forward network for 20 iterations.
Revised: During the post-hoc stage, we use Adam with batch size 10k and weight decay 0.001 to train the single layered feed-forward network for 20 iterations.

Original: We set λ to 0.1, T to 200, ∆ to 0.001.
Revised: We set λ to 0.1, T to 200, ∆ to 0.001.

Original: The learning rate is constant and set to 0.001.
Revised: The learning rate is constant and set to 0.001.

Original: ImageNet-LT : During the training stage, following the implementation of (Hong et al., 2020), we train the ResNeXt-50-32x4d (Xie et al., 2017) for 90 epochs and perform a cosine learning rate scheme with an initial learning rate of 0.05.
Revised: ImageNet-LT : During the training stage, following the implementation of (Hong et al., 2020), we train the ResNeXt-50-32x4d (Xie et al., 2017) for 90 epochs and perform a cosine learning rate scheme with an initial learning rate of 0.05.

Original: SGD is also employed to optimise the neural network with batch size 256, weight decay 0.0005, and momentum 0.9.
Revised: SGD is also employed to optimise the neural network with batch size 256, weight decay 0.0005, and momentum 0.9.

Original: During the post-hoc stage, we use
Revised: During the post-hoc stage, we use

Original: SGD with batch size 10k, weight decay 0.001 to train the single layered feed-forward network for 100 iterations.
Revised: SGD with batch size 10k, weight decay 0.001 to train the single layered feed-forward network for 100 iterations.

Original: The constant learning rate is 0.2.
Revised: The constant learning rate is 0.2.

Original: We set λ to 1.2, T to 7, ∆ to 0.001.
Revised: We set λ to 1.2, T to 7, ∆ to 0.001.

Original: iNaturelist :
Revised: iNaturelist :

Original: During the training stage, ResNet-50 (He et al., 2016) is chosen as the backbone network.
Revised: During the training stage, ResNet-50 (He et al., 2016) is chosen as the backbone network.

Original: We use SGD with momentum 0.9, batch size 256 to train the network.
Revised: We use SGD with momentum 0.9, batch size 256 to train the network.

Original: We utilise the cosine learning rate schedule gradually decaying from 0.1 to 0.
Revised: We utilise the cosine learning rate schedule gradually decaying from 0.1 to 0.

Original: During the post-hoc stage, we use SGD with batch size 23,826, weight decay 0.001, λ to 1.0, ∆ to 0.001 and T to 10.
Revised: During the post-hoc stage, we use SGD with batch size 23,826, weight decay 0.001, λ to 1.0, ∆ to 0.001 and T to 10.

Original: The constant learning rate is set to 0.05.
Revised: The constant learning rate is set to 0.05.

Original: As iNaturelist has no test data set, we directly report the performance on validation data set.
Revised: As iNaturelist has no test data set, we directly report the performance on validation data set.

Original: B. I MPLEMENTATION D ETAILS OF RIDE
Revised: C. I MPLEMENTATION D ETAILS OF RIDE

Original: The training details we use in RIDE share the same with the original paper, there are 6 experts in our
Revised: The training details we use in RIDE share the same with the original paper, there are 6 experts in our

Original: RIDE on all three data sets.
Revised: RIDE on all three data sets.

Original: CIFAR-100-LT We experimented with R = 100 and ResNet-32 as the base network.
Revised: CIFAR-100-LT We experimented with R = 100 and ResNet-32 as the base network.

Original: SGD is utilised as our solver with momentum 0.9, batch size 128, and epoch 200.
Revised: SGD is utilised as our solver with momentum 0.9, batch size 128, and epoch 200.

Original: The learning rate is 
Revised: The learning rate is initialised as 0.1 and decayed by 0.01 at epoch 120 and 160 respectively.

Original: ImageNet-LT We take experiments with Re as backbone with 100 epochs.
Revised: ImageNet-LT We take experiments with Re as backbone with 100 epochs.

Original: We set the initial learning rate to 0.1 and reduce it by 10 at epoch 60 and 80.
Revised: We set the initial learning rate to 0.1 and reduce it by 10 at epoch 60 and 80.

Original: SGD with batch size 256 and momentum 0.9 is also adopted as the optimisation solver here.
Revised: SGD with batch size 256 and momentum 0.9 is also adopted as the optimisation solver here.

Original: iNaturelist We conduct experiments with ResNet-50 as backbone.
Revised: iNaturelist We conduct experiments with ResNet-50 as backbone.

Original: The training details are the same as ImageNet-LT except from batch size 512.
Revised: The training details are the same as ImageNet-LT except from batch size 512.