Class-Dependent Label-Noise Learning with Cycle-Consistency Regularization
Abstract
In label-noise learning, estimating the transition matrix plays an important role in building statistically consistent classifier.
Current state-of-the-art consistent estimator for the transition matrix has been developed under the newly proposed sufficiently scattered assumption, through incorporating the minimum volume constraint of the transition matrix T into label-noise learning.
To compute the volume of T , it heavily relies on the estimated noisy class posterior .
However, the estimation error of the noisy class posterior could usually be large as deep learning methods tend to easily overfit the noisy labels.
Then, directly minimizing the volume of such obtained T could lead the transition matrix to be poorly estimated.
How to reduce the side-effects of the inaccurate noisy class posterior remains unsolved.
In this paper, we creatively propose to estimate the transition matrix under a forward-backward cycle-consistency regularization, of which we have greatly reduced the dependency of estimating the transition matrix T on the noisy class posterior .
Extensive experimental results consistently justify the effectiveness of the proposed method, on reducing the estimation error of the transition matrix and greatly boosting the classification performance.
Introduction
Deep learning based algorithms rely heavily on large-scale annotated training data.
However, it is often extremely costly and sometimes even infeasible to accurately annotate such massive dataset [13].
An usual way to address this issue is to collect large-scale training data using cheap methods, e g., from the crowd-sourcing platform [36] or online query search engine [3], which could inevitably yield label noise.
Training deep model with label noise can significantly degrade the performance [13, 1, 28].
Therefore, how to mitigate the side-effects of the label noise automatically becomes very important, and it has drawn increasing attention recently.
Recent studies show that estimating the transition matrix plays an important role in building statistically consistent classifiers for label-noise learning, since the transition matrix can well model the noise generation process.
These methods guarantee that classifier learned from the noisy data could approach to the optimal classifier defined on the clean risk asymptotically as the size of the noisy training data increases [13].
The basic principle is that the clean class posterior P ( Y | X = x )can be inferred by the noisy class posterior P (¯ Y | X = x ) and the transition matrix T ( x ) , where
T ij ( x ) = P ( ¯ Y =
j | Y =
i, X = x ) , i.e., P (¯ Y | X = x ) =
T ( x ) P ( Y | X = x ) .
This paper focuses on the common class-dependent and instance-independent label noise, i.e., T ( x ) = T .
Usually, the transition matrix T is unidentifiable and hard to learn without additional assumptions [37, 5, 33].
In the literature, many methods try to estimate the transition matrix under the anchor point assumption, which assumes that there exist some instances belonging to a specific class almost surely [33, 16, 22].
However, the anchor point assumption can not be always satisfied [24, 39].
Therefore, methods aiming to develop statistically consistent classifiers without anchor points have been proposed [33, 13].
Among them, the state-of-the-art method (VolMinNet [13]) proposed by far the mildest sufficiently scattered assumption, where the anchor point assumption is its special case [13].
They proved that the optimal T is the one with minimum volume enclosing the noisy class posterior of training examples.
Hence, they proposed to incorporate the minimum volume constraint of T into label-noise learning.
To compute the volume of T , the existing method heavily relies on the estimated noisy class posterior .
However, the estimation error of the noisy class posterior could usually be large as the deep learning methods tend to easily overfit the noisy labels, especially under the settings of limited training samples [37].
This could lead the transition matrix to be poorly estimated when minimizing the volume calculated by using the learned noisy class posteriors.
Therefore, how to reduce the side-effects of the inaccurate noisy class posterior has become a major concern for making use of the sufficiently scattered assumption.
To address this issue, we propose a forward-backward cycle-consistency regularized algorithm to estimate the transition matrix T , of which we could greatly reduce the dependency of estimating T on using the noisy class posterior to calculate its volume, and further build a statistically consistent classifier.
Specifically, we show that minimizing the volume of the transition matrix T is equal to maximizing the volume of corresponding clean class posterior.
More importantly, maximizing the volume of the clean class posterior could avoid the side-effects of the inaccurate noisy class posterior probability on estimating T .
To be more specific, we will further exploit the equation P (¯ Y | X = x ) =
TP ( Y | X = x ) , where P (¯ Y | X = x ) ∈ R C ,
T ∈ R C × C , P ( Y |
X = x ) ∈ R C , and C is the number of classes.
Note that P (¯ Y | X = x ) can be estimated by exploiting the noisy data.
For simplicity, we rewrite the above equation as A = TB for the following derivation, where A and B represent
P (¯ Y | X = x ) and P ( Y | X = x ) , respectively.
We adopt the commonly used determinant operation to measure the volume of T , i.e., det( T ).
As only the square matrix has determinant, we have det( AA ⊤ )
=det( TBB ⊤ T ⊤ )
=det( TT ⊤ )det( BB ⊤ ).
As P (¯ Y | X = x ) , i.e., A , can be estimated by the given noisy data, the determinant det( AA ⊤ ) is constant.
Then, minimizing det( T ) (or det( TT ⊤ )) equals maximizing det( BB ⊤ ).
Therefore, we can replace the minimum volume constraint of T by the maximum volume constraint of the clean class posterior P
( Y | X = x ) .
Maximizing the volume of the clean class posterior P ( Y | X = x ) can be achieved without directly relying on the estimated noisy class posterior probability.
Specifically, we project the C -dimensional simplex, which has the largest volume , from the noisy class posterior space onto the clean class posterior space by a diagonally dominant column stochastic matrix T ′ [13].
Let denote the projected clean class posteriors as the backward clean class posteriors P ′
( Y | X = x ) .
They will make up the largest volume in the sense that they correspond to the C -dimensional simplex in the noisy class posterior space.
Afterwards, we guide the clean class posterior P ( Y | X = x ) learned by the neural network to match to P ′
( Y | X = x ) .
Therefore, the volume of the learned clean class posteriors could be maximized to encourage the transition matrix T to converge to its optimal solution.
Besides, we further build the cycle-consistency regularization between the forward and backward transition matrices T and T ′ to better learn the clean class posterior probability.
Our main contributions are summarized as follows: 1) We propose to estimate the transition matrix
T under a forward-backward cycle-consistency regularization, of which we could greatly reduce the dependency of minimizing the volume of the transition matrix T on the estimated noisy class posterior ; 2) We show that such cycle-consistency regularization could help to minimize the volume of the transition matrix T without directly exploiting the estimated noisy class posterior, which encourages the estimated transition matrix T to converge to the optimal solution; 3) Experimental results on four datasets (two synthetic and two real-world datasets) with different label-noise settings consistently justify the effectiveness of the proposed method, on reducing the estimation error of the transition matrix and greatly improving the classification performance.
S o f t m a x
Input Image
Backbone Network
Noisy Label
Cross Entropy
CrossEntropy
CrossEntropy ' T
T ( | ) P Y x
[Figure]
Y ( | ) P Y x
( | ) P Y x ' ( | ) P Y x
Estimating Transition Matrix for Label-Noise Learning
Problem Setting.
Let D be the distribution of a pair of random variables ( X , Y ) ∈
( X × Y ) , where
X and Y denote the variable of instances and the corresponding labels, X ∈ R d × N represents the instance feature space, d is the feature dimension, N is the total number of instances, Y ∈ { 1 , · · · , C } denotes the label space, and C is the size of the classes.
However, in some real-world scenarios, it is often expensive or even infeasible to accurately draw large-scale training data independently from the clean distribution D .
Let ¯ D be the distribution of a pair of random variables ( X ,
¯ Y ) ∈
( X ×Y ) , where ¯ Y denotes the variable of noisy labels.
In learning with noisy labels, we denote the training dataset as ¯ D = { ( x i , ¯ y i ) } Ni = which is independently drawn from the noisy distribution ¯ D .
The goal is to learn a robust classifier by exploiting the noisy dataset ¯ D , which approximates well to the optimal classifier defined on the clean data.
Transition Matrix plays an important role in building statistically consistent classifier through the “loss correction” strategy.
It explicitly models the label-noise generation process from clean data distribution to the noisy data distribution, i.e., T ij ( x ) =
P ( ¯ Y =
j | Y =
i, X = x ) , and
T ( x ) ∈ R C × C .
It represents the probability that an instance x having the clean label Y
=
i is mislabeled as the noisy label ¯ Y =
j .
Intuitively, the clean class posterior probability P ( Y | x ) can be inferred by the transition matrix T ij ( x ) and the noisy class posterior probability P (¯ Y | x ) , i.e.,
P (¯ Y | x ) = T ( x ) P ( Y | x ) .
Note that, in the above equation, the noisy class posterior P (¯ Y | x ) can be estimated by the noisy data.
The transition matrix T ( x ) is generally unidentifiable and very hard to estimate without additional assumptions.
In this paper, we focus on the widely studied class-dependent and instance-independent label noise, i.e, T ( x ) =
T , T ∈ R C × C .
Current state-of-the-art method (VolMinNet [13]) proposes to estimate the class-dependent transition matrix T under the sufficiently scattered assumption, where the traditional anchor point assumption is its special case.
They proved that the optimal T is the one with minimum volume enclosing the noisy class posterior of the training examples.
However, in this work, we resort to address its equivalence problem, i.e., maximizing the volume of corresponding clean class posterior probability, which could reduce the side-effect of the estimation error of the noisy class posterior on estimating T .
Therefore, to maximize the volume of P ( Y | X ) , we propose the forward-backward cycle-consistency regularization, and further to consistently estimate T .
Specifically, as shown in Fig 1, our proposed method simultaneously optimize three objectives: the cross entropy loss between the given noisy label ¯ y and the estimated noisy class posterior; the cross entropy loss between the backward estimated clean class posteriors and the predicted clean class posteriors by the neural network; the cycleconsistency regularization to minimize the approximation error between P ( Y | X ) and T ′ TP ( Y | X ) .
The backward and cycle-consistency regularization help to minimize the volume of T indirectly without exploiting the noisy class posterior, and further encourage the estimated T to converge to its optimal solution.
2.1 Forward Transition Matrix Estimation
Given the input instance x , the probability of observing a noisy label ¯ Y can be inferred as,
[Equation]
We can rearrange Eq.
1 as P (¯ Y | x ) =
TP ( Y | x ) under the class dependent and instance independent assumption, where P (¯ Y | x ) =
[ P ( ¯ Y = 1 | x ) , ..., P ( ¯ Y = C | x )] ⊤ and P ( Y | x )
=
[ P ( Y = 1 | x ) , ..., P ( Y = C | x )]
⊤ are the noisy class posterior probability and the clean class posterior probability, respectively.
To learn the clean class posterior P ( Y | X ) , we design a neural network f
( x ; w ) parameterized with w , which projects the input instance x onto C classes with a probability output.
The transition matrix T should be trainable diagonally dominant column stochastic matrix, i.e.,
T ∈ [0 , 1] C × C , (cid:80) C i =
1 T ij = 1 and T ii > T ji for any i ̸
=
j .
Naturally, we could use Tf ( x ; w ) to learn the noisy class posterior probability P (¯ Y | X ) .
VolMinNet [13] shows us that, if Tf ( x ; w ) models P (¯ Y | X ) perfectly and the transition matrix T has the minimum volume, T will converge to the optimal transition matrix and the classifier f ( x ; w ) will converge to the optimal P ( Y | X ) .
In this paper, we introduce the cycle-consistency regularization to indirectly reduce the volume of T , without exploiting the estimated noisy class-posterior probability which usually has a large estimation error [37].
Given the training dataset ¯ D
= { ( x i , ¯ y i ) } Ni =1 , we optimize the empirical risk by jointly optimizing the transition matrix T and the consistent classifier f ( x ; w ) for label-noise learning.
Specifically, we minimize the approximation error between the inferred noisy class-posterior probability Tf ( x ; w ) and the given noisy label ¯ y as follows,
[Equation]
where N is the number of training examples.
Note that, in Eq. 2, both the transition matrix T and the output of classifier f
( x i ; w ) are not given, which makes the transition matrix T unidentifiable without any assumptions.
To address this issue, VolMinNet [13] has been proposed by minimizing the volume of the transition matrix.
It has been proved that VolMinNet theoretically guarantees the consistencies of both the transition matrix and the classifier.
2.2 Cycle-Consistency Regularization
When minimizing the volume of the transition matrix, VolMinNet exploits the estimated noisy class posteriors to calculate the volume.
Yao et al.
[37] has discussed that the estimation error of the noisy class posterior is usually large due to the randomness of label noise.
This would lead the transition matrix to be poorly estimated by VolMinNet when the noisy training sample is limited.
We will address this issue by proposing a cycle-consistency regularization method instead of minimizing the volume of the transition matrix directly.
As discussed in the Introduction, minimizing the volume of the transition matrix is equivalent to maximizing the volume of the clean class posterior.
We show that we could directly use the noisy labels to maximize the volume of the clean class posterior rather than using the estimated noisy class posterior to calculate the volume of the transition matrix and then minimizing it.
Intuitively, we maximize the volume of the learned clean class posteriors by matching it with the projected
C -dimensional simplex in the noisy class posterior which also needs to estimate.
More specifically, we first transform the one-hot noisy label into the input noisy class posterior probability by using the SoftMax function, denoting it as SoftMax (¯ y i ) .
Then, we use T ′ (an approximation of the inverse of the transition matrix) to project it onto the clean class posterior space.
Since the true clean class posterior is unobservable, here we use the output of the designed neural network f
( x i ; w ) as its approximation.
Finally, we minimize the following cross-entropy loss between T ′ SoftMax (¯ y i ) and f ( x i ; w ) , to optimize the parameters of w and T ′ simultaneously,
[Equation]
Note that this objective will help maximize the volume of the learned clean class posterior as
T ′ SoftMax (¯ y i ) has the largest volume in the sense that the noisy labels will make up the Cdimensional simplex.
Note also that T ′ should be initialized by and maintained as a diagonally dominant column stochastic matrix, i.e., T ′ ∈ [0 , 1] C × C , (cid:80) Ci =1 T ′ ij = 1 and T ′ ii > T ′ ji for any i ̸
= j .
At present, we can obtain the transition matrices from both the forward and backward ways, i.e.,
T and T ′ , by minimizing L 1 and L 2 , respectively.
Intuitively, we should also build an “indirect” cycle-consistency through minimizing the difference between P ( Y | X = x ) and T ′ ( TP ( Y | X = x )) , where “indirect” means that we would make use of the invertible relationship between these two matrices T and T ′ indirectly through the original and circularly computed clean class-posterior.
Specifically, the cycle-consistency regularization can be expressed as,
[Equation]
Note that, in Eq. 4, we just optimize the parameter w in the network classifier f ( x i ; w ) , while keep T and T ′ be constant.
This is because that our ultimate goal is to learn good clean class posteriors.
If the transition matrices are well learned, the estimated clean class posterior should satisfy this constraint.
Based on the above descriptions, the overall objective function can be expressed as,
[Equation]
where λ is the trade-off hyper-parameter to balance the two transition matrix learning objectives with the cycle-consistency regularization term.
We optimize all the model parameters simultaneously in an end-to-end manner.
2.3 Theoretical Analysis
In this section, we justify that the proposed method helps to minimize the volume of the transition matrix T without directly relying on the learned noisy class posterior, which does not prevent the estimated T from converging to its optimal solution.
As theoretically proved in VolMinNet [13], the optimal transition matrix T is the one with minimum volume enclosing the noisy class posterior of the training examples under sufficiently scattered assumption.
VolMinNet thereby theoretically guarantees the consistencies of both the transition matrix and the classifier by minimizing the volume of the transition matrix.
Under the same sufficiently scattered assumption, our proposed method also theoretically guarantees the consistencies of both the transition matrix and the classifier because, in Introduction, we show that minimizing the volume of the transition matrix is equal to maximizing the volume of the clean class posterior.
Importantly, our method could help reduce the side-effects of the estimation error of the noisy class-posterior probability.
Specifically, our proposed framework as shown in Eq.
5, simultaneously optimize the forward and backward transition matrices T and T ′ .
Especially during T ′ estimation, the input is the given one-hot noisy label which corresponds to the simplex having the maximum volume .
When they pass through the diagonally dominant matrix T ′ , the projected clean class-posterior probability should intuitively has a large volume .
Then, minimizing the cross entropy between the learned clean class posteriors and the projected clean class posteriors could help to maximize the volume of the learned clean class posteriors.
Note that the cycle-consistency constraint on T and T ′ as illustrated in Eq.
4 further helps to learn better clean class posterior as the projected clean class posterior is more accurate.
Experiments
In this section, we introduce the experiment setup, including datasets, noise types, and implementation details.
We compare our proposed method with the state-of-the-art algorithms on two synthetic and
[Table]
two real-world noisy datasets, followed by an ablation study to analyze the experimental results and some useful hyper-parameters.
3.1 Experiment Setup
Datasets .
Extensive experiments are conducted on two manually corrupted datasets with different noisy types (i.e., CIFAR-10 [9], CIFAR-100 [9]) and two real-world noisy datasets ( i.e., Clothing1M [34] and Food-101N [10]), to demonstrate the effectiveness of the proposed method.
Both
CIFAR-10 and CIFAR-100 contain 60K images of size 32 × 32, of which 50K images constitute the training set and 10K images for testing set, while CIFAR-10 contains 10 classes, and CIFARcontains 100 classes.
Clothing1M is a real-world noisy dataset that contains 1M images with about 38 .
46% noisy labels for training and 10K images with clean labels for testing.
Food-101N is also a real-world noisy dataset which contains 310K images with about 19 .
66% noisy labels for training and 55K images with clean labels for testing.
Noisy type .
For CIFAR-10 and CIFAR-100, we manually corrupted the training set according to the ground-truth transition matrices.
Specifically, we conducted experiments using three commonly used noisy types: (1) Symmetry flipping [20]; (2) Asymmetry flipping [12];
(3) Pair flipping [8];
3.2 Implementation Details
For fair comparisons, all our experiments are performed on NVIDIA GeForce RTX 3090, and implemented on the same PyTorch platform.
For CIFAR-10 and CIFAR-100, the backbone we used is ResNet-34.
We train the classification network f
( x i ; w ) , the transition matrices T and T ′ by
SGD strategy, with batchsize of 128, momentum 0.9, weight decay 10 − 3 , and learning rate 10 − 2 .
For CIFAR-10, the algorithm run 60 epochs and the learning rate is divided by 10 after the 30- th epoch.
For CIFAR-100, the algorithm run 80 epochs and the learning rate is divided by 10 after 30- th and 60- th epoch.
For Clothing 1M and Food-101N, the backbone we used is ResNet-50 which is pre-trained on ImageNet.
We train the classification network f
( x i ; w ) , the transition matrices
T and T ′ also with SGD strategy, with batchsize of 32, momentum 0.9, weight decay 10 − 3 , and learning rate 2 × 10 − 3 .
The algorithm run 80 epochs and the learning rate is divided by 10 everyepochs.
Before training, we warm up on all noisy data with early stopping technique, where we have trained 10, 10, 1 and 1 epochs on the CIFAR-10, CIFAR-100, Clothing 1M and Food 101N datasets, respectively.
3.3 Comparison with state-of-the-art methods
We compare the proposed method with the following representative works: 1) CE, which trains the classification network with the standard cross-entropy loss on the original noisy dataset; 2) GCE [40]; 3) PeerLoss [17]; 4) Co-teaching [6]; 5) Co-teaching++ [38]; 6) T-Revision [33]; 7) Dual-T [37];
VolMinNet [13] which estimates the transition matrix under sufficiently scattered assumption.
[Table] [Table]
Results on the synthetic noisy datasets .
Tables 1, 2 and 3 report the experiment results under three different synthetic noisy types on cifar10 and cifar100 datasets, respectively.
Each table compares the proposed method with 8 representative works on one synthetic noisy type.
As illustrated in Eq.
5, our overall objective contains three variants as described in the tables: 1) “T-For ( T )” means that we optimize the classifier just use L 1 ( w , T ) ; 2) “T-Back ( T ′ )” means that we optimize the classifier just use L 2 ( w , T ′ ) ; 3) “ T + T ′ ” means that we optimize the classifier use L 1 ( w , T )
+ L 2 ( w , T ′ ) ; 4) “Ours” means that we utilize the overall objective as shown in Eq.
5 to optimize the classifier.
Compared with the representative methods, the proposed method achieves the best performance on both datasets with three synthetic noise types under three different noise ratios.
The evaluation results of the three tables can be summarized as follows:
• Compared with representative methods, the proposed method has significant improvements over these methods, which specifically outperforms the baseline method "CE" and the representative work VolMinNet [13] by a large margin of 16.80%and 7.24% on average.
• As the noise rate increases, the superiority of this method gradually emerges.
As shown in these three tables: for symmetric noise, the average advantage of our method over other methods is 4.47%, 6.65% and 10.54% at noise rate of 20%, 40% and 60%, respectively; For asymmetric noise, the average advantage of our method is 4.46%, 6.61% and 10.28% at noise rate of 20%, 40% and 60%, respectively; For pair noise, the average advantage of our method is 4.61%, 15.31% and 22.88% at noise rate of 20%, 40% and 45%, respectively.
This indicates that our method can handle difficult cases with high noise rates much better.
[Table] [Table]
• By comparing the four variants of our method, it clearly shows that integrating the backward transition matrix and cycle-consistency regularization can greatly help to improve the baseline performances step by step
[Table]
Results on the real-world datasets. .
Tables 4 and 5 compare the proposed method with representative works on Clothing1M and Food-101N datasets, respectively.
We can clearly see that our methods outperforms the baseline method “CE” and VolMinNet [13] by a margin of 1.79% and 0.91% on
Clothing1M dataset, and 2.27% and 0.67% on Food-101N dataset.
What’s more, since the proposed method can be used as a plug-and-play module, we integrate this module into the representative work
DivideMix [11], and denoted as “DivideMix+Ours”.
To further illustrate its effectiveness, we also combine DivideMix with VolMinNet as another baseline, denoted as “DivideMix+VolMinNet”.
We can clearly see that our method achieves superior performances over all these methods.
3.4 Ablation Study
As shown in Figure 2 (a) to (l), we also show the estimation error of the transition matrix T of our method during the model training, with different noise types and different noise rates, on the CIFAR10 and CIFAR-100 datasets, to evaluate the estimated transition matrix T .
The estimation error is measured by the l 1 norm between the ground-truth transition matrix and the estimated transition matrix.
It shows that the proposed method always outperforms previous algorithms on T estimation.
To explore the effect of hyper-parameter λ on model performance, we conducted experiments with different λ under different noise types and noise rates on CIFAR-10 and cifar-100 datasets, as shown in Figure 2 (m) to (p).
Each experiment run five times.
We summarize Figure 2 (m) to (p) as follows:
The smaller the T estimation error is, the higher
the test
accuracy will be; 2)
When
λ = 0 .
3 , the estimation error is the smallest
and we
get the best performances.
Based on this observation, we set λ as 0.3 in all our experiments.
Related Work
Based on the statistical consistency of the learned classifier, we roughly divide existing label-noise learning methods into two categories: algorithms with statistically inconsistent classifiers (i.e., heuristic algorithms), and the algorithms with statistically consistent classifiers.
The statistically inconsistent algorithms usually explore some heuristics to reduce the side-effect of the noisy labels.
The representative works include: 1) the data cleaning methods which specially design some strategies to select reliable examples [18, 5]; 2) the “label correction” methods which aim to improve the label quality during model training [23, 21]; 3) the semi-supervised learning methods which treat the unreliable examples as unlabeled data and then adopt some self-supervised training technics for robust feature representation learning [11]; 4) some other classic machine learning techniques [19, 20], such as soft label based methods [29], early stop tricks to avoid overfitting [2, 31].
Although these methods empirically work well without explicitly modeling the label noise distribution, we can not theoretically guarantee the consistency of the learned classifiers [6].
       (SRFK       7  ( V W L P D W L R
Q  ( UU R U &,)$56\P 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (a)       
(SRFK         7  ( V
W L P D W L R Q  ( UU R U &,)$56\P 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (b)        (SRFK    
 ( V W L P D W L R Q  ( UU R U &,)$56\P 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (c)       
(SRFK         7  ( V W L P D W L R Q  ( UU R U &,)$56\P 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (d)        (SRFK       7  ( V W L P D W L R
Q  ( UU R U &,)$5$V\P 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (e)        (SRFK     
 7  ( V
W L P D W L R Q  ( UU R U &,)$5$V\P 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (f)   
[Table]
   (SRFK
[Table]
     7  ( V
W L P D W L R Q  ( UU R U &,)$5$V\P 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (g) 
      (SRFK       
 ( V W L P D W L R Q  ( UU R U &,)$5$V\P 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (h) 
   
  (SRFK
       
 ( V W L P D W L R Q  ( UU R U &,)$53DLU 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (i) 
      (SRFK     
 ( V W L P D W L R Q  ( UU R U &,)$53DLU 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (j)     
  (SRFK
     
  
  ( V W L P D W L R Q  ( UU R U &,)$53DLU 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (k)        (SRFK       

 ( V W L P D W L R Q  ( UU R U &,)$53DLU 2XUV9RO0LQ1HW7)RU75HYLVLRQ'XDO (l)        (SRFK       
 
 7 H V W  $ FF X U D F \ &,)$53DLU ȜȜȜȜ (m) 
   
  (SRFK
     
 
 ( V W L P D W L R Q  ( UU R U &,)$53DLU ȜȜȜȜ (n)        (SRFK       
   7 H V W  $ FF X U D F \ &,)$53DLU ȜȜȜȜ (o) 
      (SRFK       

 ( V W L P D W L R Q  ( UU R U &,)$53DLU ȜȜȜȜ (p)
The statistically consistent algorithms aim to learn consistent classifier with the noisy data, which will asymptotically converge to the optimal classifier obtained on its corresponding clean risk[37].
These methods are primarily designed based on the “loss correction” strategy.
Among them, the transition matrix T ( x ) has been used to modify the loss function to build consistent classifiers.
Specifically, the anchor point assumption has been widely adopted to estimate the transition matrix [16, 33], and some methods directly assume that the anchor points have already been given.
However, this assumption sometimes could be too strong in real application.
When no anchor point is given in the dataset, these algorithms can not be guaranteed statistically consistent.
Then, to remove this strong dependence on anchor points, some methods propose the slack variable trick [25], which just use the data points with high noisy class-posterior probabilities to estimate the transition matrix.
Afterwards, a series of assumptions [4, 32, 13] have been proposed to efficiently estimate the transition matrix.
For example, Dual-T estimator [37] introduces the intermediate class to factorize the original transition matrix into the product of two easy-to-estimate transition matrices.
ExtendedT [30] proposed to extend the traditional transition matrix to be able to model mixed close-set and open-set label noise.
VolMinNet [13] try to consistently estimate the transition matrix under the sufficiently scattered assumption, which empirically incorporates the minimum volume constraint of T into the label-noise learning.
However, these methods relies heavily on the estimated inaccurate noisy class-posterior probability, which could lead the transition matrix to be poorly estimated.
Conclusion and Limitation
Estimating the transition matrix T plays an important role in building statistically consistent classifier for label-noise learning.
To address the bottleneck of how to reduce the side-effects of the large estimation error of the noisy class posterior on transition matrix estimation, this paper proposes to estimate T under a forward-backward cycle-consistency regularization, of which we could greatly reduce the dependency of estimating T on the estimated noisy class posterior.
We also show that the proposed method helps to minimize the volume of T without directly exploiting the estimated noisy class posterior, which encourages the estimated transition matrix T to converge to its optimal solution.
Extensive experimental results consistently justify the effectiveness of the proposed method, from both of the superior classification accuracy and the estimation error of the learned transition matrix.
Limitation.
One major limitation in this study is that our method is constrained to work well under the sufficiently scattered assumption.
When this assumption is violated in some cases, the statistical consistency of the estimated transition matrix and classifier would not be guaranteed.
In the future, we will make deep theoretical analysis on the backward transition matrix and estimated clean class posterior probability.
Acknowledgements: This work was supported in part by the National Natural Science Foundation of China under Grant 62176198, 61922066 and 62106184, in part by the National Key Research and Development Program of China under Grant 2018AAA0103202, in part by the Technology
Innovation Leading Program of Shaanxi under Grant 2022QFY01-15, in part by Open Research
Projects of Zhejiang Lab under Grant 2021KG0AB01.
Tongliang Liu was partially supported by
Australian Research Council Projects DP180103424, DE-190101473, IC-190100031, DP-220102121, and FT-220100318.
Bo Han was supported by the RGC Early Career Scheme No. 22200720, NSFC
Young Scientists Fund No. 62006202, and Guangdong Basic and Applied Basic Research Foundation
No. 2022A1515011652.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re us- ing/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Appendix
Optionally include extra information (complete proofs, additional experiments and plots) in the appendix.
This section will often be part of the supplemental material.