# FGM: Noisy Label Learning via Fine-Grained Confidence Modeling
## 2 Related Work  
In this section, we briefly review the existing literature on noisy label learning. As listed in Table 1, previous methods can be grouped into several categories. Noise transition matrix. Many early studies tackle the task of noisy label learning by estimating a noise transition matrix which represents the probability of one class being mislabeled into another. Some methods try to improve the performance by adjusting the prediction results of DNNs using the estimated label transition probability. Other methods, such as Feorr [20], correct the loss by multiplying the estimated transition probability with the softmax outputs. The key to these methods is the reliability of the estimated noise transition matrix. Webly learning [21] estimates a noise transition matrix by training an additional noise adaptation layer. Probabilistic noise modeling [22] trains two networks which are used to predict the noise type and label transition probability, respectively. The performance of these methods is often limited in realistic cases due to the complexity of estimating noise transition matrix on real-world datasets. Robust Loss Function. Some studies try to design noise tolerant loss functions. The robust MAE [23] showed that the mean absolute error (MAE) loss is more robust to label noise than cross entropy (CE) loss, yet it is not applicable on complicated real world datasets and suffered from increased difficulty in training compared with cross entropy loss. Thus, the generalized cross entropy (GCE) [24] is introduced, trying to combine the advantages of both MAE and CE loss. Symmetric cross entropy (SCE) [25] proposes a noise robust term called reverse cross entropy loss, which can improve the learning on hard classes. SCE archives significantly better performance than CE and GCE. However, the accuracy of these \begin{table}
\begin{tabular}{c c} \hline \hline
**Category** & **Methods** \\ \hline Noise transition matrix & Feorr [20], Webly learning [21], Probabilistic noise modeling [22] \\ \hline Robust Loss Function & Robust MAE [23], Generalized Cross Entropy [24], Symmetric Cross Entropy [25] \\ \hline Sample Selection & MentorNet [16], Co-teaching [17], Co-teaching+ [18], Iterative-CV [26] \\ \hline Hybrid Methods & SELF [27], Mcorr [8], DivideMix [19], MOIT+ [28], Sel-CL+ [12] \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of literature reviewed according to the category of methods. loss function methods is still not promising and their performance will be greatly degraded in hard cases. Sample Selection. Many recent studies adopted sample selection strategies which select a certain number of possibly clean labeled samples from the training dataset and exclude other samples from the update to prevent the model from over-fitting to noisy labels. The small-loss trick is widely used to distinguish true-labeled samples from noisy samples because the losses of clean samples tend to be smaller than false-labeled samples due to the memorization effect of DNNs. MentorNet [16] trains a student network by feeding in small-loss samples selected by a teacher model. Co-teaching [17] and Co-teaching+ [18] train two networks simultaneously and each network selects small-loss samples for the training of the other model. Iterative-CV [26] randomly divides training samples into subsets and iteratively removes samples with large losses using a cross validation strategy. Despite the significant improvements these methods obtain, samples acquired by them are mostly clean simple instances. Hard samples containing rich information which can greatly help the generalization of DNNs are discarded. Moreover, in extreme circumstances where the noise rate is high, too few samples can be selected to train the model thoroughly. Last but not least, the ground truth noise rate or a clean validation dataset, which are not available in most cases, need to be provided to decide how many samples should be selected. Hybrid Methods. Recent studies attempted to combine sample selection with other techniques.