  Numerous researches have proved that deep neural networks (DNNs) can fit
everything in the end even given data with noisy labels, and result in poor
generalization performance. However, recent studies suggest that DNNs tend to
gradually memorize the data, moving from correct data to mislabeled data.
Inspired by this finding, we propose a novel method named Dynamic Loss
Thresholding (DLT). During the training process, DLT records the loss value of
each sample and calculates dynamic loss thresholds. Specifically, DLT compares
the loss value of each sample with the current loss threshold. Samples with
smaller losses can be considered as clean samples with higher probability and
vice versa. Then, DLT discards the potentially corrupted labels and further
leverages supervised learning techniques. Experiments on CIFAR-10/100 and
Clothing1M demonstrate substantial improvements over recent state-of-the-art
methods.
  In addition, we investigate two real-world problems for the first time.
Firstly, we propose a novel approach to estimate the noise rates of datasets
based on the loss difference between the early and late training stages of
DNNs. Secondly, we explore the effect of hard samples (which are difficult to
be distinguished) on the process of learning from noisy labels.
