SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 6: Test accuracies  $(\%)$  of different methods on benchmark datasets with clean or symmetric label noise  $(\eta \in \{0.2, 0.4, 0.6, 0.8\})$ . The results (mean±std) are reported over 3 random runs under different random seeds  $(1, 2, 3)$  and the top-2 best results are boldfaced.
Datasets	Methods	Clean (η=0.0)	Symmetric Noise Rate (η)
0.2	0.4	0.6	0.8
MNIST	CE	99.20±0.02	91.40±0.28	74.46±0.28	49.19±0.05	22.51±0.23
FL	99.16±0.08	91.66±0.18	75.42±0.25	50.58±0.38	22.93±0.11
MAE	99.16±0.03	99.03±0.01	98.80±0.02	97.69±0.20	70.35±1.16
GCE	99.18±0.01	98.84±0.03	96.81±0.13	80.86±0.31	33.59±0.48
SCE	99.30±0.07	98.91±0.04	97.48±0.16	88.35±0.77	48.28±0.81
NLNL	98.61±0.13	98.02±0.14	97.17±0.09	95.42±0.30	86.34±1.43
NCE+MAE	99.22±0.06	98.96±0.04	98.15±0.14	92.94±0.26	59.54±0.76
NCE+RCE	99.43±0.02	99.20±0.05	98.53±0.09	95.61±0.12	74.04±1.83
NFL+RCE	99.33±0.04	99.17±0.03	98.62±0.03	95.54±0.26	75.05±1.45
NCE+AEL	99.06±0.06	99.03±0.08	98.87±0.05	98.39±0.11	96.71±0.10
NCE+AGCE	99.10±0.03	99.00±0.05	98.91±0.04	98.50±0.07	96.93±0.13
NCE+AUL	99.19±0.01	99.04±0.05	99.00±0.03	98.52±0.14	96.97±0.12
ANL-CE	99.08±0.05	98.97±0.02	98.84±0.05	98.42±0.08	96.62±0.12
ANL-FL	99.13±0.05	98.94±0.07	98.90±0.05	98.46±0.12	95.73±0.22
CIFAR-10	CE	90.38±0.11	75.05±0.26	58.19±0.21	38.75±0.19	19.09±0.35
FL	89.84±0.28	74.52±0.10	57.54±0.75	38.83±0.49	19.33±0.58
MAE	89.15±0.27	87.19±0.19	81.76±3.17	76.82±0.84	46.42±3.66
GCE	89.66±0.20	87.17±0.01	82.44±0.26	68.62±0.35	25.45±0.51
SCE	91.38±0.12	87.86±0.12	79.96±0.25	62.16±0.33	27.98±0.98
NLNL	90.73±0.20	73.70±0.05	63.90±0.44	50.68±0.47	29.53±1.55
NCE+MAE	88.94±0.13	87.37±0.19	83.70±0.21	76.35±0.08	44.68±1.12
NCE+RCE	90.94±0.01	89.19±0.18	86.03±0.13	79.89±0.11	55.52±2.74
NFL+RCE	90.86±0.51	89.04±0.09	86.08±0.33	79.79±0.16	54.18±2.06
NCE+AEL	88.51±0.26	86.59±0.24	83.07±0.46	75.06±0.26	41.79±1.40
NCE+AGCE	91.08±0.06	89.11±0.07	86.16±0.10	80.14±0.27	55.62±4.78
NCE+AUL	91.26±0.12	89.08±0.14	86.11±0.27	79.39±0.41	54.49±2.77
ANL-CE	91.66±0.04	90.02±0.23	87.28±0.02	81.12±0.30	61.27±0.55
ANL-FL	91.79±0.19	89.95±0.20	87.25±0.11	81.67±0.19	61.22±0.85
CIFAR-100	CE	71.14±0.38	55.97±1.11	40.72±0.74	22.98±0.07	7.55±0.21
FL	71.02±0.36	55.94±0.53	39.55±1.24	23.21±0.49	7.80±0.27
MAE	7.35±1.19	7.91±0.66	3.61±0.21	3.63±0.35	2.83±1.35
GCE	61.62±0.43	61.50±1.50	56.46±0.95	46.27±1.30	19.51±0.86
SCE	70.80±0.37	55.04±0.37	39.84±0.19	21.97±0.92	7.87±0.48
NLNL	68.72±0.60	46.99±0.91	30.29±1.64	16.60±0.90	11.01±2.48
NCE+MAE	67.52±0.21	52.68±0.42	35.71±0.47	19.44±0.15	7.08±0.07
NCE+RCE	68.22±0.28	64.20±0.47	57.97±0.30	46.26±1.07	25.65±0.51
NFL+RCE	68.04±0.30	64.33±0.23	58.48±0.50	47.20±0.58	26.26±0.19
NCE+AEL	64.98±0.42	48.13±0.32	32.11±0.89	20.75±2.00	7.97±1.02
NCE+AGCE	68.61±0.12	65.30±0.21	59.74±0.68	47.96±0.44	24.13±0.07
NCE+AUL	69.91±0.18	65.26±0.17	56.67±0.21	39.98±0.42	13.30±0.08
ANL-CE	70.68±0.23	66.79±0.34	61.80±0.50	51.52±0.53	28.07±0.28
ANL-FL	70.40±0.15	66.54±0.29	61.73±0.48	51.32±0.34	27.97±0.58
Table 7: Test accuracies  $(\%)$  of different methods on benchmark datasets with asymmetric label noise  $(\eta \in \{0.1,0.2,0.3,0.4\})$ . The results (mean±std) are reported over 3 random runs under different random seeds  $(1,2,3)$  and the top-2 best results are boldfaced.
DATASETS	METHODS	ASYMMETRIC NOISE RATE (η)
0.1	0.2	0.3	0.4
MNIST	CE	97.70±0.02	94.02±0.18	88.90±0.07	81.79±0.34
FL	97.62±0.05	94.41±0.11	88.82±0.35	81.99±0.61
MAE	99.05±0.06	99.11±0.03	98.42±0.09	87.40±4.01
GCE	98.97±0.02	96.59±0.07	88.99±0.27	81.91±0.58
SCE	99.05±0.02	97.95±0.23	94.00±0.41	84.54±0.14
NLNL	98.63±0.06	98.35±0.01	97.51±0.15	95.84±0.26
NCE+MAE	99.12±0.09	98.33±0.14	95.16±0.21	85.79±0.47
NCE+RCE	99.31±0.06	98.79±0.10	95.16±0.08	91.36±0.22
NFL+RCE	99.27±0.02	98.79±0.15	96.97±0.09	91.36±0.40
NCE+AEL	99.06±0.05	99.05±0.07	98.88±0.02	98.29±0.27
NCE+AGCE	99.11±0.04	99.04±0.02	98.94±0.03	98.41±0.04
NCE+AUL	99.11±0.05	99.09±0.03	98.98±0.06	98.70±0.01
ANL-CE	99.10±0.05	99.04±0.04	98.91±0.07	98.01±0.10
ANL-FL	99.00±0.05	99.05±0.09	98.93±0.02	98.18±0.01
CIFAR-10	CE	86.85±0.15	83.00±0.33	78.15±0.17	73.69±0.20
FL	86.32±0.28	83.03±0.10	78.53±0.16	73.78±0.16
MAE	88.22±0.05	79.63±0.74	67.35±3.41	57.36±2.37
GCE	88.19±0.21	85.55±0.24	79.32±0.52	72.83±0.17
SCE	89.57±0.11	86.22±0.44	80.20±0.20	74.01±0.52
NLNL	88.54±0.25	84.74±0.08	81.26±0.43	76.97±0.52
NCE+MAE	88.22±0.25	86.16±0.18	82.98±0.15	75.23±0.24
NCE+RCE	89.98±0.19	88.36±0.13	84.84±0.16	77.75±0.37
NFL+RCE	90.11±0.06	88.26±0.27	84.72±0.18	77.29±0.30
NCE+AEL	87.64±0.16	85.64±0.07	81.95±0.38	74.55±0.42
NCE+AGCE	90.21±0.16	88.48±0.09	84.79±0.15	78.60±0.41
NCE+AUL	90.12±0.06	88.29±0.15	84.84±0.06	76.99±0.26
ANL-CE	90.90±0.13	89.13±0.11	85.52±0.24	77.63±0.31
ANL-FL	90.81±0.20	89.09±0.31	85.81±0.23	77.73±0.31
CIFAR-100	CE	65.17±0.45	58.25±1.00	50.30±0.19	41.53±0.34
FL	64.55±0.39	58.00±1.38	50.77±0.41	41.88±0.57
MAE	6.63±1.60	6.19±0.42	5.82±0.96	3.96±0.35
GCE	64.29±0.90	59.06±0.46	53.88±0.96	41.51±0.52
SCE	64.64±0.57	57.78±0.83	50.15±0.12	41.33±0.86
NLNL	59.55±1.22	50.19±0.56	42.81±1.13	35.10±0.20
NCE+MAE	60.52±0.36	52.92±0.50	44.41±0.22	36.71±0.16
NCE+RCE	66.18±0.23	62.77±0.53	55.62±0.56	42.46±0.42
NFL+RCE	66.16±0.44	63.43±0.71	55.63±0.37	42.54±0.52
NCE+AEL	57.47±0.47	50.49±0.12	42.46±0.51	35.04±0.29
NCE+AGCE	66.86±0.23	64.05±0.25	56.36±0.59	44.90±0.62
NCE+AUL	66.23±0.21	57.79±0.40	47.64±0.24	38.65±0.30
ANL-CE	68.78±0.11	66.27±0.19	59.76±0.34	45.41±0.68
ANL-FL	68.36±0.15	66.26±0.44	59.68±0.86	46.65±0.04
D.5 Comparisons on more real-world datasets.
CIFAR-10N/-100N [20] are CIFAR-10/-100 equipped with human-annotated real-world noisy labels. CIFAR-10N contains five noisy label sets with noise rates of  $9.03\%$  (Aggregate),  $17.23\%$  (Random 1),  $18.12\%$  (Random 2),  $17.64\%$  (Random 3) and  $40.21\%$  (Worst). CIFAR-100N contains one noisy label set with noise rate of  $40.20\%$  (Noisy).
We use the same experimental settings and parameters as CIFAR-10 and CIFAR-100 for CIFAR-10N and CIFAR-100N, since the only difference is the noise label distribution. The results are reported in Table 8 and Table 9.
Animal-10N [21] is a real-world noisy data set of human-labeled online images for 10 confusing animals, with 50,000 training and 5,000 testing images, and its noise rate was estimated at  $8\%$ .
We follow the experimental setting in previous works [21]. We use VGG-19 with batch normalization. The SGD optimizer is employed. We train the network for 100 epochs and use an initial learning rate of 0.1, which is divided by 5 at  $50\%$  and  $75\%$  of the total number of epochs. Batch size is set to 128. Typical data augmentations including random horizontal flip are applied.
We compare our ANL-CE with CE and GCE. We use L2 regularization (weight decay) for GCE, and L1 regularization for ANL-CE. We denote the regularization coefficient by  $\delta$ . We tune the parameters  $\{\delta\}$ ,  $\{q,\delta\}$ ,  $\{\alpha,\beta,\delta\}$  for CE, GCE and ANL-CE respectively. We use the best parameters  $\{1\times 10^{-3}\}$ ,  $\{0.5,1\times 10^{-4}\}$ ,  $\{0.5,1.0,1\times 10^{-6}\}$  for each method in our experiments. The results are reported in Table 10.
Moreover, we experiment with NCE+RCE on this dataset and tune the parameters  $\{\alpha, \beta, \delta\}$ , but we find that the performance is very poor for some unknown reason. The best test accuracy we achieve is  $28.28\%$  with  $\{10.0, 0.1, 5 \times 10^{-6}\}$ . Since this result is too low and inconsistent with its performance on other datasets, we do not include it in the table for comparison.
Clothing-1M [22] is a large-scale clothing dataset contains 14 categories and 1 million training samples with nearly  $40\%$  mislabeled samples.
We follow the experimental setting in previous works [31]. We use the  $14k$  and  $10k$  clean data for validation and test, respectively, and we do not use the  $50k$  clean training data. We use ResNet-50 pre-trained on ImageNet. For preprocessing, we resized the images to  $256 \times 256$ , performed mean subtraction, and cropped the middle  $224 \times 224$ . We use SGD with a momentum of 0.9, a weight decay of  $1 \times 10^{-3}$ , and batch size of 32. We train the network for 10 epochs with learning rate  $1 \times 10^{-3}$  and  $1 \times 10^{-4}$  for 5 epochs each. Typical data augmentations including random horizontal flip are applied.
We compare our ANL-CE with CE, GCE and NCE+RCE. In this experiment, we use L2 regularization (weight decay) for ANL-CE and set the coefficient the same as those for CE, GCE and NCE+RCE. We tune the parameters  $\{q\}$ ,  $\alpha$ ,  $\beta$  and  $\{\alpha, \beta\}$  for GCE, NCE+RCE and ANL-CE respectively. We use the best parameters  $\{0.6\}$ ,  $\{10.0, 1.0\}$  and  $\{5.0, 0.1\}$  for each method in our experiments. The results are reported in Table 11.
Table 8: Test accuracies (%) of different methods on CIFAR-10N dataset. The results (mean±std) are reported over 3 random runs under different random seeds (1, 2, 3) and the top-2 best results are boldfaced.
Methods	Clean	Aggregate	Random 1	Random 2	Random 3	Worst
CE	90.38±0.11	85.09±0.30	79.09±0.28	78.59±0.42	78.39±0.50	61.43±0.52
GCE	89.66±0.20	87.38±0.07	85.87±0.27	85.43±0.13	85.51±0.15	75.19±0.23
SCE	91.38±0.12	88.48±0.26	85.65±0.30	85.71±0.19	85.87±0.13	73.65±0.29
NCE+RCE	90.94±0.01	89.17±0.28	87.62±0.34	87.66±0.12	87.70±0.18	79.74±0.09
NCE+AGCE	91.08±0.06	89.27±0.28	87.92±0.02	87.61±0.20	87.62±0.16	79.91±0.37
ANL-CE (ours)	91.66±0.04	89.66±0.12	88.68±0.13	88.19±0.08	88.24±0.15	80.23±0.28
Table 9: Test accuracies (%) of different methods on CIFAR-100N dataset. The results (mean±std) are reported over 3 random runs under different random seeds (1, 2, 3) and the top-2 best results are boldfaced.
Methods	Clean	Noisy
CE	71.14±0.38	48.63±0.53
GCE	61.62±0.43	50.97±0.60
SCE	70.80±0.37	48.52±0.11
NCE+RCE	68.22±0.28	54.27±0.09
NCE+AGCE	68.61±0.12	55.96±0.20
ANL-CE (ours)	70.68±0.23	56.37±0.42
Table 10: Test accuracies  $(\%)$  of different methods on Animal-10N dataset. The results (mean±std) are reported over 3 random runs under different random seeds (1, 2, 3) and the top-1 best results are boldfaced.
Methods	CE	GCE	ANL-CE (ours)
Test Acc. (%)	78.92±0.76	80.39±0.17	80.72±0.37
Table 11: Test accuracies (%) of different methods on Clothing-1M dataset. The top-1 best results are boldfaced.
Methods	CE	GCE	NCE+RCE	ANL-CE (ours)
Test Acc. (%)	68.07	68.94	69.07	69.93