SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Mingyu Xu $^{1,2*}$ , Zheng Lian $^{1*}$ , Lei Feng $^{3}$ , Bin Liu $^{1,2}$ , Jianhua Tao $^{4,5}$
<sup>1</sup>The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences
$^{2}$ School of Artificial Intelligence, University of Chinese Academy of Sciences
$^{3}$ School of Computer Science and Engineering, Nanyang Technological University  $^{4}$ Department of Automation, Tsinghua University
$^{5}$ Beijing National Research Center for Information Science and Technology, Tsinghua University {xumingyu2021, lianzheng2016}@ia.ac.cn
Noisy partial label learning (noisy PLL) is an important branch of weakly supervised learning. Unlike PLL where the ground-truth label must conceal in the candidate label set, noisy PLL relaxes this constraint and allows the ground-truth label may not be in the candidate label set. To address this challenging problem, most of the existing works attempt to detect noisy samples and estimate the ground-truth label for each noisy sample. However, detection errors are unavoidable. These errors can accumulate during training and continuously affect model optimization. To this end, we propose a novel framework for noisy PLL with theoretical interpretations, called "Adjusting Label Importance Mechanism (ALIM)". It aims to reduce the negative impact of detection errors by trading off the initial candidate set and model outputs. ALIM is a plug-in strategy that can be integrated with existing PLL approaches. Experimental results on multiple benchmark datasets demonstrate that our method can achieve state-of-the-art performance on noisy PLL. Our code is available at: https://github.com/zeroQiaoba/ALIM.
Partial label learning (PLL) [1, 2] (also called ambiguous label learning [3, 4] and superset label learning [5, 6]) is a typical type of weakly supervised learning. Unlike supervised learning where each sample is associated with a ground-truth label, PLL needs to identify the ground-truth label from a set of candidate labels. Due to the low annotation cost of partially labeled samples, PLL has attracted increasing attention from researchers and has been applied to many areas, such as object recognition [4], web mining [7], and ecological informatics [8].
The basic assumption of PLL is that the ground-truth label must be in the candidate label set [9]. However, this assumption may not be satisfied due to the unprofessional judgment of annotators [10]. Recently, some researchers have relaxed this assumption and focused on a more practical setup called noisy PLL [11]. In noisy PLL, the ground-truth label may not be in the candidate label set. To deal with this task, Lv et al. [11] utilized noise-tolerant loss functions to avoid overemphasizing noisy samples. However, they cannot fully exploit the useful information in noisy data. To this end, Lian et al. [12] and Wang et al. [13] proposed to detect noisy samples and estimate pseudo labels for these samples. However, detection errors are unavoidable. These errors will accumulate and continuously affect model optimization, thereby limiting their performance on noisy PLL.
Figure 1: The core structure of ALIM. The network receives an input  $x$  and produces softmax prediction probabilities  $P(x)$ . Different from traditional PLL that fully trusts the candidate set, our method can deal with noisy samples through a weighting mechanism.
To alleviate this problem, we propose a novel framework for noisy PLL called "Adjusting Label Importance Mechanism (ALIM)". Although we may make mistakes in noisy sample detection, it allows us to leverage the initial candidate set and restart correction. To reduce manual efforts in hyperparameter tuning, we propose an adaptive strategy to determine the weighting coefficient for ALIM. To further improve noise tolerance, we equip ALIM with mixup training [14], a powerful technique in noisy label learning [15]. We also perform theoretical analysis from the perspective of objective functions and EM algorithms and prove the feasibility of our method. To verify its effectiveness, we conduct experiments on multiple benchmark datasets. Experimental results demonstrate that ALIM outperforms currently advanced approaches, setting new state-of-the-art records. The main contributions of this paper can be summarized as follows:
- We propose a plug-in framework for noisy PLL with theoretical interpretations. Our method can deal with noisy samples by trading off the initial candidate set and model outputs.
- We propose an adaptive strategy to adjust the weighting coefficient for ALIM. Furthermore, we combine our method with the mixup training to improve noise robustness.
- Experimental results on multiple datasets demonstrate the effectiveness of our method. ALIM is superior to currently advanced approaches on noisy PLL.
2.1 Problem Definition
Let  $\mathcal{X}$  be the input space and  $\mathcal{Y}$  be the label space with  $c$  distinct categories. We consider a dataset  $\mathcal{D}$  containing  $N$  partially labeled samples  $\{(x, S(x))\}$ . Here,  $S(x) \in \{0, 1\}^c$  is the candidate set for the sample  $x \in \mathcal{X}$ . We denote the  $i^{th}$  element of  $S(x)$  as  $S_i(x)$ , which is equal to 1 if the label  $i$  is in the candidate set, and 0 otherwise. Meanwhile, we denote  $P(x) \in \mathbb{R}^c$ ,  $w(x) \in \mathbb{R}^c$ , and  $y(x) \in \mathbb{R}^1$  as the prediction probabilities, the pseudo label, and the ground-truth label of the sample  $x$ . The  $i^{th}$  elements of  $P(x)$  and  $w(x)$  are represented as  $P_i(x)$  and  $w_i(x)$ , respectively. In this paper, all product operations between matrices are Hadamard products.
The goal of noisy PLL is to learn a classifier that can minimize the classification risk on  $\mathcal{D}$ . Unlike PLL, noisy PLL allows the ground-truth label may not be in the candidate set. For a fair comparison, we adopt the same data generation procedure as previous works [12]. Specifically, to generate candidate labels, we first flip incorrect labels into false positive labels with a probability  $q$  and aggregate the flipped ones with the ground-truth label. Then, we assume that each sample has a probability  $\eta$  of being the noisy sample. For each noisy sample, we further select a label from the non-candidate set, move it into the candidate set, and move the ground-truth label out of the candidate set. We denote the probability  $q$  as the ambiguity level and the probability  $\eta$  as the noise level.
2.2 Motivation
First, let us review our own exam experience. When we are unfamiliar with a test, we believe that the correct answer must be in the candidate set. Even if every option is wrong, we still choose the most likely answer. But as we become more familiar with the test, we learn to question the correctness of the candidate set. If we believe every option is wrong, we will consider answers outside the candidate set. In noisy PLL, this strategy can also be used to handle noisy samples.
2.3 ALIM Framework
Inspired by the above idea, we propose a simple yet effective framework called ALIM. The overall structure is shown in Figure 1. Specifically, we first predict the softmax probabilities  $P(x)$  for each sample  $x$ . In traditional PLL, we fully trust the candidate set and generate the pseudo label as follows:
$$
w (x) = \text {N o r m a l i z e} (S (x) P (x)), \tag {1}
$$
where  $\mathrm{Normalize}(\cdot)$  is a normalization function that ensures  $\sum_{i=1}^{c} w_i(x) = 1$ . Different from traditional PLL, ALIM introduces a coefficient  $\lambda$  to control the reliability of the candidate set:
$$
\tilde {S} (x) = S (x) + \lambda (1 - S (x)), \tag {2}
$$
$$
w (x) = \text {N o r m a l i z e} (\tilde {S} (x) P (x)). \tag {3}
$$
Here,  $\lambda = 0$  means that we fully trust the given candidate set  $S(x)$ ;  $\lambda = 1$  means that we don't believe the candidate set but trust our own judgment  $P(x)$ . According to Eq. 2, the time complexity of this process is mere  $O(cN)$ , where  $c$  is the number of categories and  $N$  is the number of samples.
In this paper, we discuss two choices of normalization functions: Onehot(·) and Scale(·). Specifically, Onehot(·) sets the maximum value to 1 and others to 0; Scale(·) introduces a scaling factor  $K > 0$  and normalizes the probabilities as follows:
$$
\operatorname {S c a l e} (z) = \left\{\frac {z _ {i} ^ {1 / K}}{\sum_ {j} z _ {j} ^ {1 / K}} \right\} _ {i = 1} ^ {c}, \tag {4}
$$
where  $z_{i}$  is the  $i^{th}$  element of  $z$ .
2.4 Theoretical Analysis
To verify the feasibility of our method, we conduct theoretical analysis from two perspectives: (1) the manually-designed objective function; (2) the classic expectation-maximization (EM) algorithm [16].
2.4.1 Interpretation from Objective Functions
Let  $P_i, S_i$ , and  $w_i$  be abbreviations for  $P_i(x), S_i(x),$  and  $w_i(x)$ , respectively. During training, we should optimize the following objectives:
- Minimize the classification loss on  $w(x)$  and  $P(x)$ .
-  $w(x)$  should be small at non-candidate labels.
- Entropy regularization on  $w(x)$  to avoid overconfidence of pseudo labels.
-  $w(x)$  should satisfy  $0 \leq w_i \leq 1$  and  $\sum_{i=1}^{c} w_i = 1$ .
Then, the final objective function is calculated as follows:
$$
\max  \sum_ {i = 1} ^ {c} w _ {i} \log P _ {i} + M \left(\sum_ {i = 1} ^ {c} w _ {i} S _ {i} - 1\right) - K \sum_ {i = 1} ^ {c} w _ {i} \log w _ {i}
$$
$$
s. t. \sum_ {i} ^ {c} w _ {i} = 1, w _ {i} \geq 0, \tag {5}
$$
where  $M$  and  $K$  are penalty factors. By using Lagrange multipliers, we can observe that the penalty factor  $K$  is different for two normalization functions:  $K = 0$  for  $\mathrm{Onehot}(\cdot)$  and  $K > 0$  for  $\mathrm{Scale}(\cdot)$ . The penalty factor  $M$  has a strong correlation with the weighting coefficient  $\lambda$  in Eq. 2, i.e.,  $\lambda = e^{-M}$ . Larger  $M$  (or smaller  $\lambda$ ) means that we have tighter constraints on  $(\sum_{i=1}^{c} w_i S_i - 1)$ , and therefore we should trust the given candidate set more. It is identical to the meaning of  $\lambda$  in our ALIM (see Section 2.3).
2.4.2 Interpretation from EM Perspective
EM aims to maximize the likelihood of the corpus  $\mathcal{D}$ . Following previous works [5, 17], we first make a mild assumption:
Assumption 1 In noisy PLL, the ground-truth label may not be in the candidate set  $S(x)$ . We assume that each candidate label  $\{i|S_i(x) = 1\}$  has an equal probability  $\alpha (x)$  of generating  $S(x)$  and each non-candidate label  $\{i|S_i(x) = 0\}$  has an equal probability  $\beta (x)$  of generating  $S(x)$ .
Besides the interpretation from objective functions, we further explain ALIM from the EM perspective in Appendix B. We prove that the E-step aims to predict the ground-truth label for each sample and the M-step aims to minimize the classification loss. Meanwhile, ALIM is a simplified version of the results derived from EM. Specifically, EM uses an instance-dependent  $\lambda(x) = \beta(x) / \alpha(x)$ , while ALIM uses a global  $\lambda$ . In the future, we will explore the performance of the instance-dependent  $\lambda(x)$  for noisy PLL. Additionally, EM connects traditional PLL with noisy PLL. In traditional PLL, we assume that the ground-truth label must be in the candidate set, i.e.,  $\beta(x) = 0$ . Then, our noisy PLL approach ALIM will degenerate to the classic PLL method RC [18].
2.5 Optional Key Components
We further introduce several key components to make our method more effective, including the adaptively adjusted strategy and the mixup training.
2.5.1 Adaptively Adjusted Strategy
Appropriate  $\lambda$  is important for ALIM. Too small  $\lambda$  makes us fully trust the candidate set, thus easily over-fitting on noise samples; too large  $\lambda$  makes us ignore the prior information in the candidate set, thus limiting the classification performance. Therefore, we propose a manually adjusted strategy to find a proper  $\lambda$  in a predefined parameter space.
To further reduce manual efforts, we also propose an adaptively adjusted strategy. Specifically, we first estimate the noise level of the dataset. Intuitively, we can randomly select a subset from  $\mathcal{D}$ , annotate the ground-truth labels by professional annotators, and estimate the noise level of the dataset. Alternatively, we can automatically estimate noise rate via the Gaussian mixture model [19, 20] or cross-validation [21, 22]. The estimated noise level is represented as  $\eta$ . Based on Appendix C, we prove that the value of Eq. 6 can be viewed as a metric, and the  $\eta$ -quantile of this value can be treated as the adaptively adjusted  $\lambda$ .
$$
\left\{\frac {\operatorname* {m a x} _ {i} S _ {i} (x) P _ {i} (x)}{\operatorname* {m a x} _ {i} \left(1 - S _ {i} (x)\right) P _ {i} (x)} \right\} _ {x \in \mathcal {D}}. \tag {6}
$$
In Section 3.2, We further present results without noise rate estimation and manually adjust  $\lambda$  as a hyper-parameter. Through experimental analysis, this approach can also achieve competitive performance. Therefore, the adaptively adjusted strategy is optional. Its main advantage is to reduce manual efforts in hyper-parameter tuning and realize a more automatic approach for noisy PLL.
2.5.2 Mixup Training
Since the mixup training is powerful in noisy label learning [14, 15], we further combine this strategy with ALIM for noisy PLL. Consider a pair of samples  $x_{i}$  and  $x_{j}$  whose pseudo labels are denoted as  $w(x_{i})$  and  $w(x_{j})$ , respectively. Then, we create a virtual training sample by linear interpolation:
$$
x _ {\text {m i x}} = \alpha x _ {i} + (1 - \alpha) x _ {j}, \tag {7}
$$
$$
w _ {\text {m i x}} = \alpha w \left(x _ {i}\right) + (1 - \alpha) w \left(x _ {j}\right). \tag {8}
$$
Algorithm 1: Pseudo-code of ALIM.
Input: Dataset  $\mathcal{D}$  with the estimated noise level  $\eta$  , predictive model  $f$  , warm-up epoch  $e_0$  , number of epoch  $E_{\mathrm{max}}$  , weighting coefficient  $\lambda$  , trade-off between losses  $\lambda_{\mathrm{mix}}$  Output: The optimized model  $f$    
1 for  $e = 1,\dots ,e_0$  do   
2 Warm up by training  $f$  with  $\mathcal{L}_{\mathrm{pll}}$    
3 end   
4   
5 for  $e = e_0,\dots ,E_{\mathrm{max}}$  do   
6 for  $\{x_i,S(x_i)\} \in \mathcal{D}$  do   
7 Calculate the output of the predictive model  $f(x_{i})$  .   
8 Obtain the pseudo label  $w(x_{i})$  by Eq.  $2\sim 3$  .   
9 Get the PLL loss  $\mathcal{L}_{\mathrm{pll}}$  between  $f(x_{i})$  and  $w(x_{i})$  . if mixup training then Sample  $\alpha \sim \mathrm{Beta}(\zeta ,\zeta)$  and sample  $x_{j}$  from  $\mathcal{D}$  . Create the virtual sample  $(x_{\mathrm{mix}},w_{\mathrm{mix}})$  by Eq.  $7\sim 8$  . Calculate the mixup loss  $\mathcal{L}_{\mathrm{mix}}$  between  $f(x_{\mathrm{mix}})$  and  $w_{\mathrm{mix}}$  . Get the final loss  $\mathcal{L} = \mathcal{L}_{\mathrm{pll}} + \lambda_{\mathrm{mix}}\mathcal{L}_{\mathrm{mix}}$  else   
16 Get the final loss  $\mathcal{L} = \mathcal{L}_{\mathrm{pll}}$  .   
17 end   
18 end   
19 Optimize  $f$  by minimizing  $\mathcal{L}$  .   
20 if adaptively adjusted  $\lambda$  then   
21 Create an empty list  $G$  .   
22 for  $\{x_i,S(x_i)\} \in \mathcal{D}$  do Calculate the output of the predictive model  $f(x_{i})$  . Store the value in Eq. 6 to  $G$  .   
24 end   
25 end   
26 end   
27 end   
28 end
$\alpha \sim \mathrm{Beta}(\zeta, \zeta)$ , where  $\zeta$  is a parameter in the beta distribution. We define the mixup objective  $\mathcal{L}_{\mathrm{mix}}$  as the cross-entropy loss on  $P(x_{\mathrm{mix}})$  and  $w_{\mathrm{mix}}$ . During training, we combine the mixup loss  $\mathcal{L}_{\mathrm{mix}}$  and the PLL loss  $\mathcal{L}_{\mathrm{pll}}$  into a joint objective function  $\mathcal{L} = \mathcal{L}_{\mathrm{pll}} + \lambda_{\mathrm{mix}}\mathcal{L}_{\mathrm{mix}}$ , where  $\lambda_{\mathrm{mix}}$  controls the trade-off between two losses. The pseudo-code of ALIM is summarized in Algorithm 1.
Corpus Description In the main experiments, we evaluate the performance on two benchmark datasets of noisy PLL, CIFAR-10 [23] and CIFAR-100 [23]. We choose the noise level  $\eta$  from  $\{0.1, 0.2, 0.3\}$ . Since CIFAR-100 has more classes than CIFAR-10, we consider  $q \in \{0.1, 0.3, 0.5\}$  for CIFAR-10 and  $q \in \{0.01, 0.03, 0.05\}$  for CIFAR-100. In Section 3.2, we also conduct experiments on fine-grained datasets (CUB-200 [17] and CIFAR-100H [24]) and consider severe noise.
Baselines To verify the effectiveness of our method, we implement the following state-of-the-art methods as baselines: 1) CC [18]: a classifier-consistent PLL method under the uniform candidate label generation assumption; 2) RC [18]: a risk-consistent PLL method under the same assumption as CC; 3) LWC [25]: a PLL method that considers the trade-off between losses on candidate and non-candidate sets; 4) LWS [25]: a PLL method that combines the weighted loss with the sigmoid activation function; 5) PiCO [17]: a PLL method using contrastive learning; 6) CRDPLL [26]: a PLL method that exploits consistency regularization on the candidate set and supervised learning on the non-candidate set; 7) IRNet [12]: a noisy PLL method that progressively purifies noisy samples; 8)  $\mathrm{PiCO + }$  [13]: a noisy PLL method using a semi-supervised contrastive framework.
Table 1: Performance of different methods.  $\diamond$  denotes the models without mixup training, and  $\heartsuit$  denotes the models with mixup training. By default, we combine ALIM with PiCO for noisy PLL.
CIFAR-10	q=0.1	q=0.3	q=0.5
η=0.1	η=0.2	η=0.3	η=0.1	η=0.2	η=0.3	η=0.1	η=0.2	η=0.3
◇CC	79.81±0.22	77.06±0.18	73.87±0.31	74.09±0.60	71.43±0.56	68.08±1.12	69.87±0.94	59.35±0.22	48.93±0.52
◇RC	80.87±0.30	78.22±0.23	75.24±0.17	79.69±0.37	75.69±0.63	71.01±0.54	72.46±1.51	59.72±0.42	49.74±0.70
◇LWC	79.13±0.53	76.15±0.46	74.17±0.48	77.47±0.56	74.02±0.35	69.10±0.59	70.59±1.34	57.42±1.14	48.93±0.37
◇LWS	82.97±0.24	79.46±0.09	74.28±0.79	80.93±0.28	76.07±0.38	69.70±0.72	70.41±2.68	58.26±0.28	39.42±3.09
◇PiCO	90.78±0.24	87.27±0.11	84.96±0.12	89.71±0.18	85.78±0.23	82.25±0.32	88.11±0.29	82.41±0.30	68.75±2.62
◇CRDPLL	93.48±0.17	89.13±0.39	86.19±0.48	92.73±0.19	86.96±0.21	83.40±0.14	91.10±0.07	82.30±0.46	73.78±0.55
◇PiCO+	93.64±0.19	93.13±0.26	92.18±0.38	92.32±0.08	92.22±0.01	89.95±0.19	91.07±0.02	89.68±0.01	84.08±0.42
◇IRNet	93.44±0.21	92.57±0.25	92.38±0.21	92.81±0.19	92.18±0.18	91.35±0.08	91.51±0.05	90.76±0.10	86.19±0.41
◇ALIM-Scale	94.15±0.14	93.41±0.04	93.28±0.08	93.40±0.03	92.69±0.11	92.01±0.19	92.52±0.12	90.92±0.10	86.51±0.21
◇ALIM-Onehot	94.15±0.15	94.04±0.16	93.77±0.27	93.44±0.16	93.25±0.08	92.42±0.17	92.67±0.12	91.83±0.08	89.80±0.38
◇PiCO+	94.58±0.02	94.74±0.13	94.43±0.19	94.02±0.03	94.03±0.01	92.94±0.24	93.56±0.08	92.65±0.26	88.21±0.37
◇ALIM-Scale	95.71±0.01	95.50±0.08	95.35±0.13	95.31±0.16	94.77±0.07	94.36±0.03	94.71±0.04	93.82±0.13	90.63±0.10
◇ALIM-Onehot	95.83±0.13	95.86±0.15	95.75±0.19	95.52±0.15	95.41±0.13	94.67±0.21	95.19±0.24	93.89±0.21	92.26±0.29
CIFAR-100	q=0.01	q=0.03	q=0.05
η=0.1	η=0.2	η=0.3	η=0.1	η=0.2	η=0.3	η=0.1	η=0.2	η=0.3
◇CC	53.63±0.46	48.84±0.19	45.50±0.28	51.85±0.18	47.48±0.30	43.37±0.42	50.64±0.40	45.87±0.36	40.87±0.47
◇RC	52.73±1.05	48.59±1.04	45.77±0.31	52.15±0.19	48.25±0.38	43.92±0.37	46.62±0.34	45.46±0.21	40.31±0.55
◇LWC	53.16±0.87	48.64±0.33	45.51±0.28	51.69±0.28	47.60±0.44	43.39±0.18	50.55±0.34	45.85±0.28	39.83±0.30
◇LWS	56.05±0.20	50.66±0.59	45.71±0.45	53.59±0.45	48.28±0.44	42.20±0.49	45.46±0.44	39.63±0.80	33.60±0.64
◇PiCO	68.27±0.08	62.24±0.31	58.97±0.09	67.38±0.09	62.01±0.33	58.64±0.28	67.52±0.43	61.52±0.28	58.18±0.65
◇CRDPLL	68.12±0.13	65.32±0.34	62.94±0.28	67.53±0.07	64.29±0.27	61.79±0.11	67.17±0.04	64.11±0.42	61.03±0.43
◇PiCO+	71.42±0.25	70.22±0.33	66.14±0.32	70.89±0.13	69.03±0.01	64.22±0.23	69.40±0.12	66.67±0.18	62.24±0.38
◇IRNet	71.17±0.14	70.10±0.28	68.77±0.28	71.01±0.43	70.15±0.17	68.18±0.30	70.73±0.09	69.33±0.51	68.09±0.12
◇ALIM-Scale	74.35±0.18	73.36±0.05	72.50±0.13	74.33±0.11	72.60±0.19	71.69±0.39	73.77±0.05	72.39±0.04	71.68±0.14
◇ALIM-Onehot	72.26±0.23	71.98±0.29	71.04±0.31	71.43±0.21	70.79±0.43	70.14±0.25	72.28±0.28	70.60±0.44	70.05±0.43
◇PiCO+	75.04±0.18	74.31±0.02	71.79±0.17	74.68±0.19	73.65±0.23	69.97±0.01	73.06±0.16	71.37±0.16	67.56±0.17
◇ALIM-Scale	77.37±0.32	76.81±0.05	76.45±0.30	77.60±0.18	76.63±0.19	75.92±0.14	76.86±0.23	76.44±0.12	75.67±0.17
◇ALIM-Onehot	76.52±0.19	76.55±0.24	76.09±0.23	77.27±0.23	76.29±0.41	75.29±0.57	76.87±0.20	75.23±0.42	74.49±0.61
Implementation Details There are mainly three user-specific parameters in ALIM:  $\lambda$ ,  $\lambda_{\mathrm{mix}}$ , and  $e_0$ . Among them,  $\lambda$  controls the trade-off between the initial candidate set and model outputs. This paper proposes two selection strategies for  $\lambda$ , i.e., manually and adaptively adjusted strategies. For the first one, we treat  $\lambda$  as a hyper-parameter and select it from  $\{0.1, 0.2, 0.3, 0.4, 0.5, 0.7\}$ . For the second one, we automatically determine  $\lambda$  using the estimated noise level.  $\lambda_{\mathrm{mix}}$  controls the trade-off between the PLL loss and the mixup loss, and we set  $\lambda_{\mathrm{mix}} = 1.0$  as the default parameter.  $e_0$  is the start epoch of ALIM, and we select it from  $\{20, 40, 80, 100, 140\}$ . Following the standard experimental setup in PLL [17, 25], we split a clean validation set from the training set to determine hyper-parameters. Then, we transform the validation set back to its original PLL form and incorporate it into the training set to accomplish model optimization. To optimize all trainable parameters, we choose the SGD optimizer with a momentum of 0.9 and set the weight decay to 0.001. We set the initial learning rate to 0.01 and adjust it using the cosine scheduler. To eliminate the randomness of the results, we run each experiment three times and report the average result and standard deviation on the test set. All experiments are implemented with PyTorch [27] and carried out with NVIDIA Tesla V100 GPU.
3.2 Experimental Results and Discussion
Main Results For a fair comparison, we reproduce all baselines using the same data generation strategy as previous works [12]. In Table 1, we observe that a large portion of improvement is due to mixup training rather than model innovation. To this end, we compare different approaches under the same mixup strategy. Experimental results demonstrate that our method succeeds over all baselines under varying ambiguity levels and noise levels. The main reason lies in two folds. On the one hand, existing PLL methods are mainly designed for clean samples but ignore the presence of noisy samples. Our method can deal with noisy samples by trading off the initial candidate set and model outputs. On the other hand, existing noisy PLL methods generally need to detect noisy samples, but detection errors are unavoidable. These errors can accumulate and continuously affect the training process. Differently, ALIM can deal with this problem by taking advantage of the initial candidate set and restarting correction. These results prove the effectiveness of ALIM in noisy PLL.
Compatibility of ALIM Since ALIM is a plug-in strategy, we integrate it with existing PLL methods and report results in Table 2. We observe that ALIM always brings performance improvement under noisy conditions, verifying the effectiveness and compatibility of our method. Meanwhile,