SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 2: Compatibility of ALIM on different PLL methods.
PLL	ALIM	CIFAR-10
q = 0.1	q = 0.3	q = 0.5
η = 0.1	η = 0.2	η = 0.3	η = 0.1	η = 0.2	η = 0.3	η = 0.1	η = 0.2	η = 0.3
◇RC	×	80.87±0.30	78.22±0.23	75.24±0.17	79.69±0.37	75.69±0.63	71.01±0.54	72.46±1.51	59.72±0.42	49.74±0.70
◇RC	✓	88.81±0.17	87.16±0.20	85.53±0.05	86.21±0.17	83.64±0.07	79.83±0.43	77.40±0.31	69.13±0.71	56.75±1.59
◇PiCO	×	90.78±0.24	87.27±0.11	84.96±0.12	89.71±0.18	85.78±0.23	82.25±0.32	88.11±0.29	82.41±0.30	68.75±2.62
◇PiCO	✓	94.15±0.15	94.04±0.16	93.77±0.27	93.44±0.16	93.25±0.08	92.42±0.17	92.67±0.12	91.83±0.08	89.80±0.38
◇CRDPLL	×	93.48±0.17	89.13±0.39	86.19±0.48	92.73±0.19	86.96±0.21	83.40±0.14	91.10±0.07	82.30±0.46	73.78±0.55
◇CRDPLL	✓	96.03±0.23	95.01±0.32	93.36±0.10	95.32±0.13	93.27±0.29	91.20±0.06	93.82±0.05	90.20±0.04	84.24±0.28
PLL	ALIM	CIFAR-100
q = 0.01	q = 0.03	q = 0.05
η = 0.1	η = 0.2	η = 0.3	η = 0.1	η = 0.2	η = 0.3	η = 0.1	η = 0.2	η = 0.3
◇RC	×	52.73±1.05	48.59±1.04	45.77±0.31	52.15±0.19	48.25±0.38	43.92±0.37	46.62±0.34	45.46±0.21	40.31±0.55
◇RC	✓	61.46±0.26	60.10±0.23	55.67±0.28	57.43±0.20	52.98±0.27	48.74±0.37	56.40±0.60	51.91±0.12	46.87±0.74
◇PiCO	×	68.27±0.08	62.24±0.31	58.97±0.09	67.38±0.09	62.01±0.33	58.64±0.28	67.52±0.43	61.52±0.28	58.18±0.65
◇PiCO	✓	72.26±0.23	71.98±0.29	71.04±0.31	71.43±0.21	70.79±0.43	70.14±0.25	72.28±0.28	70.60±0.44	70.05±0.43
◇CRDPLL	×	68.12±0.13	65.32±0.34	62.94±0.28	67.53±0.07	64.29±0.27	61.79±0.11	67.17±0.04	64.11±0.42	61.03±0.43
◇CRDPLL	✓	69.98±0.30	68.58±0.16	66.90±0.16	69.60±0.20	67.67±0.22	66.15±0.12	68.75±0.06	67.07±0.29	64.69±0.23
Table 3: Performance on fine-grained datasets.
Method	CUB-200 (q = 0.05, η = 0.2)	CIFAR-100H (q = 0.5, η = 0.2)
◇CC	26.98±1.16	34.57±0.99
◇RC	44.74±2.47	48.03±0.47
◇LWS	18.65±2.15	22.18±6.12
◇GCE	5.13±38.65	33.21±2.03
◇MSE	20.92±1.20	35.20±1.03
◇CRDPLL	44.19±0.72	53.72±0.21
◇PiCO	53.05±2.03	59.81±0.25
◇PiCO+	60.65±0.79	68.31±0.47
♥ALIM-Scale	68.38±0.47	73.42±0.18
♥ALIM-Onehot	63.91±0.35	72.36±0.20
Table 4: Performance with severe noise.
Method	CIFAR-100 (q = 0.05, η = 0.4)	CIFAR-100 (q = 0.05, η = 0.5)
◇RC	33.64±0.82	26.91±0.83
◇PiCO	44.17±0.08	35.51±1.14
◇CRDPLL	57.10±0.24	52.10±0.36
◇ALIM-Scale	70.37±0.06	64.74±0.16
◇ALIM-Onehot	68.82±0.37	63.39±0.82
♥PiCO+	66.41±0.58	60.50±0.99
♥ALIM-Scale	74.98±0.16	72.26±0.25
♥ALIM-Onehot	71.76±0.56	69.59±0.62
integrating ALIM with PiCO generally achieves better performance under severe noise ( $\eta = 0.3$ ). In noisy label learning, self-training suffers from accumulated errors caused by sample-selection bias [28]. To address this issue, researchers propose co-training [29], which contains multiple branches and cross-updates these branches. Among all PLL methods, PiCO is a natural co-training network with two branches: one for classification and one for clustering [17]. The output of the classification branch guides the model to update the clustering prototypes; the output of the clustering branch affects the update of the pseudo label for classification. These results verify the effectiveness of co-training under noisy conditions. Therefore, we combine ALIM with PiCO in the following experiments.
Fine-Grained Classification Performance Considering that similar categories are more likely to be added to the candidate set, we conduct experiments on more challenging fine-grained datasets, CIFAR100H [17] and CUB-200 [24]. Different from previous works [12] that generate candidate labels from the entire label space, this section considers the case of generating candidate labels belonging to the same superclass. Experimental results in Table 3 demonstrate that ALIM outperforms existing methods on fine-grained classification tasks, verifying the effectiveness of our method.
Robustness with Severe Noise To demonstrate noise robustness, we conduct experiments under severe noise conditions. In this section, we compare ALIM with the most competitive baselines (RC, PiCO, CRDPLL, and  $\mathrm{PiCO + }$ ) under  $\eta \in \{0.4, 0.5\}$ . Experimental results in Table 4 show that ALIM succeeds over all baselines under severe noise. Taking the results on  $\eta = 0.5$  as an example, our method outperforms the currently advanced approaches by  $11.76\%$ . These results demonstrate that our ALIM is more robust to noise than existing PLL and noisy PLL methods.
Discussion on Normalization Functions In this section, we compare the performance of two normalization functions. In Table 1, we observe that Onehot(\cdot) performs slightly better on CIFAR-10 and Scale(\cdot) performs slightly better on CIFAR-100. In Table  $3\sim 4$ , Scale(\cdot) generally achieves better performance in fine-grained datasets and severe noise conditions. The main difference between these normalization functions is that Onehot(\cdot) compresses estimated probabilities into a specific class, while Scale(\cdot) preserves prediction information for all categories. For simple datasets like CIFAR-10, the class with the highest probability is likely to be correct. Therefore, the compression operation can
(a) 100 epoch
(b) 200 epoch
(c) 400 epoch
(d) 800 epoch
Figure 2: Distribution of the value in Eq. 6 for clean and noise subsets with increasing training iterations. We conduct experiments on CIFAR-10 ( $q = 0.3$ ,  $\eta = 0.3$ ) with  $e_0 = 80$ .
(a) CIFAR-10 (w/o mixup)
(b) CIFAR-10 (w/ mixup)
(c) CIFAR-100 (w/o mixup)
(d) CIFAR-100 (w/mixup)
Figure 3: Classification performance of manually adjusted  $\lambda$  and adaptively adjusted  $\lambda$ . We conduct experiments on CIFAR-10 ( $q = 0.3$ ,  $\eta = 0.3$ ) and CIFAR-100 ( $q = 0.05$ ,  $\eta = 0.3$ ). We mark the results of the adaptively adjusted strategy with red lines.
(a)  $\lambda_{\mathrm{mix}}$  on CIFAR-10  $(e_0 = 80)$
Figure 4: Parameter sensitivity analysis on CIFAR-10 ( $q = 0.3$ ) and CIFAR-100 ( $q = 0.05$ ) with mixup training. The noise level of these datasets is fixed to  $\eta = 0.3$ .
(b)  $e_0$  on CIFAR-10  $(\lambda_{\mathrm{mix}} = 1)$
(c)  $\lambda_{\mathrm{mix}}$  on CIFAR-100  $(e_0 = 80)$
(d)  $e_0$  on CIFAR-100  $(\lambda_{\mathrm{mix}} = 1)$
reduce the negative impact of other categories and achieve better performance on noisy PLL. For challenging datasets, the class with the highest predicted value has a low probability of being correct. Therefore, this compression operation may lead to severe information loss. More importantly, ALIM consistently outperforms existing methods regardless of the normalization function. Therefore, the main improvement comes from our ALIM rather than normalization functions.
Rationality of Adaptively Adjusted Strategy In this section, we further explain the rationality of our adaptively adjusted strategy. Figure 2 visualizes the distribution of the value in Eq. 6 for clean and noise subsets with increasing training iterations. We observe that this value is an effective indicator for distinguishing clean samples from noisy samples. At the same time, our adaptively adjusted  $\lambda$  serves as a suitable boundary for clean and noisy subsets.
Role of Adaptively Adjusted Strategy It is important to select a proper  $\lambda$  in ALIM. In this paper, we propose two selection strategies: adaptively and manually adjusted strategies. Figure 3 presents the classification performance of these strategies. Experimental results show that different strategies can achieve similar performance. Therefore, the main advantage of this adaptively adjusted strategy is to reduce manual efforts in hyper-parameter tuning. The performance improvement still comes from ALIM, which exploits a coefficient to control the reliability of the candidate set.
Parameter Sensitivity Analysis In this section, we perform parameter sensitivity analysis on two key hyper-parameters: the warm-up epoch  $e_0$ , and the trade-off between the PLL loss and the mixup loss  $\lambda_{\mathrm{mix}}$ . In Figure 4, we observe that choosing an appropriate  $\lambda_{\mathrm{mix}}$  is important for ALIM, and  $\lambda_{\mathrm{mix}} = 1.0$  generally achieves competitive results. Meanwhile, with the gradual increase of  $e_0$ , the test accuracy first increases and then decreases. This phenomenon indicates that ALIM needs warm-up training. But too large  $e_0$  can also cause the model to overfit noise samples.
In PLL, the ground-truth label is concealed in the candidate set. To deal with this task, the core is to disambiguate the candidate labels. In this section, we first introduce two typical disambiguation strategies, i.e., average-based methods and identification-based methods. Then, we briefly review some recent works on noisy PLL.
Average-based PLL The most intuitive solution is the average-based approach, which assumes that each candidate label has an equal probability of being the ground-truth label. For example, Hüllermeier et al. [30] utilized k-nearest neighbors for label disambiguation. For each sample, they treated all candidate labels of its neighborhood equally and predicted the ground-truth label through the majority vote. Differently, Cour et al. [31] maximized the average output of candidate labels and non-candidate labels in parametric models. However, these average-based methods can be severely affected by false positive labels [9].
Identification-based PLL To address the shortcomings of the above methods, researchers have focused on identification-based methods. Different from average-based methods that treat all candidate labels equally, identification-based methods treat the ground-truth label as a latent variable and maximize its estimated probability by maximum margin [32] or maximum likelihood [5] criteria.
Recently, deep learning has been applied to identification-based methods and achieved promising results. For example, Lv et al. [33] proposed a self-training strategy to disambiguate candidate labels. Feng et al. [18] introduced classifier- and risk-consistent algorithms under the uniform candidate label generation assumption. Wen et al. [25] relaxed this assumption and proposed a family of loss functions for label disambiguation. To learn more discriminative representations, Wang et al. [17] exploited contrastive learning to deal with partially labeled samples. More recently, Wu et al. [26] used consistency regularization on the candidate set and supervised learning on the non-candidate set, achieving promising results under varying ambiguity levels. The above methods rely on a basic assumption that the ground-truth label must be in the candidate set. But this assumption may not be satisfied due to the unprofessional judgment of annotators.
Noisy PLL Recently, noisy PLL has attracted increasing attention from researchers due to its more practical setup. Its core challenge is how to deal with noisy samples. For example, Lv et al. [11] utilized noise-tolerant loss functions to avoid overemphasizing noisy samples during training. Lian et al. [12] proposed an iterative refinement network to purify noisy samples and reduce the noise level of the dataset. Wang et al. [13] detected clean samples through a distance-based sample selection mechanism and dealt with noisy samples via a semi-supervised contrastive framework. These noisy PLL methods generally need to detect noisy samples, but detection errors are unavoidable. These errors can accumulate and continuously affect model training. To reduce the negative impact of prediction errors, we propose a novel framework for noisy PLL. Experimental results on multiple datasets demonstrate the effectiveness of our method under noisy conditions.
This paper introduces a novel framework for noisy PLL called ALIM. To deal with noisy samples, it exploits the weighting mechanism to adjust the reliability of the initial candidate set and model outputs. To verify its effectiveness, we conduct experiments on multiple benchmark datasets under varying ambiguity levels and noise levels. Experimental results demonstrate that our ALIM achieves state-of-the-art classification performance on noisy PLL, especially in severe noise conditions and fine-grained datasets. Meanwhile, ALIM is a low-computation plug-in strategy that can be easily integrated with existing PLL frameworks. Furthermore, we interpret the rationality and effectiveness
of the adaptively adjusted strategy. We also conduct parameter sensitivity analysis and reveal the impact of different hyper-parameters. It is worth noting that this paper leverages a global  $\lambda$  to measure the reliability of the candidate set. In the future, we will explore the instance-dependent  $\lambda(x)$  during training. At the same time, we will investigate the performance in more experimental settings, such as class-imbalanced conditions.
6 Acknowledge
This work is supported by the National Natural Science Foundation of China (NSFC) (No.61831022, No.62276259, No.62201572, No.U21B2010, No.62271083), Beijing Municipal Science&Technology Commission, Administrative Commission of Zhongguancun Science Park No.Z211100004821013, Open Research Projects of Zhejiang Lab (NO. 2021KH0AB06), CCF-Baidu Open Fund (No.OF2022025), the National Natural Science Foundation of China (Grant No. 62106028), Chongqing Overseas Chinese Entrepreneurship and Innovation Support Program, Chongqing Artificial Intelligence Innovation Center, CAAI-Huawei MindSpore Open Fund, and Openl Community (https://openi.pcl.ac.cn).
A Interpretation from Objective Functions
In this section, we provide proofs of the Onehot(·) normalization function and the Scale(·) normalization function from the perspective of objective functions.
A.1 Proof for Onehot Normalization
For  $K = 0$ , we choose the following objective function during training:
$$
\max  \sum_ {i = 1} ^ {c} w _ {i} \log P _ {i} + M \left(\sum_ {i = 1} ^ {c} w _ {i} S _ {i} - 1\right)
$$
$$
s. t. \sum_ {i} ^ {c} w _ {i} = 1, w _ {i} \geq 0. \tag {9}
$$
Introduce Lagrange multipliers  $\delta_{i}, i \in [1, c]$  and  $\gamma$  into Eq. 9, we have:
$$
\mathcal {L} = \sum_ {i = 1} ^ {c} w _ {i} \log P _ {i} + M \left(\sum_ {i = 1} ^ {c} w _ {i} S _ {i} - 1\right) + \gamma \left(1 - \sum_ {i = 1} ^ {c} w _ {i}\right) + \sum_ {i = 1} ^ {c} \delta_ {i} w _ {i}. \tag {10}
$$
Combined with the Karush-Kuhn-Tucker (KKT) conditions, the optimal point should satisfy:
$$
\log P _ {i} + M S _ {i} - \gamma + \delta_ {i} = 0, \tag {11}
$$
$$
\sum_ {i = 1} ^ {c} w _ {i} = 1, \delta_ {i} \geq 0, w _ {i} \geq 0, \delta_ {i} w _ {i} = 0. \tag {12}
$$
Since  $S_{i} \in \{0,1\}$ , we have  $MS_{i} = \log (e^{M}S_{i} + (1 - S_{i}))$ . The equivalent equation of Eq. 11 is:
$$
\delta_ {i} = \gamma - \log \left(e ^ {M} S _ {i} + (1 - S _ {i})\right) P _ {i}. \tag {13}
$$
Combined with  $\delta_{i}\geq 0$  in Eq.12, we have:
$$
\gamma \geq \max  _ {i} \left(\log \left(e ^ {M} S _ {i} + (1 - S _ {i})\right) P _ {i}\right). \tag {14}
$$
$\delta_{i} > 0$  is true if  $\gamma > \max_{i}\left(\log (e^{M}S_{i} + (1 - S_{i}))P_{i}\right)$ . According to  $\delta_{i}w_{i} = 0$ , we always have  $w_{i} = 0$ , which conflicts with  $\sum_{i=1}^{c}w_{i} = 1$ . Therefore, we get:
$$
\gamma = \max  _ {i} \left(\log \left(e ^ {M} S _ {i} + (1 - S _ {i})\right) P _ {i}\right). \tag {15}
$$
We assume that only one  $i_0 \in [1, c]$  reaches the maximum  $\gamma$ , then we have  $w_i = 0, i \in [1, c] / i_0$ . Combined with  $\sum_{i=1}^{c} w_i = 1$ , we get  $w_{i_0} = 1$ . Therefore,  $w(x)$  should satisfy:
$$
w (x) = \operatorname {O n e h o t} \left(\log \left(e ^ {M} S (x) + (1 - S (x))\right) P (x)\right). \tag {16}
$$
We mark  $\lambda = e^{-M}$  and convert Eq. 16 to its equivalent version:
$$
w (x) = \operatorname {O n e h o t} \left(\left(S (x) + \lambda (1 - S (x)) P (x)\right). \right. \tag {17}
$$
A.2 Proof for Scale Normalization
For  $K > 0$ ,  $\log w_{i}$  ensures that  $w_{i}$  must be positive. Therefore, the constraint  $w_{i} \geq 0$  can be excluded. Then, the objective function can be converted to:
$$
s. t. \sum_ {i} ^ {c} w _ {i} = 1. \tag {18}
$$
Introduce the Lagrange multiplier  $\gamma$  in Eq. 18, we have:
$$
\mathcal {L} = \sum_ {i = 1} ^ {c} w _ {i} \log P _ {i} + M \left(\sum_ {i = 1} ^ {c} w _ {i} S _ {i} - 1\right) - K \sum_ {i = 1} ^ {c} w _ {i} \log w _ {i} + \gamma \left(1 - \sum_ {i} ^ {c} w _ {i}\right). \tag {19}
$$
Since the optimal point should satisfy  $\nabla_w\mathcal{L} = 0$  we have:
$$
\log P _ {i} + M S _ {i} - K (1 + \log w _ {i}) - \gamma = 0. \tag {20}
$$
Since  $S_{i} \in \{0,1\}$ , we have  $MS_{i} = \log (e^{M}S_{i} + (1 - S_{i}))$ . The equivalent equation of Eq. 20 is:
$$
\log \left(e ^ {M} S _ {i} + (1 - S _ {i})\right) P _ {i} - (K + \gamma) - K \log w _ {i} = 0, \tag {21}
$$
$$
w _ {i} ^ {K} = \frac {\left(e ^ {M} S _ {i} + (1 - S _ {i})\right) P _ {i}}{e ^ {K + \gamma}}. \tag {22}
$$
We mark  $\lambda = e^{-M}$ . Then, we have:
$$
w _ {i} = \frac {\left(\left(S _ {i} + \lambda \left(1 - S _ {i}\right)\right) P _ {i}\right) ^ {1 / K}}{e ^ {1 + (\gamma - M) / K}}. \tag {23}
$$
Since  $\sum_{i}^{c}w_{i} = 1$ , we have:
$$
\sum_ {i = 1} ^ {c} \frac {\left(\left(S _ {i} + \lambda \left(1 - S _ {i}\right)\right) P _ {i}\right) ^ {1 / K}}{e ^ {1 + (\gamma - M) / K}} = 1, \tag {24}
$$
$$
e ^ {1 + (\gamma - M) / K} = \sum_ {i = 1} ^ {c} \left(\left(S _ {i} + \lambda \left(1 - S _ {i}\right)\right) P _ {i}\right) ^ {1 / K}. \tag {25}
$$
Combine Eq. 23 and Eq. 25 and we have:
$$
w _ {i} = \frac {\left(\left(S _ {i} + \lambda \left(1 - S _ {i}\right)\right) P _ {i}\right) ^ {1 / K}}{\sum_ {i = 1} ^ {c} \left(\left(S _ {i} + \lambda \left(1 - S _ {i}\right)\right) P _ {i}\right) ^ {1 / K}}. \tag {26}
$$
Combined with the definition of  $\mathrm{Scale}(\cdot)$  in Eq. 4, this equation can be converted to:
$$
w (x) = \operatorname {S c a l e} \left(\left(S (x) + \lambda (1 - S (x))\right) P (x)\right). \tag {27}
$$
B EM Perspective of ALIM
EM aims to maximize the likelihood of the dataset  $\mathcal{D}$ :
$$
\begin{array}{l} \max  _ {\theta} \sum_ {x \in \mathcal {D}} \log P (x, S (x); \theta) = \max  _ {\theta} \sum_ {x \in \mathcal {D}} \log \sum_ {i = 1} ^ {c} P (x, S (x), y (x) = i; \theta) \\ = \max  _ {\theta} \sum_ {x \in \mathcal {D}} \log \sum_ {i = 1} ^ {c} w _ {i} (x) \frac {P (x , S (x) , y (x) = i ; \theta)}{w _ {i} (x)} \\ \geq \max  _ {\theta} \sum_ {x \in \mathcal {D}} \sum_ {i = 1} ^ {c} w _ {i} (x) \log \frac {P (x , S (x) , y (x) = i ; \theta)}{w _ {i} (x)}, \tag {28} \\ \end{array}
$$
where  $\theta$  is the trainable parameter. The last step of Eq. 28 utilizes Jensen's inequality. Since the  $\log (\cdot)$  function is strictly concave, the equal sign takes when  $P(x,S(x),y(x) = i;\theta) / w_i(x)$  is some constant  $C$ , i.e.,
$$
w _ {i} (x) = \frac {1}{C} P (x, S (x), y (x) = i; \theta). \tag {29}
$$
Considering that  $\sum_{i=1}^{c} w_i(x) = 1$ , we can further get:
$$
C = \sum_ {i = 1} ^ {c} P (x, S (x), y (x) = i; \theta). \tag {30}
$$
Then, we have:
$$
w _ {i} (x) = \frac {P (x , S (x) , y (x) = i ; \theta)}{\sum_ {i = 1} ^ {c} P (x , S (x) , y (x) = i ; \theta)} = \frac {P (x , S (x) , y (x) = i ; \theta)}{P (x , S (x) ; \theta)} = P (y (x) = i | x, S (x); \theta). \tag {31}
$$
In the EM algorithm [16], the E-step aims to calculate  $w_{i}(x)$  and the M-step aims to maximize the lower bound of Eq. 28:
$$
\begin{array}{l} \underset {\theta} {\operatorname {a r g m a x}} \sum_ {x \in \mathcal {D}} \sum_ {i = 1} ^ {c} w _ {i} (x) \log \frac {P (x , S (x) , y (x) = i ; \theta)}{w _ {i} (x)} \\ = \underset {\theta} {\operatorname {a r g m a x}} \sum_ {x \in \mathcal {D}} \sum_ {i = 1} ^ {c} w _ {i} (x) \log P (x, S (x), y (x) = i; \theta). \tag {32} \\ \end{array}
$$
E-Step. In this step, we aim to predict the ground-truth label for each sample:
$$
\begin{array}{l} w _ {i} (x) = P (y (x) = i | x, S (x); \theta) = \frac {P (S (x) | y (x) = i , x ; \theta) P (y (x) = i | x ; \theta)}{P (S (x) | x ; \theta)} \\ = \frac {P (S (x) | y (x) = i , x ; \theta) P (y (x) = i | x ; \theta)}{\sum_ {i = 1} ^ {c} P (S (x) | y (x) = i , x ; \theta) P (y (x) = i | x ; \theta)}. \tag {33} \\ \end{array}
$$
According to Assumption 1, we have:
$$
P (S (x) | y (x), x) = \left\{ \begin{array}{l} \alpha (x), S _ {y (x)} (x) = 1 \\ \beta (x), S _ {y (x)} (x) = 0. \end{array} \right. \tag {34}
$$
It can be converted to:
$$
P (S (x) \mid y (x), x) = \alpha (x) S _ {y (x)} (x) + \beta (x) \left(1 - S _ {y (x)} (x)\right). \tag {35}
$$
Then, we get the equivalent equation of Eq. 33:
$$
w _ {i} (x) = \frac {\left(\alpha (x) S _ {i} (x) + \beta (x) \left(1 - S _ {i} (x)\right)\right) P (y (x) = i | x ; \theta)}{\sum_ {i = 1} ^ {c} \left(\alpha (x) S _ {i} (x) + \beta (x) \left(1 - S _ {i} (x)\right)\right) P (y (x) = i | x ; \theta)}. \tag {36}
$$
We mark  $\lambda (x) = \beta (x) / \alpha (x)$  and  $P_{i}(x) = P(y(x) = i|x;\theta)$ . Then, we get:
$$
w _ {i} (x) = \frac {\left(S _ {i} (x) + \lambda (x) \left(1 - S _ {i} (x)\right)\right) P _ {i} (x)}{\sum_ {i = 1} ^ {c} \left(S _ {i} (x) + \lambda (x) \left(1 - S _ {i} (x)\right)\right) P _ {i} (x)}. \tag {37}
$$
It connects traditional PLL and noisy PLL. In traditional PLL, we assume that the ground-truth label must be in the candidate set, i.e.,  $\beta (x) = 0$ . Since  $\lambda (x) = \beta (x) / \alpha (x) = 0$ , Eq. 37 degenerates to:
$$
w _ {i} (x) = \frac {S _ {i} (x) P _ {i} (x)}{\sum_ {i = 1} ^ {c} S _ {i} (x) P _ {i} (x)}, \tag {38}
$$
which is identical to the classic PLL method, RC [18].
M-Step. The objective function of this step is:
$$
\begin{array}{l} \underset {\theta} {\operatorname {a r g m a x}} \sum_ {x \in \mathcal {D}} \sum_ {i = 1} ^ {c} w _ {i} (x) \log P (x, S (x), y (x) = i; \theta) \\ = \underset {\theta} {\operatorname {a r g m a x}} \sum_ {x \in \mathcal {D}} \sum_ {i = 1} ^ {c} w _ {i} (x) \log P (x; \theta) P (y (x) = i | x; \theta) P (S (x) | y (x) = i, x; \theta). \tag {39} \\ \end{array}
$$
Considering that  $P(x; \theta) = P(x)$  and  $P(S(x)|y(x) = i, x; \theta) = P(S(x)|y(x) = i, x)$ , the equivalent version of Eq. 39 is:
$$
\underset {\theta} {\operatorname {a r g m a x}} \sum_ {x \in \mathcal {D}} \sum_ {i = 1} ^ {c} w _ {i} (x) \log P (y (x) = i | x; \theta). \tag {40}
$$
Therefore, the essence of the M-step is to minimize the classification loss.
C Adaptively Adjusted  $\lambda$
Since  $\eta$  controls the noise level of the dataset, we have:
$$
P \left(S _ {y (x)} (x) = 0\right) = \eta . \tag {41}
$$
After the warm-up training, we assume that the predicted label generated by ALIM  $\hat{y}(x) = \arg \max_{1 \leq i \leq c} w(x)$  is accurate, i.e.,  $\hat{y}(x) = y(x)$ . Then we have:
$$
P \left(S _ {\hat {y} (x)} (x) = 0\right) = \eta . \tag {42}
$$
To estimate the value of  $\lambda$ , we first study the equivalent meaning of  $S_{\hat{y}(x)}(x) = 0$ :
$$
\max  _ {S _ {i} (x) = 0} \left(S _ {i} (x) + \lambda \left(1 - S _ {i} (x)\right)\right) P _ {i} (x) \geq \max  _ {S _ {i} (x) = 1} \left(S _ {i} (x) + \lambda \left(1 - S _ {i} (x)\right)\right) P _ {i} (x). \tag {43}
$$
We simplify the left and right sides of Eq.43 as follows:
$$
\begin{array}{l} \max  _ {S _ {i} (x) = 1} \left(S _ {i} (x) + \lambda \left(1 - S _ {i} (x)\right)\right) P _ {i} (x) \\ \max  _ {S _ {i} (x) = 0} \left(S _ {i} (x) + \lambda (1 - S _ {i} (x))) P _ {i} (x) \right. \quad \begin{array}{l} S _ {i} (x) = 1 \\ = \max  _ {S _ {i} (x) = 1} S _ {i} (x) P _ {i} (x) \end{array} \\ = \max  _ {S _ {i} (x) = 0} \lambda \left(1 - S _ {i} (x)\right) P _ {i} (x) \quad \begin{array}{l} S _ {i} (x) = 1 \\ = \max  _ {i} S _ {i} (x) P _ {i} (x). \end{array} (45) \\ = \max  _ {i} \lambda (1 - S _ {i} (x)) P _ {i} (x), (44) \\ \end{array}
$$
$$
\max  _ {i} \lambda (1 - S _ {i} (x)) P _ {i} (x) \geq \max  _ {i} S _ {i} (x) P _ {i} (x), \tag {46}
$$
$$
\lambda \geq \frac {\max  _ {i} S _ {i} (x) P _ {i} (x)}{\max  _ {i} (1 - S _ {i} (x)) P _ {i} (x)}. \tag {47}
$$
Therefore,  $P(S_{\hat{y}(x)}(x) = 0) = \eta$  can be converted to:
$$
P \left(\lambda \geq \frac {\max  _ {i} S _ {i} (x) P _ {i} (x)}{\max  _ {i} \left(1 - S _ {i} (x)\right) P _ {i} (x)}\right) = \eta . \tag {48}
$$
It means that  $\lambda$  is the  $\eta$ -quantile of
$$
\left\{\frac {\operatorname* {m a x} _ {i} S _ {i} (x) P _ {i} (x)}{\operatorname* {m a x} _ {i} \left(1 - S _ {i} (x)\right) P _ {i} (x)} \right\} _ {x \in \mathcal {D}}. \tag {49}
$$
D Effect of Scaling Factor