Noise Separation guided Candidate Label Reconstruction for Noisy Partial Label Learning
Authors: Xiaorui Peng
Yuheng Jia
Fuchao Yang
Ran Wang
Min-Ling Zhang
Venue: ICLR 2025 Poster (2025)
Keywords: partial label learning
weakly supervised learning
noisy partial label learning
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We propose a noisy partial label learning method with excellent classification performance and solid theoretical guarantee.

Abstract: Partial label learning is a weakly supervised learning problem in which an instance is annotated with a set of candidate labels, among which only one is the correct label. However, in practice the correct label is not always in the candidate label set, leading to the noisy partial label learning (NPLL) problem. In this paper, we theoretically prove that the generalization error of the classifier constructed under NPLL paradigm is bounded by the noise rate and the average length of the candidate label set. Motivated by the theoretical guide, we propose a novel NPLL framework that can separate the noisy samples from the normal samples to reduce the noise rate and reconstruct the shorter candidate label sets for both of them. Extensive experiments on multiple benchmark datasets confirm the efficacy of the proposed method in addressing NPLL. For example, on CIFAR100 dataset with severe noise, our method improves the classification accuracy of the state-of-the-art one by 11.57%. The code is available at: https://github.com/pruirui/PLRC.

Published as a conference paper at ICLR 2025
NOISE SEPARATION GUIDED CANDIDATE LABEL RECONSTRUCTION FOR NOISY PARTIAL LABEL LEARNING
Fuchao Yang', College of Software Engineering, Southeast University
Xiaorui Peng', Yuheng ;
Ran Wang" Min-Ling Zhang26
School of Computer Science and Engineering, Southeast University $ Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University)
*Shenzhen Key Laboratory of Advanced Machine Learning and Applications, School of Mathematical Sciences, Shenzhen University
"Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen University Keey Laboratory of Computer Network and Information Integration (Southeast University) (xiaorui, yhjia, yangfc, zhangml ) @seu. edu. cn; wangranêszu zu. edu. en
ABSTRACT
Partial label learning is a weakly supervised learning problem in which an instance is annotated with set of candidate labels, among which only is the correct label. However, in practice the correct label is not always in the candidate label set, leading the noisy partial label learning (NPLL) problem. In this paper, we theoretically prove that the generalization error of the classifier constructed under NPLL paradigm is bounded by the noise rate and the average length of the candidate label set. Motivated by the theoretical guide, we propose novel NPLL framework that can separate the noisy samples from the normal samples to reduce the noise rate and reconstruct the shorter candidate label sets for both of them. Extensive experiments on multiple benchmark datasets confirm the efficacy of the proposed method in addressing NPLL. For example, CIFAR100 dataset with severe noise, our method improves the classification accuracy of the statc-of-the-art one by 11.57% The code is available at: https://github.ee http://iihiioooopuurr///...L.../
INTRODUCTION
Partial label learning (PLL) (Cour et al., 2011; Nguyen Caruana, 2008; Jiang et al., 2024; Yang et al., 2024) is a weakly supervised learning paradigm (Zhou, 2018) that allows samples to be associated with set of candidate labels, which only one is the ground-truth label. Due to the low annotation cost of collecting partial label dataset in real- worldworld scenarios, PLL has attracted significant attention from the community with many applications, such as automatic face annotation (Chen et al., 2018), web mining (Luo & Orabona, 2010), and face age estimation (Panis & Lanitis, 2014). A variety of methods have been developed to tackle the PLL. problem, including average- methods (Hüllermeier & Beringer, 2006; Cour et al., 2009; Jin & Ghahramani, 2002), graph-based methods (Wang et al., 2022a; Jia et al., 2023b; Xu et al., 2019; Zhang et al., 2016) and pseudo-labeling-bas methods (Lv et al., 2020; Wen al., 2021; Jia et al., 2023a; 2024), etc.
Despite the promise, these PLL methods have been driven by the assumption that the ground-truie label must lay in the candidate label set (CLS), which may not hold in practice due the existence of the non-expers annotators (Li et al., 2020; Hossain & Kauranen, 2015; Shi et al., 2023). Recently, some researchers have turned their attention to more practical setting called noisy partial label learning (NPLL) (Lv et al., 2024). In NPLL, the ground-truth label of some instances may not lie in the CLS. For ease of description, in this article, samples whose ground-truth label is not included in the CLS are referred noisy samples, and those that do include the ground-truth label are termed as normal samples. To address the NPLL task, APLLS (Lv et al., 2024) proposes a family of robust
Corresponding author.

---

Published as a conference paper at ICLR 2025
average losses to against the existence of label noise. PiCO+ (Wang al., 2024) and UPLLRS (Shi et al., 2023) detect noisy samples and treat them as unlabeled samples and exploit them by a semi-supervised learning approach. ALIM (Xu et al., 2023) incorporates non-candidate labels into candidate labels through dynamic weighting to combat the noise problem. A Although these methods have achieved commendable results, they have not theoretically elucidated the essence of the NPLL problem.
To this end, we first theoretically construct the generalization error bound of NPLL, where we find that two key issues may help solve the NPLL problem: i) lower noise rate and ii) a smaller candidate label set. The noise rate is defined as the proportion of noisy samples to the overall samples. In NPLL, the key to reducing the noise rate is correctly detecting noisy samples from the NPLL dataset, because noisy samples can be easily converted into normal ones by exchanging the CLS with non-CLS (ground-truth label must locate at non-CLS for noisy samples). To achieve this goal, we propose a metric called ECK to distinguish normal samples and noisy samples. Considering that the model's output in the early stages of training is not reliable for hard samples, employ the ECK metric to partition samples into three parts: highly reliable normal samples, highly reliable noisy samples, and uncertain samples. As the performance of model improves, we gradually decrease the proportion of uncertain samples to achieve clearer separation between normal samples and noisy samples. Apart from reducing the noise rate, it is also necessary to decrease the size of the CL.S. To achieve this target, we propose to reconstruct the CLS for each instance. The restructured CLS aims to achieve two objectives: reducing its size and containing ground-truth label. We propose an mustance-adaptii parameter to balance these two objectives, thereby reconstructing a faithful shorter CLS for each instance. Through sample separation and CLS reconstruction, we can reduce the generalization bound, and accordingly promote the classification performance. Our method serves as a plug-in for PL.L methods enhance their performance on NPLL datasets. We extensively evaluate our method on several benchmarks, which validates the efficacy of our method. Our contributions can be summarized as follows:
For the first time, we provide the generalization error bound of the classifier constructed under NPLL, where we find that lower noise rate and smaller candidate label are two key factors to solve the NPLL problem.
With insights drawn from the theoretical findings, we propose a novel NPLL method which includes two components: progressive sample separation and CLS reconstruction. Based on those two novel components, our method can effectively reduce the noise rate while simultaneously decreasing the size of CLS.
Our method can serve plug-in in for existing PLL methods to enhance their performance on NPLL datasets. Extensive experimental results validate that our method outperforms the current state-of-the-ard (SOTA) methods by large margin, e.g., 11.57% % improvement on CIFAR100 with extreme noise and ambiguity level.
2 PROBLEM SETUP
Let X' = Rd be the input space of d dimensions and y = {1...., be the label space, and C be the number of classes. We denote D {(x, Y:) the training dataset with samples, and each tuple in D comprises of a vector æ; X and a CLS Y, Y. Let Y, = = Y, represents the non-CLS of æt. In traditional PLL, the ground-truth label y/ is concealed in CLS i.e. yi E Y, (Wang et al., 2022b; Chen et al., 2018), while NPLL allows some samples of which the ground-truth label y locates outside the candidate set Y, that is y1 e Y, (Lv et al., 2024; Xu et al., 2023). Let ff f(:; 8) denote a deep neural network parameterized by 8, which transforms x to into probability prediction vector p. Our goal is to train multi-class classifier f(x; f) using the NPL. dataset D.
THEORETICAL ANALYSIS
Here, we provide a generalization error bound for NPLL problem to analyze the factors that can enhance the generalization ability of the model. The true risk with respect to the classification model f( 0) is
R() Be (F([(x), v)).
2

---

Published as a conference paper at ICLR 2025
7. Ei= ((f())), y4) denote the corresponding empirical risk. However, in NPLL, cannot minimize the empirical risk directly as the ground truth label y is inaccessible. Therefore, we need to train the model with (f( 1 Ei=1 CPLL (f (x4), Yi), where Y, denotes the CLS of the instance 24. Let f arg mi (f) be the empirical risk minimizer, and f arg min f.FF R(f) be the true risk minimizer. Let LPLL (f Y.) = কे. EeEv, [(f(x;), c) be the loss for PLL. Let n' Li= Tyy Y:) be the number of normal samples. Let represent the noise rate in the NPLL dataset which is defined as the proportion of instances whose ground-truth, label does not belong to CLS, i..., n-x' and let a = / Lis.g.ey, [Ya be the mean size CLS for normal samples. Besides, we define the function space Hy for the label y E y as (h x + fy(r: E F} where fx) represents the predicted probability of the y-th class for æ. Let Ri, (Hyy) be the expected Rademacher complexity (Bartlett & Mendelson, 2002) of H, with sample size n. Then we have the following theorem.
Let
Theorem 1. Assume the loss function (af y) is p-Lipschitz with respect to (xx for all E y and uppper-bounded by M. For noise rate 0 E < and mean CLS size for normal samples 1 <<a a < for any ò > O, with probubility at least 1 ô, we have
R(F) - R(f)) s 2(1 - )M 2v Σ Rr (H2, ) 2M 1=1
log
a
2n
The proof of Theorem 1 is provided in Appendix A.1. It can be observed that the generalization performance of f primarily influenced by three factors: the noise rate the mean CLS size a of normal samples, and the sample size n. As n + oo, 6 - 0 and a - 1, Theorem shows that the generalized error bound will be reduced, and the empirical risk minimizer f will get closer to the true risk minimizer f Obviously, smaller noise rate and smaller CLS size a will bring better generalization performance.
4 PROPOSED METHOD
Based on the theoretical analyses above, we can find that the key to solving the NPLL problem lies in reducing the noise rate in the dataset and minimizing the size of the CL.S. To achieve these goals, our method comprises two components. First, we try to separate the samples into highly reliable normal samples, highly reliable noisy samples, and uncertain samples. Second, based the above sample separation, we separately perform CLS reconstruction with an pestance-adaptivn parameter for these samples. As the model is trained on reconstructed CLS, its performance improves progressively, which in turn promotes the aforementioned processes, ultimately leading to reduction in noise rate while simultaneously decreasing the size of the CLS. The pseudo-code summarized Algorithm 1. The framework of our method is showed in Fig. 4.
4.1 PROGRESSIVE SAMPLE SEPARATION
We first propose a metric, the consistency error between CLS-based and KNN- based pseudo-label (ECK), to distinguish normal samples and noisy samples. Considering that the model output is reliable in the early stages of training for some hard samples, we divide the samples into three parts based the ECK metric: highly reliable normal samples, highly reliable noisy samples, and uncertain samples. As the performance of model improves, we gradually increase the proportion of highly reliable samples to achieve an accurate separation between normal samples and noisy samples.
According to the smooth assumption (Wang et al., 2022a), similar samples tend to share the same label. For each sample I4, we select its K -enerestt neighbor samples N; with cosine distance to construct a KNN-based pseudo-label:
qi Normalize
Σ Sij Pj JEN.
where sij is cosine similarity of features between sample x; and zj. Pj = f(zz:() is the probability prediction vector of æ.j, and Normalize()) is normalization function that ensures the sum of vector
3

---

Published as a conference paper at ICLR 2025
Algorithm Training Process of the Proposed Method
Input: NPLL training dataset D = {( Yi) classifier f. PLL method LPLL, warm-up epoch
ews uncertainty-end epoch eend, number of epoch emaz hyper-parameters X, Output: The optimized muti-class classifier f.
1: for epoch = 1, .... Cur do
2: Warm-up training f by LPLL with Dataset = {(2,, k
3: end for
4: for epoch = 1, 1, .... Emaz do
5: Calculate percentage " and Tu by Eq. (5)
6:
Calculate ECK E, for each instance x; by Eq. (3)
7:
Er // E is ru-percentile of E2
8:
I F is ny-percentile of F;
9:
Calculate u, for each instance æ; by Eq. (4) // Sample Separation
Calculate Y by Eqs. (8) (9) (10) CLS Reconstruction
11: Train f by LPLL with Dataset D = { Y))
end for
equal 1. For each sample Fıg further construct CLS-based pseudo label by normalizing the model's prediction to the candidate labels:
qi Normalize (pa O S(Y)))
where S(-) transfers CLS to a vector where the i-th element equal to 1 if the i-th label is in the CLS, and 0 otherwise. Operation @ is Hadamard product, i.e., element-wise multiplication. Then, we calculate the ECK of each sample æę using the cross entropy between qs and qt i.c.,
E, 0==- Σ Ti log Gijs
where Øi and Qz are the j-th elements of the vectors Ti and Jt. respectively. When the ground-truth label of a sample is included in the candidate labels, the CL-S-based pseudo-labels and KNN-based pseudo-labels tend to be similar, resulting the ground-truth -otund label of sample locates at the non-candidate labels, there will be a significant discrepancy between the CLS-based pscudo-label and KNN-based pscudo-label, leading to larger value of ECK.
relatively small value of ECK. However, when the
We employ a dual- -olethheehhld approach to separate the samples into three distinct groups: highly reliable normal samples, highly reliable noisy samples, and uncertain samples. Then, variable o; to denote the group to which the i-th sample æ; belongs. Specifically, when v; is 1, O, and -1, it indicates that the i-th sample belongs the normal samples, uncertain samples, and noisy samples, respectively. Consequently, we have
ternary
1.
S1 ( << E, u..
0.
-1, E, u.
where the thresholds and are used to balance the quality and quantity of sample separation. As shown in Fig. 4, with model training, the value of / increases while the value of u decreases, until / equals and all samples will be separated into noisy samples and normal samples. In practice, as the ECK changes across different epochs, 1 and are assigned the and ru percentiles of ECK, respectively, to ensure stability during training. Due the typically lower proportion of noisy samples compared to normal ones, we employ the rate of percentile change between and Tu- We linearly ramp up and taper off ru. i.c.,
À = [Aril
|Aral
(0 < < 1) to regulate the
1
epoch Cw
1 max epoch Pur
X
Ti 1+ max(
.1), Tu =
(5)
Bend - Ew
Gend tw
where epoch is the current training epoch, enp is the warm up epoch and the Gend denotes the epoch where all samples are separated, i.e., ri Tu- So far, have partitioned the samples into three
4

---

Published as a conference paper at ICLR 2025
groups: normal samples (ag|u,, 1, Vi). noisy samples {æy|u, -1, Vi}. and uncertain samples (ag|u; = 0, Vil. As the model iterates, the model is better trained and accordingly the number of uncertain samples will gradually decrease.
Difference with the previous noisy sample separation methods. The previous methods, PiCO+ (Wang et al., 2024) and UPLLRS (Shi et al., 2023), also employ partitioning strategy for noisy samples. However, different with our method, they overlook the existence of the uncertain samples during the training process. Directly partitioning uncertain samples as noisy samples normal ones will inevitably lead to more errors, especially in the early stages of model training when the model's capability is weak. Our proposed dual-threshold strategy gradually increases the partitioning degree as the model's capability improves, and the experimental results in Table 5 and Table 7 demonstrate its superiority over the previous methods. Additionally, these methods just treat the partitioned noisy samples as unlabeled samples, ignoring the prior that ground-truth label of noisy samples locates in the non-CL.S. In contrast, the CLS reconstruction method we propose the next section will utilize this important prior.
4.2 RECONSTRUCTION OF CANDIDATE LABEL SET
In the previous section, we have identified subset of normal and noisy samples. A straightforward approach to handle noisy samples is treating their non-CLS as their new CLS (ground-truth must locate non- con-CLS for noisy samples). Although this approach can reduce the noise rate, will increase the size of CLS for samples (the size of CLS is usually smaller than that of the non non-CLS) thereby violating the conclusion in Theorem 1. Therefore, we propose to reconstruct the CLS for all instances, which aims to reduce the size of the CLS while ensuring that the ground-truth label is included in the CLS. To achieve those two goals simultaneously, we formulate the following optimization problem:
Σ1(r: ((us + 0) (1811 3(q: - S(P.)) i=1
min 4
(6)
s.t. Wi, v2 =; の, 8: CY,
Wf, 02 =- * cy
where Y, is reconstructed CLS of a; and 5(Y.) transfers the reconstructed CLS 9, to a vector of that the j-th element equal to 1 the label j in Y,, and otherwise. |- |is the cardinality of set and (,,-) denote the inner product of two vectors. T(-) is an indicator function which outputs 1 if the condition holds, 0 otherwise. qi the KNN-based pseudo label of x, obtained by Eq. (1).
Minimizing [& the optimization objective aims to reduce the CLS size, while minimizing -(q:, (Y.)) is designed to select more reliable labels to reconstruct the CLS. The hyper-parameter 3 balances these two objectives. The solution is showed on Appendix A.2, i.e., Ý, {ilj E Y., 9; > {) if x; a normal sample (v; = 1) while Y; = {íll E y Y., > }) if ax, is noisy sample (wy 1). Considering the various learning states of samples, for samples with high confidence, intensify the reduction of CLS with a lower value of 3. Conversely, for samples with low confidence, is appropriate to set higher 3 to ensure the ground- -oound-truth label within the CLS. Therefore, transform the hyper-parameter / into instance-adaptive parameter. Specifically, for the sample 2;. the instance-adaptiv parameter is
B, = r(:),,
where T(.) is a function, and the output of r(-) is inversely proportional to the pseudo-label confidence. In practice, we use the maximum value in the pseudo-label to represent the confidence of the sample, ie. T(q:)= = maa((g;)> where 3 is the syper-parameter in Eq. (6). Finally, the reconstructed CLS is as follows:
B
1
y, = فilا E Yi, qi
_ }.
if = 1,
(8)
B
y = (ilj y
Yú, 4s
>
1
},
if 0
-1
(9)
B
5

---

Published as a conference paper at ICLR 2025
In the early stages of training, there are fewer selected reliable samples. Directly excluding uncertain samples { = 0, vij from training will slow down the convergence to some extent. Therefore, for these uncertain samples, we choose to add the label with the highest pseudo-label probability from the non-CLS to the CLS, and remove the label with the lowest pseudo-label probability from the CLS, i.e.,
P Y, Uc} [d].
if = 0,
(10)
where c arg maxjEy-Y, Ki and c' arg minjer, qij.
Remark. Driven by theoretical analyses in Section 3, we propose a CLS reconstruction method that corrects noisy labels while reducing the size of the CLS. Note that all the previous methods do not consider the impact of CLS size on generalization performance. The experimental results in Table 5 validate the effectiveness of the proposed reconstruction method.
4.3 ITERATIVE IMPROVEMENT
Utilizing the CLS-reconstructed dataset obtained from the previous section, we optimize the model by
n 72 Σ
min 8
P.),
(11)
where LPLL can be any PLL method. Eq. (11) can be solved by applying the stochastic gradient decent (SGD) method. As the performance of the model improves, it will in turn enhance the effect of sample separation and CLS reconstruction. By alternating optimization, the model thus achieves the excellent performance on NPLL dataset. As LPLL can be any PLL method, our approach serves plug-in enhance the performance of traditional PLLL methods on NPLL datasets.
5
EXPERIMENTS
5.1
EXPERIMENTAL SETUP
Datasets. Following the previous works (Wang et al., 2024; Xu et al., 2023; Qiao et al., 2023), we first evaluated our method on two benchmark datasets, CIFARIO and CIFAR100 (Krizhevsky et al., 2009). Following the same dataset generation process of the previous works (Wang et al., 2024), for each benchmark dataset, we manually corrupted its training dataset into NPLI dataset. Specifically, we first constructed the candidate label set for each sample by manually flipping incorrect labels 7 # y to candidate labels with probability = P E YiI # y). Then, each sample has a probability y of being the noisy sample, i.e., the probability of that CLS do not contain ground-truth label is y. We denote the probability 77 as the ambiguity level and the probability r as the noise level. We consider 7 E (0.3, 0.4, 0.5} for CIFAR10 and (0.03, 0.05, 0.1) for CIFAR 100. We choose noise level E (0.2, 0.3, 0.4). Considering that similar categories are more likely to be added to the candidate set, partial label annotations are prone arise from fine-grained image scenarios. We conducted experiments on three fine-grained datasets CIFAR 100H, CUB200 (Welinder et al., 2010) and Flower (Nilsback & Zisserman, 2008). We further evaluated our method on two real-world crowdsourced datasets Treeversity and Benthic (Schmarje et al., 2022) to confirm the effectiveness of the our method in real-world applications. The detailed information of these datasets is presented in Appendix A.6.
Baselines. We compared our method with four recent PLL methods including PRODEN (Lv et al., 2020), CC (Feng et al., 2020), CRDPLL (Wu et al., 2022), and PaPi (Xia et al., 2023) and four SOTA NPLL methods including FREDIS (Qiao et al., 2023), PiCO+ (Wang et al., 2024) and ALIM (Xu et al., 2023) (two variants: ALIM-Onchot and ALIM-Scale). It is noteworthy that ALIM can also serve as a plug-in applied to other PLL methods.
Implementation Details. Following the experimental setup (Xu et al., 2023), we splited clean validation set from the training set to determine hyper- syper-parameters. Then, we transformed the validation set back to its NPLL form and incorporated them into the training set to retrain the model. Due to our approach is a plug-in method, we default to combining our method with PaPi (Xia et al., 2023) for NPL.L unless specifically stated. More experimental details showed in Appendix A.3.

---

Published as a conference paper at ICLR 2025
Table Accuracy comparisons CIFARIO and CIFAR100 under various ambiguity levels and noise levels "y. Bold indicates the best result. Accuracies are presented in percentage (%) form. experiments were conducted three times under the same three distinct random seeds.
CIFARIO
Method
7=
7 - 0.2
7- 0.3
5
0.4
0.2
7 0.3
- 0.4
7
0.2
2
0.3
0.4
PRODEN (ICMC220)
27.34
1 0.53
67.20
0.99
57.74
0.56
71.43
0.54
59.28
0.82
46.87
1.40
63.94
19.38
1.1
32.03
CC INeurilS' 20)
75.09
0.37
6348
54,12
0.34
68.08
0.9
51.16
0.36
42.24
1.31
0.24
11.38
1.60
28.57
CRDPLL (IMCL') [IMCL/23
84.61
+ 0.19
50
0.46
71.43
0.3
81.60
0.46
72.79
53.24
2.30
76.92
56.78
0.76
12.60
PaPi (CVPR
0.61
86.21
FREDIS (ICML/2]]
92.09 1: 0.29
57.91
1.74
84.15
0.19
89.25
2.18
2.50 64.78
77.74
0.70
0.59
79.73
2.70
52.68
1.22
[TPAMI
0.35
94.22
0.52
93.8
ALIM-Scale (NearIPS 23]
94.97
+ 0.27
94.10
0.16
93.31
0.27
94.38
0.24
93.41
89.62
0.66
93.92
90.10
0.56
69.78
ALIM Onshot [NeulPS'2]
95.13
0.10
94.39
93.68
0.14
94.64
0.04
58.88
0.30
0.15
90.71
0.73
65.57
2.04
Ours
96,91
0.17
96.80
96.47
0.19
96.78
0.10
96.23
0.66
96.03
0.56
96.55
0.02
94.54
82.63
1.70
Method
0.05
0.1
7=22
7
7
PRODEN [ICM 201
57.83
0.49
18.00
0.31
40.10
0.37
55.39
0.60
45.36
1.16
36.11
0.40
50.88
40.02
1.40
28.81
0.89
(NearIPS'a) 20]
57.73
0.70
48.66
0.28
38.26
1.31
1.23
35.31
40.60
28.56
0.20
CRDPLL [IMCL/2]
63.91
0.53
59.16
55.16
63.02
0.52
57.77
0.48
53.64
0.29
61.43
0.221
54.77
0.08
48.50
PuPi [CVPR'23]
69.83
0.57
61.99
0.224
59.71
0.68
68.64
0.61
62.72
58.63
0.25
67.64
61.8
0.70
55.60
FREDIS [JCML/2]]
66.94
0.10
61,83
57.99
0.35 67.48
0.57
0.77 62.72
57.19
0.68
66
$7.60
0.64
45.09
1.7.2
BCD+ [TPAMI]3]]
74.32 + 0:43
72.68
0.28
67.31
0.58
73.33
048
70.17
0.62
65.01
0.48
62.67
56.25
0.54
47.75
ALIM-Scole (NeuriPS 23]
76.02
74,49
0.22
64.61
ALIM-Oneho [NeurPS'23]
76.29
0.19
74.83
0.12
73.39
1.14
74.92
0.48
74,40
0.06
71
1.02
61 24
0.57
58.01
1.03
47.27 1.82
80.73
0.20
0.20
78,89
0.23
Table Accuracy comparisons when the methods are used a plug-in on CIFARIO and CIFARI00 under various ambiguity levels and noise levels "y. Bold indicates the best result. Accuracies are presented in percentage (5) form.
CIFARIO
Method
N 0.2
033
0.4
0.2
U.3
0.4
0.2
U.3
U.1
PRODEN
78.00
67.57
57.75
71.31
60.22
48.38
64.62
49.95
31.93
PRODEN ALIM-Onehot
90.83
88 64
84.87
89.15
84.95
77.71
86.63
79.89
42.85
PRODEN ALIM-Scale
92.05
89.83
83.22
90.58
85.78
71.27
87.10
66.34
38.14
PRODEN Ours
94.35
94.10
93.30
94.21
93.80
90.48
94.00
93.27
62.29
CRDPLL
84.40
79.61
71.46
81.97
72.43
55.06
76.93
56,40
31.96
CRDPLL ALIM- Onehot
88.30
83.64
77.04
56.58
60.72
32,95
CRDPLL + ALIM-Scale
92.06
90.42
85.86
90.81
85.36
73.85
87.06
68.57
40.50
CRDPLL Ours
95.29
95.25
94.86
95.17
94.44
90.43
94,28
83,40
54.66
PaPi
69.83
61.99
59.71
68.64
62.72
58.63
67.64
61.98
$3.60
PaPi AL.M-Oneeeee
95.64
94,88
92.57
95.67
94,26
91.01
94.45
90.50
66.50
PaPi + ALIM-Scale
95.11
94.40
92.32
95.94
94.15
90.25
93.24
88.41
58.92
PaPi Ours
96.74
96.67
96.41
96.84
95.47
95.39
96.53
95.41
81.85
CIFARTUR
Method
1)
U.00
0.05
0.1
0.2
0.3
014
0.2
0.3
0.4
0.2
0.3
0.4
PRODEN PRODEN ALIM-Onehot
58.10
40.30
$4.89
46.60
35.85
51.87
41.63
29,84
74.07
71.18
68.26
72.47
69.76
66.36
17
62.33
53.44
PRODEN ALIM-Scale
74.98
73.50
74.49
72.14
66.88
70.64
64.17
55.10
PRODEN + Ours
75.66
74.59
72.29
75.08
73.68
70.19
74.74
69.93
61.47
CRDPLL
64.36
59.01
55.20
62.45
58.26
53.37
61.26
$4.72
48.69
CRDPLL + ALIM-Onehot
68.31
63. 70
57.20
67.21
64.24
55.37
67.09
62.28
50.19
CRDPLL ALIM-Scale
70.93
67.99
58.60
70.09
67.30
57.43
68.24
63.03
53.74
CRDPLL Ours
74.46
72.93
69.96
73.58
72.39
68.19
72.27
69,55
PaPi
69.83
61.99
59.71
68.64
62.72
58.63
67.64
61.98
55.60
PaPi + ALIM-Onehot
80.37
79.20
77.58
79.64
77.77
74.04
PaPi + ALIM-Scale
81.50
80.23
78.77
80.51
78.79
77.11
79.16
75.86
63.02
PaPi Ours
81.70
80.69
80.08
80.85
79.88
79.29
79.78
78.23
78.10
5.2 MAIN RESULTS
Our method achieves SOTA classification accuracy. shown Table 1, on the CIFAR10 and CIFARI00 100 datasets with different ambiguity levels and different noise levels % method outperforms all the compared methods by large margin. For example, on CIFAR 100 with 7j = 0.1, method improves the best compared method by 4.71 %. 8.15% and 11.57% % three different noise levels = 0.2, 0.3 and 0.4 respectively. On CIFAR10 with = 0.5, our method performs 2.48% 1,8% 3.83% % and 7.04% % better than the best compared method.
Our method can promote the performance of different PLL methods on NPLL dataset. Since method is plug-in approach, combine it with various existing PI.L methods and compare the

---

Published as a conference paper at ICLR 2025
Onchot ALPAScull Original
Mcan Accuracy
Mean Accuracy
Meen Accuracy
LhLL
Dum
Gurs AUN Scale
AUM Scare
ALlM-Cnnhott
CROPLL
PRODEN
varying PLL methods
(b) varying noise level
varying ambiguity level
Figure 1: The mean accuracy CIFARIO of our method and the ALIM across different PLL methods, ambiguity levels, and noise levels. The results derived from the statisties presented in Table The results on CIFARIO 100 showed Appendix A.5.
results with that of ALIM (Xu et al., 2023) in Table 2. It can be observed that our method significantly enhances the performance of different PLL methods across various ambiguity levels and noise levels and outperforms ALIM in all cases. For example, on CIFARI00 (1) = 0.1, y 0.4), integrated with PRODEN, CRDPLL and PaPi, our method performs 6.37 % 12.42% % and 15.08 % better than the ALIM. Fig. shows the mean accuracy of our method and that of ALIM across different PLL. methods, ambiguity levels, and noise levels. As the difficulty of the problem increases (with higher levels of noise and ambiguity), our method demonstrates a more pronounced enhancement in the performance of the PLL model compared to the current SOTA method.
Our method remains effectiveness on fine-grained datasets for NPLL task. Partial label annotations are easily arisen from fine-grained image scenarios annotators are more likely to confuse similar cate- gories. Therefore we further conduct experiments on fine-grained datasets, CIFARIONS 2009), CUB200 (Welinder et al., 2010) and Flower (Nilsback & Zisserman, 2008). The results in Table 3 demonstrate that our method significantly improves
Table 3: Accuracies (%) on fine lieeggrined datasets.
CIFARTOOH
CUB-200
Flower
Method
0.03
0.05
7
0.3
(Krizhevsky et al.,
PaPi
63.94
43.56
74,95
ALIM-Onchoot
48.58
PaPi AL.MM-Scale
74.34
51.44
78.47
PaPl Ours
76.93
52.78
81.72
PLL method PaPi on fine-graieed NPLL task by 6.77% % 9.22% and 12.99% on Flower (n = 0.05, 0.2), CUB200 (1) = 0.03, 0.3) and CIFAR 100H (1) = 0.5, 7 0.2). And our method, as expected, also outperforms the current SOTA NPLL method ALIM. This results further substantiate the efficacy of our method.
Our method is skilled in real-world datasets for NPLL task. To further verify the superiority of our method, we evaluated different methods on real- world crowdsourced datasets: Treeversity and Ben- thic. The details of those datasets and the experimen- tal settings showed in Appendíx A.6. As indicated in Table 4, our method still significantly outperforms the current SOTA methods in realistic experiment. Especially, in more challenging scenarios like Treeversity2, the performance gap is even more significant.
Table 4: Accuracies (%) on real-world datasets.
Method Daraset
Treeversity2
Treeversity:
Benthic2
Papi
81.07
82.55
80.90
MM-Slle
82.72
81.46
PaPi + Ours
86.41
86.67
83.47
Our method is capable of effectively segregating the reliable samples. We validated the rationality the progressive sample separation Fig. which visualizes the distribution of ECK and the boundary values I and u. It can be observed that as the training epoch increases. ECK increasingly distinguishes between normal samples and noisy samples. Meanwhile, the boundary values and can effectively separate highly reliable normal samples from highly reliable noisy samples. Moreover, the number the highly reliable samples gradually increases with the training of the model until all samples are selected. Fig. 3(a) illustrates the trends in separation accuracy and the number of selected highly reliable samples as the model training, where we find that with the model training. the highly reliable selection quantities of both normal and noisy samples increase, while the quality maintains at high level. We further evaluated separation accuracy of normal and noisy samples for different NPLL method in Appendix A.7. We can observe that method outperforms other NPL.I
8

---

Published as a conference paper at ICLR 2025
wike
roton 73 99%
retoo 30,005
2000
20
00 110
24
ECK
BCX
BCX
(a) 100 epoch
(b) 140 epoch
(c) 180 epoch
(d) 500 epoch
Figure 2: Distribution of the ECK in Eq. (3) for real normal and noisy samples with increasing training iterations.
The experiment is conducted on CIFARIO (7) 0.5, y 0.3 ). The term "ratio" on the graph represents the proportion of cach selected subset of samples relative to the total number of samples. The results on CIFAR 100 are showed are Appendix A.5.
Number enterted strpp
Durbbat GLS
Romm
Moon CL5
ACC Notmel Sumples
ACC No Samplen
15
59
205
179
136
311
251
355
750
(a) Separation quantity Separation quality
(b) Ground-truth hit Meun CLS
first of Theorem
0.3). (a) demonstrates that with the progression of training iterations, the selection quantities of both normal and noisy samples increase, while the quality (accuracy) able be sustained high level. (b) reflects that, as training iterations continue, the reconstructed CLS size consistently decreases (green curve) while ensuring a high ground-truth label hit rate (the rate of samples that ground-truth label locates in the reconstructed CLS). There are no uncertain samples after 200 epoch, because we set Cand 200. (c) indicates that, as the model training progresses, the value of 1-s continuously decreasing. Note this value is positively related generalization error bound in Theorem 1. The results on CIFAR 100 are showed in Appendix A.5.
Figure 3:
The experiment is conducted on CIFARIO (1) = 0.5,
methods with a substantial gap especially when the noise level is high, which validates the superiority of ECK and dual-thresholds approach sample separation.
Our method achieves excellent reconstruction of the CLS. The objective of CLS reconstruction is to ensure the ground-truth label is retained within the CLS while minimizing the size of the CLS. We conducted experiments on CIFAR1O (n = 0.5, y = 0.3) to validate this in Fig. 3(b), where we can find the reconstructed CLS size decreases while ensuring high ground- -tunt label hit rate with the progression of training iterations. The above observation validates the effectiveness of our CLS reconstruction method.
Our method narrows generalization error bound. Our approach motivated by the generalization error bound in Theorem From The- orem 1, the generalization error bound of NPLL is positively correlated with factor ع= Fig. 3(c) indicates the factor I= is continu- ously decreasing as the model training progresses which confirms the proposed approach can narrow the generalization error bound.
Table 5: Ablation study of our method (%).
CIFARIO
CIFARION
Method
7 = 0.5
q 0.05
0.3
0333
Ours
All the components contribute to the proposed model. We con- ducted experiments to assess the effectiveness of each component. Specifically, we tested Ours v/ by discarding the selected uncertainty samples in model training: Ours v2 by separating normal samples and noisy samples in one-off manner, i.e., by maintaining ( through- out the training process; Ours v3 by removing CLS reconstruction and using non-CLS as CLS for noisy samples. From Table 5, can that all the ingredients of our method contribute to the performance improvement. Comparing Ours v2 with Ours, is evident that the wwo-thresholds separation strategy is important for more challenging datasets, the performance decreases 4.72% on
Ours
78.66 68
Ours
Ours Papi
81.32
74.62
61.98
62.72

---

Published as a conference paper at ICLR 2025
CIFARIO0 100. Comparing Ours v3 with Ours CIFAR-10 and CIFAR-100 datasets, the performance decreased by 14.09% and 5.26% respectively, indicating that CLS reconstruction is significantly effective in NPLL task.
6 RELATED WORK
6.1 PARTIAL LABEL LEARNING (PLL)
In PLL. an instance is associated with candidate label set in which the ground-truth label is concealed. PLL aims to learn a multi-class classifier from the ambiguous candidate label set (CLS) (Hüllermeier & Beringer, 2006; Nguyen & Caruana, 2008). The key to PLL is disambiguation, i.e., finding the correct label from the CLS. The average-based PLL methods (Hüllermeier & Beringer, 2006; Cour al., 2011 1) treat all the candidate labels as ground-truth labels equally, while the identification based PLL methods (Zhang et al., 2022; Jia et al., 2023b) treat the ground-truth label as latent variable and try to find it from the candidate labels. Graph- PLL. methods (Wang al., 2022a; Jía et al., 2023b) use the similarity (or dissimilarity) relationship of features and candidate labels to achieve label disambiguation. Recently, deep learning-bas -baedd PLL methods (Feng et al., 2020; Wen et al., 2021) have become popular due to their excellent classification performance. For instance, (Lv et al., 2020; Feng et al., 2020; Wen et al., 2021; Zhang et al., 2022) perform disambiguation by dynamically adjusting the confidence of the candidate labels as pscudo-labels to guide model training. PiCO (Wang et al., 2022b) introduces contrastive learning into PI.I and disambiguates CLS through class prototypes. CRDPLL (Wu al., 2022) performs disambiguation consistency regularization. PaPi (Xia et al., 2023) adopts the model outputs for disambiguation and utilizes the results of disambiguation to guide the learning of representations. However, the above PLL methods all assume that the ground truth label must locate in the candidate set, which may not hold in real- woald scenarios the non non-expert annotators may make erroneous judgments.
CLS with
6.2 NOISY PARTIAL LABEL LEARNING (NPLL)
Recently, some researchers have turned their attention to a more practical setting called Noisy Partial Label Learning (Lvv al., 2024) which allows the CLS of some instances do not contain the ground- truth label. The problem is also referred to as Unreliable Partial label learning (UPLL) (Shi et al., 2023; Lv et al., 2024) in some researches. UPLLRS (Shi et al., 2023) proposes a recursive separation strategy to first regard the noisy sample as unlabeled data. Then UPLLRS employs semi-supervised learning approach to train the model for these unlabeled data, while other PLI samples are tackled by PLL loss. PiCO+ (Wang et al., 2024) detects the noisy samples with KNN and then employs semi-supervised methods to handle these noisy samples. FREDIS (Qiao et al., 2023) not only moves labels from non-CLS to CLS for handling label noise, but also performs disambiguation by moving incorrect labels from CLS to non-CLS. ALIM (Xu et al., 2023) trades off the initial candidate set and model prediction with an adjusting label importance mechanism. However, these methods are designed based on empirical experience without theoretical support. Through our theoretical analysis are able identify the shortcomings of the above empirically designed methods.
in Theorem Moreover, PiCO+, UPLLRS and FREDIS do not take into account sample uncertainty in separating noisy samples. which easily cause separation errors leading large noise rate. ALIM incorporates the initial candidate set with model outputs for eliminating label noise, which expands the size of the CLS.
7 CONCLUSION
In this work, we have presented a novel method to solve the NPLL problem. Specifically, we theoretically prove that smaller noise rate and shorter length of the average candidate label set will reduce the generalization error bound constructed under the NPLL paradigm. To optimize these two factors, we propose a sample separation strategy to segregate highly reliable normal samples, highly reliable noisy samples, and uncertain samples. Based on the sample separation, we perform distinct partial label set reconstructions for these three kinds of samples to reduce the average size of the candidate label set. Our model can act as plug-in to promote different PLL methods in NPLL, and extensive experiments suggest the salient performance advantage of our method. Furthermore, the ablation study indicates that all the components contribute to the proposed method.
10

---

Published as a conference paper at ICLR 2025
8 ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China under Grants U24A20322, 62225602, and 62176160, in part by the Guangdong Basic and Applied Basic Re- search Foundation (Grant 2024 2024B1515002109) and by the Big Data Computing Center of Southeast University.
REFERENCES
Shoshana Abramovich, Graham Jameson, and Gord Sinnamon. Refining jensen's 's inequality. Bulletin mathématique de la Société des Sciences Mathématiques de Roumanie, pp. 3-14, 2004.
Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities Risk bounds and structural results. Journal of Machine Learning Research. 3:463-482, 2002.
Ching-Hui Chen, Vishal M. Patel, and Rama Chellappa. Learning from ambiguously labeled face images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(7):1653-1667, 2018.
Timothée Cour, Benjamin Sapp, Chris Jordan, and Benjamin Taskar. Learning from ambiguously labeled images. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2009, Miami, Florida, USA, June 20-25, 2009, pp. 919-926, 2009.
Timothée Cour, Benjamin Sapp, and Ben Taskar. Learning from partial labels. Journal of Machine Learning Research, 12: 12:1501-1366 201
Lei Feng, Jiaqi Lv, Bo Han, Miao Xu, Gang Niu, Xin Geng, Bo An, and Masashi Sugiyama. Provably consistent partial-label learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, virtual, December 6-12, 2020, pp. (0948-1096)) 2020.
Shuo He, Chaojie Wang, Guowu Yang, and Lei Feng. Candidate label set pruning: A data-centric perspective for deep partial-label learning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024.
Mokter Hossain and Ilkka Kauranen. Crowdsourcing: a comprehensive literature review. Strategic Outsourcing: An International Journal, 3(1):2-22, 2015.
Eyke Hüllermeier and Jürgen Beringer. Learning from ambiguously labeled examples. Intelligent Data Analysis, 10(5):419 439, 2006.
Yuheng Jia, Chongjie Si, and Min-Ling Zhang. Complementary classifier induced partial label learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023, Long Beach, CA, USA, August 6-10, 2023, pp. 974-983, 2023a.
Yuheng Jia, Fuchao Yang, and Yongqiang Dong. Partial label learning with dissimilarity propagation guided candidate label shrinkage. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, volume 36, pp. 34190-34200, 2023b.
Yuheng Jia, Xiaorui Peng, Ran Wang, and Min-Ling Zhang. Long-tailed partial label learning by head classifier and tail classifier cooperation. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Vancouver, Canada, February 20-27, 2024, pp. 12857-12865,, 2024.
Jiahao Jiang, Yuheng Jia, Hui Liu, and Junhui Hou. Fairmatch: Promoting partial label learning by unlabeled samples. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024, pp. 1269 1278, 2024.
Rong Jin and Zoubin Ghahramani. Learning with multiple labels. Advances in Neural Information Processing Systems 15: Annual Conference Neural Information Processing Systems 2002 Neural Information Processing Systems, NIPS 2002, Vancouver, British Columbia, Canada, December 9-14, 2002, pp. 897-904, 2002.
11

---

Published as a conference paper at ICLR 2025
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535 547, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Silvio Savarese, and Steven C. H. Hoi. LAVIS: one-stop library for language-vision intelligence. In Proceedings of the 6l Anmual Meeting of the Association for Computational Linguistics: System Demonstrations, ACI. 2023, Toronto, Canada, July 10-12, 2023, pp. 31-41, 2023a.
Junnan Li. Richard Socher, and Steven C. H. Hoi. Dividemix: Learning with noisy labels as semi- supervised learning. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning. ICML 2023, Honolulu, Hawaii, USA, July 23-29 2023, volume 202, pp. 19730-19742, 2023b.
Jie Luo and Francesco Orabona. Learning from candidate labeling sets, In John D. Lafferty, Christopher K. I. Williams, John Shawe-Taylor, Richard S. Zemel, and Aron Culotta (eds.), Advances Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NeurlPS 2010, Vancouver, British Columbia, Canada, December 6-9, 2010, pp. 1504-1512, 2010.
Jiaqi Lv, Miao Xu, Lei Feng, Gang Niu, Xin Geng, and Masashi Sugiyama. Progressive identification of true labels for partial-label learning. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119, pp. 6500 66006510,,50 2020.
Jiaqi Lv, Biao Liu, Lei Feng, Ning Xu, Miao Xu, Bo An, Gang Niu, Xin Geng, and Masashi Sugiyama. On the robustness of average losses for partial-label learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2569 2588,,,,6---83, 2024.
Andreas Maurer. A vector-contraction inequality for rademacher complexities. In Ronald Ortner, Hans Ulrich Simon, and Sandra Zilles (eds.), Algorithmic Learning Theory 27th International Conference, ALT 2016, Bari, Italy. October 19-21, 2016, volume 9925, pp. 3-17, 2016.
Colin McDiarmid. On the method of bounded differences. Surveys in combinatories, 141.1:148-188 1989.
Nam Nguyen and Rich Caruana. Classification with partial labels. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Nevada, USA, August 24-27, 2008, pp. 551-559, 2008
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Sixth Indian Conference Computer Vision, Graphics & Image Processing, ICVGIP 2008, Bhubaneswar, India, 16-19 December 2008, pp- 722-729, 2008
Gabriel Panis and Andreas Lanitis. An overview of research activities in facial age estimation using the FG-NET aging database. In Computer Vision ECCV 2014 Workshops Zurich, Switzerland, September 6-7 and 12, 2014, Proceedings. Part 11, volume 8926, pp. 737-750, 2014.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner. Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, Vancouver, BC, Canada, December 141 2019, pp. 8024-8035, 2019.
Congyu Qiao, Ning Xu, Jiaqi Lv, Yi Ren, and Xin Geng. FREDIS: A fusion framework of refinement and disambiguation for unreliable partial label learning. In International Conference on Machine Learning. ICML 2023, Honolulu, Hawaii, USA, July 23-29, 2023, volume 202. pp. 28321-28336, 2023.
12

---

Published as a conference paper at ICLR 2025
Lars Schmarje, Vasco Grossmann, Claudius Zelenka, Sabine Dippel, Rainer Kiko, Mariusz Oszust, Matti Pastell, Jenny Stracke, Anna Valros, Nina Volkmann, and Reinhard Koch. Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimation. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Informa- fion Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, pp. 33215-332322 2022.
Yu Shi, Ning Xu, Hua Yuan, and Xin Geng. Unreliable partial label learning with recursive separation. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, Macao, SAR, China, August 19-25, 2023, pp.
2023.
Deng-Bac Bao Wang, Min-Ling Zhang, and Li Li. Adaptive graph guided disambiguation for partial label learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44((2):8796-88 8811. I, 2022a.
Haobo Wang, Ruixuan Xiao, Yixuan Li, Lei Feng, Gang Niu, Gang Chen, and Junbo Zhao. Pico: Contrastive label disambiguation for partial label learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022b.
Haobo Wang. Ruixuan Xiao, Yixuan Li, Lei Feng, Gang Niu, Gang Chen, and Junbo Zhao. Picot: Contrastive label disambiguation for robust partial label learning. IEEE Transactions on Pattern Analysis and Machine Intelligene, 46(5):3183 183-3318. 2024.
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pictro Perona. Caltech-ucsd birds 200. 2010.
Hongwei Wen, Jingyi Cui, Hanyuan Hang, Jiabin Liu, Yisen Wang, and Zhouchen Lin. Leveraged weighted loss for partial label learning. In Proceedings of the 38th International Conference Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139, pp. 11091-11100, 2021.
Dong-Dong Wu, Deng-Bao Bao Wang, and Min-Ling Zhang. Revisiting consistency regularization for deep partial label learning. In International Conference on Machine Learning, ICML 2022, Baltimore, Maryland, USA, July 17-23, 2022, volume 162, pp. 24212-24225, 2022.
Shiyu Xia, Jiaqi Lv. Ning Xu, Gang Niu, and Xin Geng. Towards effective visual representations for partial-label learning. In Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pp. 15589-15598, 2023.
Ming-Kun Xie, Jiahao Xiao, Hao-Zhe Liu, Gang Niu, Masashi Sugiyama, and Sheng-Jun Huang. Slass-distibuttionawwaa wware pseudo-labeling for semi-supervised multi-label learning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16. 2023, pp. 25731-25747, 2023.
Mingyu Xu, Zheng Lian, Lei Feng, Bin Liu, and Jianhua Tao. ALIM: adjusting label importance mechanism for noisy partial label learning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, pp. 18668-38684, 2023.
Ning Xu, Jiaqi Lv, and Xin Geng. Partial label learning via label enhancement. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, Honolulu, Hawaii, USA, January 27 February 1, 2019, pp. 5557-5564, 2019.
Fuchao Yang, Yuheng Jia, Hui Liu, Yongqiang Dong, and Junhui Hou. Noisy label removal for partial multi-label learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024, pp. 3724-3735, 2024.
Fei Zhang, Lei Feng, Bo Han, Tongliang Liu, Gang Niu, Tao Qin, and Masashi Sugiyama. Exploiting class activation value for partial-label learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
13

---

Published as a conference paper at ICLR 2025
Min-Ling Zhang. Bin-Bin Zhou, and Xu- Xu-Ying Liu. Partial label learning via feature-aware disam- biguation. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1335-1344, 2016.
Zhi-Hua Zhou. A brief introduction to weakly supervised learning. National Science Review, pp. 44-53, Jan 2018.
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 PROOF OF THEOREM 1
Theorem. Assume the loss function (x)( y) is p-Lipschitz with respect to f(x) for all y and uppper-bounded by M. For noise rate 0 < E < 1 and mean CLS size for normal samples 1 < < C, for any ó > with probability at least 1
have
R(f) - R(f) S 2(1 - M + 4V 2p Σ R. (Hy) + 2M
log
2n
y=1
Proof. Before proving the theorem, we first provide two useful lemmas as follows.
We primarily derive the uniform deviation bound between R(f) and R()), which is simple extension of result in muti-class setting (Xie al., 2023).
Lemma 1. Suppose that the loss function ((f(x),)) y) is p-Lipschitz with respect to f(x) for all y E y and
by M. For any O, with probability least 6, we have
log 2
R(f) -((F) S 2p
Rn (Hy) + 1 2n
(12)
y
where the function space Hy for the label y is {h: x (x))f E F}.
Proof. We first define the Rademacher complexity of L and JF with training instances as follows: (. FF)
En,yo
sup (C(((x))) fer im1
S <2p
Hr (H,y)
(13)
We apply the Rademacher vector contraction inequality (Maurer, 2016) in the second line.
Then, we proceed the proof by showing that the one direction supjer R(f) R(f) is bounded with probability at least 1 - 5/2, and the other direction can be proved similarly. Note that replacing an example (z,, y;) leads to change of sup
fejr R(f)) R(f) at most M due to the L is bounded by M. According to McDiarmid's 's inequality (McDiarmid, 1989), for any > O, with probability at least 1 5/2, we have
sup R()) R(()) < sup R() - (R(S) + +M 2n
log
E
(14)
Jef
fer
According the result in (Maurer, 2016) that shows E(sup R()) P((S)) S 29, (F), and further FEF
taking the other direction sup R(f)) R(f) into account. With probability at least 8, we have
JEF
log 2
R(f) - R(() 22V2p Σ Ri (H,y + M y=
(15)
2n
which completes the proof.
Lemma 2. Assume the loss function ((f(x), yy) is p p-Lipschitz with respect to f(x) for all y and
uppper-bounded by M. For noise rate e < and mean CLS size for normal samples < a C, we have
R'(F) - R(F) s 1-9MM.
(16)
15

---

Published as a conference paper at ICLR 2025
Proof. Let's first expand Rf))):
R(( - Σ [(f(x)),))
Y 1 [()
Iv EY)
Σ
C (((()))))))
(17)
|
[1/
i=1
nEVincaon in
+
I(y Yi)
Σ LEEY.
1 Y
[(((m))) (((())))))(((()))))))))))))))
1=1
Define m -i Iy E Yi), which implies the noise rate Define the function
|Y|-1 which is concave. By Jensen's 's inequality (Abramovich et al., 2004) and the f((1) = |Y
definition of a (a= = 1 دド= (gD we have
1 1E 1zgl =((a).
(18)
==1
Then, prove its lower bound:
('(f) 2 A(f)
Σ I( E Y;)
[(f(x;),)))
---1 Σ n
I( $ (.) yi)
71
Xi
2 R( Σ 1=1
le E Y;)
Yg -1 1
M
1
1( Y.)
|y|
2 A(f)
m
Y
1
M
71 m M
(19)
Ø=1
2 R(() (1 - 0
M
cM
2 R( - (1 111 M
where the second line holds because ((f(x))))) is upper-bounded by (C((()),)) S M) and the fourth line holds by Eq. (18).
Then, we prove its upper bound:
i(( S RO))
Mv EY)
Σ
1 ((((2)),
Σ
T(y $ Yi) Σ
(((())))
IV
Y
cEY,,ouy.
.EY,
< R(f) +
I(UI EY)
Σ
1
M
Σ
I $ Yi)
Yil M
|Y
cEYi.ogu
i=1
OEN,
m RR() Σ
Y1
1
1
II Y,)
M
+
Σ
T(u $ Y.JM
Yj1
n
< Rf()+ + m 1
Y -1
n m
M +
M
|Y]|
<R()) + (1 -66) a
M + EM
sA()) + (1 MM,
(20)
16

---

Published as a conference paper at ICLR 2025
where the second line holds because ((f()), y) is upper-bounded by M (C(f(z), y) S M) the fifth line holds by Eq. (18).
By combining these two sides, obtain the following result:
[ - R()) s (1
1 a
M,
(21)
which concludes the proof.
For any 6 > O, with probability at least 6, we have:
log oo
R(f) R(f) + 2/22p Σ
Rn (Pp)) + MV
2n
y=1
C
log
$ +
JM + +2v2p
Σ
Ri (Hb, + M
a
2n
y=1
log
- (1
Hn (Hyy + +MV JM + 2p Σ
(22)
2n
y=1
Hn (Hy, + M S (((*) + 1 )M + 2p Σ
log
2n
y-1
R. (Hly + 2M S R(F())+++ + + 1 )M + 4v 2p
log 2/
2n
y=1
where the first and fifth lines are based on Lemma 1, and second and fourth lines holds due to Lemma 2. The third line hold by the definition of f. At this point, we have proven the Theorem I.
A.2 SOLUTION OF EQUATION 6
min Σ [(u # 0) (1E) - 3(q.
- S(Y)))
1=1
s.t. Vi, 0 = 2 CY.
Vi, 12 = + 2, cy-
We denote Siy the j-th element of vector S(Y)). Then the original optimization objective can be writed as
C
Sij - 3 n
I(v + 0)
Σ
Ilus + 0) Σ Sij jml
j=1
Σ I(u # 0) Σ (1 BBss SSg
Considering Sig E (0, 1}, we can obtain that S.j if if > and Sij = 0 otherwise. The inclusion
constraint: the ground-truth label of the normal sample (w) = 1) is in the CLS (% # 2, 9, C Y,), while the ground-truth label of the noisy sample (e; 11) is in the non-CLS (Y, + 2, Y, cy Y2). Therefore we can get the solution that
P = Gil E Y, and qij > 3h if = 1,
p Gi E y Y; and 4i
Bl
1
if ef 11
17

---

Published as a conference paper at ICLR 2025
A.3 IMPLEMENTATION DETAILS
There are four nyper-parameters in our method: Ear, Send. X and B. Since our approach is plug-in method, the number of warm-up rounds ew related the training rounds of the original method. For the CIFAR-10 dataset, we used 0.25 times the number of training rounds of the original method, and used 0.1 times for Treeversity and Benthic ,while for CIFAR- and other fine- aree-graned datasets, we used 0.4 times. The uncentainty-en epoch eend equals 0.6 times the number of training rounds of original method. We selected from {0.2, 0.3, 0.4, 0.5) and 3 from {1.5, 2, 2.5, 3.0). We used the softmax function as the N Normalize(-). Following the standard experimental setup (Xu et al., 2023), we splited clean validation set from the training set to determine nyper-parameters. Then, we transformed the validation set back to its NPLL form and incorporated them into the training set to optimize the model. Following previous research (He et al., 2024), we employed BLIP-2 (Li et al., 2023b) the feature extractor based the open-souree library LAVIS (Li et al., 2023a), thereby securing a more reliable K -neihhbor relationship. For KNN searching, the number chosen neighhors was set to 5 for all experiments and employed Faiss (Johnson al., 2019), a library for efficient similarity search and clustering of dense vectors. For all the methods, we employed the same backbone. For the CIFAR dataset, we utilized ResNet18, while for the fine- gin-ggriiedd datasets CUB200 and Flower and the real-world datasets Treeversity and Benthic, we employed ResNet34 and loaded the pre-trained weights from ImageNet for the feature extractor enhance training efficiency. For all methods, the SGD was used as the optimizer with momentum of 0.9 and weight decay of 0.001. We set the initial learning rate to 0.01 and adjusted it using the cosine scheduler. Due to our approach is plug-in method, we default to combining our method with PaPi (Xia et al., 2023) for NPLL unless specifically stated. All experiments were implemented with Py Torchh (Paszke al., 2019) 19) and carried out with 6 NVIDIA RTX 3090 GPUs and 8 NVIDIA RTX 4090 GPUs.
A.4 THE FRAMEWORK OF OUR METHOD
Ground wuth label
1: Candidate label
0 Non-candidate
Highly
Ya
1
0 0
Y
0
Reconstruction
Y
0
0
Y,
1
0
0
surpless
0 0
8.
0
0
tampples
D
( YO)
D ((z1,
P:)
Figure 4: The framework of our method. We employed thal--hhrehold approach and u) to separate the samples into three parts, based on that we proposed CLS reconstruction for faithful shorter CLS for each instance.
A.5
OTHER RESULTS ON CIFAR100 100
Mcan Accuracy
75
Mean Accuracy
Mean Accuracy
20
AUN ALIN-Scale AUN Onehot
Original
23
AUM Onehct Orginal
Paf
CROPEL
MROGEN
0-02
P-0.3
y
6
0.0
0.0h
y 0.
(a) varying PLI methods
(b) varying noise level
(c) varying ambiguity level
Figure 5: The mean accuracy on CIFARIO0 of our method and the ALIM across different PLL method, ambiguity levels, and noise levels. The results are derived from the statistics presented in Table 2.
18

---

Published as a conference paper at ICLR 2025
(a) 200 epoch
(h) 240 epoch
(c) 280 epoch
(d) 500 epoch
Figure 6: Distribution of the ECK for real normal and noisy samples with increasing training iterations. The experiment conducted on CIFARIO0 (1) = 0.05, 7 = 0.3).
373
Numbo solected 53
Noma
Wol
humber
CLS
ACC. Norma Sa Sample
6CC. Natao
Nam
Eboch
(a) Separation quantity vs Separation quality (b) Ground-trath hit rate vs Mean CLS size
(c) The factor of generalization error
Figure 7: The experiment is conducted on CIFARIO0 (n 0.05, 7 0.3). (a) demonstrates that
with the progression of training iterations, the selection quantity of both normal and noisy samples increases, while the quality is able to be sustained a high level. (b) reflects that, as training iterations continue, the reconstructed CLS size consistently decreases while ensuring high ground-truth label hit rate. There are no uncertain samples after 200 epoch, i.c., Cend = 300. (c) indicates that, as the model training progresses, the generalization error bound is continuously decreasing, with - 1-c being an important factor derived from Section for the generalization error bound.
A.6 REAL-WORLD DATASETS
We further evaluate our method two the real-world crowdsourced datasets Treeversity and Benthic. In these datasets, each image annotated by multiple individuals. For our study, we randomly selected annotations to selected annotations to construct a dataset Treeversity3. Given the limited number of annotations per sample in the Benthic dataset, randomly selected I annotation for 60% of the samples and 2 annotations for the remaining samples, resulting in the Benthic2# dataset. Below is the detailed information about these datasets:
sample's CLS construct dataset Treeversity2, and randomly
Table 6: The detailed information of real-world datasets.
Real-world Datasets
Noise rate
Avg.
size
Sample num
Classes
Input size
Treeversity3
0.07
1.72
9826
224 224
Treeversity2
0.14
* 224
Benthic2il 0.12
1.12
112
We used the ResNet34 as the backbone for all methods and loaded the pre-trained weights from ImageNet enhance training efficiency. The experimental results the Table indicate that our method still significantly outperforms the current SOTA methods real-world datasets. Especially, more challenging scenarios like Treeversity2, the performance gap even more significant.
A.7 SEPARATION ACCURACY COMPARISONS TO CURRENT NPLL METHODS
The y is noise level and 7) is the ambiguity level Table 7. Our method outperforms other NPLL methods with substantial gap especially when noise level is high, which validates the superiority
19

---

Published as a conference paper at ICLR 2025
Table The separation accuracy of normal/noisy samples for each NPLL method.
Method Dataset
CIFARIO (:002 (=0.5)
C) y=0.5)
CIFARIO (5=0.4 /=0.5)
PICO+
99.2 9:48, 90%
96.25%/66 14%
83.92%/76. 1
98.53%%/94.44%
97.315%/93.36%
86.80% 92.68% %6888058
Ours
99.35%
98.91 09.91%/99..88 %
of ECK on sample separation and demonstrates that our method is more robust against label noise compared to other NPLL methods.
A.8 LIMITATION
While our method has achieved superior performances in handling NPLL tasks, it still has short- coming. Due to our method being applied during the training process given PLL model, necessitates additional computations for sample separation and CLS reconstruction which incurs extra computational and space costs. However, correspondingly, as our method gradually reduces the length of the CLS during reconstruction, it can accelerate the disambiguation process of the PLL methods, thereby decreasing the number of training iterations. This can mitigate the issue of increased training time per iteration certain extent.
A.9 PARAMETER SENSITIVITY ANALYSIS
In this section, perform parameter sensitivity analysis on four hyper-parameters: the warm-up epoch (ew), the rate of percentile change between and T., (A), the trade-off of two objectives on CLS Reconstruction (8), and the KNN parameter (K). Figure 8 presents the results of the hyper- parameters sensitivity study conducted on various
on CIFAR10 (n 0.5, y 0.3) and CIFAR 100 (1) 0.05, 7 0.3). To investigate the influence of noise on ayper-parameter sensitivity, we further conducted sensitivity analyses at higher noise level on CIFAR 10 (1 0.05, 7 = 0.4) on Figure 9. The results show that 33 has stable performance across a range of values, with CIFAR10 10 (7) 0.5, 7 0.3) achieving the best accuracy (95. 195.46%. at 3 = 1.5, and CIFAR 100 (n) = 0.05, 7 = 0.3) peaking at 80.39% when 8 = 2.0. The KNN parameter K shows stable results with CIFARIO (17 0.5, 7 0.3) achieving 95.55% at K 7 and CIFARI00 (17 0.05, 7 0.3) reaching 80.1 at K = 5. The comparison between Figures 8 and reveals that the hyperparameters and K demonstrate robustness against noise. The model achieves optimal performance when is set between 1.5 and 2 and K is set between 5 and 7 across different noise levels. For X, CIFARIO (1) 0.5, 7 0.3) and CIFAR 100 (1 = 0.05, 7 0.3) reach their highest accuracy 13% and 80.46%) at À = 0.4, with performance declining at lower or higher values. Its selection is influenced by the noise rate in the dataset, as controls the percentile change rate between normal and noisy samples during the separation process. Therefore. we recommend inereasing À under high noise conditions and reducing it under low noise scenarios. The warm-up epoch e, crucial for training stability. The insufficient pretraining with small ew may lead to poor discrimination, hindering the generation of reliable pseudo-labels. Conversely, an excessively large e., can result in overfitting to noisy samples, reducing the model's ability to correct errors in later stages. Nonetheless, setting en within the range of 100 to 200 consistently yields satisfactory performance across different datasets and noise levels. For instance, achieved at e,, = 150, while CIFARI00 (n = 0.05, ? = 0.3) peaks at 19% with e,, = 150. Overall, while and K robust, tuning of and exp is necessary.
CIFAR10 (n = 0.5,
0.3), the best accuracy (96.23%) is
20

---

Published as a conference paper at ICLR 2025
120
156
24
(a) Ew on CI
(b) Ew on CIFAR100
(c) on CIFARIO
(d) A on CIFARI00
955
550
15
1.5
(c) 3 on CIFARIO
(f) B on CIFAR100
(g) K on CIFAR1O
(h) K on CIFARIO0
Figure 8: Parameter sensitivity analysis on CIFARIO (1) 0.5, 7 0.3) and CIFAR100 (n 0.05,
0.3).
The Acuuney
(a) Cur on CIFAR100
(b) À on CIFAR100
(c) 8 on CIFARI00
(d) K CIFARI00
Figure 9: Parameter sensitivity analysis on CIFAR100 (1) = 0.05, = 0.4).
21


Reviews:
[{"review_id": "qWzWFgR1Zy", "reviewer": "Reviewer_xLDw", "summary": "This paper studies the noisy partial label learning (NPLL) problem, where the correct label is not always in the candidate label set (CLS). This setting is more practice in real world. To address this problem, firstly, the authors theoretically prove the generalization error bound of the classifier constructed under NPLL paradigm and reveal that the error bound depends on the noise rate and the average length of the candidate label set. Secondly, the authors propose a method to reduce the noise rate and reconstruct shorter candidate label sets. Finally, experiments on benchmark datasets are conducted to confirm the efficacy of the proposed method in addressing NPLL.", "questions": "1.\tAccording to Eq.(2), do you need to know the CLS of the normal samples in prior?\n2.\tSince the computation of Eq.(3) relies on the KNN-based pseudo-label, how to decide the parameter of K when applying KNN?\n3.\tIn lines 192-195, the authors claim that “when the ground-truth label of a sample locates at the non-candidate labels, there will be a significant discrepancy between the CLS-based pseudo-label and KNN-based pseudo-label, leading to a larger value of ECK”, could you provide some experimental results to support this?\n4.\tIn lines 272-274, the authors claim that “add the label with the highest pseudo-label probability from the non-CLS to the CLS, and remove the label with the lowest pseudo-label probability from the CLS”, this operation may destroy the diversity of CLS. Is it beneficial to the performance? And please explain Eq.(10).\n5.\tPlease give the detailed derivation of second inequality of Eq.(18).\n6.\tIn lines 648-654, there are two repeated references.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThis paper provides a theoretical generalization analysis of the NPLL, highlighting the noise rate and the length of the candidate label sets are two key factors, which is insightful.\n2.\tThe proposed method is grounded in theoretical findings, making it reasonable and intuitive.\n3.\tExtensive experiments have been conducted and the reported results verify the effectiveness of the proposed method.", "weaknesses": "1.\tSince the whole label space is very large, then the size of non-CLS may also be very large, which will bring challenges to find the ground truth label.\n2.\tSome parts of this paper is unclear, see the questions below for details.\n3.\tSensitivity analyses of the hyperparameters are absent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d4K17h70gu", "reviewer": "Reviewer_B6bW", "summary": "This paper addresses the problem of noisy partial label learning (NPLL), where the true label may not be included in the candidate label set (CLS). The authors provide a generalization bound on the empirical risk and, based on this, propose an NPLL method that progressively separates samples and reconstructs the CLS, reducing the noise rate while shrinking the CLS size. Experiments are conducted on both synthetic and real-world noisy datasets.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Building on the generalization error bound, the authors propose a method of progressive sample separation and CLS reconstruction, which reduces the noise rate while simultaneously shrinking the CLS size.\n2. The performance of the proposed method is promising in the experiments.\n3. The writing is very clear and easy to understand.", "weaknesses": "**Weaknesses and questions**\n\n1. Theoretical part.\n   * \"$f_y(x)$\" (Line 125) is not defined in the paper. I assume this represents the predicted probability of the $y$-th class for $x$, following the notation in Xie et al. (2023), as cited by the authors in Appendix 1?\n   * The authors state, \"Theorem 1 shows that the empirical risk minimizer $\\hat{f}$ converges to the true risk minimizer $f^*$ as $n \\rightarrow \\infty$, $\\epsilon \\rightarrow 0$, and $\\alpha \\rightarrow 1$\" (Line 138). However, even under these conditions, the Rademacher complexity will not necessarily converge to zero, which implies that $R(\\hat{f})$ does not necessarily converge to $R(f^)$. Furthermore, even if $R(\\hat{f})$ does converge to $R(f^*)$, this does not guarantee that the minimizer $\\hat{f}$ itself converges to $f^*$.\n2. Proposed Method and Experimental Part\n   * Could you clarify the rationale behind the statement, \"When the ground-truth label of a sample is included in the candidate labels, the CLS-based pseudo-labels and KNN-based pseudo-labels tend to be similar... however, when the ground-truth label of a sample is in the non-candidate labels, there will be a significant discrepancy between the CLS-based pseudo-label and KNN-based pseudo-label\" (Lines 190-195)?\n   * In constructing the KNN-based pseudo-label and CLS-based pseudo-label for $x_i$, model probability prediction vectors and features are used. How does the warm-up stage affect this process? What if the warm up stage that relies on noisy labels is not good enough to produce meaningful features, especially when the noise ratio is high?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iBYyQkEpNg", "reviewer": "Reviewer_1LBQ", "summary": "The authors provide the generalization error bound of the classifier constructed under NPLL, which depends on the noise rate and the mean size of candidate label sets for normal samples. Empirically, they propose a novel NPLL method which includes two components: progressive sample separation and CLS reconstruction. Their method can serve as a plug-in for existing PLL methods to enhance their performance on NPLL datasets.", "questions": "1. Can we use the prediction from the network $p_j$ instead of the KNN-based $q_j$ in Equ. (6)? Are there any difference in performance? In other words, can you empirically show that the KNN-based pseudo-label is better than probability prediction vector from the network?\n2. Can you compare the performance with (Lv et al., 2024)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. From generalization error bounds in NPLL problem, they find that the lower noise rate and smaller candidate label are two key\nfactors to solve the NPLL problem.\n\n2. By iterative learning, their method can effectively reduce the noise rate while simultaneously decreasing the size of CLS.\n\n3. Extensive experimental results validate that our method outperforms the current state-of-the-art (SOTA) methods by a large margin", "weaknesses": "The authors did not compare with the recent work (Lv et al., 2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kEaGARH2JO", "reviewer": "Reviewer_Xu7T", "summary": "The paper addresses the problem of Noisy Partial Label Learning (NPLL), a challenging subset of weakly supervised learning where instances are annotated with a set of candidate labels, only one of which is correct. The authors theoretically demonstrate that the generalization error of a classifier constructed under the NPLL paradigm is constrained by the noise rate and the average length of the candidate label set. Building on this theoretical foundation, they propose a novel NPLL framework designed to distinguish noisy samples from normal ones, thereby reducing the noise rate and reconstructing shorter candidate label sets for both categories.", "questions": "Could the authors provide a more detailed explanation or theoretical justification for the assumption that reliable samples become increasingly reliable, while unreliable samples become increasingly unreliable as training progresses?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper presents a groundbreaking contribution to the field of weakly supervised learning, specifically addressing the Noisy Partial Label Learning (NPLL) problem. The authors’ novel approach to bounding the generalization error of classifiers under NPLL is a significant theoretical advancement. The proposed method, which integrates progressive sample separation and CLS reconstruction, demonstrates a clear and innovative approach to solving NPLL. This originality in problem formulation and theoretical analysis is commendable.\n2. The article's structure is well-organized and easy to comprehend.\n3. The experiments are generally comprehensive.\n4. Theoretical analysis contributes to establishing the effectiveness of the proposed model.", "weaknesses": "1.The computational efficiency of the proposed model is a concern.  The requirement to train the classifier on the entire dataset at every epoch introduces a high time complexity, which may limit the practical applicability of the method, especially for large datasets.\n2.The computational efficiency of the proposed model is a concern.  The requirement to train the classifier on the entire dataset at every epoch introduces a high time complexity, which may limit the practical applicability of the method, especially for large datasets.\n3.When there are many noisy samples, for the calculation of KNN, if the neighboring samples are also noisy, is this method reliable?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jqYrTELUyY", "reviewer": "Reviewer_RE2k", "summary": "The paper introduces a new framework for noisy partial label learning (NPLL) that combines progressive sample separation and candidate label reconstruction to reduce noise rates and improve classification accuracy, achieving a good  performance enhancement on the CIFAR10 dataset compared to state-of-the-art methods.", "questions": "Please refer to the weakness", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is clearly written and easy to understand.\n\n2. The paper provides the generalization error bound for the classifier constructed under NPLL.\n\n3. Extensive experimental results demonstrate that our method significantly outperforms the current state-of-the-art (SOTA) methods.", "weaknesses": "1. For ablation study,  $v_2$ seems ineffective in separating normal samples and noisy samples in CIFAR10.\n\n2. For Eq.(3), the author calculates the ECK of each sample $x_i$ by CE and uses $\\widetilde q$ as target. What about using a KNN-based pseudo-label as the target instead?\n\n3. The sentence below Eq.(10), $c^\\prime$ can be  $c^\\prime = \\mathrm{arg min}_{j \\in Y_i}$ instead of   arg max\n\n4. The datasets adopt in the experiment are few and small. It would be more effective to utilize larger datasets to further validate the effectiveness of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)