Decompositional Generation Process for Instance-Dependent Partial Label Learning
Authors: Congyu Qiao
Ning Xu
Xin Geng
Venue: ICLR 2023 notable top 25% (2023)
Keywords: partial label learning
weakly supervised learning
decompositional generation process

Abstract: Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true. Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels and model the generation process of the candidate labels in a simple way.  However, these approaches usually do not perform as well as expected due to the fact that the generation process of the candidate labels is always instance-dependent. Therefore, it deserves to be modeled in a refined way.  In this paper, we consider instance-dependent PLL and assume that the generation process of the candidate labels could decompose into two sequential parts, where the correct label emerges first in the mind of the annotator but then the incorrect labels related to the feature are also selected with the correct label as candidate labels due to uncertainty of labeling. Motivated by this consideration, we propose a novel PLL method that performs Maximum A Posterior(MAP) based on an explicitly modeled generation process of candidate labels via decomposed probability distribution models. Extensive experiments on manually corrupted benchmark datasets and real-world datasets validate the effectiveness of the proposed method.

Published as a conference paper at ICLR 2023
DECOMPOSITIONAL GENERATION PROCESS FOR INSTANCE- DEPENDENT DNSTANCE-DEPENDEEN PARTIAL LABEL LEARNING
Congyu Qiao, Ning Xu', Xin Geng" School of Computer Science and Engineering, Southeast University, Nanjing 210096, China (qiaocy, xning, xgeng) @euu edu.cn
ABSTRACT
Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true. Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels and model the generation process of the candidate labels in simple way. However, these approaches usually do not perform as well as expected due to the fact that the generation process of the candidate labels always instance-dependent. Therefore, it deserves to be modeled in a refined way. In this paper, we consider instance- dependent PLL and assume that the generation process of the candidate labels could decompose into two sequential parts, where the correct label emerges first in the mind of the annotator but then the incorrect labels related to the feature are also selected with the correct label candidate labels due to uncertainty of labeling. Motivated by this consideration, we propose a novel PLL method that performs Maximum A Posterior (MAP) based candidate labels via decomposed probability distribution models. Extensive experiments manually corrupted benchmark datasets and real-world datasets validate the effectiveness of the proposed method. Source code available at Ittpp::/ /tttp:://gttub.. cww/pa lmmmlmidgg
explicitly modeled generation process
INTRODUCTION
Partial label learning (PLL) aims deal with the problem where each instance is provided with a set of candidate labels, only one of which is the correct label. The problem of learning from partial label examples naturally arises in a number of real-world scenarios such as web data mining
Luo & Orabona (2010), multimedia content analysis Zeng et al. (2013); Chen et al. (2017). and
ecoinformatics/ Liu & Dietterich (2012); Tang & Zhang (0017).
A number of methods have been proposed to improve the practical performance of PLL. Identification-
based PLL approaches JJin & Ghahramani (2002): Nguyen & Caruana (2008): Liu & Dietterich (2012):
Chen et al. y; Yu & Zhang (2016) regard the correct label latent variable and try identify
it. Average- Average-bused approaches Hillermeeir & Beringer (2006); Cour et al. (201T); Zhang & Yul (2015)
treat all the candidate labels equally and average the modeling outputs the prediction. In addition. risk-consistent methods Feng et al. (2020): Wen et al. (202)) and classifier-consistent methods Lv et al. (2020): Feng al. (2020) are proposed for deep models. Furthermore, aimed at deep models, Wang et al. (2022) investigate contrastive representation learning, Zhang et al. (2021) adapt the class activation map, and Wu et al. (2022) revisit consistency regularization in PLL.
It is challenging to avoid overfitting on candidate labels, especially when the candidate labels depend on instances. Therefore, the previous methods assume that the candidate labels are instance- independent. Unfortunately, this often tends to be the case that the incorrect labels related to the feature are more likely to be picked as candidate label set for each instance. Recent work Xu et al. (2021) has also shown that the presence of instance-dependent PL. imposes additional challenges but more realistic in practice than the instance-independent case.
"Corresponding authors.

---

Published as a conference paper at ICLR 2023
In this paper, focus on the iustance-dependeen PLL via considering the essential generating process candidate labels in PLL. To begin with, let us rethink meticulously how candidate labels arise in most manual annotation scenarios. When one annotates an instance, though the correct label has already emerged in the mind of the annotator first, the incorrect labels which are related to the feature of the instance confuse the annotator, then leading the result that the correct label and some incorrect labels are packed together as the candidate labels. Therefore, the generating process of the candidate labels in instance-dependent PLL could be decomposed into stages, i.e., the generation of the correct label of the instance and the generation of the incorrect labels related to the instance, which could be described by Categorical distribution and Bernuuli distribution, respectively.
Motivated by the above consideration, we propose a novel PLL method named IDGP, i.e., Instance- dependent partial label learning via Decompositional Generation Process. Before performing IDGP, the distributions of the correct label and the incorrect label given the training example should be modeled explicitly by decoupled probability distributions Categorical distribution and Bernoulli distribution. Then we perform Maximum A Posterior (MAP) estimation on the PLL training dataset to deduce a risk minimizer. To optimize the risk minimizer, Dirichlet distribution and Beta distri- bution are leveraged to model the condition prior inside and estimate the parameters of Categorical distribution and Bernouli distribution due to the conjugacy. Finally, we refine prior information by updating the parameters of the corresponding conjugate distributions iteratively improve the performance of the predictive model in each epoch. Our contributions can be summarized as follows:
We for the first time explicitly model the generation process of candidate labels in instance- dependent PLL. The entire generating process is decomposed into the generation of the correct label of the instance and the generation of the incorrect labels, which could be described by Categorical distribution and Bernoulii distribution, respectively.
We optimize the models of Categorical distribution and Bernoulli distribution via the MAP technique, where the corresponding conjugate distributions, i.e., Dirichlet distribution and Beta distribution are induced.
We derive an estimation error bound of our approach, which demonstrates that the empirical risk minimizer would upproximately converge to the optimal risk minimizer as the number of training data grows infinity.
2 RELATED WORK
In this section, we briefly review the literature for PL.. from two aspects, i.e., traditional PLI and deep PI.I. The former absorbs many classical machine learning techniques and usually utilizes linear models while the latter embraces deep learning and builds upon deep neural networks. We focus on the underlying assumptions on the generation of candidate labels behind part them.
Traditional PLL usually uses average-based or iientification-based approaches to disambiguate the candidate label set. Average-based approaches treat each candidate label of the instance equally Hüllermeier & Beringer (2006); Cour et al. (201T); Zhang & Yu (2015). Typically, Hüllermeier & Beringer[(0066)) Zhang & Yu (2015) apply the K-nearest neighbor technique to predict a new instance constantly trying to identify the possible correct label, either explicitly or implicitly during the training phase, in order to reduce label ambiguity Jin &
through voting. Identification-bant based approaches
Ghahramani Nguyen & Caruana (2008); : Liu & Dietterich (2012); Chen al. (2014); Yu &
Zhang (016)). For instance, Nguyen & Caruana 2008); : Yu & Zhang (2016) formulate their objective Tunctions by treating the correct label as latent variable via the maximum margin criterion. Zhang
et al.] 2016); & An (2018); Wang et al. 2019); Xu et al.] (2019b) use topological information in the feature space to iteratively update the confidence of each candidate label or the label distribution.
It should be noted that most traditional PL. methods ignore the process of generating candidate labels. The Logistic Stick-Breaking Conditional Multinomial Model is proposed by Liu & Dietterich (2012) to depict the generation process, but the candidate labels are assumed to be instance- inn.uncceiddeeeddeet...
Deep PLL has recently been studied and advanced the practical application of PLL, where the PLL approaches are not restricted to linear models and low-efficiency optimization. Yao et al. (200aa) pioneer the use of deep convolutional neural networks and employ a regularization term of uncertainty and temporal- -ensprlllnnssmmliigg term to train the deep model. [v et al. (2020) propose a progressive identification method that allows PLL to be compatible with arbitrary models and optimizers while also performing impressively on image classification benchmarks for the first.
2

---

Published as a conference paper at ICLR 2023
time. Yao et al. 2020b) introduce network-cooperraion cooperation mechanism in deep PLL which trains two networks simul tancously to reduce their respective disambiguation errors. |Feng et al. (2020) employ the importance reweighting strategy to derive risk-consistent estimator and the transition matrix for PLL to derive a classifier-consisentt estimator; Wen et al. (2021) introduce a leverage parameter partial labels and non-partial labels. Xu et al. (202T) apply variational label enhancement Xu et al. (2020) (2119)) to iteratively recover label distribution for each instance. Lv a.. (022)) for the first time propose unreliable PLL, a realistic and challenge setting. ZZhang et al.
that weights losses
(2021) discover class activation map could be used for disambiguation, and class activation value
could be used to capture the learned representation information a more general way. Lyy al.
(2022) transform the PLL problem into an "instance-label" matching selection problem. He et al.
(2022) utilize semantic label representations to enhance the label disambiguation, [Wang et al. (2022)
create contrastive learning framework for PL. disambiguation. To preserve the manifold structure in both feature and label space, Wu et al. (2022) employ manifold consistency regularization.
From Feng et al. (2020), the generation process of candidate labels has been paid attention to in Deep PLL, making a naive assumption that the candidate label set is uniformly sampled for each instance.
Wen et al. 2021) extend the uniform case the class-dependent one but remain instance- instance-indepente independent.
Xu et al. (2021) for the first time consider PLL under the instance- deettnnce-deeddeent case, a more realistic but challenging setting. However, Xu al. (202T) do not explicitly model the generation process of instance-dependent candidate labels. The generative model in it is employed only to generate label distribution. In this paper, we not only consider the instance-dependent PLL but also explicitly model the generation process of instance-dependent candidate labels via decoupled probability distributions.
3 PROPOSED METHOD
First of all, we introduce some necessary notations for our approach. In PLL. labeling information of every instance in the training dataset is corrupted from a correct label into candidate label set, which contains the correct label and incorrect candidate labels but is not a complete label set. Let X' S R be the g-dimensional feature space of the instances and y {1, 2, ........... c} be the label space with e class labels. Then PLL training data set can be formulated as D
( S, [1 < i < n). where a; E X denotes the i-th y-dimensional feature vector and S; E C = 2 \ (0, ) is the candidate label set annotated for a; St consists of the correct label y/ E ) and the incorrect candidate label set S = =S,Y \ {v.). We aim to find multi-class classifier f X + y according to D.
3.1 OVERVIEW
To deal with instance- deepede- PLL., we introduce the explicit generation model of instance- dependent candidate labels, which decouples the distribution of the correct and incorrect candidate labels. Then Category distribution and Bernoulli distribution are leveraged to depict them respectively with Dirichlet distribution and Beta distribution as their prior distributions. Based on the probabilistic generation model, we simply deduce the log-likeldhood loss function for optimization and then move forward to the MAP optimization problem in consideration of the prior distributions Dirichlet and Beta. Finally, we further propose the algorithm IDGP, which keeps and refines the prior information.
3.2 GENERATION MODEL OF CANDIDATE LABELS
Focusing on instance- dependent partial label learning, we propose a novel generation model of candidate labels, which explicitly models the generation process and decouples the distribution of the candidate labels into the distribution of the correct label and incorrect candidate labels respectively. To form candidate label set, we suppose the correct label is first selected according distribution dependent on the instance. And then the incorrect candidate labels related to the feature of instance emerge to disturb the annotator and are sampled from another posterior distribution dependent on the instance. The concrete generation model is demonstrated as follows.
posterior
Given an instance x;, its candidate label set S,, consisting of the correct label yi and the incorrect candidate labels 3. is drawn from a probability distribution with the following density:
p(S. (x.)
Σ JES.
p(u: )|=:)p (x))
(1)
3

---

Published as a conference paper at ICLR 2023
Here, p(,, suggests generation model is entirely dependent the instance. Our generation model builds upon the PLL assumption that the correct label is always in the candidate set, which allows for directly splitting S; into Y; and S For the given candidate label set St each label j E S have the possibility p(y; j|æ;)) sampled by the annotator the correct label in the first stage of
generation, and then the incorrect candidate labels S; are sampled with the possibility p(S) (x:) in the second stage to form the entire candidate label set S,. The generation of the correct label and incorrect labels are conditioned on the instance In this way, we decompose the generation process of the instance-dependent candidate labels.
After the decomposition of generation, the corresponding probability distributions are required to depict p(ui|m:) and p(S," Jr;)) th )x;)) We assume that the correct label yi of each instance is drawn from Categorical distribution with the parameters O,, where 02 = [0), 02..0..... .....,, 09] E [0, 1] is
c-dimension vector with the constraint مj=1 0f = 1, i.e.,
pll,,,, 0,) = Cat((
II (0)), j=1
(2)
where the vector L; = 14,4 14., .... E 10 is utilized to represent whether the j-th label is the correct label Vi i.e.!? 1 if Ut. otherwise = O.
Besides, for the incorrect candidate label S. can also be denoted by logical label vector 3 = [ 3 ,.
3. E {0, 13 to represent whether the j-th label is the incorrect candidate label,
i..., 3/- = 1f S. उ 0 if j & 5." In order to describe p(5 (x;)) with probabilistic model, we further decouple the incorrect candidate labels by assuming that the distribution of each variable of is independent from each other, i.e, (( = IIJe3. p(> a::) 15 (1 - p(>{(=))) and
the incorrect candidate label set S is drawn from multivariate Bernoulli distribution with the parameters 2,, where 2; | 25 ...,,,,, 2f] E [0, 1] is c-dimension vector, i.e.,
123" 2. 21)
II Ber (3 = II (z 2 j=1
(3)
1=1
For the latter estimation of 0g in MAP, we introduce Dirichlet distribution, the conjugate distribution of the Categorical distribution, its conditional prior, i.e., p(0,(z,) = -Dir(0,(A.) where A, > 0) is c-dimensional vector the output of our main branch inference
T][T (N model / parameterized by θ for an instance X;, i.e, X = a exp (ax 9)/7) + b, where a, and used resolve the scale ambiguity. Note that the main branch inference model is leveraged as the final predictive model to accomplish the PLL target X' + y by using
[N,, X?, ,.....: T
(a 2 1, b 2 0, >
arg max, 0/. where 0: will be estimated by X; with the conjugacy later.
I Likewise, to estimate z; latter in MAP, we introduce Beta distribution, the conjugate distribution of Bernoulli distribution, the conditional prior parameterized by [a oz. ................................ og]] and 3
..... [5] (af > 3 > 0), i.e., p(2, x;; 11; Beta(2 (o2,,88 B.). We employ auxil- iary branch model parameterized by Ω to output AT = aa B. A, = a e-exp (g(x;; 2)/>/ 1//>>++b +b. The same model and constants to scale A; are implemented to simplify our approaches. It should be noted that the predictive model f(æ.; θ) or auxiliary model g( 12) can be any deep neural network once its output satisfies the corresponding constraint.
3.3 OPTIMIZATION USING MAXIMUM A POSTERIOR
Based on the generation model of candidate labels denoted by Bq.01 Eq. 2 and Fq.3), by using the technique of Maximum Likelihood (MI.) estimation we can immediately induce the log-likelihood loss function for the PI.I. training data set to be optimized:
LML Σ
log Dog 0, 21)
log
II
(1-
II
(4)
K
4

---

Published as a conference paper at ICLR 2023
The Eq.44) demonstrates how the distribution of the correct label interacts with that of the incorrect candidate labels after decoupling the generation. In the backpropagation the training process, they provide weight coefficients for each other.
In order to bring in prior information for PLL, we further introduce IDGP, which performs MAP in the training data set. In IDGP, we more concerned about maximizing the joint distribution p(0, z x, S). This will lead to the following optimization problem LMAP = LMIL + Lreg= where
Leg = log p x;)) + log (z2,(a))).
(5)
Compared ML, our IDGP framework provides a natural way of leveraging prior information via optimizing the extra condition prior illustrated by the (5. in the training process, which is significant for PL.L due to the implicit supervision information. Combined with the prior distribution Dirichlet and Beta, frgg can be analytically calculated as follows:
Lag ΣΣ log 6. + (ag - log of + (8) - :=1
-1) log (1 (1-2)).
(6)
The mathematical derivations of Eq. 4 and Eq. 66 provided in Appendix A.I By combining the (4. and the (6). the MAP optimization problem can be calculated as follows:
log
II
II kg
21)
jES,
KE
"
x 1) log 0¹ + (af log of + (8) lolo (1 2().
C can also accommodate the uniform case, which can be seen in Appendix A.2 conjugacy of Dirichlet and Categorical distribution Minka (2000) we can estimate 0 by using
Then, due to the
of X
0 1 Ao
(8)
Ei
X +
where of denotes the number of occurrences on the label for 2,, i.e., =f E S,, otherwise of O. Similarly, we can leverage the conjugacy of Beta and Bernoulli distribution to estimate af iie.
+.+
D ai 8.] a. + + +o
E
The mathematical derivations of Fq. and Eq. 9 are provided in Appendix A.3
As it is shown in the Eq. 6g. IDGP provides prior information that can be included in the parameters X, a and / for the predictive model fa: θ) and the auxiliary model gxx; S2) by transforming them into the weights exerted on the corresponding label. The prior information comes from the memorization effects Han et al. (2020) of the neural network. It makes the neural network always likely to recognize and remember the correct label in priority, leading to kind of initial disambiguation at the beginning epochs. Hence, to keep fine prior information, we replace X; with
A; in the following way:
+(1 - m) (1 j S jij
(10)
1 1+ e, otherwise,
where (-)(9 denotes the vector scalar (-) at the t epoch, r(s t) denotes the beginning epoch set to reserve the fine prior information for f(æ; O), m E (0, 1) 1) is a positive constant used to replenish the present information X with the prior information (²(r), and is minor value, which means that the weight excerted on each incorrect label is negligible. In the similar way, we replace a and
3. with at and @(0) i.e, the vectors a!!) and ((0) will be calculated by
(ي(()) = do. (4) + (1 d)a
(11)
(,(00 = dz)(1)) + (1)
5

---

Published as a conference paper at ICLR 2023
Algorithm 1 IDGP Algorithm
Input: PLL training dataset = { S S n). Epoch T, Iteration K;; Output: The predictive model f(;; ))
1: Initialize the parameters of the predictive model f(x;; ①) and the auxiliary model g(;; St);
2: Initialize ( .0
(1) _3(1) for each instance æ;; . T do
3: for 1, ..........
4: Randomly shuffle the training dataset D and divide it into IC
5: for k 1, ......K.. K do
6: Calculate the parameters of Categorical distribution and Bernoulli distribution 0, and 2; for the instance æ; by the (8. and
7:
Calculate the parameters X 8: Fix θ update f2 by the Eq.
(+) 3
for the instance æ; by the Eq. and
9:
Fix 52, update by the Eq.
10: end for
11: end for
where q denotes the beginning epoch we set reserve the fine prior information for the auxiliary
model g(:; 12), and d E (0, 1) is also positive constant used to provide the present model informa-
tion or 1 (t(()) with the prior information az ((()) (²(()) Before the epoch r, X((1) X j(t)
if j E S., otherwise + e. Before the epoch q, az = =x, and 3:10) ((((), In the above way, we refine the prior information epoch by epoch.
The optimization of the two model f(::; θ) and g(;; 22) at the epoch / for an instance æ: is as follows. First, fx;; +) outputs À while g(æ; Q2) outputs a and B, based on which, according to Eq. and Eq. 9 we replace O and z with the estimation and 2 in 7 Next, to introduce prior information, we use X. a and ( in 10 and 11) to replace X, and /3r in Eq. Note that 0 and 2 are variables, of which constants. Finally, we update the predictive model ((:; Q) and the auxiliary model (a;; 52) by fixing one and updating the other. The whole algorithmic description of the IDGP is shown in Algorithm []] After implementing IDGP in the PLL training dataset, we can use the output À of the main branch inference model f(æ; ②) to calculate the predict results 0 as the label confidence in the test dataset.
calculate the gradient perform backpropagation, while X, x and ]
4 THEORETICAL ANALYSIS
In this section, we pay attention to the estimation eтror bound of the predictive model f(x; O). According to Eq. 7 the empirical risk estimator for the predictive model f(x; θ) denoted by R(f) = > fMAP (f(z;), and for further analysis, we give an upper bound of LMAP (f(x),, Si) as follows:
EMAP (i) S K; Σ (((((()))))))) j=1
(12)
where e¹ denotes the standard canonical vector in R denotes the cross-entropy function, w! X
1 [sT j E St, otherwise w! X 11 and K; log [ [.. Ejes, log I = II II (1
={*) + (ag - 1) log af + (8 - 1) log (1 - 2²). We denote K = max K1, K2, K₂ and scale w to [0, during the training process. The detailed induction of Eq. can be seen in Appendix A.4 Then to formulate the estimation error bound of f, we give the following definition and lemmas.
Definition 1. Let F., F2,......, In E X be n. i.i.d random variables drawn from a probability distribution, 11, ..., {-1,+1} be Rademacher variables with even probabilities, and H (h:X0 + R. be a class of measurable funtuutioble Then the expected Rademacher complexity of H. is defined
Rg, = Exto
sup MEH n i=1
1
Ea,((z)))

---

Published as a conference paper at ICLR 2023
Table 1: Classification accuracy (mean:tstd) of each comparing approach benchmark datasets for instance-
dependent PLL
Datasets
MNIST
FMNIST
KMNIST
CIFARIO
CIFARIO0
IDGP
98.92
0.05%
91.48 0.32%
96.88 0.17%
87.57 0.26%
64.59 0.17%
PLCR
98.56=0.0899
90.10:0.211
95.29-1.219%
86.37+0.388%
64.12
PICO
98.61=0. 1
88.41=0.205%
94.78410.19 19%
86.16:0..2%% %
62.98=003889
VALEN
98.72=0.055% +
90.63=0.300%
96.19=0.75%
83,48+00.2%%
62.96=0.9669
CAVL
98.84=0.05%
19%%
93.69-10.28%
59.67++3.30%
52.59
LWS
98.56=0.06% 88.99-10.66%
92.27=
37.4942.822%
53.98=+0.99%
RC
98.41=0.09%% 89.60-+0.19%
93.78-10.177 17%%
85.95++0.0%%
63,41
CC
98.16=0.14%% 89.8610.11%
94.08:10.35%
79.96:10.99%
62.40=0.84%
PRODEN
98.39=0.10%%
89.79-10.24%
93.79-10.24%
86.04:10211%
62.56 12.56.11.999 1.49%
According Definition 1
given the function space g ((x, + LAAP (f(z), E FJ, the
expected Rademacher complexity of G can be defined as follows:
Rt (9) S,,
sup gEg n
(zg Si)
(13)
We pre-limit both the main model f and the auxiliary model g by clamping their output to [-A, A, A), and the loss LMAP can be bounded, though it would not have extended to infinity in practice.
Lemma 1. Suppose the loss function LMaP (f()), S) is bounded by M,ie., sUPaex, EMAP (f(x),, S),, then for any & > with probability at least 1 - &.
R(f) R(() S (9)
M
log
sup FET
2
2n
Lemma 2. Assume the loss function (f((x), e¹) is L L-Lipschitz with respect f(x) (0 < L x0) for all 1. E y. Let H. [h + [f()) E F} and R,, (Hr)) Ex,o [SUPNEH. D M(x;)]. then the following inequality holds:
(g) S 22pLL Σ 2n, (H, K.
NVV
The proof of Lemma T and is provided in Appendix A.5 and A.6 Based on Lemma and 2 we induce an estimation error bound for our IDGP method. Let f arg minjer R(f) be the empirical risk minimizer and f = arg min R(f) be the true minimizer. The function space H, for the label i. E y is defined as {h + fy(x)]f E F). and R, (H.) is defined as the expected Rademacher complexity of H. with sample size n. Then we have the following theorem.
Theorem Assume the loss function {(f(x), et) L--liphchitz with respect f(x) (0 < T. < 20) for all E y and LMAP (fz),, S) is bounded by M, i... = SUPaEX,SEC.fet LMAP ()) Then, for any &. > O, with probability at least 1 - E.
log E + 4K.
R(A - - SV2C Σ Rn (HL) + M
2n
The proof can be found in Appendix A.7 Theorem 1 means that
the empirical risk minimizer I will converge to the optimal risk minimizer f" for models with a bounded norm.
5 EXPERIMENTS
In this section, validate the effectiveness of our proposed IDGP by performing it augmented benchmark and real-world datasets and comparing its results against DNN-based PLL algorithms. Furthermore, the ablation study and sensitive analysis of parameters conducted to explore IDGP.
7

---

Published as a conference paper at ICLR 2023
Table Classification accuracy (mean:istd) of comparing algorithms the real-world datasets.
Lost
BirdSong
MSRCv2
Soccer Player
Yahoo!News
IDGP
77.02++0.82%
7..23=0.17%
50.45++0.44%%
55.99::0228%
66.62:#0.19%
VALEN
76.87-10.86%
73.39110.26%
49.97.10.33%
55.8110.110%
66.26.10.13%
CAVL
55.89-00.42%
03.47:=0.13%
44.73=0.965%
14.06=0.667%
65.44=00.33%
LWS
73.13::0.32%
51.45=0.26%%
49.85=0 +
30.24:10.45%%
48.21
RC
76,26:t0.46%
69.33=0.32%
49.47=0043%%
56,02=+0.59 %
63.51=0.20%
CC
63.54-1...554 69.90:10.58%
41.50:10.44%%
49.07-10.36%
54.86:10.48%
PRODEN
76.47:10.25%
73.44+0.112%
45.10:10.16%
54.05
66. 16.14%
5.1 DATASETS
We implement IDGP with compared DNN-based algorithms on five widely used benchmark datasets in
deep learning, including MNTST LeCun al. 1998), Kuzushi ST Clanuwat al. (018).
ST Xiao et al. (2017) CIFAR-10 0 and CIFAR- 1--R--100 100 Krizhevsky et al. 2009).
Instance-dependent partial labels for these datasets are generated through the same strategy Xu et al. 2021). which for the first time consider instance-dependent PLL.
Besides, part of the comparing algorithms also performed on five frequently used real-world datasets, which come from different practical application domains, including Lost Cour et al. (201T), BirdSong Briggs al. (2012), MSRCV2 Liu & Dietterich (2012), Soccer Player Zeng et al.
(2013) and Yahoo NNws Guillaumin et al. (2010).
For benchmark datasets, we split 10% samples from the training datasets for validating. For each real-world dataset, we run the methods with 80%/10%//0% train/validation/test split. Then we run five trials on each datasets with different random seeds and report the mean accuracy and standard deviation of all comparing algorithms.
5.2 BASELINES
We compare IDGP with eight DNN-based methods:1) PLCR Wu et al. (2022), , regularized training framework which is based on data augmentation and utilizes the manifold consistency regularization term to preserve the manifold structure both in feature space and label space. 2) Pico Wang et al.) (2022), contrastive learning framework which is based data augmentation and performs label disambiguation based on the contrastive prototypes. 3) VALEN Xu et al. (2021), an instance-dependent
PLL framework which guides the training process via the recovered latent label distributions. 4) CAVL Zhang et al. (2021), a discriminative approach which identifies correct labels from candidate labels by class activation value. 5) Lws Wen al. 2 identification based method which introduces leverage parameter to consider the trade-off between losses on candidate and non-candidate labels. 6) RC Feng et al. (2020), risk-consistent PL.I method which is induced by an importance reweighting strategy. 7) CC Feng et al. (2020), a conssiteeee PLL method which leverages the transition matrix describing the probability of the candidate label given a correct label. 8) PRODEN Lv et al. 2020). self-training style algorithm which provides a framework to equip arbitrary stochastic optimizers and models in PLL. Note that PLCR and PiCO will not be compared on the real-world datasets due to the requirement of data augmentation.
To ensure the fairness, we employ the same network backbone, optimizer and data augmentation strategy for all the comparing methods. For MNTST, Kuzushi ST and Fashi on-MNT we take LeNet-5 as their backbone. For CIFAR-10 and CIFAR-100 00, the network backbone changed ResNet-32 He et al. (2016). For all the real-world datasets. simply adopt the linear model. The optimizer is stochastic gradient descent (SGD) Robbins & Monro (195I) with momentum 0.9 and batch size 256. The details of data augmentation strategy shown in Appendix A.8 Besides, the learning rate is selected from {10-4 10 10-3 10 10-2), and the weight decay selected from (10-5, 10-4 10-3 10 10-2) according the performance the validation.
5.3 EXPERIMENTAL RESULTS
The performance of each DNN- based method on each corrupted benchmark dataset is summarized in Table where the best results are highlighted in bold and e/o indicates whether IDGP statistically
8

---

Published as a conference paper at ICLR 2023
Table 3: Classification accuracy (meaniist)) for comparison against IDGP-ML.
Dataset
IDGP
IDGP-ML
Lost
77.02++0.82%
52.50
BirdSong
74.23:10.17%
71.13:10.45%
MSRCv2
50.45:10..47% %
43.04:10.98%
Soccer Player
55.99-10.28%9 %
49.56-10.11%
Yahoo!News
66.62-10.19%
51.48:0.07%
21
(b) Bi rdSong
Yahoo NNew
Figure 1: Parameter sensitivity analysis for IDGP on Lost, BirdSong and Yahoo !Ne
wins/loses to the comparing method on each dataset additionally (pairwise t-test at 0.05 significance level). We can overall see that IDGP sigmificantly outperforms all comparing approaches on all benchmark datasets (except Kuzushi ji -i-MMIST where VALEN performs comparably against IDGP), and the improvements are particularly noticeable on Fashion and CIFAR-10,
Table 2 demonstrates the ability of IDGP solve the PLL problem in real-world datasets. PLCR and PICO are not compared on the real- world PLL datasets due to the inability of data augmentation be employed the extracted features from various domains. We can find that method has stronger competence than others in all datasets except Soccer Player where IDGP loses to RC but still ranks second. As for Bi rdSong, MSRCv2 and Y Yahoo News, the performance of IDGP is sigmificantly better than all other comparing algorithms. And when it comes to Lost. our method is comparable VALEN, PRODEN and RC, while obviously better than the rest.
5.4 FURTHER ANALYSIS
To demonstrate the effectiveness of the iteratively refined prior information introduced by IDGP, we remove the loss function Lrgg to reverse IDGP to IDGP-ML, which only uses the log-likelihood function for optimization. The performance of IDGP-ML against IDGP is also measured by the classification accuracy (with pairwise t-test at 0.05 significance level). As is illustrated in Table B with the assistance of the prior information which IDGP provides and improves epoch by epoch, IDGP achieves superior performance all real-world datasets compared IDGP-MI..
Furthermore, we conduct parameter sensitivity analysis to study the influence of the two hyper- parameters a, 7 on our algorithm, which decides the scale of Dirichlet and Beta distribution parameters. Figure iillustrate the sensitivity of IDGP in the real-world datasets including Lost, Birdd rdSong, Yahoo New when varies from 0.001 1000 and increases from 0.1 3. We can easily find that as for the small-scale real-world datasets like Lost, and are suggested around 0.1 and 0.5, respectively. For the more sarge-scale real-worke datasets like BirdSong, Yahoo! News, IDGP seems insensitive to a, and are suggested around 1.
CONCLUSION
this paper, we consider more realistic scenario, instance-dependent PLL, and explicitly decompose and model the generation process of instance-dependen candidate labels. Then based the decompositional generation process, a novel instance-dependee PLL approach IDGP proposed by to further introduce and refine the prior information in every training epoch via MAP. The experimental comparisons with other DaN-based algorithms on both -edendedd corrupted benchmark datasets and real-wor sal-world datasets demonstrate the effectiveness of proposed method.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This research was supported by the National Key Research & Development Plan of China (No. 2021ZD01 14202), the National Science Foundation of China (62206050, 62125602, and 62076063), China Postdoctoral Science Foundation (2021M700023), Jiangsu Province Science Foundation for Youths (120210220), Young Elite Scientists Sponsorship Program of Jiangsu Association for Science and Technology (TJ-2022-078).
REFERENCES
Forrest Briggs, Xiaoli Z Fern, and Raviv Raich. Rank-loss support instance machines for miml instance annotation. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 534-542, 2012.
Ching-Hui Chen, Vishal M Patel, and Rama Chellappa. Learning from ambiguously labeled face images. IEEE transactions on pattern analysis and machine intelligence, 40(7):1653-1667, 2017.
Yi-Chen Chen, Vishal M Patel, Rama Chellappa, and P Jonathon Phillips. Ambiguously labeled learning using dictionaries. IEEE Transactions on Information Forensics and Security, 9(12): 2076-2088, 2014.
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. arXiv preprint arXiv:181.001718, 2018.
Timothee Cour, Ben Sapp. and Ben Taskar. Learning from partial labels. The Journal of Machine Learning Research, 12:1501-1536, 2011.
Lci Feng and Bo An. Leveraging latent label distributions for partial label learning. In ICCAI, pp. 2107-2113 13, 2018.
Lei Feng, Jiaqi Lv, Bo Han, Miao Xu, Gang Niu, Xin Geng, Bo An, and Masashi Sugiyama. Provably consistent partial-label learning. arXiv preprint trXiv:2007.08929 2020.
Matthieu Guillaumin, Jakob Verbeek, and Cordelia Schmid. Multiple instance metric learning from automatically labeled bags of faces. In European conference computer vision, pp. 634-647. Springer, 2010.
B Han, Q Yao, X Yu, G Niu, M Xu, W Hu, Tsang, and M Sugiyama. Robust training of deep neural networks with extremely noisy labels. In Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS), volume 2, pp. 4, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Shuo He, Lei Feng, Fengmao Lv, Wen Li, and Guowu Yang. Partial label learning with semantic label representations. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 545-553, 2022.
Eyke Hüllermeier and Jürgen Beringer. Learning from ambiguously labeled examples. Intelligent Data Analysis, 10(5):419-444 439, 2006.
Rong Jin and Zoubin Ghahramani. Learning with multiple labels. In NIPS, volume 2, pp. 897-904. Citeseer, 2002.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFF, 86(11):2278-2322 1998.
Liping Liu and Thomas Dietterich. conditional multinomial mixture model for superset label learning. In Advances in neural information processing systems, pp. 348-556. Citeseer, 2012.
10

---

Published as a conference paper at ICLR 2023
Jie Luo and Francesco Orabona. Learning from candidate labeling sets. Technical report, MIT Press, 2010.
Jiaqi Lv, Miao Xu, Lei Feng, Gang Niu, Xin Geng, and Masashi Sugiyama. Progressive identification of true labels for partial-label learning. In International Conference Machine Learning, pp. 6500-6510. PMLR, 2020.
Jiaqi Lv, Biao Liu, Lei Feng, Ning Xu, Miao Xu, Bo An, Gang Niu, Xin Geng, and Masashi Sugiyama.
On the robustness of average losses for partial-label learning. arXiv preprint arXiv:2106.06152, 2021.
Gengyu Lyu, Yanan Wu, and Songhe Feng. Deep graph matching for partial label learning. In Proceedings of the International Joint Conference on Artificial Intelligence, pp. 3306-3312, 2022.
Thomas Minka. Estimating dirichlet distribution, 2000.
Nam Nguyen and Rich Caruana. Classification with partial labels. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 551-559, 2008.
Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pp. 400-407, 1951.
Cai-Zhi Tang and Min-Ling Zhang. Confidence- discriminative partial label learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
Deng-Bao Bag Wang. Li Li, and Min-Ling Zhang. Adaptive graph guided disambiguation for partial label learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 83-91, 2019.
Haobo Wang, Ruixuan Xiao, Yixuan Li, Lei Feng, Gang Niu, Gang Chen, and Junbo Zhao. Pico: Contrastive label disambiguation for partial label learning. arXiv preprint arXiv:2201.0888 2022.
Hongwei Wen, Jingyi Cui, Hanyuan Hang, Jiabin Liu, Yisen Wang, and Zhouchen Lin. Leveraged weighted loss for partial label learning. In International Conference Machine Learning, pp. 11091-11100, PMLR, 2021.
Dong-Dong Wu, Deng-Bao Wang, and Min-Ling Zhang. Revisiting consistency regularization for deep partial label learning. In International Conference Machine Learning, pp. 24212-24225. PMLR, 2022.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint wwXiv:1708.077444 2017.
Ning Xu, Yun-Peng Liu, and Xin Geng. Label enhancement for label distribution learning. IEEE Transactions on Knowledge and Data Engineering. 33(4): 2019a.
Ning Xu, Jiaqi Lv, and Xin Geng. Partial label learning via label enhancement. In Proceedings of the
AAAI Conference on Artificial Intelligence, volume 33, pp. 5557-5564, 2019b.
Ning Xu, Jun Shu. Yun-Peng Liu, and Xin Geng. Variational label enhancement. In International Conference on Machine Leurning, pp. 10597-10606. PMLR, 2020.
Ning Xu, Congyu Qiao, Xin Geng, and Min-Ling Zhang. destance-depender partial label learning. Advances in Neural Information Processing Systems, 34, 2021.
Yao Yao, Jiehui Deng, Xiuhua Chen, Chen Gong, Jianxin Wu, and Jian Yang. Deep discriminative cnn with temporal ensembling for ambiguously-labelee image classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 12669-12676, 2020a.
Yao Yao, Chen Gong, Jichui Deng, and Jian Yang. Network cooperation with progressive disambigua- tion for partial label learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 471-488 Springer, 2020b.
Fei Yu and Min- Ling Zhang. Maximum margin partial label learning. Asian conference on machine learning, pp. 96-111 PMLR, 2016.
11

---

Published as a conference paper at ICLR 2023
Zinan Zeng, Shijie Xiao, Kui Jia, Tsung- Hann Chan, Shenghua Gao, Dong Xu, and Yi Ma. Learning by associating ambiguously labeled images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 708-715, 2013.
Fei Zhang, Lei Feng, Bo Han, Tongliang Liu, Gang Niu, Tao Qin, and Masashi Sugiyama. Ex- ploiting class activation value for partial-label learning. In International Conference on Learning Representations, 2021.
Min-Ling Zhang and Fei Yu. Solving the partial label learning problem: An instance- -basd approach. In Twenty-fourth international joint conference artificial intelligence, 2015.
Min-Ling Zhang, Bin-Bin Zhou, and Xu- Sung Liu. Partial label learning via feature-awar disam- biguation. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1335-1344, 2016.
A APPENDIX
A.1 DERIVATIONS OF BQ
44)
AND BQ.(6)
The derivation of Eq. as follows:
LML
log (0,
log
Σ jES;
p(ys = j|z;)) (p5/pp()
Σ log Σ xx 0.) II
(e =
(14)
1=1
JES
km
log
21-2))) - 111 =))
JES; k=
log
II
II =/)
JES.
KES
kgs!
The derivation of Eq. 6 is as follows:
Creg
log p( 0, (z.) + p(z((x))
log r(E; x
I(a + B) (2)) 2/)
=1 [ (0)) (0)))t== + log
II
j=1 I(a ((5))
(15)
log II (01)x1 -1 + log II
(=)) x() Cr
j=1
j=1
n
ΣV (X 1) log(!?) + (a log( (2 (3) 1) log(1 2/) + Cr.
T 7 x C- log I1; r(x
T(a Due to that we later use E, and P in Eq.
log I1
P(a ) and (11 to replace X, a and /3 in (7), which are fixed to be constants, Cr will be a constant and ignored. Here, the derivation of (6) has been finished.
12

---

Published as a conference paper at ICLR 2023
A.2 DEGENERATION OF EQ.
The proposed process can accommodate the uniform generation process of candidate labels by setting the parameters of Bernoulli distribution to constant p. which also means the flipping probability. In this case, we can degenerate LML, Lreg and L to the following form.
L =
log
(1-
JES;
RES!
kg5
log
0 (1 10 10 11-1
(16)
log(1 p) ct 1ct1l11:lp1 181
0
jES;
log
log(1 - p) |S: 1 181-1
JES.
Due to that the second term is constant, the final LML is formulated as
LMLDD = Σ log 0
(17)
JES,
Due to the parameters of Bernoulli Distribution constant, do not need Beta Distribution. Hence,
Lree-D = Σ
(2)
1)log0
(18)
1=1 j= j=1
Finally,
CD = LML-D + Lreg.D Σ
log 0f
(2
1)log03.
(19)
jES.
i=1
1=
A.3 DERIVATIONS OF (8) AND (Q)
0, ~ p(0, A,) & Dir(O,1A,) Ca.(L,10))
r(E X))
II
(B/))/
(0))
f=1
==1
(20)
r( X²) 1 j=1
(
Dir(0,A, 4.).
The above has proved the conjugacy of Dirichlet and Categorical distribution. For Dirichlet distribu- tion, its expectation be calculated
X + D Ei A +1 Ei X +
X of
(21)
The mathematical derivations of (8) completed. (9) can be proved in similar way due to Dirichlet distribution and Categorical distribution are the generalization forms Beta distribution and Bernoulli distribution respectively.
13

---

Published as a conference paper at ICLR 2023
A.4 CALCULATION DETAILS OF EQ. (12)
According to multivariate basic inequality, we can obtain:
- -log Σ jES;
II
-
log |
Σ JES;
log
0¹
+ log II KES
o II
(1- (1-
[S
RES
kgs!
kgS!
(22)
Then we can calculate Eq. (12) as
[APP Si) log Σ
0f II
II (1 (1-
JES.
KES!
kg...
(2] 1) log 02 + (a} 1) log of + (8) - log (1 - z)
j=1
< log |Sil
log 02 + +log II
II (1 )
|S
jES:
KE
kg
E (A - 1) log 0² + (a) 1) log of + (3% - 1og (1 - -1))
1=1
log 18:1 + |Si jES;
log II
II (1 - 2/ + (a} - 1) log 2/ + + 11)log (1 -2x))
KES
kgS!
Σ JES,
N 1 1+
log
0!
Σ
(xf - 11) log 0!
JES,
K₁
of ((f(((()))
Ø=1
(23)
where el denotes the standard canonical vector in RS, and l denotes the cross-entropy func- tion, w 1 ST. j E S. otherwise ii X/ 1, and K; log [S. S.. Ejes, log IREE 2 IIkez (1 - =/ + (af - 1) log य + (B) - 1) (1 z!{-
A.5 PROOF OF LEMMA
In order to prove this lemma, we first show CMAP can be bounded. Due to the ouput of f.g is limited in [-A, A], the following inequations hold:
a exp (A/7) +b B
(24)
acexp A/7) + bc
= acxp(-A/1) +b 2(aexp(AA7) + b) +
acxp(A/7)+br +b+
$ 2 dexp(A/7)
F (25)
+ A/7) + 26 +1
Hence,
[MAP < log looo|S,,BB(111 - F) 18:1 1 clogl
BllogBBa BE F))0))))))x(p(/1111)
When M takes the value larger than S, B( F)
clogiBE(1
ex]a the loss LMAP will be bounded. Note that the limitation to the output of the model excludes extreme (conditional) probabilities, the effect which could be ignored which is large enough.
14

---

Published as a conference paper at ICLR 2023
Then, we show that the one direction Su R(f) R(f) is bounded with probability at least 1 - f/2, and the other direction can be similarly shown. Suppose example (æ.,, St) is replaced by another arbitrary example (a',, St), then the change of sur R(S) - R(f) is no greater than M/(2n), the loss function L are bounded by M. By applying McDiarmid's inequality, for any g > with probability at least - E/2.
M
log 2
sup R(f) R(F) E fer
sup R(f) - R(() fEJ
+
(26)
2
2n
By sysmetrization, can obtain
R(f) - R(()) < 20
sup
(27)
fEF
By further taking into account the other side sup R(f) R(f). we have for any & 0, with probability at least 1 E.
M
log \
suflR(f) - R(()) & (9) + 2
(28)
feF
2n
A.6 PROOF OF LEMMA 2
The upper bound loss function of LmAr(f (m is denoted by
(.)())), = Ki +
w ((f((()))
(29)
Correspondingly, the function space for can be defined as:
n ((f()) F}
(30)
Then the expected Rademacher complexity of & can be defined as follows:
1
Ri (g) = EE.
sup ozr S:) GEG i=1
(31)
For each example (æ,, St), since wi is bounded [0, pl and K; S K, we can obtain (g) s RR,() < pcR,, (loF)-K where eo denotes [foFFif E F}. Since H. = [h: x ++ fz()) E F} and the loss function (f(x, e))) is L-Lipschitz for all E V. by the Rademacher vector contraction inequality, we have (f, F) S V2L E.ey R, (H.). Then the proof completed.
A.7 PROOF OF THEOREM 1
Theorem is proven through
R()) R(f)) -RF) R( + R()) R(f*) +R() -R(F)
S R(f) R(F) + R(F) R((")
R(f) -R(A)
s 2 sup
fer
(32)
log 2
< (g) + MM
2n
log 2/
< 2/ Σ
Hi (H..) + M V
+ 4K
2n
15

---

Published as a conference paper at ICLR 2023
Table 4: Classification accuracy (meanistd) of each comparing approach benchmark datasets, of which instance-dependent partial labels are generated by our generation model.
Datasets
MNIST
FMNIST
KMNIST
CIFARIO
CIFAR100
IDGP
98.87 0.05%
91.20
0.21%
97.28 0.19%
90.18 0.32%
64.67 0.23%
PLCR
98.46=0.2255
90.18:0.2279
95.19:1.1139 13%
87.07+0.34%%
64.43+0.44%
PICO
1.110..1
88.32:40.1 17%
94.78410.19 19%
86.56+22.749 7
1.005-0.266%
VALEN
98.77=0.04%% 90.58:0.366%
96.41=0.333%
86.31+0.37%
63.77=0.511 %
CAVL
98.77=0.09%% 88.09:10.32%
93.30-10.55%
60.20+2.64%%
52.85 %
LWS
98.52=0..14% 88.90-10.25%
92.69--0.76%%
39.75++2.22%%
13.52-+1.3%%
RC
98.35=0.114%
89.78:+..29%
94.06:1..05%%
85.48:10.27%
63.24
CC
98.15=0.07% 89.67:10.11%
94.12:10.1%%
80.19:10.66%
62.11=1003%
PRODEN
98.43=00110%
89.72.10366%
94.08110.5%
83.60:10.20%
0..87=00.5%%
Table Classification accuracy (meanistd) for comparison against IDGP-ML benchmark datasets, of which instance-dependent partial labels are generated by our generation model.
Dataset MNIST
IDGP
IDGP-MP ML
98.87:10.05%
91.79:10.122
Kuzushiji-MNB
97.28:00.9%%
76.67:10.34%%
Dashion-MINIST
91.27=:0.21% %
78.3810..9%%
CIFAR-10
90.181:0.22%
69.47:10.5%%
CIFAR-100
64.67:10.23%
40.23:10.57%
A.8 DETAILS DATA AUGMENTATION
For all benchmark datasets, we apply Random Horizontal Flipping, Random Cropping, and Cutout. For CIFAR 10 and CIFAR- -00, AutoAugment additionally applied.
A.9 EXTENDING EXPERIMENTS
Our Generation Model. We also synthesize the sampled correct label from Categorical Distribution ply = j(x) and incorrect candidate labels from Bernoulli Distribution p(3 1/x). For the former, correct labels has been contained in these datasets. For the latter, use the confidence prediction of clean neural network g(æ: ②2) (trained only with correct labels) model the Bernoulli Distribution, ie., p(5) = 1 1|x) = sigmoid(g, (g (æ; s2)). Table illustrates the performance of IDGP and comparing benchmark datasets, of which destance-dependent partial labels generated by our generation model. From the table, we can overall see that IDGP consistently outperforms all comparing approaches on all benchmark datasets, and the improvements particularly noticeable on Kuzushi ji -NIIST and CIFAR-10.
approaches
Ablation Study. As is illustrated in Table IDGP also achieves superior performance all benchmark datasets, of which instance-dependent partial labels are generated by our generation model.
Sensitive Analysis. For the benchmark dataset MMNNIISTT the performance of IDGP is stable and effective in the case that is around 10 and y around 1.5.
120
(a) Fashi on MINISS
Figure 2: Parameter sensitivity analysis for IDGP on Fashion-ooei MMNNS.....
16


Reviews:
[{"review_id": "hBZ2iVTnzVX", "reviewer": "Reviewer_XV9C", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an instance-dependent PLL approach by decomposing the generation process of candidate labels into two processes and explicitly modeling the processes using different probability distributions, which the risk estimator is built upon. The MAP technique is employed to create the final empirical risk estimator. Then, the generation error bound of it is proposed for it in the theoretical analysis. Finally, experiments on corrupted benchmark datasets and real-world datasets validate the effectiveness of the proposed approach.", "summary_of_the_review": "This paper is intriguing and makes good contributions to Instance-Dependent Partial Label Learning.", "strength_and_weaknesses": "Strength:\n1.  It makes sense that the paper decomposes and models the generation process of candidate labels with two separate processes. The generation process is reasonable and interesting, and it has the potential to benefit future research.\n\n2.  This definition of the data generation process is mathematically clear. The reasoning that follows is standard and rigorous, including the derivation of the risk estimator and the statistical analysis of the generalization error bound, is standard and rigorous.  All of these contribute to the soundness of the paper.\n\n3.  The proposed approach IDGP indicates the superiority when it is compared to the existing PLL methods in benchmark and real-world datasets.\n\nWeakness:\n1. In the paper, the authors respectively utilize Categorical and Bernoulli distributions to capture these two processes. More explanations can be provided on it to further clarify why the instance-dependent partial labels could be modeled by these two distributions.\n\n2. The presentation is good, but it might be better to use visuals and figures to provide an additional explanation of the learning procedure. \n\n3. The approach in Proposed Method includes two branch models, the main branch $f$ and the auxiliary branch $g$. The manually corrupted benchmark datasets in Experiments also use two sub-models $f$ and $g$. Overall, the former corresponds to the latter. However, the former $g$ appears to output two variables, $alpha$ and $beta$, whereas the latter $g$ appears to output only one. The difference inside should be explained in detail. \n\n4. Some related works [1, 2] are suggested to be cited and discussed. \n[1] Lyu, Gengyu, Yanan Wu, and Songhe Feng. \"Deep Graph Matching for Partial Label Learning.\"\n[2] He, Shuo, et al. \"Partial Label Learning with Semantic Label Representations.\"\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "fZvyRnuDSUW", "reviewer": "Reviewer_Zc1G", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims to address the challenging problem of Instance-Dependent Partial Label Learning, which is caused by overfitting on candidate labels, by rethinking the generation process of instance-dependent partial labels. In detail, the authors consider a two-step decomposition-based process. Categorical Distribution generates the correct label for a given instance, and Bernoulli Distribution samples the incorrect labels. In this way, they explicitly model the generation process. Then, to form their optimization objective, Maximum A Posterior (MAP) is performed, with Dirichlet Distribution and Beta Distribution introduced as prior distributions.", "summary_of_the_review": "Overall, the paper proposes an insightful and novel approach to dealing with the instance-dependent PLL, and the experimental results demonstrate its effectiveness. As a result, I vote for acceptance.\n\n\n", "strength_and_weaknesses": "Pros: \n1.\tThe proposed generation process is inspiring and offers a new perspective for me on Instance-Dependent Partial Label Learning. The generation process is distinct from traditional Multi-class Learning and Multi-Label Learning, but it also shares a few similarities, which is significant for understanding this problem.\n2.\tThe paper is well-organized and logical. It begins by explicitly modeling the process with decoupled probabilistic distributions, then forms the optimization objective using a MAP technique, and finally theoretically analyzes the estimation error of the optimization objective.\n3.\tThe experimental results are superior to the baselines, demonstrating the effectiveness of the proposed approach.\n\nCons and Qs:\n1. Since the optimization is based on the proposed generation model, the log-likelihood\nloss function in Eq.(4), which is derived from Eq.(1)~(3), could be more detailed. On the other hand, it is also used for ablation studies. Hence, more explanations need to be made to convince me.\n2. Could you explain whether the proposed process is adaptable to a uniform generation process of candidate labels or the relationship between them?\n3. I am curious about why Eq. (6) holds. Because the derivation is not in the appendix, a detailed explanation would be greatly appreciated.\n4. I'm not sure how to estimate the parameters $\\lambda$, $\\alpha$, $\\beta$, $\\theta$ and $z$. Eq.(8) and Eq.(9) use $\\lambda$, $\\alpha$, $\\beta$ to calculate $\\hat{\\alpha}$, $\\hat{\\beta}$, while Eq.(10) and Eq.(11) replace $\\lambda$, $\\alpha$, $\\beta$ with $\\hat{\\lambda}$, $\\hat{\\alpha}$, $\\hat{\\beta}$.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "TGXt4jK_a7", "reviewer": "Reviewer_YnxC", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates the problem of partial label learning (PLL) on the instance-dependent scenario, which is a more realistic and practical setting. A new generative model is proposed that decomposes the generative process into two parts related to the ground-truth labels and other candidate labels, respectively. The distribution of the correct label and the incorrect label in the candidate label set of each training example are modeled explicitly by decoupled probability distributions Categorical distribution and Bernoulli distribution. MAP estimation is employed on the PLL training dataset to deduce a risk minimizer. Experiments on benchmark and real-world datasets validate the effectiveness of the proposed method. ", "summary_of_the_review": "Overall, this is a well-written paper and proposes a novel approach for instance-dependent partial label learning.", "strength_and_weaknesses": "Strengths:\n1. This paper provides a new perspective on instance-dependent partial label learning. \n2. This paper provides a theoretical analysis for the estimation error bound of the proposed empirical risk. \n3. As far as I know, it is the first attempt to explicitly models the generation process of the correct labels and incorrect positive labels from different distributions. \n4. The theoretical justifications and empirical validations are solid.\n\nWeaknesses:\n1. The authors should give more discussion about the results on real-world PLL datasets. \n2. The authors should give more details the MAP optimization problem in Eq. (7)\n3. “Then we perform Maximum A Posterior(MAP) estimation” should be revised as “Then we perform Maximum A Posterior (MAP) estimation”.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "K0qD4CkLL4", "reviewer": "Reviewer_bU6i", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Authors extend VALEN with an instance-dependent incorrect label process.", "summary_of_the_review": "**Don't panic.**  This paper wants to be accepted you just have to address my concerns and my score will change.  \n\nFactors that are creating a low score right now are:\n1.  a confusing exposition in section 3.2 [have someone with strong writing skills read this section and help you edit it]\n2. an apparently disqualifying precondition in lemma 1 and theorem 1 [justify it or change it]\n3. use of your own generative model for evaluation [drop all use of these datasets and only use datasets from Table 2].\n\n\n(Updated score from 3->6)", "strength_and_weaknesses": "Strengths:\n  * Clearly a good idea to improve an already strong baseline.\n\nWeaknesses:\n  * Clarity is lacking.\n    * In equation (1), is the generation of candidiate labels conditioned on the correct label?  or perhaps conditionally independent of the correct label given the instance?  the notation in equation (1) is not clear and the surrounding text is silent.  (later in equation (3), there's a latent variable that is jointly conditioned with the instance, but not the correct label; and strangely equation (3) apparently attempts to model the probability that the correct label is [not] in the candidate set, whereas in PLL this is assumed always in the candidate set.) \n    * I find the discussion around equation (10) completely unintelligible, but possibly related to my previous concern listed above.\n  * A theoretical correctness concern:\n    * In Lemma 1 and Theorem 1, boundedness of $\\mathcal{L}_{MAP}$ does not superficially appear to be a reasonable assumption, as it has a negative log in it, but this is presented without comment.\n  * Some experimental concerns:\n    * The transformation of classification datasets to PPL datasets uses the assumed generative model, which degrades from all the conclusions drawn from it.  Consequently, I'm not drawing any of the conclusions you are hoping for:\n      * Table 1 does not convince me the proposed approach is better.\n      * Table 3 does not convince me regularization is beneficial.\n      * Figure 1 does not convince me the technique is not overly sensitive to choice of hyperparameter, except for Figure 1b.\n      * **Note**: Table 2 *is* persuasive.\n      * **Constructive feedback**: \n        * Drop Table 1 entirely.\n        * Populate Figure 1 with 3 datasets from Table 2.\n        * Do the comparison in Table 3 on datasets from Table 2.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%