Original: D ECOMPOSITIONAL

Original: G

Original: ENERATION

Original:  P ROCESS FOR I NSTANCE -D EPENDENT P ARTIAL L ABEL L EARNING
Revised: D ECOMPOSITIONAL G ENERATION P ROCESS FOR I NSTANCE -D EPENDENT P ARTIAL L ABEL L EARNING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true.
Revised: Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true.

Original: Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels and model the generation process of the candidate labels in a simple way.
Revised: Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels and model the generation process of the candidate labels in a simple way.

Original: However, these approaches usually do not perform as well as expected due to the fact that the generation process of the candidate labels is always instance-dependent.
Revised: However, these approaches usually do not perform as well as expected due to the fact that the generation process of the candidate labels is always instance-dependent.

Original: Therefore, it deserves to be modeled in a refined way.
Revised: Therefore, it deserves to be modeled in a refined way.

Original: In this paper, we consider instancedependent PLL and assume that the generation process of the candidate labels could decompose into two sequential parts, where the correct label emerges first in the mind of the annotator
Revised: In this paper, we consider instancedependent PLL and assume that the generation process of the candidate labels could decompose into two sequential parts, where the correct label emerges first in the mind of the annotator

Original: but then the incorrect labels related to the feature are also selected with the correct label as candidate labels due to uncertainty of labeling.
Revised: but then the incorrect labels related to the feature are also selected with the correct label as candidate labels due to uncertainty of labeling.

Original: Motivated by this consideration, we propose a novel PLL method that performs
Revised: Motivated by this consideration, we propose a novel PLL method that performs

Original: Maximum A Posterior(MAP)
Revised: Maximum A Posterior (MAP)

Original: based on an explicitly modeled generation process of candidate labels via decomposed probability distribution models.
Revised: based on an explicitly modeled generation process of candidate labels via decomposed probability distribution models.

Original: Extensive experiments on manually corrupted benchmark datasets and real-world datasets validate the effectiveness of the proposed method.
Revised: Extensive experiments on manually corrupted benchmark datasets and real-world datasets validate the effectiveness of the proposed method.

Revised: Source code is available at https://github.com/palm-ml/idgp .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Partial label learning(PLL) aims to deal with the problem where each instance is provided with a set of candidate labels, only one of which is the correct label.
Revised: Partial label learning (PLL) aims to deal with the problem where each instance is provided with a set of candidate labels, only one of which is the correct label.

Original: The problem of learning from partial label examples naturally arises in a number of real-world scenarios such as web data miningLuo &
Revised: The problem of learning from partial label examples naturally arises in a number of real-world scenarios such as web data mining

Original:  Orabona (2010), multimedia content analysis Zeng et al.
Revised: Luo & Orabona (2010), multimedia content analysis Zeng et al.

Original: ; Chen et al.
Revised: ; Chen et al.

Original: (2017),  ecoinformatics Liu & Dietterich (2012); Tang & Zhang (2017).
Revised: (2017), and ecoinformatics Liu & Dietterich (2012) ; Tang & Zhang (2017).

Original: A number of methods have been proposed to improve the practical performance of PLL.
Revised: A number of methods have been proposed to improve the practical performance of PLL.

Original: Identificationbased PLL approaches Jin & Ghahramani (2002); Nguyen & Caruana (2008); Liu & Dietterich (2012);
Revised: Identificationbased PLL approaches Jin & Ghahramani (2002); Nguyen & Caruana (2008); Liu & Dietterich (2012);

Original: Chen et al.
Revised: Chen et al.

Original: ; Yu & Zhang (2016) regard the correct label as a latent variable and try to identify it.
Revised: ; Yu & Zhang (2016) regard the correct label as a latent variable and try to identify it.

Original: Average-based approaches Hüllermeier & Beringer (2006);
Revised: Average-based approaches Hüllermeier & Beringer (2006);

Original: Cour et al.
Revised: Cour et al.

Original: (2011); Zhang & Yu (2015) treat all the candidate labels equally and average the modeling outputs as the prediction.
Revised: (2011); Zhang & Yu (2015) treat all the candidate labels equally and average the modeling outputs as the prediction.

Original: In addition,the risk-consistent methods Feng et al.
Revised: In addition, risk-consistent methods Feng et al.

Original: Wen et al.
Revised: Wen et al.

Original: (2021) and classifier-consistent methods
Revised: (2021) and classifier-consistent methods

Original: Lv et al.
Revised: Lv et al.

Original: Feng et al.
Revised: Feng et al.

Original: (2020) are proposed for deep models.
Revised: (2020) are proposed for deep models.

Original: Furthermore, aimed at deep models,
Revised: Furthermore, aimed at deep models,

Original: Wang et al.
Revised: Wang et al.

Original: (2022) investigated contrastive representation learning, Zhang et al.
Revised: (2022) investigate contrastive representation learning, Zhang et al.

Original: (2021) adapted class activation map and Wu et al.
Revised: (2021) adapt the class activation map, and Wu et al.

Original: revisited consistency regularization in PLL.
Revised: (2022) revisit consistency regularization in PLL.

Original: It is challenging to avoid overfitting on candidate labels, especially when the candidate labels depend on instances.
Revised: It is challenging to avoid overfitting on candidate labels, especially when the candidate labels depend on instances.

Original: Therefore, the previous methods assume that the candidate labels are instanceindependent.
Revised: Therefore, the previous methods assume that the candidate labels are instanceindependent.

Original: Unfortunately, this often tends to be the case that the incorrect labels related to the feature are more likely to be picked as candidate label set for each instance.
Revised: Unfortunately, this often tends to be the case that the incorrect labels related to the feature are more likely to be picked as candidate label set for each instance.

Original: Recent work Xu et al.
Revised: Recent work Xu et al.

Original: (2021) has also shown that the presence of instance-dependent PLL imposes additional challenges but is more realistic in practice than the instance-independent case.
Revised: (2021) has also shown that the presence of instance-dependent PLL imposes additional challenges but is more realistic in practice than the instance-independent case.

Original: In this paper, we focus on the instance-dependent PLL via considering the essential generating process of candidate labels in PLL.
Revised: In this paper, we focus on the instance-dependent PLL via considering the essential generating process of candidate labels in PLL.

Original: To begin with, let us rethink meticulously how candidate labels arise in most manual annotation scenarios.
Revised: To begin with, let us rethink meticulously how candidate labels arise in most manual annotation scenarios.

Original: When one annotates an instance, though the correct label has already emerged in the mind of the annotator first, the incorrect labels which are related to the feature of the instance confuse the annotator, then leading to the result that the correct label and some incorrect labels are packed together as the candidate labels.
Revised: When one annotates an instance, though the correct label has already emerged in the mind of the annotator first, the incorrect labels which are related to the feature of the instance confuse the annotator, then leading to the result that the correct label and some incorrect labels are packed together as the candidate labels.

Original: Therefore, the generating process of the candidate labels in instance-dependent PLL could be decomposed into two stages, i.e., the generation of the correct label of the instance and the generation of the incorrect labels related to the instance, which could be described by Categorical distribution and Bernoulli distribution, respectively.
Revised: Therefore, the generating process of the candidate labels in instance-dependent PLL could be decomposed into two stages, i.e., the generation of the correct label of the instance and the generation of the incorrect labels related to the instance, which could be described by Categorical distribution and Bernoulli distribution, respectively.

Original: Motivated by the above consideration, we propose a novel PLL method named I DGP , i.e., Instancedependent partial label learning via Decomposed Generation Process.
Revised: Motivated by the above consideration, we propose a novel PLL method named I DGP , i.e., Instancedependent partial label learning via Decompositional Generation Process.

Original: Before performing I DGP , the distribution of the correct label and the incorrect label in thecandidate label set of each training example should be modeled explicitly by decoupled probability distributions Categorical distribution and Bernoulli distribution.
Revised: Before performing I DGP , the distributions of the correct label and the incorrect label given the training example should be modeled explicitly by decoupled probability distributions Categorical distribution and Bernoulli distribution.

Original: Then we perform Maximum A Posterior (MAP) estimation on the PLL training dataset to deduce a risk minimizer.
Revised: Then we perform Maximum A Posterior (MAP) estimation on the PLL training dataset to deduce a risk minimizer.

Original: To optimize the risk minimizer, Dirichlet distribution and Beta distribution are leveraged to model the condition prior in it and estimate the parameters of Categorical distribution and Bernoulli distribution due to the conjugacy.
Revised: To optimize the risk minimizer, Dirichlet distribution and Beta distribution are leveraged to model the condition prior inside and estimate the parameters of Categorical distribution and Bernoulli distribution due to the conjugacy.

Original: Finally, we refine prior information by updating the parameters of the corresponding conjugate distributions iteratively to improve the performance of the predictive model in each epoch.
Revised: Finally, we refine prior information by updating the parameters of the corresponding conjugate distributions iteratively to improve the performance of the predictive model in each epoch.

Original: Our contributions can be summarized as follows
Revised: Our contributions can be summarized as follows:

Original: : • We for the first time explicitly model the generation process of candidate labels in instancedependent PLL.
Revised: • We for the first time explicitly model the generation process of candidate labels in instancedependent PLL.

Original: The entire generating process is decomposed into the generation of the correct label of the instance and the generation of the incorrect labels, which could be described by Categorical distribution and Bernoulli distribution, respectively.
Revised: The entire generating process is decomposed into the generation of the correct label of the instance and the generation of the incorrect labels, which could be described by Categorical distribution and Bernoulli distribution, respectively.

Original: • We optimize the models of Categorical distribution and Bernoulli distribution via  MAP technique, where the corresponding conjugate distributions,
Revised: • We optimize the models of Categorical distribution and Bernoulli distribution via the MAP technique, where the corresponding conjugate distributions,

Original: i
Revised: i

Original: e., Dirichlet distribution and Beta distribution are induced.
Revised: e., Dirichlet distribution and Beta distribution are induced.

Original: • We derive an estimation error bound of our approach, which demonstrates that the empirical risk minimizer would approximately converge to the optimal risk minimizer as the number of training data grows to infinity.
Revised: • We derive an estimation error bound of our approach, which demonstrates that the empirical risk minimizer would approximately converge to the optimal risk minimizer as the number of training data grows to infinity.

Original: Experiments on benchmark and real-world datasets validate the effectiveness of our proposed method.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: In this section, we briefly review the literature for PLL from two aspects, i.e., traditional PLL and deep PLL.
Revised: In this section, we briefly review the literature for PLL from two aspects, i.e., traditional PLL and deep PLL.

Original: The former absorbs many classical machine learning techniques and usually utilizes linear models while the latter embraces deep learning and builds upon deep neural networks.
Revised: The former absorbs many classical machine learning techniques and usually utilizes linear models while the latter embraces deep learning and builds upon deep neural networks.

Original: We focus on the underlying assumptions on the generation of candidate labels behind part of them.
Revised: We focus on the underlying assumptions on the generation of candidate labels behind part of them.

Original: Traditional PLL usually aims to disambiguate the candidate label set with average-based or identification-based approaches.
Revised: Traditional PLL usually uses average-based or identification-based approaches to disambiguate the candidate label set.

Original: Average-based approaches treat each candidate label of the instance equally Hüllermeier & Beringer (2006);
Revised: Average-based approaches treat each candidate label of the instance equally Hüllermeier & Beringer (2006);

Original: Cour et al.
Revised: Cour et al.

Original: ; Zhang & Yu (2015).
Revised: ; Zhang & Yu (2015).

Original: Typically, 
Revised: Typically, Hüllermeier &

Original: Hüllermeier & Beringer (2006); Zhang & Yu (2015) apply  K-nearest neighbour technique and predict a new instance by voting.
Revised: Beringer (2006); Zhang & Yu (2015) apply the K-nearest neighbor technique to predict a new instance through voting.

Original: Identification-based approaches  constantly try to identify the possible correct label explicitly or implicitlyto alleviate the label ambiguity during the training phase Jin &
Revised: Identification-based approaches are constantly trying to identify the possible correct label, either explicitly or implicitly during the training phase, in order to reduce label ambiguity Jin &

Original: Ghahramani (2002); Nguyen & Caruana (2008); Liu & Dietterich (2012); Chen et al.
Revised: Ghahramani (2002); Nguyen & Caruana (2008); Liu & Dietterich (2012); Chen et al.

Original: ; Yu &
Revised: ; Yu &

Original: Zhang (2016).
Revised: Zhang (2016).

Original: For instance, Nguyen & Caruana (2008); Yu & Zhang (2016) formulate the objective function by treating the correct label as a latent variable via the maximum margin criterion.
Revised: For instance, Nguyen & Caruana (2008); Yu & Zhang (2016) formulate their objective functions by treating the correct label as a latent variable via the maximum margin criterion.

Original: Zhang et al.
Revised: Zhang et al.

Original: ; Feng & An (2018); Wang et al.
Revised: ; Feng & An (2018); Wang et al.

Original: ; Xu et al. (2019) iteratively update the confidence of each candidate label or the label distribution via leveraging the topological information in the feature space.
Revised: ; Xu et al. (2019b) use topological information in the feature space to iteratively update the confidence of each candidate label or the label distribution.

Original: Note that most traditional PLL methods do not take thegeneration process of  candidate labels into consideration.
Revised: It should be noted that most traditional PLL methods ignore the process of generating candidate labels.

Original: Liu & Dietterich (2012) propose the Logistic Stick-Breaking Conditional Multinomial Model  to depict the generation process butsimply assume the candidate labels  to be instance-indpendent.
Revised: The Logistic Stick-Breaking Conditional Multinomial Model is proposed by Liu & Dietterich (2012) to depict the generation process, but the candidate labels are assumed to be instance-independent.

Original: Deep PLL has  been studiedrecently and advanced the practical application of PLL, where the PLL approaches are not limited to linear models and low-efficiency optimization.
Revised: Deep PLL has recently been studied and advanced the practical application of PLL, where the PLL approaches are not restricted to linear models and low-efficiency optimization.

Original: Yao et al.
Revised: Yao et al.

Original: (2020a) pioneer the use of deep convolutional neural networks and employ a regularization term of uncertainty and a temporal-ensembling term to train the deep model.
Revised: (2020a) pioneer the use of deep convolutional neural networks and employ a regularization term of uncertainty and a temporal-ensembling term to train the deep model.

Original: Lv et al.
Revised: Lv et al.

Original: (2020) propose a progressive identification method that makes it possible for PLL to be compatible with arbitrary models and optimizers and performs impressively on image classification benchmarks for the first time.
Revised: (2020) propose a progressive identification method that allows PLL to be compatible with arbitrary models and optimizers while also performing impressively on image classification benchmarks for the first time.

Original: Yao et al.
Revised: Yao et al.

Original: (2020b) introduce a network-cooperation mechanism in deep PLL which trains two networks simultaneously to reduce their  disambiguation errors respectively.
Revised: (2020b) introduce a network-cooperation mechanism in deep PLL which trains two networks simultaneously to reduce their respective disambiguation errors.

Original: Feng et al.
Revised: Feng et al.

Original: (2020) derive a riskconsistent estimator via employing the importance reweighting strategy and a classifier-consistent estimator via utilizing the transition matrix for PLL.
Revised: (2020) employ the importance reweighting strategy to derive a risk-consistent estimator and the transition matrix for PLL to derive a classifier-consistent estimator.

Original: Wen et al.
Revised: Wen et al.

Original: (2021) introduce a leverage parameter that weights losses on partial labels and non-partial labels.
Revised: (2021) introduce a leverage parameter that weights losses on partial labels and non-partial labels.

Original: Xu et al.
Revised: Xu et al.

Original: (2021) apply variational inference to iteratively recover label distribution for each instance.
Revised: (2021) apply variational label enhancement Xu et al. (2020; 2019a) to iteratively recover label distribution for each instance.

Revised: Lv et al.

Revised: (2021) for the first time propose unreliable PLL, a realistic and challenge setting.

Revised: Zhang et al.

Revised: (2021) discover class activation map could be used for disambiguation, and class activation value could be used to capture the learned representation information in a more general way.

Original: Lyu et al.
Revised: Lyu et al.

Original: (2022) transform the PLL problem into an “instance-label” matching selection problem.
Revised: (2022) transform the PLL problem into an “instance-label” matching selection problem.

Original: He et al.
Revised: He et al.

Original: (2022) utilize semantic label representations to enhance the label disambiguation.
Revised: (2022) utilize semantic label representations to enhance the label disambiguation.

Original: Wang et al.
Revised: Wang et al.

Original: (2022) build a contrastive learning framework for PLLto perform disambiguation.
Revised: (2022) create a contrastive learning framework for PLL disambiguation.

Original: Zhang et al.

Original: (2021) discover class activation map could be utilized for disambiguation and further purpose class activation value to capture the learned representation information in a more general way.

Original: Wu et al. (2022) employ manifold consistency regularization to preserve the manifold structureboth in  feature and label space.
Revised: To preserve the manifold structure in both feature and label space, Wu et al. (2022) employ manifold consistency regularization.

Original: From Feng et al.
Revised: From Feng et al.

Original: (2020), the generation process of candidate labels has been paid attention  in Deep
Revised: (2020), the generation process of candidate labels has been paid attention to in Deep

Original: PLL. Feng et al. (2020) make a naive assumption that the candidate label set is uniformly sampled for each instance.
Revised: PLL, making a naive assumption that the candidate label set is uniformly sampled for each instance.

Original: Wen et al.
Revised: Wen et al.

Original: (2021) extend the uniform case to the class-dependent one but remain instance-independent.
Revised: (2021) extend the uniform case to the class-dependent one but remain instance-independent.

Original: Xu et al.
Revised: Xu et al.

Original: (2021) for the first time consider PLL under the instance-dependent case, a more realistic but challenging setting.
Revised: (2021) for the first time consider PLL under the instance-dependent case, a more realistic but challenging setting.

Original: However, Xu et al.
Revised: However, Xu et al.

Original: (2021) do not explicitly model the generation process of instance-dependent candidate labels.
Revised: (2021) do not explicitly model the generation process of instance-dependent candidate labels.

Original: The generative model in it is employed only to generate label distribution.
Revised: The generative model in it is employed only to generate label distribution.

Original: In this paper, we not only consider the instance-dependent PLL but also explicitly model the generation process of instance-dependent candidate labels via decoupled probability distributions.
Revised: In this paper, we not only consider the instance-dependent PLL but also explicitly model the generation process of instance-dependent candidate labels via decoupled probability distributions.

Original:  P ROPOSED M ETHOD
Revised:  P ROPOSED M ETHOD

Original: First of all, we introduce  necessary notations for our approach.
Revised: First of all, we introduce some necessary notations for our approach.

Original: In PLL,the labeling information of every instance in the training dataset is corrupted from one correct label into  candidate label set, which contains the correct label andcandidate incorrect  labels but is not a whole label set.
Revised: In PLL, labeling information of every instance in the training dataset is corrupted from a correct label into a candidate label set, which contains the correct label and incorrect candidate labels but is not a complete label set.

Original: Let X ⊆
Revised: Let X ⊆

Original: R q be the q -dimensional feature space of the instances and Y = { 1 , 2 , . .
Revised: R q be the q -dimensional feature space of the instances and Y = { 1 , 2 , . .

Original: , c } be the label space with c class labels.
Revised: , c } be the label space with c class labels.

Original: Then a PLL training data set can be formulated as D = { ( x i , S i ) | 1 ≤ i ≤ n } , where x i ∈
Revised: Then a PLL training data set can be formulated as D = { ( x i , S i ) | 1 ≤ i ≤ n } , where x i ∈

Original: X denotes the i -th q -dimensional feature vector and
Revised: X denotes the i -th q -dimensional feature vector and

Original: S i ∈ C
Revised: S i ∈ C

Original: ( C ={ C | 2 Y \ ∅} ) is the candidate label set annotated for x i .
Revised: = 2 Y \ {∅ , Y} is the candidate label set annotated for x i .

Original: S i consists of the correct label y i ∈ Y and thecandidate incorrect  label set
Revised: S i consists of the correct label y i ∈ Y and the incorrect candidate label set

Original: S y
Revised: S y

Original: i
Revised: i

Original: i =
Revised: i =

Original: S i \ { y i } .
Revised: S i \ { y i } .

Original: We aim to find a multi-class classifier f : X (cid:55)→ Y according to D .
Revised: We aim to find a multi-class classifier f : X (cid:55)→ Y according to D .

Original: 3. O VERVIEW
Revised: 3. O VERVIEW

Original: To deal withthe instance-dependent PLL problem, we introduce the explicit generation model of instance-dependent candidate labels, which decouples the distribution of the correct andcandidate incorrect  labels.
Revised: To deal with instance-dependent PLL, we introduce the explicit generation model of instancedependent candidate labels, which decouples the distribution of the correct and incorrect candidate labels.

Original: Then Category distribution and Bernoulli distribution are leveraged to depict them respectively with Dirichlet distribution and Beta distribution as their prior distributions.
Revised: Then Category distribution and Bernoulli distribution are leveraged to depict them respectively with Dirichlet distribution and Beta distribution as their prior distributions.

Original: Based on the probabilistic generation model, we simply deduce the log-likelihood loss function for optimization and then move forward to  MAP optimization problem in consideration of the prior distributions 
Revised: Based on the probabilistic generation model, we simply deduce the log-likelihood loss function for optimization and then move forward to the MAP optimization problem in consideration of the prior distributions Dirichlet and

Original: Dirichlet and Beta.
Revised: Beta.

Original: Finally, we further propose the algorithm I DGP , which keep and refine the prior information.
Revised: Finally, we further propose the algorithm I DGP , which keeps and refines the prior information.

Original: 3. G ENERATION M ODEL OF C ANDIDATE L ABELS
Revised: 3. G ENERATION M ODEL OF C ANDIDATE L ABELS

Original: Focusing on instance-dependent partial label learning, we propose a novel generation model of candidate labels, which explicitly models the generation process and decouples the distribution of the candidate labels into the distribution of the correct label andcandidate incorrect  labels respectively.
Revised: Focusing on instance-dependent partial label learning, we propose a novel generation model of candidate labels, which explicitly models the generation process and decouples the distribution of the candidate labels into the distribution of the correct label and incorrect candidate labels respectively.

Original: To form a candidate label set, we suppose the correct labelemerges and isselected first  according to a posterior distribution about the instance first.
Revised: To form a candidate label set, we suppose the correct label is first selected according to a posterior distribution dependent on the instance.

Original: And then thecandidate incorrect  labels related to the feature of instance emerge to disturb the annotator and are sampled from another posterior distribution dependent on the instance.
Revised: And then the incorrect candidate labels related to the feature of instance emerge to disturb the annotator and are sampled from another posterior distribution dependent on the instance.

Original: The concrete generation model is demonstrated as follows.
Revised: The concrete generation model is demonstrated as follows.

Original: Given an instance x i , its candidate label set S i , consisting of the correct label y i and the incorrect candidate labels S
Revised: Given an instance x i , its candidate label set S i , consisting of the correct label y i and the incorrect candidate labels S

Original: y i
Revised: y i

Original: i , is drawn from a probability distribution with the following density:
Revised: i , is drawn from a probability distribution with the following density:

Original: [Equation]
Revised: [Equation]

Original: S i p [Equation] [Equation] Here, p ( S i | x i ) suggests our generation model is entirely dependent on the instance.
Revised: Here, p ( S i | x i ) suggests our generation model is entirely dependent on the instance.

Original: Our generation model builds upon the PLL assumption that the correct label is always in the candidate set, which allows for directly splitting S i into y i and S
Revised: Our generation model builds upon the PLL assumption that the correct label is always in the candidate set, which allows for directly splitting S i into y i and S

Original: y i
Revised: y i

Original: i .
Revised: i .

Original: For the given candidate label set S i , each label j ∈ S
Revised: For the given candidate label set S i , each label j ∈ S

Original: i have the possibility p ( y i =
Revised: i have the possibility p ( y i =

Original: j | x i ) sampled by the annotator as the correct label in the first stage of generation and  thecandidate incorrect  labels S ji arethen sampled with the possibility p ( S ji | x i ) in the second stage to form the entire candidate label set S i .
Revised: j | x i ) sampled by the annotator as the correct label in the first stage of generation, and then the incorrect candidate labels S ji are sampled with the possibility p ( S ji | x i ) in the second stage to form the entire candidate label set S i .

Original: The generation of the correct label and incorrect labels are conditioned on the instance x i .
Revised: The generation of the correct label and incorrect labels are conditioned on the instance x i .

Original: In this way, we decompose the generation process of the instance-dependent candidate labels.
Revised: In this way, we decompose the generation process of the instance-dependent candidate labels.

Original: After the decomposition of generation, the corresponding probability distributions are leveraged to depict p ( y i | x i ) and p ( S y
Revised: After the decomposition of generation, the corresponding probability distributions are required to depict p ( y i | x i ) and p ( S y

Original: i
Revised: i

Original: i | x i ) .
Revised: i | x i ) .

Original: We assume that the correct label y i of each instance x i is drawn from a Categorical distribution with the parameters θ i , where θ i
Revised: We assume that the correct label y i of each instance x i is drawn from a Categorical distribution with the parameters θ i , where θ i

Original: = [ θ 1 i , θ 2 i , . . .
Revised: = [ θ 1 i , θ 2 i , . . .

Original: , θ ci ] ∈ [0 , 1] c is a c -dimension vector with the constraint (cid:80) cj
Revised: , θ ci ] ∈ [0 , 1] c is a c -dimension vector with the constraint (cid:80) cj

Original: =

Revised: =

Original: θ ji = 1 , i.e.,
Revised: θ ji = 1 , i.e.,

Original: [Equation]
Revised: [Equation]

Original: where  l i =
Revised: where the vector l i =

Original: [ l 1 i , l 2 i . . .
Revised: [ l 1 i , l 2 i . . .

Original: , l ci ] ∈ { 0 , 1 } c  to represents whether the j -th label is the correct label
Revised: , l ci ] ∈ { 0 , 1 } c is utilized to represent whether the j -th label is the correct label

Original: y i , i.e., l ji = 

Revised: y i , i.e., l ji =

Original: if j = y i , otherwise l ji = 0 .
Revised: if j = y i , otherwise l ji = 0 .

Original: Besides, for thecandidate incorrect  label set S
Revised: Besides, for the incorrect candidate label set S

Original: y i
Revised: y i

Original: i , it can also be denoted by a logical label vector s y
Revised: i , it can also be denoted by a logical label vector s y

Original: i i =
Revised: i i =

Original: [ s 1 i , s 2 i . . .
Revised: [ s 1 i , s 2 i . . .

Original: , s ci ] ∈ { 0 , 1 } c to represent whether the j -th label is thecandidate incorrect  label, i.e., s ji = 1 if j ∈ S
Revised: , s ci ] ∈ { 0 , 1 } c to represent whether the j -th label is the incorrect candidate label, i.e., s ji = 1 if j ∈ S

Original: y i
Revised: y i

Original: i , s ji = 

Revised: i , s ji =

Original: if j / ∈ S
Revised: if j / ∈ S

Original: y i
Revised: y i

Original: i .
Revised: i .

Original: In order to describe p ( S y i i | x i ) with a probabilistic model, we further decouple thecandidate incorrect  labels by assuming that the distribution of each variable s ji is independent from each other, i.e, p ( s y
Revised: In order to describe p ( S y i i | x i ) with a probabilistic model, we further decouple the incorrect candidate labels by assuming that the distribution of each variable s ji is independent from each other, i.e, p ( s y

Original: i i | x i ) =
Revised: i i | x i ) =

Original: (cid:81) j ∈ S yii p ( s ji | x i ) (cid:81) j/ ∈ S yii (1 − p ( s ji | x i )) and thecandidate incorrect  label set S
Revised: (cid:81) j ∈ S yii p ( s ji | x i ) (cid:81) j/ ∈ S yii (1 − p ( s ji | x i )) and the incorrect candidate label set S

Original: y i
Revised: y i

Original: i is drawn from a multivariate Bernoulli distribution with the parameters
Revised: i is drawn from a multivariate Bernoulli distribution with the parameters

Original: z i , where z i =
Revised: z i , where z i =

Original: [ z 1 i , z 2 i , . . .
Revised: [ z 1 i , z 2 i , . . .

Original: , z ci ] ∈ [0 , 1] c is a c-dimension vector, i.e.,
Revised: , z ci ] ∈ [0 , 1] c is a c-dimension vector, i.e.,

Original: [Equation]
Revised: [Equation]

Original: For the latter estimation of θ i in MAP, we introduce Dirichlet distribution, the conjugate distribution of the Categorical distribution, as its conditional prior, i.e., p ( θ i | x i ) =
Revised: For the latter estimation of θ i in MAP, we introduce Dirichlet distribution, the conjugate distribution of the Categorical distribution, as its conditional prior, i.e., p ( θ i | x i ) =

Original: Dir ( θ i | λ i ) , where λ i =
Revised: Dir ( θ i | λ i ) , where λ i =

Original: [ λ 1 i , λ 2 i , . . .
Revised: [ λ 1 i , λ 2 i , . . .

Original: , λ ci ] ⊤ ( λ ji > 0) is a c -dimensional vector as the output of our main branch inference model f parameterized by Θ for an instance x
Revised: , λ ci ] ⊤ ( λ ji > 0) is a c -dimensional vector as the output of our main branch inference model f parameterized by Θ for an instance x

Original: i , i.e, λ i = a · exp ( f ( x i ; Θ ) /γ )
Revised: i , i.e, λ i = a · exp ( f ( x i ; Θ ) /γ )

Original: + b , where a , b and γ ( a ≥ 1 , b ≥ 0 , γ > 0 ) are used to resolve the scale ambiguity.
Revised: + b , where a , b and γ ( a ≥ 1 , b ≥ 0 , γ > 0 ) are used to resolve the scale ambiguity.

Original: Note that the main branch inference model f is leveraged as the final predictive model to accomplish the PLL target X (cid:55)→ Y by using (cid:98)
Revised: Note that the main branch inference model f is leveraged as the final predictive model to accomplish the PLL target X (cid:55)→ Y by using (cid:98)

Original: y
Revised: y

Original: i = arg max j (cid:98) θ ji , where (cid:98) θ ji will be estimated by λ i .
Revised: i = arg max j (cid:98) θ ji , where (cid:98) θ ji will be estimated by λ i with the conjugacy later.

Original: Likewise, to estimate z i latter in MAP, we introduce Beta distribution, the conjugate distribution of
Revised: Likewise, to estimate z i latter in MAP, we introduce Beta distribution, the conjugate distribution of

Original: Bernoulli distribution, as the conditional prior parameterized by α i =
Revised: Bernoulli distribution, as the conditional prior parameterized by α i =

Original: [ α 1 i , α 2 i , . .
Revised: [ α 1 i , α 2 i , . .

Original: α ci ] ⊤ and β i = [ β 1 i , β 2 i , . . .
Revised: α ci ] ⊤ and β i = [ β 1 i , β 2 i , . . .

Original: , β ci ] ⊤ ( α ji > 0 , β ji > 0) , i.e., p ( z i | x i ) =
Revised: , β ci ] ⊤ ( α ji > 0 , β ji > 0) , i.e., p ( z i | x i ) =

Original: (cid:81) cj
Revised: (cid:81) cj

Original: =1 Beta ( z ji | α ji , β ji ) .
Revised: =1 Beta ( z ji | α ji , β ji ) .

Original: We employ an auxiliary branch model g parameterized by Ω to output Λ ⊤ i
Revised: We employ an auxiliary branch model g parameterized by Ω to output Λ ⊤ i

Original: =
Revised: =

Original: [ α ⊤ i , β ⊤ i ] ,i.e, Λ i = a · exp ( g ( x i ; Ω ) /γ )
Revised: [ α ⊤ i , β ⊤ i ] ,i.e, Λ i = a · exp ( g ( x i ; Ω ) /γ )

Original: + b .
Revised: + b .

Original: The same model and constants to scale Λ i are implemented to simplify our approaches.
Revised: The same model and constants to scale Λ i are implemented to simplify our approaches.

Original: It should be noted that the predictive model f ( x i ; Θ ) or auxiliary model g ( x i ; Ω ) can be any deep neural network once its output satisfies the corresponding constraint.
Revised: It should be noted that the predictive model f ( x i ; Θ ) or auxiliary model g ( x i ; Ω ) can be any deep neural network once its output satisfies the corresponding constraint.

Original: 3. O PTIMIZATION U SING M AXIMUM A P OSTERIOR
Revised: 3. O PTIMIZATION U SING M AXIMUM A P OSTERIOR

Original: Based on the generation model of candidate labels denoted by Eq.(1), Eq.(2) and Eq.(3), by using the technique of Maximum Likelihood (ML) estimation we can immediately induce the log-likelihood loss function for the PLL training data set to be optimized:
Revised: Based on the generation model of candidate labels denoted by Eq.(1), Eq.(2) and Eq.(3), by using the technique of Maximum Likelihood (ML) estimation we can immediately induce the log-likelihood loss function for the PLL training data set to be optimized:

Original: [Equation]
Revised: [Equation]

Original: The Eq.(4)clearly demonstrates how the distribution of the correct label interacts with that of thecandidate incorrect  labels after decoupling the generation.
Revised: The Eq.(4) demonstrates how the distribution of the correct label interacts with that of the incorrect candidate labels after decoupling the generation.

Original: In the backpropagation of the training process, they provide weight coefficients for each other.
Revised: In the backpropagation of the training process, they provide weight coefficients for each other.

Original: In order to bring in prior information for PLL, we further introduce I DGP , which performs MAP in the training data set.
Revised: In order to bring in prior information for PLL, we further introduce I DGP , which performs MAP in the training data set.

Original: In I DGP , we are more concerned about maximizing the joint distribution p ( θ , z | x , S ) .
Revised: In I DGP , we are more concerned about maximizing the joint distribution p ( θ , z | x , S ) .

Original: This will lead to the following optimization problem L MAP =
Revised: This will lead to the following optimization problem L MAP =

Original: L ML + L reg , where
Revised: L ML + L reg , where

Original: [Equation]
Revised: [Equation]

Original: Compared to ML, our I DGP framework provides a natural way of leveraging prior information via optimizing the extra condition prior illustrated by the Eq.(5) in the training process, which is significant for PLL due to the implicit supervision information.
Revised: Compared to ML, our I DGP framework provides a natural way of leveraging prior information via optimizing the extra condition prior illustrated by the Eq.(5) in the training process, which is significant for PLL due to the implicit supervision information.

Original: Combined with the prior distribution
Revised: Combined with the prior distribution

Original: Dirichlet and Beta, L reg can be analytically calculated as follows:
Revised: Dirichlet and Beta, L reg can be analytically calculated as follows:

Original: [Equation]
Revised: [Equation]

Original: The mathematical derivations of Eq.(4) and Eq.(6) is provided in Appendix A.1.
Revised: The mathematical derivations of Eq.(4) and Eq.(6) are provided in Appendix A.1.

Original: By combining the
Revised: By combining the

Original: Eq.(4) and the Eq.(6), the MAP optimization problem L can be calculated as follows:
Revised: Eq.(4) and the Eq.(6), the MAP optimization problem L can be calculated as follows:

Original: [Equation]
Revised: [Equation]

Original: L can also accommodate the uniform case, which can be seen in Appendix A.2.
Revised: L can also accommodate the uniform case, which can be seen in Appendix A.2.

Original: Then, due to the conjugacy of Dirichlet and Categorical distribution Minka (2000), we can estimate θ ji by using
Revised: Then, due to the conjugacy of Dirichlet and Categorical distribution Minka (2000), we can estimate θ ji by using

Original: [Equation]
Revised: [Equation]

Original: where o ji denotes the number of occurrences of label j for x i , i.e., o ji = 1 if j ∈ S
Revised: where o ji denotes the number of occurrences on the label j for x i , i.e., o ji = 1 if j ∈ S

Original: i , otherwise o ji = 0 .
Revised: i , otherwise o ji = 0 .

Original: Similarly, we can leverage the conjugacy of Beta and Bernoulli distribution to estimate z ji ,i.e,
Revised: Similarly, we can leverage the conjugacy of Beta and Bernoulli distribution to estimate z ji ,i.e,

Original: [Equation]
Revised: [Equation]

Original: The mathematical derivations of Eq.(8) and Eq.(9) is provided in Appendix A.3.
Revised: The mathematical derivations of Eq.(8) and Eq.(9) are provided in Appendix A.3.

Original: As it is shown in the Eq.(6), I DGP provides prior information that can be included in the parameters λ, α and β for the predictive model f ( x ; Θ ) and the auxiliary model g ( x ; Ω ) by transforming them into the weights exerted on the corresponding label.
Revised: As it is shown in the Eq.(6), I DGP provides prior information that can be included in the parameters λ, α and β for the predictive model f ( x ; Θ ) and the auxiliary model g ( x ; Ω ) by transforming them into the weights exerted on the corresponding label.

Original: The prior information comes from the memorization effects Han et al.
Revised: The prior information comes from the memorization effects Han et al.

Original: (2020) of the neural network.
Revised: (2020) of the neural network.

Original: It makes the neural network always likely to recognize and remember the correct label in priority, leading to a kind of initial disambiguation at the beginning epochs.
Revised: It makes the neural network always likely to recognize and remember the correct label in priority, leading to a kind of initial disambiguation at the beginning epochs.

Original: Hence, to keep fine prior information, we replace λ i with (cid:98) λ i in the following way:
Revised: Hence, to keep fine prior information, we replace λ i with (cid:98) λ i in the following way:

Original: [Equation]
Revised: [Equation]

Original: where ( · ) ( t ) denotes the vector or scalar ( · ) at the t epoch, r ( ≤ t ) denote the beginning epoch we set to reserve the fine prior information for f ( x ; Θ ) , m ∈ (0 , 1) is a positive constant used to replenish the present information λ j ( t )
Revised: where ( · ) ( t ) denotes the vector or scalar ( · ) at the t epoch, r ( ≤ t ) denotes the beginning epoch we set to reserve the fine prior information for f ( x ; Θ ) , m ∈ (0 , 1) is a positive constant used to replenish the present information λ j ( t )

Original: i with the prior information λ j ( r ) i  and ϵ is a minor value, which means that the weight excerted on each incorrect label is negligible.
Revised: i with the prior information λ j ( r ) i , and ϵ is a minor value, which means that the weight excerted on each incorrect label is negligible.

Original: In the similar way, we replace α ( t ) i and β ( t ) i with (cid:98) α ( t ) i and (cid:98) β ( t ) i , i.e, the vectors (cid:98) α
Revised: In the similar way, we replace α ( t ) i and β ( t ) i with (cid:98) α ( t ) i and (cid:98) β ( t ) i , i.e, the vectors (cid:98) α

Original: ( t )
Revised: ( t )

Original: i and (cid:98) β ( t ) i will be calculated by
Revised: i and (cid:98) β ( t ) i will be calculated by

Original: [Equation]
Revised: [Equation]

Original: where q denote the beginning epoch we set to reserve the fine prior information for the auxiliary model g ( x ; Ω ) , and d ∈ (0 , 1) is also a positive constant used to provide the present model information α j ( t ) i , β j ( t ) i with the prior information α j ( q ) i , β j ( q ) i .
Revised: where q denotes the beginning epoch we set to reserve the fine prior information for the auxiliary model g ( x ; Ω ) , and d ∈ (0 , 1) is also a positive constant used to provide the present model information α j ( t ) i , β j ( t ) i with the prior information α j ( q ) i , β j ( q ) i .

Original: Before the epoch r , (cid:98) λ
Revised: Before the epoch r , (cid:98) λ

Original: j ( t ) i =
Revised: j ( t ) i =

Original: λ j ( t ) i if j ∈ S i , otherwise 1 + ϵ .
Revised: λ j ( t ) i if j ∈ S i , otherwise 1 + ϵ .

Original: Before the epoch q , (cid:98) α j ( t ) i = α j ( t ) i and (cid:98) β j ( t ) i =
Revised: Before the epoch q , (cid:98) α j ( t ) i = α j ( t ) i and (cid:98) β j ( t ) i =

Original: β j ( t ) i .
Revised: β j ( t ) i .

Original: In the above way, we refine the prior information epoch by epoch.
Revised: In the above way, we refine the prior information epoch by epoch.

Original: The optimization of the two model f ( x ; Θ ) and g ( x ; Ω ) at the epoch t for an instance x is as follows.
Revised: The optimization of the two model f ( x ; Θ ) and g ( x ; Ω ) at the epoch t for an instance x is as follows.

Original: First, f ( x ; Θ ) outputs λ while g ( x ; Ω ) outputs α and β , with which, according to Eq.(8) and Eq.(9), we replace θ and z with the estimation (cid:98) θ and (cid:98) z in Eq.(7).
Revised: First, f ( x ; Θ ) outputs λ while g ( x ; Ω ) outputs α and β , based on which, according to Eq.(8) and Eq.(9) , we replace θ and z with the estimation (cid:98) θ and (cid:98) z in Eq.(7).

Original: Next, to introduce prior information, we use (cid:98) λ , (cid:98) α and (cid:98) β in Eq.(10) and Eq.(11) to replace λ , α and β in Eq.(7).
Revised: Next, to introduce prior information, we use (cid:98) λ , (cid:98) α and (cid:98) β in Eq.(10) and Eq.(11) to replace λ , α and β in Eq.(7).

Original: Note that (cid:98) θ and (cid:98) z are variables, of which we calculate the gradient to perform backpropagation, while (cid:98) λ , (cid:98) α and (cid:98) β are constants.
Revised: Note that (cid:98) θ and (cid:98) z are variables, of which we calculate the gradient to perform backpropagation, while (cid:98) λ , (cid:98) α and (cid:98) β are constants.

Original: Finally, we update the predictive model f
Revised: Finally, we update the predictive model f

Original: ( x ; Θ ) and the auxiliary model g ( x ; Ω ) by fixing one and updating the other.
Revised: ( x ; Θ ) and the auxiliary model g ( x ; Ω ) by fixing one and updating the other.

Original: The whole algorithmic description of the I DGP is shown in Algorithm 1.
Revised: The whole algorithmic description of the I DGP is shown in Algorithm 1.

Original: After implementing I DGP in the PLL training dataset, we can use the output λ of the main branch inference model f ( x ; Θ ) to calculate the predict results ˆ θ as the label confidence in the test dataset.
Revised: After implementing I DGP in the PLL training dataset, we can use the output λ of the main branch inference model f ( x ; Θ ) to calculate the predict results (cid:98) θ as the label confidence in the test dataset.

Original:  T HEORETICAL A NALYSIS
Revised:  T HEORETICAL A NALYSIS

Original: In this section, we pay attention to the estimation error bound of the predictive model f ( x ; Θ ) .
Revised: In this section, we pay attention to the estimation error bound of the predictive model f ( x ; Θ ) .

Original: According to Eq.(7), the empirical risk estimator for the predictive model f ( x ; Θ ) is denoted by (cid:98) R ( f )
Revised: According to Eq.(7), the empirical risk estimator for the predictive model f ( x ; Θ ) is denoted by (cid:98) R ( f )

Original: = 1 n (cid:80) ni =1 L MAP ( f ( x i ) , S i ) and for further analysis, we give an upper bound of
Revised: = 1 n (cid:80) ni =1 L MAP ( f ( x i ) , S i ) and for further analysis, we give an upper bound of

Original: L MAP ( f ( x i ) , S i ) as follows:
Revised: L MAP ( f ( x i ) , S i ) as follows:

Original: [Equation]
Revised: [Equation]

Original:   , (12) where e j denotes the standard canonical vector in R c , ℓ denotes the cross-entropy function, w ji = λji − 1+ 1 | S i | if j ∈ S i , otherwise w ji = λ
Revised: (12) where e j denotes the standard canonical vector in R c , ℓ denotes the cross-entropy function, w ji = λ

Original: ji −  1 , and K i = log | S i | + 1 | S i | (cid:80) j ∈ S i log (cid:81)
Revised: ji − 1+ 1 | S i | if j ∈ S i , otherwise w ji = λ ji − 1 , and K i = log | S i | + 1 | S i | (cid:80) j ∈ S i log (cid:81)

Original: k ∈ S ji z ki (cid:81)
Revised: k ∈ S ji z ki (cid:81)

Original: k/ ∈ S ji (1 − z ki )
Revised: k/ ∈ S ji (1 − z ki )

Original: + ( α ji − 1) log z ji
Revised: + ( α ji − 1) log z ji

Original: + ( β ji − 1) log (1 − z ji ) .
Revised: + ( β ji − 1) log (1 − z ji ) .

Original: We denote K = max { K 1 , K 2 , ..., K n } and scale w ji to [0 , ρ ] during the training process.
Revised: We denote K = max { K 1 , K 2 , ..., K n } and scale w ji to [0 , ρ ] during the training process.

Original: The detailed induction of Eq.(12) can be seen in Appendix A.4.
Revised: The detailed induction of Eq.(12) can be seen in Appendix A.4.

Original: Then to formulate the estimation error bound of f , we give the following definition and lemmas.
Revised: Then to formulate the estimation error bound of f , we give the following definition and lemmas.

Original: Definition 1.
Revised: Definition 1.

Original: Let x 1 , x 2 , . . .
Revised: Let x 1 , x 2 , . . .

Original: x n ∈ X be n i.i.d random variables drawn from a probability distribution and σ 1 , σ 2 , . . .
Revised: x n ∈ X be n i.i.d random variables drawn from a probability distribution, σ 1 , σ 2 , . . .

Original: , σ n ∈ {− 1 , +1 } be Rademacher variables with even probabilities, 
Revised: , σ n ∈ {− 1 , +1 } be Rademacher variables with even probabilities, and

Original: H =
Revised: H =

Original: { h : X (cid:55)→ R } be a class of measurable functions.
Revised: { h : X (cid:55)→ R } be a class of measurable functions.

Original: Then the expected Rademacher complexity of
Revised: Then the expected Rademacher complexity of

Original: H is defined as
Revised: H is defined as

Original: [Equation]
Revised: [Equation]

Original:  According Definition 1, given the function space G = { ( x , S ) (cid:55)→ L MAP ( f ( x ) , S ) | f ∈ F} , the expected Rademacher complexity of G can be defined as follows:
Revised: [Table] [Equation] According Definition 1, given the function space G = { ( x , S ) (cid:55)→ L MAP ( f ( x ) , S ) | f ∈ F} , the expected Rademacher complexity of G can be defined as follows:

Original: [Equation]
Revised: [Equation]

Original: We pre-limit both the main model f and the auxiliary model g by clamping their output to [ − A, A ] , and the loss L MAP can be bounded, though it would not have extended to infinity in practice.
Revised: We pre-limit both the main model f and the auxiliary model g by clamping their output to [ − A, A ] , and the loss L MAP can be bounded, though it would not have extended to infinity in practice.

Original: Lemma 1.
Revised: Lemma 1.

Original: Suppose the loss function L MAP ( f ( x ) , S ) is bounded by M ,i.e., M = sup x ∈X ,S ∈C ,f ∈F L MAP ( f ( x ) , S ) , then for any ξ > 0 , with probability at least 1 − ξ ,
Revised: Suppose the loss function L MAP ( f ( x ) , S ) is bounded by M ,i.e., M = sup x ∈X ,S ∈C ,f ∈F L MAP ( f ( x ) , S ) , then for any ξ > 0 , with probability at least 1 − ξ ,

Original: [Equation]
Revised: [Equation]

Original: Lemma 2.
Revised: Lemma 2.

Original: Assume the loss function ℓ ( f ( x ) , e ι ) is L -Lipschitz with respect to f ( x ) (0 < L < ∞ ) for all ι ∈ Y .
Revised: Assume the loss function ℓ ( f ( x ) , e ι ) is L -Lipschitz with respect to f ( x ) (0 < L < ∞ ) for all ι ∈ Y .

Original: Let H ι
Revised: Let H ι

Original: = { h : x
Revised: = { h : x

Original: (cid:55)→ f ι ( x ) | f ∈ F} and R n ( H ι ) =
Revised: (cid:55)→ f ι ( x ) | f ∈ F} and R n ( H ι ) =

Original: E x ,σ [Equation] h ∈H [Equation]
Revised: E x ,σ (cid:2) sup h ∈H ι 1 n (cid:80) ni

Original: 1 n [Equation] ni =1 h ( x i ) (cid:3) , then the following inequality holds:
Revised: =1 h ( x i ) (cid:3) , then the following inequality holds:

Original: [Equation]
Revised: [Equation]

Original: The proof of Lemma 1 and 2 is provided in Appendix A.5 and A.6.
Revised: The proof of Lemma 1 and 2 is provided in Appendix A.5 and A.6.

Original: Based on Lemma 1 and 2, we induce an estimation error bound for our I DGP method.
Revised: Based on Lemma 1 and 2, we induce an estimation error bound for our I DGP method.

Original: Let (cid:98) f = arg min f ∈F (cid:98) R ( f ) be the empirical risk minimizer and f ⋆ = arg
Revised: Let (cid:98) f = arg min f ∈F (cid:98) R ( f ) be the empirical risk minimizer and f ⋆ = arg

Original: min f ∈F R ( f ) be the true minimizer.
Revised: min f ∈F R ( f ) be the true minimizer.

Original: The function space H ι for the label ι ∈ Y is defined as { h : x (cid:55)→ f y ( x ) | f ∈ F} .
Revised: The function space H ι for the label ι ∈ Y is defined as { h : x (cid:55)→ f y ( x ) | f ∈ F} , and R n ( H ι ) is defined as the expected

Original: Let R n ( H ι ) be the expected Rademacher complexity of H ι with sample size n , then we have the following theorem.
Revised: Rademacher complexity of H ι with sample size n . Then we have the following theorem.

Original: Theorem 1.
Revised: Theorem 1.

Original: Assume the loss function ℓ ( f ( x ) , e ι ) is L -Lipschitz with respect to f ( x ) (0 < L < ∞ ) for all ι ∈ Y and L MAP ( f ( x ) , S ) is bounded by M , i.e., M = sup x ∈X ,S ∈C ,f ∈F L  ( f ( x ) , S ) ,.
Revised: Assume the loss function ℓ ( f ( x ) , e ι ) is L -Lipschitz with respect to f ( x ) (0 < L < ∞ ) for all ι ∈ Y and L MAP ( f ( x ) , S ) is bounded by M , i.e., M = sup x ∈X ,S ∈C ,f ∈F L MAP ( f ( x ) , S ) ,.

Original: Then, for any ξ > 0 , with probability at least 1 − ξ ,
Revised: Then, for any ξ > 0 , with probability at least 1 − ξ ,

Original: [Equation]
Revised: [Equation]

Original: The proof is provided in Appendix A.7.
Revised: The proof can be found in Appendix A.7.

Original: Theorem 1 means that when n → ∞ the empirical risk minimizer f will converge to the optimal risk minimizer f ⋆ as to all parametric models with a bounded norm.
Revised: Theorem 1 means that as n → ∞ the empirical risk minimizer f will converge to the optimal risk minimizer f ⋆ for models with a bounded norm.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we validate the effectiveness of our proposed I DGP by performing it on  benchmarkdatasets and real-world datasets and comparing its results against DNN-based PLL algorithms.
Revised: In this section, we validate the effectiveness of our proposed I DGP by performing it on augmented benchmark and real-world datasets and comparing its results against DNN-based PLL algorithms.

Original: Furthermore,  ablation study and sensitive analysis of parameters arealso conducted to explore I DGP .
Revised: Furthermore, the ablation study and sensitive analysis of parameters are conducted to explore I DGP .

Original: 5. DATASETS
Revised: [Table] D

Original:  We implement I DGP with compared DNN-based algorithms on five widely used benchmark datasets in deep learning, including MNIST LeCun et al.
Revised: [Table] We implement I DGP with compared DNN-based algorithms on five widely used benchmark datasets in deep learning, including MNIST LeCun et al.

Original: , Kuzushiji-MNIST Clanuwat et al.
Revised: , Kuzushiji-MNIST Clanuwat et al.

Original: Fashion-MINIST Xiao et al.
Revised: Fashion-MINIST Xiao et al.

Original: (2017), CIFAR-10 and CIFAR-100 Krizhevsky et al.
Revised: (2017), CIFAR-10 and CIFAR-100 Krizhevsky et al.

Original: Instance-dependent partial labels for these datasets are generated through the same strategy as Xu et al.
Revised: Instance-dependent partial labels for these datasets are generated through the same strategy as Xu et al.

Original: (2021), which for the first time consider instance-dependent PLL.
Revised: (2021), which for the first time consider instance-dependent PLL.

Original: Besides,  the comparing algorithms are also performed on five frequently used real-world datasets, which come from different practical application domains, including Lost Cour et al.
Revised: Besides, part of the comparing algorithms are also performed on five frequently used real-world datasets, which come from different practical application domains, including Lost Cour et al.

Original: BirdSong Briggs et al.
Revised: BirdSong Briggs et al.

Original: (2012), MSRCv2 Liu & Dietterich (2012), Soccer Player Zeng et al.
Revised: (2012), MSRCv2 Liu & Dietterich (2012), Soccer Player Zeng et al.

Original: (2013) and Yahoo!News Guillaumin et al.
Revised: (2013) and Yahoo!News Guillaumin et al.

Original: For benchmark datasets, we split 10% samples from the training datasets for validating.
Revised: For benchmark datasets, we split 10% samples from the training datasets for validating.

Original: For each real-world dataset, we run the methods with 80% / 10% / 10% train/validation/test split.
Revised: For each real-world dataset, we run the methods with 80% / 10% / 10% train/validation/test split.

Original: Then we run five trials on each datasets with different random seeds and report the mean accuracy and standard deviation of all comparing algorithms.[Table] [Table] [Table]
Revised: Then we run five trials on each datasets with different random seeds and report the mean accuracy and standard deviation of all comparing algorithms.

Original:  B ASELINES
Revised: 5. B ASELINES

Original: We compare I DGP with nine DNN-based methods:1) P LCR Wu et al.
Revised: We compare I DGP with eight DNN-based methods:1) P LCR Wu et al.

Original: (2022), a regularized training framework which is based on data augmentation and utilizes the manifold consistency regularization term to preserve the manifold structure both in feature space and label space.
Revised: (2022), a regularized training framework which is based on data augmentation and utilizes the manifold consistency regularization term to preserve the manifold structure both in feature space and label space.

Original: 2) P ICO [Table] et al.
Revised: 2) P ICO Wang et al.

Original: (2022), a contrastive learning framework which is based on data augmentation and performs label disambiguation based on the contrastive prototypes.
Revised: (2022), a contrastive learning framework which is based on data augmentation and performs label disambiguation based on the contrastive prototypes.

Original: 3) V ALEN Xu et al.
Revised: 3) V ALEN Xu et al.

Original: (2021), an instance-dependent
Revised: (2021), an instance-dependent

Original: PLL framework which guides the training process via the recovered latent label distributions.
Revised: PLL framework which guides the training process via the recovered latent label distributions.

Original: 4) C AVL
Revised: 4) C AVL

Original: Zhang et al.
Revised: Zhang et al.

Original: (2021), a discriminative approach which identifies correct labels from candidate labels by class activation value.
Revised: (2021), a discriminative approach which identifies correct labels from candidate labels by class activation value.

Original: L WS Wen et al.
Revised: L WS Wen et al.

Original: (2021), an identification-based method which introduce a leverage parameter to consider the trade-off between losses on candidate and non-candidate labels.
Revised: (2021), an identification-based method which introduces a leverage parameter to consider the trade-off between losses on candidate and non-candidate labels.

Original: RC Feng et al.
Revised: RC Feng et al.

Original: (2020), a risk-consistent PLL method which is induced by an importance reweighting strategy.
Revised: (2020), a risk-consistent PLL method which is induced by an importance reweighting strategy.

Original: 7) CC Feng et al.
Revised: 7) CC Feng et al.

Original: (2020), a classifier-consistent PLL method which leverages the transition matrix describing the probability of the candidate label set given a correct label.
Revised: (2020), a classifier-consistent PLL method which leverages the transition matrix describing the probability of the candidate label set given a correct label.

Original: 8) P RODEN Lv et al.
Revised: 8) P RODEN Lv et al.

Original: (2020), a self-training style algorithm which provides a framework to equip arbitrary stochastic optimizers and models in PLL.
Revised: (2020), a self-training style algorithm which provides a framework to equip arbitrary stochastic optimizers and models in PLL.

Original: Note that P LCR and P ICO will not be compared on the real-world datasets due to the requirement of data augmentation.
Revised: Note that P LCR and P ICO will not be compared on the real-world datasets due to the requirement of data augmentation.

Original: To ensure the comparisons as fair as possible, we employ the same network backbone, optimizer and data augmentation strategy for all the comparing methods.
Revised: To ensure the fairness, we employ the same network backbone, optimizer and data augmentation strategy for all the comparing methods.

Original: For MNIST , Kuzushiji-MNIST and Fashion-MNIST , we take LeNet-5 as their backbone.
Revised: For MNIST , Kuzushiji-MNIST and Fashion-MNIST , we take LeNet-5 as their backbone.

Original: For CIFAR-10 and CIFAR-100 , the network backbone is changed to ResNet-32 He et al.
Revised: For CIFAR-10 and CIFAR-100 , the network backbone is changed to ResNet-32 He et al.

Original: For all the real-world datasets, we simply adopt the linear model.
Revised: For all the real-world datasets, we simply adopt the linear model.

Original: The optimizer is stochastic gradient descent (SGD) Robbins & Monro (1951) with momentum 0 .
Revised: The optimizer is stochastic gradient descent (SGD) Robbins & Monro (1951) with momentum 0 .

Original: 9 and batch size 256 .
Revised: 9 and batch size 256 .

Original: The details of data augmentation strategy are shown in Appendix A.8.
Revised: The details of data augmentation strategy are shown in Appendix A.8.

Original: Besides, the learning rate is selected from { 10 − 4 , 10 − 3 , 10 − 2 }  and the weight decay are selected from { 10 − 5 , 10 − 4 , 10 − 3 , 10 − 2 } according to the performance on the validation.
Revised: Besides, the learning rate is selected from { 10 − 4 , 10 − 3 , 10 − 2 } , and the weight decay is selected from { 10 − 5 , 10 − 4 , 10 − 3 , 10 − 2 } according to the performance on the validation.

Original:  E XPERIMENTAL R ESULTS
Revised: 5. E XPERIMENTAL R ESULTS

Original: The performance of each DNN-based method on each corrupted benchmark dataset is summarized in
Revised: The performance of each DNN-based method on each corrupted benchmark dataset is summarized in

Original: Table 1, where the best results are highlighted in bold and • / ◦ indicates whether I DGP statistically  wins/loses to the comparing method on each dataset additionally (pairwise t-test at 0.05 significance level).
Revised: Table 1, where the best results are highlighted in bold and • / ◦ indicates whether I DGP statistically [Table] [Figure] wins/loses to the comparing method on each dataset additionally (pairwise t-test at 0.05 significance level).

Original: We can overall see that I DGP significantly outperforms all comparing approaches on all benchmark datasets (except on Kuzushiji-MNIST where V ALEN performs comparably against
Revised: We can overall see that I DGP significantly outperforms all comparing approaches on all benchmark datasets (except on Kuzushiji-MNIST where V ALEN performs comparably against

Original: I DGP ), and the improvements are particularly noticeable on Fashion-MNIST and CIFAR-10 .
Revised: I DGP ), and the improvements are particularly noticeable on Fashion-MNIST and CIFAR-10 .

Original: [Table] [Figure] Table 2 demonstrates the ability of I DGP to solve the PLL problem in real-world datasets.
Revised: Table 2 demonstrates the ability of I DGP to solve the PLL problem in real-world datasets.

Original: P LCR and P ICO are not compared on the real-world PLL datasets due to the inability of data augmentation to be employed on the extracted features from various domains.
Revised: P LCR and P ICO are not compared on the real-world PLL datasets due to the inability of data augmentation to be employed on the extracted features from various domains.

Original: We can find that our method has stronger competence than others in all datasets except Soccer Player where I DGP loses to RC but still ranks second.
Revised: We can find that our method has stronger competence than others in all datasets except Soccer Player where I DGP loses to RC but still ranks second.

Original: As for BirdSong , MSRCv2 and Yahoo!News , the performance of I DGP is significantly better than all other comparing algorithms.
Revised: As for BirdSong , MSRCv2 and Yahoo!News , the performance of I DGP is significantly better than all other comparing algorithms.

Original: And when it comes to Lost , our method is comparable to V ALEN , P RODEN and RC, while obviously better than the rest.
Revised: And when it comes to Lost , our method is comparable to V ALEN , P RODEN and RC, while obviously better than the rest.

Original: 5. F URTHER A NALYSIS
Revised: 5. F URTHER A NALYSIS

Original: To demonstrate the effectiveness of the iteratively refined prior information introduced by I DGP , we remove the loss function L reg to reversethe I DGP to I DGP - ML  which only uses the log-likelihood function for optimization.
Revised: To demonstrate the effectiveness of the iteratively refined prior information introduced by I DGP , we remove the loss function L reg to reverse I DGP to I DGP - ML , which only uses the log-likelihood function for optimization.

Original: The performance of I DGP - ML against I DGP is also measured by the classification accuracy (with pairwise t-test at 0.05 significance level).
Revised: The performance of I DGP - ML against I DGP is also measured by the classification accuracy (with pairwise t-test at 0.05 significance level).

Original: As is illustrated in Table 3,I DGP achieves superior performance on all real-world datasets compared to I DGP - ML with the assistance of the prior information which I DGP provides and improves epoch by epoch.
Revised: As is illustrated in Table 3, with the assistance of the prior information which I DGP provides and improves epoch by epoch, I DGP achieves superior performance on all real-world datasets compared to I DGP - ML .

Original: Furthermore, we conduct parameter sensitivity analysis to study the influence of the two hyperparameters a , γ on our algorithm, which decides the scale of Dirichlet and Beta distribution parameters.
Revised: Furthermore, we conduct parameter sensitivity analysis to study the influence of the two hyperparameters a , γ on our algorithm, which decides the scale of Dirichlet and Beta distribution parameters.

Original: Figure 2 illustrates the sensitivity of I DGP in the real-world datasets including Lost , BirdSong ,
Revised: Figure 2 illustrates the sensitivity of I DGP in the real-world datasets including Lost , BirdSong ,

Original: Yahoo!New when a varies from 0 .
Revised: Yahoo!New when a varies from 0 .

Original: 001 to 1000 and γ increases from 0 .
Revised: 001 to 1000 and γ increases from 0 .

Original: 1 to 3 .
Revised: 1 to 3 .

Original: We can easily find that as for the small-scale real-world datasets like Lost , a and γ are suggested around 0 .
Revised: We can easily find that as for the small-scale real-world datasets like Lost , a and γ are suggested around 0 .

Original: 1 and 0 . 5 , respectively.
Revised: 1 and 0 . 5 , respectively.

Original: For the more large-scale real-world datasets like BirdSong , Yahoo!News , I DGP seems insensitive to a , and γ are suggested around 1 .
Revised: For the more large-scale real-world datasets like BirdSong , Yahoo!News , I DGP seems insensitive to a , and γ are suggested around 1 .

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we consider a more realistic scenario, instance-dependent PLL, and explicitly decompose and model the generation process of instance-dependent candidate labels.
Revised: In this paper, we consider a more realistic scenario, instance-dependent PLL, and explicitly decompose and model the generation process of instance-dependent candidate labels.

Original: Then based on the decompositional generation process, a novel instance-dependent PLL approach I DGP is proposed by us to further introduce and refine the prior information in every training epoch via MAP.
Revised: Then based on the decompositional generation process, a novel instance-dependent PLL approach I DGP is proposed by us to further introduce and refine the prior information in every training epoch via MAP.

Original: The experimental comparisons with other DNN-based algorithms on both instance-dependent corrupted benchmark datasets and real-world datasets demonstrate the effectiveness of our proposed method. 
Revised: The experimental comparisons with other DNN-based algorithms on both instance-dependent corrupted benchmark datasets and real-world datasets demonstrate the effectiveness of our proposed method. A CKNOWLEDGMENTS

Revised: This research was supported by the National Key Research & Development Plan of China (No.

Revised: 2021ZD0114202), the National Science Foundation of China (62206050, and 62076063),

Revised: China Postdoctoral Science Foundation (2021M700023), Jiangsu Province Science Foundation for

Revised: Youths (BK20210220), Young Elite Scientists Sponsorship Program of Jiangsu Association for

Original:  A A PPENDIX A. D ERIVATIONS OF E Q .(4) AND E Q .(6)
Revised: Science and Technology (TJ-2022-078) A A PPENDIX A. D ERIVATIONS OF E Q .(4) AND E Q .(6)

Original: The derivation of Eq.(4) is as follows: 
Revised: The derivation of Eq.(4) is as follows: [Equation] =

Original: [Equation]
Revised: [Equation]

Original: The derivation of Eq.(6) is as follows:
Revised: log (cid:88) j [Equation] S i p ( y

Original: [Equation]
Revised: [Equation]

Revised: =

Revised: j | x i ) p ( ¯ S ji | x i ) = − n (cid:88)

Revised: i = log (cid:88)

Original: Due to that we later use ˆ λ , ˆ α and ˆ β in Eq.(10) and Eq.(11) to replace λ , α and β in Eq.(7), which are fixed to be constants, C Γ will be a constant and ignored.
Revised: j ∈ S i ( Cat ( l i | x i , θ i ) c (cid:89) k = Ber ( ¯ s ki | z ki ) | y i = j )

Revised: = − n (cid:88)

Revised: i = log (cid:88)

Revised: j ∈ S i ( c (cid:89) k

Revised: = ( θ ki ) l ki c (cid:89) k

Revised: = ( z ki ) ¯ s ki (1 − z ki ) 1 − ¯ s ki | y i =

Revised: j )

Revised: = − n (cid:88)

Revised: [Equation]

Revised: The derivation of Eq.(6) is as follows:

Revised: [Equation]

Revised: C Γ

Revised: [Equation]

Revised: 1 λ ji )

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: Γ( λ ji )

Revised: +

Revised: [Equation]

Revised: (cid:81) cj =1 Γ( α ji + β ji ) Γ( α ji )Γ( β ji ) .

Revised: Due to that we later use ˆ λ , ˆ α and ˆ β in Eq.(10) and Eq.(11) to replace λ , α and β in Eq.(7), which are fixed to be constants, C Γ will be a constant and ignored.

Original: Here, the derivation of Eq.(6) has been finished.
Revised: Here, the derivation of Eq.(6) has been finished.

Original: A. D EGENERATION OF E Q .(7)
Revised: A. D EGENERATION OF E Q .(7)

Original: The proposed process can accommodate the uniform generation process of candidate labels by settng the parameters of Bernoulli distribution to a constant p , which also means the flipping probability.
Revised: The proposed process can accommodate the uniform generation process of candidate labels by settng the parameters of Bernoulli distribution to a constant p , which also means the flipping probability.

Original: In this case, we can degenerate L ML , L reg and L to the following form.
Revised: In this case, we can degenerate L ML , L reg and L to the following form.

Original: [Equation]
Revised: [Equation]

Original: Due to that the second term is a constant, the final L ML is formulated as
Revised: Due to that the second term is a constant [Equation] the final L ML is formulated as

Original: [Equation]
Revised: [Equation]

Original: Due to the parameters of Bernoulli Distribution are constant
Revised: Due to the parameters of Bernoulli Distribution are constant

Original: [Equation]
Revised: [Equation]

Original: we do not need Beta Distribution.
Revised: we do not need Beta Distribution.

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: Finally
Revised: Finally,

Original: [Equation]
Revised: [Equation]

Original: A. D ERIVATIONS OF E Q .(8) AND E Q .(9)
Revised: A. D ERIVATIONS OF E Q .(8) AND E Q .(9)

Original: [Equation]
Revised: [Equation]

Original: The above has proved the conjugacy of Dirichlet and Categorical distribution.
Revised: The above has proved the conjugacy of Dirichlet and Categorical distribution.

Original: For Dirichlet distribution , its expectation can be calculated as
Revised: For Dirichlet distribution , its expectation can be calculated as

Original: [Equation]
Revised: [Equation]

Original: The mathematical derivations of Eq.(8) are completed.
Revised: The mathematical derivations of Eq.(8) are completed.

Original: Eq.(9) can be proved in a similar way due to
Revised: Eq.(9) can be proved in a similar way due to

Original: Dirichlet distribution and Categorical distribution are the generalization forms of Beta distribution and Bernoulli distribution respectively.
Revised: Dirichlet distribution and Categorical distribution are the generalization forms of Beta distribution and Bernoulli distribution respectively.

Original: A. C ALCULATION D ETAILS OF E Q . (12)
Revised: A. C ALCULATION D ETAILS OF E Q . (12)

Original: According to multivariate basic inequality, we can obtain:
Revised: According to multivariate basic inequality [Equation] we can obtain:

Original: [Equation]
Revised: [Equation]

Original: Then we can calculate Eq.
Revised: Then we can calculate Eq.

Original: (12) as
Revised: (12) as

Original: [Equation]
Revised: [Equation]

Original: where e j denotes the standard canonical vector in R c [Equation] ℓ denotes the cross-entropy function, w ji= λ
Revised: where e j denotes the standard canonical vector in R c , and ℓ denotes the cross-entropy function, w ji

Original:  ji − 1+ 1 | S i | if j ∈ S i , otherwise w ji = λ ji − 1 , and K i = log | S i | + 1 | S i |(cid:80) j ∈ S i log (cid:81)
Revised: = λ ji − 1 + 1 | S i | if j ∈ S i , otherwise w ji = λ ji − 1 , and K i = log | S i | + 1 | S i |

Original: k ∈ S ji z ki (cid:81) k/ ∈ S ji (1 − z ki ) + ( α ji − 1) log z ji
Revised: [Equation] S i

Original: + ( β ji − 1) log (1 − z ji ) .
Revised: [Equation] [Equation]

Original: A. P ROOF OF L EMMA
Revised: A. P ROOF OF L EMMA

Original: In order to prove this lemma, we first show L MAP can be bounded.
Revised: In order to prove this lemma, we first show L MAP can be bounded.

Original: Due to the ouput of f, g is limited in [ − A, A ] , the following inequations hold:
Revised: Due to the ouput of f, g is limited in [ − A, A ] , the following inequations hold:

Original: [Equation]
Revised: [Equation]

Original: Hence,
Revised: Hence,

Original: L MAP ≤ − [Equation]
Revised: L MAP ≤ − log | S i | B (1 − F ) c

Original:  | 
Revised: +1 −| S i | E | S

Original: [Equation] log[ BE (1 − F )] a exp( A/γ )
Revised: i |− 1 − c log[ BE (1 − F )] a exp( A/γ )

Original: + b
Revised: + b

Original: When M takes the value larger than − log | S i | B (1 − F )
Revised: When M takes the value larger than − log | S i | B (1 − F )

Original: c +1 −| S i | E
Revised: c +1 −| S i | E

Original: | S
Revised: | S

Original: i |− 1 − c log[ BE (1 −
Revised: i |− 1 − c log[ BE (1 −

Original: F )] a exp( A/γ )+ b , the loss L MAP will be bounded.
Revised: F )] a exp( A/γ )+ b , the loss L MAP will be bounded.

Original: Note that the limitation to the output of the model excludes extreme (conditional) probabilities, the effect of which could be ignored which A is large enough.
Revised: Note that the limitation to the output of the model excludes extreme (conditional) probabilities, the effect of which could be ignored which A is large enough.

Original: Then, we show that the one direction sup f ∈F R ( f ) − (cid:98) R ( f ) is bounded with probability at least 1 − ξ/ 2 , and the other direction can be similarly shown.
Revised: Then, we show that the one direction sup f ∈F R ( f ) − (cid:98) R ( f ) is bounded with probability at least 1 − ξ/ 2 , and the other direction can be similarly shown.

Original: Suppose an example ( x i , S i ) is replaced by another arbitrary example ( x ′ i , S ′ i ) , then the change of sup f ∈F R ( f ) − (cid:98) R ( f ) is no greater than
Revised: Suppose an example ( x i , S i ) is replaced by another arbitrary example ( x ′ i , S ′ i ) , then the change of sup f ∈F R ( f ) − (cid:98) R ( f ) is no greater than

Original: M/ (2 n ) , the loss function L are bounded by M .
Revised: M/ (2 n ) , the loss function L are bounded by M .

Original: By applying McDiarmid’s inequality, for any ξ > 0 , with probability at least 1 − ξ/ 2 ,
Revised: By applying McDiarmid’s inequality, for any ξ > 0 , with probability at least 1 − ξ/ 2 ,

Original: [Equation]
Revised: [Equation]

Original: By sysmetrization, we can obtain
Revised: By sysmetrization, we can obtain

Original: [Equation]
Revised: [Equation]

Original: [Equation] By further taking into account the other side sup f ∈F R ( f ) − [Equation] R ( f ) , we have for any ξ > 0 , with probability at least 1 − ξ ,
Revised: By further taking into account the other side sup f ∈F R ( f ) − (cid:98) R ( f ) , we have for any ξ > 0 , with probability at least 1 − ξ ,

Original: [Equation]
Revised: [Equation]

Original: A. P ROOF OF L EMMA
Revised: A. P ROOF OF L EMMA

Original: The upper bound loss function of L MAP ( f ( x i ) , S i ) is denoted by
Revised: The upper bound loss function of L MAP ( f ( x i ) , S i ) is denoted by

Original: [Equation]
Revised: [Equation]

Original: Correspondingly, the function space for ¯ L can be defined as:
Revised: Correspondingly, the function space for ¯ L can be defined as:

Original: [Equation]
Revised: [Equation]

Original: Then the expected Rademacher complexity of ¯ G can be defined as follows:
Revised: Then the expected Rademacher complexity of ¯ G can be defined as follows:

Original: [Equation]
Revised: [Equation]

Original: For each example ( x i , S i ) , since w ji is bounded in [0 , ρ ] and K i ≤ K , we can obtain (cid:101) R n ( G ) ≤ (cid:101) R n ( ¯ G ) ≤ ρc R n ( ℓ ◦F )
Revised: For each example ( x i , S i ) , since w ji is bounded in [0 , ρ ] and K i ≤ K , we can obtain (cid:101) R n ( G ) ≤ (cid:101) R n ( ¯ G ) ≤ ρc R n ( ℓ ◦F )

Original: + K where ℓ ◦F denotes { ℓ ◦F| f ∈ F} .
Revised: + K where ℓ ◦F denotes { ℓ ◦F| f ∈ F} .

Original: Since H ι = { h : x (cid:55)→ f y ( x ) | f ∈ F} and the loss function ℓ ( f ( x , e ι )) is L -Lipschitz for all ι ∈ Y , by the Rademacher vector contraction inequality, we have R n ( ℓ ◦ F ) ≤ √ 2 L (cid:80) ι ∈Y R n ( H ι ) .
Revised: Since H ι = { h : x (cid:55)→ f y ( x ) | f ∈ F} and the loss function ℓ ( f ( x , e ι )) is L -Lipschitz for all ι ∈ Y , by the Rademacher vector contraction inequality, we have R n ( ℓ ◦ F ) ≤ √ 2 L (cid:80) ι ∈Y R n ( H ι ) .

Original: Then the proof is completed.
Revised: Then the proof is completed.

Original: A. P ROOF OF T HEOREM
Revised: A. P ROOF OF T HEOREM

Original: Theorem is proven through
Revised: Theorem is proven through

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Revised: [Table]

Original: A. D ETAILS OF DATA AUGMENTATION
Revised: A. D ETAILS OF DATA AUGMENTATION

Original: For all benchmark datasets, we apply Random Horizontal Flipping, Random Cropping, and Cutout.
Revised: For all benchmark datasets, we apply Random Horizontal Flipping, Random Cropping, and Cutout.

Original: For CIFAR-10 and CIFAR-100 , AutoAugment is additionally applied.
Revised: For CIFAR-10 and CIFAR-100 , AutoAugment is additionally applied.

Original: A. E XTENDING E XPERIMENTS
Revised: A. E XTENDING E XPERIMENTS

Original: Our Generation Model .
Revised: Our Generation Model .

Original: We also synthesize the sampled correct label from Categorical Distribution p ( y = j | x ) and incorrect candidate labels from Bernoulli Distribution p (¯ s j = 1 | x ) .
Revised: We also synthesize the sampled correct label from Categorical Distribution p ( y = j | x ) and incorrect candidate labels from Bernoulli Distribution p (¯ s j = 1 | x ) .

Original: For the former,[Table] [Table] correct labels has been contained in these datasets.
Revised: For the former, correct labels has been contained in these datasets.

Original: For the latter, we use the confidence prediction of a clean neural network g ( x ; ˆ Ω ) (trained only with correct labels) to model the Bernoulli Distribution, i.e., p (¯ s j = 1 | x ) = sigmoid ( g j ( x ; ˆ Ω )) .
Revised: For the latter, we use the confidence prediction of a clean neural network g ( x ; ˆ Ω ) (trained only with correct labels) to model the Bernoulli Distribution, i.e., p (¯ s j = 1 | x ) = sigmoid ( g j ( x ; ˆ Ω )) .

Original: Table 4 illustrates the performance of I DGP and comparing approaches on on benchmark datasets, of which instance-dependent partial labels generated by our generation model.
Revised: Table 4 illustrates the performance of I DGP and comparing approaches on on benchmark datasets, of which instance-dependent partial labels generated by our generation model.

Original: From the table, we can overall see that I DGP consistently outperforms all comparing approaches on all benchmark datasets, and the improvements are particularly noticeable on Kuzushiji-MNIST and CIFAR-10 .
Revised: From the table, we can overall see that I DGP consistently outperforms all comparing approaches on all benchmark datasets, and the improvements are particularly noticeable on Kuzushiji-MNIST and CIFAR-10 .

Original: Ablation Study .
Revised: Ablation Study .

Original: As is illustrated in Table 5, I DGP also achieves superior performance on all benchmark datasets, of which instance-dependent partial labels are generated by our generation model.
Revised: As is illustrated in Table 5, I DGP also achieves superior performance on all benchmark datasets, of which instance-dependent partial labels are generated by our generation model.

Original: Sensitive Analysis .
Revised: Sensitive Analysis .

Original: For the benchmark dataset Fashion-MINIST , the performance of I DGP is stable and effective in the case that a is around 10 and γ is around 1 .
Revised: For the benchmark dataset Fashion-MINIST , the performance of I DGP is stable and effective in the case that a is around 10 and γ is around 1 .

Original: [Figure]
Revised: [Figure]