Original: M UTUAL P ARTIAL L ABEL L EARNING WITH C OMPETITIVE L ABEL N OISE
Revised: M UTUAL P ARTIAL L ABEL L EARNING WITH C OMPETITIVE L ABEL N OISE

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Partial label learning (PLL) is an important weakly supervised learning problem, where each training instance is associated with a set of candidate labels that include both the true label and noise labels.
Revised: Partial label learning (PLL) is an important weakly supervised learning problem, where each training instance is associated with a set of candidate labels that include both the true label and additional noisy labels.

Original: Most existing PLL methods assume the candidate noise labels are randomly chosen, which hardly holds in the realworld learning scenarios.
Revised: Most existing PLL methods assume the candidate noisy labels are randomly chosen, which hardly holds in real-world learning scenarios.

Original: In this paper, we consider a more realistic PLL scenario with competitive  noiselabels that are more difficult to distinguish from the true label than the random noise labels.
Revised: In this paper, we consider a more realistic PLL scenario with competitive label noise that is more difﬁcult to distinguish from the true label than the random label noise.

Original: We propose a novel Mutual Learning based PLL approach named ML-PLL to address this challenging problem.
Revised: We propose a novel Mutual Learning based PLL approach named ML-PLL to address this challenging problem.

Original: ML-PLL learns a prediction network based classifier and a class-prototype based classifier cooperatively through interactive mutual learning and label correction.
Revised: ML-PLL learns a prediction network based classiﬁer and a class-prototype based classiﬁer cooperatively through interactive mutual learning and label correction.

Original: Moreover, we use a transformation network to model the association relationships between the true label and candidatenoise labels, and learn it together with the prediction network to match the observed candidate labels in the training data and enhance label correction.
Revised: Moreover, we use a transformation network to model the association relationships between the true label and candidate labels, and learn it together with the prediction network to match the observed candidate labels in the training data and enhance label correction.

Original: Extensive experiments are conducted on several benchmark PLL datasets, and the proposed ML-PLL approach demonstratesthe state-of-the-art performance for partial label learning.
Revised: Extensive experiments are conducted on several benchmark PLL datasets, and the proposed ML-PLL approach demonstrates state-of-the-art performance for partial label learning.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: As it is costly and difficult to annotate each instance with a precise label, weakly supervised learning (WSL) has been widely studied in recent years (Zhou, 2018), which includes, but not limited to, semi-supervised learning (Van Engelen & Hoos, 2020; Ouali et al., 2020), noise label learning (Natarajan et al., 2013; Feng et al., 2021), positive-unlabeled learning (Kiryo et al., 2017; Shu et al., 2020), and partial multi-label learning (Xie & Huang, 2018; Yan & Guo, 2021).
Revised: As it is costly and difﬁcult to annotate each instance with a precise label weakly supervised learning (WSL) has been widely studied in recent years (Zhou, 2018), which includes, but not limited to, semi-supervised learning (Van Engelen & Hoos, 2020; Ouali et al., 2020), noisy label learning (Natarajan et al., 2013; Feng et al., 2021), positive-unlabeled learning (Kiryo et al., 2017; Shu et al., 2020), and partial multi-label learning (Xie & Huang, 2018; Yan & Guo, 2021).

Original: Partial label learning (PLL) is a typical WSL problem and aims to learn a model from training samples with overcomplete labels; that is, each training sample is associated with a set of candidate labels that include both the true label and noise labels.
Revised: Partial label learning (PLL) is a typical WSL problem and aims to learn a model from training samples with overcomplete labels; that is, each training sample is associated with a set of candidate labels that include both the true label and additional label noise—noisy labels.

Original: PLL has been widely applied in many real-world learning scenarios, including automatic face naming (Hüllermeier & Beringer, 2006; Zeng et al., 2013), web mining (Luo & Orabona, 2010), and multimedia content analysis (Zeng et al., 2013).
Revised: PLL has been widely applied in many real-world learning scenarios, including automatic face naming (Hüllermeier & Beringer, 2006; Zeng et al., 2013), web mining (Luo & Orabona, 2010), and multimedia content analysis (Zeng et al., 2013).

Original: Since the ground-truth label is hidden in the candidate label set and not available to the learning algorithms, the main challenge of PLL lies in candidate label disambiguation.
Revised: Since the ground-truth label is hidden in the candidate label set and not available to the learning algorithms, the main challenge of PLL lies in candidate label disambiguation.

Original: To address this challenge, two main label disambiguation strategies have been proposed: average-based disambiguation strategy and identification-based disambiguation strategy.
Revised: To address this challenge, two main label disambiguation strategies have been proposed: average-based disambiguation strategy and identiﬁcation-based disambiguation strategy.

Original: Average-based disambiguation treats each candidate label equally in the model training phase and averages all the modeling outputs from each candidate label in the testing phase(Cour et al., 2011; Hüllermeier & Beringer, 2006; Zhang & Yu, 2015).
Revised: Average-based disambiguation treats each candidate label equally in the model training phase and averages all the modeling outputs from each candidate label in the testing phase (Cour et al., 2011; Hüllermeier & Beringer, 2006; Zhang & Yu, 2015).

Original: Although this strategy is simple and clear, it can make the true label overwhelmed by noise labels without sufficient differentiation, and lead to poor prediction performance.
Revised: Although this strategy is simple and clear, it can make the true label overwhelmed by noisy labels without sufﬁcient differentiation, and lead to poor prediction performance.

Original: The identificationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving different confidences for the candidate labels (Feng & An, 2018; 2019; 
Revised: The identiﬁcationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving different conﬁdence scores for the candidate labels (Feng & An, 2018; 2019; Yao et al., 2020b; Yu & Zhang, 2016;

Original: Yao et al., 2020b; Yu & Zhang, 2016; Xu et al., 2021).
Revised: Xu et al., 2021).

Original: The identification-based disambiguation approaches  achieve relatively better prediction performance than the average-based disambiguation approaches by handling the candidate labels with discrimination, but they can still suffer from the potential drawback that the identified label may not be the true label, which could severely disrupt the subsequent model training.
Revised: The identiﬁcation-based disambiguation approaches are able to achieve relatively better prediction performance than the average-based disambiguation approaches by handling the candidate labels with discrimination, but they can still suffer from the potential drawback of accumulating label identiﬁcation errors and severely disrupting the subsequent model training.

Original: In addition, these existing methods are usually restricted to standard machine learning frameworks with liner or kernel-based approaches, which have difficulties in dealing with large-scale datasets.
Revised: In addition, these existing methods are usually restricted to standard machine learning frameworks with linear or kernel-based approaches, which have difﬁculties to deal with large-scale datasets.

Original: Recently, deep learning has been widely used in dealing with PLL problems (Feng et al., 2020b;
Revised: Recently, deep learning has been widely used in addressing PLL problems (Feng et al., 2020b;

Original: Wu et al., 2022).
Revised: Wu et al., 2022).

Original: For example, the work in (Feng et al., 2020b) proposes two PLL methods, riskconsistent and classifier-consistent algorithms, based on deep neural networks.
Revised: For example, the work in (Feng et al., 2020b) proposes two PLL methods, a risk-consistent and a classiﬁer-consistent algorithms, based on deep neural networks.

Original: The methods in (Wen et al., 2021; Lv et al., 2020) try to progressively identify the ground-truth label during training procedures by employing a self-training technique.
Revised: The methods in (Wen et al., 2021; Lv et al., 2020) try to progressively identify the ground-truth label during training procedures by employing a self-training technique.

Original: A more recent work in (Wang et al., 2022) tackles PLL by using contrastive representation learning with class-prototype based label disambiguation.
Revised: A more recent work in (Wang et al., 2022b) tackles PLL by using contrastive representation learning with class-prototype based label disambiguation.

Original: Although it achieves satisfactory prediction performance, the contrastive learning procedure isusually time-consuming and resource-demanding.
Revised: Although it achieves satisfactory prediction performance, the contrastive learning procedure is time-consuming and resource-demanding.

Original: Moreover, all these methods have a common drawback: they automatically assume random noise in the label space; that is, they assume the candidate noise labels are randomly sampled from a uniform generating procedure.
Revised: Moreover, all these methods have a common drawback: they automatically assume random noise in the label space; that is, they assume the candidate noisy labels are randomly sampled from a uniform generating procedure.

Original: However, during the noise label creation procedure in real-world scenarios, such as crowd-sourcing, the noise labels are typically associated with the true label and dependent on the input sample, which makes the  noiselabels to be more difficult to distinguish from the ground-truth label than a random label.
Revised: However, during the noisy label creation process in real-world scenarios, such as crowd-sourcing, the noisy labels are typically associated with the true label and dependent on the input sample, which makes the label noise more difﬁcult to distinguish from the ground-truth label than a random label.

Original: In this paper, we consider a more realistic and challenging PLL learning scenario, where the noise labels are competitive and difficult to distinguish from the true label given the input data sample.
Revised: In this paper, we consider a more realistic and challenging PLL learning scenario, where the noisy labels are competitive and difﬁcult to distinguish from the true label given the input data sample.

Original: Intuitively, competitive  noiselabels can demonstrate stronger association relationships with a true label than a random label noise, and hence are more likely to be chosen as candidate labels.
Revised: Intuitively, competitive label noise can demonstrate stronger association relationships with a true label than a random label noise, and hence are more likely to be chosen as candidate labels.

Original: For example, for online image annotation, when the object contained in an image is an “alpaca”, a competitive label “camel” could have a large probability to be chosen by an annotator with limited expertise due to the similar appearances of “alpaca” and “camel”, while labels such as “dog” or “duck” are less likely to be picked as part of the candidate label set due to their relatively weak association with the ground-truth “alpaca”.
Revised: For example, for online image annotation, when the object contained in an image is an “alpaca”, a competitive label “camel” could have a large probability to be chosen by an annotator with limited expertise due to the similar appearances of “alpaca” and “camel”, while labels such as “dog” or “duck” are less likely to be picked as part of the candidate label set due to their relatively weak association with the ground-truth “alpaca”.

Original: Motivated by the above consideration, we propose a novel and effective Mutual Partial Label Learning approach (ML-PLL) under the competitive label noise learning scenario, which learns a prediction network based classifier and  class-prototype based classifier interactively through label correction and mutual learning.
Revised: Motivated by the above consideration, we propose a novel and effective Mutual Partial Label Learning approach (ML-PLL) under the competitive label noise learning scenario, which learns a prediction network based classiﬁer and a class-prototype based classiﬁer interactively through label correction and mutual learning.

Original: Specially, ML-PLL performs noise label correction by integrating the outputs of both the prediction network classifier and the class-prototype based classifier, while using the corrected pseudo-labels as the targets for the prediction network training and using the outputs of the prediction network as the targets for the prototype-based classifier training.
Revised: Speciﬁcally, ML-PLL performs noisy label correction by integrating the outputs of both the prediction network classiﬁer and the class-prototype based classiﬁer, while using the corrected pseudo-labels as the targets for the prediction network training and using the output of the prediction network as target for the class-prototype based classiﬁer training.

Original: In addition, a transformation network is proposed to model the association relationships between the true label and the noise candidate labels, which can further enhance the classifier training with respect to ground-truth label disambiguation.
Revised: In addition, a transformation network is proposed to model the association relationships between the true label and the noisy candidate labels, which can further enhance the classiﬁer training with respect to ground-truth label disambiguation.

Original: Extensive experiments are conducted on benchmarks PLL datasets, while the proposed approach ML-PLL demonstratesthe state-of-the-art performance for partial label learning.
Revised: Extensive experiments are conducted on several benchmark PLL datasets, while the proposed approach, ML-PLL, demonstrates state-of-the-art performance.

Original:  R ELATED W ORK 2. S TANDARD P ARTIAL L ABEL L EARNING
Revised:  R ELATED W ORK 2. S TANDARD P ARTIAL L ABEL L EARNING

Original: The main challenge for dealing with PLL problem lies in how to disambiguate the candidate labels.
Revised: The main challenge for addressing the PLL problem lies in how to disambiguate the candidate labels.

Original: Average-based disambiguation strategy and identification-based disambiguation strategy are two main strategies deployed in PLL.
Revised: Average-based disambiguation strategy and identiﬁcation-based disambiguation strategy are two main strategies deployed in PLL.

Original: The average-based disambiguation strategy treats each candidate label equally in the model induction and averages all the modeling outputs from all the candidate labels as the final prediction.
Revised: The average-based disambiguation strategy treats each candidate label equally in the model induction and averages all the modeling outputs from all the candidate labels as the ﬁnal prediction.

Original: For example, the works in (Cour et al., 2011; Zhang & Yu, 2015) distinguishall the averaged candidate label prediction from the non-candidate ones.
Revised: For example, the works in (Cour et al., 2011; Zhang & Yu, 2015) distinguish the averaged candidate label prediction from the non-candidate ones.

Original: The identificationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving confidence scores for all the candidate labels (Jin & Ghahramani, 2002; Feng & An, 2018; 2019; Yu & Zhang, 2016).
Revised: The identiﬁcation-based disambiguation strategy treats the ground-truth label as a latent variable and identiﬁes the true label by deriving conﬁdence scores for all the candidate labels (Feng & An, 2018; 2019; Yu & Zhang, 2016).

Original: For example, the works in (Zhang et al., 2016; Xu et al., 2019; Wang et al., 2021) try to identify the true label by employing iterative label refining procedures and leveraging the topological information in the feature space.
Revised: For example, the works in (Zhang et al., 2016; Xu et al., 2019; Wang et al., 2022a) try to identify the true label by employing iterative label reﬁning procedures and leveraging the topological information in the feature space.

Original: However, these methods may suffer from the cumulative errors induced in the error-prone label confidence estimation along the topological structure.
Revised: However, these methods may suffer from the cumulative errors induced in the error-prone label conﬁdence estimation along the topological structure.

Original: In addition, a number of PLL methods propose to employ the off-the-shelf learning techniques to tackle PLL problems.
Revised: In addition, a number of PLL methods propose to employ the off-the-shelf learning techniques, such as maximum likelihood, k-nearest-neighbor, maximum margin, boosting, and error-correcting output codes (ECOC), to tackle PLL problems.

Original: For the maximum likelihood technique, the likelihood of each PL training sample is defined over its candidate label set instead of its implicit ground-truth label (Jin & Ghahramani, 2002; Liu & Dietterich, 2012).
Revised: For the maximum likelihood technique, the likelihood of each PL training sample is deﬁned over its candidate label set instead of its implicit ground-truth label (Liu & Dietterich, 2012).

Original: For the k -nearest neighbor technique, the candidate labels from neighbor instances are aggregated to produce the final prediction on a test instance (Hüllermeier & Beringer, 2006;
Revised: For the k -nearest neighbor technique, the candidate labels from neighbor instances are aggregated to produce the ﬁnal prediction on a test instance (Hüllermeier & Beringer, 2006;

Original: Gong et al., 2017; Zhang & Yu, 2015).
Revised: Gong et al., 2017; Zhang & Yu, 2015).

Original: For the maximum margin technique, the classification margin is defined over the predictive difference between the candidate labels and the non-candidate labels for each PL training sample (Nguyen & Caruana, 2008)(Yu & Zhang, 2016).
Revised: For the maximum margin technique, the classiﬁcation margin is deﬁned over the predictive difference between the candidate labels and the non-candidate labels for each PL training sample (Nguyen & Caruana, 2008)(Yu & Zhang, 2016).

Original: For the boosting technique, the weight of each PL training instance and the confidence value of each candidate label being the ground-truth label are refined in each boosting round (Tang & Zhang, 2017).
Revised: For the boosting technique, the weight of each PL training instance and the conﬁdence value of each candidate label being the truth label are reﬁned in each boosting round (Tang & Zhang, 2017).

Original: For the error-correcting output codes (ECOC) technique, multiple binaryclassifiers to ECOC coding matrix are built based on thetransformed binary  (Zhang et al., 2017).
Revised: For the ECOC technique, the binary ECOC classiﬁers are built based on the binary training sets ﬁltered by the candidate labels (Zhang et al., 2017).

Original: 2. D EEP N EURAL N ETWORK BASED P ARTIAL L ABEL L EARNING
Revised: 2. D EEP N EURAL N ETWORK BASED P ARTIAL L ABEL L EARNING

Original: Standard PLL methods are usually restricted to liner based models, which has difficulty in dealing with large-scale datasets.
Revised: Standard PLL methods are usually restricted to linear based models, which have difﬁculty to deal with large-scale datasets.

Original: Deep learningparadigm based PLL has recently started gaining attention from the research community due to its powerful capability of training on large-scale datasets.
Revised: Deep learning based PLL has recently started gaining attention from the research community due to its powerful capability of training on large-scale datasets.

Original: The work in (Yao et al., 2020a) adopts deep convolutional neural networks as backbones and exploits the temporal-ensembling technique for model training.
Revised: Yao et al. (2020a) adopt deep convolutional neural networks as backbones and exploit the temporalensembling technique for model training.

Original: The work in (Yao et al., 2020b) proposes to train two networks in a mutual learning manner, which helps both networks interact with each other and alleviate the error accumulation problem.
Revised: Yao et al. (2020b) propose to train two networks in a mutual learning manner, where both networks interact with each other and alleviate the error accumulation problem.

Original: The work in (Lv et al., 2020) attempts to progressively identify the true label in a self-training manner, which can be compatibly trained with stochastic optimization.
Revised: Lv et al. (2020) attempt to progressively identify the true labels in a self-training manner, which can be compatibly trained with stochastic optimization.

Original: The work in (Feng et al., 2020b) assumes that the probability of each noise label being chosen is uniform and presents two provably consistent methods: a risk-consistent method and a classifier-consistent method.
Revised: Feng et al. (2020b) assume that the probability of each candidate label being chosen is uniform and present two provably consistent methods: a risk-consistent method and a classiﬁer-consistent method.

Original: Another work in (Wen et al., 2021) induces a family of loss functions, which generalize the uniform generation procedures.
Revised: Another work in (Wen et al., 2021) induces a family of loss functions, which generalizes the uniform generation procedures.

Original: In addition, a recent work in (Wang et al., 2022) handles PLLproblems by conducting class prototype-based label disambiguation following the idea of contrastive representation learning (He et al., 2020; Li et al., 2020), which achieves impressive prediction performance.
Revised: In addition, a recent work in (Wang et al., 2022b) handles PLL by conducting class prototype-based label disambiguation following the idea of contrastive representation learning (He et al., 2020; Li et al., 2020), which achieves impressive prediction performance.

Original: Although the above approaches produce competitive performance, most of them assume the noise labels in the candidate label set are randomly produced, whichhowever does not hold in many realworld learning scenarios.
Revised: Although the abovementioned approaches produce competitive performance, most of them assume the noisy labels in the candidate label set are randomly produced, which does not hold in many real-world learning scenarios.

Original: By contrast, we tackle a more realistic and challenging PLL scenario, where the noise candidate labels are competitive and difficult to distinguish from the true label.
Revised: By contrast, we tackle a more realistic and challenging PLL scenario, where the noisy candidate labels are competitive and difﬁcult to distinguish from the true label.

Original:  P ROPOSED A PPROACH
Revised:  P ROPOSED A PPROACH

Original: In this section, we delineate the details of our proposed mutual partial label learning(ML-PLL) model  in the  competitive label noise scenario.
Revised: In this section, we delineate the details of our proposed mutual partial label learning model (MLPLL) in the learning scenario with competitive label noise.

Original: Given a partial label training set D =
Revised: Given a partial label training set D =

Original: ( X, Y )
Revised: ( X, Y )

Original: = { ( x i , y i ) } ni
Revised: = { ( x i , y i ) } ni

Original: =1 , where x i ∈
Revised: =1 , where x i ∈

Original: X denotes the i -th instance and X denotes the input space; y i ⊆ Y denotes the candidate label indicator vector associated with x i , and Y = { 1 , 2 , · · · , L } denotes the multi-class label space.
Revised: X denotes the i -th instance expressed in the input space X , and y i ∈ { 0 , 1 } L denotes the candidate label indicator vector associated with x i .

Original: The multiple 1 values in y i denotes either the ground-truth label or irrelevant label noise, which forms the candidate label set S i associated with x i .
Revised: The multiplevalues in y i denote either the ground-truth label or irrelevant noisy labels, which form the candidate label set S i ⊆ Y for x i , where Y = { 1 , 2 , · · · , L } denotes the multi-class label space.

Original: The task of PLL is to induce a multi-class classifier from the PLL training set D .
Revised: The task of PLL is to induce a multi-class classiﬁer from the PLL training set D .

Original: We assume a challenging realistic PLL scenario, where the noise labels in the candidate label set are difficult to distinguish from the ground-truth label given the input instance.
Revised: We assume a challenging realistic PLL scenario, where the noisy labels in the candidate label set are difﬁcult to distinguish from the ground-truth label given the input instance.

Original: The proposed ML-PLL model is illustrated in Figure 1, which mainly comprises the following component networks: a feature extractor G that extract features from the input instance v =
Revised: The proposed ML-PLL model is illustrated in Figure 1, which mainly comprises the following component networks: a feature extractor G that extracts features from the input instance v =

Original: G ( x ) ∈ R m ; a projection network H that maps the feature vector v into a low-dimensional
Revised: G ( x ) ∈ R m ; a projection network H that maps the feature vector v into a low-dimensional

Original: embedding z
Revised: embedding z

Original: = H ( v ) ∈ R d ; a prediction network based classifier F ; a class-prototype based classifier; and a transformation network T that estimates the association relationships between the true label and candidate labels.
Revised: = H ( v ) ∈ R d ; a prediction network based classiﬁer F ; a class-prototype based classiﬁer; and a transformation network T that estimates the association relationships between the true label and candidate labels.

Original: The prediction network based classifier and the class-prototype based classifier are learned together through noise label correction and interactive mutual learning, while the prediction network and the transformation network are learned together to match the observed candidate labels in the training data and enhance label disambiguation.
Revised: The prediction network based classiﬁer and the class-prototype based classiﬁer are learned together through noisy label correction and interactive mutual learning, while the prediction network and the transformation network are learned together to match the observed candidate labels in the training data and enhance label disambiguation.

Original: We elaborate these components and the proposed approach in the following subsections.
Revised: We elaborate these components and the proposed approach in the following subsections.

Original: 3. N OISE L ABEL C ORRECTION
Revised: 3. N OISY L ABEL C ORRECTION

Original: The main challenge of PLL lies in that the ground-truth label coexists with the irreverent noise labels in the candidate label set and is unknown to the learning algorithm.
Revised: The main challenge of PLL lies in that the ground-truth label coexists with the irreverent noisy labels in the candidate label set and is unknown to the learning algorithm.

Original: The key of PLL is to correct the candidate label vectors and identify the true label vectors.
Revised: The key of PLL is to correct the candidate label vectors and identify the true label vectors.

Original: Given an interactive pair of  prediction network classifier F and prototype-based classifier, we propose the following simple mutual label correction procedure.
Revised: Given an interactive pair of classiﬁers—a prediction network classiﬁer F and a class-prototype based classiﬁer, we propose the following simple mutual label correction procedure.

Original: For any training instance x i , we correct the partial label indicator vector towards the true label indicator vector by exploiting the predicted pseudo-label vector.
Revised: For any training instance x i , we correct its partial label indicator vector towards the true label indicator vector by exploiting the predicted pseudo-label vector.

Original: Specifically, we produce the soft pseudo-label vector q i ∈ [0 , 1] L for x i by taking a weighted convex combination of the prediction network’s softmax output probability vector F ( v i ) and the class prototype-based prediction probability vector p i :
Revised: Speciﬁcally, we produce the soft pseudo-label vector q i ∈ [0 , 1] L for x i by taking a weighted convex combination of the prediction network’s softmax output probability vector F ( v i ) and the class-prototype based prediction probability vector p i :

Original: [Equation]
Revised: [Equation]

Original: where λ ∈ (0 , 1) is the combination weight parameter.
Revised: where λ ∈ (0 , 1) is the combination weight parameter.

Original: Here the prediction network F( v i ) is built above the feature extractor G , such as F ( v i ) =
Revised: Here the prediction network F is built above the feature extractor G , such as F ( v i ) =

Original: F ( G ( x i )) , while p i is produced by the similaritybased class-prototype classifier as follows:
Revised: F ( G ( x i )) , while p i is produced by the similarity-based class-prototype classiﬁer as follows:

Original: [Equation]
Revised: [Equation]

Original: where τ is a temperature parameter, L is the number of class categories, z i
Revised: where τ is a temperature parameter, L is the number of class categories, z i

Original: = H ( G ( x i )) denotes the low-dimensional embedding of the features for instance x i , and c k denotes the prototype  for the k -th category, which will be elaborated in the next subsection.
Revised: = H ( G ( x i )) denotes the low-dimensional embedding of the features for instance x i , and c k denotes the prototype vector for the k -th category, which will be elaborated in the next subsection.

Original: Instead of using Euclidean distance, we use the dot-product operator “ · ” to compute the inner product similarity between the instance and the class prototypes in the embedding space for class prediction probability calculation.
Revised: Instead of using Euclidean distance, we use the dot-product operator “ · ” to compute the inner product similarity between the instance and the class prototypes in the embedding space for class prediction probability calculation.

Original: To maintain a effective soft pseudo-label vector for each instance and eliminate the noise predictions outside of the candidate label set, we  restrict the soft pseudo-labels within the candidate label set and further re-normalize q i into (cid:98) q i with row normalization,
Revised: To maintain an effective soft pseudo-label vector for each instance and eliminate the noise predictions outside of the candidate label set, we further restrict the soft pseudo-labels within the candidate label set and transform q i into (cid:98) q i with row normalization,

Original: [Equation]
Revised: [Equation]

Original: A corrected soft pseudo-label matrix (cid:98) Q can be obtained by performing the abovementioned noise label correction  on all the n training instances, such that (cid:98) Q = [ (cid:98) qik ] n × L .
Revised: This noisy label correction procedure can be applied on all the n training instances to produce corrected soft pseudo-label vectors { (cid:98) q 1 , · · · , (cid:98) q n } .

Original: With the progress of prediction network training and class-prototype update (elaborated in the next subsection), we expect the produced soft pseudo-label vector will be better aligned with the  true label indicator vector, which can interactively provide a good target for the subsequent prediction network training and gradually mitigate the negative impact of the label noise during the training process.
Revised: With the progress of prediction network training and class-prototype update (elaborated in the next subsection), we expect each produced soft pseudolabel vector will be better aligned with the corresponding true label indicator vector, which can interactively provide a good target for the subsequent prediction network training and gradually mitigate the negative impact of the label noise during the training process.

Original: 3. B ATCH - WISE M OVING A VERAGE BASED P ROTOTYPE U PDATE
Revised: 3. B ATCH - WISE M OVING A VERAGE BASED P ROTOTYPE U PDATE

Original: The effectiveness of noise label correction relies on accurate class probability predictions, which depend on both the prediction network and the class-prototypes.
Revised: The effectiveness of noisy label correction relies on accurate class probability predictions, which depend on both the prediction network and the class-prototypes.

Original: Instead of training an entirely independent prototype-based classifier, we propose to produce the class prototypes based on the outputs of the prediction network classifier.
Revised: Instead of training an entirely independent class-prototype based classiﬁer, we propose to produce the class prototypes based on the output of the prediction network classiﬁer.

Original: Specifically, we compute the class prototypes in the low-dimensional embedding space from the partial label training set D in the following way:
Revised: Speciﬁcally, we compute the class prototypes in the low-dimensional embedding space from the partial label training set D in the following way:

Original: [Equation]
Revised: [Equation]

Original: where 1 is an indicator function, (cid:98)
Revised: where 1 is an indicator function; (cid:98)

Original: y i denotes predicted one-hot label vector for x i , which is determined by the output of the prediction network F and restricted to the candidate label set S i .
Revised: y i denotes predicted one-hot label vector for x i , which is determined by the output of the prediction network F and restricted to the candidate label set S i .

Original: In this way, we can automatically eliminate the prediction noise outside of the candidate labels.
Revised: In this way, we can automatically eliminate the prediction noise outside of the candidate labels.

Original: With the prototype computing method above, we can update the class-prototype by recomputing it during each training iteration.
Revised: With the prototype computing method above, we can update each class-prototype by recomputing it during each training iteration.

Original: However, such an update process will be computational-intensive and lead to intolerable training latency, since it requires going through the whole training set.
Revised: However, such an update process will be computationally intensive and lead to intolerable training latency, since it requires going through the whole training set.

Original: To address this problem, we propose to perform batch-wise moving average based prototype update; that is, we update class-prototype for each mini-batch by consulting the cluster centroid over each mini-batch in a moving averaging manner:
Revised: To address this problem, we propose to perform batch-wise moving average based prototype update; that is, we update the class-prototypes for the current mini-batch by consulting the prototypes computed from previous mini-batches in a moving average manner:

Original: [Equation]
Revised: [Equation]

Original: where c ′ k is the mean embedding for the k -th class within the current mini-batch, which can be computed with Eq.(4) by limiting D to the current mini-batch; γ  is a momentum coefficient; and 
Revised: where c (cid:48) k is the mean embedding for the k -th class within the current mini-batch, which can be computed with Eq.(4) by limiting D to the current mini-batch; γ ∈ [0 , 1] is a momentum coefﬁcient; and Normalize( c )

Original: Normalize( c ) = c / ∥ c ∥ 2 .
Revised: = c / (cid:107) c (cid:107) 2 .

Original: We expect such a batch-wise prototype update method can be computational efficient and enhance the gradual adjustment and stability of the class prototypes.
Revised: We expect such a batch-wise prototype update method can be computationally efﬁcient and enhance the gradual adjustment and stability of the class prototypes.

Original: 3. P REDICTION N ETWORK T RAINING WITH M UTUAL L EARNING
Revised: 3. P REDICTION N ETWORK T RAINING WITH M UTUAL L EARNING

Original: With the noise label correction method above, the corrected soft pseudo-label produced can be treated asthe target for prediction network training.
Revised: With the noisy label correction method above, the corrected soft pseudo-label vectors can be used as target for prediction network training.

Original: Thus, we can learn the prediction network based probabilistic classifier on  training data D by minimizing the following classification loss:
Revised: Speciﬁcally, we can learn the prediction network based probabilistic classiﬁer on the training data D by minimizing the following classiﬁcation loss:

Original: [Equation]
Revised: [Equation]

Original: wherek is the index of the class, (cid:98) q i is the corrected soft pseudo-label vector  for the i -th instance, and F k ( · ) denotes the k -th entry of  prediction network’s softmax outputs, indicating the predicted probability of the instance belonging to the k -th class.
Revised: where (cid:98) q ik denotes the k -th entry of the corrected soft pseudo-label vector (cid:98) q i for the i -th instance, and F k ( · ) denotes the k -th entry of the prediction network’s softmax output, indicating the predicted probability of the instance belonging to the k -th class.

Original: It is expected that the prediction network based classifier F can be improved by training with the corrected soft pseudo-labels.
Revised: We expect the prediction network based classiﬁer F can be improved by training with the corrected soft pseudo-labels.

Original: Meanwhile, we can learn an additional class-prototype based classifier supervised by  the outputs of the prediction network by deploying the followinginteractive mutual learning loss:
Revised: Meanwhile, we can learn an additional class-prototype based classiﬁer by using the output of the prediction network as prediction target and deploying the following mutual learning loss:

Original: [Equation]
Revised: [Equation]

Original: where p ik is the prediction probability produced by  prototype based classifier using Eq.(2).
Revised: where p ik is the predicted probability for the i -th instance belonging to the k -th class by the prototype based classiﬁer using Eq.(2).

Original: We expect this mutual learning loss can enforce both classifiers to collaborate with each other and gradually lead to consistent predictions that can better align with  underlying true label indicator vectors.
Revised: This mutual learning loss is expected to enforce both classiﬁers to collaborate with each other and gradually lead to consistent predictions that can better align with the underlying true label indicator vectors.

Original: 3. T RANSFORMATION BASED L ABEL A SSOCIATION L EARNING
Revised: 3. T RANSFORMATION BASED L ABEL A SSOCIATION L EARNING

Original: As previously discussed, the appearances of noise labels in the candidate label set typically havean association relationship with the ground-truth label and can be competitive to the true label given the input instance in the real-world learning scenarios.
Revised: As previously discussed, the noisy labels in the candidate label set typically have association relationships with the ground-truth label and can be competitive with the true label given the input instance in the real-world learning scenarios.

Original: In this subsection, we propose to learn the competitive association relationship between noise candidate labels and the true label using a transformation network T , aiming to enhance the label disambiguation process.
Revised: In this subsection, we propose to learn the competitive association relationships between the noisy candidate labels and the true label using a transformation network T , aiming to enhance the label disambiguation process.

Original: We assumethat the transformation network T can be learned to produce a transformation matrix R =
Revised: We assume the transformation network T can be learned to produce a transformation matrix R =

Original: [ r ij ] L × L , such that R can capture the transformation relationship from a given ground-truth label to the associated  candidatenoise labels and produce the candidate label set.
Revised: [ r ij ] L × L , such that R can capture the transformation relationships from a given ground-truth label to the associated noisy candidate labels and recover the candidate label set.

Original: The entry r ij of R reflects the probability of transforming  j -th label to a candidate label when the i -th label is the ground-truth.
Revised: The entry r ij of R reﬂects the probability of transforming the j -th label to a candidate label when the i -th label is the ground-truth.

Original: When r ij is large, the j -th label has a strong association relationship with the i -th ground-truth label and hence has a larger probability to be chosen as a candidate label.
Revised: When r ij is large, the j -th label has a strong association relationship with the i -th ground-truth label and hence has a larger probability to be chosen as a candidate label.

Original: To achieve the goal, we propose to produce the transformation matrix R by applying the transformation network  on the observed candidatepartial label matrix Y =
Revised: Towards this goal, we propose to produce the transformation matrix R by applying the transformation network T on the observed candidate label matrix Y =

Original: [ y ij ] n × L from the training set, such that R = T ( Y ) , since the observed partial label matrix contains both the true label and  candidatenoise labels.
Revised: [ y ij ] n × L from the training set, such that R = T ( Y ) , since the observed candidate label matrix contains both the true labels and the associated noisy candidate labels.

Original: We propose to learn the transformation network T  through prediction based candidate label matrix reconstruction.
Revised: The transformation network T can then be learned through prediction based candidate label matrix reconstruction.

Original: Specifically, we propose to reconstruct the normalized candidate label matrix by transforming the predicted outputs of the prediction network classifier F with the transformation matrix R .
Revised: Speciﬁcally, we propose to reconstruct the normalized candidate label matrix by transforming the prediction outputs of the prediction network based classiﬁer F with the transformation matrix R .

Original: We use the symmetric Kullback-Leibler (KL) divergenceloss to match the observed reconstructed candidate label distribution and the observed candidate label distribution through the following consistency loss: 
Revised: We use the symmetric Kullback-Leibler (KL) divergence to match each reconstructed candidate label distribution with its corresponding observed candidate label distribution and form the following consistency loss: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) Y i
Revised: where (cid:101) Y i

Original: =
Revised: =

Original: Y i / ∥
Revised: Y i / (cid:107)

Original: Y i ∥ 2 is a row normalized candidate label matrix in the training set.
Revised: Y i (cid:107) 2 is a row normalized candidate label matrix, expressing the candidate label distributions in the training set.

Original: It is however computational-intensive to recompute the transformation matrix R on the whole  matrix Y .
Revised: It is however computationally intensive to compute the transformation matrix R by taking the whole candidate label matrix Y as input.

Original: We  propose to compute and update R on each mini-batch in a moving averaging style, such as
Revised: We further propose to compute and update R on each mini-batch in a moving average style, such as

Original: [Equation]
Revised: [Equation]

Original: where R ′ is the transformation matrix computed from the observed partial label matrix within the current mini-batch, and η  is a momentum smooth coefficient.
Revised: where R (cid:48) is the transformation matrix computed from the observed partial label matrix on the current mini-batch, and η ∈ [0 , 1] is a momentum coefﬁcient.

Original: We expect the consistency loss in Eq.(8) can enhance the noise label correction and prediction network learning by exploiting the label association relationships between the true-label the candidate labels, thus consequently pushing the outputs of  prediction network towards the true labels.
Revised: By exploiting the label association relationships between the true-label and the candidate labels, we expect the consistency loss in Eq.(8) can enhance noisy label correction and prediction network learning, thus consequently pushing the outputs of the prediction network towards the true labels.

Original: 3. L EARNING WITH ML-PLL M ODEL
Revised: 3. L EARNING WITH ML-PLL M ODEL

Original: By integrating the classification loss in Eq.(6), the mutual learning loss in Eq.(7) and the consistency loss in Eq.(8) together, we get the following overall training loss for  proposed ML-PLL model, 
Revised: By integrating the classiﬁcation loss in Eq.(6), the mutual learning loss in Eq.(7) and the consistency loss in Eq.(8) together, we get the following overall training loss for the proposed ML-PLL model, [Equation]

Original: [Equation]
Revised: [Equation]

Original:  where α and β are trade-off hyperparameters that control the relative importance of the mutual learning loss and the consistency loss respectively.
Revised: ) where α and β are trade-off hyperparameters that control the relative importance of the mutual learning loss and the consistency loss, respectively.

Original: For optimization, we minimize this objective using a mini-batch-based stochastic gradient descent (SGD) algorithm.
Revised: We perform training by minimizing this objective with a mini-batch based stochastic gradient descent (SGD) algorithm.

Original: The full training algorithm of ML-PLL is outlined in Appendix A.1.

Original:  E XPERIMENT 4. E XPERIMENTAL S ETTING
Revised:  E XPERIMENT 4. E XPERIMENTAL S ETTING

Original: Datasets We perform the proposed ML-PLL on four widely used benchmark image datasets including Fashion-MNIST (Xiao et al., 2017), Kuzushiji-MNIST (Clanuwat et al., 2018), CIFARand CIFAR-100 (Krizhevsky et al., 2009).
Revised: Datasets We conducted experiments on four widely used benchmark image datasets: FashionMNIST (Xiao et al., 2017), Kuzushiji-MNIST (Clanuwat et al., 2018), CIFAR-10 and CIFAR- (Krizhevsky et al., 2009).

Original: We manually corrupt these datasets into partially labeled version by  competitive and uniform label noise(Lv et al., 2020) generation process.
Revised: We transform these datasets into partial-label datasets by using the following competitive and uniform label noise generation processes.

Original: To synthesize the competitive candidate noise labels, we  predict the probability of each category  being the true label corresponding to an instance by using a clean neural network, which can be trained on the original clean datasets.
Revised: First , to generate competitive label noise, we train a neural network on the original clean dataset to predict the probability of each category label being the true label for a given instance, and then randomly choose the irrelevant labels with high predicted probabilities as noisy candidate labels.

Original: Assume we have a clean training sample ( x i , (cid:98) y i ) , ρ i is its prediction probability of instance x i , y ij is the true label associated with x i .
Revised: Speciﬁcally, assume ˆp i is the predicted label probability vector for an instance x i whose true label is class j .

Original: Firstly, we set the probability of j -th label to zero, i.e., ρ ij = 0. Then, we sort ρ i in a descending order and keep its original category index.
Revised: We ﬁrst exclude the true label by setting ˆp ij = 0 , and then randomly choose labels from the top- K predictions in ˆp i —i.e., the K irrelevant label categories with highest predicted probabilities—as the noisy candidate labels.

Original: Intuitively, the irrelevant labels with higher prediction values demonstratethe stronger association relationshipswith the true label, which can be chosen as the competitive labels andhence are more likely to be chosen as the candidate labels.
Revised: Intuitively, these irrelevant labels with higher predicted probabilities demonstrate stronger association relationships and are more competitive with the true label.

Original: In our experiments, we choose the irrelevant labels within the top- k predictions as the competitive  labels,from which we randomly choose several competitive labels as the candidate labels that together with the true label (cid:98) y ij to form the candidate label set associated with the instance x i .
Revised: The chosen competitive noisy labels, together with the true label j , form the candidate label set for instance x i .

Original: Following above competitivecandidate noise label generation process, we corrupt the benchmark datasets Fashion-MNIST, Kuzushiji-MNIST and CIFAR-
Revised: Following this competitive

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original:  from the top-6 predictions  while CIFAR-100 from the top-20 predictions.
Revised: candidate label noise generation process, we produce partial-label datasets by choosing competitive label noise from the top-6 predictions for Fashion-MNIST, Kuzushiji-MNIST and CIFAR-10, while choosing from the top-20 predictions for CIFAR-100.

Original: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise generation process is recorded in Appendix A.4.
Revised: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise are reported in Appendix A.2.

Original: To synthesize the uniform candidate noise labels, we corrupt these datasets by the uniform generation procedureused in (Lv et al., 2020).
Revised: Second , we use the uniform generation procedure in (Lv et al., 2020) to generate uniform label noise.

Original: Specially, any irrelevant label can be chosen as a candidate  label with the probability ρ uniformly.
Revised: It generates noisy candidate labels by ﬂipping each incorrect label with a probability ρ , which indicates the label ambiguity level.

Original: When encountering a special case that noone irrelevant label is chosen, we randomly pick one irrelevant label as the  candidate label to ensure  all training set arefully corrupted.
Revised: When encountering a special case where no irrelevant label is chosen, we randomly select one irrelevant label as the additional candidate label to ensure the label sets for all training instances are corrupted.

Original: Comparison Methods We compare the proposed ML-PLL method with the following deep neural network based PLL methods: PiCO (Wang et al., 2022), a contrastive learning based  PLL method which disambiguates the candidate labels by  the refined pseudo-label in moving-average manner. LWS (Wen et al., 2021), a discriminative PLL method which proposes to trade-off the losses by mean of weighting the risk function on the candidate labels and non-candidate ones; CC (Feng et al., 2020b), a classifier-consistent method which leverages  transition matrix to induce an empirical risk estimator; MSE and EXP (Feng et al., 2020a) are two baselines that exploit mean square error and exponential loss as risks for inducingthe unbiased risk estimator.
Revised: Comparison Methods We compare the proposed ML-PLL method with the following deep neural network based PLL methods: PiCO (Wang et al., 2022b) is a contrastive learning based state-of-theart PLL method, which disambiguates the candidate labels by reﬁning the pseudo-labels in a movingaverage manner; LWS (Wen et al., 2021) is a discriminative PLL method, which considers the loss trade-off between the candidate labels and non-candidate ones; CC (Feng et al., 2020b) is a classiﬁerconsistent method, which leverages a transition matrix to induce an empirical risk estimator; MSE and EXP (Feng et al., 2020a) are two baselines that exploit mean square error and exponential loss, respectively, as risks for inducing unbiased risk estimators.

Original: For each comparison methods, we configure it with the same backbone structure for feature extraction and adopt the same predictive model for prediction.
Revised: For each comparison method, we use the same backbone structure for feature extraction and adopt the same predictive model.

Original: Moreover, all the comparison methods are configured with the suggested parameters according to the respective literatures.
Revised: Moreover, all the comparison methods are conﬁgured with the suggested parameters according to the source literature.

Original: For all the experiments, we report  average test accuracy and standard deviation based on 5 independent runs.
Revised: For each experiment, we report the average test accuracy and standard deviation based on 5 independent runs.

Original: Implementation Details We use LeNet-5 as  backbone on  Fashion-MNIST and Kuzushiji-MNIST datasets, and  18-layer ResNet as  backbone on CIFAR-10 and CIFAR-100 for feature extraction.
Revised: Implementation Details We use LeNet-5 as the backbone on the Fashion-MNIST and KuzushijiMNIST datasets, and use 18-layer ResNet as the backbone on CIFAR-10 and CIFAR-100, for feature extraction.

Original: The projection network is a two-layer MLP that outputs 128-dimensional embeddings.
Revised: The projection network is a two-layer MLP that outputs 128-dimensional embeddings.

Original: The prediction network is a two-layer MLP, while the Leaky ReLu activation function is used in the middle layer and a softmax activation is adopted in the output layer.
Revised: The prediction network is a linear classiﬁer.

Original: The weighted combination parameter λ in Eq.(1),  temperature parameter τ in Eq.(2),  momentum coefficient in Eq.(5) andmomentum smooth coefficient in Eq.(9) are set to 0.99, 0.07, 0.999 and 0.99 respectively.
Revised: The weighted combination parameter λ in Eq.(1), the temperature parameter τ in Eq.(2), and the momentum coefﬁcients in Eq.(5) and Eq.(9) are set to 0.99, 0.07, 0.999, and 0.99 respectively.

Original: For all comparison experiments, we employ a standard SGD optimizerwith a momentum of 0.9, a weight decay of 1e-3 for model training. The mini-batch size, learning rate and total epochs are set to 128, 0.01 and
Revised: In all the experiments, we utilize a standard SGD optimizer

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  respectively.
Revised: with a momentum of 0.9 and a weight decay of 1e-3 for model training. The mini-batch size, learning rate and total training epochs are set to 128, 0.01 and 400 respectively.

Original: The parameter α and β are chosen from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } and { 1 , 2 , 3 , 4 , 5 , 6 } respectively according to the accuracy on a validation dataset (10% of the training dataset)
Revised: The parameter α and β are chosen from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } and { 1 , 2 , 3 , 4 , 5 , 6 } , respectively, according to the accuracy on a validation dataset (10% of the training dataset)

Original: [Table]
Revised: [Table]

Original: 4. E XPERIMENTAL R ESULTS ON C OMPETITIVE L ABEL N OISE
Revised: 4. E XPERIMENTAL R ESULTS WITH C OMPETITIVE L ABEL N OISE

Original: We compared the proposed ML-PLL method with multiple state-of-art comparison methods on  fourwidely used benchmark datasets corrupted bythe competitive label noise and uniform label noise respectively.
Revised: We compared the proposed ML-PLL method with the ﬁve state-of-the-art comparison methods on the four datasets corrupted by competitive label noise. The comparison results on Fashion-MNIST,

Original: The comparison results on Fashion-MNIST, Kuzushiji-MNIST, and CIFAR-10 with competitive noise labels is reported in Table 1, from which we can see that the proposed method consistently outperforms all the  comparison methods, and the performance gains yield by ML-PLL overall the other methods are quite notable in many cases.
Revised: Kuzushiji-MNIST and CIFAR-10 with different label ambiguity levels, indicated by the average number of candidate labels, are reported in Table 1. From the table we can see that the proposed method consistently outperforms all the other comparison methods, and the performance gains yield by ML-PLL over the other methods are quite notable in many cases.

Original: For example, the proposed method outperforms the best alternative comparison method by 1.48%, and 2.83% where Avg.#CL = 4, 5on CIFAR-10 respectively.
Revised: For example, ML-PLL outperforms the best alternative comparison method by 1.48% and 2.83% on CIFAR-10 with Avg.#CL = 4 and 5 respectively.

Original: It is worth noting that with the increase of the average number of candidate labels, consequently boosts the ambiguity level, the performance gains become larger.

Original: This clearly demonstrates that the effectiveness of the proposed method in addressing the competitive label noise.

Original: In addition, we further challenge a complicated dataset CIFAR-100 with large label space ( L = 100).
Revised: The experimental results on the more challenging dataset CIFAR-100, which has a large label space ( L = 100), are reported in Table 2.

Original: The experimental results are reported in Table 2, from which we can see that the proposed method still outperforms all the  comparison methods and has a remarkable performance improvementeven on such a complicated dataset.
Revised: From the table we can see that the proposed ML-PLL again outperforms all the other comparison methods and demonstrates remarkable performance improvement on such a complicated dataset.

Original: For example, we improve upon the best comparison method by 1.53%, 3.49%, and 4.31% where Avg.#CL = 2, 6, 10 respectively.
Revised: For example, ML-PLL improves the best comparison method, PiCO, by 1.53%, 3.49%, and 4.31% respectively across the three label ambiguity levels.

Original: Moreover, the comparison methods like PiCO and LWS also perform well on the lower ambiguity levels, while the performance drops dramatically with the increase of the  ambiguity level.
Revised: Moreover, the comparison methods—e.g, PiCO, LWS, and CC—perform well with the low label ambiguity level (Avg.#CL=2), but their performance degrades dramatically with the increase of the label ambiguity level.

Original: By contrast, theperformance of the proposed ML-PLL method  consistently outperforms the other algorithms witha notable performance gain.
Revised: By contrast, the proposed ML-PLL method not only consistently outperforms the other methods with notable performance gains, but also produces larger gains with higher label ambiguity levels.

Original: This again validates the efficacy of  ML-PLL  in addressing the competitive label noise.
Revised: All these results validate the efﬁcacy of the proposed ML-PLL approach in addressing the competitive label noise.

Original: 4. E XPERIMENTAL R ESULTS ON U NIFORM L ABEL N OISE
Revised: 4. E XPERIMENTAL R ESULTS WITH U NIFORM L ABEL N OISE

Original: To further validate the effectiveness of the proposed method, we also perform experiments on Fashion-MNIST, Kuzushiji-MNIST, CIFAR-10 and CIFAR-100 corrupted by the  uniform label noise.
Revised: To further validate the effectiveness of the proposed method, we also conducted experiments on the four datasets (Fashion-MNIST, Kuzushiji-MNIST, CIFAR-10 and CIFAR-100) corrupted by the [Table] uniform label noise.

Revised: The label ambiguity level in this scenario is controlled by the ρ value—a larger ρ probability for producing noisy candidate labels leads to larger candidate label sets.

Original: All the comparison results are reported in Table 3 and Table 4.
Revised: All the comparison results are reported in Table 3 and Table 4.

Original: As shown in Table3, we
Revised: From Table

Original: [Table]
Revised: [Table]

Original:  can see that although the proposed method is not specially designed for uniform label noise, it still achievesthe competitive performance on the whole datasets.
Revised: we can see that although the proposed method is not specially designed for uniform label noise [Table] it still achieves competitive performance with the best method across all cases.

Original:  Table4 shows that the superiority of our proposed method is more significant compared with the other methods on such a complicated dataset; moreover, the performance gains are getting larger with the increase of the ambiguity level. For the complicated dataset CIFAR-100 with large label space, no matter in competitive label noise or uniform label noise learning scenario, the performance gains become larger with ambiguity level increases, i
Revised: In Table

Original: [Table]
Revised: [Table]

Original: e., the average number of candidate labels increase.
Revised: the proposed method even demonstrates substantial superiority over the other methods and produces larger performance gains with higher label ambiguity levels. These results on the challenging CIFAR-100 dataset are consistent with the ones reported in the scenario with competitive label noise—that is, the proposed ML-PLL method yields substantial and increasing performance gains when the label ambiguity level increases.

Original: All these results demonstrate the efficacy of the proposed method in handing  competitivelabel noise and uniform label noise for PLL
Revised: All these results demonstrate the efﬁcacy of the proposed method in handing both competitive and uniform label noise for PLL.

Original: [Table]

Original: 4. A BLATION S TUDY
Revised: 4. A BLATION S TUDY

Original: The objective of the proposed method contains three parts that contribute to PLL: the classification loss, the interactive mutual learning loss  and the consistency loss.
Revised: The objective of the proposed method contains three loss terms: the classiﬁcation loss, the interactive mutual learning loss [Table] and the consistency loss.

Original: In addition, noise label correction is an indispensable component in the proposed mutual learning model.
Revised: In addition, noisy label correction is an indispensable component in the proposed mutual learning model.

Original: To validate each part’s importance, we perform an ablation study to compare the proposed  method with  following ablation variants: 1) CLS-w/o-ml , which drops  mutual learning loss; 2) CLS-w/o-con, which drops  consistency loss; 3) CLS-w/o-lc, which drop noise label correction procedure; 4) CLS, which only uses classification loss by dropping  mutual learning loss,  consistency loss and noise label correction procedure.
Revised: To validate each part’s contribution, we conduct an ablation study to compare the proposed ML-PLL method with the following ablation variants: (1) ML-PLL-w/o-ml, which drops the mutual learning loss from the full model; (2) MLPLL-w/o-con, which drops the consistency loss; (3) ML-PLL-w/o-lc, which drops the noisy label correction procedure; and (4) CLS, which only uses a standard classiﬁcation loss by dropping the mutual learning loss, the consistency loss and the noisy label correction procedure.

Original: The comparison results are reported in Table 5.
Revised: The comparison results are reported in Table 5.

Original: From Table 5, we can see that compared with the full model, all  four variants produced inferior results, which demonstrates that all the components contribute to the full model in different degree.
Revised: From the table we can see that compared with the full model, all the four variants produced inferior results, which suggests that all the components contribute to the full model in different degrees.

Original: Among the three variants, CLS-w/o-ml, CLS-w/ocon and CLS-w/o-lc, we can see that CLS-w/o-lc, which drops the noise label correction procedure, produces a large performance degradation.
Revised: Among the three variants of ML-PLL-w/o-ml, ML-PLL-w/o-con and ML-PLL-w/o-lc, ML-PLL-w/o-lc produces the largest performance degradation, which demonstrates the importance of the noisy label correction procedure for label disambiguation and classiﬁer learning.

Original: This makes sense since dropping noise label correction means giving up correcting the competitive label noise, the prediction network based classifier has to be induced on the competitive noise labels and models a noise label distribution rather than the true label distribution.

Original: Among the two variants, CLS-w/o-ml and CLS-w/o-con, we can see that both of them produce a performancedegradation to some extent, since CLS-w/o-ml drops  the mutual learning loss that destroys the mutual learning framework, andCLS-w/o-con drops the  consistency loss that makes the proposed model can only address the uniform label noise while powerless to the competitive label noise.
Revised: Meanwhile, ML-PLL-w/o-ml and ML-PLL-w/o-con also produced consistent and very notable performance drops from the full model, which validates the contributions of both the mutual learning loss and the transformation network based consistency loss. The results suggest exploiting the association relationships between the competitive label noise and the ground-truth label can help

Original: Moreover, CLS-w/o-con produces more inferior performance than CLSw/
Revised: PLL.

Original: o-ml in most cases, which demonstrates the efficacy of the proposed transformation network in exploring the association relationships between the competitive noise labels and the ground-truth label. Overall, these ablation results demonstrates that the proposedML-PLL method is effective by integrating these parts for addressing partial label learning with competitive label noise.
Revised: Moreover, the large performance gap between the full model and the baseline variant CLS further validates the integrating power of the proposed method on combining all these three contributing components for PLL with competitive label noise: noisy label correction, mutual learning, and transformation network based label association exploitation.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: This paper considers a more realistic PLL scenario with competitive label noise which can be more difficult to distinguish with the ground-truth label thanthe random noise labels.
Revised: This paper tackles a more realistic PLL scenario with competitive label noise that can be more difﬁcult to distinguish from the ground-truth label than random noisy labels.

Original: We propose a novel mutual partial label learning approach called ML-PLL to address this challenging problem.
Revised: We propose a novel mutual partial label learning approach called ML-PLL to address this challenging problem.

Original: MLPLL learns a prediction network based classifier and a class-prototype based classifier cooperatively with label correction under the mutual learning framework.
Revised: ML-PLL learns a prediction network based classiﬁer and a class-prototype based classiﬁer cooperatively with label correction under a mutual learning framework.

Original: Moreover,  a transformation networkis introduced to model the association relationships between the ground-truth label and  candidate noise labels, and learn it together with the prediction network to match the observed candidate labels and enhance label correction.
Revised: Moreover, we introduce a transformation network to model the association relationships between the ground-truth label and the candidate noisy labels, and learn it together with the prediction network through candidate label set reconstruction to enhance label correction.

Original: The comprehensive experiments on several benchmarkPLL datasets  demonstrate that the proposed approach significantly outperforms the state-of-art PLL approaches. A A PPENDIX In this appendix we provide the complete implementation of our proposed approach in subsection A.1.
Revised: The comprehensive experiments on several benchmark datasets corrupted with both competitive and uniform label noise demonstrate that the proposed approach signiﬁcantly outperforms the state-of-the-art PLL methods.

Original: Moreover, the results of parameter sensitivity analysis is reported in subsection A.2, which studies how the trade-off parameters impact performance of the proposed model. Moreover, subsection A.3 conducts extended experiments on the CIFAR-10 dataset to explore how would the noise level affect performance of the proposed ML-PLL method. The characteristics of the dataset is recorded in A.4. In addition, an additional experiment is conducted to further validate the effectiveness of the proposed ML-PLLmethod in subsection A.5.
Revised: A A PPENDIX A. T RAINING A LGORITHM FOR ML-PLL

Original: We summarize the pseudo-code of our proposed ML-PLLmethod in Algorithm  Algorithm 
Revised: We present the mini-batch based training algorithm for ML-PLL in Algorithm 1. Algorithm 1 Training Algorithm for ML-PLL.

Original: Input :
Revised: do do

Original: // Noise label correction
Revised: Noisy label correction

Original: Get the class prototype-based prediction probability vector p i  via Eq.(2).
Revised: Compute the class prototype-based prediction probability vectors { p i } via Eq

Original: Conduct noise label correction via Eq.(1). Get normalized soft pseudo-label (cid:98) q i  via Eq.(3). Batch-
Revised: Perform noisy label correction to get { q i } via Eq

Original:  moving average  prototype update Computethe class prototype via Eq
Revised: Eq // Batch-wise moving average based prototype update Compute

Original: Update the class prototype viaEq.(5).
Revised: Update the class prototypes via

Original: // Update of the transformation matrix
Revised: // Update of the transformation matrix

Original: Compute and update the transformation matrix via Eq.(9).
Revised: Compute and update the transformation matrix via Eq

Original: // Classification loss calculation
Revised: // Classiﬁcation loss calculation

Original: [Equation]
Revised: L c ( F, G, H ) = 1 m (cid:80) x i ∼ B (cid:80) Lk =1 − (cid:98) q ik log( F k ( G ( x i ))) //

Original: Interactive mutual learning loss calculation
Revised: Interactive mutual learning loss calculation

Original: L [Equation] F, G, H ) 
Revised: L ml ( F, G, H ) =

Original: [Equation]{+1 m (cid:80) x i ∼ B (cid:80) Lk =
+}
Revised: {+1 m (cid:80) x i ∼ B (cid:80) Lk =
+}

Original: [Equation] i [Equation] p ik ) //
Revised: − F k ( G ( x i ))log( p ik ) //

Original: Consistency loss calculation
Revised: Consistency loss calculation

Revised: L con ( F, G, H, T ) =

Revised: (cid:80) x i ∼ B (cid:16) KL (cid:0) F ( G ( x i ))

Original: [Equation]
Revised: R, (cid:101) Y i (cid:1) + KL (cid:0)(cid:101) Y i , F ( G ( x i )) R (cid:1)(cid:17) //

Original: Network updating
Revised: Network updating

Original: [Equation]
Revised: L = L c + α L ml + β L con

Original: Update network F , T, G and H to minimize L .
Revised: Update network F , T G via stochastic gradient descent by minimizing L .

Original: end for 
Revised: end for end for

Revised: A

Revised: . D ATASET

Revised: We produce PLL datasets with different average number of candidate labels (Avg.#CL) from each benchmark dataset using the competitive label noise generation process described in subsection 4.1.

Revised: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise are recorded in Table 6.

Original: A. P ARAMETER S ENSITIVITY A NALYSIS
Revised: A. P ARAMETER S ENSITIVITY A NALYSIS

Original: In this subsection, we study the trade-off parameters α and β which controls the relative importance of the mutual learning loss andconsistency loss respectively, and perform the  experiments on CIFAR-10 dataset to studyhow would the trade-off parameters α and β affect the performance of ML-PLL.The experimental ranges for the parameters α and β are chosen from [Figure]
Revised: The trade-off parameters α and β control the relative importance of the mutual learning loss and the consistency loss, respectively. We conduct experiments on CIFAR-10 at the label ambiguity level of Avg#CL = 4 to study the impacts of α and β on the performance of ML-PLL.

Original:  9 , 1 }  and  { 1 , 2 , 3 , 4 , 5 , 6 }respectively. Note that a larger α means higher weight of the mutual learning loss; while a larger β means higher importance of the consistency loss. The experimental results with different parameter of α and β on the corrupted CIFAR-10 dataset with
Revised: We investigate the impact of α by experimenting with different α values from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } while setting β = 4 , and investigate the impact of β by experimenting with different β values from { 1 , 2 , 3 , 4 , 5 , 6 } while setting α = 0 .

Original: Avg#CL = 4 are reported in Figure 2(a) and Figure 2(b) respectively.
Revised: The results are presented in Figure 2(a) and Figure 2(b) respectively.

Original: We vary β while keep α with its best.
Revised: From Figure 2(a), we can see the best performance is achieved when α = 0 . 6 , while the performance difference is very small for varying α within [0 . 5 , 0 . But further increasing α towards 1 leads to performance degradation.

Original: As shown in Figure 2(b), we can see that when β is very small, the performance  is relatively poor.
Revised: The results in Figure 2(b) however demonstrate larger variations with different β values, suggesting the performance of ML-PLL is more sensitive to the β value. When β is very small, the performance of ML-PLL is relatively poor.

Original: With the increase of  β , the performance improves as the consistency loss begin to work.
Revised: With the increase of the β value, the performance improves as the consistency loss begins to work.

Original: This suggests that the consistency loss is an important loss term for the proposed model.

Original: When β is too large, the performance degrades  since the consistency loss dominates.
Revised: But when β is too large, the performance degrades again since the consistency loss can over-dominate.

Original: This make sense since we just hope the consistency loss to assist the model training, rather than dominates.
Revised: [Table]

Original: We can also observe the similar performance changes by varying the value of the trade-off parameter α .
Revised: [Figure]

Original: A. I MPACT OF C OMPETITIVE N OISE L EVELS
Revised: A. I MPACT OF C OMPETITIVE N OISE L EVELS

Original: In subsections 4.2, the experimental results has shown the effectiveness of the proposed method by enhancing the competitive levelthrough increasing number of the average candidate labels. In this subsection, we further validate its effectiveness by increasing the competitive level in another perspective: decreasing the  competitive label space.
Revised: As described in subsection 4.1, we generate the competitive label noise by randomly selecting irrelevant labels from the top- K predicted categories, where K controls the competitivity level of the chosen noisy candidate labels. A smaller K value forces the noisy candidate labels to be chosen from the fewer irrelevant labels with the highest few predicted probabilities of being the true label, and hence indicates a higher competitive noise level.

Original: We conduct an extension experiment to demonstrate how the performance ofPLL algorithms changes on thedifferent competitive level, the comparison results according to the accuracy on the corrupted CIFAR-10 dataset with Avg#CL = 4 are reported in Figure 3.
Revised: Here we conduct experiments on CIFAR-10 to investigate the impact of the competitive noise level by producing partial-label training data using different K values from { 4 , 5 , 6 , 8 } but the same average number of candidate labels, Avg#CL = 4.

Original: Note that k is the number of competitive label space, we can choose the candidate labels from the competitive label set. With the decrease of the competitive label space k , the competitive level increases since the noisy labels from the smaller competitive label space are more difficult for label disambiguation.
Revised: The comparison results are presented in Figure 3.

Original: As shown in Figure 3, we can see that with the decrease of the k , consequently  increase  the competitive  level, the performance of all the comparison methods[Table] [Table] degrades in general.
Revised: We can see that with the decrease of the K value and consequently the increase of the competitive noise level, the performance of all the comparison methods degrades in general.

Original: However [Table] the proposed ML-PLL method consistently outperforms all the otherfour comparison methods.
Revised: However, the proposed ML-PLL method consistently outperforms all the other comparison methods.

Original: It is worth noting that the performance of ML-PLLactually is relatively  stable  with the increase of the competitive  level (the decrease of the k ) [Table] and the performance gain becomes larger when the other methods encounter dramatically performance degradation.
Revised: Moreover, the performance of ML-PLL is relatively more stable across different K values, and its performance gains over the other methods become larger with the increase of the competitive noise level (the decrease of K ), while the other methods encounter more dramatic performance degradation with smaller K values.

Original: This again validates the effectiveness of ML-PLL in addressing competitive label noise.
Revised: These results again validate the effectiveness of ML-PLL in addressing competitive label noise.

Original: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise generation process is recorded in Table 6.
Revised: [Table]

Original: A.T HE I NFLUENCE OF N OISE L ABEL G ENERATION
Revised: A. I NFLUENCE OF N OISY L ABEL G ENERATION

Original: To further explore the impact of noise label generation on model performance, we follow (Wen et al., 2021) to conduct additional experiment with following setting: some irrelevant labels that have larger association relationships than other labels would have larger probability of label flipping ρ than other labels. We perform the experiments on CIFAR-10 with  following  commonly used learning cases:
Revised: To further investigate the impact of noisy label generation on model performance, we follow (Wen et al., 2021) to conduct additional experiments with alternative noisy label generation processes. Speciﬁcally, we generate noisy labels on CIFAR-10 with the following two commonly used label ﬂipping matrices in the form of [ ρ ij ] L × L (Wang et al., 2022b): (1)

Original: [Equation] [Equation]
Revised: =   0 . · · ·  0 . 5 · · ·...  · · ·  , (2) =

Original: [Equation]
Revised: [Table]

Original:  
Revised: 

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original: From the two flipping matrices we can see that each entry denotes the probability of  label being  a candidate label coupled with the ground-truth label.
Revised: [Table] [Table]   Each entry ρ ij of the ﬂipping matrix speciﬁes the probability of the j -th label being selected as a candidate label when the i -th label is the ground-truth label.

Revised: We conduct experiments with the partial label data generated using these two ﬂipping matrices and the comparison results are reported in Table 7, where case (1) and case (2) correspond to using the partial label data generated with the ﬂipping matrix (1) and (2), respectively.

Original: The comparison results is reported in Table 7, from which we can see that: in a simple case (1), the proposed ML-PLL method outperforms  the best comparison methods with a slight advantages; it is worth noting that in case (2), ML-PLL method still achieves a competitive performance and has a remarkable performanceimprovement even in such a harder case.
Revised: From the table, we can see that the proposed ML-PLL method outperforms all the other methods in both cases. As case (2) is more challenging than case (1), all the other comparison methods produce much inferior performance in case (2). However the proposed ML-PLL maintains similar good performance in both cases. In the simple case (1),

Revised: ML-PLL outperforms the best comparison method, LWS, by 4.09%, while in the more challenging case (2), ML-PLL outperforms the best comparison method, CC, by 23.58%.

Original: These additional experimental results further demonstrates the effectiveness of the proposed method in candidate label disambiguation.
Revised: These additional experimental results further validate the effectiveness of the proposed method in addressing the partial label learning problem.