Original: M UTUAL P ARTIAL L ABEL L EARNING WITH C OMPETITIVE L ABEL N OISE
Revised: M UTUAL P ARTIAL L ABEL L EARNING WITH C OMPETITIVE L ABEL N OISE

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Partial label learning (PLL) is an important weakly supervised learning problem, where each training instance is associated with a set of candidate labels that include both the true label and noise labels.
Revised: Partial label learning (PLL) is an important weakly supervised learning problem, where each training instance is associated with a set of candidate labels that include both the true label and noise labels.

Original: Most existing PLL methods assume the candidate noise labels are randomly chosen, which hardly holds in the realworld learning scenarios.
Revised: Most existing PLL methods assume the candidate noise labels are randomly chosen, which hardly holds in the realworld learning scenarios.

Original: In this paper, we consider a more realistic PLL scenario with competitive noise labels that are more difficult to distinguish from the true label than the random noise labels.
Revised: In this paper, we consider a more realistic PLL scenario with competitive noise labels that are more difficult to distinguish from the true label than the random noise labels.

Original: We propose a novel Mutual Learning based
Revised: We propose a novel Mutual Learning based

Original: PLL approach named ML-PLL to address this challenging problem.
Revised: PLL approach named ML-PLL to address this challenging problem.

Original: ML-PLL learns a prediction network based classifier and a class-prototype based classifier cooperatively through interactive mutual learning and label correction.
Revised: ML-PLL learns a prediction network based classifier and a class-prototype based classifier cooperatively through interactive mutual learning and label correction.

Original: Moreover, we use a transformation network to model the association relationships between the true label and candidate noise labels, and learn it together with the prediction network to match the observed candidate labels in the training data and enhance label correction.
Revised: Moreover, we use a transformation network to model the association relationships between the true label and candidate noise labels, and learn it together with the prediction network to match the observed candidate labels in the training data and enhance label correction.

Original: Extensive experiments are conducted on several benchmark PLL datasets, and the proposed ML-PLL approach demonstrates the state-of-the-art performance for partial label learning.
Revised: Extensive experiments are conducted on several benchmark PLL datasets, and the proposed ML-PLL approach demonstrates the state-of-the-art performance for partial label learning.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: As it is costly and difficult to annotate each instance with a precise label, weakly supervised learning (WSL) has been widely studied in recent years (Zhou, 2018), which includes, but not limited to, semi-supervised learning (Van Engelen & Hoos, 2020; Ouali et al., 2020), noise label learning (Natarajan et al., 2013;
Revised: As it is costly and difficult to annotate each instance with a precise label, weakly supervised learning (WSL) has been widely studied in recent years (Zhou, 2018), which includes, but not limited to, semi-supervised learning (Van Engelen & Hoos, 2020; Ouali et al., 2020), noise label learning (Natarajan et al., 2013;

Original: Feng et al., 2021), positive-unlabeled learning (Kiryo et al., 2017; Shu et al., 2020), and partial multi-label learning (Xie & Huang, 2018; Yan & Guo, 2021).
Revised: Feng et al., 2021), positive-unlabeled learning (Kiryo et al., 2017; Shu et al., 2020), and partial multi-label learning (Xie & Huang, 2018; Yan & Guo, 2021).

Original: Partial label learning (PLL) is a typical WSL problem and aims to learn a model from training samples with overcomplete labels; that is, each training sample is associated with a set of candidate labels that include both the true label and noise labels.
Revised: Partial label learning (PLL) is a typical WSL problem and aims to learn a model from training samples with overcomplete labels; that is, each training sample is associated with a set of candidate labels that include both the true label and noise labels.

Original: PLL has been widely applied in many real-world learning scenarios, including automatic face naming (Hüllermeier & Beringer, 2006; Zeng et al., 2013), web mining (Luo & Orabona, 2010), and multimedia content analysis (Zeng et al., 2013).
Revised: PLL has been widely applied in many real-world learning scenarios, including automatic face naming (Hüllermeier & Beringer, 2006; Zeng et al., 2013), web mining (Luo & Orabona, 2010), and multimedia content analysis (Zeng et al., 2013).

Original: Since the ground-truth label is hidden in the candidate label set and not available to the learning algorithms, the main challenge of PLL lies in candidate label disambiguation.
Revised: Since the ground-truth label is hidden in the candidate label set and not available to the learning algorithms, the main challenge of PLL lies in candidate label disambiguation.

Original: To address this challenge, two main label disambiguation strategies have been proposed: average-based disambiguation strategy and identification-based disambiguation strategy.
Revised: To address this challenge, two main label disambiguation strategies have been proposed: average-based disambiguation strategy and identification-based disambiguation strategy.

Original: Average-based disambiguation treats each candidate label equally in the model training phase and averages all the modeling outputs from each candidate label in the testing phase(Cour et al., 2011; Hüllermeier & Beringer, 2006; Zhang & Yu, 2015).
Revised: Average-based disambiguation treats each candidate label equally in the model training phase and averages all the modeling outputs from each candidate label in the testing phase(Cour et al., 2011; Hüllermeier & Beringer, 2006; Zhang & Yu, 2015).

Original: Although this strategy is simple and clear, it can make the true label overwhelmed by noise labels without sufficient differentiation, and lead to poor prediction performance.
Revised: Although this strategy is simple and clear, it can make the true label overwhelmed by noise labels without sufficient differentiation, and lead to poor prediction performance.

Original: The identificationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving different confidences for the candidate labels (Feng & An, 2018; 2019;
Revised: The identificationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving different confidences for the candidate labels (Feng & An, 2018; 2019;

Original: Yao et al., 2020b; Yu & Zhang, 2016; Xu et al., 2021).
Revised: Yao et al., 2020b; Yu & Zhang, 2016; Xu et al., 2021).

Original: The identification-based disambiguation approaches achieve relatively better prediction performance than the average-based disambiguation approaches by handling the candidate labels with discrimination, but they can still suffer from the potential drawback that the identified label may not be the true label, which could severely disrupt the subsequent model training.
Revised: The identification-based disambiguation approaches achieve relatively better prediction performance than the average-based disambiguation approaches by handling the candidate labels with discrimination, but they can still suffer from the potential drawback that the identified label may not be the true label, which could severely disrupt the subsequent model training.

Original: In addition, these existing methods are usually restricted to standard machine learning frameworks with liner or kernel-based approaches, which have difficulties in dealing with large-scale datasets.
Revised: In addition, these existing methods are usually restricted to standard machine learning frameworks with liner or kernel-based approaches, which have difficulties in dealing with large-scale datasets.

Original: Recently, deep learning has been widely used in dealing with PLL problems (Feng et al., 2020b;
Revised: Recently, deep learning has been widely used in dealing with PLL problems (Feng et al., 2020b;

Original: Wu et al., 2022).
Revised: Wu et al., 2022).

Original: For example, the work in (Feng et al., 2020b) proposes two PLL methods, riskconsistent and classifier-consistent algorithms, based on deep neural networks.
Revised: For example, the work in (Feng et al., 2020b) proposes two PLL methods, riskconsistent and classifier-consistent algorithms, based on deep neural networks.

Original: The methods in (Wen et al., 2021; Lv et al., 2020) try to progressively identify the ground-truth label during training procedures by employing a self-training technique.
Revised: The methods in (Wen et al., 2021; Lv et al., 2020) try to progressively identify the ground-truth label during training procedures by employing a self-training technique.

Original: A more recent work in (Wang et al., 2022) tackles PLL by using contrastive representation learning with class-prototype based label disambiguation.
Revised: A more recent work in (Wang et al., 2022) tackles PLL by using contrastive representation learning with class-prototype based label disambiguation.

Original: Although it achieves satisfactory prediction performance, the contrastive learning procedure is usually time-consuming and resource-demanding.
Revised: Although it achieves satisfactory prediction performance, the contrastive learning procedure is usually time-consuming and resource-demanding.

Original: Moreover, all these methods have a common drawback: they automatically assume random noise in the label space; that is, they assume the candidate noise labels are randomly sampled from a uniform generating procedure.
Revised: Moreover, all these methods have a common drawback: they automatically assume random noise in the label space; that is, they assume the candidate noise labels are randomly sampled from a uniform generating procedure.

Original: However, during the noise label creation procedure in real-world scenarios, such as crowd-sourcing, the noise labels are typically associated with the true label and dependent on the input sample, which makes the noise labels to be more difficult to distinguish from the ground-truth label than a random label.
Revised: However, during the noise label creation procedure in real-world scenarios, such as crowd-sourcing, the noise labels are typically associated with the true label and dependent on the input sample, which makes the noise labels to be more difficult to distinguish from the ground-truth label than a random label.

Original: In this paper, we consider a more realistic and challenging PLL learning scenario, where the noise labels are competitive and difficult to distinguish from the true label given the input data sample.
Revised: In this paper, we consider a more realistic and challenging PLL learning scenario, where the noise labels are competitive and difficult to distinguish from the true label given the input data sample.

Original: Intuitively, competitive noise labels can demonstrate stronger association relationships with a true label than a random label noise, and hence are more likely to be chosen as candidate labels.
Revised: Intuitively, competitive noise labels can demonstrate stronger association relationships with a true label than a random label noise, and hence are more likely to be chosen as candidate labels.

Original: For example, for online image annotation, when the object contained in an image is an “alpaca”, a competitive label “camel” could have a large probability to be chosen by an annotator with limited expertise due to the similar appearances of “alpaca” and “camel”, while labels such as “dog” or “duck” are less likely to be picked as part of the candidate label set due to their relatively weak association with the ground-truth “alpaca”.
Revised: For example, for online image annotation, when the object contained in an image is an “alpaca”, a competitive label “camel” could have a large probability to be chosen by an annotator with limited expertise due to the similar appearances of “alpaca” and “camel”, while labels such as “dog” or “duck” are less likely to be picked as part of the candidate label set due to their relatively weak association with the ground-truth “alpaca”.

Original: Motivated by the above consideration, we propose a novel and effective Mutual Partial Label Learning approach (ML-PLL) under the competitive label noise learning scenario, which learns a prediction network based classifier and class-prototype based classifier interactively through label correction and mutual learning.
Revised: Motivated by the above consideration, we propose a novel and effective Mutual Partial Label Learning approach (ML-PLL) under the competitive label noise learning scenario, which learns a prediction network based classifier and class-prototype based classifier interactively through label correction and mutual learning.

Original: Specially, ML-PLL performs noise label correction by integrating the outputs of both the prediction network classifier and the class-prototype based classifier, while using the corrected pseudo-labels as the targets for the prediction network training and using the outputs of the prediction network as the targets for the prototype-based classifier training.
Revised: Specially, ML-PLL performs noise label correction by integrating the outputs of both the prediction network classifier and the class-prototype based classifier, while using the corrected pseudo-labels as the targets for the prediction network training and using the outputs of the prediction network as the targets for the prototype-based classifier training.

Original: In addition, a transformation network is proposed to model the association relationships between the true label and the noise candidate labels, which can further enhance the classifier training with respect to ground-truth label disambiguation.
Revised: In addition, a transformation network is proposed to model the association relationships between the true label and the noise candidate labels, which can further enhance the classifier training with respect to ground-truth label disambiguation.

Original: Extensive experiments are conducted on benchmarks PLL datasets, while the proposed approach ML-PLL demonstrates the state-of-the-art performance for partial label learning.
Revised: Extensive experiments are conducted on benchmarks PLL datasets, while the proposed approach ML-PLL demonstrates the state-of-the-art performance for partial label learning.

Original:  R ELATED W ORK 2. S TANDARD P ARTIAL L ABEL L EARNING
Revised:  R ELATED W ORK 2. S TANDARD P ARTIAL L ABEL L EARNING

Original: The main challenge for dealing with PLL problem lies in how to disambiguate the candidate labels.
Revised: The main challenge for dealing with PLL problem lies in how to disambiguate the candidate labels.

Original: Average-based disambiguation strategy and identification-based disambiguation strategy are two main strategies deployed in PLL.
Revised: Average-based disambiguation strategy and identification-based disambiguation strategy are two main strategies deployed in PLL.

Original: The average-based disambiguation strategy treats each candidate label equally in the model induction and averages all the modeling outputs from all the candidate labels as the final prediction.
Revised: The average-based disambiguation strategy treats each candidate label equally in the model induction and averages all the modeling outputs from all the candidate labels as the final prediction.

Original: For example, the works in (Cour et al., 2011; Zhang & Yu, 2015) distinguish all the averaged candidate label prediction from the non-candidate ones.
Revised: For example, the works in (Cour et al., 2011; Zhang & Yu, 2015) distinguish all the averaged candidate label prediction from the non-candidate ones.

Original: The identificationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving confidence scores for all the candidate labels (Jin & Ghahramani, 2002;
Revised: The identificationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving confidence scores for all the candidate labels (Jin & Ghahramani, 2002;

Original: Feng & An, 2018; 2019; Yu & Zhang, 2016).
Revised: Feng & An, 2018; 2019; Yu & Zhang, 2016).

Original: For example, the works in (Zhang et al., 2016; Xu et al., 2019; Wang et al., 2021) try to identify the true label by employing iterative label refining procedures and leveraging the topological information in the feature space.
Revised: For example, the works in (Zhang et al., 2016; Xu et al., 2019; Wang et al., 2021) try to identify the true label by employing iterative label refining procedures and leveraging the topological information in the feature space.

Original: However, these methods may suffer from the cumulative errors induced in the error-prone label confidence estimation along the topological structure.
Revised: However, these methods may suffer from the cumulative errors induced in the error-prone label confidence estimation along the topological structure.

Original: In addition, a number of PLL methods propose to employ the off-the-shelf learning techniques to tackle PLL problems.
Revised: In addition, a number of PLL methods propose to employ the off-the-shelf learning techniques to tackle PLL problems.

Original: For the maximum likelihood technique, the likelihood of each PL training sample is defined over its candidate label set instead of its implicit ground-truth label (Jin & Ghahramani, 2002; Liu & Dietterich, 2012).
Revised: For the maximum likelihood technique, the likelihood of each PL training sample is defined over its candidate label set instead of its implicit ground-truth label (Jin & Ghahramani, 2002; Liu & Dietterich, 2012).

Original: For the k -nearest neighbor technique, the candidate labels from neighbor instances are aggregated to produce the final prediction on a test instance (Hüllermeier & Beringer, 2006;
Revised: For the k -nearest neighbor technique, the candidate labels from neighbor instances are aggregated to produce the final prediction on a test instance (Hüllermeier & Beringer, 2006;

Original: Gong et al., 2017; Zhang & Yu, 2015).
Revised: Gong et al., 2017; Zhang & Yu, 2015).

Original: For the maximum margin technique, the classification margin is defined over the predictive difference between the candidate labels and the non-candidate labels for each PL training sample (Nguyen & Caruana, 
Revised: For the maximum margin technique, the classification margin is defined over the predictive difference between the candidate labels and the non-candidate labels for each PL training sample (Nguyen & Caruana, 2008)(Yu & Zhang, 2016).

Original: Yu & Zhang, the boosting technique, the weight of each PL training instance and the confidence  of each candidate label being the ground- label are refined  round 
Revised: For the boosting technique, the weight of each PL training instance and the confidence value of each candidate label being the ground-truth label are refined in each boosting round (Tang & Zhang, 2017). For the error-correcting output codes (ECOC) technique, multiple binary classifiers to ECOC coding matrix are built based on the transformed binary (Zhang et al., 2017).

Original: 2. D EEP N EURAL N ETWORK BASED P ARTIAL L ABEL L EARNING
Revised: 2. D EEP N EURAL N ETWORK BASED P ARTIAL L ABEL L EARNING

Original: Standard PLL methods are usually restricted to liner based models, which has difficulty in dealing with large-scale datasets.
Revised: Standard PLL methods are usually restricted to liner based models, which has difficulty in dealing with large-scale datasets.

Original: Deep learning paradigm based PLL has recently started gaining attention from the research community due to its powerful capability of training on large-scale datasets.
Revised: Deep learning paradigm based PLL has recently started gaining attention from the research community due to its powerful capability of training on large-scale datasets.

Original: The work in (Yao et al., 2020a) adopts deep convolutional neural networks as backbones and exploits the temporal-ensembling technique for model training.
Revised: The work in (Yao et al., 2020a) adopts deep convolutional neural networks as backbones and exploits the temporal-ensembling technique for model training.

Original: The work in (Yao et al., 2020b) proposes to train two networks in a mutual learning manner, which helps both networks interact with each other and alleviate the error accumulation problem.
Revised: The work in (Yao et al., 2020b) proposes to train two networks in a mutual learning manner, which helps both networks interact with each other and alleviate the error accumulation problem.

Original: The work in (Lv et al., 2020) attempts to progressively identify the true label in a self-training manner, which can be compatibly trained with stochastic optimization.
Revised: The work in (Lv et al., 2020) attempts to progressively identify the true label in a self-training manner, which can be compatibly trained with stochastic optimization.

Original: The work in (Feng et al., 2020b) assumes that the probability of each noise label being chosen is uniform and presents two provably consistent methods: a risk-consistent method and a classifier-consistent method.
Revised: The work in (Feng et al., 2020b) assumes that the probability of each noise label being chosen is uniform and presents two provably consistent methods: a risk-consistent method and a classifier-consistent method.

Original: Another work in (Wen et al., 2021) induces a family of loss functions, which generalize the uniform generation procedures.
Revised: Another work in (Wen et al., 2021) induces a family of loss functions, which generalize the uniform generation procedures.

Original: In addition, a recent work in (Wang et al., 2022) handles PLL problems by conducting class prototype-based label disambiguation following the idea of contrastive representation learning (He et al., 2020;
Revised: In addition, a recent work in (Wang et al., 2022) handles PLL problems by conducting class prototype-based label disambiguation following the idea of contrastive representation learning (He et al., 2020;

Original: Li et al., 2020), which achieves impressive prediction performance.
Revised: Li et al., 2020), which achieves impressive prediction performance.

Original: Although the above approaches produce competitive performance, most of them assume the noise labels in the candidate label set are randomly produced, which however does not hold in many realworld learning scenarios.
Revised: Although the above approaches produce competitive performance, most of them assume the noise labels in the candidate label set are randomly produced, which however does not hold in many realworld learning scenarios.

Original: By contrast, we tackle a more realistic and challenging PLL scenario, where the noise candidate labels are competitive and difficult to distinguish from the true label.
Revised: By contrast, we tackle a more realistic and challenging PLL scenario, where the noise candidate labels are competitive and difficult to distinguish from the true label.

Original:  P ROPOSED A PPROACH
Revised:  P ROPOSED A PPROACH

Original: In this section, we delineate the details of our proposed mutual partial label learning (ML-PLL) model in the competitive label noise scenario.
Revised: In this section, we delineate the details of our proposed mutual partial label learning (ML-PLL) model in the competitive label noise scenario.

Original: Given a partial label training set D =
Revised: Given a partial label training set D =

Original: ( X, Y )
Revised: ( X, Y )

Original: = { ( x i , y i ) } ni
Revised: = { ( x i , y i ) } ni

Original: =1 , where x i ∈
Revised: =1 , where x i ∈

Original: X denotes the i -th instance and X denotes the input space; y i ⊆
Revised: X denotes the i -th instance and X denotes the input space; y i ⊆

Original: Y denotes the candidate label indicator vector associated with x i , and Y = { 1 , 2 , · · · , L } denotes the multi-class label space.
Revised: Y denotes the candidate label indicator vector associated with x i , and Y = { 1 , 2 , · · · , L } denotes the multi-class label space.

Original: The multiple 1 values in y i denotes either the ground-truth label or irrelevant label noise, which forms the candidate label set S i associated with x i .
Revised: The multiple 1 values in y i denotes either the ground-truth label or irrelevant label noise, which forms the candidate label set S i associated with x i .

Original: The task of PLL is to induce a multi-class classifier from the PLL training set D .
Revised: The task of PLL is to induce a multi-class classifier from the PLL training set D .

Original: We assume a challenging realistic PLL scenario, where the noise labels in the candidate label set are difficult to distinguish from the ground-truth label given the input instance.
Revised: We assume a challenging realistic PLL scenario, where the noise labels in the candidate label set are difficult to distinguish from the ground-truth label given the input instance.

Original: The proposed ML-PLL model is illustrated in Figure 1, which mainly comprises the following component networks: a feature extractor G that extract features from the input instance v =
Revised: The proposed ML-PLL model is illustrated in Figure 1, which mainly comprises the following component networks: a feature extractor G that extract features from the input instance v =

Original: G ( x ) ∈ R m ; a projection network H that maps the feature vector v into a low-dimensional
Revised: G ( x ) ∈ R m ; a projection network H that maps the feature vector v into a low-dimensional

Original: embedding z
Revised: embedding z

Original: = H ( v ) ∈ R d ; a prediction network based classifier F ; a class-prototype based classifier; and a transformation network T that estimates the association relationships between the true label and candidate labels.
Revised: = H ( v ) ∈ R d ; a prediction network based classifier F ; a class-prototype based classifier; and a transformation network T that estimates the association relationships between the true label and candidate labels.

Original: The prediction network based classifier and the class-prototype based classifier are learned together through noise label correction and interactive mutual learning, while the prediction network and the transformation network are learned together to match the observed candidate labels in the training data and enhance label disambiguation.
Revised: The prediction network based classifier and the class-prototype based classifier are learned together through noise label correction and interactive mutual learning, while the prediction network and the transformation network are learned together to match the observed candidate labels in the training data and enhance label disambiguation.

Original: We elaborate these components and the proposed approach in the following subsections.
Revised: We elaborate these components and the proposed approach in the following subsections.

Original: 3. N OISE L ABEL C ORRECTION
Revised: 3. N OISE L ABEL C ORRECTION

Original: The main challenge of PLL lies in that the ground-truth label coexists with the irreverent noise labels in the candidate label set and is unknown to the learning algorithm.
Revised: The main challenge of PLL lies in that the ground-truth label coexists with the irreverent noise labels in the candidate label set and is unknown to the learning algorithm.

Original: The key of PLL is to correct the candidate label vectors and identify the true label vectors.
Revised: The key of PLL is to correct the candidate label vectors and identify the true label vectors.

Original: Given an interactive pair of prediction network classifier F and prototype-based classifier, we propose the following simple mutual label correction procedure.
Revised: Given an interactive pair of prediction network classifier F and prototype-based classifier, we propose the following simple mutual label correction procedure.

Original: For any training instance x i , we correct the partial label indicator vector towards the true label indicator vector by exploiting the predicted pseudo-label vector.
Revised: For any training instance x i , we correct the partial label indicator vector towards the true label indicator vector by exploiting the predicted pseudo-label vector.

Original: Specifically, we produce the soft pseudo-label vector q i ∈ [0 , 1] L for x i by taking a weighted convex combination of the prediction network’s softmax output probability vector F ( v i ) and the class prototype-based prediction probability vector p i :
Revised: Specifically, we produce the soft pseudo-label vector q i ∈ [0 , 1] L for x i by taking a weighted convex combination of the prediction network’s softmax output probability vector F ( v i ) and the class prototype-based prediction probability vector p i :

Original: [Equation]
Revised: [Equation]

Original: where λ ∈ (0 , 1) is the combination weight parameter.
Revised: where λ ∈ (0 , 1) is the combination weight parameter.

Original: Here the prediction network F ( v i ) is built above the feature extractor G , such as F ( v i ) =
Revised: Here the prediction network F ( v i ) is built above the feature extractor G , such as F ( v i ) =

Original: F ( G ( x i )) , while p i is produced by the similaritybased class-prototype classifier as follows:
Revised: F ( G ( x i )) , while p i is produced by the similaritybased class-prototype classifier as follows:

Original: [Equation]
Revised: [Equation]

Original: where τ is a temperature parameter, L is the number of class categories, z i
Revised: where τ is a temperature parameter, L is the number of class categories, z i

Original: = H ( G ( x i )) denotes the low-dimensional embedding of the features for instance x i , and c k denotes the prototype for the k -th category, which will be elaborated in the next subsection.
Revised: = H ( G ( x i )) denotes the low-dimensional embedding of the features for instance x i , and c k denotes the prototype for the k -th category, which will be elaborated in the next subsection.

Original: Instead of using Euclidean distance, we use the dot-product operator “ · ” to compute the inner product similarity between the instance and the class prototypes in the embedding space for class prediction probability calculation.
Revised: Instead of using Euclidean distance, we use the dot-product operator “ · ” to compute the inner product similarity between the instance and the class prototypes in the embedding space for class prediction probability calculation.

Original: To maintain a effective soft pseudo-label vector for each instance and eliminate the noise predictions outside of the candidate label set, we restrict the soft pseudo-labels within the candidate label set and further re-normalize q i into (cid:98) q i with row normalization,
Revised: To maintain a effective soft pseudo-label vector for each instance and eliminate the noise predictions outside of the candidate label set, we restrict the soft pseudo-labels within the candidate label set and further re-normalize q i into (cid:98) q i with row normalization,

Original: [Equation]
Revised: [Equation]

Original: A corrected soft pseudo-label matrix (cid:98) Q can be obtained by performing the abovementioned noise label correction on all the n training instances, such that (cid:98) Q
Revised: A corrected soft pseudo-label matrix (cid:98) Q can be obtained by performing the abovementioned noise label correction on all the n training instances, such that (cid:98) Q

Original: =
Revised: =

Original: [ (cid:98) q ik ] n × L .
Revised: [ (cid:98) q ik ] n × L .

Original: With the progress of prediction network training and class-prototype update (elaborated in the next subsection), we expect the produced soft pseudo-label vector will be better aligned with the true label indicator vector, which can interactively provide a good target for the subsequent prediction network training and gradually mitigate the negative impact of the label noise during the training process.
Revised: With the progress of prediction network training and class-prototype update (elaborated in the next subsection), we expect the produced soft pseudo-label vector will be better aligned with the true label indicator vector, which can interactively provide a good target for the subsequent prediction network training and gradually mitigate the negative impact of the label noise during the training process.

Original: 3. B ATCH - WISE M OVING A VERAGE BASED P ROTOTYPE U PDATE
Revised: 3. B ATCH - WISE M OVING A VERAGE BASED P ROTOTYPE U PDATE

Original: The effectiveness of noise label correction relies on accurate class probability predictions, which depend on both the prediction network and the class-prototypes.
Revised: The effectiveness of noise label correction relies on accurate class probability predictions, which depend on both the prediction network and the class-prototypes.

Original: Instead of training an entirely independent prototype-based classifier, we propose to produce the class prototypes based on the outputs of the prediction network classifier.
Revised: Instead of training an entirely independent prototype-based classifier, we propose to produce the class prototypes based on the outputs of the prediction network classifier.

Original: Specifically, we compute the class prototypes in the low-dimensional embedding space from the partial label training set D in the following way:
Revised: Specifically, we compute the class prototypes in the low-dimensional embedding space from the partial label training set D in the following way:

Original: [Equation]
Revised: [Equation]

Original: where 1 is an indicator function, (cid:98)
Revised: where 1 is an indicator function, (cid:98)

Original: y i denotes predicted one-hot label vector for x i , which is determined by the output of the prediction network F and restricted to the candidate label set S i .
Revised: y i denotes predicted one-hot label vector for x i , which is determined by the output of the prediction network F and restricted to the candidate label set S i .

Original: In this way, we can automatically eliminate the prediction noise outside of the candidate labels.
Revised: In this way, we can automatically eliminate the prediction noise outside of the candidate labels.

Original: With the prototype computing method above, we can update the class-prototype by recomputing it during each training iteration.
Revised: With the prototype computing method above, we can update the class-prototype by recomputing it during each training iteration.

Original: However, such an update process will be computational-intensive and lead to intolerable training latency, since it requires going through the whole training set.
Revised: However, such an update process will be computational-intensive and lead to intolerable training latency, since it requires going through the whole training set.

Original: To address this problem, we propose to perform batch-wise moving average based prototype update; that is, we update class-prototype for each mini-batch by consulting the cluster centroid over each mini-batch in a moving averaging manner:
Revised: To address this problem, we propose to perform batch-wise moving average based prototype update; that is, we update class-prototype for each mini-batch by consulting the cluster centroid over each mini-batch in a moving averaging manner:

Original: [Equation]
Revised: [Equation]

Original: where c ′ k is the mean embedding for the k -th class within the current mini-batch, which can be computed with Eq.(4) by limiting D to the current mini-batch; γ is a momentum coefficient; and
Revised: where c ′ k is the mean embedding for the k -th class within the current mini-batch, which can be computed with Eq.(4) by limiting D to the current mini-batch; γ is a momentum coefficient; and

Original: Normalize( c ) =
Revised: Normalize( c ) =

Original: c / ∥ c ∥ 2 .
Revised: c / ∥ c ∥ 2 .

Original: We expect such a batch-wise prototype update method can be computational efficient and enhance the gradual adjustment and stability of the class prototypes.
Revised: We expect such a batch-wise prototype update method can be computational efficient and enhance the gradual adjustment and stability of the class prototypes.

Original: 3. P REDICTION N ETWORK T RAINING WITH M UTUAL L EARNING
Revised: 3. P REDICTION N ETWORK T RAINING WITH M UTUAL L EARNING

Original: With the noise label correction method above, the corrected soft pseudo-label produced can be treated as the target for prediction network training.
Revised: With the noise label correction method above, the corrected soft pseudo-label produced can be treated as the target for prediction network training.

Original: Thus, we can learn the prediction network based probabilistic classifier on training data D by minimizing the following classification loss:
Revised: Thus, we can learn the prediction network based probabilistic classifier on training data D by minimizing the following classification loss:

Original: [Equation]
Revised: [Equation]

Original: where k is the index of the class, (cid:98)
Revised: where k is the index of the class, (cid:98)

Original: q
Revised: q

Original: i is the corrected soft pseudo-label vector for the i -th instance, and F k ( · ) denotes the k -th entry of prediction network’s softmax outputs, indicating the predicted probability of the instance belonging to the k -th class.
Revised: i is the corrected soft pseudo-label vector for the i -th instance, and F k ( · ) denotes the k -th entry of prediction network’s softmax outputs, indicating the predicted probability of the instance belonging to the k -th class.

Original: It is expected that the prediction network based classifier F can be improved by training with the corrected soft pseudo-labels.
Revised: It is expected that the prediction network based classifier F can be improved by training with the corrected soft pseudo-labels.

Original: Meanwhile, we can learn an additional class-prototype based classifier supervised by the outputs of the prediction network by deploying the following interactive mutual learning loss:
Revised: Meanwhile, we can learn an additional class-prototype based classifier supervised by the outputs of the prediction network by deploying the following interactive mutual learning loss:

Original: [Equation]
Revised: [Equation]

Original: where p ik is the prediction probability produced by prototype based classifier using Eq.(2).
Revised: where p ik is the prediction probability produced by prototype based classifier using Eq.(2).

Original: We expect this mutual learning loss can enforce both classifiers to collaborate with each other and gradually lead to consistent predictions that can better align with underlying true label indicator vectors.
Revised: We expect this mutual learning loss can enforce both classifiers to collaborate with each other and gradually lead to consistent predictions that can better align with underlying true label indicator vectors.

Original: 3. T RANSFORMATION BASED L ABEL A SSOCIATION L EARNING
Revised: 3. T RANSFORMATION BASED L ABEL A SSOCIATION L EARNING

Original: As previously discussed, the appearances of noise labels in the candidate label set typically have an association relationship with the ground-truth label and can be competitive to the true label given the input instance in the real-world learning scenarios.
Revised: As previously discussed, the appearances of noise labels in the candidate label set typically have an association relationship with the ground-truth label and can be competitive to the true label given the input instance in the real-world learning scenarios.

Original: In this subsection, we propose to learn the competitive association relationship between noise candidate labels and the true label using a transformation network T , aiming to enhance the label disambiguation process.
Revised: In this subsection, we propose to learn the competitive association relationship between noise candidate labels and the true label using a transformation network T , aiming to enhance the label disambiguation process.

Original: We assume that the transformation network T can be learned to produce a transformation matrix R =
Revised: We assume that the transformation network T can be learned to produce a transformation matrix R =

Original: [ r ij ] L × L , such that R can capture the transformation relationship from a given ground-truth label to the associated candidate noise labels and produce the candidate label set.
Revised: [ r ij ] L × L , such that R can capture the transformation relationship from a given ground-truth label to the associated candidate noise labels and produce the candidate label set.

Original: The entry r ij of R reflects the probability of transforming j -th label to a candidate label when the i -th label is the ground-truth.
Revised: The entry r ij of R reflects the probability of transforming j -th label to a candidate label when the i -th label is the ground-truth.

Original: When r ij is large, the j -th label has a strong association relationship with the i -th ground-truth label and hence has a larger probability to be chosen as a candidate label.
Revised: When r ij is large, the j -th label has a strong association relationship with the i -th ground-truth label and hence has a larger probability to be chosen as a candidate label.

Original: To achieve the goal, we propose to produce the transformation matrix R by applying the transformation network on the observed candidate partial label matrix Y =
Revised: To achieve the goal, we propose to produce the transformation matrix R by applying the transformation network on the observed candidate partial label matrix Y =

Original: [ y ij ] n × L from the training set, such that R = T ( Y ) , since the observed partial label matrix contains both the true label and candidate noise labels.
Revised: [ y ij ] n × L from the training set, such that R = T ( Y ) , since the observed partial label matrix contains both the true label and candidate noise labels.

Original: We propose to learn the transformation network T through prediction based candidate label matrix reconstruction.
Revised: We propose to learn the transformation network T through prediction based candidate label matrix reconstruction.

Original: Specifically, we propose to reconstruct the normalized candidate label matrix by transforming the predicted outputs of the prediction network classifier F with the transformation matrix R .
Revised: Specifically, we propose to reconstruct the normalized candidate label matrix by transforming the predicted outputs of the prediction network classifier F with the transformation matrix R .

Original: We use the symmetric Kullback-Leibler (KL) divergence loss to match the observed reconstructed candidate label distribution and the observed candidate label distribution through the following consistency loss:
Revised: We use the symmetric Kullback-Leibler (KL) divergence loss to match the observed reconstructed candidate label distribution and the observed candidate label distribution through the following consistency loss:

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) Y i
Revised: where (cid:101) Y i

Original: =
Revised: =

Original: Y i / ∥
Revised: Y i / ∥

Original: Y i ∥ 2 is a row normalized candidate label matrix in the training set.
Revised: Y i ∥ 2 is a row normalized candidate label matrix in the training set.

Original: It is however computational-intensive to recompute the transformation matrix R on the whole matrix
Revised: It is however computational-intensive to recompute the transformation matrix R on the whole matrix

Original: Y .
Revised: Y .

Original: We propose to compute and update R on each mini-batch in a moving averaging style, such as
Revised: We propose to compute and update R on each mini-batch in a moving averaging style, such as

Original: [Equation]
Revised: [Equation]

Original: where R ′ is the transformation matrix computed from the observed partial label matrix within the current mini-batch, and η is a momentum smooth coefficient.
Revised: where R ′ is the transformation matrix computed from the observed partial label matrix within the current mini-batch, and η is a momentum smooth coefficient.

Original: We expect the consistency loss in Eq.(8) can enhance the noise label correction and prediction network learning by exploiting the label association relationships between the true-label the candidate labels, thus consequently pushing the outputs of prediction network towards the true labels.
Revised: We expect the consistency loss in Eq.(8) can enhance the noise label correction and prediction network learning by exploiting the label association relationships between the true-label the candidate labels, thus consequently pushing the outputs of prediction network towards the true labels.

Original: 3. L EARNING WITH ML-PLL M ODEL
Revised: 3. L EARNING WITH ML-PLL M ODEL

Original: By integrating the classification loss in Eq.(6), the mutual learning loss in Eq.(7) and the consistency loss in Eq.(8) together, we get the following overall training loss for proposed ML-PLL model,
Revised: By integrating the classification loss in Eq.(6), the mutual learning loss in Eq.(7) and the consistency loss in Eq.(8) together, we get the following overall training loss for proposed ML-PLL model,

Original: [Equation]
Revised: [Equation]

Original: where α and β are trade-off hyperparameters that control the relative importance of the mutual learning loss and the consistency loss respectively.
Revised: where α and β are trade-off hyperparameters that control the relative importance of the mutual learning loss and the consistency loss respectively.

Original: For optimization, we minimize this objective using a mini-batch-based stochastic gradient descent (SGD) algorithm.
Revised: For optimization, we minimize this objective using a mini-batch-based stochastic gradient descent (SGD) algorithm.

Original: The full training algorithm of ML-PLL is outlined in Algorithm 1 in Appendix.
Revised: The full training algorithm of ML-PLL is outlined in Appendix A.1.

Original:  E XPERIMENT 4. E XPERIMENTAL S ETTING
Revised:  E XPERIMENT 4. E XPERIMENTAL S ETTING

Original: Datasets We perform the proposed ML-PLL on four widely used benchmark image datasets including Fashion-MNIST (Xiao et al., 2017), Kuzushiji-MNIST (Clanuwat et al., 2018), CIFARand CIFAR-100 (Krizhevsky et al., 2009).
Revised: Datasets We perform the proposed ML-PLL on four widely used benchmark image datasets including Fashion-MNIST (Xiao et al., 2017), Kuzushiji-MNIST (Clanuwat et al., 2018), CIFARand CIFAR-100 (Krizhevsky et al., 2009).

Original: We manually corrupt these datasets into partially labeled version by competitive and uniform label noise (Lv et al., 2020) generation process.
Revised: We manually corrupt these datasets into partially labeled version by competitive and uniform label noise (Lv et al., 2020) generation process.

Original: To synthesize the competitive candidate noise labels, we predict the probability of each category being the true label corresponding to an instance by using a clean neural network, which can be trained on the original clean datasets.
Revised: To synthesize the competitive candidate noise labels, we predict the probability of each category being the true label corresponding to an instance by using a clean neural network, which can be trained on the original clean datasets.

Original: Assume we have a clean training sample ( x i , (cid:98) y i ) , ρ i is its prediction probability of instance x
Revised: Assume we have a clean training sample ( x i , (cid:98) y i ) , ρ i is its prediction probability of instance x

Original: i , y ij is the true label associated with x i .
Revised: i , y ij is the true label associated with x i .

Original: Firstly, we set the probability of j -th label to zero, i.e., ρ ij = 0.
Revised: Firstly, we set the probability of j -th label to zero, i.e., ρ ij = 0.

Original: Then, we sort ρ i in a descending order and keep its original category index.
Revised: Then, we sort ρ i in a descending order and keep its original category index.

Original: Intuitively, the irrelevant labels with higher prediction values demonstrate the stronger association relationships with the true label, which can be chosen as the competitive labels and hence are more likely to be chosen as the candidate labels.
Revised: Intuitively, the irrelevant labels with higher prediction values demonstrate the stronger association relationships with the true label, which can be chosen as the competitive labels and hence are more likely to be chosen as the candidate labels.

Original: In our experiments, we choose the irrelevant labels within the top- k predictions as the competitive labels, from which we randomly choose several competitive labels as the candidate labels that together with the true label (cid:98) y ij to form the candidate label set associated with the instance x i .
Revised: In our experiments, we choose the irrelevant labels within the top- k predictions as the competitive labels, from which we randomly choose several competitive labels as the candidate labels that together with the true label (cid:98) y ij to form the candidate label set associated with the instance x i .

Original: Following above competitive candidate noise label generation process, we corrupt the benchmark datasets Fashion-MNIST, Kuzushiji-MNIST and CIFARfrom the top-6 predictions while CIFAR-100 from the top-20 predictions. The average number of candidate labels (Avg.#CL) for each benchmark dataset corrupted by the competitive label noise
Revised: Following above competitive candidate noise label generation process, we corrupt the benchmark datasets Fashion-MNIST, Kuzushiji-MNIST and CIFAR-

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original:  generation process is recorded in Table 1.
Revised: from the top-6 predictions while CIFAR-100 from the top-20 predictions. The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise generation process is recorded in Appendix A.4.

Original: To synthesize the uniform candidate noise labels, we corrupt these datasets by the uniform generation procedure used in (Lv et al., 2020).
Revised: To synthesize the uniform candidate noise labels, we corrupt these datasets by the uniform generation procedure used in (Lv et al., 2020).

Original: Specially, any irrelevant label can be chosen as a candidate label with the probability ρ uniformly.
Revised: Specially, any irrelevant label can be chosen as a candidate label with the probability ρ uniformly.

Original: When encountering a special case that no one irrelevant label is chosen, we randomly pick one irrelevant label as the candidate label to ensure all training set are fully corrupted.
Revised: When encountering a special case that no one irrelevant label is chosen, we randomly pick one irrelevant label as the candidate label to ensure all training set are fully corrupted.

Original: Comparison Methods We compare the proposed ML-PLL method with the following deep neural network based PLL methods:  LWS (Wen et al., 2021), a discriminative PLL method which proposes to trade-off the losses by mean of weighting the risk function on the candidate labels and noncandidate ones; CC (Feng et al., 2020b), a classifier-consistent method which leverages transition matrix to induce an empirical risk estimator; MSE and EXP (Feng et al., 2020a) are two baselines that exploit mean square error and exponential loss as risks for inducing the unbiased risk estimator.
Revised: Comparison Methods We compare the proposed ML-PLL method with the following deep neural network based PLL methods: PiCO (Wang et al., 2022), a contrastive learning based PLL method which disambiguates the candidate labels by the refined pseudo-label in moving-average manner. LWS (Wen et al., 2021), a discriminative PLL method which proposes to trade-off the losses by mean of weighting the risk function on the candidate labels and non-candidate ones; CC (Feng et al., 2020b), a classifier-consistent method which leverages transition matrix to induce an empirical risk estimator; MSE and EXP (Feng et al., 2020a) are two baselines that exploit mean square error and exponential loss as risks for inducing the unbiased risk estimator.

Original: For each comparison methods, we configure it with the same backbone structure for feature extraction and adopt the same predictive model for prediction.
Revised: For each comparison methods, we configure it with the same backbone structure for feature extraction and adopt the same predictive model for prediction.

Original: Moreover, all the comparison methods are configured with the suggested parameters according to the respective literatures.
Revised: Moreover, all the comparison methods are configured with the suggested parameters according to the respective literatures.

Original: For all the experiments, we report average test accuracy and standard deviation based on 5 independent runs.
Revised: For all the experiments, we report average test accuracy and standard deviation based on 5 independent runs.

Original: Implementation Details We use LeNet-5 as backbone on Fashion-MNIST and Kuzushiji-MNIST datasets, and 18-layer ResNet as backbone on CIFAR-10 and CIFAR-100 for feature extraction.
Revised: Implementation Details We use LeNet-5 as backbone on Fashion-MNIST and Kuzushiji-MNIST datasets, and 18-layer ResNet as backbone on CIFAR-10 and CIFAR-100 for feature extraction.

Original: The projection network is a two-layer MLP that outputs 128-dimensional embeddings.
Revised: The projection network is a two-layer MLP that outputs 128-dimensional embeddings.

Original: The prediction network is a two-layer MLP, while the Leaky ReLu activation function is used in the middle layer and a softmax activation is adopted in the output layer.
Revised: The prediction network is a two-layer MLP, while the Leaky ReLu activation function is used in the middle layer and a softmax activation is adopted in the output layer.

Original: The weighted combination parameter λ in
Revised: The weighted combination parameter λ in

Original: Eq.(1), temperature parameter τ in Eq.(2), momentum coefficient in Eq.(5) and momentum smooth coefficient in Eq.(9) are set to 0.99, 0.07, 0.999 and 0.99 respectively.
Revised: Eq.(1), temperature parameter τ in Eq.(2), momentum coefficient in Eq.(5) and momentum smooth coefficient in Eq.(9) are set to 0.99, 0.07, 0.999 and 0.99 respectively.

Original: For all comparison experiments, we employ a standard SGD optimizer with a momentum of 0.9, a weight decay of 1e-3 for 
Revised: For all comparison experiments, we employ a standard SGD optimizer with a momentum of 0.9, a weight decay of 1e-3 for model training.

Revised: The mini-batch size, learning rate and total epochs are set to 128, 0.01 and

Original: [Table] [Table]
Revised: [Table] [Table]

Original: model training. The mini-batch size, learning rate and total epochs are set to 128, 0.01 andrespectively.
Revised: respectively.

Original: The parameter α and β are chosen from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } and { 1 , 2 , 3 , 4 , 5 , 6 } respectively according to the accuracy on a validation dataset (10% of the training dataset).
Revised: The parameter α and β are chosen from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } and { 1 , 2 , 3 , 4 , 5 , 6 } respectively according to the accuracy on a validation dataset (10% of the training dataset) [Table]

Original: 4. E XPERIMENTAL R ESULTS ON C OMPETITIVE L ABEL N OISE
Revised: 4. E XPERIMENTAL R ESULTS ON C OMPETITIVE L ABEL N OISE

Original: We compared the proposed ML-PLL method with multiple state-of-art comparison methods on four widely used benchmark datasets corrupted by the competitive label noise and uniform label noise respectively.
Revised: We compared the proposed ML-PLL method with multiple state-of-art comparison methods on four widely used benchmark datasets corrupted by the competitive label noise and uniform label noise respectively.

Original: Table 2 reports the comparison results on Fashion-MNIST, Kuzushiji-MNIST, and CIFAR-10 with competitive noise labels. Table 2 shows that the proposed method consistently outperforms all the comparison methods, and the performance gains yield by ML-PLL over all the other methods are quite notable in many cases.
Revised: The comparison results on Fashion-MNIST, Kuzushiji-MNIST, and CIFAR-10 with competitive noise labels is reported in Table 1, from which we can see that the proposed method consistently outperforms all the comparison methods, and the performance gains yield by ML-PLL over all the other methods are quite notable in many cases.

Original: For example, the proposed method outperforms the best alternative comparison method by 3.27%, 4.29%, and 17.14% where Avg.#CL =3, 4, 5 on
Revised: For example, the proposed method outperforms the best alternative comparison method by 1.48%, and 2.83% where Avg.#CL = 4, 5 on

Original: CIFAR-10 respectively.
Revised: CIFAR-10 respectively.

Original: It is worth noting that with the increase of the average number of candidate labels, consequently boosts the ambiguity level, the performance gains become larger.
Revised: It is worth noting that with the increase of the average number of candidate labels, consequently boosts the ambiguity level, the performance gains become larger.

Original: This clearly demonstrates that the effectiveness of the proposed method in addressing the competitive label noise.
Revised: This clearly demonstrates that the effectiveness of the proposed method in addressing the competitive label noise.

Original: Note that the previous work merely conduct a performance validation on datasets with a small label space ( L = 10) (Wen et al., 2021).

Original: We challenge this by conducting an additional experiments on

Original:  CIFAR-100 with a larger label space ( L = 100) and reports the comparison results in Table 3.
Revised: In addition, we further challenge a complicated dataset CIFAR-100 with large label space ( L = 100).

Original: The  Table 3 shows that the proposed method still outperforms all the comparison methods and has a remarkable performance improvement even on such a complicated dataset with a large label space.
Revised: The experimental results are reported in Table 2, from which we can see that the proposed method still outperforms all the comparison methods and has a remarkable performance improvement even on such a complicated dataset.

Original: For example, we improve upon the best comparison method by 4.1%, 6.56%, and 8.53% where Avg.#CL = 2, 6, 10 respectively.
Revised: For example, we improve upon the best comparison method by 1.53%, 3.49%, and 4.31% where Avg.#CL = 2, 6, 10 respectively.

Original: Moreover, the comparison methods like LWS and CC also perform well on the lower ambiguity levels, while the performance drops dramatically with the increase of the ambiguity level.
Revised: Moreover, the comparison methods like PiCO and LWS also perform well on the lower ambiguity levels, while the performance drops dramatically with the increase of the ambiguity level.

Original: By contrast, the performance of the proposed ML-PLL method is quite stable and robust against the competitive label noise.
Revised: By contrast, the performance of the proposed ML-PLL method consistently outperforms the other algorithms with a notable performance gain.

Original: This again validates the efficacy of the proposed method in addressing the competitive label noise.
Revised: This again validates the efficacy of ML-PLL in addressing the competitive label noise.

Original: 4. E XPERIMENTAL R ESULTS ON U NIFORM L ABEL N OISE
Revised: 4. E XPERIMENTAL R ESULTS ON U NIFORM L ABEL N OISE

Original: To further validate the effectiveness of the proposed method, we also perform experiments on
Revised: To further validate the effectiveness of the proposed method, we also perform experiments on

Original: Fashion-MNIST, Kuzushiji-MNIST, CIFAR-10 and CIFAR-100 corrupted by the uniform label noise.
Revised: Fashion-MNIST, Kuzushiji-MNIST, CIFAR-10 and CIFAR-100 corrupted by the uniform label noise.

Original: All the comparison results are reported in Table 4 and Table 5.
Revised: All the comparison results are reported in Table 3 and Table 4.

Original: As shown in Table 4, we
Revised: As shown in Table 3, we

Original: [Table]
Revised: [Table]

Original: can see that although the proposed method has a relatively small performance gain, it still outperforms all the other methods on the whole datasets.
Revised: can see that although the proposed method is not specially designed for uniform label noise, it still achieves the competitive performance on the whole datasets.

Original: Table 5 shows that the superiority of our proposed method is more significant compared with the other methodseven on such a complicated dataset; moreover, the performance gains are getting larger with the increase of the ambiguity level.
Revised: Table 4 shows that the superiority of our proposed method is more significant compared with the other methods on such a complicated dataset; moreover, the performance gains are getting larger with the increase of the ambiguity level.

Original: Note that the proposed method beats all the comparison methods with superior performance in both learning scenarios with different noise levels, which is not easy given different comparison methods with different strength on different datasets.

Original: Moreover, no matter in competitive label noise or uniform label noise learning scenario, the performance gains become larger with ambiguity level increases, i
Revised: For the complicated dataset CIFAR-100 with large label space, no matter in competitive label noise or uniform label noise learning scenario, the performance gains become larger with ambiguity level increases, i

Original: [Table]
Revised: [Table]

Original: e., the average number of candidate labels increase.
Revised: e., the average number of candidate labels increase.

Original: All these results demonstrate the efficacy of the proposed method in handing competitive label noise and uniform label noise for PLL
Revised: All these results demonstrate the efficacy of the proposed method in handing competitive label noise and uniform label noise for PLL

Original: [Table]
Revised: [Table]

Original: 4. A BLATION S TUDY
Revised: 4. A BLATION S TUDY

Original: The objective of the proposed method contains three parts that contribute to PLL: the classification loss, the interactive mutual learning loss and the consistency loss.
Revised: The objective of the proposed method contains three parts that contribute to PLL: the classification loss, the interactive mutual learning loss and the consistency loss.

Original: In addition, noise label correction is an indispensable component in the proposed mutual learning model.
Revised: In addition, noise label correction is an indispensable component in the proposed mutual learning model.

Original: To validate each part’s importance, we perform an ablation study to compare the proposed method with following ablation variants:
Revised: To validate each part’s importance, we perform an ablation study to compare the proposed method with following ablation variants:

Original: 1) CLS-w/o-ml, which drops mutual learning loss; 2) CLS-w/o-con, which drops consistency loss; 3) CLS-w/o-lc, which drop noise label correction procedure; 4) CLS, which only uses classification loss by dropping mutual learning loss, consistency loss and noise label correction procedure.
Revised: 1) CLS-w/o-ml , which drops mutual learning loss; 2) CLS-w/o-con, which drops consistency loss; 3) CLS-w/o-lc, which drop noise label correction procedure; 4) CLS, which only uses classification loss by dropping mutual learning loss, consistency loss and noise label correction procedure.

Original: The comparison results are reported in Table 6.
Revised: The comparison results are reported in Table 5.

Original: From Table 6, we can see that compared with the full model, all four variants produced inferior results, which demonstrates that all the components contribute to the full model in different degree.
Revised: From Table 5, we can see that compared with the full model, all four variants produced inferior results, which demonstrates that all the components contribute to the full model in different degree.

Original: Among the three variants, CLS-w/o-ml, CLS-w/ocon and CLS-w/o-lc, we can see that CLS-w/o-lc, which drops the noise label correction procedure, produces a large performance degradation.
Revised: Among the three variants, CLS-w/o-ml, CLS-w/ocon and CLS-w/o-lc, we can see that CLS-w/o-lc, which drops the noise label correction procedure, produces a large performance degradation.

Original: This makes sense since dropping noise label correction means giving up correcting the competitive label noise, the prediction network based classifier has to be induced on the competitive noise labels and models a noise label distribution rather than the true label distribution.
Revised: This makes sense since dropping noise label correction means giving up correcting the competitive label noise, the prediction network based classifier has to be induced on the competitive noise labels and models a noise label distribution rather than the true label distribution.

Original: Among the two variants, CLS-w/o-ml and CLS-w/o-con, we can see that both of them produce a performance degradation to some extent, since CLS-w/o-ml drops the mutual learning loss that destroys the mutual learning framework, and CLS-w/o-con drops the consistency loss that makes the proposed model can only address the uniform label noise while powerless to the competitive label noise.
Revised: Among the two variants, CLS-w/o-ml and CLS-w/o-con, we can see that both of them produce a performance degradation to some extent, since CLS-w/o-ml drops the mutual learning loss that destroys the mutual learning framework, and CLS-w/o-con drops the consistency loss that makes the proposed model can only address the uniform label noise while powerless to the competitive label noise.

Original: Moreover, CLS-w/o-con produces more inferior performance than CLSw/
Revised: Moreover, CLS-w/o-con produces more inferior performance than CLSw/

Original: o-ml in most cases, which demonstrates the efficacy of the proposed transformation network in exploring the association relationships between the competitive noise labels and the ground-truth label.
Revised: o-ml in most cases, which demonstrates the efficacy of the proposed transformation network in exploring the association relationships between the competitive noise labels and the ground-truth label.

Original: Overall, these ablation results demonstrates that the proposed ML-PLL method is effective by integrating these parts for addressing partial label learning with competitive label noise.
Revised: Overall, these ablation results demonstrates that the proposed ML-PLL method is effective by integrating these parts for addressing partial label learning with competitive label noise.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: This paper considers a more realistic PLL scenario with competitive label noise which can be more difficult to distinguish with the ground-truth label than the random noise labels.
Revised: This paper considers a more realistic PLL scenario with competitive label noise which can be more difficult to distinguish with the ground-truth label than the random noise labels.

Original: We propose a novel mutual partial label learning approach called ML-PLL to address this challenging problem.
Revised: We propose a novel mutual partial label learning approach called ML-PLL to address this challenging problem.

Original: MLPLL learns a prediction network based classifier and a class-prototype based classifier cooperatively with label correction under the mutual learning framework.
Revised: MLPLL learns a prediction network based classifier and a class-prototype based classifier cooperatively with label correction under the mutual learning framework.

Original: Moreover, a transformation network is introduced to model the association relationships between the ground-truth label and candidate noise labels, and learn it together with the prediction network to match the observed candidate labels and enhance label correction.
Revised: Moreover, a transformation network is introduced to model the association relationships between the ground-truth label and candidate noise labels, and learn it together with the prediction network to match the observed candidate labels and enhance label correction.

Original: The comprehensive experiments on several benchmark PLL datasets demonstrate that the proposed approach significantly outperforms the state-of-art PLL approaches.
Revised: The comprehensive experiments on several benchmark PLL datasets demonstrate that the proposed approach significantly outperforms the state-of-art PLL approaches.

Original: A A PPENDIX
Revised: A A PPENDIX

Original: In this appendix, we provide the complete implementation of our proposed approach in Algorithm 1.
Revised: In this appendix we provide the complete implementation of our proposed approach in subsection A.1.

Original: Moreover, the results of parameter sensitivity analysis is reported to study how the trade-off parameters impact performance of the proposed model.
Revised: Moreover, the results of parameter sensitivity analysis is reported in subsection A.2, which studies how the trade-off parameters impact performance of the proposed model.

Original: In addition, we conduct extended experiments on the CIFAR-10 dataset to show how would the noise level affect performance of the proposed 
Revised: Moreover, subsection A.3 conducts extended experiments on the CIFAR-10 dataset to explore how would the noise level affect performance of the proposed ML-PLL method.

Original:  ML-PLL method.
Revised: The characteristics of the dataset is recorded in A.4. In addition, an additional experiment is conducted to further validate the effectiveness of the proposed ML-PLL method in subsection A.5. A. P SEUDO - CODE OF ML-PLL We summarize the pseudo-code of our proposed ML-PLL method in Algorithm

Original: Algorithm
Revised: Algorithm

Original: Input :do
Revised: Input :

Original: Sample a batch B with m samples from

Original: // Noise label correction
Revised: // Noise label correction

Original: Get the class prototype-based prediction probability vector p i via Eq.(2).
Revised: Get the class prototype-based prediction probability vector p i via Eq.(2).

Original: Conduct noise label correction via Eq.(1).
Revised: Conduct noise label correction via Eq.(1).

Original: Get normalized soft pseudo-label (cid:98) q i via Eq.
Revised: Get normalized soft pseudo-label (cid:98) q i via Eq.(3).

Original: // Batch-wise moving averagebased prototype update
Revised: Batch- moving average

Original:  Compute the class prototype via Eq.(4). Update the class prototype via Eq.(5).
Revised: prototype update Compute the class prototype via Eq Update the class prototype via Eq.(5).

Original: // Update of the transformation matrix
Revised: // Update of the transformation matrix

Original: Compute and update the transformation matrix via Eq.(9).
Revised: Compute and update the transformation matrix via Eq.(9).

Original: // Classification loss calculation
Revised: // Classification loss calculation

Original: [Equation]
Revised: [Equation]

Original: Interactive mutual learning loss calculation
Revised: Interactive mutual learning loss calculation

Original: L ml ( F, G, H )=
Revised: L [Equation] F, G, H )

Original: [-1 m (cid:80) x i ∼ B (cid:80) Lk =
-]
Revised: [-1 m (cid:80) x i ∼ B (cid:80) Lk =
-][Equation]

Original: − F k ( G ( x i ))log( p ik ) //
Revised: [Equation] i [Equation] p ik ) //

Original: Consistency loss calculation
Revised: Consistency loss calculation

Original: L con ( F, G, H, T ) =

Original: (cid:80) x i ∼ B (cid:16) KL (cid:0) F ( G ( x i ))

Original: R, (cid:101) Y i (cid:1) + KL (cid:0)(cid:101) Y i , F ( G ( x i )) R (cid:1)(cid:17) //
Revised: [Equation]

Original: Network updating
Revised: Network updating

Original: L = L c + α L ml + β L con
Revised: [Equation]

Original: Update network F , T , G and H to minimize L .
Revised: Update network F , T , G and H to minimize L .

Original: end for
Revised: end for

Original: A. P ARAMETER S ENSITIVITY A NALYSIS
Revised: A. P ARAMETER S ENSITIVITY A NALYSIS

Original: In this subsection, we study the trade-off parameters α and β which controls the relative importance of the mutual learning loss and consistency loss respectively, and perform the experiments on CIFAR-10 dataset to study how would the trade-off parameters α and β affect the performance of ML-PLL.
Revised: In this subsection, we study the trade-off parameters α and β which controls the relative importance of the mutual learning loss and consistency loss respectively, and perform the experiments on CIFAR-10 dataset to study how would the trade-off parameters α and β affect the performance of ML-PLL.

Original: The experimental ranges for the parameters α and β are chosen from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 .
Revised: The experimental ranges for the parameters α and β are chosen from [Figure]

Original: 9 , 1 } and { 1 , 2 , 3 , 4 , 5 , 6 }
Revised: 9 , 1 } and { 1 , 2 , 3 , 4 , 5 , 6 }

Original: respectively.
Revised: respectively.

Original: Note that a larger α means higher weight of the mutual learning loss; while a larger β means higher importance of the consistency loss.
Revised: Note that a larger α means higher weight of the mutual learning loss; while a larger β means higher importance of the consistency loss.

Original: The experimental results with different parameter of α and β on the corrupted CIFAR-10 dataset with
Revised: The experimental results with different parameter of α and β on the corrupted CIFAR-10 dataset with

Original: Avg#CL = 4 are reported in Figure 2(a) and Figure 2(b) respectively.
Revised: Avg#CL = 4 are reported in Figure 2(a) and Figure 2(b) respectively.

Original: We vary β while keep α with its best.
Revised: We vary β while keep α with its best.

Original: As shown in Figure 2(b), we can see that when β is very small, the performance is relatively poor.
Revised: As shown in Figure 2(b), we can see that when β is very small, the performance is relatively poor.

Original: With the increase of β , the performance improves as the[Figure] consistency loss begin to work.
Revised: With the increase of β , the performance improves as the consistency loss begin to work.

Original: This suggests that the consistency loss is an important loss term for the proposed model.
Revised: This suggests that the consistency loss is an important loss term for the proposed model.

Original: When β is too large, the performance degrades since the consistency loss dominates.
Revised: When β is too large, the performance degrades since the consistency loss dominates.

Original: This make sense since we just hope the consistency loss to assist the model training, rather than dominates.
Revised: This make sense since we just hope the consistency loss to assist the model training, rather than dominates.

Original: We can also observe the similar performance changes by varying the value of the trade-off parameter α .
Revised: We can also observe the similar performance changes by varying the value of the trade-off parameter α .

Original: A. I MPACT OF C OMPETITIVE N OISE L EVELS
Revised: A. I MPACT OF C OMPETITIVE N OISE L EVELS

Original: In subsections 4.2, the experimental results has shown the effectiveness of the proposed method by enhancing the competitive level through increasing number of the average candidate labels.
Revised: In subsections 4.2, the experimental results has shown the effectiveness of the proposed method by enhancing the competitive level through increasing number of the average candidate labels.

Original: In this subsection, we further validate its effectiveness by increasing the competitive level in another perspective: decreasing the competitive label space.
Revised: In this subsection, we further validate its effectiveness by increasing the competitive level in another perspective: decreasing the competitive label space.

Original: We conduct an extension experiment to demonstrate how the performance of PLL algorithms changes on the different competitive level, the comparison results according to the accuracy on the corrupted CIFAR-10 dataset with Avg#CL = 4 are reported in Figure 3.
Revised: We conduct an extension experiment to demonstrate how the performance of PLL algorithms changes on the different competitive level, the comparison results according to the accuracy on the corrupted CIFAR-10 dataset with Avg#CL = 4 are reported in Figure 3.

Original: Note that k is the number of competitive label space, we can choose the candidate labels from the competitive label set.
Revised: Note that k is the number of competitive label space, we can choose the candidate labels from the competitive label set.

Original: With the decrease of the competitive label space k , the competitive level increases since the noisy labels from the smaller competitive label space are more difficult for label disambiguation.
Revised: With the decrease of the competitive label space k , the competitive level increases since the noisy labels from the smaller competitive label space are more difficult for label disambiguation.

Original: As shown in Figure 3, we can see that with the decrease of the k , consequently increase the competitive level, the performance of all the comparison methods degrades in general.
Revised: As shown in Figure 3, we can see that with the decrease of the k , consequently increase the competitive level, the performance of all the comparison meth- [Table] [Table] ods degrades in general.

Original: However, the proposed ML-PLL method consistently outperforms all the other four comparison methods.
Revised: However, the proposed ML-PLL method consistently outperforms all the other four comparison methods.

Original: It is worth noting that the performance of ML-PLL actually is quite stable with the increase of the competitive level (the decrease of the k ), and the performance gain becomes larger when the other methods encounter dramatically performance degradation.
Revised: It is worth noting that the performance of ML-PLL actually is quite stable with the increase of the competitive level (the decrease of the k ) [Table] and the performance gain becomes larger when the other methods encounter dramatically performance degradation.

Original: This again validates the effectiveness of ML-PLL in addressing competitive label noise.
Revised: This again validates the effectiveness of ML-PLL in addressing competitive label noise.

Revised: A. D ATASET

Revised: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise generation process is recorded in Table 6.

Revised: A. T HE I NFLUENCE OF N OISE L ABEL G ENERATION

Revised: To further explore the impact of noise label generation on model performance, we follow (Wen et al., 2021) to conduct additional experiment with following setting: some irrelevant labels that have larger association relationships than other labels would have larger probability of label flipping ρ than other labels.

Revised: We perform the experiments on CIFAR-10 with following commonly used learning cases:

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised:  

Revised: From the two flipping matrices we can see that each entry denotes the probability of label being a candidate label coupled with the ground-truth label.

Revised: The comparison results is reported in Table 7, from which we can see that: in a simple case (1), the proposed ML-PLL method outperforms the best comparison methods with a slight advantages; it is worth noting that in case (2), ML-PLL method still achieves a competitive performance and has a remarkable performance improvement even in such a harder case.

Revised: These additional experimental results further demonstrates the effectiveness of the proposed method in candidate label disambiguation.