Original: M UTUAL P ARTIAL L ABEL L EARNING WITH C OMPETITIVE L ABEL N OISE
Revised: M UTUAL P ARTIAL L ABEL L EARNING WITH C OMPETITIVE L ABEL N OISE

Original: A BSTRACT
Revised: A BSTRACT

Original: Partial label learning (PLL) is an important weakly supervised learning problem, where each training instance is associated with a set of candidate labels that include both the true label and additional noisy labels.
Revised: Partial label learning (PLL) is an important weakly supervised learning problem, where each training instance is associated with a set of candidate labels that include both the true label and additional noisy labels.

Original: Most existing PLL methods assume the candidate noisy labels are randomly chosen, which hardly holds in real-world learning scenarios.
Revised: Most existing PLL methods assume the candidate noisy labels are randomly chosen which hardly holds in real-world learning scenarios.

Original: In this paper, we consider a more realistic PLL scenario with competitive label noise that is more difﬁcult to distinguish from the true label than the random label noise.
Revised: In this paper, we consider a more realistic PLL scenario with competitive label noise that is more difficult to distinguish from the true label than the random label noise.

Original: We propose a novel Mutual Learning based PLL approach named ML-PLL to address this challenging problem.
Revised: We propose a novel Mutual Learning based PLL approach named ML-PLL to address this challenging problem.

Original: ML-PLL learns a prediction network based classiﬁer and a class-prototype based classiﬁer cooperatively through interactive mutual learning and label correction.
Revised: ML-PLL learns a prediction network based classifier and a class-prototype based classifier cooperatively through interactive mutual learning and label correction.

Original: Moreover, we use a transformation network to model the association relationships between the true label and candidate labels, and learn it together with the prediction network to match the observed candidate labels in the training data and enhance label correction.
Revised: Moreover, we use a transformation network to model the association relationships between the true label and candidate labels and learn it together with the prediction network to match the observed candidate labels in the training data and enhance label correction.

Original: Extensive experiments are conducted on several benchmark PLL datasets, and the proposed ML-PLL approach demonstrates state-of-the-art performance for partial label learning.
Revised: Extensive experiments are conducted on several benchmark PLL datasets and the proposed ML-PLL approach demonstrates state-of-the-art performance for partial label learning

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: As it is costly and difﬁcult to annotate each instance with a precise label weakly supervised learning (WSL) has been widely studied in recent years (Zhou, 2018), which includes, but not limited to, semi-supervised learning (Van Engelen & Hoos, 2020; Ouali et al., 2020), noisy label learning (Natarajan et al., 2013; Feng et al., 2021), positive-unlabeled learning (Kiryo et al., 2017; Shu et al., 2020), and partial multi-label learning (Xie & Huang, 2018; Yan & Guo, 2021).
Revised: As it is costly and difficult to annotate each instance with a precise label, weakly supervised learning (WSL) has been widely studied in recent years (Zhou, 2018), which includes, but not limited to, semi-supervised learning (Van Engelen & Hoos, 2020; Ouali et al., 2020), noisy label learning (Natarajan et al., 2013; Feng et al., 2021), positive-unlabeled learning (Kiryo et al., 2017; Shu et al., 2020), and partial multi-label learning (Xie & Huang, 2018; Yan & Guo, 2021).

Original: Partial label learning (PLL) is a typical WSL problem and aims to learn a model from training samples with overcomplete labels; that is, each training sample is associated with a set of candidate labels that include both the true label and additional label noise—noisy labels.
Revised: Partial label learning (PLL) is a typical WSL problem and aims to learn a model from training samples with overcomplete labels; that is, each training sample is associated with a set of candidate labels that include both the true label and additional label noise—noisy labels.

Original: PLL has been widely applied in many real-world learning scenarios, including automatic face naming (Hüllermeier & Beringer, 2006; Zeng et al., 2013), web mining (Luo & Orabona, 2010), and multimedia content analysis (Zeng et al., 2013).
Revised: PLL has been widely applied in many real-world learning scenarios, including automatic face naming (Hüllermeier & Beringer, 2006; Zeng et al., 2013), web mining (Luo & Orabona, 2010), and multimedia content analysis (Zeng et al., 2013).

Original: Since the ground-truth label is hidden in the candidate label set and not available to the learning algorithms, the main challenge of PLL lies in candidate label disambiguation.
Revised: Since the ground-truth label is hidden in the candidate label set and not available to the learning algorithms, the main challenge of PLL lies in candidate label disambiguation.

Original: To address this challenge, two main label disambiguation strategies have been proposed: average-based disambiguation strategy and identiﬁcation-based disambiguation strategy.
Revised: To address this challenge, two main label disambiguation strategies have been proposed: average-based disambiguation strategy and identification-based disambiguation strategy.

Original: Average-based disambiguation treats each candidate label equally in the model training phase and averages all the modeling outputs from each candidate label in the testing phase (Cour et al., 2011; Hüllermeier & Beringer, 2006; Zhang & Yu, 2015).
Revised: Average-based disambiguation treats each candidate label equally in the model training phase and averages all the modeling outputs from each candidate label in the testing phase (Cour et al., 2011; Hüllermeier & Beringer, 2006; Zhang & Yu, 2015).

Original: Although this strategy is simple and clear, it can make the true label overwhelmed by noisy labels without sufﬁcient differentiation, and lead to poor prediction performance.
Revised: Although this strategy is simple and clear, it can make the true label overwhelmed by noisy labels without sufficient differentiation, and lead to poor prediction performance.

Original: The identiﬁcationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving different conﬁdence scores for the candidate labels (Feng & An, 2018; 2019; Yao et al., 2020b; Yu & Zhang, 2016;
Revised: The identificationbased disambiguation strategy treats the ground-truth label as a latent variable and tries to identify the true label by deriving different confidence scores for the candidate labels (Feng & An, 2018; 2019; Yao et al., 2020b; Yu & Zhang, 2016;

Original: Xu et al., 2021).
Revised: Xu et al., 2021).

Original: The identiﬁcation-based disambiguation approaches are able to achieve relatively better prediction performance than the average-based disambiguation approaches by handling the candidate labels with discrimination, but they can still suffer from the potential drawback of accumulating label identiﬁcation errors and severely disrupting the subsequent model training.
Revised: The identification-based disambiguation approaches are able to achieve relatively better prediction performance than the average-based disambiguation approaches by handling the candidate labels with discrimination, but they can still suffer from the potential drawback of accumulating label identification errors and severely disrupting the subsequent model training.

Original: In addition, these existing methods are usually restricted to standard machine learning frameworks with linear or kernel-based approaches, which have difﬁculties to deal with large-scale datasets.
Revised: In addition, these existing methods are usually restricted to standard machine learning frameworks with linear or kernel-based approaches, which have difficulties to deal with large-scale datasets.

Original: Recently, deep learning has been widely used in addressing PLL problems (Feng et al., 2020b;
Revised: Recently, deep learning has been widely used in addressing PLL problems (Feng et al., 2020b;

Original: Wu et al., 2022).
Revised: Wu et al., 2022).

Original: For example, the work in (Feng et al., 2020b) proposes two PLL methods, a risk-consistent and a classiﬁer-consistent algorithms, based on deep neural networks.
Revised: For example, the work in (Feng et al., 2020b) proposes two PLL methods, a risk-consistent and a classifier-consistent algorithms, based on deep neural networks.

Original: The methods in (Wen et al., 2021; Lv et al., 2020) try to progressively identify the ground-truth label during training procedures by employing a self-training technique.
Revised: The methods in (Wen et al., 2021; Lv et al., 2020) try to progressively identify the ground-truth label during training procedures by employing a self-training technique.

Original: A more recent work in (Wang et al., 2022b) tackles PLL by using contrastive representation learning with class-prototype based label disambiguation.
Revised: A more recent work in (Wang et al., 2022) tackles PLL by using contrastive representation learning with class-prototype based label disambiguation.

Original: Although it achieves satisfactory prediction performance, the contrastive learning procedure is time-consuming and resource-demanding.
Revised: Although it achieves satisfactory prediction performance, the contrastive learning procedure is time-consuming and resource-demanding.

Original: Moreover, all these methods have a common drawback: they automatically assume random noise in the label space; that is, they assume the candidate noisy labels are randomly sampled from a uniform generating procedure.
Revised: Moreover, all these methods have a common drawback: they automatically assume random noise in the label space; that is, they assume the candidate noisy labels are randomly sampled from a uniform generating procedure.

Original: However, during the noisy label creation process in real-world scenarios, such as crowd-sourcing, the noisy labels are typically associated with the true label and dependent on the input sample, which makes the label noise more difﬁcult to distinguish from the ground-truth label than a random label.
Revised: However, during the noisy label creation process in real-world scenarios, such as crowd-sourcing, the noisy labels are typically associated with the true label and dependent on the input sample, which makes the label noise more difficult to distinguish from the ground-truth label than a random label.

Original: In this paper, we consider a more realistic and challenging PLL learning scenario, where the noisy labels are competitive and difﬁcult to distinguish from the true label given the input data sample.
Revised: In this paper, we consider a more realistic and challenging PLL learning scenario, where the noisy labels are competitive and difficult to distinguish from the true label given the input data sample.

Original: Intuitively, competitive label noise can demonstrate stronger association relationships with a true label than a random label noise, and hence are more likely to be chosen as candidate labels.
Revised: Intuitively, competitive label noise can demonstrate stronger association relationships with a true label than a random label noise, and hence are more likely to be chosen as candidate labels.

Original: For example, for online image annotation, when the object contained in an image is an “alpaca”, a competitive label “camel” could have a large probability to be chosen by an annotator with limited expertise due to the similar appearances of “alpaca” and “camel”, while labels such as “dog” or “duck” are less likely to be picked as part of the candidate label set due to their relatively weak association with the ground-truth “alpaca”.
Revised: For example, for online image annotation, when the object contained in an image is an “alpaca”, a competitive label “camel” could have a large probability to be chosen by an annotator with limited expertise due to the similar appearances of “alpaca” and “camel”, while labels such as “dog” or “duck” are less likely to be picked as part of the candidate label set due to their relatively weak association with the ground-truth “alpaca”.

Original: Motivated by the above consideration, we propose a novel and effective Mutual Partial Label Learning approach (ML-PLL) under the competitive label noise learning scenario, which learns a prediction network based classiﬁer and a class-prototype based classiﬁer interactively through label correction and mutual learning.
Revised: Motivated by the above consideration, we propose a novel and effective Mutual Partial Label Learning approach (ML-PLL) under the competitive label noise learning scenario, which learns a prediction network based classifier and a class-prototype based classifier interactively through label correction and mutual learning.

Original: Speciﬁcally, ML-PLL performs noisy label correction by integrating the outputs of both the prediction network classiﬁer and the class-prototype based classiﬁer, while using the corrected pseudo-labels as the targets for the prediction network training and using the output of the prediction network as target for the class-prototype based classiﬁer training.
Revised: Specifically, ML-PLL performs noisy label correction by integrating the outputs of both the prediction network classifier and the class-prototype based classifier, while using the corrected pseudo-labels as the targets for the prediction network training and using the output of the prediction network as target for the class-prototype based classifier training.

Original: In addition, a transformation network is proposed to model the association relationships between the true label and the noisy candidate labels, which can further enhance the classiﬁer training with respect to ground-truth label disambiguation.
Revised: In addition, a transformation network is proposed to model the association relationships between the true label and the noisy candidate labels, which can further enhance the classifier training with respect to ground-truth label disambiguation.

Original: Extensive experiments are conducted on several benchmark
Revised: Extensive experiments are conducted on several benchmark

Original: PLL datasets, while the proposed approach, ML-PLL, demonstrates state-of-the-art performance.
Revised: PLL datasets, while the proposed approach, ML-PLL, demonstrates state-of-the-art performance.

Original:  R ELATED W ORK 2. S TANDARD P ARTIAL L ABEL L EARNING
Revised:  R ELATED W ORK 2. S TANDARD P ARTIAL L ABEL L EARNING

Original: The main challenge for addressing the PLL problem lies in how to disambiguate the candidate labels.
Revised: The main challenge for addressing the PLL problem lies in how to disambiguate the candidate labels.

Original: Average-based disambiguation strategy and identiﬁcation-based disambiguation strategy are two main strategies deployed in PLL.
Revised: Average-based disambiguation strategy and identification-based disambiguation strategy are two main strategies deployed in PLL.

Original: The average-based disambiguation strategy treats each candidate label equally in the model induction and averages all the modeling outputs from all the candidate labels as the ﬁnal prediction.
Revised: The average-based disambiguation strategy treats each candidate label equally in the model induction and averages all the modeling outputs from all the candidate labels as the final prediction.

Original: For example, the works in (Cour et al., 2011; Zhang & Yu, 2015) distinguish the averaged candidate label prediction from the non-candidate ones.
Revised: For example, the works in (Cour et al., 2011; Zhang & Yu, 2015) distinguish the averaged candidate label prediction from the non-candidate ones.

Original: The identiﬁcation-based disambiguation strategy treats the ground-truth label as a latent variable and identiﬁes the true label by deriving conﬁdence scores for all the candidate labels (Feng & An, 2018; 2019; Yu &Zhang, 2016).
Revised: The identificationbased disambiguation strategy treats the ground-truth label as a latent variable and identifies the true label by deriving confidence scores for all the candidate labels (Feng & An, 2018; 2019; Yu &

Original:  For example, the works in (Zhang et al., 2016; Xu et al., 2019; Wang et al., 2022a) try to identify the true label by employing iterative label reﬁning procedures and leveraging the topological information in the feature space.
Revised: Zhang, 2016). For example, the works in (Zhang et al., 2016; Xu et al., 2019; Wang et al., 2021) try to identify the true label by employing iterative label refining procedures and leveraging the topological information in the feature space.

Original: However, these methods may suffer from the cumulative errors induced in the error-prone label conﬁdence estimation along the topological structure.
Revised: However, these methods may suffer from the cumulative errors induced in the error-prone label confidence estimation along the topological structure.

Original: In addition, a number of PLL methods propose to employ the off-the-shelf learning techniques, such as maximum likelihood, k-nearest-neighbor, maximum margin, boosting, and error-correcting output codes (ECOC), to tackle PLL problems.
Revised: In addition, a number of PLL methods propose to employ the off-the-shelf learning techniques, such as maximum likelihood, k-nearest-neighbor, maximum margin, boosting, and error-correcting output codes (ECOC), to tackle PLL problems.

Original: For the maximum likelihood technique, the likelihood of each PL training sample is deﬁned over its candidate label set instead of its implicit ground-truth label (Liu & Dietterich, 2012).
Revised: For the maximum likelihood technique, the likelihood of each PL training sample is defined over its candidate label set instead of its implicit ground-truth label (Liu & Dietterich, 2012).

Original: For the k -nearest neighbor technique, the candidate labels from neighbor instances are aggregated to produce the ﬁnal prediction on a test instance (Hüllermeier &Beringer, 2006;
Revised: For the k -nearest neighbor technique, the candidate labels from neighbor instances are aggregated to produce the final prediction on a test instance (Hüllermeier &

Original:  Gong et al., 2017; Zhang & Yu, 2015).
Revised: Beringer, 2006; Gong et al., 2017; Zhang & Yu, 2015).

Original: For the maximum margin technique, the classiﬁcation margin is deﬁned over the predictive difference between the candidate labels and the non-candidate labels for each PL training sample (Nguyen & Caruana, 2008)(Yu & Zhang, 2016).
Revised: For the maximum margin technique, the classification margin is defined over the predictive difference between the candidate labels and the non-candidate labels for each PL training sample (Nguyen & Caruana, 2008)(Yu & Zhang, 2016).

Original: For the boosting technique, the weight of each PL training instance and the conﬁdence value of each candidate label being the truth label are reﬁned in each boosting round (Tang & Zhang, 2017).
Revised: For the boosting technique, the weight of each PL training instance and the confidence value of each candidate label being the truth label are refined in each boosting round (Tang & Zhang, 2017).

Original: For the ECOC technique, the binary ECOC classiﬁers are built based on the binary training sets ﬁltered by the candidate labels (Zhang et al., 2017).
Revised: For the ECOC technique, the binary ECOC classifiers are built based on the binary training sets filtered by the candidate labels (Zhang et al., 2017).

Original: 2. D EEP N EURAL N ETWORK BASED P ARTIAL L ABEL L EARNING
Revised: 2. D EEP N EURAL N ETWORK BASED P ARTIAL L ABEL L EARNING

Original: Standard PLL methods are usually restricted to linear based models, which have difﬁculty to deal with large-scale datasets.
Revised: Standard PLL methods are usually restricted to linear based models, which have difficulty to deal with large-scale datasets.

Original: Deep learning based PLL has recently started gaining attention from the research community due to its powerful capability of training on large-scale datasets.
Revised: Deep learning based PLL has recently started gaining attention from the research community due to its powerful capability of training on large-scale datasets.

Original: Yao et al.
Revised: Yao et al.

Original: (2020a) adopt deep convolutional neural networks as backbones and exploit the temporalensembling technique for model training.
Revised: (2020a) adopt deep convolutional neural networks as backbones and exploit the temporalensembling technique for model training.

Original: Yao et al.
Revised: Yao et al.

Original: (2020b) propose to train two networks in a mutual learning manner, where both networks interact with each other and alleviate the error accumulation problem.
Revised: (2020b) propose to train two networks in a mutual learning manner, where both networks interact with each other and alleviate the error accumulation problem.

Original: Lv et al.
Revised: Lv et al.

Original: (2020) attempt to progressively identify the true labels in a self-training manner, which can be compatibly trained with stochastic optimization.
Revised: (2020) attempt to progressively identify the true labels in a self-training manner, which can be compatibly trained with stochastic optimization.

Original: Feng et al.
Revised: Feng et al.

Original: (2020b) assume that the probability of each candidate label being chosen is uniform and present two provably consistent methods: a risk-consistent method and a classiﬁer-consistent method.
Revised: (2020b) assume that the probability of each candidate label being chosen is uniform and present two provably consistent methods: a risk-consistent method and a classifier-consistent method.

Original: Another work in (Wen et al., 2021) induces a family of loss functions, which generalizes the uniform generation procedures.
Revised: Another work in (Wen et al., 2021) induces a family of loss functions, which generalizes the uniform generation procedures.

Original: In addition, a recent work in (Wang et al., 2022b) handles PLL by conducting class prototype-based label disambiguation following the idea of contrastive representation learning (He et al., 2020; Li et al., 2020), which achieves impressive prediction performance.
Revised: In addition, a recent work in (Wang et al., 2022) handles PLL by conducting class prototype-based label disambiguation following the idea of contrastive representation learning (He et al., 2020; Li et al., 2020), which achieves impressive prediction performance.

Original: Although the abovementioned approaches produce competitive performance, most of them assume the noisy labels in the candidate label set are randomly produced, which does not hold in many real-world learning scenarios.
Revised: Although the abovementioned approaches produce competitive performance, most of them assume the noisy labels in the candidate label set are randomly produced, which does not hold in many real-world learning scenarios.

Original: By contrast, we tackle a more realistic and challenging PLL scenario, where the noisy candidate labels are competitive and difﬁcult to distinguish from the true label.
Revised: By contrast, we tackle a more realistic and challenging PLL scenario, where the noisy candidate labels are competitive and difficult to distinguish from the true label.

Original:  P ROPOSED A PPROACH
Revised:  P ROPOSED A PPROACH

Original: In this section, we delineate the details of our proposed mutual partial label learning model (MLPLL) in the learning scenario with competitive label noise.
Revised: In this section, we delineate the details of our proposed mutual partial label learning model (MLPLL) in the learning scenario with competitive label noise.

Original: Given a partial label training set D =
Revised: Given a partial label training set D =

Original: ( X, Y )
Revised: ( X, Y )

Original: = { ( x i , y i ) } ni
Revised: = { ( x i , y i ) } ni

Original: =1 , where x i ∈
Revised: =1 , where x i ∈

Original: X denotes the i -th instance expressed in the input space X , and y i ∈ { 0 , 1 } L denotes the candidate label indicator vector associated with x i .
Revised: X denotes the i -th instance expressed in the input space X , and y i ∈ { 0 , 1 } L denotes the candidate label indicator vector associated with x i .

Original: The multiplevalues in y i denote either the ground-truth label or irrelevant noisy labels, which form the candidate label set S i ⊆
Revised: The multiplevalues in y i denote either the ground-truth label or irrelevant noisy labels, which form the candidate label set S i ⊆

Original: Y for x i , where Y = { 1 , 2 , · · · , L } denotes the multi-class label space.
Revised: Y for x i , where Y = { 1 , 2 , · · · , L } denotes the multi-class label space.

Original: The task of PLL is to induce a multi-class classiﬁer from the PLL training set D .
Revised: The task of PLL is to induce a multi-class classifier from the PLL training set D .

Original: We assume a challenging realistic PLL scenario, where the noisy labels in the candidate label set are difﬁcult to distinguish from the ground-truth label given the input instance.
Revised: We assume a challenging realistic PLL scenario, where the noisy labels in the candidate label set are difficult to distinguish from the ground-truth label given the input instance.

Original: The proposed ML-PLL model is illustrated in Figure 1, which mainly comprises the following component networks: a feature extractor G that extracts features from the input instance v =
Revised: The proposed ML-PLL model is illustrated in Figure 1, which mainly comprises the following component networks: a feature extractor G that extracts features from the input instance v =

Original: G ( x ) ∈ R m ; a projection network H that maps the feature vector v into a low-dimensional
Revised: G ( x ) ∈ R m ; a projection network H that maps the feature vector v into a low-dimensional

Original: embedding z
Revised: embedding z

Original: = H ( v ) ∈ R d ; a prediction network based classiﬁer F ; a class-prototype based classiﬁer; and a transformation network T that estimates the association relationships between the true label and candidate labels.
Revised: = H ( v ) ∈ R d ; a prediction network based classifier F ; a class-prototype based classifier; and a transformation network T that estimates the association relationships between the true label and candidate labels.

Original: The prediction network based classiﬁer and the class-prototype based classiﬁer are learned together through noisy label correction and interactive mutual learning, while the prediction network and the transformation network are learned together to match the observed candidate labels in the training data and enhance label disambiguation.
Revised: The prediction network based classifier and the class-prototype based classifier are learned together through noisy label correction and interactive mutual learning, while the prediction network and the transformation network are learned together to match the observed candidate labels in the training data and enhance label disambiguation.

Original: We elaborate these components and the proposed approach in the following subsections.
Revised: We elaborate these components and the proposed approach in the following subsections.

Original: 3. N OISY L ABEL C ORRECTION
Revised: 3. N OISY L ABEL C ORRECTION

Original: The main challenge of PLL lies in that the ground-truth label coexists with the irreverent noisy labels in the candidate label set and is unknown to the learning algorithm.
Revised: The main challenge of PLL lies in that the ground-truth label coexists with the irreverent noisy labels in the candidate label set and is unknown to the learning algorithm.

Original: The key of PLL is to correct the candidate label vectors and identify the true label vectors.
Revised: The key of PLL is to correct the candidate label vectors and identify the true label vectors.

Original: Given an interactive pair of classiﬁers—a prediction network classiﬁer F and a class-prototype based classiﬁer, we propose the following simple mutual label correction procedure.
Revised: Given an interactive pair of classifiers—a prediction network classifier F and a class-prototype based classifier, we propose the following simple mutual label correction procedure.

Original: For any training instance x i , we correct its partial label indicator vector towards the true label indicator vector by exploiting the predicted pseudo-label vector.
Revised: For any training instance x i , we correct its partial label indicator vector towards the true label indicator vector by exploiting the predicted pseudo-label vector.

Original: Speciﬁcally, we produce the soft pseudo-label vector q i ∈ [0 , 1] L for x i by taking a weighted convex combination of the prediction network’s softmax output probability vector
Revised: Specifically, we produce the soft pseudo-label vector q i ∈ [0 , 1] L for x i by taking a weighted convex combination of the prediction network’s softmax output probability vector

Original: F ( v i ) and the class-prototype based prediction probability vector p i :
Revised: F ( v i ) and the class-prototype based prediction probability vector p i :

Original: [Equation]
Revised: [Equation]

Original: where λ ∈ (0 , 1) is the combination weight parameter.
Revised: where λ ∈ (0 , 1) is the combination weight parameter.

Original: Here the prediction network F is built above the feature extractor G , such as F ( v i ) =
Revised: Here the prediction network F is built above the feature extractor G , such as F ( v i ) =

Original: F ( G ( x i )) , while p i is produced by the similarity-based class-prototype classiﬁer as follows:
Revised: F ( G ( x i )) , while p i is produced by the similarity-based class-prototype classifier as follows:

Original: [Equation]
Revised: [Equation]

Original: where τ is a temperature parameter, L is the number of class categories, z i
Revised: where τ is a temperature parameter, L is the number of class categories, z i

Original: = H ( G ( x i )) denotes the low-dimensional embedding of the features for instance x i , and c k denotes the prototype vector for the k -th category, which will be elaborated in the next subsection.
Revised: = H ( G ( x i )) denotes the low-dimensional embedding of the features for instance x i , and c k denotes the prototype vector for the k -th category, which will be elaborated in the next subsection.

Original: Instead of using Euclidean distance, we use the dot-product operator “ · ” to compute the inner product similarity between the instance and the class prototypes in the embedding space for class prediction probability calculation.
Revised: Instead of using Euclidean distance, we use the dot-product operator “ · ” to compute the inner product similarity between the instance and the class prototypes in the embedding space for class prediction probability calculation.

Original: To maintain an effective soft pseudo-label vector for each instance and eliminate the noise predictions outside of the candidate label set, we further restrict the soft pseudo-labels within the candidate label set and transform q i into (cid:98) q i with row normalization,
Revised: To maintain an effective soft pseudo-label vector for each instance and eliminate the noise predictions outside of the candidate label set, we further restrict the soft pseudo-labels within the candidate label set and transform q i into (cid:98) q i with row normalization,

Original: [Equation]
Revised: [Equation]

Original: This noisy label correction procedure can be applied on all the n training instances to produce corrected soft pseudo-label vectors { (cid:98) q 1 , · · · , (cid:98) q n } .
Revised: This noisy label correction procedure can be applied on all the n training instances to produce corrected soft pseudo-label vectors { (cid:98) q 1 , · · · , (cid:98) q n } .

Original: With the progress of prediction network training and class-prototype update (elaborated in the next subsection), we expect each produced soft pseudolabel vector will be better aligned with the corresponding true label indicator vector, which can interactively provide a good target for the subsequent prediction network training and gradually mitigate the negative impact of the label noise during the training process.
Revised: With the progress of prediction network training and class-prototype update (elaborated in the next subsection), we expect each produced soft pseudolabel vector will be better aligned with the corresponding true label indicator vector, which can interactively provide a good target for the subsequent prediction network training and gradually mitigate the negative impact of the label noise during the training process.

Original: 3. B ATCH - WISE M OVING A VERAGE BASED P ROTOTYPE U PDATE
Revised: 3. B ATCH - WISE M OVING A VERAGE BASED P ROTOTYPE U PDATE

Original: The effectiveness of noisy label correction relies on accurate class probability predictions, which depend on both the prediction network and the class-prototypes.
Revised: The effectiveness of noisy label correction relies on accurate class probability predictions, which depend on both the prediction network and the class-prototypes.

Original: Instead of training an entirely independent class-prototype based classiﬁer, we propose to produce the class prototypes based on the output of the prediction network classiﬁer.
Revised: Instead of training an entirely independent class-prototype based classifier, we propose to produce the class prototypes based on the output of the prediction network classifier.

Original: Speciﬁcally, we compute the class prototypes in the low-dimensional embedding space from the partial label training set D in the following way:
Revised: Specifically, we compute the class prototypes in the low-dimensional embedding space from the partial label training set D in the following way:

Original: [Equation]
Revised: [Equation]

Original: where 1 is an indicator function; (cid:98)
Revised: where 1 is an indicator function; (cid:98)

Original: y i denotes predicted one-hot label vector for x i , which is determined by the output of the prediction network F and restricted to the candidate label set S i .
Revised: y i denotes predicted one-hot label vector for x i , which is determined by the output of the prediction network F and restricted to the candidate label set S i .

Original: In this way, we can automatically eliminate the prediction noise outside of the candidate labels.
Revised: In this way, we can automatically eliminate the prediction noise outside of the candidate labels.

Original: With the prototype computing method above, we can update each class-prototype by recomputing it during each training iteration.
Revised: With the prototype computing method above, we can update each class-prototype by recomputing it during each training iteration.

Original: However, such an update process will be computationally intensive and lead to intolerable training latency, since it requires going through the whole training set.
Revised: However, such an update process will be computationally intensive and lead to intolerable training latency, since it requires going through the whole training set.

Original: To address this problem, we propose to perform batch-wise moving average based prototype update; that is, we update the class-prototypes for the current mini-batch by consulting the prototypes computed from previous mini-batches in a moving average manner:
Revised: To address this problem, we propose to perform batch-wise moving average based prototype update; that is, we update the class-prototypes for the current mini-batch by consulting the prototypes computed from previous mini-batches in a moving average manner:

Original: [Equation]
Revised: [Equation]

Original: where c (cid:48) k is the mean embedding for the k -th class within the current mini-batch, which can be computed with Eq.(4) by limiting D to the current mini-batch; γ ∈ [0 , 1] is a momentum coefﬁcient; and Normalize( c ) 
Revised: where c ′ k is the mean embedding for the k -th class within the current mini-batch, which can be computed with Eq.(4) by limiting D to the current mini-batch; γ ∈ [0 , 1] is a momentum coefficient; and Normalize( c ) =

Original: = c / (cid:107) c (cid:107) 2 .
Revised: c / ∥ c ∥ 2 .

Original: We expect such a batch-wise prototype update method can be computationally efﬁcient and enhance the gradual adjustment and stability of the class prototypes.
Revised: We expect such a batch-wise prototype update method can be computationally efficient and enhance the gradual adjustment and stability of the class prototypes.

Original: 3. P REDICTION N ETWORK T RAINING WITH M UTUAL L EARNING
Revised: 3. P REDICTION N ETWORK T RAINING WITH M UTUAL L EARNING

Original: With the noisy label correction method above, the corrected soft pseudo-label vectors can be used as target for prediction network training.
Revised: With the noisy label correction method above, the corrected soft pseudo-label vectors can be used as target for prediction network training.

Original: Speciﬁcally, we can learn the prediction network based probabilistic classiﬁer on the training data D by minimizing the following classiﬁcation loss:
Revised: Specifically, we can learn the prediction network based probabilistic classifier on the training data D by minimizing the following classification loss:

Original: [Equation]
Revised: [Equation]

Original: where (cid:98) q ik denotes the k -th entry of the corrected soft pseudo-label vector (cid:98) q i for the i -th instance, and F k ( · ) denotes the k -th entry of the prediction network’s softmax output, indicating the predicted probability of the instance belonging to the k -th class.
Revised: where (cid:98) q ik denotes the k -th entry of the corrected soft pseudo-label vector (cid:98) q i for the i -th instance, and F k ( · ) denotes the k -th entry of the prediction network’s softmax output, indicating the predicted probability of the instance belonging to the k -th class.

Original: We expect the prediction network based classiﬁer F can be improved by training with the corrected soft pseudo-labels.
Revised: We expect the prediction network based classifier F can be improved by training with the corrected soft pseudo-labels.

Original: Meanwhile, we can learn an additional class-prototype based classiﬁer by using the output of the prediction network as prediction target and deploying the following mutual learning loss:
Revised: Meanwhile, we can learn an additional class-prototype based classifier by using the output of the prediction network as prediction target and deploying the following mutual learning loss:

Original: [Equation]
Revised: [Equation]

Original: where p ik is the predicted probability for the i -th instance belonging to the k -th class by the prototype based classiﬁer using Eq.(2).
Revised: where p ik is the predicted probability for the i -th instance belonging to the k -th class by the prototype based classifier using Eq.(2).

Original: This mutual learning loss is expected to enforce both classiﬁers to collaborate with each other and gradually lead to consistent predictions that can better align with the underlying true label indicator vectors.
Revised: This mutual learning loss is expected to enforce both classifiers to collaborate with each other and gradually lead to consistent predictions that can better align with the underlying true label indicator vectors.

Original: 3. T RANSFORMATION BASED L ABEL A SSOCIATION L EARNING
Revised: 3. T RANSFORMATION BASED L ABEL A SSOCIATION L EARNING

Original: As previously discussed, the noisy labels in the candidate label set typically have association relationships with the ground-truth label and can be competitive with the true label given the input instance in the real-world learning scenarios.
Revised: As previously discussed, the noisy labels in the candidate label set typically have association relationships with the ground-truth label and can be competitive with the true label given the input instance in the real-world learning scenarios.

Original: In this subsection, we propose to learn the competitive association relationships between the noisy candidate labels and the true label using a transformation network T , aiming to enhance the label disambiguation process.
Revised: In this subsection, we propose to learn the competitive association relationships between the noisy candidate labels and the true label using a transformation network T , aiming to enhance the label disambiguation process.

Original: We assume the transformation network T can be learned to produce a transformation matrix
Revised: We assume the transformation network T can be learned to produce a transformation matrix

Original: R =
Revised: R =

Original: [ r ij ] L × L , such that R can capture the transformation relationships from a given ground-truth label to the associated noisy candidate labels and recover the candidate label set.
Revised: [ r ij ] L × L , such that R can capture the transformation relationships from a given ground-truth label to the associated noisy candidate labels and recover the candidate label set.

Original: The entry r ij of
Revised: The entry r ij of

Original: R reﬂects the probability of transforming the j -th label to a candidate label when the i -th label is the ground-truth.
Revised: R reflects the probability of transforming the j -th label to a candidate label when the i -th label is the ground-truth.

Original: When r ij is large, the j -th label has a strong association relationship with the i -th ground-truth label and hence has a larger probability to be chosen as a candidate label.
Revised: When r ij is large, the j -th label has a strong association relationship with the i -th ground-truth label and hence has a larger probability to be chosen as a candidate label.

Original: Towards this goal, we propose to produce the transformation matrix R by applying the transformation network T on the observed candidate label matrix Y =
Revised: Towards this goal, we propose to produce the transformation matrix R by applying the transformation network T on the observed candidate label matrix Y =

Original: [ y ij ] n × L from the training set, such that R = T ( Y ) , since the observed candidate label matrix contains both the true labels and the associated noisy candidate labels.
Revised: [ y ij ] n × L from the training set, such that R = T ( Y ) , since the observed candidate label matrix contains both the true labels and the associated noisy candidate labels.

Original: The transformation network T can then be learned through prediction based candidate label matrix reconstruction.
Revised: The transformation network T can then be learned through prediction based candidate label matrix reconstruction.

Original: Speciﬁcally, we propose to reconstruct the normalized candidate label matrix by transforming the prediction outputs of the prediction network based classiﬁer F with the transformation matrix R .
Revised: Specifically, we propose to reconstruct the normalized candidate label matrix by transforming the prediction outputs of the prediction network based classifier F with the transformation matrix R .

Original: We use the symmetric Kullback-Leibler (KL) divergence to match each reconstructed candidate label distribution with its corresponding observed candidate label distribution and form the following consistency loss:[Equation]
Revised: We use the symmetric Kullback-Leibler (KL) divergence to match each reconstructed candidate label distribution with its corresponding observed candidate label distribution and form the following consistency loss:

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) Y i
Revised: where (cid:101) Y i

Original: =
Revised: =

Original: Y i / (cid:107)
Revised: Y i / ∥

Original: Y i (cid:107) 2 is a row normalized candidate label matrix, expressing the candidate label distributions in the training set.
Revised: Y i ∥ 2 is a row normalized candidate label matrix, expressing the candidate label distributions in the training set.

Original: It is however computationally intensive to compute the transformation matrix R by taking the whole candidate label matrix Y as input.
Revised: It is however computationally intensive to compute the transformation matrix R by taking the whole candidate label matrix Y as input.

Original: We further propose to compute and update R on each mini-batch in a moving average style, such as
Revised: We further propose to compute and update R on each mini-batch in a moving average style, such as

Original: [Equation]
Revised: [Equation]

Original:  where R (cid:48) is the transformation matrix computed from the observed partial label matrix on the current mini-batch, and η ∈ [0 , 1] is a momentum coefﬁcient.
Revised: ) where R ′ is the transformation matrix computed from the observed partial label matrix on the current mini-batch, and η ∈ [0 , 1] is a momentum coefficient.

Original: By exploiting the label association relationships between the true-label and the candidate labels, we expect the consistency loss in Eq.(8) can enhance noisy label correction and prediction network learning, thus consequently pushing the outputs of the prediction network towards the true labels.
Revised: By exploiting the label association relationships between the true-label and the candidate labels, we expect the consistency loss in Eq.(8) can enhance noisy label correction and prediction network learning, thus consequently pushing the outputs of the prediction network towards the true labels.

Original: 3. L EARNING WITH ML-PLL M ODEL
Revised: 3. L EARNING WITH ML-PLL M ODEL

Original: By integrating the classiﬁcation loss in Eq.(6), the mutual learning loss in Eq.(7) and the consistency loss in Eq.(8) together, we get the following overall training loss for the proposed ML-PLL model,[Equation]
Revised: By integrating the classification loss in Eq.(6), the mutual learning loss in Eq.(7) and the consistency loss in Eq.(8) together, we get the following overall training loss for the proposed ML-PLL model,

Original: [Equation]
Revised: [Equation]

Original: ) where α and β are trade-off hyperparameters that control the relative importance of the mutual learning loss and the consistency loss, respectively.
Revised: ) where α and β are trade-off hyperparameters that control the relative importance of the mutual learning loss and the consistency loss, respectively.

Original: We perform training by minimizing this objective with a mini-batch based stochastic gradient descent (SGD) algorithm.
Revised: We perform training by minimizing this objective with a mini-batch based stochastic gradient descent (SGD) algorithm.

Original:  E XPERIMENT 4. E XPERIMENTAL S ETTING
Revised:  E XPERIMENT 4. E XPERIMENTAL S ETTING

Original: Datasets We conducted experiments on four widely used benchmark image datasets: FashionMNIST (Xiao et al., 2017), Kuzushiji-MNIST (Clanuwat et al., 2018), CIFAR-10 and CIFAR-
Revised: Datasets We conducted experiments on four widely used benchmark image datasets: FashionMNIST (Xiao et al., 2017), Kuzushiji-MNIST (Clanuwat et al., 2018), CIFAR-10 and CIFAR-

Original: (Krizhevsky et al., 2009).
Revised: (Krizhevsky et al., 2009).

Original: We transform these datasets into partial-label datasets by using the following competitive and uniform label noise generation processes.
Revised: We transform these datasets into partial-label datasets by using the following competitive and uniform label noise generation processes.

Original: First , to generate competitive label noise, we train a neural network on the original clean dataset to predict the probability of each category label being the true label for a given instance, and then randomly choose the irrelevant labels with high predicted probabilities as noisy candidate labels.
Revised: First , to generate competitive label noise, we train a neural network on the original clean dataset to predict the probability of each category label being the true label for a given instance, and then randomly choose the irrelevant labels with high predicted probabilities as noisy candidate labels.

Original: Speciﬁcally, assume ˆp i is the predicted label probability vector for an instance x i whose true label is class j .
Revised: Specifically, assume ˆp i is the predicted label probability vector for an instance x i whose true label is class j .

Original: We ﬁrst exclude the true label by setting ˆp ij = 0 , and then randomly choose labels from the top- K predictions in ˆp i —i.e., the K irrelevant label categories with highest predicted probabilities—as the noisy candidate labels.
Revised: We first exclude the true label by setting ˆp ij = 0 , and then randomly choose labels from the top- K predictions in ˆp i —i.e., the K irrelevant label categories with highest predicted probabilities—as the noisy candidate labels.

Original: Intuitively, these irrelevant labels with higher predicted probabilities demonstrate stronger association relationships and are more competitive with the true label.
Revised: Intuitively, these irrelevant labels with higher predicted probabilities demonstrate stronger association relationships and are more competitive with the true label.

Original: The chosen competitive noisy labels, together with the true label j , form the candidate label set for instance x i .
Revised: The chosen competitive noisy labels, together with the true label j , form the candidate label set for instance x i .

Original: Following this competitive
Revised: Following this competitive

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: candidate label noise generation process, we produce partial-label datasets by choosing competitive label noise from the top-6 predictions for Fashion-MNIST, Kuzushiji-MNIST and CIFAR-10, while choosing from the top-20 predictions for CIFAR-100.
Revised: candidate label noise generation process, we produce partial-label datasets by choosing competitive label noise from the top-6 predictions for Fashion-MNIST, Kuzushiji-MNIST and CIFAR-10, while choosing from the top-20 predictions for CIFAR-100.

Original: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise are reported in Appendix A.2.
Revised: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise are reported in Appendix A.2.

Original: Second , we use the uniform generation procedure in (Lv et al., 2020) to generate uniform label noise.
Revised: Second , we use the uniform generation procedure in (Lv et al., 2020) to generate uniform label noise.

Original: It generates noisy candidate labels by ﬂipping each incorrect label with a probability ρ , which indicates the label ambiguity level.
Revised: It generates noisy candidate labels by flipping each incorrect label with a probability ρ , which indicates the label ambiguity level.

Original: When encountering a special case where no irrelevant label is chosen, we randomly select one irrelevant label as the additional candidate label to ensure the label sets for all training instances are corrupted.
Revised: When encountering a special case where no irrelevant label is chosen, we randomly select one irrelevant label as the additional candidate label to ensure the label sets for all training instances are corrupted.

Original: Comparison Methods We compare the proposed ML-PLL method with the following deep neural network based PLL methods: PiCO (Wang et al., 2022b) is a contrastive learning based state-of-theart PLL method, which disambiguates the candidate labels by reﬁning the pseudo-labels in a movingaverage manner; LWS (Wen et al., 2021) is a discriminative PLL method, which considers the loss trade-off between the candidate labels and non-candidate ones; CC (Feng et al., 2020b) is a classiﬁerconsistent method, which leverages a transition matrix to induce an empirical risk estimator; MSE and EXP (Feng et al., 2020a) are two baselines that exploit mean square error and exponential loss, respectively, as risks for inducing unbiased risk estimators.
Revised: Comparison Methods We compare the proposed ML-PLL method with the following deep neural network based PLL methods: PiCO (Wang et al., 2022) is a contrastive learning based state-of-theart PLL method, which disambiguates the candidate labels by refining the pseudo-labels in a movingaverage manner; LWS (Wen et al., 2021) is a discriminative PLL method, which considers the loss trade-off between the candidate labels and non-candidate ones; CC (Feng et al., 2020b) is a classifierconsistent method, which leverages a transition matrix to induce an empirical risk estimator; MSE and EXP (Feng et al., 2020a) are two baselines that exploit mean square error and exponential loss, respectively, as risks for inducing unbiased risk estimators.

Original: For each comparison method, we use the same backbone structure for feature extraction and adopt the same predictive model.
Revised: For each comparison method, we use the same backbone structure for feature extraction and adopt the same predictive model.

Original: Moreover, all the comparison methods are conﬁgured with the suggested parameters according to the source literature.
Revised: Moreover, all the comparison methods are configured with the suggested parameters according to the source literature.

Original: For each experiment, we report the average test accuracy and standard deviation based on 5 independent runs.
Revised: For each experiment, we report the average test accuracy and standard deviation based on 5 independent runs.

Original: Implementation Details We use LeNet-5 as the backbone on the Fashion-MNIST and KuzushijiMNIST datasets, and use 18-layer ResNet as the backbone on CIFAR-10 and CIFAR-100, for feature extraction.
Revised: Implementation Details We use LeNet-5 as the backbone on the Fashion-MNIST and KuzushijiMNIST datasets, and use 18-layer ResNet as the backbone on CIFAR-10 and CIFAR-100, for feature extraction.

Original: The projection network is a two-layer MLP that outputs 128-dimensional embeddings.
Revised: The projection network is a two-layer MLP that outputs 128-dimensional embeddings.

Original: The prediction network is a linear classiﬁer.
Revised: The prediction network is a linear classifier.

Original: The weighted combination parameter λ in Eq.(1), the temperature parameter τ in Eq.(2), and the momentum coefﬁcients in Eq.(5) and Eq.(9) are set to 0.99, 0.07, 0.999, and 0.99 respectively.
Revised: The weighted combination parameter λ in Eq.(1), the temperature parameter τ in Eq.(2), and the momentum coefficients in Eq.(5) and Eq.(9) are set to 0.99, 1, 0.999, and 0.99 respectively.

Original: In all the experiments, we utilize a standard SGD optimizer
Revised: In all the experiments, we utilize a standard SGD optimizer

Original: [Table] [Table]
Revised: [Table] [Table]

Original: with a momentum of 0.9 and a weight decay of 1e-3 for model training.
Revised: with a momentum of 0.9 and a weight decay of 1e-3 for model training.

Original: The mini-batch size, learning rate and total training epochs are set to 128, 0.01 and 400 respectively.
Revised: The mini-batch size, learning rate and total training epochs are set to 128, 0.01 and 400 respectively.

Original: The parameter α and β are chosen from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } and { 1 , 2 , 3 , 4 , 5 , 6 } , respectively, according to the accuracy on a validation dataset (10% of the training dataset)
Revised: The parameter α and β are chosen from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } and { 1 , 2 , 3 , 4 , 5 , 6 } , respectively, according to the accuracy on a validation dataset (10% of the training dataset)

Original: [Table]
Revised: [Table]

Original: 4. E XPERIMENTAL R ESULTS WITH C OMPETITIVE L ABEL N OISE
Revised: 4. E XPERIMENTAL R ESULTS WITH C OMPETITIVE L ABEL N OISE

Original: We compared the proposed ML-PLL method with the ﬁve state-of-the-art comparison methods on the four datasets corrupted by competitive label noise.
Revised: We compared the proposed ML-PLL method with the five state-of-the-art comparison methods on the four datasets corrupted by competitive label noise.

Original: The comparison results on Fashion-MNIST,
Revised: The comparison results on Fashion-MNIST,

Original: Kuzushiji-MNIST and CIFAR-10 with different label ambiguity levels, indicated by the average number of candidate labels, are reported in Table 1.
Revised: Kuzushiji-MNIST and CIFAR-10 with different label ambiguity levels, indicated by the average number of candidate labels, are reported in Table 1.

Original: From the table we can see that the proposed method consistently outperforms all the other comparison methods, and the performance gains yield by ML-PLL over the other methods are quite notable in many cases.
Revised: From the table we can see that the proposed method consistently outperforms all the other comparison methods, and the performance gains yield by ML-PLL over the other methods are quite notable in many cases.

Original: For example, ML-PLL outperforms the best alternative comparison method by 1.48% and 2.83% on CIFAR-10 with Avg.#CL = 4 and 5 respectively.
Revised: For example, ML-PLL outperforms the best alternative comparison method by 1.48% and 2.83% on CIFAR-10 with Avg.#CL = 4 and 5 respectively.

Original: The experimental results on the more challenging dataset CIFAR-100, which has a large label space ( L = 100), are reported in Table 2.
Revised: The experimental results on the more challenging dataset CIFAR-100, which has a large label space ( L = 100), are reported in Table 2.

Original: From the table we can see that the proposed ML-PLL again outperforms all the other comparison methods and demonstrates remarkable performance improvement on such a complicated dataset.
Revised: From the table we can see that the proposed ML-PLL again outperforms all the other comparison methods and demonstrates remarkable performance improvement on such a complicated dataset.

Original: For example, ML-PLL improves the best comparison method, PiCO, by 1.53%, 3.49%, and 4.31% respectively across the three label ambiguity levels.
Revised: For example, ML-PLL improves the best comparison method, PiCO, by 1.53%, 3.49%, and 4.31% respectively across the three label ambiguity levels.

Original: Moreover, the comparison methods—e.g, PiCO, LWS, and CC—perform well with the low label ambiguity level (Avg.#CL=2), but their performance degrades dramatically with the increase of the label ambiguity level.
Revised: Moreover, the comparison methods—e.g, PiCO, LWS, and CC—perform well with the low label ambiguity level (Avg.#CL=2), but their performance degrades dramatically with the increase of the label ambiguity level.

Original: By contrast, the proposed ML-PLL method not only consistently outperforms the other methods with notable performance gains, but also produces larger gains with higher label ambiguity levels.
Revised: By contrast, the proposed ML-PLL method not only consistently outperforms the other methods with notable performance gains, but also produces larger gains with higher label ambiguity levels.

Original: All these results validate the efﬁcacy of the proposed ML-PLL approach in addressing the competitive label noise.
Revised: All these results validate the efficacy of the proposed ML-PLL approach in addressing the competitive label noise.

Original: 4. E XPERIMENTAL R ESULTS WITH U NIFORM L ABEL N OISE
Revised: 4. E XPERIMENTAL R ESULTS WITH U NIFORM L ABEL N OISE

Original: To further validate the effectiveness of the proposed method, we also conducted experiments on the four datasets (Fashion-MNIST, Kuzushiji-MNIST, CIFAR-10 and CIFAR-100) corrupted by the
Revised: To further validate the effectiveness of the proposed method, we also conducted experiments on the four datasets (Fashion-MNIST, Kuzushiji-MNIST, CIFAR-10 and CIFAR-100) corrupted by the

Original: [Table]
Revised: [Table]

Original: uniform label noise.
Revised: uniform label noise.

Original: The label ambiguity level in this scenario is controlled by the ρ value—a larger ρ probability for producing noisy candidate labels leads to larger candidate label sets.
Revised: The label ambiguity level in this scenario is controlled by the ρ value—a larger ρ probability for producing noisy candidate labels leads to larger candidate label sets.

Original: All the comparison results are reported in Table 3 and Table 4.
Revised: All the comparison results are reported in Table 3 and Table 4.

Original: From Table
Revised: From Table

Original: [Table]
Revised: [Table]

Original: we can see that although the proposed method is not specially designed for uniform label noise
Revised: we can see that although the proposed method is not specially designed for uniform label noise

Original: [Table]
Revised: [Table]

Original: it still achieves competitive performance with the best method across all cases.
Revised: it still achieves competitive performance with the best method across all cases.

Original: In Table
Revised: In Table

Original: [Table]
Revised: [Table]

Original: the proposed method even demonstrates substantial superiority over the other methods and produces larger performance gains with higher label ambiguity levels.
Revised: the proposed method even demonstrates substantial superiority over the other methods and produces larger performance gains with higher label ambiguity levels.

Original: These results on the challenging CIFAR-100 dataset are consistent with the ones reported in the scenario with competitive label noise—that is, the proposed ML-PLL method yields substantial and increasing performance gains when the label ambiguity level increases.
Revised: These results on the challenging CIFAR-100 dataset are consistent with the ones reported in the scenario with competitive label noise—that is, the proposed ML-PLL method yields substantial and increasing performance gains when the label ambiguity level increases.

Original: All these results demonstrate the efﬁcacy of the proposed method in handing both competitive and uniform label noise for PLL.
Revised: All these results demonstrate the efficacy of the proposed method in handing both competitive and uniform label noise for PLL.

Original: 4. A BLATION S TUDY
Revised: 4. A BLATION S TUDY

Original: The objective of the proposed method contains three loss terms: the classiﬁcation loss, the interactive mutual learning loss
Revised: The objective of the proposed method contains three loss terms: the classification loss, the interactive mutual learning loss

Original: [Table]
Revised: [Table]

Original: and the consistency loss.
Revised: and the consistency loss.

Original: In addition, noisy label correction is an indispensable component in the proposed mutual learning model.
Revised: In addition, noisy label correction is an indispensable component in the proposed mutual learning model.

Original: To validate each part’s contribution, we conduct an ablation study to compare the proposed ML-PLL method with the following ablation variants: (1) ML-PLL-w/o-ml, which drops the mutual learning loss from the full model; (2) MLPLL-w/o-con, which drops the consistency loss; (3) ML-PLL-w/o-lc, which drops the noisy label correction procedure; and (4) CLS, which only uses a standard classiﬁcation loss by dropping the mutual learning loss, the consistency loss and the noisy label correction procedure.
Revised: To validate each part’s contribution, we conduct an ablation study to compare the proposed ML-PLL method with the following ablation variants: (1) ML-PLL-w/o-ml, which drops the mutual learning loss from the full model; (2) MLPLL-w/o-con, which drops the consistency loss; (3) ML-PLL-w/o-lc, which drops the noisy label correction procedure; and (4) CLS, which only uses a standard classification loss by dropping the mutual learning loss, the consistency loss and the noisy label correction procedure.

Original: The comparison results are reported in Table 5.
Revised: The comparison results are reported in Table 5.

Original: From the table we can see that compared with the full model, all the four variants produced inferior results, which suggests that all the components contribute to the full model in different degrees.
Revised: From the table we can see that compared with the full model, all the four variants produced inferior results, which suggests that all the components contribute to the full model in different degrees.

Original: Among the three variants of ML-PLL-w/o-ml, ML-PLL-w/o-con and ML-PLL-w/o-lc, ML-PLL-w/o-lc produces the largest performance degradation, which demonstrates the importance of the noisy label correction procedure for label disambiguation and classiﬁer learning.
Revised: Among the three variants of ML-PLL-w/o-ml, ML-PLL-w/o-con and ML-PLL-w/o-lc, ML-PLL-w/o-lc produces the largest performance degradation, which demonstrates the importance of the noisy label correction procedure for label disambiguation and classifier learning.

Original: Meanwhile, ML-PLL-w/o-ml and ML-PLL-w/o-con also produced consistent and very notable performance drops from the full model, which validates the contributions of both the mutual learning loss and the transformation network based consistency loss.
Revised: Meanwhile, ML-PLL-w/o-ml and ML-PLL-w/o-con also produced consistent and very notable performance drops from the full model, which validates the contributions of both the mutual learning loss and the transformation network based consistency loss.

Original: The results suggest exploiting the association relationships between the competitive label noise and the ground-truth label can help
Revised: The results suggest exploiting the association relationships between the competitive label noise and the ground-truth label can help

Original: PLL.
Revised: PLL.

Original: Moreover, the large performance gap between the full model and the baseline variant CLS further validates the integrating power of the proposed method on combining all these three contributing components for PLL with competitive label noise: noisy label correction, mutual learning, and transformation network based label association exploitation.
Revised: Moreover, the large performance gap between the full model and the baseline variant CLS further validates the integrating power of the proposed method on combining all these three contributing components for PLL with competitive label noise: noisy label correction, mutual learning, and transformation network based label association exploitation.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: This paper tackles a more realistic PLL scenario with competitive label noise that can be more difﬁcult to distinguish from the ground-truth label than random noisy labels.
Revised: This paper tackles a more realistic PLL scenario with competitive label noise that can be more difficult to distinguish from the ground-truth label than random noisy labels.

Original: We propose a novel mutual partial label learning approach called ML-PLL to address this challenging problem.
Revised: We propose a novel mutual partial label learning approach called ML-PLL to address this challenging problem.

Original: ML-PLL learns a prediction network based classiﬁer and a class-prototype based classiﬁer cooperatively with label correction under a mutual learning framework.
Revised: ML-PLL learns a prediction network based classifier and a class-prototype based classifier cooperatively with label correction under a mutual learning framework.

Original: Moreover, we introduce a transformation network to model the association relationships between the ground-truth label and the candidate noisy labels, and learn it together with the prediction network through candidate label set reconstruction to enhance label correction.
Revised: Moreover, we introduce a transformation network to model the association relationships between the ground-truth label and the candidate noisy labels, and learn it together with the prediction network through candidate label set reconstruction to enhance label correction.

Original: The comprehensive experiments on several benchmark datasets corrupted with both competitive and uniform label noise demonstrate that the proposed approach signiﬁcantly outperforms the state-of-the-art PLL methods.
Revised: The comprehensive experiments on several benchmark datasets corrupted with both competitive and uniform label noise demonstrate that the proposed approach significantly outperforms the state-of-the-art PLL methods.

Original: A A PPENDIX A. T RAINING A LGORITHM FOR ML-PLL
Revised: A A PPENDIX A. T RAINING A LGORITHM FOR ML-PLL

Original: We present the mini-batch based training algorithm for ML-PLL in Algorithm 1.
Revised: We present the mini-batch based training algorithm for ML-PLL in Algorithm 1.

Original: Algorithm 1 Training Algorithm for ML-PLL.
Revised: Algorithm 1 Training Algorithm for ML-PLL.

Original:  do do 
Revised: for num epochs do do Sample m

Original: Noisy label correction
Revised: Noisy label correction

Original: Compute the class prototype-based prediction probability vectors { p i } via Eq
Revised: Compute the class prototype-based prediction probability vectors { p i } via Eq.(2).

Original: Perform noisy label correction to get {  q i } via Eq Eq
Revised: Perform noisy label correction to get { (cid:98) q i } via Eq.(1) and Eq.(3).

Original: // Batch-wise moving average based prototype update
Revised: // Batch-wise moving average based prototype update

Original: Compute 
Revised: Compute the class prototypes via Eq.(4) on the current batch B .

Original: Update the class prototypes via 
Revised: Update the class prototypes via Eq.(5).

Original: // Update of the transformation matrix
Revised: // Update of the transformation matrix

Original: Compute and update the transformation matrix via Eq
Revised: Compute and update the transformation matrix via Eq.(9).

Original: // Classiﬁcation loss calculation
Revised: // Classification loss calculation

Original: L c ( F, G, H ) =
Revised: L c ( F, G, H ) =

Original: 1 m (cid:80) x i ∼ B (cid:80)
Revised: 1 m (cid:80) x i ∼ B (cid:80)

Original: Lk =1 − (cid:98) q ik log( F k ( G ( x i ))) //
Revised: Lk =1 − (cid:98) q ik log( F k ( G ( x i ))) //

Original: Interactive mutual learning loss calculation
Revised: Interactive mutual learning loss calculation

Original: L ml ( F, G, H ) =
Revised: L ml ( F, G, H ) =

Original: 1 m (cid:80) x i ∼ B (cid:80)
Revised: 1 m (cid:80) x i ∼ B (cid:80)

Original: Lk =

Revised: Lk =

Original: − F k ( G ( x i ))log( p ik ) //
Revised: − F k ( G ( x i ))log( p ik ) //

Original: Consistency loss calculation
Revised: Consistency loss calculation

Original: L con ( F, G, H, T ) =
Revised: L con ( F, G, H, T ) =

Original: (cid:80) x i ∼ B (cid:16) KL (cid:0) F ( G ( x i ))
Revised: (cid:80) x i ∼ B (cid:16) KL (cid:0) F ( G ( x i ))

Original: R, (cid:101) Y i (cid:1)
Revised: R, (cid:101) Y i (cid:1)

Original: + KL (cid:0)(cid:101) Y
Revised: + KL (cid:0)(cid:101) Y

Original: i , F ( G ( x i ))
Revised: i , F ( G ( x i ))

Original: R (cid:1)(cid:17) //
Revised: R (cid:1)(cid:17) //

Original: Network updating
Revised: Network updating

Original: L =
Revised: L =

Original: L c + α L ml + β L con
Revised: L c + α L ml + β L con

Original: Update network F , T  G  via stochastic gradient descent by minimizing L . end for end for
Revised: Update network F , T , G and H via stochastic gradient descent by minimizing L . end for end for

Original: A . D ATASET
Revised: A. D ATASET

Original: We produce PLL datasets with different average number of candidate labels (Avg.#CL) from each benchmark dataset using the competitive label noise generation process described in subsection 4.1.
Revised: We produce PLL datasets with different average number of candidate labels (Avg.#CL) from each benchmark dataset using the competitive label noise generation process described in subsection 4.1.

Original: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise are recorded in Table 6.
Revised: The average number of candidate labels (Avg.#CL) and the characteristics of each benchmark dataset corrupted by the competitive label noise are recorded in Table 6.

Original: A. P ARAMETER S ENSITIVITY A NALYSIS
Revised: A. P ARAMETER S ENSITIVITY A NALYSIS

Original: The trade-off parameters α and β control the relative importance of the mutual learning loss and the consistency loss, respectively.
Revised: The trade-off parameters α and β control the relative importance of the mutual learning loss and the consistency loss, respectively.

Original: We conduct experiments on CIFAR-10 at the label ambiguity level of Avg#CL = 4 to study the impacts of α and β on the performance of ML-PLL.
Revised: We conduct experiments on CIFAR-10 at the label ambiguity level of Avg#CL = 4 to study the impacts of α and β on the performance of ML-PLL.

Original: We investigate the impact of α by experimenting with different α values from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } while setting β = 4 , and investigate the impact of β by experimenting with different β values from { 1 , 2 , 3 , 4 , 5 , 6 } while setting α = 0 .
Revised: We investigate the impact of α by experimenting with different α values from { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 } while setting β = 4 , and investigate the impact of β by experimenting with different β values from { 1 , 2 , 3 , 4 , 5 , 6 } while setting α = 0 .

Original: The results are presented in Figure 2(a) and Figure 2(b) respectively.
Revised: The results are presented in Figure 2(a) and Figure 2(b) respectively.

Original: From Figure 2(a), we can see the best performance is achieved when α = 0 .
Revised: From Figure 2(a), we can see the best performance is achieved when α = 0 .

Original: 6 , while the performance difference is very small for varying α within [0 . 5 , 0 .
Revised: 6 , while the performance difference is very small for varying α within [0 . 5 , 0 .

Original: But further increasing α towards 1 leads to performance degradation.
Revised: But further increasing α towards 1 leads to performance degradation.

Original: The results in Figure 2(b) however demonstrate larger variations with different β values, suggesting the performance of ML-PLL is more sensitive to the β value.
Revised: The results in Figure 2(b) however demonstrate larger variations with different β values, suggesting the performance of ML-PLL is more sensitive to the β value.

Original: When β is very small, the performance of ML-PLL is relatively poor.
Revised: When β is very small, the performance of ML-PLL is relatively poor.

Original: With the increase of the β value, the performance improves as the consistency loss begins to work.
Revised: With the increase of the β value, the performance improves as the consistency loss begins to work.

Original: But when β is too large, the performance degrades again since the consistency loss can over-dominate.
Revised: But when β is too large, the performance degrades again since the consistency loss can over-dominate.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: A. I MPACT OF C OMPETITIVE N OISE L EVELS
Revised: A. I MPACT OF C OMPETITIVE N OISE L EVELS

Original: As described in subsection 4.1, we generate the competitive label noise by randomly selecting irrelevant labels from the top- K predicted categories, where K controls the competitivity level of the chosen noisy candidate labels.
Revised: As described in subsection 4.1, we generate the competitive label noise by randomly selecting irrelevant labels from the top- K predicted categories, where K controls the competitivity level of the chosen noisy candidate labels.

Original: A smaller K value forces the noisy candidate labels to be chosen from the fewer irrelevant labels with the highest few predicted probabilities of being the true label, and hence indicates a higher competitive noise level.
Revised: A smaller K value forces the noisy candidate labels to be chosen from the fewer irrelevant labels with the highest few predicted probabilities of being the true label, and hence indicates a higher competitive noise level.

Original: Here we conduct experiments on CIFAR-10 to investigate the impact of the competitive noise level by producing partial-label training data using different K values from { 4 , 5 , 6 , 8 } but the same average number of candidate labels, Avg#CL = 4.
Revised: Here we conduct experiments on CIFAR-10 to investigate the impact of the competitive noise level by producing partial-label training data using different K values from { 4 , 5 , 6 , 8 } but the same average number of candidate labels, Avg#CL = 4.

Original: The comparison results are presented in Figure 3.
Revised: The comparison results are presented in Figure 3.

Original: We can see that with the decrease of the K value and consequently the increase of the competitive noise level, the performance of all the comparison methods degrades in general.
Revised: We can see that with the decrease of the K value and consequently the increase of the competitive noise level, the performance of all the comparison methods degrades in general.

Original: However, the proposed ML-PLL method consistently outperforms all the other comparison methods.
Revised: However, the proposed ML-PLL method consistently outperforms all the other comparison methods.

Original: Moreover, the performance of ML-PLL is relatively more stable across different K values, and its performance gains over the other methods become larger with the increase of the competitive noise level (the decrease of K ), while the other methods encounter more dramatic performance degradation with smaller K values.
Revised: Moreover, the performance of ML-PLL is relatively more stable across different K values, and its performance gains over the other methods become larger with the increase of the competitive noise level (the decrease of K ), while the other methods encounter more dramatic performance degradation with smaller K values.

Original: These results again validate the effectiveness of ML-PLL in addressing competitive label noise.
Revised: These results again validate the effectiveness of ML-PLL in addressing competitive label noise.

Original: [Table]
Revised: [Table]

Original: A. I NFLUENCE OF N OISY L ABEL G ENERATION
Revised: A. I NFLUENCE OF N OISY L ABEL G ENERATION

Original: To further investigate the impact of noisy label generation on model performance, we follow (Wen et al., 2021) to conduct additional experiments with alternative noisy label generation processes.
Revised: To further investigate the impact of noisy label generation on model performance, we follow (Wen et al., 2021) to conduct additional experiments with alternative noisy label generation processes.

Original: Speciﬁcally, we generate noisy labels on CIFAR-10 with the following two commonly used label ﬂipping matrices in the form of [ ρ ij ] L × L (Wang et al., 2022b): (1) = 
Revised: Specifically, we generate noisy labels on CIFAR-10 with the following two commonly used label flipping matrices in the form of [ ρ ij ] L × L (Wang et al., 2022): (1) =   0 .

Original:   0 . · · ·  0 . 5 · · ·...

Original:  · · ·  , (2) =
Revised:  · · ·  , (2) =

Original: [Table]
Revised: [Table]

Original: 

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  
Revised:  

Original: Each entry ρ ij of the ﬂipping matrix speciﬁes the probability of the j -th label being selected as a candidate label when the i -th label is the ground-truth label.
Revised: Each entry ρ ij of the flipping matrix specifies the probability of the j -th label being selected as a candidate label when the i -th label is the ground-truth label.

Original: We conduct experiments with the partial label data generated using these two ﬂipping matrices and the comparison results are reported in Table 7, where case (1) and case (2) correspond to using the partial label data generated with the ﬂipping matrix (1) and (2), respectively.
Revised: We conduct experiments with the partial label data generated using these two flipping matrices and the comparison results are reported in Table 7, where case (1) and case (2) correspond to using the partial label data generated with the flipping matrix (1) and (2), respectively.

Original: From the table, we can see that the proposed ML-PLL method outperforms all the other methods in both cases.
Revised: From the table, we can see that the proposed ML-PLL method outperforms all the other methods in both cases.

Original: As case (2) is more challenging than case (1), all the other comparison methods produce much inferior performance in case (2).
Revised: As case (2) is more challenging than case (1), all the other comparison methods produce much inferior performance in case (2).

Original: However the proposed ML-PLL maintains similar good performance in both cases.
Revised: However the proposed ML-PLL maintains similar good performance in both cases.

Original: In the simple case (1),
Revised: In the simple case (1),

Original: ML-PLL outperforms the best comparison method, LWS, by 4.09%, while in the more challenging case (2), ML-PLL outperforms the best comparison method, CC, by 23.58%.
Revised: ML-PLL outperforms the best comparison method, LWS, by 4.09%, while in the more challenging case (2), ML-PLL outperforms the best comparison method, CC, by 23.58%.

Original: These additional experimental results further validate the effectiveness of the proposed method in addressing the partial label learning problem.
Revised: These additional experimental results further validate the effectiveness of the proposed method in addressing the partial label learning problem.