Estimating Transition Matrix with Diffusion Models for Instance-Dependent Label Noise

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 16:36:02

Abstract: Learning with noisy labels is a common problem in weakly supervised learning, where the transition matrix approach is a prevalent method for dealing with label noise. It estimates the transition probabilities from a clean label distribution to a noisy label distribution and has garnered continuous attention. However, existing transition matrix methods predominantly focus on class-dependent noise, making it challenging to incorporate feature information for learning instance-dependent label noise. This paper proposes the idea of using diffusion models for estimating transition matrix in the context of instance-dependent label noise. Specifically, we first estimate grouped transition matrices through clustering. Then, we introduce a process of adding noise and denoising with the transition matrix, incorporating features extracted by unsupervised pre-trained models. The proposed method enables the estimation of instance-dependent transition matrix and extends the application of transition matrix method to a broader range of noisy label data. Experimental results demonstrate the significant effectiveness of our approach on both synthetic and real-world datasets with instance-dependent noise. The code will be open sourced upon acceptance of the paper.

Main paper: 
 introduction: 
 For classification problems with given labels, deep neural networks have demonstrated significant improvements compared to traditional methods in recent years [25] . The efficacy of deep neural networks heavily relies on the accuracy of the labels. Directly incorporating polluted erroneous labels into network learning can result in the network fitting the noise, potentially severely impacting the predictive performance of the network [8] . However, in reality, obtaining accurate annotated data can be prohibitively expensive, and a substantial amount of data comes from the Internet or is annotated by non-expert annotators, inevitably containing noisy labels. Therefore, researching and promoting methods to mitigate the damage to models and make them more robust in the face of label noise data is a highly worthwhile problem to investigate, known as the problem of learning with noisy labels [23, 10, 34, 1] .Different approaches have been proposed to address the problem of label noise. One category [31, 22] involves the design of specialized loss functions or network structures to enhance the model's robustness against noisy labels. Another major category focuses on sample selection [2, 10, 14] ,where samples are partitioned into a set of clean samples and a set of contaminated noisy samples based on the magnitude of the loss or the similarity of extracted features. The labels of the noisy samples are then modified or their weights are reduced, followed by learning using semi-supervised methods. Sample selection methods are currently mainstream and have achieved promising results.However, the selection process relies heavily on intuition and lacks theoretical support. Additionally, the sample selection procedure is often complex and computationally intensive. In contrast, another Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute. significant category of methods is the transition matrix method [34, 17, 12, 42] , which estimates the transition probabilities from the clean label distribution to the noisy label distribution. This class of methods reveals the generation process of noisy labels and exhibits statistical consistency, often accompanied by theoretical analyses as methodological support. As a result, they have garnered continuous attention and occupy an important position in various algorithms for learning with noisy labels.In transition matrix methods, accurate estimation of the transition matrix is crucial. If an accurate estimation of the transition matrix can be obtained, along with the observed data for estimating the posterior distribution of the noisy labels, it is possible to infer the distribution of clean labels for neural network learning. Previous transition matrix methods [34, 17, 39] have mainly focused on class-dependent label noise, where a single transition matrix is estimated for all samples, which is typically straightforward. However, for instance-dependent label noise and complex real-world data, the label transition probabilities for each sample are not entirely identical. The transition matrix often depends on the specific features of individual samples, requiring the estimation of a separate transition matrix for each sample. However, in most cases, a single observed label corresponds to each sample in the dataset, making it an identifiability problem to estimate a separate transition matrix for each sample [20] . Although some methods [33, 41, 15] have utilized separate small networks to generate the transition matrix or divided the data into groups to transform it into a grouped class-dependent scenario, there still exist significant estimation errors and a lack of incorporating features effectively into the estimation of the transition matrix.To better incorporate the feature information of images into the estimation of the transition matrix, this work employs conditional diffusion models. The diffusion model originates from generative models and has been widely applied in various computer vision tasks in recent years [36, 7] , showing remarkable results. The proposed method revolves around the core idea of replacing image samples in the original diffusion process with a transition matrix. The matrix undergoes a process of adding noise and denoising, where the denoising step incorporates the sample features extracted by a pre-trained model as conditions. This generates a feature-dependent transition matrix. The constructed diffusion module is illustrated in Figure 1 . Additionally, considering the assumption that instance-dependent label noise is usually correlated with features [6] , clustering methods are utilized at the feature level to group samples. Preliminary estimations of the transition matrices are obtained for each group, which are then incorporated into the diffusion module for learning. The overall framework of the method is depicted in Figure 2 .The subsequent sections are organized as follows. Section 2 presents an in-depth review of the relevant works. In Section 3, we introduce our proposed model framework. Section 4 outlines the experimental analysis conducted on diverse synthetic and real-world noisy datasets, along with comparisons against other existing methods. Finally, we provide concluding in Section 5. The primary contributions of this paper can be summarized as follows:• We propose a method that utilizes diffusion models to add noise and denoise on the transition matrix, incorporating image features extracted through pre-trained encoder.• By combining the transition matrix-based diffusion model with feature-based clustering, we establish a framework capable of addressing instance-dependent label noise problems. • Our method demonstrates significant improvements over other transition matrix methods on both synthetic and real-world noisy datasets, and it achieves comparable performance to state-of-the-art methods. 
transition matrix methods: 
 Most previous methods for estimating transition matrix in the presence of label noise have primarily focused on class-dependent noise scenarios, simplifying the estimation process. Methods such as [24, 34] assume the existence of anchor points to identify the transition matrix. [17] and [39] introduce different regularization techniques to relax the anchor point assumption. Additionally, [26, 38] apply techniques such as meta-learning to estimate the transition matrix, but these approaches may require more clean data and computational resources. While these methods are effective for handling class-dependent label noise, they are not suitable for instance-dependent noise or real-world noisy data.However, estimating an individual transition matrix for each sample without additional assumptions or multiple noisy labels is infeasible [20] . To approximate the estimation of the instance-dependent transition matrix, [9] utilize an adaptation layer that estimates the transition matrix based on the output of each sample. [37] employs a separate network to estimate the transition matrix based on Bayesian labels. Some methods, such as [33, 30, 41] , employ clustering to learn part-dependent or group-dependent matrices, which can be viewed as a compromise between instance-dependent and class-dependent methods. Other approaches, including [6, 12] , utilize the similarity in the feature space to aid in learning the transition matrix. Although these instance-dependent transition matrix methods achieve identifiability through specialized treatments, they have not effectively utilized feature information in the learning process, resulting in errors in estimating feature-dependent transition matrices.Diffusion models, as generative models, have played a significant role in computer vision [36, 7] .Prominent examples include DDPM [11] , DDIM [27] , score matching methods [28] , and methods based on stochastic differential equations [29] . Diffusion models and their variants have been applied to various computer vision tasks such as image generation, image-to-image translation, text-to-image generation, among others. However, their application to the problem of label noise is relatively novel.To the best of our knowledge, only one existing work [3] has utilized diffusion models for addressing this problem. However, this work treats labels as the output of the diffusion model, which limits their expressive power due to the low dimension of the labels. Moreover, it overly relies on directly incorporating image features as conditions in the label generation process, which depends heavily on pre-trained models and may not be as reasonable as incorporating them into the transition matrix that reveals the process of noise generation. Experimental results also support this perspective.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: In the experimental section, we analyze the applicability and limitations of our method.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
method: 
 In this section, we present the definitions of symbols and introduce our method of using Diffusion models to construct the Transition Matrix (DTM).Let X ⊂ R d be the input image space, Y = {1, 2, • • • , C} be the label space, where C is the number of classes. Random variables (X, Y ), (X, Ỹ ) ∈ X × Y denote the underlying data distributions with true and noisy labels respectively. In general, we can not observe the latent true data samplesD = {(x i , y i )} N i=1 , but can only obtain the corrupted data D = {(x i , ỹi )} N i=1, where ỹ ∈ Y is the noisy label corrupted from the true label y, while denote corresponding one-hot label as y and ỹ.Transition matrix methods use a matrix T (x) ∈ [0, 1] C×C to represent the probability from clean label to noisy label, where the ij-th entry of the transition matrix is the probability that the instance x with the clean label i corrupted to a noisy label j. The matrix satisfies the requirement that the sum of each row C j=1 T ij (x) is 1, and usually has the requirement forT ii (x) > T ij (x), ∀j ̸ = i. Let P (Y |X = x) = [P (Y = 1|X = x), • • • , P (Y = C|X = x)] ⊤ be the clean class-posterior probability andP ( Ỹ |X = x) = [P ( Ỹ = 1|X = x), • • • , P ( Ỹ = C|X = x)] ⊤ be the noisy class-posterior probability, the formula can be write as:EQUATIONBy estimating the transition matrix and the noisy class-posterior probability, the clean class-posterior probability can be inferred byEQUATIONwhere the symbol -⊤ denotes the transpose of the inverse matrix.The majority of existing methods [24, 10, 17] focus on studying the class-dependent and instanceindependent transition matrix, i.e., T (x) ≡ T for ∀x. However, these methods are not applicable to instance-dependent noise scenarios where the transition matrix T (x) varies with respect to the input X. The main focus of our work is to utilize the feature information from input images to construct a instance-dependent transition matrix T (x).We adopt the classic DDPM model [11] from diffusion models as a reference to perform noise addition and denoising on the transition matrix. The diagram is illustrated in Figure 1 .For the forward diffusion process beginning with transition matrix T 0 ∼ q(T ), the process of gradually adding noise is obtained according to the following Markov process:EQUATIONfor m = 1, 2, • • • , M, where we use M to replace T , which is usually used in other diffusion models, in above equation for distinguishing from the symbol of transition matrix T .We aim to make the distribution of q(T M ) approach a standard normal distribution N (0, I) and through T M to conduct the reverse denoising process by fitting a neural network µ θ to fit the disttibution:EQUATIONwhere define βm = 4) denotes the pre-trained encoder for feature extraction.1-ᾱm-1 1-ᾱm β m , α m = 1 -β m , ᾱm = m i=1 α i . The f p in equation (The diffusion model can be learned by optimizing the evidence lower bound:EQUATIONwhereEQUATIONSimilar to the derivation and simplification process of DDPM, when a pre-trained encoder f p is provided along with the training data incorporating the initial transition matrix T , the learning algorithm for the diffusion model is presented in Algorithm 1.Input: Training data {x i , T i } N i=1 , pre-trained encoder f p . while not converged do Sample (x 0 , T 0 ) from data Sample m ∼ {1, • • • , M } Sample noise ϵ ∼ N (0, I)Take gradient descent step on the loss:∇ θ ϵ -ϵ θ √ ᾱm T 0 + √ 1 -ᾱm ϵ, x 0 , f p , m 2 end whileNext, for each image x, we can sample the corresponding transition matrix T (x) as shown in Algorithm 2.Sample T M ∼ N (0, I)for m = M, • • • , 1 do z ∼ N (0, I) if t > 1, else z = 0 T m-1 = 1 √ αm T m -1-αm √ 1-ᾱm ϵ θ (T m , x, f p , m) + σ m z end for Output: T 0 3.3 Feature-Dependent FrameworkFrom Algorithm 1, it can be observed that there are two components of the diffusion process that need to be provided in advance: the pre-trained encoder f p and the initial input T (x).The pre-trained encoder f p can be obtained through self-supervised learning or directly using the large model like CLIP. In our experiments, we employ the commonly used SimCLR [4] method in contrastive learning as the feature extraction model.On the other hand, the part involving the transition matrix T (x) used for learning the diffusion model is also related to the pre-trained encoder f p . Based on the assumption that the noise transition probability depends on image features, we adopt a group-dependent transition matrix as the initial input. We perform clustering algorithms at the feature extraction level f p (x), using the K-means method in our experiments, to group the image data. Then, based on the method VolMinNet [17] , we train class-dependent transition matrices for each group and obtain the initial transition matrix T (x) for each image x, which is then used as input in Algorithm 1. It is worth to note that the initial T (x) used as input for the diffusion process does not require different for each x. However, the denoising process of the diffusion model will further incorporate the feature information into the learning of the transition matrix.After obtaining the instance-dependent estimated transition matrix T (x), the neural network can be learned to fit the clean label distribution by the loss function:EQUATIONwheref ϕ (•) : X → ∆ C-1 (∆ C-1 ⊂ [0, 1] C is the C-dimensional simplex) is a differentiablefunction represented by a neural network with parameters ϕ and ℓ is a loss function usually using cross-entropy (CE) loss.The schematic diagram of the proposed framework is shown in Figure 2 , and the pseudocode is presented in Algorithm 3.Algorithm 3 A framework of DTMInput: Training set {(x i , y i )} N i=1 , pre-trained encoder f p , diffusion model ϵ θ , classification neural network f ϕ .1: Utilize input data to train f p or directly utilizing f p to extract features. 2: Perform K-means on feature space and estimate the transition matrix for each group to get data {x i , T i } N i=1 . 3: Train the diffusion model ϵ θ with Algorithm 1. 4: Sample instance-dependent train matrix T (x) for any input image x i with Algorithm 2. 5: Update the parameters of the classification network by incorporating the transition matrix T (x i ) into equation (7) . Output: Network parameters ϕ.Considering that the transition matrix typically require the sum of each row Through this transformation, we ensure that the learned transition matrix has row sums equal to 1 and that the diagonal elements are the largest in each row. In practical experiments, we apply the diffusion modeling discussed in subsection 3.2 to the matrix W , and then transform it into the transition matrix T for application. To simplify the notation, we uniformly use the term of transition matrix W to represent it, unless it leads to singularity.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: Upon acceptance of our paper, we will provide open-source code. The data we used is from commonly available open-source datasets.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experiments: 
 In this section, we present experimental findings to showcase the effectiveness of our proposed method compared to other methods. We evaluate our approach on both synthetic instance-dependent noisy datasets and real-world noisy datasets.We conduct experiments on following image classification datasets: CIFAR-10 and CIFAR-100 [13] ,CIFAR-10N and CIFAR-100N [32] , Clothing1M [35] , Webvision and ILSVRC12 [16] . Among them, CIFAR-10 and CIFAR-100 both have 32 × 32 × 3 color images including 50,000 training images and 10,000 test images. CIFAR-10 has 10 classes while CIFAR-100 has 100 classes. We generate instance-dependent noisy data on CIFAR-10 and CIFAR-100 with noise rates ranging from 10% to 50%, following the same generation method as in [33] . CIFAR-10N has three annotated labels, namely Random1, Random 2 and Random 3. The "Aggregate" is the aggregation of three noisy labels by majority voting, and the "Worst" is the dataset with the worst case. For CIFAR-100N, each image contains a coarse label and a fine label given by a human annotator. Clothing1M is a real-world dataset consisting of 1 million training images, consisting of 14 categories. WebVision contains 2.4 million images crawled from the websites using the 1,000 concepts in ImageNet ILSVRC12, but only the first 50 classes of the Google image subset are used in our experiments. For the validation set selection in our BTR method, we randomly sampled 10 samples from each observed class for each dataset to form the validation set, while the remaining samples were used for the training set.For the pre-trained model, we employ the commonly used SimCLR model [4] from contrastive learning, which directly performs self-supervised learning on input images without utilizing additional datasets. For the diffusion model, we follow the setup similar to DDPM [11] to set β 1 = 10 -4 , β M = 0.02 and utilize a similar U-Net network architecture but we reduce the M from 1000 to 10 to accelerate the learning process. As for the classification network, it may vary depending on the specific dataset. More specifically, for CIFAR-10/10N, we use ResNet-18 as the backbone network with batch size 128 and learning rate 0.05. For CIFAR-100/100N, we use ResNet-34 network with batch size 128, learning rate 0.02. For clothing1M, we use a ResNet-50 pre-trained with 10 epochs, batch size 64, learning rate 0.002 for network and divided by 10 after the 5th epoch. We use InceptionResNetV2 network on Webvision, with 100 epochs, batch size 32, learning rate 0.02 for network and divided by 10 after the 30th and 60th epoch. For clustering, we utilize the K-means method, where the number of clusters is set to 10 times the number of classes in the datasets. For the initialization of transition matrix, the update method and setting are consistent with [17] . While the updates for other parameters are performed using the stochastic gradient descent optimization method. In our experiments, we included the following common transition matrix and baseline methods as comparison: (1) VolMinNet [17] , (2) PeerLoss [21] (3) BLTM [37] , (4) PartT [33] , (5) MEIDTM [6] , as well as state-of-the-art methods for learning with noisy labels: (6) Co-teaching [10] , (7) ELR+ [18] , (8) DivideMix [14] , (9) SOP and SOP+ [19] , (10) PGDF [5] , (11) CC [40] , (12) LRA [3] with SimCLR as encoder similarly. We primarily validated our proposed method DTM against previous instance-based transition matrix methods on synthetic CIFAR-10/100 noise datasets. These methods mainly focus on estimating the transition matrix and some methods applicable to instance-dependent label noise. We performed 5independent runs for each experimental configuration, and the average values and standard deviations of each experiment are presented in Table 1 .The results demonstrate that our proposed DTR method outperforms other methods of the same category across various noise rates. It is evident that traditional transition matrix methods for classdependent noise as VolMinNet exhibit subpar performance when handling instance-dependent noise.While even advanced transition matrix methods for instance-dependent label noise such as BLTM, ParT and MEIDTM, still show significant gaps compared to our method.Furthermore, as the noise rates increase, the test accuracy of existing transition matrix methods significantly decline. This is particularly pronounced in the case of CIFAR-100 with 50% instancedependent noise (IDN) data, where all transition matrix methods achieve test accuracy below 60%.In contrast, our proposed DTR method achieves a remarkable test accuracy of 74.85%, showcasing its exceptional performance. That demonstrates relatively robust performance of DTM with only a slight decrease as the noise rate increases.This experiment clearly demonstrates that there is a significant performance gap between previous transition matrix methods and other advanced techniques, such as CC and LRA, when dealing with instance-dependent noise problems. However, the experimental results indicate that our proposed method DTM, which incorporates the diffusion model into the estimation of the transition matrix, outperforms these advanced techniques, except for the case of 40% noise in CIFAR-100, where our method slightly underperforms CC. It is evident that by leveraging the diffusion modeling to estimate the transition matrix, we effectively incorporate the image's feature information, leading to a substantial improvement in the effectiveness of the transition matrix.In addition to synthetic datasets, we also applied our method to real-world datasets and compared it with other state-of-the-art techniques for handling label noise problems. The results are presented in Table 2 and Table 3 . The results demonstrate that regardless of the type of noise labels, whether it is aggregated, random, or the worst-case scenario in CIFAR-10N, as well as in CIFAR-100N with more label categories, our method consistently achieves the best results in handling real-world noise. When dealing with large datasets like Clothing1M and complex image datasets like Webvision, DTM also performs comparably to other state-of-the-art methods.Through extensive experiments on five real-world datasets and the rusults on synthetic datasets above, our method outperforms the LRA method, which also utilizes the diffusion model for label noise problems. The LRA method models label diffusion with fewer dimensional information and lacks the rationale of our method, which considers noise generation from a transfer probability distribution perspective. The experiments demonstrate that our method achieves better learning performance by effectively integrating the transition matrix with the diffusion model.Besides the aforementioned experiments, we conducted ablation studies on proposed DTM method to assess the importance of each component. From the results in Table 4 , it can be observed that regardless of which component of diffusion module, pre-trained encoder module and clustering module is missing, the performance is consistently weaker compared to the original DTM. This indicates that each module plays a crucial role in our method. Our approach effectively combines the transition matrix, diffusion model, and pre-trained feature extraction, leading to significant improvements. 
conclusion: 
 In this paper, we propose a method that models the transition matrix using diffusion models, incorporating the feature information extracted by a pre-trained encoder into the estimation of the transition matrix. This approach enables the model to handle instance-dependent label noise with a wider range of applicability. Experimental results on both synthetic and real-world noisy datasets demonstrate the effectiveness of our proposed method.NeurIPS Paper Checklist Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The experimental section of the paper provides details of the model and data.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We conducted multiple repeated experiments to validate our approach and performed ablation experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We list the relevant details in the experimental section.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the positive implications of our work and ensure it does not have any negative societal impact.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? 
 
 Appendix: Justification: We submitted the paper following the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).Justification: There are no concerns in this regard regarding this work.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The data and code used in our work are all publicly available and open-source.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 


Decision: This work proposes to tackle the noisy label learning problem by modeling instance-dependent label transition matrices using a diffusion model.

All reviewers expressed their dissatisfaction with the lack of motivation of using a diffusion model to approximate the label transition matrix. The reviewers are also not convinced by the soundness of the approach. The training method using pseudo transition matrices and the initialization methods also raised concerns. The authors did not respond in the rebuttal phase.

Reviews: review: paper_summary: paper_summary: In this work, the authors proposed an approach to estimate the instance-dependent transition matrix in order to reliably learn from noisy labels. The idea is to use a condition diffusion model to estimate the transition matrix by using the pretrained extracted image features as the conditions. Once the transition matrices are estimated, the classifier is learned through the corrected cross entropy loss. Experiments are presented to compare the performance of the approach with other baselines using both synthetic and real noisy datasets.
strength_weakness: strength_weakness: The main weakness is the lack of support and discussion in substantiating the idea. Experiments are insufficient to support the claims.
questions: questions: Questions:

1.	A key concern is that estimation of the transition matrix is heavily dependent on the initializations given to the diffusion model learning. The diffusion model intuitively tries to approximate the distribution of its inputs through its forward and reverse process. In the traditional setting, the original image features is the input. But in your case, the initializations estimated through clustering and volmin optimization are the inputs. This part is quite unclear how does it help learn the true instance-dependent transition matrices.

2.	In the experiments, in Table 4, I do not see the ablation study with just using the initialized transition matrix and training the classifier, which is important to see the effect of the diffusion model-based learning for the TM. The ablation study corresponding to “w/o diffusion” says that it is using the pre-trained model. 

3.	Experiment results all look good compared to the baselines, but I do not see any clear intuition/discussion to substantiate this idea of instance-dependent transition matrix estimation
limitations: limitations: No limitations are discussed
paper_summary: paper_summary: This paper focuses on the estimation of the transition matrix with instance-dependent label noise. They used a diffusion model for this estimation. By applying a diffusion process to the transition matrix, the diffusion model is trained to generate transition matrices from a prior distribution. The instance-wise generated transition matrix is then used to train the classifier with a forward cross-entropy loss. The improvement of the method is demonstrated by experiments on benchmark and real-world datasets.
strength_weakness: strength_weakness: * The reason for generating the transition matrix using a diffusion model is unclear.
  * The instance-dependent transition matrix is the target to be estimated, but it is uncertain what role training a diffusion model to generate the transition matrix without a fixed target.
  * In addition, as mentioned by the authors, the transition matrix must be satisfied: the entries are greater than 0, the row sum is to be 1, and the diagonal entry is typically the largest. However, these considerations have not been taken into account in the construction of the diffusion process. Although a transformation method is proposed in Section 3.4, there is no discussion of how this affects the training of the diffusion model.

* Pre-trained features are fed into the diffusion network, but their impact on the diffusion process has not been analysed. This could be seen as providing additional conditional information during the diffusion process, implying that this diffusion model might be a conditional diffusion model. It would be better to discuss these consideration.

* In Algorithm 3, it appears that the diffusion model is trained in order to generate the initialized $T_i$. I wonder if the desired training is for the initialized $T_i$ to be generated perfectly as is. This could lead to a transition matrix that might not contain instance-dependent information, raising questions about the mechanism by which diffusion training introduces variance.

* The diffusion training seems to take a considerable amount of time, which needs to be analysed. If it takes a long time, the performance improvement may not be significant in comparison.
questions: questions: Please see the Weaknesses part.
limitations: limitations: They mentioned the limitations only briefly in the experimental section. I have noted additional limitations that I perceive in the Weaknesses part.
paper_summary: paper_summary: This paper deals with the problem of supervised learning from noisy labels, where the label noise is modeled using instance-dependent label transition probability matrix. Mainly, this work attempts to leverage conditional diffusion model in order to obtain a generative model of transition matrix conditioned on the sample features. To that end, this work first generate pseudo paired samples $( x_i, T_i )_{i=1}^N$ using existing method (VolMinNet). Secondly, a conditional diffusion model is trained that generates $T_i$ given $x_i$. Finally, the classifier is trained taking into consideration the estimated transition matrix from the diffusion model.
strength_weakness: strength_weakness: 1. The technical soundness of the proposed method is questionable. Essentially, the proposed method trains a conditional diffusion model using paired samples $(x_i, T_i)$. If we consider the true transition matrix as $T(x)$ for a sample $x$, then the idea of the proposed method is to train a conditional generative model $p( T(x) | x )$. There are several issues with this attempt and the proposed implementation:
   (a) The authors use pseudo transition matrix $T_i$ generated from a sample-independent method (VolMinNet). $T_i$ only depends upon the cluster assignment of $x_i$. The diffusion model, at best, can approximate the conditional distribution $p( T_i | x_i )$. This has no clear relation to $p(T(x) | x)$. Therefore, in principle, the transition matrix generated by the trained diffusion model cannot be better than that returned by VolMinNet.
   (b) Second, the transition matrix is modeled as a deterministic function of sample, i.e., only one $T(x)$ exists for a given $x$. Therefore, it does not make sense to learn a generative model for $p(T(x) | x)$, since it is a degenerate distribution (probability of all other matrices should be zero except the true $T(x)$). 

2. Another hint at why the proposed method should be limited by the pseudo paired sample distribution is that the diffusion model training part (which is ultimately used as transition matrix estimator) does not require available noisy labels. Hence, no extra information can be extracted about the true transition matrix $T(x)$ beyond the information captured by the pseudo paired samples $(x_i, T_i)$. 

2. It is unclear where the performance gain in empirical results is coming from. The manuscript does not provide any intuitive or theoretical explanation to justify the quality of their estimator. Moreover, no rationale for the algorithm design is provided.
questions: questions: Please provide a rebuttal for each of the points in weaknesses section.
limitations: limitations: Limitations are not adequately discussed.