SiDyP: Simplex Diffusion with Dynamic Prior for Denoising Llama-Generated Labels

Authors: Liqin Ye, Agam Shah, Chao Zhang, Sudheer Chava
Primary area: generative models
Keywords: Diffusion Model, Learning from Noisy Labels, Soft Labels

Abstract:
The traditional process of creating labeled datasets is not only labor-intensive but also expensive. Recent breakthroughs in open-source large language models (LLMs), such as Llama-3, have opened a new avenue in generating labeled datasets automatically for various natural language processing (NLP) tasks to provide an alternative to such expensive annotation process. However, the reliability of such auto-generated labels remains a significant concern due to inherent inaccuracies. When learning from such noisy labels, the model's generalization is likely to be harmed as it is prone to overfit those label noises. In this paper, we propose the \textbf{Si}mplex Diffusion with a \textbf{Dy}namic \textbf{P}rior (\textbf{SiDyP}) model to calibrate \hl{classifier's} predication, thus enhancing its robustness towards noisy labels. Our framework leverages simplex diffusion model to iteratively correct noisy labels conditioned on training dynamic trajectories obtained from classifier finetuning. \hl{The \textbf{P}rior in SiDyP refers} to the potential true label candidates which was obtained according to neighborhood label distribution in text embedding space. \hl{It is \textbf{Dy}namic because we} progressively distill these candidates based on the feedback of the diffusion model. Our SiDyP model can increase the performance of the BERT classifier fine-tuned on both zero-shot and few-shot Llama-3 generated noisy label datasets by an average of 5.33\% and 7.69\% respectively. Our extensive experiments, which explore different LLMs, diverse noise types (real-world and synthetic), ablation studies, and multiple baselines, demonstrate the effectiveness of SiDyP across a range of NLP tasks. We will make code and data publicly (under a CC BY 4.0 license) available on GitHub upon publication of the work.

Decision: reject
Reviewer scores: 5, 5, 5, 5

Reviews and discussion:
<thread>
creation date:
1730554942669

thread_type:
official_review

<thread_object>
creator: reviewer_a1cc

<content>
summary:
This paper presents a work on learning with noisy labels in datasets annotated by LLMs. The proposed framework, SiDyP, is based on a noisy-supervised model called PLC and a simplex diffusion model. In this framework, the PLC model, fine-tuned with the noisy labeled data, is used to obtain dynamic training trajectories and induce a prior distribution, while the diffusion model serves as a posterior to iteratively calibrate PLC’s prior. Specifically, this paper introduces a true label candidates retrieval algorithm and a prior dynamic distillation
algorithm to mitigate the negative impact brought by noisy labels. Extensive experiments demonstrate that the proposed framework achieves competitive classification performance on multiple tasks.

strengths:
1. Denoising LLM-generated labels may be a promising direction and is worth further investigation.
2. Experimental results are good, which shows that the proposed method is a state-of-the-art one.

weaknesses:
1.The novelty of this paper is limited, capturing the label prior distribution based on training dynamics and using the simplex model for iterative refinement are the technical contributions of DyGen and TESS, respectively. The contributions of this paper seem like engineering innocations.

2.The proposed method introduces a few more hyperparameters. In practice, when we don't have a clean validation set, these hyperparameters can be difficult to set.

3.Writing needs to be improved. the dynamic prior needs to be clearly explained in the abstract and introduction, which can easily be confused with training dynamics. Additionally, the organization of Section 3 is not reasonable.

4.A more detailed ablation study would have been desirable, such as the number of candidate labels in Algorithm 1 and the number of iterations in Algorithm 2.

questions:
1.Compared to DyGen, what are the main contributions of SiDyP? It seems that SiDyP just replaces the VAE part in DyGEN with a simplex diffusion model. There are no insightful innovations.

2.Line 165, why do we only consider the two candidate categories with the highest probabilities? In fact, the types and distributions of labels vary significantly across different tasks.

3.Some descriptions and settings of hyperparameters can be confusing. Does the estimated error rate refer to the noise ratio? What are the bases for setting these hyperparameters?

4.The symbols in Figure 1 are inconsistent with those in the main text.

5.Section 4.5, the effectiveness of the proposed label candidate retrieval algorithm has not been validated.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730581056929

thread_type:
official_review

<thread_object>
creator: reviewer_uucy

<content>
summary:
This paper proposes a method called SiDyP to address the learning-from-noisy-labels problem.  
In the experiments, the authors validated the effectiveness of the proposed method using noisy data generated from the zero-shot and few-shot inference of Llama-3.

strengths:
The authors conducted experiments using noisy labels generated by LLM Llama and demonstrated the effectiveness of the proposed method, which is one of the contributions of this paper.

weaknesses:
1)	On the novelty of introducing LLMs for generating noisy data for learn-from-noisy-labels methods.  
The authors claimed in the Introduction that, this is the first time that LLMs-generated datasets have been introduced in the realm of learning from noisy labels.
Actually, this statement is not rigorous. 
There are already some works that focus on learning from noisy labels generated by LLMs [1,2,3].
These works concern the topic of Weak Supervision, which can be seen as a subset of the learning-from-noisy-labels problem; even though Weak Supervision considers the context of each one sample with multiple noisy labels, the learning-from-noisy-labels problem often refers to the context of one sample with one label.
Anyway, the authors should take these prior works into account while writing.

[1] Alfred: A System for Prompted Weak Supervision. ACL-2023 (demo).  
[2] Combining prompt-based language models and weak supervision for labeling named entity recognition on legal documents. Artificial Intelligence and Law, 2024.  
[3] https://snorkel.ai/blog/few-shot-learning-large-language-models/. Blog from Snorkel.


2) On the core research motivation and contributions.  
Related to the first issue mentioned above, and more importantly, the core research motivation and contributions of this paper are not sufficiently clear:  

    &emsp;2.1) If the designed method is more specifically tailored to noisy labels generated by LLMs (rather than noisy labels in the general learning-from-noisy-labels works), or in other words, if the proposed method is specifically designed for noisy labels generated by LLMs, then:
    The authors should analyze from theoretical or experimental perspectives whether noisy labels generated by LLMs have different characteristics compared to noisy labels in the traditional sense, and why the designed method is well-suited to handle these characteristics.

    &emsp;2.2) If the designed method is not specifically aimed at noisy labels generated by LLMs, then this paper essentially presents a new typical learning-from-noisy-labels method.    

    &emsp;2.3) In summary, the authors do not clarify in the paper whether noisy labels generated by LLMs have different characteristics compared to traditional noisy labels, nor do they specify whether their contributions are specifically designed for noisy labels generated by LLMs.

questions:
Please refer to the above Weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730592667296

thread_type:
official_review

<thread_object>
creator: reviewer_svro

<content>
summary:
The paper presents a novel approach using a simplex diffusion model for denoising labels generated by large language models (LLMs). The method aims to address the inherent noise in LLM-generated labels by leveraging an iterative denoising process within a continuous probability simplex space. The approach is motivated by the high cost and time demands of manual labeling, coupled with the practicality of LLMs for fast, albeit noisy, label generation. The paper introduces several key assumptions, including the local consistency of embedding spaces and the ability of diffusion models to gradually refine noisy labels into accurate representations. Experiments across different noise types, including symmetric, asymmetric, and instance-dependent noise, demonstrate the model's robustness. The results suggest that the method effectively reduces noise without requiring additional labeled data, making it a potentially valuable contribution to applications reliant on large-scale automatically generated labels.

strengths:
The paper has several notable strengths across originality, quality, clarity, and significance:

Originality: The approach is innovative in applying a simplex diffusion model to iteratively denoise noisy LLM-generated labels. Using the probability simplex space to handle label noise is a creative application, enabling a gradual refinement of noisy labels, which is distinct from traditional denoising methods. This method also addresses the growing need for handling noise in labels generated by LLMs, making the approach highly relevant and timely.

Quality: The paper presents a structured approach with a combination of techniques, such as embedding-based candidate selection, to effectively manage noise. The experiment setup is comprehensive, covering multiple noise types (symmetric, asymmetric, instance-dependent), which reflects the robustness of the method. By comparing the proposed model to various baselines, the paper demonstrates a clear performance advantage in multiple scenarios, supporting the claim that the simplex diffusion model can enhance accuracy in noisy data environments.

Clarity: The overall structure of the paper is clear, with a logical flow from the problem formulation to the model design and experimental results. The motivation for the proposed method is well-presented, with a clear description of how each component of the simplex diffusion process contributes to the final denoised output.

Significance: The work is significant for applications that rely on LLM-generated labels, which often contain noise. By introducing a method that reduces noise without needing additional labeled data, the paper addresses a practical challenge, potentially improving the scalability and quality of labeling in real-world scenarios. This could be valuable across fields where large-scale data labeling is a bottleneck, such as natural language processing, computer vision, and medical imaging.

weaknesses:
While the paper is promising, several areas could benefit from further improvement:

The paper lacks in-depth theoretical discussion regarding why the simplex diffusion model is particularly well-suited to the task of LLM label denoising. For example, the rationale behind using iterative denoising in the simplex space could be strengthened by exploring why this approach may perform better than existing denoising methods in managing specific types of noise common to LLM-generated labels.

Key Assumptions Verification: The paper is based on several assumptions, such as the local consistency of embeddings and the relationship between training dynamics and sample quality. However, these assumptions are not directly validated in the experiments. For instance, the assumption that noisy labels will have larger mean and standard deviation distances in the embedding space could be tested and quantified to provide stronger empirical support.

The paper could provide a more detailed analysis of the noise characteristics specific to LLM-generated labels. The motivation would be more convincing if it explained how these labels differ from other types of noise and why standard denoising methods are insufficient. Understanding these nuances would clarify why the simplex diffusion model is a more suitable choice.

Although the experiments include different noise types, they do not explore the method’s limitations in scenarios where noise is highly complex or includes extreme cases. Additionally, there is a lack of quantitative analysis regarding the training dynamics feature assumption, which would help validate the model’s robustness and provide further insights.

The paper lacks a failure analysis that would highlight the model's limitations. Discussing scenarios where the model underperforms could provide readers with a more realistic understanding of its applicability and suggest areas for future improvement.

questions:
Noise Characteristics in LLM Labels: Could the authors provide more insights into the specific noise patterns in LLM-generated labels that this model targets? Understanding these patterns would clarify why traditional denoising methods might struggle with this type of noise.

Embedding Space Consistency Assumption: Can the authors validate the assumption that the local consistency of embeddings allows clean and noisy labels to be differentiated? This could be demonstrated by visualizing the embedding space to show the distribution differences between noisy and clean labels.

Training Dynamics Validation: Could the authors conduct a more detailed analysis of the training dynamics related to noisy samples? Quantifying the mean and standard deviation differences in distances for noisy versus clean samples would strengthen the paper’s empirical foundation.

Application Scenarios and Limitations: In which specific scenarios would the simplex diffusion model be most applicable or potentially limited? Additional examples would help readers understand the practical applicability of the method.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730681775295

thread_type:
official_review

<thread_object>
creator: reviewer_7vxe

<content>
summary:
This paper proposes applying simplex diffusion to denoise annotations generated by large language models (LLMs). The core idea is to first obtain noisy annotations from LLMs and then iteratively refine these labels with a simplex diffusion model, which maps labels to a continuous probability space. The main experiments are conducted across four common text classification datasets using Llama 3.1-70b. Overall the results show that the proposed method achieves competitive performance in both zero-shot and few-shot settings. Additionally, the authors explore the method’s robustness by testing with different 4 LLM backbones and different types of label noises.

strengths:
Overall, I think this paper is well-written and relatively easy to follow. The results are very competitive and clearly show the advantage of the proposed method. I think the construction of experiments is correct.  Additionally, the authors clearly communicate the problem and the potential impact of their approach on improving LLM-generated annotations, which would be a valuable contribution to the field.

weaknesses:
= The comparisons with Mixtral models might not be entirely fair. If the authors used the base Mixtral-8x22b model, which is not instruction/chat-finetuned, it may not perform comparably to other models that have been finetuned on instruction data. For a fairer comparison, using Mixtral-8x22b-Instruct-v* or similar instruction-tuned models would better align with the capabilities of the other models tested.

= Insufficient literature coverage: Overall, this paper aims to solve the problem of LLM for data annotation and reduce annotation noise. I think it would be justifiable to have a better coverage of related work especially on LLM for data annotations. Give the limitation in space for the initial submission, I expect the authors to conduct a more rigorous literature survey of related work.

= Although the approach is novel for this specific problem of denoising llm annotaions, the method largely adapts simplex diffusion for signal denoising directly to noisy label refinement. This may raise questions regarding the technical novelty, as the adaptation itself does not appear to involve significant theoretical modifications.

questions:
It would be helpful if the authors could clarify the following:

= For missing labels, why was a random label assignment chosen? Given the low failure rate, might it be preferable to disregard these cases instead?

= Were any specific system prompts or chat templates used when applying the prompts for LLM label generation? If so, could the authors provide details?

= In the few-shot experiments, the authors selected one example per label. Were these demonstration examples sampled from the training set or the validation set?

= For the 20News dataset, is there a reason why no results are reported for the few-shot experiments?

= Since the validation set is used for model selection, as an exploratory experiment, have the authors considered prompting the LLM with a larger few-shot sample (e.g., 50+ examples) from the validation set, given the support for long contexts in the LLMs used? It would be interesting to see how the efficiency of this approach compares with the proposed diffusion-based denoising pipeline.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731769607202

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Rebuttal by Authors

comment:
We sincerely thank all the reviewers for their time, thoughtful comments, and constructive feedback. We are encouraged that the reviewers appreciate the following aspects of our work:

- Our work introduces a novel framework, SiDyP, which effectively addresses the challenge of noisy labels in datasets generated by large language models like Llama-3 (sVro).
- Our experimental setup is robust and comprehensive, providing a solid evaluation across various noise conditions and multiple text classification datasets. (a1Cc) The reviewers find that the experimental results are competitive and demonstrate state-of-the-art performance. (7vxE, sVro, uUcy, a1Cc)
- We have clearly communicated our contributions, and reviewers have found the paper well-written and easy to follow (7vxE, sVro). 

We attempted our best to address the questions. We believe the comments and revisions have made the paper stronger and thank all the reviewers for their help. We try our best to highlight the revision we make in the rebuttal submission PDF. Please find individual responses to your questions below.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732293523358

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Thank You and a Follow-Up to Reviewers

comment:
Dear Reviewers,  

Thank you for your positive reviews of our work.  

We would appreciate your consideration of our responses to the comments you provided. We hope they address your concerns and aid in your overall assessment of our work.  

Thank You,  
Authors
</content>
</thread_object>

</thread>
<thread>
creation date:
1733285029953

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Rebuttal Summary

comment:
Dear Reviewers, AC, SAC, 

We sincerely thank the AC and SAC for their invaluable efforts in assigning reviews and guiding the process. We also deeply appreciate the reviewers for taking the time to read our paper and provide constructive feedback. 

Although all reviewers remain silent during the discussion phase, here we summarize our rebuttal to present a high-level perspective that could hopefully help grasp our contribution and modification quickly. 

- We focus on enhancing the learning from LLM-generated label noise, a new type of label noise that has not been explored before. Previous studies mainly focus on synthetic noise and real-world noise. 
- We introduce label candidates and label distillation algorithm to improve the prior information ($y$) quality when estimating the posterior $p(y|\tilde{y}, x)$.
- We propose a simplex denoise label diffusion model to approximate the posterior $p(y|\tilde{y}, x)$, which provides a new perspective of reconstructing true labels from noisy labels.
- Extensive experiments across 4 NLP tasks, 5 LLMs, and 3 different types of noises are conducted to show the effectiveness of our framework. Although our focus is on LLM-generated label noise, we also test SiDyP under synthetic noise and real-world noise to demonstrate its robustness and applicability to all types of noises

We try our best to address all reviewers’ concerns and questions, and modify our draft accordingly:

- We add more related works on LLM data annotation and LLMs under weak supervision to address Reviewer uUcy and Reviewer 7vxE’s concern of insufficient literature coverage. 
- We clarify our research question and motivation in Section 1&2 (Reviewer uUcy) 
- We explain the term “dynamic prior” in our content and reorganize section 3 according to Reviewer a1Cc’s feedback on the presentation. 
- We incorporate noise characteristic analysis of LLM-generated label noise to show its difference from other types of noise. (Reviewer sVro)
- We enrich our theoretical discussion in Section 5.5. (Reviewer sVro)
- We clarify our choice of using a noisy validation set for model selection, inspired by Zhu et al (2023) to address Reviewer a1Cc and 7vxE’s concerns. 
- We expand the ablation experiments to show the effectiveness of our distillation algorithm (Reviewer a1Cc). 
- We clarify our hyper-parameter search procedure. (Reviewer a1Cc).

Over the past month, we have put forth our best efforts to improve the quality of this paper and address each concern raised by the reviewers. We sincerely hope that our work will make a valuable contribution to the community. Thank you once again for your kind support and constructive feedback.

Sincerely,

The Authors



[1] Zhu, Dawei, et al. "Weaker than you think: A critical look at weakly supervised learning." arXiv preprint arXiv:2305.17442 (2023).
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper presents a new method for denoising labels generated by large language models (LLMs) using a simplex diffusion model. Reviewers mostly agreed that the paper is clearly written and the experiments are well designed. However, there are also some major limitations of this work, such as novelty, assumptions, theoretical analysis, missing related work, etc. Overall, the current version of this work is not ready for publication at ICLR.