Original: S ELF -G UIDED N OISE -F REE D ATA G ENERATION FOR E FFICIENT Z ERO -S HOT L EARNING
Revised: S ELF -G UIDED N OISE -F REE D ATA G ENERATION FOR E FFICIENT Z ERO -S HOT L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: There is a rising interest in further exploring the zero-shot learning potential of large pretrained language models (PLMs).
Revised: There is a rising interest in further exploring the zero-shot learning potential of large pre-trained language models (PLMs).

Original: A new paradigm called data-generation-based zero-shot learning has achieved impressive success.
Revised: A new paradigm called data-generationbased zero-shot learning has achieved impressive success.

Original: In this paradigm, the synthesized data from the PLM acts as the carrier of knowledge, which is used to train a task-specific model with orders of magnitude fewer parameters than the PLM, achieving both higher performance and efficiency than prompt-based zero-shot learning methods on PLMs.
Revised: In this paradigm, the synthesized data from the PLM acts as the carrier of knowledge, which is used to train a task-specific model with orders of magnitude fewer parameters than the PLM, achieving both higher performance and efficiency than prompt-based zero-shot learning methods on PLMs.

Original: The main hurdle of this approach is that the synthesized data from PLM usually contains a significant portion of low-quality samples.
Revised: The main hurdle of this approach is that the synthesized data from PLM usually contains a significant portion of low-quality samples.

Original: Fitting on such data will greatly hamper the performance of the taskspecific model, making it unreliable for deployment.
Revised: Fitting on such data will greatly hamper the performance of the taskspecific model, making it unreliable for deployment.

Original: Previous methods remedy this issue mainly by filtering synthetic data using heuristic metrics(e.g., output confidence), or refining the data with the help of a human expert, which comes with excessive manual tuning or expensive costs.
Revised: Previous methods remedy this issue mainly by filtering synthetic data using heuristic metrics(e.g., output confidence), or refining the data with the help of a human expert, which comes with excessive manual tuning or expensive costs.

Original: In this paper, we propose a novel noise-robust re-weighting framework S UN G EN to automatically construct high-quality data for zero-shot classification problems.
Revised: In this paper, we propose a novel noise-robust re-weighting framework S UN G EN to automatically construct high-quality data for zero-shot classification problems.

Original: Our framework features the ability to learn the sample weights indicating data quality without requiring any human annotation.
Revised: Our framework features the ability to learn the sample weights indicating data quality without requiring any human annotation.

Original: We theoretically and empirically verify the ability of our method to help construct good-quality synthetic datasets.
Revised: We theoretically and empirically verify the ability of our method to help construct good-quality synthetic datasets.

Original: Notably, S UN G EN -LSTM yields a 9.8% relative improvement than the baseline on average accuracy across eight different established text classification tasks.
Revised: Notably, S UN G EN -LSTM yields a 9.8% relative improvement than the baseline on average accuracy across eight different established text classification tasks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Owing to the superior generative capacity of large-scale pre-trained language models (PLMs), there has been an emerging trend of using these powerful models (e.g., GPT) to generate training data for downstream tasks (Anaby-Tavor et al., 2020; Puri et al., 2020; Kumar et al., 2020; Lee et al., 2021, inter alia ).
Revised: Owing to the superior generative capacity of large-scale pre-trained language models (PLMs), there has been an emerging trend of using these powerful models (e.g., GPT) to generate training data for downstream tasks (Anaby-Tavor et al., 2020; Puri et al., 2020; Kumar et al., 2020; Lee et al., 2021, inter alia ).

Original: Among them, a new line of generation-based zero-shot learning using the unfinetuned PLM pushes the envelope further (Schick & Schütze, 2021;
Revised: Among them, a new line of generation-based zero-shot learning using the unfinetuned PLM pushes the envelope further (Schick & Schütze, 2021;

Original: Ye et al., 2022; Meng et al., 2022), featuring total annotation-free training for downstream tasks.
Revised: Ye et al., 2022; Meng et al., 2022), featuring total annotation-free training for downstream tasks.

Original: Ye et al.
Revised: Ye et al.

Original: (Z ERO G EN ) further boosts the efficiency by using the generated data to train tiny task models (TAM), which have orders-of-magnitude fewer parameters than the PLM.
Revised: (Z ERO G EN ) further boosts the efficiency by using the generated data to train tiny task models (TAM), which have ordersof-magnitude fewer parameters than the PLM.

Original: Specifically, they first design prompts incorporating the task description and label information, then use them to guide the data generation from the PLM.
Revised: Specifically, they first design prompts incorporating the task description and label information, then use them to guide the data generation from the PLM.

Original: Subsequently, the synthesized dataset is used to train the tiny task-specific models.
Revised: Subsequently, the synthesized dataset is used to train the tiny task-specific models.

Original: Compared with the classic prompt-based zero-shot learning on PLM, this new paradigm enjoys two favorable properties: (1) since the task model has orders-of-magnitude fewer parameters than the PLM, it demonstrates much lower inference latency; (2) with the large amount of PLM-generated training data, the task model often shows better performance than prompt-based zero-shot PLM counterparts.
Revised: Compared with the classic prompt-based zero-shot learning on PLM, this new paradigm enjoys two favorable properties: (1) since the task model has orders-of-magnitude fewer parameters than the PLM, it demonstrates much lower inference latency; (2) with the large amount of PLM-generated training data, the task model often shows better performance than prompt-based zero-shot PLM counterparts.

Original: In the above paradigm, the amount and the quality of the generated data arethe crucial factors for the task model’s performance.
Revised: In the above paradigm, the amount and the quality of the generated data are crucial factors for the task model’s performance.

Original: Unfortunately, despite we can generate unlimited training data  in theory, the data quality is not always guaranteed.
Revised: Unfortunately, despite the unlimited training data that one can generate in theory, the data quality is not always guaranteed.

Original: Our experimental observation across many downstream tasks verifies the existence of this issue: in Z ERO G EN , after a few training epochs on the PLM-generated dataset, although the training accuracy steadily improves, the actual test accuracy of the model starts declining rapidly (e.g., IMDb in Figure 1) – a clear indication of the model overfitting to low-quality data (noisy data) (Arpit et al., 2017).
Revised: Our experimental observation across many downstream tasks verifies the existence of this issue: in Z ERO G EN , after a few training epochs on the PLM-generated dataset, although the training accuracy steadily improves, the actual test accuracy of the model starts declining rapidly (e.g., IMDb in Figure 1) – a clear indication of the model overfitting to low-quality data (noisy data) (Arpit et al., 2017).

Original: More specifically, we identify two major cases of noisy samples in the synthetic dataset: corrupted labels and task-irrelevant samples (Table 6 in Appendix).
Revised: More specifically, we identify two major cases of noisy samples in the synthetic dataset: corrupted labels and task-irrelevant samples (Table 6 in Appendix).

Original: Without any task-related fine-tuning, it is challenging for PLM to follow a user’s instruction (task-specific prompt including label information) to generate accurate samples in the target domain (Ouyang et al., 2022).
Revised: Without any task-related fine-tuning, it is challenging for PLM to follow a user’s instruction (task-specific prompt including label information) to generate accurate samples in the target domain (Ouyang et al., 2022).

Original: To alleviate the data quality issue, recent work adopts human-active labeling to correct the corrupted label or revise the example (Wang et al., 2021a;
Revised: To alleviate the data quality issue, recent work adopts human-active labeling to correct the corrupted label or revise the example (Wang et al., 2021a;

Original: Liu et al., 2022).
Revised: Liu et al., 2022).

Original: However, such methods introduce considerable costs and may be unrealistic.
Revised: However, such methods introduce considerable costs and may be unrealistic.

Original: [Figure]
Revised: [Figure]

Original: To avoid human intervention, the classic approach to eliminate the effect of noisy data is to re-weight the samples.
Revised: To avoid human intervention, the classic approach to eliminate the effect of noisy data is to re-weight the samples.

Original: The core idea is to design a weighting function w , such that the correct samples are associated with larger weights and the noisy ones with smaller weights.
Revised: The core idea is to design a weighting function w , such that the correct samples are associated with larger weights and the noisy ones with smaller weights.

Original: Compared with heuristic design of w (e.g., according to output confidence, loss value) (Liu & Tao, 2015; Wang et al., 2021b), which requires task-specific knowledge and excessive manual tuning, the adaptive methods that learn the sample weights in an end-to-end manner demonstrate better performances in practice (Ren et al., 2018;
Revised: Compared with heuristic design of w (e.g., according to output confidence, loss value) (Liu & Tao, 2015; Wang et al., 2021b), which requires taskspecific knowledge and excessive manual tuning, the adaptive methods that learn the sample weights in an end-to-end manner demonstrate better performances in practice (Ren et al., 2018;

Original: Shu et al., 2019; Zheng et al., 2021).
Revised: Shu et al., 2019; Zheng et al., 2021).

Original: Those methods typically formulate the learning of sample weights into a bi-level optimization problem, with a clean validation set in the outer loop to guide the learning of w .
Revised: Those methods typically formulate the learning of sample weights into a bi-level optimization problem, with a clean validation set in the outer loop to guide the learning of w .

Original: Despite remarkable success was achieved, their dependence on a clean validation set becomes a major limitation, which is especially impractical in zero-shot setting.
Revised: Despite remarkable success was achieved, their dependence on a clean validation set becomes a major limitation, which is especially impractical in zero-shot setting.

Original: Our solution comes from rethinking the choice of the outer objective in the bi-level framework: can we design an objective such that the sample weights can be optimized with only access to the noisy synthetic data ?
Revised: Our solution comes from rethinking the choice of the outer objective in the bi-level framework: can we design an objective such that the sample weights can be optimized with only access to the noisy synthetic data ?

Original: To this end, we resort to a family of noise-robust loss functions ( ℓ robust )
Revised: To this end, we resort to a family of noise-robust loss functions ( ℓ robust )

Original: (Ghosh et al., 2017; Zhang & Sabuncu, 2018).
Revised: (Ghosh et al., 2017; Zhang & Sabuncu, 2018).

Original: These functions were adopted by previous work to train the neural network under label noise due to their theoretically noise-tolerant property (Ghosh et al., 2017; Zhang & Sabuncu, 2018; Wang et al., 2019).
Revised: These functions were adopted by previous work to train the neural network under label noise due to their theoretically noise-tolerant property (Ghosh et al., 2017; Zhang & Sabuncu, 2018; Wang et al., 2019).

Original: However, from the optimization point of view, such loss functions suffer from instability and difficulty when training the neural networks (Zhang & Sabuncu, 2018), which limits their effectiveness.
Revised: However, from the optimization point of view, such loss functions suffer from instability and difficulty when training the neural networks (Zhang & Sabuncu, 2018), which limits their effectiveness.

Original: Remarkably, our approach leverages the noise-tolerant property of these losses, while avoiding their pathology.
Revised: Remarkably, our approach leverages the noise-tolerant property of these losses, while avoiding their pathology.

Original: We propose a novel bi-level re-weighting framework S UN G EN : in the inner loop, we train the task model using weighted training loss based on current sample weights; in the outer loop, the noise-robust loss is adopted to guide the learning of the sample weights.
Revised: We propose a novel bi-level re-weighting framework S UN G EN : in the inner loop, we train the task model using weighted training loss based on current sample weights; in the outer loop, the noise-robust loss is adopted to guide the learning of the sample weights.

Original: The two procedures are performed alternatively to generate a set of weights indicating the importance of samples.
Revised: The two procedures are performed alternatively to generate a set of weights indicating the importance of samples.

Original: Notably, our method focuses on enhancing the quality of generated data, while improving the generator (e.g., modify PLM parameter, prompt engineering) is an orthogonal direction and can be applied jointly with our method.
Revised: Notably, our method focuses on enhancing the quality of generated data, while improving the generator (e.g., modify PLM parameter, prompt engineering) is an orthogonal direction and can be applied jointly with our method.

Original: Our main contributions are threefold.
Revised: Our main contributions are threefold.

Original: First, we propose a novel end-to-end framework to construct highquality noise-free synthetic dataset, without the aid of any human annotation (§3).
Revised: First, we propose a novel end-to-end framework to construct high-quality noise-free synthetic dataset, without the aid of any human annotation (§3).

Original: Second, we offer theoretical justification (§4) and empirical verification (§5.2) for S UN G EN ’s ability of recovering a noise-free dataset reliably with synthetic data only.
Revised: Second, we offer theoretical justification (§4) and empirical verification (§5.2) for S UN G EN ’s ability of recovering a noise-free dataset reliably with synthetic data only.

Original: Third, we conduct experiments on eight text classification datasets and show our method outperforms the current baseline by large margins (§5.2).
Revised: Third, we conduct experiments on eight text classification datasets and show our method outperforms the current baseline by large margins (§5.2).

Original:  B ACKGROUND 2. P ROMPT - BASED Z ERO -S HOT L EARNING
Revised:  B ACKGROUND 2. P ROMPT - BASED Z ERO -S HOT L EARNING

Original: We first introduce prompt-based zero-shot prediction (named P ROMPTING ).
Revised: We first introduce prompt-based zero-shot prediction (named P ROMPTING ).

Original: Given a manually-designed prompt T ( · ) and a query example x i ∈ X , P ROMPTING constructs a sentence T ( x i )
Revised: Given a manuallydesigned prompt T ( · ) and a query example x i ∈ X , P ROMPTING constructs a sentence T ( x i )

Original: (e.g., “The movie review in < y i > sentiment is: < x i > ” ).
Revised: (e.g., “The movie review in < y i > sentiment is: < x i > ” ).

Original: The PLM P is expected to model the probability distribution of a set of label words
Revised: The PLM P is expected to model the probability distribution of a set of label words

Original: y i ∈ Y (e.g., “positive”, “negative”) and select the class with highest probability for x i .
Revised: y i ∈ Y (e.g., “positive”, “negative”) and select the class with highest probability for x i .

Original: Even though P ROMPTING has achieved remarkable success in zero-shot learning (Radford et al., 2019; Brown et al., 2020), it is difficult for P ROMPTING to fully leverage the task-specific knowledge from PLMs.
Revised: Even though P ROMPTING has achieved remarkable success in zero-shot learning (Radford et al., 2019; Brown et al., 2020), it is difficult for P ROMPTING to fully leverage the task-specific knowledge from PLMs.

Original: Besides, P ROMPTING still needs to conduct inference on a cumbersome PLM.
Revised: Besides, P ROMPTING still needs to conduct inference on a cumbersome PLM.

Original: 2. E FFICIENT Z ERO -S HOT L EARNING VIA D ATA G ENERATION
Revised: 2. E FFICIENT Z ERO -S HOT L EARNING VIA D ATA G ENERATION

Original: A new line of research (Ye et al., 2022; Meng et al., 2022) endeavors to make zero-shot learning more practical and efficient, among which the generative efficient zero-shot learning paradigm proposed by Z ERO G EN (Ye et al., 2022) is as follows:
Revised: A new line of research (Ye et al., 2022; Meng et al., 2022) endeavors to make zero-shot learning more practical and efficient, among which the generative efficient zero-shot learning paradigm proposed by Z ERO G EN (Ye et al., 2022) is as follows:

Original: Synthetic Data Generation.
Revised: Synthetic Data Generation.

Original: Given a task, the paradigm first generates a synthetic dataset S syn =
Revised: Given a task, the paradigm first generates a synthetic dataset S syn =

Original: ( X syn , Y syn ) with the help of large-scale PLM P and task-related prompts.
Revised: ( X syn , Y syn ) with the help of large-scale PLM P and task-related prompts.

Original: The idea is to use model P to generate the input x syn based on a pseudo label y syn .
Revised: The idea is to use model P to generate the input x syn based on a pseudo label y syn .

Original: For example, in text classification task, a class label y syn is uniformly sampled: y syn ∼ U ( y 1 , y 2 , . . .
Revised: For example, in text classification task, a class label y syn is uniformly sampled: y syn ∼ U ( y 1 , y 2 , . . .

Original: , y K ) , where K is the number of classes.
Revised: , y K ) , where K is the number of classes.

Original: The pseudo label y syn is transformed into a label-descriptive prompt T ( y syn ) to generate x syn : x syn ∼ P ( ·|T ( y syn )) .
Revised: The pseudo label y syn is transformed into a label-descriptive prompt T ( y syn ) to generate x syn : x syn ∼ P ( ·|T ( y syn )) .

Original: The generated x syn and pseudo label y syn are paired to construct a pseudo training dataset S syn .
Revised: The generated x syn and pseudo label y syn are paired to construct a pseudo training dataset S syn .

Original: Efficient Training and Inference.
Revised: Efficient Training and Inference.

Original: To achieve efficiently training and inference, the paradigm then trains a t iny t a sk m odel (TAM) (e.g., 1-layer Bi-LSTM) on the synthetic dataset S syn .
Revised: To achieve efficiently training and inference, the paradigm then trains a t iny t a sk m odel (TAM) (e.g., 1-layer Bi-LSTM) on the synthetic dataset S syn .

Original: TAMs typically have much fewer parameters, which is thus easy to train and more efficient during inference.
Revised: TAMs typically have much fewer parameters, which is thus easy to train and more efficient during inference.

Original: Although Z ERO G EN achieved promising results by training a TAM with S syn , we find that model’s performance rapidly declines after several training epochs, indicating overfitting to noisy samples (Figure 1).
Revised: Although Z ERO G EN achieved promising results by training a TAM with S syn , we find that model’s performance rapidly declines after several training epochs, indicating overfitting to noisy samples (Figure 1).

Original: A classic approach to improve the quality of a dataset containing low-quality data is to re-weight the samples using some function w .
Revised: A classic approach to improve the quality of a dataset containing low-quality data is to re-weight the samples using some function w .

Original: Intuitively, if we assign large weights to correct samples and small weights to noisy ones, the negative influence of noisy samples during training can be reduced.
Revised: Intuitively, if we assign large weights to correct samples and small weights to noisy ones, the negative influence of noisy samples during training can be reduced.

Original: However, the heuristic design of w (e.g., according to output confidence, loss value) suffers unstable performances and requires task-specific knowledge (Shu et al., 2019).
Revised: However, the heuristic design of w (e.g., according to output confidence, loss value) suffers unstable performances and requires task-specific knowledge (Shu et al., 2019).

Original: Therefore, our paper seeks to optimize w automatically.
Revised: Therefore, our paper seeks to optimize w automatically.

Original:  M ETHOD
Revised:  M ETHOD

Original: To automatically learn sample weights, the bi-level optimization approaches (Ren et al., 2018; Shu et al., 2019) has been proven to be effective.
Revised: To automatically learn sample weights, the bi-level optimization approaches (Ren et al., 2018; Shu et al., 2019) have been proven to be effective.

Original: However, such methods are impractical in our zero-shot scenario, since they depend on a clean validation set to guide the optimization of sample weights.
Revised: However, such methods are impractical in our zero-shot scenario, since they depend on a clean validation set to guide the optimization of sample weights.

Original: To circumvent the absence of human-labeled data, we propose a S elf-g U ided N oise-free data GEN eration framework, named S UN G EN (Figure 2).
Revised: To circumvent the absence of human-labeled data, we propose a S elf-g U ided N oise-free data GEN eration framework, named S UN G EN (Figure 2).

Original: Concretely, we propose a sample reweighting framework via bilevel optimization using a noise-robust loss (Ghosh et al., 2017; Wang et al., 2019; Zhang & Sabuncu, 2018) as the outer objective.
Revised: Concretely, we propose a sample reweighting framework via bilevel optimization using a noise-robust loss (Ghosh et al., 2017; Wang et al., 2019; Zhang & Sabuncu, 2018) as the outer objective.

Original: Due to the appealing property of such loss functions, our method is able to learn meaningful sample weights with only a synthetic (noisy) validation dataset.
Revised: Due to the appealing property of such loss functions, our method is able to learn meaningful sample weights with only a synthetic (noisy) validation dataset.

Original: Notations.
Revised: Notations.

Original: As elaborated in Section 2.2, we first generate a synthetic dataset using a left-to-right PLM and task-related prompts for the given task.
Revised: As elaborated in Section 2.2, we first generate a synthetic dataset using a left-to-right PLM and task-related prompts for the given task.

Original: Let S syn , S clean ⊂ X

Original: × Y = R d × { 1 , ..., K } denote distribution of synthetic (noisy) and clean (gold) data, respective.
Revised: Let S syn , S clean ⊂ X ×Y = R d ×{ 1 , ..., K } denote distribution of synthetic (noisy) and clean (gold) data, respective.

Original: Here d is the dimension of the input space and K is the number of classes.
Revised: Here d is the dimension of the input space and K is the number of classes.

Original: We draw a training dataset S tsyn :
Revised: We draw a training dataset S tsyn :

Original: = { ( x , y ) ( i ) } Ni =1 and a validation dataset 
Revised: = { ( x , y ) ( i ) } Ni =1 and a validation dataset S vsyn :

Original: S vsyn := { ( x , y ) ( j ) } Mj =1 from S syn .
Revised: = { ( x , y ) ( j ) } Mj =1 from S syn .

Original: Denote f ( x , θ ) as the classifier (TAM) with θ as its parameters.
Revised: Denote f ( x , θ ) as the classifier (TAM) with θ as its parameters.

Original: w ∈ W := { w ( · , · ) :
Revised: w ∈ W := { w ( · , · ) :

Original: X × Y −→ [0 , 1] } is a re-weighting function that assign a weight to each sample.
Revised: X × Y −→ [0 , 1] } is a re-weighting function that assign a weight to each sample.

Original: The bold w is a sample weight vector of length N indicating training samples’ importance, which contains per-sample weight w i :=
Revised: The bold w is a sample weight vector of length N indicating training samples’ importance, which contains per-sample weight w i :=

Original: w ( x i , y i ) .
Revised: w ( x i , y i ) .

Original: Overall Framework.
Revised: Overall Framework.

Original: Without the presence of clean validation set, our proposed bi-level optimization framework S UN G EN (as shown in Figure 2) can be outlined as follows:
Revised: Without the presence of clean validation set, our proposed bi-level optimization framework S UN G EN (as shown in Figure 2) can be outlined as follows:

Original: [Equation]
Revised: [Equation]

Original: where w ∗ denotes the optimal sample weights, which is obtained from the outer loop (Eqn. 1); ˆ θ ( w ) denotes the classifier’s parameters after weighted training with w , which is obtained from the inner loop (Eqn. 2); ℓ robust denotes the noise-robust loss (to be illustrated below) calculated on validation set; ℓ ce denotes the cross-entropy(CE) loss calculated on training set.
Revised: where w ∗ denotes the optimal sample weights, which is obtained from the outer loop (Eqn. 1); ˆ θ ( w ) denotes the classifier’s parameters after weighted training with w , which is obtained from the inner loop (Eqn. 2); ℓ robust denotes the noise-robust loss (to be illustrated below) calculated on validation set; ℓ ce denotes the cross-entropy(CE) loss calculated on training set.

Original: The whole process is: (1) in the inner loop (Eqn. 2), we fix w and optimize the classifier f ( x , θ ) using the weighted loss L ce over a synthetic training set S tsyn , and derive the trained classifier ˆ θ ( w ) ; (2) in the outer loop (Eqn. 1), we calculate the noise-robust loss L robust on the synthetic validation set S vsyn at the optimized ˆ θ ( w ) .
Revised: The whole process is: (1) in the inner loop (Eqn. 2), we fix w and optimize the classifier f ( x , θ ) using the weighted loss L ce over a synthetic training set S tsyn , and derive the trained classifier ˆ θ ( w ) ; (2) in the outer loop (Eqn. 1), we calculate the noise-robust loss L robust on the synthetic validation set S vsyn at the optimized ˆ θ ( w ) .

Original: The outer loss is minimized to guide the optimization of w , and the outer gradient ∇ w L robust is calculated via truncated back-propagation (Appendix A).
Revised: The outer loss is minimized to guide the optimization of w , and the outer gradient ∇ w L robust is calculated via truncated back-propagation (Appendix H).

Original: The two procedures are performed alternatively for T iterations.
Revised: The two procedures are performed alternatively for T iterations.

Original: Notably, our framework prevents the need for a clean validation set, which is the main obstacle for the previous bi-level re-weighting approaches under zero-shot scenario (Ren et al., 2018; Shu et al., 2019).
Revised: Notably, our framework prevents the need for a clean validation set, which is the main obstacle for the previous bi-level re-weighting approaches under zero-shot scenario (Ren et al., 2018; Shu et al., 2019).

Original: The magic of this appealing feature lies in the choice of the outer objective L robust in Eqn.
Revised: The magic of this appealing feature lies in the choice of the outer objective L robust in Eqn.

Original: Recall that the goal of sample reweighting is to find w ∗ , with which the model f
Revised: Recall that the goal of sample reweighting is to find w ∗ , with which the model f

Original: ( x ; θ ) can be trained using weighted loss over the synthetic training set, such that the model performs well on the clean data (from the same distribution as test data).
Revised: ( x ; θ ) can be trained using weighted loss over the synthetic training set, such that the model performs well on the clean data (from the same distribution as test data).

Original: In Sec. 4,  we theoretically show that using L robust as the outer objective, our method can find a set of sample weights w ∗ with just the synthetic validation set, such that w ∗ maximizes the model performance on the clean data.
Revised: In Sec. 4, under the condition that the majority of data are correctly labelled (Ghosh et al., 2017; Wang et al., 2019), we theoretically show that using L robust as the outer objective, our method can find a set of sample weights w ∗ with just the synthetic validation set, such that w ∗ maximizes the model performance on the clean data.

Original: The whole procedure is illustrated in Algorithm 1.
Revised: The whole procedure is illustrated in Algorithm 1.

Original: In the following, we elaborate more on the design of ℓ robust in our framework.
Revised: In the following, we elaborate more on the design of ℓ robust in our framework.

Original: Algorithm Require: a TAM with parameters θ , synthetic training dataset S tsyn and validation dataset S vsyn , outer step size λ , outer iterations T.
Revised: Require: a TAM with parameters θ , synthetic training dataset S tsyn and validation dataset S vsyn , outer step size λ , outer iterations T.

Original: 1: Initialize sample weightingsw with each w i to be 0.5.
Revised: : Initialize sample weightings with each w i to be 0.5.

Original: 2: for training iteration t =1 , 2 . . .
Revised: : for training iteration t =

Original: T do 3:
Revised: T do 3:

Original: Conduct weighted training ( L ce ) on TAM using w t , S tsyn and obtain ˆ θ ( w t ) 4:
Revised: Conduct weighted training ( L ce ) on TAM using w t , S tsyn and obtain ˆ θ ( w t ) 4:

Original: Evaluate ˆ θ ( w t ) on S vsyn , then obtain meta gradient ∇ w L robust via Eqn.
Revised: Evaluate ˆ θ ( w t ) on S vsyn , then obtain meta gradient ∇ w L robust via Eqn.

Original: Update w using ∇ w L robust by gradient descent w t +1 ← w t − λ ∇ w L robust 6: end for
Revised: Update w using ∇ w L robust by gradient descent w t +1 ← w t − λ ∇ w L robust 6: end for

Original: Output: Optimized sample weights w ∗ .
Revised: Output: Optimized sample weights w ∗ .

Original: Noise-robust Loss Functions.
Revised: Noise-robust Loss Functions.

Original: In the noisy label learning literature, there is a family of loss functions that possess the following property:
Revised: In the noisy label learning literature, there is a family of loss functions that possess the following property:

Original: [Equation]
Revised: [Equation]

Original: where f ( · , θ ) denotes a classifier, x is the input, K is the number of classes, and C is a constant.
Revised: where f ( · , θ ) denotes a classifier, x is the input, K is the number of classes, and C is a constant.

Original: Previous work (Wang et al., 2019; Zhang & Sabuncu, 2018; Ghosh et al., 2017) has shown that these loss functions have consistent global minimum under label noise.
Revised: Previous work (Wang et al., 2019; Zhang & Sabuncu, 2018; Ghosh et al., 2017) has shown that these loss functions have consistent global minimum under label noise.

Original: More formally, under suitable conditions, the global minimizer ( θ ∗ ) of ℓ robust is the same regardless of whether the training data is clean or noisy.
Revised: More formally, when the majority of the training samples are correctly labelled, the global minimizer ( θ ∗ ) of ℓ robust is the same regardless of whether the training data is clean or noisy.

Original: This noise-robust property enables these losses to optimize the sample weights given only a noisy validation dataset.
Revised: This noiserobust property enables these losses to optimize the sample weights given only a noisy validation dataset.

Original: Detailed proof will be given in Sec. 4.
Revised: Detailed proof will be given in Sec. 4.

Original: In particular, we consider the reversed cross-entropy loss ℓ rce with the following form for sample ( x , y ) :
Revised: In particular, we consider the reversed cross-entropy loss ℓ rce with the following form for sample ( x , y ) :

Original: [Equation]
Revised: [Equation]

Original: where f k ( x , θ )
Revised: where f k ( x , θ )

Original: =
Revised: =

Original: e zk (cid:80)
Revised: e zk (cid:80)

Original: Ki =1 e zi denotes the predicted probability for each label k ∈ { 1 , ..., K } , and z i are the logits.
Revised: Ki =1 e zi denotes the predicted probability for each label k ∈ { 1 , ..., K } , and z i are the logits.

Original: We denote the ground-truth distribution over labels by q ( k | x ) , and (cid:80)
Revised: We denote the ground-truth distribution over labels by q ( k | x ) , and (cid:80)

Original: Kk
Revised: Kk

Original: =1 q ( k | x ) = 1 .
Revised: =1 q ( k | x ) = 1 .

Original: Given the ground-truth label is y , then q ( y | x ) = 1 and q ( k | x ) = 0 for all k ̸
Revised: Given the ground-truth label is y , then q ( y | x ) = 1 and q ( k | x ) = 0 for all k ̸

Original: =
Revised: =

Original: y .
Revised: y .

Original: log (0) is approximated to a constant A .
Revised: log (0) is approximated to a constant A .

Original: One can easily check that ℓ rce satisfies Property (3), and
Revised: One can easily check that ℓ rce satisfies Property (3), and

Original: C = − ( K − 1) A in this case.
Revised: C = − ( K − 1) A in this case.

Original: Remark.
Revised: Remark.

Original: Even though ℓ robust is theoretically noise-tolerant, it has been shown that using them to train the network parameters θ leads to difficulty in optimization and hampers the network’s performance (Wang et al., 2019; Zhang & Sabuncu, 2018).
Revised: Even though ℓ robust is theoretically noise-tolerant, it has been shown that using them to train the network parameters θ leads to difficulty in optimization and hampers the network’s performance (Wang et al., 2019; Zhang & Sabuncu, 2018).

Original: Remarkably, adopting ℓ robust as the objective in the outer loop of Eqn.
Revised: Remarkably, adopting ℓ robust as the objective in the outer loop of Eqn.

Original: (1) implicitly overcomes the side effects of ℓ robust .
Revised: (1) implicitly overcomes the side effects of ℓ robust .

Original: Since ℓ robust is now used to optimize the sample weights w , which is in a much smaller search space and has simpler structure than θ , thus is easier to optimize.
Revised: Since ℓ robust is now used to optimize the sample weights w , which is in a much smaller search space and has simpler structure than θ , thus is easier to optimize.

Original: This "decouples" noise removal from network training, and thereby prevents hampering the TAM’s performance.
Revised: This "decouples" noise removal from network training, and thereby prevents hampering the TAM’s performance.

Original: Clean Subset Sampling.
Revised: Clean Subset Sampling.

Original: Our framework enables us to derive a set of continuous weights w , which encodes the data quality and can well separate the noisy samples from clean ones, as shown in Figure 3.
Revised: Our framework enables us to derive a set of continuous weights w , which encodes the data quality and can well separate the noisy samples from clean ones, as shown in Figure 3.

Original: With those weights, we can sample subsets with arbitrary budgets and use them to train the task models with unweighted CE loss.
Revised: With those weights, we can sample subsets with arbitrary budgets and use them to train the task models with unweighted CE loss.

Original: More specifically, suppose the budget is D , we first normalize w i to w ′
Revised: More specifically, suppose the budget is D , we first normalize w i to w ′

Original: i (i.e. (cid:80) ni
Revised: i (i.e. (cid:80) ni

Original: =1 w ′ i = D ), based on which we then sample a Bernoulli random variable
Revised: =1 w ′ i = D ), based on which we then sample a Bernoulli random variable

Original: I i ∼ Ber ( w ′ i ) for each sample to indicate whether they should be included.
Revised: I i ∼ Ber ( w ′ i ) for each sample to indicate whether they should be included.

Original: Denote the size of sampled subset as ˆ D .
Revised: Denote the size of sampled subset as ˆ D .

Original: Because E I ∼ p ( I | w ) ∥ I ∥ 

Revised: Because E I ∼ p ( I | w ) ∥ I ∥

Original: = (cid:80) n i =1 w ′ i
Revised: = (cid:80) ni =1 w ′ i

Original: =
Revised: =

Original: D , it is clear that ˆ DD P −→ 1 when n −→ ∞ , meaning that ˆ D is close to D .
Revised: D , it is clear that ˆ DD P −→when n −→ ∞ , meaning that ˆ D is close to D .

Original: Notably, because the important and noise-free data are associated with larger weights, training on the sampled subset can perform on par with weighted training over the entire dataset, while being much more efficient.
Revised: Notably, because the important and noise-free data are associated with larger weights, training on the sampled subset can perform on par with weighted training over the entire dataset, while being much more efficient.

Original:  T HEORETICAL A NALYSIS
Revised:  T HEORETICAL A NALYSIS

Original: Though we have no clean data as validation set, our method still enjoys favorable theoretical properties.
Revised: Though we have no clean data as validation set, our method still enjoys favorable theoretical properties.

Original: Recall that S clean and S syn are as the clean and synthetic (noisy) distribution, respectively.
Revised: Recall that S clean and S syn are as the clean and synthetic (noisy) distribution, respectively.

Original: We further denote L ( θ , S ) =
Revised: We further denote L ( θ , S ) =

Original: E ( x ,y ) ∼S [ ℓ ( f ( x ; θ ) , y )] given a data distribution S .
Revised: E ( x ,y ) ∼S [ ℓ ( f ( x ; θ ) , y )] given a data distribution S .

Original: Let the optimal network parameters obtained with CE loss over clean dataset be θ ∗ :=
Revised: Let the optimal network parameters obtained with CE loss over clean dataset be θ ∗ :=

Original: arg min θ L ce ( θ , S clean ) .
Revised: arg min θ L ce ( θ , S clean ) .

Original: We assume the θ ∗ is unique, or we focus on the θ ∗ with minimum norm.
Revised: We assume the θ ∗ is unique, or we focus on the θ ∗ with minimum norm.

Original: Property 1.
Revised: Property 1.

Original: We call a loss function ℓ a robust loss, if  its minimizer on the noisy dataset collides with that one  the clean dataset, i.e.,
Revised: We call a loss function ℓ a robust loss, if under mild conditions as in (Ghosh et al., 2017), its minimizer on the noisy dataset collides with the one on the clean dataset, i.e.,

Original: [Equation]
Revised: [Equation]

Original: Previous work on noise-robust loss functions (Ghosh et al., 2017; Wang et al., 2019; Zhang &Sabuncu, 2018;
Revised: Previous work on noise-robust loss functions (Ghosh et al., 2017; Wang et al., 2019; Zhang &

Original:  Xu et al., 2019) have shown that the losses satisfying Eqn.
Revised: Sabuncu, 2018; Xu et al., 2019) have shown that the losses satisfying Eqn.

Original: (3) have the above noise-tolerant property.
Revised: (3) have the above noise-tolerant property.

Original: We give the  complete proof in the Appendix.
Revised: We give the detailed assumptions and complete proof in Appendix C.2.

Original: Assumption 1.
Revised: Assumption 1.

Original: Let P clean and P syn denote the probability density function of S clean and S syn , respectively.
Revised: Let P clean and P syn denote the probability density function of S clean and S syn , respectively.

Original: There exists a weighting function w ∗ such that P clean ( x , y ) =
Revised: There exists a weighting function w ∗ such that P clean ( x , y ) =

Original: w ∗ ( x , y )
Revised: w ∗ ( x , y )

Original: P syn ( x , y ) .
Revised: P syn ( x , y ) .

Original: Assumption 1 is reasonable because synthetic data generated by the PLM has a wide coverage.
Revised: Assumption 1 is reasonable because synthetic data generated by the PLM has a wide coverage.

Original: Therefore, with a proper weight function w , we may recover the clean distributional by reweighting the synthetic data.
Revised: Therefore, with a proper weight function w , we may recover the clean distribution by reweighting the synthetic data.

Original: Assumption 2.
Revised: Assumption 2.

Original: The optimal θ ∗ uniquely minimizes L robust ( θ , S clean ) , i.e., L robust ( θ ∗ , S clean ) <
Revised: The optimal θ ∗ uniquely minimizes L robust ( θ , S clean ) , i.e., L robust ( θ ∗ , S clean ) <

Original: L robust ( θ , S clean ) for all θ ̸
Revised: L robust ( θ , S clean ) for all θ ̸

Original: = θ ∗ .
Revised: = θ ∗ .

Original: Assumption 2 is natural since the robust losses are originally designed to train the model.
Revised: Assumption 2 is natural since the robust losses are originally designed to train the model.

Original: Thus minimizing L robust ( θ , S clean ) is expected to achieve promising performance, as justified by Ghosh et al.
Revised: Thus minimizing L robust ( θ , S clean ) is expected to achieve promising performance, as justified by Ghosh et al.

Original:  (though the optimization difficulty mentioned in the Remark of Section 3 poses challenges for model training).
Revised: (2017) (though the optimization difficulty mentioned in the Remark of Section 3 poses challenges for model training).

Original: We also experimentally show that when training a model with L ce ( θ , S t clean ) , L robust ( θ , S t clean ) also decreases, and reaches the plateau at a similar point (Appendix D.1).
Revised: We also experimentally show that when training a model with L ce ( θ , S t clean ) , L robust ( θ , S t clean ) also decreases, and reaches the plateau at a similar point (Appendix C.1).

Original: This indicates that L ce ( θ , S clean ) and L robust ( θ , S clean ) have close optimal solutions.
Revised: This indicates that L ce ( θ , S clean ) and L robust ( θ , S clean ) have close optimal solutions.

Original: Note that even if θ and θ ∗ differ by a small quantity, i.e., L robust ( θ ∗ , S clean ) <
Revised: Note that even if θ and θ ∗ differ by a small quantity, i.e., L robust ( θ ∗ , S clean ) <

Original: L robust ( θ , S clean ) + ε holds with a small ε , the following proof holds with just minor modifications.
Revised: L robust ( θ , S clean ) + ε holds with a small ε , the following proof holds with just minor modifications.

Original: Theorem 1.
Revised: Theorem 1.

Original: If Assumption 1 holds, there exists a w ∗ such that ˆ θ ( w ∗ )
Revised: If Assumption 1 holds, there exists a w ∗ such that ˆ θ ( w ∗ )

Original: = θ ∗ .
Revised: = θ ∗ .

Original: Further with Assumption 2 and Property D.2, our method can uniquely return w ∗ and the resulting θ ∗ with only synthetic (noisy) data S noisy .
Revised: Further with Assumption 2 and Property 1, our method can uniquely return w ∗ and the resulting θ ∗ with only synthetic (noisy) data S noisy .

Original: Given Assumption 2 holds, L ce ( θ , S clean ) and L robust ( θ , S clean ) have consistent optimal solution θ ∗ .
Revised: Given Assumption 2 holds, L ce ( θ , S clean ) and L robust ( θ , S clean ) have consistent optimal solution θ ∗ .

Original: Furthermore, Property D.2 of L robust indicates that θ ∗ can be found with just the noisy synthetic data S syn .
Revised: Furthermore, Property 1 of L robust indicates that θ ∗ can be found with just the noisy synthetic data S syn .

Original: Therefore, we can optimize L robust ( θ , S syn ) to find θ ∗ .
Revised: Therefore, we can optimize L robust ( θ , S syn ) to find θ ∗ .

Original: Finally, since θ ∗ can be parameterized by w ∗ (Theorem 1), which is achieved by the inner loop (Eqn.
Revised: Finally, since θ ∗ can be parameterized by w ∗ (Theorem 1), which is achieved by the inner loop (Eqn.

Original: (2)), we can optimize L robust (ˆ θ ( w ) , S syn ) over w in the outer loop (Eqn.
Revised: (2)), we can optimize L robust (ˆ θ ( w ) , S syn ) over w in the outer loop (Eqn.

Original: (1)) to find the optimal corresponding w ∗ as well as the θ ∗ .
Revised: (1)) to find the optimal corresponding w ∗ as well as the θ ∗ .

Original: See Appendix D.3 for detailed proof.
Revised: See Appendix C.3 for detailed proof.

Original: The above analysis theoretically shows our S UN G EN is able to learn the optimal sample weights w ∗ and the resulting optimal model parameters θ ∗ with just synthetic data.
Revised: The above analysis theoretically shows our S UN G EN is able to learn the optimal sample weights w ∗ and the resulting optimal model parameters θ ∗ with just synthetic data.

Original: Notably, Theorem 1 is based on population losses with infinite samples.
Revised: Notably, Theorem 1 is based on population losses with infinite samples.

Original: We further characterize the generalization bound when there is finite samples:
Revised: We further characterize the generalization bound when there is finite samples:

Original: Theorem 2 (Finite Sample Generalization) .
Revised: Theorem 2 (Finite Sample Generalization) .

Original: Suppose we have access to synthetic datasets S v syn and S t syn both with finite samples N .
Revised: Suppose we have access to synthetic datasets S v syn and S t syn both with finite samples N .

Original: Let ˆ θ ∗ ( w ) be the deterministic mapping from w to θ defined in the inner of Eqn. given ˆ S t syn .
Revised: Let ˆ θ ∗ ( w ) be the deterministic mapping from w to θ defined in the inner of Eqn.2 given ˆ S t syn .

Original: Assuming the output of loss function ℓ robust is upper bounded by M , the carnality of W is |W| , the outer loop of Eqn.1 is solved within ϵ -approximately, and the solution ˆ w satisfies the following condition:
Revised: Assuming the output of loss function ℓ robust is upper bounded by M , the carnality of W is |W| , the outer loop of Eqn.1 is solved within ϵ -approximately, and the solution ˆ w satisfies the following condition:

Original: [Equation]
Revised: [Equation]

Original: we then have with probability at least 1 − δ ,
Revised: we then have with probability at least 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: Refer to Appendix D.4 for full proof.
Revised: Refer to Appendix C.4 for full proof.

Original: Theorem 2 characterizes the generalization ability of S UN G EN .
Revised: Theorem 2 characterizes the generalization ability of S UN G EN .

Original: If we obtain an approximate solution of the bilevel problem Eqn.(1)-(2) given finite samples, Eqn. (4) shows that such solution has a test performance close to the oracle model.[Table]
Revised: If we obtain an approximate solution of the bilevel problem Eqn.(1)-(2) given finite samples, Eqn. (4) shows that such solution has a test performance close to the oracle model.

Original:  E XPERIMENTS 5. S ETUP
Revised:  E XPERIMENTS 5. S ETUP

Original: Datasets & Baselines.
Revised: Datasets & Baselines.

Original: We evaluate S UN G EN on eight text classification tasks, including IMDb (Maas et al., 2011), SST-

Revised: We evaluate S UN G EN on eight text classification tasks, including IMDb (Maas et al., 2011), SST-

Original: (Socher et al., 2013), Rotten Tomatoes (Pang & Lee, 2005), Amazon (McAuley & Leskovec, 2013), Yelp (Zhang et al., 2015), Subj (Pang & Lee, 2004), AGNews (Zhang et al., 2015) and DBpedia (Zhang et al., 2015).
Revised: (Socher et al., 2013), Rotten Tomatoes (Pang & Lee, 2005), Amazon (McAuley & Leskovec, 2013), Yelp (Zhang et al., 2015), Subj (Pang & Lee, 2004), AGNews (Zhang et al., 2015) and DBpedia (Zhang et al., 2015).

Original: These tasks have various number of classes, ranging from 2 to 14.
Revised: These tasks have various number of classes, ranging fromto 14.

Original: Other details about datasets are in Appendix F.
Revised: Other details about datasets are in Appendix F.

Original: We compare our proposed method with the following baselines: (1) P ROMPTING .
Revised: We compare our proposed method with the following baselines: (1) P ROMPTING .

Original: The prompt-based zero-shot classification method based on PLMs (Brown et al., 2020; Gao et al., 2021).
Revised: The prompt-based zero-shot classification method based on PLMs (Brown et al., 2020; Gao et al., 2021).

Original:  Z ERO G EN .
Revised: (2) Z ERO G EN .

Original: A recent zero-shot learning work via dataset generation (Ye et al., 2022).
Revised: A recent zero-shot learning work via dataset generation (Ye et al., 2022).

Original: Implementation Details.
Revised: Implementation Details.

Original: We compare the baselines using GPT2-XL (Radford et al., 2019) as PLM.
Revised: We compare the baselines using GPT2-XL (Radford et al., 2019) as PLM.

Original: For text generation, we use Nucleus Sampling (Holtzman et al., 2020) with p = 0 .
Revised: For text generation, we use Nucleus Sampling (Holtzman et al., 2020) with p = 0 .

Original: 9 as the decoding strategy and use GPT2-XL as the generator.
Revised: 9 as the decoding strategy and use GPT2-XL as the generator.

Original: To make a fair comparison, we use the best prompts designed by (Ye et al., 2022) in both P ROMPTING and data-generation settings (Appendix Table 8 ).
Revised: To make a fair comparison, we use the best prompts designed by (Ye et al., 2022) in both P ROMPTING and data-generation settings (Appendix Table 10 ).

Original: For task model training, we use 1-layer Bi-LSTM and DistilBERT-base as the lightweight classifiers.
Revised: For task model training, we use 1-layer Bi-LSTM and DistilBERT-base as the lightweight classifiers.

Original: The bilevel procedure is iteratedtimes for each task.
Revised: The bilevel procedure is iterated 50 times for each task.

Original: For more details(e.g., full prompts, training details), please refer to Appendix F. 
Revised: For more details(e.g., full prompts, training details), please refer to Appendix F. [Table] [Table] [Table]

Revised: [Table]

Original: E XPERIMENTAL R ESULTS
Revised: E XPERIMENTAL R ESULTS

Original: Main Experiments.
Revised: Main Experiments.

Original: We present our main experiment results in Table 1.
Revised: We present our main experiment results in Table 1.

Original: We observe that our S UN G EN achieves considerable performance gain over the Z ERO G EN baseline across all the tasks.
Revised: We observe that our S UN G EN achieves considerable performance gain over the Z ERO G EN baseline across all the tasks.

Original: Interestingly, the improvement is more prominent for LSTM compared with DistilBERT, which shows relative improvement over Z ERO G EN by 9.8% on average across all tasks.
Revised: Interestingly, the improvement is more prominent for LSTM compared with DistilBERT, which shows relative improvement over Z ERO G EN by 9.8% on average across all tasks.

Original: We conjecture the reason is that the pre-trained models such as DistilBERT are inherently more robust to noisy training data, which is also pointed out in (Hendrycks et al., 2019).
Revised: We conjecture the reason is that the pre-trained models such as DistilBERT are inherently more robust to noisy training data, which is also pointed out in (Hendrycks et al., 2019).

Original: Surprisingly, on Rotten Tomatoes and Yelp, S UN G EN -LSTM even outperforms Z ERO G EN -DistilBERT, which requires no pre-training and has much fewer parameters.
Revised: Surprisingly, on Rotten Tomatoes and Yelp, S UN G EN -LSTM even outperforms Z ERO G EN -DistilBERT, which requires no pre-training and has much fewer parameters.

Original: Effectiveness of S UN G EN on Constructing Noise-Free Data.
Revised: Effectiveness of S UN G EN on Constructing Noise-Free Data.

Original: Figure 1 shows that while ZeroGen suffers from overfitting on noisy data, the problem disappears when training with the subset selected by S UN G EN .
Revised: Figure 1 shows that while ZeroGen suffers from overfitting on noisy data, the problem disappears when training with the subset selected by S UN G EN .

Original: With longer training epochs, the data of S UN G EN consistently helps improve the performance of TAM on test set and significantly surpasses the result of ZeroGen, which proves that S UN G EN can effectively construct noise-free high-quality data.
Revised: With longer training epochs, the data of S UN G EN consistently helps improve the performance of TAM on test set and significantly surpasses the result of ZeroGen, which proves that S UN G EN can effectively construct noise-free high-quality data.

Original: Synthetic Data vs. Gold Data Supervision.
Revised: Synthetic Data vs. Gold Data Supervision.

Original: Note that a key advantage of our proposed framework is that we do not require gold data to optimize the sample weights.
Revised: Note that a key advantage of our proposed framework is that we do not require gold data to optimize the sample weights.

Original: Here, we compare our S UN G EN to the bilevel reweighting method which uses gold data for calculating the outer objective in Table 2.
Revised: Here, we compare our S UN G EN to the bilevel reweighting method which uses gold data for calculating the outer objective in Table 2.

Original: Specifically, our S UN G EN calculates the robust outer objective ℓ rce on the noisy synthetic data, while the counterpart calculates the normal outer objective ℓ ce on gold data.
Revised: Specifically, our S UN G EN calculates the robust outer objective ℓ rce on the noisy synthetic data, while the counterpart calculates the normal outer objective ℓ ce on gold data.

Original: Table 2 shows that our method achieves similar performances as

Original: [Table]

Original: [Table] [Table] [Table] the counterpart, which verifies that our method using the noise validation set can equivalently supervise the optimization of sample weights as the clean validation set does.
Revised: Table 2 shows that our method achieves similar performances as the counterpart, which verifies that our method using the noise validation set can equivalently supervise the optimization of sample weights as the clean validation set does.

Original: S UN G EN vs. Popular Denoise Baselines.
Revised: S UN G EN vs. Popular Denoise Baselines.

Original: We compare with other de-noise methods, which are (1) directly removing data with low predicted confidence (Swayamdipta et al., 2020), (2) Co-Teaching (Han et al., 2018), which distinguishes noisy data by large loss value, and (3) Meta-Weight-Net (Shu et al., 2019), which relies on meta samples and a loss-based proxy model to learn the sample weights.
Revised: We compare with other de-noise methods, which are (1) directly removing data with low predicted confidence (Swayamdipta et al., 2020), (2) CoTeaching (Han et al., 2018), which distinguishes noisy data by large loss value, and (3) Meta-WeightNet (Shu et al., 2019), which relies on meta samples and a loss-based proxy model to learn the sample weights.

Original: Since in zero-shot setting we have no access to gold data, we use the synthetic data as validation set for Co-Teaching and Meta-Weight-Net.
Revised: Since in zero-shot setting we have no access to gold data, we use the synthetic data as validation set for Co-Teaching and Meta-Weight-Net.

Original: Results in Table 3 prove our method’s superiority in zero-shot setting.
Revised: Results in Table 3 prove our method’s superiority in zero-shot setting.

Original: We further analyze why the loss-value-based methods are not able to distinguish noisy data in our situation in Appendix G [Table]
Revised: We further analyze why the loss-value-based methods are not able to distinguish noisy data in our situation in Appendix K.

Original: 5. A BLATION S TUDY AND A NALYSIS
Revised: 5. A BLATION S TUDY AND A NALYSIS

Original: Performance of Different Data Sizes.
Revised: Performance of Different Data Sizes.

Original: Table 4 shows that even with much less data, model trained by S UN G EN achieves better performance than Z ERO G EN . 
Revised: Table 4 shows that even with much less data, model trained by S UN G EN achieves better performance than Z ERO G EN . For example, in IMDb, by removing [Table]

Original: For example, in IMDb, by removing low-quality data, subset with 2% data achieves better performance than the original data (20k vs. 1,000k), which shows superior data quality of S UN G EN .
Revised: [Table] low-quality data, subset with 2% data achieves better performance than the original data (20k vs. 1,000k), which shows superior data quality of S UN G EN .

Original: Besides, from Figure 3(d), we find the percentage of erroneous data is small.
Revised: Besides, from Figure 3(d), we find the percentage of erroneous data is small.

Original: However, those erroneous data significantly degrades the model performance: without weighted training to remove the effect of noisy data, the IMDB accuracy of full set is decreased from 86.56 to 78.29.
Revised: However, those erroneous data significantly degrades the model performance: without weighted training to remove the effect of noisy data, the IMDB accuracy of full set is decreased from 86.56 to 78.29.

Original: Analysis of Data Diversity and Correctness.
Revised: Analysis of Data Diversity and Correctness.

Original: Table 5 shows that our selected noise-free dataset is more diverse than data generated by Z ERO G EN .
Revised: Table 5 shows that our selected noise-free dataset is more diverse than data generated by Z ERO G EN .

Original: One interesting thing to note is that in Amazon and Yelp, average correctness of S UN G EN is slightly lower than Z ERO G EN .
Revised: One interesting thing to note is that in Amazon and Yelp, average correctness of S UN G EN is slightly lower than Z ERO G EN .

Original: In addition, the top samples (with highest weights) have slightly lower correctness than average.
Revised: In addition, the top samples (with highest weights) have slightly lower correctness than average.

Original: This is expected as the data with highest correctness may be redundant or too simple, while the challenging and informative samples are often harder to classify and thus have lower correctness.
Revised: This is expected as the data with highest correctness may be redundant or too simple, while the challenging and informative samples are often harder to classify and thus have lower correctness.

Original: The results further verify that S UN G EN effectively separates the informative samples from the ones that are redundant or erroneous.
Revised: The results further verify that S UN G EN effectively separates the informative samples from the ones that are redundant or erroneous.

Original: This can not be done with the heuristic methods that manually sets a threshold to separate clean and noisy data, which may keep redundant samples and remove hard ones.
Revised: This can not be done with the heuristic methods that manually sets a threshold to separate clean and noisy data, which may keep redundant samples and remove hard ones.

Original: [Figure]

Original: Analysis of Removed and Selected Examples.
Revised: Analysis of Removed and Selected Examples.

Original: We take IMDb (sentiment classification task of movie review) as the example task and find that the majority of samples associated with small weights are wrongly labeled, as shown inAppendix Table 6.
Revised: We take IMDb (sentiment classification task of movie review) as the example task and find that the majority of samples associated with small weights are wrongly labeled, as shown in Table 6.

Original: Besides, there is a small part of erroneous data which contains task-irrelevant text, meaning the generated text is not a movie review and has no obvious emotional tendency.
Revised: Besides, there is a small part of erroneous data which contains task-irrelevant text, meaning the generated text is not a movie review and has no obvious emotional tendency.

Original: For samples with large weights, we actually find them to be well-written transitional complex sentences (bottom part ofAppendix Table 6), which verifies that S UN G EN tends to select correct and important samples.
Revised: For samples with large weights, we actually find them to be well-written transitional complex sentences (bottom part of Table 6), which verifies that S UN G EN tends to select correct and important samples.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Zero-shot Learning via PLM.
Revised: Zero-shot Learning via PLM.

Original: The popular prompt-based zero-shot prediction is proposed by GPT (Radford et al., 2019).
Revised: The popular prompt-based zero-shot prediction is proposed by GPT (Radford et al., 2019).

Original: With well-designed prompts, large-scale PLMs have shown its notable zero-shot learning ability in various tasks (Jiang et al., 2020; Shin et al., 2020; Reynolds & McDonell, 2021).
Revised: With well-designed prompts, large-scale PLMs have shown its notable zero-shot learning ability in various tasks (Jiang et al., 2020; Shin et al., 2020; Reynolds & McDonell, 0. [Figure] [Figure] [Figure]

Original: More recently, data-generation-based work via PLM has gained popularity and shown superior ability on synthesizing task-specific data (Anaby-Tavor et al., 2020;
Revised: [Figure] [Table] The [Table]

Revised: This long, pompous

Revised: [Table]

Revised: chain-smoking movie makes a big

Revised: [Table]

Revised: out of not very much at all

Revised: [Table]

Revised: One of the worst cult films ever made.

Revised: 2D

Revised: [Table]

Revised: zombies and comic-book characters.

Revised: Some bad acting, technical problems

Revised: [Table]

Revised: cheap gimmicks and script that is.

Revised: [Table]

Revised: The four-hour look at family dysfunction brings forth a richer character study.

Revised: [Table]

Revised: Despite its oddball structure and topsy-turvy interactions between characters, this surprisingly zany animated film succeeds where so many animated films fail.

Revised: [Table]

Revised: Not worth the time for the main actors, but for the almost movie has a very good story that puts many sci-fi movies of the past to shame.

Revised: [Table]

Revised: An outrageously stupid movie that has been thoroughly disproved by fact and logic.

Revised: [Table]

Revised: Wonder Woman is a big-budget superhero blockbuster that turns the spotlight on the potential of a woman leader...but the movie is ultimately unfulfilling and laden with female stereotypes.

Revised: [Table]

Revised: While a satire on class and power

Revised: [Table]

Revised: the film’s dismissal of human misery is shallow and the actors’ portrayal of the weak and needy are gratingly self-pitying.

Revised: [Table]

Revised: More recently, data-generation-based work via PLM has gained popularity and shown superior ability on synthesizing task-specific data (Anaby-Tavor et al., 2020;

Original: Puri et al., 2020; Kumar et al., 2020; Lee et al., 2021; Wang et al., 2021b; Yoo et al., 2021; Bonifacio et al., 2022).
Revised: Puri et al., 2020; Kumar et al., 2020; Lee et al., 2021; Wang et al., 2021b; Yoo et al., 2021; Bonifacio et al., 2022).

Original: Apart from work that still relies on task-related human-annotated data to instruct or fine-tune the generative PLM, a recent line of research explores this direction in zero-shot scenario:
Revised: Apart from work that still relies on task-related human-annotated data to instruct or fine-tune the generative PLM, a recent line of research explores this direction in zero-shot scenario:

Original: Schick & Schütze (2021); Meng et al.
Revised: Schick & Schütze (2021); Meng et al.

Original: (2022) use PLM with task-dependent prompts to generate data, and finetune another PLM on such data for task prediction.
Revised: (2022) use PLM with task-dependent prompts to generate data, and finetune another PLM on such data for task prediction.

Original: To further investigate PLM’s zero-shot ability and alleviate the computation cost of PLM, Ye et al.
Revised: To further investigate PLM’s zero-shot ability and alleviate the computation cost of PLM, Ye et al.

Original: (2022) study an extreme scenario, which trains a tiny model from scratch using the synthetic data.
Revised: (2022) study an extreme scenario, which trains a tiny model from scratch using the synthetic data.

Original: Noise Robust Learning.
Revised: Noise Robust Learning.

Original: Previous methods tackling data noise problem can be roughly categorized into two groups: (1) heuristic approaches based on loss values that rely on the assumption that the network learns easy samples first, which adopt either resampling (Han et al., 2018; Jiang et al., 2018; Yu et al., 2019), loss reweighting (Thulasidasan et al., 2019; Konstantinov & Lampert, 2019;
Revised: Previous methods tackling data noise problem can be roughly categorized into two groups: (1) heuristic approaches based on loss values that rely on the assumption that the network learns easy samples first, which adopt either resampling (Han et al., 2018; Jiang et al., 2018; Yu et al., 2019), loss reweighting (Thulasidasan et al., 2019; Konstantinov & Lampert, 2019;

Original: Ren et al., 2018; Shu et al., 2019), or label correction (Ma et al., 2018; Kremer et al., 2018;
Revised: Ren et al., 2018; Shu et al., 2019), or label correction (Ma et al., 2018; Kremer et al., 2018;

Original: Reed et al., 2014).
Revised: Reed et al., 2014).

Original: These methods require either manually set a threshold for the loss value or a clean validation set, which makes their performance questionable in zero-shot scenario.
Revised: These methods require either manually set a threshold for the loss value or a clean validation set, which makes their performance questionable in zero-shot scenario.

Original: (2) methods in another line train the network with noise-robust loss (Ghosh et al., 2017; Ma et al., 2020; Liu &Guo, 2020;
Revised: (2) methods in another line train the network with noise-robust loss (Ghosh et al., 2017; Ma et al., 2020; Liu &

Original:  Xu et al., 2019; Wang et al., 2019).
Revised: Guo, 2020; Xu et al., 2019; Wang et al., 2019).

Original: Despite they learn a robust classifier in theory, they are typically difficult to train the DNNs and result require more hyper-parameter tuning (Zhang &Sabuncu, 2018;
Revised: Despite they learn a robust classifier in theory, they are typically difficult to train the DNNs and result require more hyper-parameter tuning (Zhang &

Original:  Wang et al., 2019).
Revised: Sabuncu, 2018; Wang et al., 2019).

Original: To this end, we take advantages from both lines of research and design an end-to-end framework which can reliably filter out harmful data without requiring a clean validation set.
Revised: To this end, we take advantages from both lines of research and design an end-to-end framework which can reliably filter out harmful data without requiring a clean validation set.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: This paper focuses on high-quality data generation in efficient zero-shot learning.
Revised: This paper focuses on high-quality data generation in efficient zero-shot learning.

Original: To attack the noise  data, we design an end-to-end framework to construct a clean synthetic dataset without relying on any humanlabeled data or human intervention.
Revised: To address the noise in data, we design an end-to-end framework to construct a clean synthetic dataset without relying on any human-labeled data or human intervention.

Original: Our method can be jointly applied to other data generation pipelines to automatically select high-quality data.
Revised: Our method can be jointly applied to other data generation pipelines to automatically select high-quality data.

Original: We hope this paper can provide insights for improving the quality of synthetic datasets, and inspire more exploration in data-generation-based zero-shot learning via PLM.
Revised: We hope this paper can provide insights for improving the quality of synthetic datasets, and inspire more exploration in data-generation-based zero-shot learning via PLM.

Original:  R EPRODUCIBILITY S TATEMENT
Revised:  R EPRODUCIBILITY S TATEMENT

Original: Our code for data generation is based on Ye et al.
Revised: Our code for data generation is based on Ye et al.

Original: (2022), which is publically available on GitHub.
Revised: (2022), which is publically available on GitHub.

Original: We submit our code together with ICLR submission and we will publically release the code upon acceptance of the paper.
Revised: We submit our code together with ICLR submission, and we will publically release the code upon acceptance of the paper.

Original: The hyper-parameters and network choices used throughout the experiments are elaborated in 
Revised: The hyper-parameters and network choices used throughout the experiments are elaborated in Section 5.1 and Appendix F.

Original: Section 5.1 and Appendix F. For theoretical analysis given in our paper, the assumptions and theorems are in Section 4, and the full proofs are given in Appendix D
Revised: For theoretical analysis given in our paper, the assumptions and theorems are in Section 4, and the full proofs are given in Appendix C. A PPENDIX A A BLATION S TUDY OF O UTER O BJECTIVES

Revised: In the main paper, we directly use RCE as an example of noise-robust loss to verify our framework.

Original: A A PPENDIX A T RUNCATED B ACK - PROPAGATION FOR M ETA G RADIENT . 
Revised: Here we conduct additional experiments using the other noise-robust Mean Absolute Error (MAE) loss (Ghosh et al., 2017). The results in Table 7 show that S UN G EN framework using RCE and MAE losses both achieve promising performance, which significantly surpasses baseline methods. Besides, if we consider the standard cross-entropy(CE) loss as the outer objective, the bi-level framework can only achieve marginal improvement or even worse than Z ERO G EN . The result is reasonable as the standard CE loss in the outer loop run on synthetic data, which cannot provide accurate guidance to update the per-sample weights.

Original: For solving the bilevel optimization problem, the gradient of w can be calculated as follows:
Revised: [Table] B R ELAXING A SSUMPTION Here we provide additional results on relaxed assumptions. We relax Assumption 2 as follows:

Revised: Assumption 3.

Revised: The optimal θ ∗ achieves reasonable robust loss L robust ( θ , S clean ) on the clean dataset, i.e., there exists ϵ > 0 such that L robust ( θ ∗ , S clean ) <

Revised: L robust ( θ , S clean ) + ϵ for all θ ̸

Revised: = θ ∗ .

Revised: Then we have the following results that extend Theorem 1:

Revised: Theorem 3.

Revised: If Assumption 1 holds, there exists a w ∗ such that ˆ θ ( w ∗ )

Revised: = θ ∗ .

Revised: Further with Assumption 3 and Property 1 and assume L robust ( θ ( w ) , S clean ) is µ -strongly convex and differentiable w.r.t.

Revised: w , our method can return a w that is close to w ∗ as follows:

Original: [Equation]
Revised: [Equation]

Original: where Eqn. (5) follows chain rule.
Revised: Further, if the minimizer of the robust loss on the clean data collides with θ ∗ , i.e., ϵ = 0 , we have w = w ∗ and θ = θ ∗ . Proof.

Original: For computational efficiency, we do not unroll the entire T steps, but perform 1-step truncated backpropagation as in Eqn. (6) (Shaban et al., 2019).
Revised: Similar to the first step of the proof of Theorem 1 in Appendix C.3, we know ˆ θ ( w ∗ )

Original: Our framework has the following advantages against other noise-robust learning methods: • Compared with heuristic methods (Han et al., 2018; Jiang et al., 2018), our framework is end-to-end and does not require excessive manual tuning and task-specific knowledge. • Since both the inner and outer objectives are calculated on the same synthetic training set, we do not need any in-domain labeled data, which is a must in the previous end-to-end reweighting methods (Ren et al., 2018; Shu et al., 2019).
Revised: = θ ∗ .

Original: • Compared with methods that train the model with ℓ robust (Zhang & Sabuncu, 2018; Wang et al., 2019), our approach leverages ℓ robust to learn the sample weights, which enables removing the low-quality data without hurting the model’s performance [Table]
Revised: Further by Property 1 we know the solution of arg min θ L robust ( θ , S syn ) is the same with that of arg min θ L robust ( θ , S clean ) .

Original: The film does a great
Revised: We further have

Original: [Table]
Revised: [Equation]

Original: This long, pompous, chain-smoking movie makes a big hit out of not very much at all.
Revised: Denote ¯ w = arg min θ L robust (ˆ θ ( w ) , S clean ) , it follows that

Original: [Table]
Revised: [Equation]

Original: One of the worst cult films ever made. 2D CGI animations of zombies and comic-book characters. Some bad acting, technical problems, cheap gimmicks and script that is.
Revised: By Assumption 3, we then have

Original: [Table]
Revised: [Equation]

Original: The four-hour look at family dysfunction brings forth a richer character study. [Table] visceral
Revised: We further have

Original: [Table]
Revised: [Equation]

Original: of [Table] life
Revised: The equality in the first line is due to ˆ θ ( w ∗ )

Original: [Table]
Revised: = θ ∗ .

Original: Despite its oddball structure and topsy-turvy interactions between characters, this surprisingly zany animated film succeeds where so many animated films fail. [Table] Not worth the time for the main actors, but for the almost movie has a very good story that puts many sci-fi movies of the past to shame.
Revised: The inequality comes from the strong convexity assumption.

Original: [Table] An outrageously stupid movie that has been thoroughly disproved by fact and logic. [Table] Wonder Woman is a big-budget superhero blockbuster that turns the spotlight on the potential of a woman leader...but the movie is ultimately unfulfilling and laden with female stereotypes.
Revised: Combining Eqn equation 5, equation 6 and equation 7, we have

Original: [Table]
Revised: [Equation]

Original: While a satire on class and power [Table] the film’s dismissal of human misery isshallow and the actors’ portrayal of the weak and needy are gratingly self-pitying. [Table]
Revised: The result follows immediately by noting that ¯ w is the output of our method.

Original: C S ELECTED ANDR EMOVED DATA
Revised: C P ROOFS C. E MPIRICAL V ERIFICATION FOR A SSUMPTION 2 AND

Original: The selected and removed examples are listed in Table 6.
Revised: Summary and Discussion.

Original: We take IMDb as the example task. The observation shows that most of  the removed data (data with low weights) have noise label, which indicates the class of text is wrongly labeled by PLM during generation(Noisy Y). Besides, there is a small part of erroneous data which contains unrelated text to  the task, meaning the generated text is not a movie review showing obvious emotional tendency(Unrelated X).
Revised: In the sections below, we first show that (1) the poor performance of training the neural network with robust losses is due to the optimization difficulty, rather than the lack of good solution . We verify this by showing that the loss value of RCE is able to steadily decrease when the network is trained with CE loss.

Original: From Figure 3(d), we find the percentage of the erroneous data is small, but it significantly degrades the model performance(e.g., IMDB accuracy is decreased from 86.56 to 78.29 in Table 4). For selected data by S UN G EN , we find they are actually well-written transitional complex sentences (bottom part of Table 6), which verifies that S UN G EN tends to select correct and challenging samples.
Revised: However, when training the network with

Original: D P ROOFS D. E MPIRICAL V ERIFICATION FOR A SSUMPTION
Revised: RCE, the loss value fails to decrease.

Original: We run experiments on both NLP and CV tasks (IMDb and CIFAR-10) to verify the rationality of  Assumption 2.
Revised: Then, we show that (2) the optimal solutions of CE and RCE losses are close by plotting the loss surface for both losses around the optimal solution obtained with CE loss. We are able to observe that the RCE loss is also close to minumum around the optimal solution of CE loss. We believe the above two experiments can verify that the Assumption 2 andare reasonable.

Revised: [Figure]

Revised: C.1. O PTIMIZATION D IFFICULTY OF N OISE -R OBUST L OSS .

Revised: First, we analyze the cause of difficulty in the optimization of robust loss : (1) (Zhang & Sabuncu, 2018) show that the gradients of the cross-entropy loss have an implicit weighting term(as shown in Equation 8), which prioritizes the harder samples during training.

Revised: On the other hand, this weighting does not exist in noise robust loss functions, which means these functions treat all samples equally.

Revised: The lack of implicit weighting is claimed by (Zhang & Sabuncu, 2018) to be the cause of difficulty in training.

Revised: [Equation]

Revised: (2) Our experiment shows that the surface of the robust loss has a wide flat region, so when the parameters are not close to the optimal solution, the gradients could vanish, which leads to difficulty in optimization.

Revised: The 3D visualization of the loss surface is shown in Figure 4.

Revised: More details about drawing the figure are shown in C.1.2.

Revised: Second, we verify the performance degradation when training the network with robust loss is caused by optimization difficulty, rather than the lack of good solution.

Original: Without loss of generality, we compare the robust-loss ℓ rce to ℓ ce .
Revised: Without loss of generality, we compare the robust-loss ℓ rce to ℓ ce .

Revised: More specifically, we train the model with one loss, and use another loss to evaluate the model trajectory.

Original: The results in Figure 4 show that when training network with ℓ ce , the ℓ rce demonstrates a similar curve as ℓ ce and reaches the plateau at  a similar point.
Revised: From the results in Figure 5(a), we can clearly see that when optimizing the network with CE, both CE and RCE continue to decline throughout the training, and reach the plateau at the same time, which means that the RCE loss does indeed have a good solution. On the other hand, from the results in Figure 5(b), when training with RCE, both CE, and RCE hardly decrease, this further verifies that RCE is difficult to optimize when training the network. [Figure]

Revised: C.1. T HE O PTIMAL S OLUTIONS OF CE AND RCE ARE C LOSE .

Original: This empirically shows that the robust loss ℓ rce has a close optimal  solution with ℓ ce .
Revised: Despite the optimization difficulty of RCE loss, the loss surfaces of CE and RCE losses indicate that the two loss functions have close optimal solutions. More specifically, we plot the loss surfaces of CE and RCE losses centered around the solution of CE loss following (Li et al., 2018), which visualizes the loss surfaces by modifying the model weights in two random directions: [Equation] where θ ∗ is the model parameters optimized by CE, δ and η are the random directions in the vector space of model parameters.

Revised: α and β are the scaling coefficients of the two directions, which control how far the parameters are perturbed.

Revised: We parameterize the surface w.r.t the values of α , β ranging from [-1, 1], and calculate the loss values at different positions alongside a 2D grid.

Revised: We show the visualization in Figure 4.

Revised: From sub-figure 4 (d), we can see that the optimal solution of CE is also close to the optimal solution of RCE (also has the lowest value near zero), which is direct experimental support for the

Revised: Assumption 3.

Revised: In addition, when the solution is not close to the optimal position, the rce loss surface is flat and wide, which is the cause of optimization difficulty.

Original: D. P ROOF FOR P ROPERTY
Revised: C. P ROOF FOR P ROPERTY

Original: The noise-robust property of robust loss functions have been proven in previous work (Ghosh et al., 2017; Wang et al., 2019), we add them here for completeness.
Revised: The noise-robust property of robust loss functions have been proven in previous work (Ghosh et al., 2017; Wang et al., 2019), we add them here for completeness.

Original: Specifically, wefollow Ghosh et al. and consider three cases of label noise: uniform noise, simple non-uniform noise and class-dependent noise. 15 Epoch 0. L o ss
Revised: Specifically, we consider three cases of label noise: uniform noise, simple non-uniform noise and class-dependent noise following (Ghosh et al., 2017).

Original: IMDb ce  15 Epoch 0.

Original: IMDb rce    80 100 120 Epoch 0.

Original: CIFAR ce    80 100 120 Epoch 0.

Original: CIFAR rce

Original: Uniform Noise.
Revised: Uniform Noise.

Original: Given a classification problem with K classes, for a loss function ℓ robust satisfying Eqn.
Revised: Given a classification problem with K classes, for a loss function ℓ robust satisfying Eqn.

Original: Then ℓ robust is noise-tolerant under uniform label noise if the noise rate η < K − 1 K .
Revised: Then ℓ robust is noise-tolerant under uniform label noise if the noise rate η < K − 1 K .

Original: Let ˜ y denote the noisy label from S syn , the proof is as follows:
Revised: Let ˜ y denote the noisy label from S syn , the proof is as follows:

Original: [Equation]
Revised: [Equation]

Original: where C is a constant due to the property of symmetric loss functions.
Revised: where C is a constant due to the property of symmetric loss functions.

Original: Suppose θ ∗ is the optimal solution for the clean dataset S clean .
Revised: Suppose θ ∗ is the optimal solution for the clean dataset S clean .

Original: Therefore, we have the following inequality for any θ : L ( θ ∗ , S syn ) − L ( θ , S syn )
Revised: Therefore, we have the following inequality for any θ : L [Equation] [Equation]

Original: = K − 1 − Kη
Revised: [Equation]

Original: K −( L ( θ ∗ , S clean ) − L ( θ , S clean )) ≤ 0 .
Revised: ( L ( θ ∗ , S clean ) − L ( θ , S clean )) ≤ 0 .

Original: Therefore, θ ∗ is also the optimal solution for S syn .
Revised: Therefore, θ ∗ is also the optimal solution for S syn .

Original: Class Dependent Noise.
Revised: Class Dependent Noise.

Original: For a loss function ℓ robust satisfying Eq 3, and 0 ≤ ℓ robust ( f ( x ; θ ) , i ) ≤ CK − 1 , ∀ i ∈ [ K ] , and suppose min θ L ( θ , S clean ) = 0 .
Revised: For a loss function ℓ robust satisfying Eq 3, and 0 ≤ ℓ robust ( f ( x ; θ ) , i ) ≤ CK − 1 , ∀ i ∈ [ K ] , and suppose min θ L ( θ , S clean ) = 0 .

Original: Then ℓ robust is noise-tolerant under classs dependent label noise if η ij < 1 − η i , ∀ j ̸
Revised: Then ℓ robust is noise-tolerant under classs dependent label noise if η ij < 1 − η i , ∀ j ̸

Original: = i, ∀ i, j ∈ [ K ] , η ij represents the probability of class i mislabelled into class j .
Revised: = i, ∀ i, j ∈ [ K ] , η ij represents the probability of class i mislabelled into class j .

Original: The proof is as follows:
Revised: The proof is as follows:

Original: [Equation]
Revised: [Equation]

Original: Denote θ ∗ and ˆ θ ∗ as arg min θ L robust ( θ , S clean ) and arg min θ L robust ( θ , S syn ) , respectively.
Revised: Denote θ ∗ and ˆ θ ∗ as arg min θ L robust ( θ , S clean ) and arg min θ L robust ( θ , S syn ) , respectively.

Original: Given the above result, we have the following inequality for ˆ θ ∗ and θ ∗ :
Revised: Given the above result, we have the following inequality for ˆ θ ∗ and θ ∗ :

Original: [Equation]
Revised: [Equation]

Original: Since ℓ robust is always non-negative and L robust ( θ ∗ , S clean ) = 0 , we have that ℓ robust ( f ( x ; θ ∗ ) , y )
Revised: Since ℓ robust is always non-negative and L robust ( θ ∗ , S clean ) = 0 , we have that ℓ robust ( f ( x ; θ ∗ ) , y )

Original: = 0 , ∀ x .
Revised: = 0 , ∀ x .

Original: Thus, we have that ℓ robust ( f ( x ; θ ∗ ) , i ) =
Revised: Thus, we have that ℓ robust ( f ( x ; θ ∗ ) , i ) =

Original: CK − 1 , ∀ i ̸
Revised: CK − 1 , ∀ i ̸

Original: =
Revised: =

Original: y by the symmetric property of ℓ robust .
Revised: y by the symmetric property of ℓ robust .

Original: Given the assumption on the label noise, (1 − η y − η yj ) > 0 .
Revised: Given the assumption on the label noise, (1 − η y − η yj ) > 0 .

Original: Therefore, for the equality to hold, we musts have ℓ robust ( f ( x ; ˆ θ ∗ , i ) =
Revised: Therefore, for the equality to hold, we musts have ℓ robust ( f ( x ; ˆ θ ∗ , i ) =

Original: CK − 1 , ∀ i ̸
Revised: CK − 1 , ∀ i ̸

Original: = y .
Revised: = y .

Original: According to the symmetric property of ℓ robust , we know that ℓ robust ( f ( x ; ˆ θ ∗ ) , y )
Revised: According to the symmetric property of ℓ robust , we know that ℓ robust ( f ( x ; ˆ θ ∗ ) , y )

Original: = 0 , ∀ x .
Revised: = 0 , ∀ x .

Original: Which means that ˆ θ ∗ achieve zero losss on all clean samples as well.
Revised: Which means that ˆ θ ∗ achieve zero losss on all clean samples as well.

Original: Therefore, ˆ θ ∗ is also the minimizer for L robust ( θ , S clean ) .
Revised: Therefore, ˆ θ ∗ is also the minimizer for L robust ( θ , S clean ) .

Original: Non-Uniform Noise.
Revised: Non-Uniform Noise.

Original: For a loss function ℓ robust satisfying Eq 3, and suppose min θ L ( θ , S clean ) = 0 .
Revised: For a loss function ℓ robust satisfying Eq 3, and suppose min θ L ( θ , S clean ) = 0 .

Original: Then ℓ robust is noise-tolrant under non-uniform label noise if η x < K − 1 K , ∀ x .
Revised: Then ℓ robust is noise-tolrant under non-uniform label noise if η x < K − 1 K , ∀ x .

Original: The proof is as follows:
Revised: The proof is as follows:

Original: [Equation]
Revised: [Equation]

Original: Therefore, we have the following inequality for any θ : L robust ( θ ∗ , S syn ) − L robust ( θ , S syn) =
Revised: Therefore, we have the following inequality for any θ : L robust ( θ ∗ , S syn ) − L robust ( θ [Equation] S syn

Original: E x ,y [(1 − Kη x K − 1 )( ℓ robust ( f ( x ; θ ∗ ) , y ) − ℓ robust ( f ( x ; θ ) , y ))] .
Revised: [Equation] ℓ [Equation] ( f ( x ; θ ) , y ))] .

Original: Since ℓ robust is always non-negative and L robust ( θ ∗ , S clean ) = 0 , we have that ℓ robust ( f ( x ; θ ∗ ) , y )
Revised: Since ℓ robust is always non-negative and L robust ( θ ∗ , S clean ) = 0 , we have that ℓ robust ( f ( x ; θ ∗ ) , y )

Original: = 0 , ∀ x .
Revised: = 0 , ∀ x .

Original: Therefore, θ ∗ is also the optimal solution for S syn .
Revised: Therefore, θ ∗ is also the optimal solution for S syn .

Original: D. P ROOF FOR T HEOREM
Revised: C. P ROOF FOR T HEOREM

Original: Proof.
Revised: Proof.

Original: We first show θ ∗ exists in the space induced by ˆ θ ( w ) .
Revised: We first show θ ∗ exists in the space induced by ˆ θ ( w ) .

Original: Then we show our framework uniquely return θ ∗ .
Revised: Then we show our framework uniquely return θ ∗ .

Original: Step 1: Existence.
Revised: Step 1: Existence.

Original: By Assumption 1, we know that P syn ( x , y ) w ∗ ( x , y )
Revised: By Assumption 1, we know that P syn ( x , y ) w ∗ ( x , y )

Original: =
Revised: =

Original: P clean ( x , y ) in distribution.
Revised: P clean ( x , y ) in distribution.

Original: So
Revised: So

Original: [Equation]
Revised: [Equation]

Original: Step 2: Uniqueness.
Revised: Step 2: Uniqueness.

Original: By Assumption 2, we have for all θ ̸
Revised: By Assumption 2, we have for all θ ̸

Original: = θ ∗ , we have L robust ( θ ∗ , S clean ) <
Revised: = θ ∗ , we have L robust ( θ ∗ , S clean ) <

Original: L robust ( θ , S clean ) .
Revised: L robust ( θ , S clean ) .

Original: So θ ∗ =

Original: [Equation]
Revised: [Equation]

Original: L robust ( θ , S clean ) .

Original: Since arg min θ L robust ( θ , S syn ) =
Revised: Since arg min θ L robust ( θ , S syn ) =

Original: arg min θ L robust ( θ , S clean ) .
Revised: arg min θ L robust ( θ , S clean ) .

Original: So we have
Revised: So we have

Original: [Equation]
Revised: [Equation]

Original: Putting the existence and uniqueness parts together, we finish the proof.
Revised: Putting the existence and uniqueness parts together, we finish the proof.

Original: The above theorem shows that there is an explicit mapping between the optimal weighting function w ∗ and the optimal θ ∗ .
Revised: The above theorem shows that there is an explicit mapping between the optimal weighting function w ∗ and the optimal θ ∗ .

Original: This mapping is achieved by the inner loop in formulation 2.
Revised: This mapping is achieved by the inner loop in formulation 2.

Original: We can then optimize L ce ( θ ( w ) , S clean ) over w to find the optimal w ∗ .
Revised: We can then optimize L ce ( θ ( w ) , S clean ) over w to find the optimal w ∗ .

Original: Given that Assumption 2 holds, we can switch L ce ( θ ( w ) , S clean ) to L robust ( θ ( w ) , S clean ) .
Revised: Given that Assumption 2 holds, we can switch L ce ( θ ( w ) , S clean ) to L robust ( θ ( w ) , S clean ) .

Original: Finally, thanks to Property D.2 of ℓ robust , we may simply optimize L robust ( θ ( w ) , S syn ) over w instead.
Revised: Finally, thanks to Property 1 of ℓ robust , we may simply optimize L robust ( θ ( w ) , S syn ) over w instead.

Original: This gives rise to the outer loop in formulation 1.
Revised: This gives rise to the outer loop in formulation 1.

Original: D. P ROOF FOR T HEOREM
Revised: C. [Equation]

Original: Proof.
Revised: Proof.

Original: Let ˆ S v, − 1 syn denotes the dataset that replaces any one element of ˆ S vsyn with arbitrary x , it is easy to know that
Revised: Let ˆ S v, − 1 syn denotes the dataset that replaces any one element of ˆ S vsyn with arbitrary x , it is easy to know that

Original: [Equation]
Revised: [Equation]

Original: holds for any w .
Revised: holds for any w .

Original: Then by the bounded difference inequality (Corollary 2.21 of Wainwright (2019)) [Equation] given w [Equation]
Revised: Then by the bounded difference inequality (Corollary 2.21 of Wainwright (2019)), given w , we have with probability 1 − δ ,

Original: we have with probability 1 − δ ,

Original: [Equation]
Revised: [Equation]

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: The first inequality because we require inequality equation 7 to hold uniformly for all |W| functions.
Revised: The first inequality because we require inequality equation 9 to hold uniformly for all |W| functions.

Original: The second inequality is because ˆ w is the ϵ -approximated solution.
Revised: The second inequality is because ˆ w is the ϵ -approximated solution.

Original: The third inequality is applying inequality equation 7.
Revised: The third inequality is applying inequality equation 9.

Original: The forth inequality is because |W| > 1 .
Revised: The forth inequality is because |W| > 1 .

Original: Taking infimum over w on the right hand side, we obtain the desired bound.
Revised: Taking infimum over w on the right hand side, we obtain the desired bound.

Original: E B I -L EVEL VS . O NE -L EVEL O PTIMIZATION
Revised: D S UN G EN VS . O THER N OISE -R OBUST T RAINING S TRATEGIES

Original: We empirically verify the effectiveness of bi-level S UN G EN than (1) one-level optimazation using l rce ( Onelevel, ℓ rce ) and (2) bi-level optimization using ℓ ce as both inner and outer objectives ( Bi-level, ℓ ce ).
Revised: We compare our method with the label smoothing and temporal ensemble noise-training strategies in

Original: From the results in Table 7, we can observe that our framework with bi-level ℓ rce outperforms both counterparts significantly.
Revised: Table 8. The experimental results show that our method achieved a significant improvement over these two counterparts.

Revised: In addition, label smoothing and temporal ensemble are training strategies to alleviate the noise issue, which cannot be used to select a high-quality subset.

Original: [Table]
Revised: [Table]

Revised: E E XPERIMENTS ON O THER C LASSIFICATION T ASKS

Revised: As suggested, we conduct experiments on more difficult tasks of GLUE in Table 9, including the NLI tasks (RTE, QNLI) and paraphrase task (MRPC).

Revised: The experimental results show that SunGen can consistently surpass the baseline methods.

Revised: For more challenging tasks that need expert knowledge or reasoning ability, we leave them as our future work.

Revised: [Table]

Original: F I MPLEMENTATION D ETAILS F. D ATASET
Revised: F I MPLEMENTATION D ETAILS F. D ATASET

Original: We evaluate our method on eight widely-used text classification tasks, including IMDb (Maas et al., 2011), SST-2 Socher et al.
Revised: We evaluate our method on eight widely-used text classification tasks, including IMDb (Maas et al., 2011), SST-2 Socher et al.

Original: (2013), Rotten Tomatoes (Pang & Lee, 2005), Amazon (McAuley &Leskovec, 2013),
Revised: Rotten Tomatoes (Pang & Lee, 2005), Amazon (McAuley &

Original:  Yelp (Zhang et al., 2015), Subj (Pang & Lee, 2004), AGNews (Zhang et al., 2015) and DBpedia (Zhang et al., 2015).
Revised: Leskovec, 2013), Yelp (Zhang et al., 2015), Subj (Pang & Lee, 2004), AGNews (Zhang et al., 2015) and DBpedia (Zhang et al., 2015).

Original: IMDB, SST-2, and Rotten Tomatoes are sentiment classification benchmarks containing positive/negative movie reviews.
Revised: IMDB, SST-2, and Rotten Tomatoes are sentiment classification benchmarks containing positive/negative movie reviews.

Original: Amazon and Yelp are comments classification tasks consisting of electronic product reviews and restaurant reviews respectively.
Revised: Amazon and Yelp are comments classification tasks consisting of electronic product reviews and restaurant reviews respectively.

Original: We choose electronics and restaurant reviews as they are very different from movie reviews.
Revised: We choose electronics and restaurant reviews as they are very different from movie reviews.

Original: Subj is a subjectivity detection task to justify whether the text contains factual contents or expresses opinions.
Revised: Subj is a subjectivity detection task to justify whether the text contains factual contents or expresses opinions.

Original: AGNews (4-class classification) and DBpedia (14-class classification) are the topic and otologogy classification tasks respectively.
Revised: AGNews (4-class classification) and DBpedia (14-class classification) are the topic and otologogy classification tasks respectively.

Original: Apart from AGNews and DBpedia, other tasks are binary classification tasks.
Revised: Apart from AGNews and DBpedia, other tasks are binary classification tasks.

Original: We use full test set for evaluation except for DBpedia, for which we randomly sample 5000 test examples to reduce computational cost.
Revised: We use full test set for evaluation except for DBpedia, for which we randomly sample 5000 test examples to reduce computational cost.

Original: Sample sizes are listed in Table 1.
Revised: Sample sizes are listed in Table 1.

Original: We report accuracy for evaluation.
Revised: We report accuracy for evaluation.

Original: F. F ULL IMPLEMENTATION DETAILS
Revised: F. F ULL IMPLEMENTATION DETAILS

Original: We compare the baselines using GPT2-XL (Radford et al., 2019) as PLM.
Revised: We compare the baselines using GPT2-XL (Radford et al., 2019) as PLM.

Original: For text generation, we useNucleus
Revised: For text generation, we use

Original:  Sampling (Holtzman et al., 2020) with p = 0 .
Revised: Nucleus Sampling (Holtzman et al., 2020) with p = 0 .

Original: 9 as the decoding strategy and use GPT2-XL as the generator.
Revised: 9 as the decoding strategy and use GPT2-XL as the generator.

Original: For fair comparison, we use the best prompts designed by (Ye et al., 2022) for data generation.
Revised: For fair comparison, we use the best prompts designed by (Ye et al., 2022) for data generation.

Original: For task model training, we use 1-layer Bi-LSTM and DistilBERT-base as the light-weight classifiers.
Revised: For task model training, we use 1-layer Bi-LSTM and DistilBERT-base as the light-weight classifiers.

Original: For LSTM, we use Adam optimizer(Kingma & Ba, 2015) with learning rate 1e-3.
Revised: For LSTM, we use Adam optimizer(Kingma & Ba, 2015) with learning rate 1e-3.

Original: For DistilBERT-base, we finetune each dataset using Adam optimizer with learning rate 2e-5, and other default hyper-parameters as suggested by HuggingFace Transformers library(Wolf et al., 2019).
Revised: For DistilBERT-base, we finetune each dataset using Adam optimizer with learning rate 2e-5, and other default hyper-parameters as suggested by HuggingFace Transformers library(Wolf et al., 2019).

Original: We run LSTM for 5 epochs, and run DistilBERT-base for 3 epochs for prediction.
Revised: We run LSTM for 5 epochs, and run DistilBERT-base for 3 epochs for prediction.

Original: Unless otherwise stated, we run our experiments on 200k data.
Revised: Unless otherwise stated, we run our experiments on 200k data.

Original: We compute the average accuracy on test set over 3 runs using different random seeds.
Revised: We compute the average accuracy on test set over 3 runs using different random seeds.

Original: During the optimization of sample weights,  Adam optimizer with learning rate 2.5e-1 is used to update the sample-weights.
Revised: During the optimization of sample weights, we use Adam optimizer. For selecting the appropriate value of the outer learning rate, we select from {2.5e-1, 1e-1, 1e-2} by looking at the value of RCE loss in the outer loop.

Revised: If the outer loss steadily decreases and reaches a low value, then it indicates that the optimization is going well.

Original: In the inner loop, 1,000k synthetic data are used as the training data; in the outer loop, 50k synthetic samples are randomly sampled as the training data for fast iteration.
Revised: In the inner loop, 1,000k synthetic data are used as the training data; in the outer loop, 50k synthetic samples are randomly sampled as the training data for fast iteration.

Original: We use 1-layer Bi-LSTM and DistilBERT-base as the tiny task model and run it for 1 epoch each time for fast iteration.
Revised: We use 1-layer Bi-LSTM and DistilBERT-base as the tiny task model and run it for 1 epoch each time for fast iteration.

Original: The bilevel procedure is iterated for 50 times for each task.
Revised: The bilevel procedure is iterated for 50 times for each task.

Original: For the baseline using gold data in Table 2, to simulate the scenario where gold data is scarce, we randomly select 1,000 samples from the standard training set as the training data in outer loop.
Revised: For the baseline using gold data in Table 2, to simulate the scenario where gold data is scarce, we randomly select 1,000 samples from the standard training set as the training data in outer loop.

Original: For comparison with other denoising baselines shown in Table 3, we use the techniques as described in the original papers.
Revised: For comparison with other denoising baselines shown in Table 3, we use the techniques as described in the original papers.

Original: Specifically, for Confidence (Swayamdipta et al., 2020), we use the mean model probability of the true label across epochs as the confidence value and select top 200k examples; for Co-teaching (Han et al., 2018), we use two networks, each is trained with samples selected by the other network based on the loss value; for meta-weight-net (Shu et al., 2019), since we do not have access to clean data, we use a part of synthetic data as validation.
Revised: Specifically, for Confidence (Swayamdipta et al., 2020), we use the mean model probability of the true label across epochs as the confidence value and select top 200k examples; for Co-teaching (Han et al., 2018), we use two networks, each is trained with samples selected by the other network based on the loss value; for meta-weight-net (Shu et al., 2019), since we do not have access to clean data, we use a part of synthetic data as validation.

Original: Co-teaching and meta-weight-net are conducted on 200k synthetic data.
Revised: Co-teaching and meta-weight-net are conducted on 200k synthetic data.

Original: F. P ROMPTS
Revised: F. P ROMPTS

Original: For IMDb, SST-2, and Rotten Tomatoes, we use the best prompts designed by Ye et al.
Revised: For IMDb, SST- [Table] and Rotten Tomatoes, we use the best prompts designed by Ye et al.

Original: (2022) in both P ROMPTING and data-generation settings.
Revised: (2022) in both P ROMPTING and data-generation settings.

Original: For tasks that are not investigated by Z ERO G EN , following Ye et al.
Revised: For tasks that are not investigated by Z ERO G EN , following Ye et al.

Original: [Table]
Revised: [Table]

Original: we manually design five prompts for each task and report the result of the best prompt on P ROMPTING , then we use the same prompt(or minor revision version) for Z ERO G EN and S UN G EN to generate data.
Revised: we manually design five prompts for each task and report the result of the best prompt on P ROMPTING , then we use the same prompt(or minor revision version) for Z ERO G EN and S UN G EN to generate data.

Original: The details of prompts are shown in Table 
Revised: The details of prompts are shown in Table [Table] [Table]

Revised: G B I -L EVEL VS . O NE -L EVEL O PTIMIZATION

Revised: We empirically verify the effectiveness of bi-level S UN G EN than one-level optimization using l rce ( One-level, ℓ rce ).

Revised: From the results in Table

Original: [Table]
Revised: [Table]

Revised: we can observe that our framework with bi-level ℓ rce outperforms both one-level counterparts significantly

Original: [Table] 
Revised: [Table] [Table]

Revised: H T RUNCATED B ACK - PROPAGATION FOR M ETA G RADIENT .

Revised: For solving the bilevel optimization problem, the gradient of w can be calculated as follows:

Original: [Figure]
Revised: [Equation]

Revised: where Eqn.

Revised: (10) follows chain rule.

Revised: For computational efficiency, we do not unroll the entire T steps, but perform 1-step truncated backpropagation as in Eqn.

Revised: (11) (Shaban et al., 2019).

Revised: I C OMPARISON WITH OTHER NOISE - ROBUST LEARNING METHODS

Revised: Our framework has the following advantages against other noise-robust learning methods:

Revised: • Compared with heuristic methods (

Revised: Han et al., 2018; Jiang et al.,

Revised: ), our framework is end-to-end and does not require excessive manual tuning and task-specific knowledge.

Revised: • Since both the inner and outer objectives are calculated on the same synthetic training set, we do not need any in-domain labeled data, which is a must in the previous end-to-end reweighting methods (Ren et al., 2018; Shu et al., 2019).

Revised: • Compared with methods that train the model with ℓ robust (Zhang & Sabuncu, 2018;

Revised: Wang et al., 2019), our approach leverages ℓ robust to learn the sample weights, which enables removing the low-quality data without hurting the model’s performance

Revised: J S ELECTED AND R EMOVED DATA

Revised: The selected and removed examples are listed in Table 6.

Revised: We take IMDb as the example task.

Revised: The observation shows that most of the removed data (data with low weights) have noise label, which indicates the class of text is wrongly labeled by PLM during generation(Noisy Y).

Revised: Besides, there is a small part of erroneous data which contains unrelated text to the task, meaning the generated text is not a movie review showing obvious emotional tendency(Unrelated X).

Revised: From Figure 3(d), we find the percentage of the erroneous data is small, but it significantly degrades the model performance(e.g.,

Revised: IMDB accuracy is decreased from 86.56 to 78.29 in Table 4).

Revised: For selected data by S UN G EN , we find they are actually well-written transitional complex sentences (bottom part of Table 6), which verifies that S UN G EN tends to select correct and challenging samples.

Original: G W HY LOSS - VALUE - BASED DE - NOISE METHODS DOES NOT WORK IN OUR SITUATION ?
Revised: K W HY LOSS - VALUE - BASED DE - NOISE METHODS DOES NOT WORK IN OUR SITUATION ?

Original: We further empirically analyze why the popular loss-value-based methods (Shu et al., 2019; Han et al., 2018) fail to help sample noise-free dataset in our scenario.
Revised: We further empirically analyze why the popular loss-value-based methods (Shu et al., 2019; Han et al., 2018) fail to help sample noise-free dataset in our scenario.

Original: For this experiment, we manually construct a dataset with a 30% label noise using the training set of SST-2 by randomly flipping the class label.
Revised: For this experiment, we manually construct a dataset with a 30% label noise using the training set of SST-2 by randomly flipping the class label.

Original: As shown in figure 5, the loss values of the correctly labelled data and the mislabelled data are still clustered together.
Revised: As shown in figure 6, the loss values of the correctly labelled data and the mislabelled data are still clustered together.

Original: We conjecture that this is due to the nature of the task: not all tasks demonstrate the phenomenon that noisy data demonstrate higher loss values, which is also mentioned in works related to instance dependent noise learning (Cheng et al., 2020).
Revised: We conjecture that this is due to the nature of the task: not all tasks demonstrate the phenomenon that noisy data demonstrate higher loss values, which is also mentioned in works related to instance dependent noise learning (Cheng et al., 2020).

Original: Therefore, selecting subset based on the loss value is not applicable in our scenario. 
Revised: Therefore, selecting subset based on the loss value is not applicable in our scenario. [Figure]