Original: G RADUATED N ON -C ONVEXITY FOR R OBUST S ELFT RAINED L ANGUAGE U NDERSTANDING
Revised: G RADUATED N ON -C ONVEXITY FOR R OBUST S ELFT RAINED L ANGUAGE U NDERSTANDING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Self-training has been proven to be an efficient strategy for unsupervised finetuning of language models using unlabeled data and model-generated pseudolabels.
Revised: Self-training has been proven to be an efficient strategy for unsupervised finetuning of language models using unlabeled data and model-generated pseudolabels.

Original: However, the performance of self-trained models is unstable under different conditions of the training and evaluation data, influenced by both data distribution and pseudo-label accuracy.
Revised: However, the performance of self-trained models is unstable under different conditions of the training and evaluation data, influenced by both data distribution and pseudo-label accuracy.

Original: In this work, we propose an outlier robust self-training method based on graduated non-convexity (GNC) to mitigate the problem.
Revised: In this work, we propose an outlier robust self-training method based on graduated non-convexity (GNC) to mitigate the problem.

Original: We construct self-training as a non-convex optimization problem with outlier training examples.
Revised: We construct self-training as a non-convex optimization problem with outlier training examples.

Original: The models are self-trained with robust cost functions according to Black-Rangarajan Duality.
Revised: The models are self-trained with robust cost functions according to Black-Rangarajan Duality.

Original: The algorithm learns slack variables as the loss weights for all training samples.
Revised: The algorithm learns slack variables as the loss weights for all training samples.

Original: The slack variables are used to calibrate the loss items during training to update the model parameters.
Revised: The slack variables are used to calibrate the loss items during training to update the model parameters.

Original: The calibrated loss items lead to more robust self-trained models against different training and evaluation data and tasks.
Revised: The calibrated loss items lead to more robust self-trained models against different training and evaluation data and tasks.

Original: We conduct experiments on few-shot natural language understanding tasks with labeled and unlabeled data examples.
Revised: We conduct experiments on few-shot natural language understanding tasks with labeled and unlabeled data examples.

Original: Experimental results show that the proposed loss calibration method improves the performance and stability of self-training on different tasks, benefiting the robustness against incorrect pseudo-labels, imbalanced training data, overfitting and adversarial evaluation data.
Revised: Experimental results show that the proposed loss calibration method improves the performance and stability of self-training on different tasks, benefiting the robustness against incorrect pseudo-labels, imbalanced training data, overfitting and adversarial evaluation data.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Recent developments in large-scale pretrained language models has significantly improved the performance of natural language understanding tasks (Devlin et al., 2018; Liu et al., 2019; Clark et al., 2020;
Revised: Recent developments in large-scale pretrained language models has significantly improved the performance of natural language understanding tasks (Devlin et al., 2018; Liu et al., 2019; Clark et al., 2020;

Original: He et al., 2020; Brown et al., 2020).
Revised: He et al., 2020; Brown et al., 2020).

Original: After pretraining, these models are typically fine-tuned on task-specific training data with human-generated labels.
Revised: After pretraining, these models are typically fine-tuned on task-specific training data with human-generated labels.

Original: However, human-generated labels are not available (or large enough) for all tasks of interest.
Revised: However, human-generated labels are not available (or large enough) for all tasks of interest.

Original: When there is a significant number of unlabeled examples, a pretrained model can utilize techniques such as self-training to improve performance (He et al., 2019; Zoph et al., 2020).
Revised: When there is a significant number of unlabeled examples, a pretrained model can utilize techniques such as self-training to improve performance (He et al., 2019; Zoph et al., 2020).

Original: An issue with this approach is that the generated pseudo-labels can be noisy (Zhao et al., 2021; Lang et al., 2022; Zhang & Zhou, 2011), since a pretrained model can make wrong predictions for unseen examples.
Revised: An issue with this approach is that the generated pseudo-labels can be noisy (Zhao et al., 2021; Lang et al., 2022; Zhang & Zhou, 2011), since a pretrained model can make wrong predictions for unseen examples.

Original: We propose a learning-based loss calibration strategy that tunes the loss weights for each data example during self-training.
Revised: We propose a learning-based loss calibration strategy that tunes the loss weights for each data example during self-training.

Original: In this approach, a subset of training data are assigned low weights for calculating the overall training loss, leading to less influence in parameter updates.
Revised: In this approach, a subset of training data are assigned low weights for calculating the overall training loss, leading to less influence in parameter updates.

Original: To learn the loss weights, we employ the graduated non-convexity (GNC) strategy
Revised: To learn the loss weights, we employ the graduated non-convexity (GNC) strategy

Original: (Yang et al., 2020) based on
Revised: (Yang et al., 2020) based on

Original: Black-Rangarajan Duality (Black & Rangarajan, 1996).
Revised: Black-Rangarajan Duality (Black & Rangarajan, 1996).

Original: Under the fully supervised and outlier-free setting, the model parameters are updated to optimize the total cost of all training samples, while under the self-training setting, the training set usually contains outliers with wrong pseudo-labels.
Revised: Under the fully supervised and outlier-free setting, the model parameters are updated to optimize the total cost of all training samples, while under the self-training setting, the training set usually contains outliers with wrong pseudo-labels.

Original: We thus optimize a robust cost function
Revised: We thus optimize a robust cost function

Original: [Equation]
Revised: [Equation]

Original: where l ( · ) is the selected loss function and D train
Revised: where l ( · ) is the selected loss function and D train

Original: = { ( x i , y i ) | i ∈
Revised: = { ( x i , y i ) | i ∈

Original: [0 , N ] } is the training set, and θ stands for the set of trainable parameters.
Revised: [0 , N ] } is the training set, and θ stands for the set of trainable parameters.

Original: ρ ( · ) is a robust cost function and ˆ y i stands for an element of a noisy pseudo label set.
Revised: ρ ( · ) is a robust cost function and ˆ y i stands for an element of a noisy pseudo label set.

Original: Such methods have been widely applied in computer vision (Black & Rangarajan, 1996) and robot perception (Yang et al., 2020) where there exists a ground-truth optimization target for the algorithm to predict.
Revised: Such methods have been widely applied in computer vision (Black & Rangarajan, 1996) and robot perception (Yang et al., 2020) where there exists a ground-truth optimization target for the algorithm to predict.

Original: Few application examples have been seen in other domains and the approach has not been applied in the area of self-training.
Revised: Few application examples have been seen in other domains and the approach has not been applied in the area of self-training.

Original: The difficulty is that unlike prediction, an optimal self-training loss on training examples does not guarantee minimal loss on evaluation sets.
Revised: The difficulty is that unlike prediction, an optimal self-training loss on training examples does not guarantee minimal loss on evaluation sets.

Original: In fact, adding noise in self-training can produce better performance, for example with Dropout (He et al., 2019) and confidence regularization (Zou et al., 2019).
Revised: In fact, adding noise in self-training can produce better performance, for example with Dropout (He et al., 2019) and confidence regularization (Zou et al., 2019).

Original: In this work, we propose GNC self-training that produces robust performance against incorrect pseudo labels, imbalanced training data, overfitting, and adversarial evaluation data.
Revised: In this work, we propose GNC self-training that produces robust performance against incorrect pseudo labels, imbalanced training data, overfitting, and adversarial evaluation data.

Original: The core idea is penalizing outliers with wrong pseudo labels and over-confident training samples by assigning low loss weights.
Revised: The core idea is penalizing outliers with wrong pseudo labels and over-confident training samples by assigning low loss weights.

Original: As a result, such training samples contribute less to the updated model parameters than other samples.
Revised: As a result, such training samples contribute less to the updated model parameters than other samples.

Original: In this work we make the following contributions,
Revised: In this work we make the following contributions,

Original: Applying graduated non-convexity (GNC) in self-training.
Revised: Applying graduated non-convexity (GNC) in self-training.

Original: To the best of our knowledge, this work is the first attempt of applying GNC for self-trained language models.
Revised: To the best of our knowledge, this work is the first attempt of applying GNC for self-trained language models.

Original: While traditional GNC methods treat high-loss observations as outliers, we noticed that in self-training, outliers include both high-loss and over-confident examples.
Revised: While traditional GNC methods treat high-loss observations as outliers, we noticed that in self-training, outliers include both high-loss and over-confident examples.

Original: We propose a shifted robust cost function to deal with over-confident cases, and the loss weights are tuned during self-training.
Revised: We propose a shifted robust cost function to deal with over-confident cases, and the loss weights are tuned during self-training.

Original: Robustness against different training and evaluation data.
Revised: Robustness against different training and evaluation data.

Original: The difference within different training and evaluation corpora leads to significant performance gaps between the two because of the accuracy of pseudo-labels and the difficulty/coverage of data examples.
Revised: The difference within different training and evaluation corpora leads to significant performance gaps between the two because of the accuracy of pseudo-labels and the difficulty/coverage of data examples.

Original: We find that the proposed method is robust against different tasks, training corpora, and adversarial evaluation data, outperforming label-free self-training strategies and label-dependent few-shot learning baselines.
Revised: We find that the proposed method is robust against different tasks, training corpora, and adversarial evaluation data, outperforming label-free self-training strategies and label-dependent few-shot learning baselines.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Recent pretrained language models are trained with a self-supervised learning strategy, including predicting masked words (Devlin et al., 2018;
Revised: Recent pretrained language models are trained with a self-supervised learning strategy, including predicting masked words (Devlin et al., 2018;

Original: Liu et al., 2019; He et al., 2020), predicting next words (Brown et al., 2020; Raffel et al., 2020; Lewis et al., 2019), predicting the correctness of words (Clark et al., 2020), and representing sentences (Gao et al., 2021; Chuang et al., 2022).
Revised: Liu et al., 2019; He et al., 2020), predicting next words (Brown et al., 2020; Raffel et al., 2020; Lewis et al., 2019), predicting the correctness of words (Clark et al., 2020), and representing sentences (Gao et al., 2021; Chuang et al., 2022).

Original: Besides pretraining on corpora without additional human-generated labels, textual entailment corpora, including SNLI (Bowman et al., 2015) and MultiNLI (Williams et al., 2018) are also used for both sentence-level pretraining (Reimers & Gurevych, 2019) and downstream tasks; for example relation extraction (Obamuyide & Vlachos, 2018;
Revised: Besides pretraining on corpora without additional human-generated labels, textual entailment corpora, including SNLI (Bowman et al., 2015) and MultiNLI (Williams et al., 2018) are also used for both sentence-level pretraining (Reimers & Gurevych, 2019) and downstream tasks; for example relation extraction (Obamuyide & Vlachos, 2018;

Original: Yin et al., 2019) and fact checking (Thorne & Vlachos, 2018).
Revised: Yin et al., 2019) and fact checking (Thorne & Vlachos, 2018).

Original: While most self-supervised learning methods are agnostic to downstream tasks, self-training models learn to handle any downstream task by learning from synthetic data or pseudo-labels that are automatically generated by the same or other models.
Revised: While most self-supervised learning methods are agnostic to downstream tasks, self-training models learn to handle any downstream task by learning from synthetic data or pseudo-labels that are automatically generated by the same or other models.

Original: Among recent research under the context of self-training, back translation (He et al., 2019) and data augmentation methods (Xie et al., 2020;
Revised: Among recent research under the context of self-training, back translation (He et al., 2019) and data augmentation methods (Xie et al., 2020;

Original: Chen et al., 2020) use synthetic texts as inputs, pseudo labeling methods (Zoph et al., 2020;
Revised: Chen et al., 2020) use synthetic texts as inputs, pseudo labeling methods (Zoph et al., 2020;

Original: Zou et al., 2019) use synthetic labels as training targets, and self-trained question answering methods (Bartolo et al., 2021;
Revised: Zou et al., 2019) use synthetic labels as training targets, and self-trained question answering methods (Bartolo et al., 2021;

Original: Luo et al., 2022) use synthetic questions as inputs and synthetic answers as target outputs.
Revised: Luo et al., 2022) use synthetic questions as inputs and synthetic answers as target outputs.

Original: Recent studies have discussed the efficiency and robustness of language models from different aspects, including different settings on training and evaluation data.
Revised: Recent studies have discussed the efficiency and robustness of language models from different aspects, including different settings on training and evaluation data.

Original: The authors of Zang et al.
Revised: The authors of Zang et al.

Original: Jin et al.
Revised: Jin et al.

Original: ; Bartolo et al.
Revised: ; Bartolo et al.

Original: ; Wang et al.
Revised: ; Wang et al.

Original: (2021) indicated that even state-or-the-art language models make mistakes on adversarial evaluation examples.
Revised: (2021) indicated that even state-or-the-art language models make mistakes on adversarial evaluation examples.

Original: On the other hand, even for language models pretrained on large corpora, their performance on downstream tasks might not be satisfying if not enough task-specific training data is presented (Schick & Sch¨utze, 2021; Le Scao & Rush, 2021).
Revised: On the other hand, even for language models pretrained on large corpora, their performance on downstream tasks might not be satisfying if not enough task-specific training data is presented (Schick & Sch¨utze, 2021; Le Scao & Rush, 2021).

Original: There is little work simultaneously dealing with the limitations of small training sets and missing human-generated labels.
Revised: There is little work simultaneously dealing with the limitations of small training sets and missing human-generated labels.

Original:  B ACKGROUND : G RADUATED N ON -C ONVEXITY
Revised:  B ACKGROUND : G RADUATED N ON -C ONVEXITY

Original: Graduated non-convexity (GNC) is a popular optimization method in vision (Blake & Zisserman, 1987) and machine learning (Rose, 1998; Mobahi & Fisher, 2015).
Revised: Graduated non-convexity (GNC) is a popular optimization method in vision (Blake & Zisserman, 1987) and machine learning (Rose, 1998; Mobahi & Fisher, 2015).

Original: The core idea is optimizing a robust cost function instead of a standard loss function, as shown in Equation 1.
Revised: The core idea is optimizing a robust cost function instead of a standard loss function, as shown in Equation 1.

Original: The robust cost function ρ ( · ) is adjustable give a coefficient µ .
Revised: The robust cost function ρ ( · ) is adjustable give a coefficient µ .

Original: When µ is large enough, ρ ( l ( y i , x i )) is approximately convex.
Revised: When µ is large enough, ρ ( l ( y i , x i )) is approximately convex.

Original: During the optimization, the value of µ is decreased at each step and the optimization prob- lem is gradually recovered to the original non-convex problem.
Revised: During the optimization, the value of µ is decreased at each step and the optimization prob- lem is gradually recovered to the original non-convex problem.

Original: A more convex iteration provides a guess for the following optimization.
Revised: A more convex iteration provides a guess for the following optimization.

Original: A typical robust surrogate cost function is the Geman-McClure (GM) function.
Revised: A typical robust surrogate cost function is the Geman-McClure (GM) function.

Original: The GM function and the controlled GM function are shown as follows, as mentioned in (Yang et al., 2020),
Revised: The GM function and the controlled GM function are shown as follows, as mentioned in (Yang et al., 2020),

Original: [Equation]
Revised: [Equation]

Original: where ¯ c is a parameter that decides the shape of the GM function, and represents the largest cost of a threshold of outlier losses.
Revised: where ¯ c is a parameter that decides the shape of the GM function, and represents the largest cost of a threshold of outlier losses.

Original: According to Equation 2, when µ → ∞ , ρ µ ( l ) → l 2 ; when µ = 1 , ρ µ ( l ) = ρ ( l ) .
Revised: According to Equation 2, when µ → ∞ , ρ µ ( l ) → l 2 ; when µ = 1 , ρ µ ( l ) = ρ ( l ) .

Original: The optimization result of ρ ( l ) is reached by optimizing a high-quality guess of ρ µ ( l ) .
Revised: The optimization result of ρ ( l ) is reached by optimizing a high-quality guess of ρ µ ( l ) .

Original: In practice, following (Yang et al., 2020), the optimization target we apply is a mutation of ρ µ ( l ) based on the Black-Rangarajan Duality (Black & Rangarajan, 1996), which proved that the problem of optimizing ρ µ ( l ) is equivalent to optimizing a weighted sum of l ( · ) and a weight penalty term Φ .
Revised: In practice, following (Yang et al., 2020), the optimization target we apply is a mutation of ρ µ ( l ) based on the Black-Rangarajan Duality (Black & Rangarajan, 1996), which proved that the problem of optimizing ρ µ ( l ) is equivalent to optimizing a weighted sum of l ( · ) and a weight penalty term Φ .

Original: In other words, the optimization problem in Equation 1 has the same optimal solution with the following task,
Revised: In other words, the optimization problem in Equation 1 has the same optimal solution with the following task,

Original: [Equation]
Revised: [Equation]

Original: The set of w = { w 1 , w 2 , . . . , w n } are slack variables to control each loss item
Revised: The set of w = { w 1 , w 2 , . . . , w n } are slack variables to control each loss item

Original: l ( y i , x ) .
Revised: l ( y i , x ) .

Original: In the robotic perception and computer vision areas, y i stands for a noisy observation of the i -th sensor and x stands for the ground-truth location or shape of the physical object of interest.
Revised: In the robotic perception and computer vision areas, y i stands for a noisy observation of the i -th sensor and x stands for the ground-truth location or shape of the physical object of interest.

Original: According to
Revised: According to

Original: Black & Rangarajan (1996), when the GM function is applied as the robust cost function, the weight penalty function Φ( w i ) can be written as
Revised: Black & Rangarajan (1996), when the GM function is applied as the robust cost function, the weight penalty function Φ( w i ) can be written as

Original: [Equation]
Revised: [Equation]

Original: In this work, we jointly optimize the weighted sum of the standard loss l ( y i , x ) and the weight penalty loss Φ( w i ) .
Revised: In this work, we jointly optimize the weighted sum of the standard loss l ( y i , x ) and the weight penalty loss Φ( w i ) .

Original:  M ETHOD : GNC S ELF - TRAINING WITH S HIFTED G EMAN M C C LURE 4. E NTAILMENT P RETRAINING AND S UPPOSITION V ERIFICATION
Revised:  M ETHOD : GNC S ELF - TRAINING WITH S HIFTED G EMAN M C C LURE 4. E NTAILMENT P RETRAINING AND S UPPOSITION V ERIFICATION

Original: In this work, we investigate weakly supervised domain and task adaption of textual entailment models, where the size of training texts and the access to human-generated labels are limited.
Revised: In this work, we investigate weakly supervised domain and task adaption of textual entailment models, where the size of training texts and the access to human-generated labels are limited.

Original: To enable the adaptation, we formulate natural language understanding tasks to classifications tasks where a model just needs to predict the truth value of input suppositions.
Revised: To enable the adaptation, we formulate natural language understanding tasks to classifications tasks where a model just needs to predict the truth value of input suppositions.

Original: We first pretrain entailment classifiers on MultiNLI (Williams et al., 2018) that learns to predict the truth values of the following suppositions based on a hypothesis { x } and the premise { y } ,
Revised: We first pretrain entailment classifiers on MultiNLI (Williams et al., 2018) that learns to predict the truth values of the following suppositions based on a hypothesis { x } and the premise { y } ,

Original: s 1 mnli : { x } is entailed by { y } ; s 2 mnli : { x } is  true when { y } is true.
Revised: s 1 mnli : { x } is entailed by { y } ; s 2 mnli : { x } is not true when { y } is true.

Original: According to the definition of entailment, s 1 p and s 2 p are contradictory, thus they must have different truth values.
Revised: According to the definition of entailment, s 1 p and s 2 p are contradictory, thus they cannot both be true of false.

Original: A model trained to predict the truth value suppositions constructed by each hypothesispremise pair ( h i , p i ) can be applied to other tasks, if similar suppositions can be constructed.
Revised: A model trained to predict the truth value suppositions constructed by each hypothesispremise pair ( h i , p i ) can be applied to other tasks, if similar suppositions can be constructed.

Original: For example, an entailment classifier can be applied to handle the movie review classification task (SST2) (Socher et al., 2013).
Revised: For example, an entailment classifier can be applied to handle the movie review classification task (SST2) (Socher et al., 2013).

Original: A sentiment analysis task can also be formulated as an entailment prediction or supposition verification task, with the following potential suppositions, s 1 sst 2 : I like the movie is entailed by my comment { x } ; s 2 sst 2 : The movie is good because { x } .
Revised: A sentiment analysis task can also be formulated as an entailment prediction or supposition verification task, with the following potential suppositions, s 1 sst 2 : I like the movie is entailed by my comment { x } ; s 2 sst 2 : The movie is good because { x } .

Original: We construct the suppositions following Anonymous (2022) for different tasks, including SST2,
Revised: We construct the suppositions following Anonymous (2022) for different tasks, including SST2,

Original: RTE (Dagan et al., 2005), QNLI (Rajpurkar et al., 2016), QQP (Wang et al., 2018), and their adversarial evaluation corpora (Wang et al., 2021).
Revised: RTE (Dagan et al., 2005), QNLI (Rajpurkar et al., 2016), QQP (Wang et al., 2018), and their adversarial evaluation corpora (Wang et al., 2021).

Original: 4. S ELF -T RAINING OF THE E NTAILMENT C LASSIFIER
Revised: 4. S ELF -T RAINING OF THE E NTAILMENT C LASSIFIER

Original: In this work, we investigate learning task-specific models with a pretrained entailment classifier with a limited amount of unlabeled training data D task = { x 1 , x 2 , . . .
Revised: In this work, we investigate learning task-specific models with a pretrained entailment classifier with a limited amount of unlabeled training data D task = { x 1 , x 2 , . . .

Original: , x K } , where x i stands for an unlabeled data example.
Revised: , x K } , where x i stands for an unlabeled data example.

Original: Given a pretrained entailment model M , We generated pseudo-labels for each data example by feeding suppositions constructed based on x i ,
Revised: Given a pretrained entailment model M , We generated pseudo-labels for each data example by feeding suppositions constructed based on x i ,

Original: [Equation]
Revised: [Equation]

Original: The pseudo-labels are then used to train task-specific models.
Revised: The pseudo-labels are then used to train task-specific models.

Original: A new training set is constructed as D train =
Revised: A new training set is constructed as D train =

Original: { ( x 1 , ˆ y 1 ) , ( x 1 , ˆ y 2 ) , . . .
Revised: { ( x 1 , ˆ y 1 ) , ( x 1 , ˆ y 2 ) , . . .

Original: , ( x K , ˆ y K ) } .
Revised: , ( x K , ˆ y K ) } .

Original: The new training set is used to fine-tune the pretrained entailment classifier.
Revised: The new training set is used to fine-tune the pretrained entailment classifier.

Original: Note that the pretrained entailment model has three classes (entailment, neutral, and contradiction), we just use the entailment and contradiction scores for the fine-tuning and evaluation of downstream tasks.
Revised: Note that the pretrained entailment model has three classes (entailment, neutral, and contradiction), we just use the entailment and contradiction scores for the fine-tuning and evaluation of downstream tasks.

Original: 4. C ONVEXITY C ONTROLLED T RAINING
Revised: 4. C ONVEXITY C ONTROLLED T RAINING

Original: The Black-Rangarajan Duality (Black & Rangarajan, 1996) suggests that optimizing the surrogate robust cost function (cid:80) i ρ ( l ( y i , x )) is equivalent to optimizing the following cost function
Revised: The Black-Rangarajan Duality (Black & Rangarajan, 1996) suggests that optimizing the surrogate robust cost function (cid:80) i ρ ( l ( y i , x )) is equivalent to optimizing the following cost function

Original: [Equation]
Revised: [Equation]

Original: In this work, we apply the Geman McClure (GM) function as the robust cost ρ ( · ) .
Revised: In this work, we apply the Geman McClure (GM) function as the robust cost ρ ( · ) .

Original: Before the beginning of training, all slack variables w i are set to 1 .
Revised: Before the beginning of training, all slack variables w i are set to 1 .

Original: We use the the alternating optimization strategy proposed in Yang et al.
Revised: We use the the alternating optimization strategy proposed in Yang et al.

Original: At each optimization step t , we first update the model parameters θ based on the weighted sum of base costs,
Revised: At each optimization step t , we first update the model parameters θ based on the weighted sum of base costs,

Original: [Equation]
Revised: [Equation]

Original: then we fix the learned θ and update the slack variables W =
Revised: then we fix the learned θ and update the slack variables W =

Original: [ w 1 , w 2 , . .
Revised: [ w 1 , w 2 , . .

Original: . , w n ] , Note that in Yang et al.
Revised: . , w n ] , Note that in Yang et al.

Original: (2020) and in related work in visual perception, n stands for the number of sensor observations, but in our work n stands for the number of training examples.
Revised: (2020) and in related work in visual perception, n stands for the number of sensor observations, but in our work n stands for the number of training examples.

Original: At each time step, w i is updated as
Revised: At each time step, w i is updated as

Original: [Equation]
Revised: [Equation]

Original: This analytic solution is obtained by solving ∂L ρ ∂w i = 0 when θ t is fixed.
Revised: This analytic solution is obtained by solving ∂L ρ ∂w i = 0 when θ t is fixed.

Original: The output w ti is used to calculate θ t +1 in the next episode.
Revised: The output w ti is used to calculate θ t +1 in the next episode.

Original: At the first training step, µ is initialized as µ = 2 l 2 max ( y i , x i ; θ 0 ) / ¯ c 2 .
Revised: At the first training step, µ is initialized as µ = 2 l 2 max ( y i , x i ; θ 0 ) / ¯ c 2 .

Original: In the following training steps, we decrease it by µ = µ/ 2 . 8 .
Revised: In the following training steps, we decrease it by µ = µ/ 2 . 8 .

Original: The training stops when µ < 1 .
Revised: The training stops when µ < 1 .

Original: It is straightforward to see that an observation, or training example ( y i , x i ) receives a low weight w i when its corresponding loss l ( y i , x i ; θ ) is high.
Revised: It is straightforward to see that an observation, or training example ( y i , x i ) receives a low weight w i when its corresponding loss l ( y i , x i ; θ ) is high.

Original: In visual perception, this feature helps the algorithm to find outlier sensors.
Revised: In visual perception, this feature helps the algorithm to find outlier sensors.

Original: In these tasks, excluding outliers benefits the accuracy of the predictions because 1) the perception functions of the sensors are usually known, and 2) accurate object predictions must lead to low cost if the observation is correct.
Revised: In these tasks, excluding outliers benefits the accuracy of the predictions because 1) the perception functions of the sensors are usually known, and 2) accurate object predictions must lead to low cost if the observation is correct.

Original: The situation is different under the context of training neural language models.
Revised: The situation is different under the context of training neural language models.

Original: In neural language model training, the models are presented with ( y i , x i ) pairs instead of ( y i , θ ) , so there is no guarantee if the model tuning step in Equation 7 could find a global optimal solution of θ under the current setting of w or not.
Revised: In neural language model training, the models are presented with ( y i , x i ) pairs instead of ( y i , θ ) , so there is no guarantee if the model tuning step in Equation 7 could find a global optimal solution of θ under the current setting of w or not.

Original: On the other hand, a high-loss example in machine learning is not necessarily an outlier - it might bring new knowledge that improves the model.
Revised: On the other hand, a high-loss example in machine learning is not necessarily an outlier - it might bring new knowledge that improves the model.

Original: Another difficulty is that an outlier might also improve the model performance and robustness.
Revised: Another difficulty is that an outlier might also improve the model performance and robustness.

Original: Under the self-training setting, a low cost might be associated with either a correct pseudo-label or a wrong pseudo-label that the model is over-confident about.
Revised: Under the self-training setting, a low cost might be associated with either a correct pseudo-label or a wrong pseudo-label that the model is over-confident about.

Original: These difficulties prevent the direct application of GNC on machine learning, especially self-training of neural language models.
Revised: These difficulties prevent the direct application of GNC on machine learning, especially self-training of neural language models.

Original: To mitigate the problem, we introduce the shifted Geman McClure (SGM) function for graduated non-convexity optimization.
Revised: To mitigate the problem, we introduce the shifted Geman McClure (SGM) function for graduated non-convexity optimization.

Original: Besides the parameter ¯ c that controls the maximum cost of non-outlier examples, we set another parameter ϵ that bars the minimal normal cost of non-outlier examples.
Revised: Besides the parameter ¯ c that controls the maximum cost of non-outlier examples, we set another parameter ϵ that bars the minimal normal cost of non-outlier examples.

Original: Under this setting, the outlier set consists of two parts, including high-loss examples associated with wrong pseudo-labels, and over-confident examples whose loss is too low that might lead to overfitting of the trained model.
Revised: Under this setting, the outlier set consists of two parts, including high-loss examples associated with wrong pseudo-labels, and over-confident examples whose loss is too low that might lead to overfitting of the trained model.

Original: The SGM function applied as our robust cost is
Revised: The SGM function applied as our robust cost is

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: When l < ϵ , the robust cost ρ increases if l continues to decrease.
Revised: When l < ϵ , the robust cost ρ increases if l continues to decrease.

Original: We use this method to prevent overfitting caused by over-confident examples.
Revised: We use this method to prevent overfitting caused by over-confident examples.

Original: The shapes of GM and SGM functions are shown in
Revised: The shapes of GM and SGM functions are shown in

Original: Figure 1.
Revised: Figure 1.

Original: Higher µ values result in higher curves.
Revised: Higher µ values result in higher curves.

Original:  E XPERIMENTS 5. T ASK AND D ATA
Revised:  E XPERIMENTS 5. T ASK AND D ATA

Original: MultiNLI.
Revised: MultiNLI.

Original: We use the multi-genre natural language inference (MultiNLI)
Revised: We use the multi-genre natural language inference (MultiNLI)

Original: (Williams et al., 2018) corpus for pretraining entailment classifiers.
Revised: (Williams et al., 2018) corpus for pretraining entailment classifiers.

Original: The MultiNLI corpus contains 393k sentence pairs that can be classified into 3 categories: entailed, neutral, and contradictory.
Revised: The MultiNLI corpus contains 393k sentence pairs that can be classified into 3 categories: entailed, neutral, and contradictory.

Original: We train a classifier to predict the truth value of the constructed suppositions as mentioned in Section 4.1.
Revised: We train a classifier to predict the truth value of the constructed suppositions as mentioned in Section 4.1.

Original: QNLI/AdvQNLI.
Revised: QNLI/AdvQNLI.

Original: The question natural language inference (QNLI) corpus contains questioncontext pairs.
Revised: The question natural language inference (QNLI) corpus contains questioncontext pairs.

Original: The task is to predict if the answer to a given question is contained in its paired passage.
Revised: The task is to predict if the answer to a given question is contained in its paired passage.

Original: We construct the following supposition for the task, s qnli : The answer to question { q } is entailed by { p } .
Revised: We construct the following supposition for the task, s qnli : The answer to question { q } is entailed by { p } .

Original: QQP/AdvQQP.
Revised: QQP/AdvQQP.

Original: The Quora question pairs (QQP) task is proposed in Wang et al.
Revised: The Quora question pairs (QQP) task is proposed in Wang et al.

Original: In the
Revised: In the

Original: QQP task, a model learns to predict if two questions are duplicated.
Revised: QQP task, a model learns to predict if two questions are duplicated.

Original: In most cases, the task is to predict if two questions can be answer by the same answer.
Revised: In most cases, the task is to predict if two questions can be answer by the same answer.

Original: The adaptation from MultiNLI to QQP is a cross-task adaptation, and the supposition for the QQP task is s qqp : The answer to { q 1 } is entailed by the answer to { q 2 } .
Revised: The adaptation from MultiNLI to QQP is a cross-task adaptation, and the supposition for the QQP task is s qqp : The answer to { q 1 } is entailed by the answer to { q 2 } .

Original: RTE/AdvRTE.
Revised: RTE/AdvRTE.

Original: The recognizing textual entailment (RTE) task is a binary (entailment/contradiction) entailment classification task based in the news domain.
Revised: The recognizing textual entailment (RTE) task is a binary (entailment/contradiction) entailment classification task based in the news domain.

Original: Each data example is a claim-passage pair and the task is predicting if the claim is entailed by the paired passage or not.
Revised: Each data example is a claim-passage pair and the task is predicting if the claim is entailed by the paired passage or not.

Original: The adaptation from MultiNLI to RTE is cross-domain adaptation and the suppositions are the same as MultiNLI suppositions, as defined in Section 4.1.
Revised: The adaptation from MultiNLI to RTE is cross-domain adaptation and the suppositions are the same as MultiNLI suppositions, as defined in Section 4.1.

Original: SST2/AdvSST2.
Revised: SST2/AdvSST2.

Original: The Stanford Sentiment Treebank (SST-2) is a sentimental analysis corpus that requires a model to predict if the attitude of a movie review is positive or negative.
Revised: The Stanford Sentiment Treebank (SST-2) is a sentimental analysis corpus that requires a model to predict if the attitude of a movie review is positive or negative.

Original: The adaptation from MultiNLI to SST-2 is a cross-task adaptation.
Revised: The adaptation from MultiNLI to SST-2 is a cross-task adaptation.

Original: The construct supposition template is s sst 2 : The movie is good is entailed by the comment { x } .
Revised: The construct supposition template is s sst 2 : The movie is good is entailed by the comment { x } .

Original: 5. I MPLEMENTATION D ETAILS
Revised: 5. I MPLEMENTATION D ETAILS

Original: In this work, we train BERT (Devlin et al., 2018) and DeBERTa He et al.
Revised: In this work, we train BERT (Devlin et al., 2018) and DeBERTa He et al.

Original: (2020) models as the pretrained entailment classifiers, which are trained on the MultiNLI corpus Williams et al.
Revised: (2020) models as the pretrained entailment classifiers, which are trained on the MultiNLI corpus Williams et al.

Original: In GNC self-training, there are 2 key hyper-parameters: the threshold of outlier losses ¯ c and the amount of shift in the SGM function ϵ .
Revised: In GNC self-training, there are 2 key hyper-parameters: the threshold of outlier losses ¯ c and the amount of shift in the SGM function ϵ .

Original: For each task, we select a value from { 1 e − 2 , 1 e − 3 , 1 e − 4 } for ¯ c .
Revised: For each task, we select a value from { 1 e − 2 , 1 e − 3 , 1 e − 4 } for ¯ c .

Original: To find an appropriate ϵ , we calculate the losses l ∈ R n on all training samples after each training epoch and set ϵ = µ ( l ) − α ( σ ( l ) / √ n ) , α ∈ [1 . 2 , 2 . 8] .
Revised: To find an appropriate ϵ , we calculate the losses l ∈ R n on all training samples after each training epoch and set ϵ = µ ( l ) − α ( σ ( l ) / √ n ) , α ∈ [1 . 2 , 2 . 8] .

Original: More details are in Appendix B.
Revised: More details are in Appendix B.

Original: 5. E XPERIMENTAL R ESULTS
Revised: 5. E XPERIMENTAL R ESULTS

Original: In this work, we investigated different experimental settings, including few-shot self-training and the combination of few-shot fine-tuning and self-training.
Revised: In this work, we investigated different experimental settings, including few-shot self-training and the combination of few-shot fine-tuning and self-training.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: -shot self-training.
Revised: -shot self-training.

Original: Under the few-shot self-training setting, a pretrained entailment classifier can only access a limited number ( K = 12 or 132 ) of unlabeled data examples.
Revised: Under the few-shot self-training setting, a pretrained entailment classifier can only access a limited number ( K = 12 or 132 ) of unlabeled data examples.

Original: The pretrained models generate pseudo labels for fine-tuning.
Revised: The pretrained models generate pseudo labels for fine-tuning.

Original: We evaluate pretrained DeBERTa and RoBERTa entailment classifiers on both regular and adversarial evaluation sets.
Revised: We evaluate pretrained DeBERTa and RoBERTa entailment classifiers on both regular and adversarial evaluation sets.

Original: We conduct the evaluation with 20 independent experiments and report the average performance.
Revised: We conduct the evaluation with 20 independent experiments and report the average performance.

Original: Combined Few-shot fine-tuning and self-training.
Revised: Combined Few-shot fine-tuning and self-training.

Original: In many situations, the model can access a limited number of training examples with human-generated labels and a larger number of unlabeled data examples.
Revised: In many situations, the model can access a limited number of training examples with human-generated labels and a larger number of unlabeled data examples.

Original: In this case, we first fine-tune the pretrained models with the labeled data, and self-train the fine-tuned model using unlabeled data with generated pseudo-labels.
Revised: In this case, we first fine-tune the pretrained models with the labeled data, and self-train the fine-tuned model using unlabeled data with generated pseudo-labels.

Original: We evaluate the regular self-training strategy with GNC self-training with few-shot fine-tuned models.
Revised: We evaluate the regular self-training strategy with GNC self-training with few-shot fine-tuned models.

Original: In each experiment, we use 12 human-labeled training examples and 132 unlabeled data examples.
Revised: In each experiment, we use 12 human-labeled training examples and 132 unlabeled data examples.

Original: We compare the performance of the proposed methods with strong few-shot and fully supervised baselines.
Revised: We compare the performance of the proposed methods with strong few-shot and fully supervised baselines.

Original: While the baseline models access task-specific human-generated labels, our pure selftraining method only process unlabeled data examples from each task.
Revised: While the baseline models access task-specific human-generated labels, our pure selftraining method only process unlabeled data examples from each task.

Original: The few-shot learning baselines include PET Schick & Sch¨utze (2021), LM-BFF Gao et al.
Revised: The few-shot learning baselines include PET Schick & Sch¨utze (2021), LM-BFF Gao et al.

Original: (2020), P-tuning Liu et al.
Revised: (2020), P-tuning Liu et al.

Original: PPT Gu et al.
Revised: PPT Gu et al.

Original: (2021), and UPT Wang et al.
Revised: (2021), and UPT Wang et al.

Original: (2022), which are medium-sized models with around 350M parameters trained with 16 human-labeled examples.
Revised: (2022), which are medium-sized models with around 350M parameters trained with 16 human-labeled examples.

Original: FLAN Wei et al.
Revised: FLAN Wei et al.

Original: (2021) and LaMDA
Revised: (2021) and LaMDA

Original: Thoppilan et al.
Revised: Thoppilan et al.

Original: (2022) are large-scale language models with 137B parameters and are tuned with up to 12 data examples.
Revised: (2022) are large-scale language models with 137B parameters and are tuned with up to 12 data examples.

Original: The robust baselines R3F Aghajanyan et al.
Revised: The robust baselines R3F Aghajanyan et al.

Original: CT T Xu et al.
Revised: CT T Xu et al.

Original: (2021), and MT Tong et al.
Revised: (2021), and MT Tong et al.

Original: (2022) are trained on the full training corpora.
Revised: (2022) are trained on the full training corpora.

Original: The experiment results are shown in Table 1.
Revised: The experiment results are shown in Table 1.

Original: We found that the self-training methods receive significant improvement on adversarial evaluation data, while matching or outperforming the pretrained entailment classifiers on regular evaluation sets.
Revised: We found that the self-training methods receive significant improvement on adversarial evaluation data, while matching or outperforming the pretrained entailment classifiers on regular evaluation sets.

Original: We cite the performance of baseline methods reported by the authors, and summarize the self-training performance by averaging the results ofindependent experiments.
Revised: We cite the performance of baseline methods reported by the authors, and summarize the self-training performance by averaging the results ofindependent experiments.

Original: We found the self-training methods that are tuned with 12 unlabeled data examples outperform the 16-shot and fully-supervised baselines in most experiments.
Revised: We found the self-training methods that are tuned with 12 unlabeled data examples outperform the 16-shot and fully-supervised baselines in most experiments.

Original: Comparing
Revised: Comparing

Original: [Figure]
Revised: [Figure]

Original: the best few-shot and fully-supervised results of medium language models, our GNC self-training method improved 18.8%/7.4% on QNLI/AdvQNLI and 7.7%/8.2% on QQP/AdvQQP.
Revised: the best few-shot and fully-supervised results of medium language models, our GNC self-training method improved 18.8%/7.4% on QNLI/AdvQNLI and 7.7%/8.2% on QQP/AdvQQP.

Original: Although GNC constantly outperforms the regular self-training method, its performance is significantly lower than fully-supervised method on AdvRTE and AdvSST2.
Revised: Although GNC constantly outperforms the regular self-training method, its performance is significantly lower than fully-supervised method on AdvRTE and AdvSST2.

Original: With 12 labeled andunlabeled data examples, the GNC self-training method achieved the highest RTE and AdvSST2 performance in addition, and the accuracy on AdvRTE and AdvSST2 is boosted to the fully-supervised level (78.1% vs. 79.0%, 57.8% vs. 57.8%).
Revised: With 12 labeled andunlabeled data examples, the GNC self-training method achieved the highest RTE and AdvSST2 performance in addition, and the accuracy on AdvRTE and AdvSST2 is boosted to the fully-supervised level (78.1% vs. 79.0%, 57.8% vs. 57.8%).

Original: In the self-training experiments, we use all unlabeled training data and pseudo labels.
Revised: In the self-training experiments, we use all unlabeled training data and pseudo labels.

Original: We will analyze the effect of data selection in the next section.
Revised: We will analyze the effect of data selection in the next section.

Original:  A NALYSIS AND A BLATION S TUDY
Revised:  A NALYSIS AND A BLATION S TUDY

Original: Stability and robustness of self-training.
Revised: Stability and robustness of self-training.

Original: We characterize the stability of the model performance from different perspectives, including the mean value, standard deviation, and minimum/maximum accuracy with different pretrained models and training strategy.
Revised: We characterize the stability of the model performance from different perspectives, including the mean value, standard deviation, and minimum/maximum accuracy with different pretrained models and training strategy.

Original: To assess the effect of the proposed method, we propose the following metrics,
Revised: To assess the effect of the proposed method, we propose the following metrics,

Original: • A strictly more stable model achieves higher mean and minimum/maximum accuracy, with lower standard deviation.
Revised: • A strictly more stable model achieves higher mean and minimum/maximum accuracy, with lower standard deviation.

Original: • A softly more stable model achieves higher mean and minimum accuracy.
Revised: • A softly more stable model achieves higher mean and minimum accuracy.

Original: • A less risky model achieves higher minimum accuracy and lower standard deviation.
Revised: • A less risky model achieves higher minimum accuracy and lower standard deviation.

Original: Under the few-shot self-training setting, a pretrained entailment classifier can only access a limited number ( K = 12 ) of unlabeled data examples.
Revised: Under the few-shot self-training setting, a pretrained entailment classifier can only access a limited number ( K = 12 ) of unlabeled data examples.

Original: The pretrained models generate pseudo labels for fine-tuning.
Revised: The pretrained models generate pseudo labels for fine-tuning.

Original: We evaluate pretrained DeBERTa and RoBERTa entailment classifiers with the confidence regularization (CR) method proposed in Zou et al.
Revised: We evaluate pretrained DeBERTa and RoBERTa entailment classifiers with the confidence regularization (CR) method proposed in Zou et al.

Original: on both regular and adversarial evaluation sets.
Revised: on both regular and adversarial evaluation sets.

Original: We conduct the evaluation with 20 independent experiments and report the mean, minimum, maximum performance, and standard deviation in Figure 2.
Revised: We conduct the evaluation with 20 independent experiments and report the mean, minimum, maximum performance, and standard deviation in Figure 2.

Original: The experimental results show the effect of GNC in self-training.
Revised: The experimental results show the effect of GNC in self-training.

Original: In all 48 comparisons in Figure 2, the GNC method is strictly more stable than the baselines in 23 cases, where all metrics (mean, std, min, max) indicate that GNC perform better.
Revised: In all 48 comparisons in Figure 2, the GNC method is strictly more stable than the baselines in 23 cases, where all metrics (mean, std, min, max) indicate that GNC perform better.

Original: There are 35 ( 72 . 9% ) cases where GNC is softly more stable or less-risky than its baselines.
Revised: There are 35 ( 72 . 9% ) cases where GNC is softly more stable or less-risky than its baselines.

Original: There are only 2 cases where the baselines are strictly better than GNC (AdvSST2, K = 12 , 132 , RoBERTa) and another 4 cases where the baselines are softly better than GNC (AdvRTE, K = 12 ; RTE, K = 132 ).
Revised: There are only 2 cases where the baselines are strictly better than GNC (AdvSST2, K = 12 , 132 , RoBERTa) and another 4 cases where the baselines are softly better than GNC (AdvRTE, K = 12 ; RTE, K = 132 ).

Original: [Table] [Table]
Revised: [Table] [Table]

Original: We compare the model performance across methods and tasks.
Revised: We compare the model performance across methods and tasks.

Original: We first note that in 15 out oftasks, the performance of DeBERTa+GNC is at least softly more stable than DeBERTa+CR.
Revised: We first note that in 15 out oftasks, the performance of DeBERTa+GNC is at least softly more stable than DeBERTa+CR.

Original: The only exception is the AdvSST2 task when K = 12 , where DeBERTa+GNC is less risky than CR as defined.
Revised: The only exception is the AdvSST2 task when K = 12 , where DeBERTa+GNC is less risky than CR as defined.

Original: We also found that the DeBERTa-based models generally outperform RoBERTa, except for adversarial QNLI and SST2 evaluations.
Revised: We also found that the DeBERTa-based models generally outperform RoBERTa, except for adversarial QNLI and SST2 evaluations.

Original: We also compare the model performance under different model and training data settings.
Revised: We also compare the model performance under different model and training data settings.

Original: Table 2 indicates that the GNC strategy is more effective on
Revised: Table 2 indicates that the GNC strategy is more effective on

Original: DeBERTa and the improvement is most obvious when K = 12 .
Revised: DeBERTa and the improvement is most obvious when K = 12 .

Original: This proves that when the size of training data is limited, self-training is likely to be biased and the performance would be unstable.
Revised: This proves that when the size of training data is limited, self-training is likely to be biased and the performance would be unstable.

Original: Note that each experiment stands for the summarized result of 20 independent runs.
Revised: Note that each experiment stands for the summarized result of 20 independent runs.

Original: The performance analysis across different tasks is presented in Table 3, summarizing all model and training data settings.
Revised: The performance analysis across different tasks is presented in Table 3, summarizing all model and training data settings.

Original: We found that the QQP and AdvQQP tasks benefits most from the GNC strategy, while the RTE task does not.
Revised: We found that the QQP and AdvQQP tasks benefits most from the GNC strategy, while the RTE task does not.

Original: We believe the reason is that the RTE task is also an entailment classification task, so it is not difficult for the baseline self-training method to adapt.
Revised: We believe the reason is that the RTE task is also an entailment classification task, so it is not difficult for the baseline self-training method to adapt.

Original: Few-shot + self-training performance.
Revised: Few-shot + self-training performance.

Original: We also analyze the experimental results under the combined few-shot supervised and self-training settings on different tasks, where we first fine-tune the pretrained entailment classifiers with 12 labeled data examples and then conduct self-training with 132 unlabeled examples.
Revised: We also analyze the experimental results under the combined few-shot supervised and self-training settings on different tasks, where we first fine-tune the pretrained entailment classifiers with 12 labeled data examples and then conduct self-training with 132 unlabeled examples.

Original: The results of RoBERTa and DeBERTa models are visualized in Figure 3.
Revised: The results of RoBERTa and DeBERTa models are visualized in Figure 3.

Original: Out of 8 tasks, DeBERTa+GNC is the most stable strategy in general, while RoBERTa+GNC achieves the best performance on the AdvSST2 task.
Revised: Out of 8 tasks, DeBERTa+GNC is the most stable strategy in general, while RoBERTa+GNC achieves the best performance on the AdvSST2 task.

Original: The results indicate that besides the pretrained entailment classifier, the GNC method is also robust for the self-training of few-shot supervised models.
Revised: The results indicate that besides the pretrained entailment classifier, the GNC method is also robust for the self-training of few-shot supervised models.

Original: Geman McClure (GM) and Shifted Geman McClure (SGM) functions.
Revised: Geman McClure (GM) and Shifted Geman McClure (SGM) functions.

Original: We analyze the performance of using GM and SGM as the robust cost function on QNLI and SST2 tasks.
Revised: We analyze the performance of using GM and SGM as the robust cost function on QNLI and SST2 tasks.

Original: [Figure]
Revised: [Figure]

Original: Pseudo label bias stands for the percentage of the pseudo label in all selected data examples.
Revised: Pseudo label bias stands for the percentage of the pseudo label in all selected data examples.

Original: The experimental results are shown in Table 4.
Revised: The experimental results are shown in Table 4.

Original: The results indicate that shifting the optimum point of the GM function can lead to more stable performance, by comparing the mean and minimum performance on QNLI and SST2 tasks.
Revised: The results indicate that shifting the optimum point of the GM function can lead to more stable performance, by comparing the mean and minimum performance on QNLI and SST2 tasks.

Original: However, the variance of GM performance is larger and it achieves higher maximum performance on the adversarial evaluations.
Revised: However, the variance of GM performance is larger and it achieves higher maximum performance on the adversarial evaluations.

Original: Despite this fact, we can still conclude that the SGM cost function introduces more stable and robust self-training performance.
Revised: Despite this fact, we can still conclude that the SGM cost function introduces more stable and robust self-training performance.

Original: [Table]
Revised: [Table]

Original: What Makes Self-training Unstable and How Does
Revised: What Makes Self-training Unstable and How Does

Original: GNC Help?
Revised: GNC Help?

Original: We randomly selected 20 different training sets of the QNLI task, and then conduct the baseline and GNC self-training on each training set.
Revised: We randomly selected 20 different training sets of the QNLI task, and then conduct the baseline and GNC self-training on each training set.

Original: We plot the relation of self-training performance and related factors as shown in Figure 4 with data pairs and linear regressions.
Revised: We plot the relation of self-training performance and related factors as shown in Figure 4 with data pairs and linear regressions.

Original: From the upper plots, we find that the performance is roughly related to the pseudo-label accuracy and the pseudo-labeling perplexity.
Revised: From the upper plots, we find that the performance is roughly related to the pseudo-label accuracy and the pseudo-labeling perplexity.

Original: Figure 4.a indicates that the confidence of the pseudo-labels affects the selftraining performance, and GNC is more robust against the change.
Revised: Figure 4.a indicates that the confidence of the pseudo-labels affects the selftraining performance, and GNC is more robust against the change.

Original: Figure 4.b
Revised: Figure 4.b

Original: and c suggest that the accuracy of pseudo-labeling has more impact.
Revised: and c suggest that the accuracy of pseudo-labeling has more impact.

Original: When the accuracy labeling accuracy is high ( ∼ 90% ), the baseline and GNC self-training methods achieve similarly high performance.
Revised: When the accuracy labeling accuracy is high ( ∼ 90% ), the baseline and GNC self-training methods achieve similarly high performance.

Original: However, when the pseudo-labeling accuracy is low, the performance of the baseline self-training method significantly drops, while the GNC self-training performance is still robust even when the labeling accuracy is lower than 50% .
Revised: However, when the pseudo-labeling accuracy is low, the performance of the baseline self-training method significantly drops, while the GNC self-training performance is still robust even when the labeling accuracy is lower than 50% .

Original: In Figure 4.a, b, and c we plot the loss weights w i in the first self-training epoch.
Revised: In Figure 4.a, b, and c we plot the loss weights w i in the first self-training epoch.

Original: We find that GNC tends to highlight low-confident data to avoid overfitting as shown in Figure4.d, while it does not highlight all data examples with correct pseudo labels (Figure 4.e).
Revised: We find that GNC tends to highlight low-confident data to avoid overfitting as shown in Figure4.d, while it does not highlight all data examples with correct pseudo labels (Figure 4.e).

Original: Figure 4.f indicates that when the pseudo label set of the randomly selected data is imbalanced, the data examples of the majority pseudo labels receives lower loss weights.
Revised: Figure 4.f indicates that when the pseudo label set of the randomly selected data is imbalanced, the data examples of the majority pseudo labels receives lower loss weights.

Original: This can mitigate the bias during self-training.
Revised: This can mitigate the bias during self-training.

Original: Additional analysis on data selection and adversarial evaluation data is presented in Appendix C.
Revised: Additional analysis on data selection and adversarial evaluation data is presented in Appendix C.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We apply graduated non-convexity in robust self-training for natural language understanding.
Revised: We apply graduated non-convexity in robust self-training for natural language understanding.

Original: With the Black-Rangarajan Duality, we transfer the optimization of a shifted robust cost function to optimizing a weight loss function, where the loss weights are learned to improve the stability and robustness of self-trained language understanding models.
Revised: With the Black-Rangarajan Duality, we transfer the optimization of a shifted robust cost function to optimizing a weight loss function, where the loss weights are learned to improve the stability and robustness of self-trained language understanding models.

Original: Experimental results summarized from 20 independent runs show that the GNC method improves the performance, stability, and robustness of self-training on 4 tasks and corresponding regular and adversarial evaluation corpora.
Revised: Experimental results summarized from 20 independent runs show that the GNC method improves the performance, stability, and robustness of self-training on 4 tasks and corresponding regular and adversarial evaluation corpora.

Original: Our labelfree method significantly outperforms strong label-dependent baselines.
Revised: Our labelfree method significantly outperforms strong label-dependent baselines.

Original: Through the analysis and visualization, we found the GNC self-training strategy is more robust against wrong pseudo-labels, imbalanced training data, overfitting, and adversarial evaluation data.
Revised: Through the analysis and visualization, we found the GNC self-training strategy is more robust against wrong pseudo-labels, imbalanced training data, overfitting, and adversarial evaluation data.

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: In this work we propose a self-training method for language understanding.
Revised: In this work we propose a self-training method for language understanding.

Original: Traditionally, training a language understanding model needs human annotation on large-scale corpora that contributes to a number of annotation jobs.
Revised: Traditionally, training a language understanding model needs human annotation on large-scale corpora that contributes to a number of annotation jobs.

Original: Our method requires much less human-generated labels, leading to the risk of reducing the number of annotation jobs and pays.
Revised: Our method requires much less human-generated labels, leading to the risk of reducing the number of annotation jobs and pays.

Original: On the other hand, since the model learns from its own outputs, it is still possible for the model to generate risky predictions, although the method proposed in this manuscript helps mitigate the risk.
Revised: On the other hand, since the model learns from its own outputs, it is still possible for the model to generate risky predictions, although the method proposed in this manuscript helps mitigate the risk.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: Data.
Revised: Data.

Original: We introduce the tasks and corpora we used for training and evaluation in Section 5.1 and
Revised: We introduce the tasks and corpora we used for training and evaluation in Section 5.1 and

Original: Appendix A. We mentioned how we use the data in Section 5.3.
Revised: Appendix A. We mentioned how we use the data in Section 5.3.

Original: Method.
Revised: Method.

Original: We introduce the background of the method in Section 3, the details of our method in
Revised: We introduce the background of the method in Section 3, the details of our method in

Original: Section 4, and present the pseudo-code of our algorithm in Appendix B.2.
Revised: Section 4, and present the pseudo-code of our algorithm in Appendix B.2.

Original: Hyper-parameter.
Revised: Hyper-parameter.

Original: We describe the key hyper-parameters of GNC self-training in Section 5.2, and list other regular hyper-parameters in Appendix B.1.
Revised: We describe the key hyper-parameters of GNC self-training in Section 5.2, and list other regular hyper-parameters in Appendix B.1.

Original: Experiments.
Revised: Experiments.

Original: We describe the experiment results, number of independent runs, and visualized analysis in Section 5.3, 6, and Appendix C to prove the statistical significance.
Revised: We describe the experiment results, number of independent runs, and visualized analysis in Section 5.3, 6, and Appendix C to prove the statistical significance.

Original: Source code.
Revised: Source code.

Original: We will open-source the source code for reproducing the experiment results.
Revised: We will open-source the source code for reproducing the experiment results.

Original: The code is currently hosted anonymously on https://anonymous.4open.science/r/
Revised: The code is currently hosted anonymously on https://anonymous.4open.science/r/

Original: GNC-ST-DC8F/readme.md .
Revised: GNC-ST-DC8F/readme.md .

Original: A D ATA AND T ASK
Revised: A D ATA AND T ASK

Original: MultiNLI.
Revised: MultiNLI.

Original: We use the multi-genre natural language inference (MultiNLI)
Revised: We use the multi-genre natural language inference (MultiNLI)

Original: (Williams et al., 2018) corpus for pretraining entailment classifiers.
Revised: (Williams et al., 2018) corpus for pretraining entailment classifiers.

Original: The MultiNLI corpus contains 393k sentence pairs that can be classified into 3 categories: entailed, neutral, and contradictory.
Revised: The MultiNLI corpus contains 393k sentence pairs that can be classified into 3 categories: entailed, neutral, and contradictory.

Original: The entailment relation defined in the MultiNLI task is not strict logical entailment.
Revised: The entailment relation defined in the MultiNLI task is not strict logical entailment.

Original: The textual entailment is defined by “the hypothesis is likely to be true when the premise is true.”
Revised: The textual entailment is defined by “the hypothesis is likely to be true when the premise is true.”

Original: The contradiction means “the hypothesis cannot be true when the premise is ture” and when the truth value of the hypothesis cannot be inferred by the premise, the text pair is classified as neutral.
Revised: The contradiction means “the hypothesis cannot be true when the premise is ture” and when the truth value of the hypothesis cannot be inferred by the premise, the text pair is classified as neutral.

Original: We train a classifier to predict the truth value of the constructed suppositions as mentioned in Section 4.1.
Revised: We train a classifier to predict the truth value of the constructed suppositions as mentioned in Section 4.1.

Original: QNLI/AdvQNLI.
Revised: QNLI/AdvQNLI.

Original: The question natural language inference (QNLI) corpus contains questioncontext pairs.
Revised: The question natural language inference (QNLI) corpus contains questioncontext pairs.

Original: The task is to predict if the answer to a given question is contained in its paired passage.
Revised: The task is to predict if the answer to a given question is contained in its paired passage.

Original: The regular validation set of QNLI contains 5.6k question-passage pairs for evaluation.
Revised: The regular validation set of QNLI contains 5.6k question-passage pairs for evaluation.

Original: AdvQNLI is the adversarial evaluation set proposed by Wang et al.
Revised: AdvQNLI is the adversarial evaluation set proposed by Wang et al.

Original: (2021), containing examples generated by textual adversarial attack.
Revised: (2021), containing examples generated by textual adversarial attack.

Original: The AdvQNLI evaluation set contains 0.9k text examples.
Revised: The AdvQNLI evaluation set contains 0.9k text examples.

Original: The adaptation from MultiNLI to QNLI is a cross-task adaptation.
Revised: The adaptation from MultiNLI to QNLI is a cross-task adaptation.

Original: The supposition for the task is s qnli :
Revised: The supposition for the task is s qnli :

Original: The answer to question { q } is entailed by { p } .
Revised: The answer to question { q } is entailed by { p } .

Original: QQP/AdvQQP.
Revised: QQP/AdvQQP.

Original: The Quora question pairs (QQP) task is proposed in Wang et al.
Revised: The Quora question pairs (QQP) task is proposed in Wang et al.

Original: In the QQP task, a model learns to predict if two questions are duplicated.
Revised: In the QQP task, a model learns to predict if two questions are duplicated.

Original: In most cases, the task is to predict if two questions can be answer by the same answer.
Revised: In most cases, the task is to predict if two questions can be answer by the same answer.

Original: The regular evaluation set of QQP contains 391k question-question pairs and the adversarial set contains 0.4k examples.
Revised: The regular evaluation set of QQP contains 391k question-question pairs and the adversarial set contains 0.4k examples.

Original: The adaptation from
Revised: The adaptation from

Original: MultiNLI to QQP is a cross-task adaptation, an the supposition for the QQP task is s qqp : The answer to { q 1 } is entailed by the answer to { q 2 } .
Revised: MultiNLI to QQP is a cross-task adaptation, an the supposition for the QQP task is s qqp : The answer to { q 1 } is entailed by the answer to { q 2 } .

Original: RTE/AdvRTE.
Revised: RTE/AdvRTE.

Original: The recognizing textual entailment (RTE) task is a binary (entailment/contradiction) entailment classification task based in the news domain.
Revised: The recognizing textual entailment (RTE) task is a binary (entailment/contradiction) entailment classification task based in the news domain.

Original: Each data example is a claim-passage pair and the task is predicting if the claim is entailed by the paired passage or not.
Revised: Each data example is a claim-passage pair and the task is predicting if the claim is entailed by the paired passage or not.

Original: The different between
Revised: The different between

Original: RTE and MultiNLI is that the texts of RTE is generally longer than MultiNLI.
Revised: RTE and MultiNLI is that the texts of RTE is generally longer than MultiNLI.

Original: The regular evaluation set of RTE contains 3k text pairs, and the adversarial evaluation set contains 0.3k data examples.
Revised: The regular evaluation set of RTE contains 3k text pairs, and the adversarial evaluation set contains 0.3k data examples.

Original: The adaptation from MultiNLI to RTE is cross-domain adaptation and the suppositions are the same as
Revised: The adaptation from MultiNLI to RTE is cross-domain adaptation and the suppositions are the same as

Original: MultiNLI suppositions, as defined in Section 4.1.
Revised: MultiNLI suppositions, as defined in Section 4.1.

Original: SST2/AdvSST2.
Revised: SST2/AdvSST2.

Original: The Stanford Sentiment Treebank (SST-2) is a sentimental analysis corpus that requires a model to predict if the attitude of a movie review is positive or negative.
Revised: The Stanford Sentiment Treebank (SST-2) is a sentimental analysis corpus that requires a model to predict if the attitude of a movie review is positive or negative.

Original: The regular evaluation set contains 1.8k test reviews and the adversarial evaluation set contains 1.4k examples.
Revised: The regular evaluation set contains 1.8k test reviews and the adversarial evaluation set contains 1.4k examples.

Original: The adaptation from MultiNLI to SST-2 is a cross-task adaptation with a gap between input formats.
Revised: The adaptation from MultiNLI to SST-2 is a cross-task adaptation with a gap between input formats.

Original: While the inputs of all tasks described above are text pairs, the inputs of SST-2 are movie reviews without any pair text.
Revised: While the inputs of all tasks described above are text pairs, the inputs of SST-2 are movie reviews without any pair text.

Original: To apply the pretrained entailment classifier, we construct both label descriptions and suppositions.
Revised: To apply the pretrained entailment classifier, we construct both label descriptions and suppositions.

Original: The construct supposition template is s sst 2 : The movie is good is entailed by the comment { x } .
Revised: The construct supposition template is s sst 2 : The movie is good is entailed by the comment { x } .

Original: B A DDITION I MPLEMENTATION D ETAILS B. H YPERPARAMETER TUNING
Revised: B A DDITION I MPLEMENTATION D ETAILS B. H YPERPARAMETER TUNING

Original: The models are pretrained and fine-tuned with the AdamW optimizer Loshchilov & Hutter (2017).
Revised: The models are pretrained and fine-tuned with the AdamW optimizer Loshchilov & Hutter (2017).

Original: In pretrained, the learning rate is set to 5 e - 6 for BERT and 3 e - 6 for DeBERTa, and the weight decay weight is set to 1 e - 5 .
Revised: In pretrained, the learning rate is set to 5 e - 6 for BERT and 3 e - 6 for DeBERTa, and the weight decay weight is set to 1 e - 5 .

Original: For the downstream self-training step, the learning rate for both models is set to 4 e - 6 and the weight decay is 1 e - 2 .
Revised: For the downstream self-training step, the learning rate for both models is set to 4 e - 6 and the weight decay is 1 e - 2 .

Original: The learning rates and weight decay coefficients are fixed across all tasks.
Revised: The learning rates and weight decay coefficients are fixed across all tasks.

Original: We tune the baseline self-training models and found that the best epoch number is 6.
Revised: We tune the baseline self-training models and found that the best epoch number is 6.

Original: In GNC self-training, the model automatically decides the epoch number.
Revised: In GNC self-training, the model automatically decides the epoch number.

Original: B. GNC I MPLEMENTATION
Revised: B. GNC I MPLEMENTATION

Original: We summarize the process of GNC self-training with pseudo code in Algorithm 1.
Revised: We summarize the process of GNC self-training with pseudo code in Algorithm 1.

Original: C A DDITIONAL A NALYSIS
Revised: C A DDITIONAL A NALYSIS

Original: Data selection.
Revised: Data selection.

Original: Lang et al.
Revised: Lang et al.

Original: (2022) have mentioned that selecting high-quality data-pseudo label pairs can benefit self-training.
Revised: (2022) have mentioned that selecting high-quality data-pseudo label pairs can benefit self-training.

Original: We hereby analyze the effect of model c onfidence-based data s election (CS) on all tasks we investigate in this work.
Revised: We hereby analyze the effect of model c onfidence-based data s election (CS) on all tasks we investigate in this work.

Original: The experiment results are presented in Figure 5.
Revised: The experiment results are presented in Figure 5.

Original: We noticed that under our setting where a limited number of unlabeled data is used for self-training, the data selection strategy does not guarantee improvement on all tasks.
Revised: We noticed that under our setting where a limited number of unlabeled data is used for self-training, the data selection strategy does not guarantee improvement on all tasks.

Original: However, we found the the
Revised: However, we found the the

Original: GNC method can also benefit the data selection strategy on all tasks except AdvRTE, improving the average performance of 20 experiments and reduce the risk of getting low performance.
Revised: GNC method can also benefit the data selection strategy on all tasks except AdvRTE, improving the average performance of 20 experiments and reduce the risk of getting low performance.

Original: On QQP,
Revised: On QQP,

Original: RTE, and SST2, GNC self-training without data selection is still the most robust strategy.
Revised: RTE, and SST2, GNC self-training without data selection is still the most robust strategy.

Original: Require:
Revised: Require:

Original: Pseudo training set D
Revised: Pseudo training set D

Original: = { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . .
Revised: = { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . .

Original: , ( x n , y n ) } , model M 0 , ¯ c , ϵ δ ← 2 .▷ update step size of µ µ ←t ←▷ number of steps [ w 0 , w 1 , . . . , w n ] ← [1 , 1 , . . .
Revised: , ( x n , y n ) } , model M 0 , ¯ c , ϵ δ ← 2 .▷ update step size of µ µ ←t ←▷ number of steps [ w 0 , w 1 , . . . , w n ] ← [1 , 1 , . . .

Original: , 1] while µ > 1 do l it = loss fn ( M t ( x i ) , y i )
Revised: , 1] while µ > 1 do l it = loss fn ( M t ( x i ) , y i )

Original: ▷ Calculate training loss
Revised: ▷ Calculate training loss

Original: L t = (cid:80) i l it · w i
Revised: L t = (cid:80) i l it · w i

Original: ▷ Calculate the weighted total loss
Revised: ▷ Calculate the weighted total loss

Original: M t +1 ← train ( M t , L t )
Revised: M t +1 ← train ( M t , L t )

Original: ▷ update model parameters using the total loss e it =
Revised: ▷ update model parameters using the total loss e it =

Original: loss fn
Revised: loss fn

Original: ( M t +1 ( x i ) , y i )
Revised: ( M t +1 ( x i ) , y i )

Original: ▷ calculate evaluation loss of training data if t == 0 then µ = 2 · max i e 2 i / ¯ c▷ initialize µ else if t > 0 then µ ← µ/δ ▷ update µ end if w i = (cid:104)
Revised: ▷ calculate evaluation loss of training data if t == 0 then µ = 2 · max i e 2 i / ¯ c▷ initialize µ else if t > 0 then µ ← µ/δ ▷ update µ end if w i = (cid:104)

Original: [Figure]
Revised: [Figure]

Original: ▷ update w i end while
Revised: ▷ update w i end while

Original: [Figure]
Revised: [Figure]

Original: (CS) on baseline and self-training methods on each
Revised: (CS) on baseline and self-training methods on each

Original: DeBERTa model.
Revised: DeBERTa model.

Original: Adversarial evaluation analysis.
Revised: Adversarial evaluation analysis.

Original: We visualize the experiment results with baseline and GNC selftraining on AdvQNLI.
Revised: We visualize the experiment results with baseline and GNC selftraining on AdvQNLI.

Original: The results are shown in 6.
Revised: The results are shown in 6.

Original: As the regular evaluation set, the GNC method is also more robust against high pseudo label noises and labeling perplexities.
Revised: As the regular evaluation set, the GNC method is also more robust against high pseudo label noises and labeling perplexities.

Original: [Figure]
Revised: [Figure]