Graduated Non-Convexity for Robust Self-Trained Language Understanding
Authors: Hongyin Luo
Yoon Kim
James R. Glass
Venue: Submitted to ICLR 2023 (2023)

Abstract: Self-training has been proved an efficient strategy for unsupervised fine-tuning of language models using unlabeled data and model-generated pseudo-labels. However, the performance of self-trained models is unstable under different settings of training and evaluation data, influenced by both data distribution and pseudo-label accuracy. In this work, we propose an outlier robust self-training method based on graduated non-convexity (GNC) to mitigate the problem. We construct self-training as a non-convex optimization problem with outlier training examples. The models are self-trained with robust cost functions based according to Black-Rangarajan Duality. The algorithm learns slack variables as the loss weights for all training samples. The slack variables are used to calibrate the loss items during training to update the model parameters. The calibrated loss items lead to more robust self-trained models against different training and evaluation data and tasks. We conducted experiments on few-shot natural language understanding tasks with labeled and unlabeled data examples. Experiment results show that the proposed loss calibration method improves the performance and stability of self-training under different training tasks and data examples, and also benefits the robustness against adversarial evaluation corpora. 

Under review as a conference paper at ICLR 2023
GRADUATED NON-CONVEXITY FOR ROBUST SELF- TRAINED LANGUAGE UNDERSTANDING
Anonymous authors Paper under double-blind review
ABSTRACT
Self-training has been proven to be an efficient strategy for unsupervised fine- tuning of language models using unlabeled data and model-generated pseudo- labels. However, the performance of self-trained models is unstable under dif- ferent conditions of the training and evaluation data, influenced by both data dis- tribution and pseudo-label accuracy. In this work, we propose outlier robust self-training method based on graduated non-convexity (GNC) to mitigate the problem. We construct self-training outlier training examples. The models are self-trained with robust cost functions according to Black- Rangaaaaa Duality. The algorithm learns slack variables the loss weights for all training samples. The slack variables are used to calibrate the loss items during training update the model parameters. The calibrated loss items lead to more robust self-traine models against different training and eval- uation data and tasks. We conduct experiments on few-shot natural language un- derstanding tasks with labeled and unlabeled data examples. Experimental results show that the proposed loss calibration method improves the performance and sta- bility of self-training on different tasks, benefiting the robustness against incorrect pseudo-labels, imbalanced training data, overfitting, and adversarial evaluation data.
non-convex optimization problem with
INTRODUCTION
Recent developments in large-scale pretrained language models has significantly improved the per- formance of natural language understanding tasks (Devlin et al.) 2018 Liu et al. 2019 Clark et al.] 2020 He et al.) 2020; Brown et al. 2020). After pretraining, these models are typically fine-tuned on task-specific training data with
labels are not available (or large enough) for all tasks of interest. When there is a significant number of unlabeled examples, a pretrained model can utilize techniques such as self-training to improve performance (He al. 2019 Zoph et al. 2020). An issue with this approach is that the generated pseudo-labels can be noisy (Zhao et al.) |021]] Lang et al.] [022]] ZZhang & Zhou 201T), since a pretrained model can make wrong predictions for unseen examples.
labels. However, numan-generann
We propose learning-basse based loss calibration strategy that tunes the loss weights for each data ex- ample during self-training. In this approach, subset of training data are assigned low weights for calculating the overall training loss, leading to less influence in parameter updates. To learn the loss weights, we employ the graduated non non-convexity (GNC) strategy (Yang al.) (020)) based on Black-Ranggrajan Duality (Black & Rangarajan] 1996). Under the fully supervised and outlier-free setting, the model parameters are updated optimize the total cost of all training samples, while under the self-training setting, the training set usually contains outliers with wrong pseudo-labels. We thus optimize a robust cost function
Ly Σ pll(in x.):
where I(-) is the selected loss function and Derain = (1,, vi E [0, N]} is the training set, and stands for the set of trainable parameters. p(-) is a robust cost function and 9J stands for an element of a noisy pscudo label set. Such methods have been widely applied in computer vision (Black) & Rangarajann 1996) and robot perception (Yang et al. 2020) where there exists ground-truth

---

Under review as a conference paper at ICLR 2023
optimization target for the algorithm to prediet. Few application examples have been seen in other domains and the approach has not been applied in the area of self-training. The difficulty is that unlike prediction, an optimal self-training loss training examples does not guarantee minimal loss on evaluation sets. In fact, adding noise in self-training can produce better performance, for example with Dropout (He et al. 2019) and confidence regularization (Zou et al. 2019).
In this work, we propose GNC self-training that produces robust performance against incorrect pseudo labels, imbalanced training data, overfitting, and adversarial evaluation data. The core idea is penalizing outliers with wrong pseudo labels and over-confident training samples by assigning low loss weights. As result, such training samples contribute less to the updated model parameters than other samples. In this work we make the following contributions,
Applying graduated non-convexity (GNC) in self-training, To the best of our knowledge, this work is the first attempt of applying GNC for self- language models. While traditional GNC methods treat high-loss observations as outliers, we noticed that in self-training, outliers include both high-loss and over-confident examples. We propose shifted robust cost function to deal with over-confident cases, and the loss weights are tuned during self-training.
Robustness against different training and evaluation data. The difference within different train- ing and evaluation corpora leads to significant performance gaps between the two because of the accuracy of pseudo-labels and the difficulty/coverage of data examples. We find that the proposed method is robust against different tasks, training corpora, and adversarial evaluation data, outper- forming label-free self- self-training strategies and label-dependent few-shot learning baselines.
2 RELATED WORK
Recent pretrained language models are trained with self-supervised learning strategy, including predicting masked words (Devlin et al. 2018 [ [Lu et al. al.|| 2019 He et al. 2020), predicting next words
(Brown et al. 2020 Raffel et al. 2020 Lewis et al. (2019). predicting the correctness of words (Clark) ct al. 2020), and representing sentences (Gao et al. 2202 Chuang et al. 2022). Besides pretraining on corpora without additional human- human-generatee generated labels, textual entailment corpora, including SNLI
[Bowman et al. 2013) and MultiNLI (Williams et al. 2018) are also used for both sentence-level pretraining (Reimers & Gurevych 2019) and downstream tasks; for example relation extraction (Obamuyide & Vlachos 2018 Yin et al. 2019) and fact checking (Thorne & Vlachos (208)).
While most self-superviser learning methods are agnostic to downstream tasks, self- mod- els learn to handle any downstream task by learning from synthetic data or pseudo-labels that are automatically generated by the same or other models. Among recent research under the context of self-training, back translation (He et al.) (219)) and data augmentation methods (Xie et al. 2020: Chen al. 2020) use synthetic texts as inputs. pseudo labeling methods (Zoph al.) 2020 Zou et al.) (019) use synthetic labels as training targets, and self-trained question answering methods (Bartolo et al. 2021 Luo et al. 2022) use synthetic questions as inputs and synthetic answers as target outputs.
Recent studies have discussed the efficiency and robustness of language models from different as- pects. including different settings training and evaluation data. The authors of Zf et al. (2019); (202))/Baartloo Barttloo et al. (2020); Waag et al. (200)) indicated that even state-or-the-art language adversarial evaluation examples. On the other hand, even for language
Jin al. (2020);
models make mistakes models pretrained on large corpora, their performance on downstream tasks might not be satisfying
if not enough task-specific training data is presented (Schick & Schüize 2021 Le Scao & Rush, 2021). There is little work simultaneously dealing with the limitations of small training sets and missing human-generated labels.
BACKGROUND: GRADUATED NON-CONVEXITY
Graduated non-convexity (GNC) is a popular optimization method in vision (Blake & Zisserman, 1987) and machine learning Rose 1998; Mobahi & Fisher 2013). The core idea is optimizing robust cost function instead of standard loss function, as shown in Equation The robust cost function p(-) is adjustable give a coefficient μ. When μ is large enough, p(l(y,, x;.] approximately convex. During the optimization, the value of y is decreased at each step and the optimization prob-
2

---

Under review as a conference paper at ICLR 2023
lem is gradually recovered to the original non-convex problem. A more convex iteration provides guess for the following optimization.
A typical robust surrogate cost function is the Geman-Mcclus (GM) function. The GM function and the controlled GM function are shown as follows, as mentioned Yang et al. 2020).
22 p((()) ( 22 Pa((())
μ.c² +
where č is a parameter that decides the shape of the GM function, and represents the largest cost of threshold of outlier losses. According to Equation 2 when μ - oc, (x - 12; when l 1, (t) = p(l). The optimization result p()) reached by optimizing a high-quality guess of (T)
In practice, following (Yang al. 2020), the optimization target apply is mutation of Py based on the Black-Rangarajan Duality (Black & Rangarajan (199)), which proved that the problem of optimizing (t( is equívalent to optimizing weighted sum of I(+) and weight penalty term Φ. In other words, the optimization problem in Equation [ has the same optimal solution with the following task,
I = argmina D(ur Juoe Uyy, Ty; 0)² + ((w))), wi E [0,
(3)
The set of w = f W2,,.... wn] are slack variables to control each loss item lit, z).. In the robotic perception and computer vision areas, Yi stands for noisy observation of the i -th sensor and x stands for the ground-truth location or shape of the physical object of interest. According to Black & Rangarajan (1996), when the GM function is applied the robust cost function, the weight penalty function Φ (w:) can be written as
Pp (w,) = (C -1) 1)²
(4)
In this work, we jointly optimize the weighted sum of the standard loss ((1,,)) and the weight penalty loss (ww)).
METHOD: GNC SELF-TRAINING WITH SHIFTED GEMAN MCCLURE
4.1 ENTAILMENT PRETRAINING AND SUPPOSITION VERIFICATION
In this work, we investigate weakly supervised domain and task adaption of textual entailment mod- els, where the size of training texts and the access to human- ueman-generated labels are limited. To enable the adaptation,
formulate natural language understanding tasks classifications tasks where model just needs to predict the truth value of input suppositions. We first pretrain entailment clas- sifiers MultiNLI (Williams et al., 2018) that learns predict the truth values of the following suppositions based on a hypothesis { and the premise {
/ {x] entailed by {y); s2 stnnts {x} is not true when {y} is true.
According to the definition of entailment, s/ and 5² are contradictory, thus they cannot both be true of false. A model trained to predict the truth value suppositions constructed by each hypothesis- premise pair (h,, P:) can be applied other tasks, if similar suppositions can be constructed. For ample, an entailment classifier can be applied to handle the movie review classification task (SST2) (Socher al. 2013). A sentiment analysis task can also be formulated or supposition verification task, with the following potential suppositions,
entailment prediction
s sst2: I like the movie entailed by my comment {x]; 22 The movie good because {x}.
We construct the suppositions following Anonymous 2022 for different tasks, including SST2,
RTE (Dagan al. (2055). QNLI (Rajpurkar et al. 2016). QQP
(Wang al. 2018), and their adver-
sarial evaluation corpora (Wang al. 2021).
4.2 SELF-TRAINNN OF THE ENTAILMENT CLASSIFIER
In this work, we investigate learning speciffcc models with a pretrained entailment classifier with limited amount of unlabeled training data Drask (21, -2, F..) where o; stands for an
3

---

The pseudo-labels are then used to train task-specific models. new training set is constructed as Dtrain { 21), (I1, .2)..... (zx, (x)]. The new training set is used to fine-tune the pre- trained entailment classifier. Note that the pretrained entailment model has three classes (entailment, Under review as a conference paper at ICLR 2023
unlabeled data example. Given a pretrained entailment model M, We generated pseudo-labels for each data example by feeding suppositions constructed based on 24
= [stssk((D))]]
(5)
neutral, and contradiction), we just use the entailment and contradiction scores for the fine-tuning and evaluation of downstream tasks.
4.3 CONVEXITY CONTROLLED TRAINING
The Black-Rangaraiii Duality (Black & Rangarajan 1996) suggests that optimizing the surrogate robust cost function >; ((((v, r)) is equivalent to optimizing the following cost function
Lp = Ewu (un.-((ye (yr 21; 0) + (w:)),, w, E [0. 1]
(6)
In this work, we apply the Geman McClure (GM) function as the robust cost p(.). Before the beginning of training, all slack variables w; are set to 1. We use the the alternating optimization strategy proposed in Yang al. (2020). At each optimization step t, we first update the model parameters 0 based on the weighted sum of base costs,
0² = argming Lut Pyi, xz;;
then we fix the learned 0 and update the slack variables W = [wn,, W2,-.., Wal. Note that in Yang let al. (2020) and in related work in visual perception, stands for the number of sensor observations, but in our work n stands for the number of training examples. At each time step, w, is updated as
: argmin gns Σlur- Cyr,, 24; 2); + x(w,)]
(8)
( 24; 02) + pi
This analytic solution is obtained by solving Bws
8hg 0 when 0 is fixed. The output w is used to cal-
culate 0++1 in the next episode. At the first training step, μ is initialized as μ 2/2 maz (yi, T;; /²² In the following training steps, we decrease it by μ - p/2.8. The training stops when μ < 1.
It is straightforward to see that an observation, or training example (J;,;;) receives low weight w; when its corresponding loss ([y,, xy; 0) is high. In visual perception, this feature helps the algorithm to find outlier sensors. In these tasks, excluding outliers benefits the accuracy of the predictions because 1) the perception functions of the sensors are usually known, and 2) accurate object predic- tions must lead to low cost if the observation is correct. The situation is different under the context of training neural language models.
In neural language model training, the models presented with (;,,;;) pairs instead of (y:, f). so there is no guarantee if the model tuning step in Equation could find a global optimal solution of 0 under the current setting of not. On the other hand, high-loss example in machine learning is not necessarily an outlier - it might bring new knowledge that improves the model. Another difficulty that an outlier might also improve the model performance and robustness. Under the self-training setting, low cost might be associated with either correct pseudo-labe -abel or a wrong pseudo-label that the model is ever-confident about. These difficulties prevent the direct application of GNC on machine learning, especially self self-training traininin of neural language models.
To mitigate the problem, we introduce the shifted Geman McClure (SGM) function for graduated non-convexity optimization. Besides the parameter ō that controls the maximum cost of non-outlier examples, we set another parameter 6 that bars the minimal normal cost of non-outlier examples. Under this setting, the outlier set consists of two parts, including high-loss examples associated with wrong pseudo-labels, and over-confident examples whose loss is too low that might lead to overfitting of the trained model. The SGM function applied as robust cost is
ud² (1-66) e)²
(n(()= = ni²2t + (1 e)²
(9)
4

---

Under review as a conference paper at ICLR 2023
Figure 1: Example of GM (left) and SGM (right) functions under different j and loss values.
When < c, the robust cost p increases I continues to decrease. We use this method to prevent overfitting caused by over-confident examples. The shapes of GM and SGM functions are shown in Higher μ values result in higher curves.
Figure T
5
EXPERIMENTS
5.1 TASK AND DATA
MultiNLI We use the multi-genre natural language inference (MultiNLI) (Williams al. 2018) corpus for pretraining entailment classifiers. The MultiNLI corpus contains 393k sentence pairs that can be classified into 3 categories: entailed, neutral, and contradictory. We train a classifier to predict the truth value of the constructed suppositions mentioned in Section 4.1
ONLI/AdVQNLI The question natural language inference (QNLI) corpus contains question-
context pairs. The task is to predict if the answer to a given question is contained in its paired pas- sage. We construct the following supposition for the task, Spnti The answer to question {q} is entailed by {p}.
QQP/AdvQQP. The Quora question pairs (QQP) task proposed Wang et al. (2018). In the QQP task, model learns predict if two questions duplicated. In most the task is to predict two questions can answer by the same answer. The adaptation from MultiNLI to QQP cross-task adaptation, and the supposition for the QQP task is 8ggp2 The answer to {q1 is entailed by the answer to {92}.
The recognizing textual entailment (RTE) task is binary (entailment/contraacttion entailment classification task based in the news domain. Each data example is a claim-passage pair and the task is predicting if the claim is entailed by the paired passage or not. The adaptation from MultiNLI to RTE is cross-domain adaptation and the suppositions are the same as MultiNLI suppositions, defined in Section 4.1
RTE/AdvRTE.
SST2/AdvSST2. The Stanford Sentiment Treebank (SST-2) is a sentimental analysis corpus that requires model to predict if the attitude of movie review is positive or negative. The adaptation from MultiNLI to SST-2 is a cross-task adaptation. The construct supposition template is S The movie is good entailed by the comment {x}
5.2 IMPLEMENTATION DETAILS
In this work, train BERT (Devlin et al. 2018) and DeBERTa He et al. 2020 models the pretrained entailment classifiers, which are trained on the MultiNLI corpus Williams et al. (2018). In GNC self-training, there
key hyper-parameters: the threshold of outlier losses é and the amount of shift in the SGM function e. For each task, we select value from (le-2, 2, le -e- 3, le 4} for ε. To find an appropriate e, we calculate the losses | E Ra on all training samples after each training epoch and set e = pul)= ())( (n), E [1.2, 2.8). More details are in Appendix B
5.3 EXPERIMENTAL RESULTS
In this work, we investigated different experimental settings, including few-shot self- tet--iriiing and the combination of fho- fine-tuning and self-training.
5

---

Under review as a conference paper at ICLR 2023
GLUE
AdvGLUE
Method
Method
QNLI
QQP
RTE
SST2
QNLI QQP
RTE
SST2
Few-shot (left) and supervised (right) medium LMs (350M) with human- -uman-generated labels
PET
61.3
67.6
65.7
91.8
R3F
47.5
40.6
50.1
38.5
LM-BFF
69.2
69.8
83.9
CTT
49.6
40.7
46.2
39.2
P-tuning
58.8
67.6
70.8
92.6
MT
47.5
41.5
52.5
51.3
PPT
68.8
67.2
67.9
92.3
BERT
39.8
37.9
40.5
33.0
UPT
70.1
72.1
68.9
92.9
DeBERTa
57.9
60.4
79.0
57.8
Few-shot large L.Ms
(137B) with human-generated labels
LaMDA
35.7
58.9
70.8
"92.3"
FLAN
63.3
75.9
84.5
94.6
Zero-shot pretrained entailment classifiers based on medium LM (350M)
RoBERTA-PT
71.5
78.6
81.2
87.7
62.1
32.6
61.7
59.9
DeBERTa-PT
77.3
79.9
84.5
90.1
61.5
64.1
66.7
42.6
Few-shot Sclf-trained medium LMs (350M) without human- generated labels
ROBERTa-ST
71.2
78.9
81.4
86.0
63.9
56.3
61.1
54.7
RoBERTa-GNO 77.4
79.1
80.8
88.1
66.1
60.2
65.6
48.2
DeBERTa-ST 76.1
79.4
82.9
89.1
63.2
64.7
69.2
47.0
DoBERTa-GNG 78.7
79.8
83.8
90.4
65.3
68.6
69.0
47.5
Few-shot supervised + telf-trained medium LMs (350M) with labeled and unlabeled data
RoBERTA-ST
79.3
75.7
84,4
88.7
61.9
60.0
64.9
52.7
RoBERTa-GNC 80.2
76.9
84.9
90.3
62.9
60.0
64.7
57.8
DcBERTa-ST 79.6
78.2
85.1
65.4
64,9
74.9
DeBERTa-GNC 83.1
79.4
85.5
91.1
66.5
67.5
78.1
48.7
Table 1: Experimental results of few-shot supervised, sully-supervised, and few-shot self-training (unsupervised) models and methods. BERT and DeBERTa results for AdvGLUE are trained with entire training corpora. The self-training results averaged with 20 independent experiments.
Few-shot self-training. Under the few-shot self relf-training setting, a pretrained entailment classifier can only access a limited number (K 12 or 132) of unlabeled data examples. The pretrained models generate pseudo labels for fine-tuning. We evaluate pretrained DeBERTa and RoBERTa entailment classifiers on both regular and adversarial evaluation sets. We conduct the evaluation with 20 independent experiments and report the average performance.
Combined Few-shot fine-tuning and self-training. In many situations, the model can access a limited number of training examples with human-generated labels and a larger number of unlabeled data examples. In this case, we first fine-tune the pretrained models with the labeled data, and self-train the fine-tuned model using unlabeled data with generated pseudo-labels. We evaluate the regular self-training strategy with GNC self-training with few-shot fine-tuned models. In each experiment, use 12 human-labeled training examples and 132 unlabeled data examples.
We compare the performance of the proposed methods with strong few-shot and fully supervised baselines. While the baseline models access task-specific human-generated labels, our pure self- training method only process unlabeled data examples from each task. The few-shot learning base- lines include PET Schick & Schütze (202T). LM-BFF Gao al. (2020), P-tuning Liu et al. (202T). PPT Gu et al. (2021). and UPT Wang et al. 22022), which are medium-sized models with around 350M parameters trained with 16 numan-labeler examples. FLAN Wei et al.) (202I) and LaMDA Thoppilan et al. (2022) are large-scale language models with 137B parameters and are tuned with up to 12 data examples. The robust baselines R3F R3F Aghajanyan al. (2020), CT Xu et al. (202)) and
et al. (2222) are trained the full training corpora.
The experiment results are shown Table We found that the self-training methods receive signif- icant improvement on adversarial evaluation data, while matching outperforming the pretrained entailment classifiers on regular evaluation sets. We cite the performance of baseline methods re- ported by the authors, and summarize the self-training performance by averaging the results of 20 independent experiments. We found the self-training methods that are tuned with unlabeled data examples outperform the 16-shot and fully-supervised baselines in most experiments. Comparing

---

Under review as a conference paper at ICLR 2023
Contearca
DVC ST
Serly--
Cp CMC Looeritee
em CNC lightrt The
CNC St
GMG Seadyy-
00-
20-
24
30-
90
Adve caniia E-13
Ang.ark 6.
Figure The performance (%) DeBERTa- and RoBERTa-large models with different self- training strategies under the few-shot self self-training setting. The black dots stand for mean perfor- mance, the colored bars stand for the range of +1 standard deviation, and the grey lines indicate the minimum and maximum accuracy. All results summarized with 20 independent experiments. ST stands for the regular self-training and GNC stands for self-training with graduated non-convexity. K stands for the number of unlabeled training examples processed by the models.
the best few-shot and suply-sue results of medium language models, our GNC self-training method improved 18.8%/7.4%% on QNLI/AdVONLI and 7.7%/8.29% QQP/AdvQQP.
Although GNC constantly outperforms the regular self-training method, its performance is signif- icantly lower than sully-sup method AdvRTE and AdvSST2. With 12 labeled and 132 unlabeled data examples, the GNC self-training method achieved the highest RTE and AdvSST2 per- formance in addition, and the accuracy AdyRTE and AdvSST2 is boosted to the fully-supervised level (78. % vs. 79.0%, 57.8% vs. 57.8%). In the self-training experiments, we use all unlabeled training data and pseudo labels. We will analyze the effect of data selection in the next section.
ANALYSIS AND ABLATION STUDY
Stability and robustness of self-training. We characterize the stability of the model performance from different perspectives, including the mean value, standard deviation, and miniiumm/maximmn accuracy with different pretrained models and training strategy. To assess the effect of the proposed method, we propose the following metrics,
A strictly more stable model achieves higher mean and minimum/maximmm accuracy, with lower standard deviation.
A softly more stable model achieves higher mean and minimum accuracy.
less risky model achieves higher minimum accuracy and lower standard deviation.
Under the few-shot self-training setting, pretrained entailment classifier can only access limited number (K 12) of unlabeled data examples. The pretrained models generate pseudo labels for fine-tuning. We evaluate pretrained DeBERTa and RoBERTa entailment classifiers with the confidence regularization (CR) method proposed in Zou et al. (2019) both regular and adversarial evaluation sets. We conduct the evaluation with 20 independent experiments and report the mean, minimum, maximum performance, and standard deviation in Figure
The experimental results show the effect GNC in self self-training. In all 48 comparisons in Figure the GNC method is strictly more stable than the baselines in 23 cases, where all metrics (mean, std,

---

Under review as a conference paper at ICLR 2023
min, max) indicate that GNC perform better. There are 35 (72.9%) cases where GNC is softly more stable or less-risky than its baselines. There are only cases where the baselines are strictly better than GNC (AdvSST2, K 12, 132, RoBERTa) and another 4 cases where the baselines are softly better than GNC (AdvRTE, K = 12; RTE, K = 132).
GNC ST
DeBERTa
DeBERTA-CR
RoBERia
K=12
Adv, K=12
Reg. K-132
Adv, K=132
Strictyy Stable
11
6
Sonty Stable
Ricky
Total Num of Exp.
14/16
11/16
12/16
11/12
W/12
X/12
9V12
Table 2: Number of evaluations in each performance category with different models and data.
GNC ST
QNLI
QQP
RTE
SST2
AdvONLI
Ad+QOP
AdvRTE
AdvSST2
Stricly Stable
3
2
Sofily Stable
2
Ricky
lotaliipp.
516
2/6
3/6
4/6
6/6
4/6
3/6
Table 3: Number of evaluations in each performance category in different tasks.
We compare the model performance across methods and tasks. We first note that in 15 out of 16 tasks, the performance DeBERTa+GNC is at least softly more stable than DeBERTa+CR, The only exception is the AdvSST2 task when K 12, where DeBERTa+GNC is less risky than CR as defined. We also found that the DeBERTa-based models generally outperform RoBERTa, except for adversarial QNLI and SST2 evaluations. We also compare the model performance under different model and training data settings. Table indicates that the GNC strategy is more effective on DeBERTa and the improvement is most obvious when
12. This proves that when the size of training data is limited, ooffttrainnng is likely to be biased and the performance would be unstable. Note that each experiment stands for the summarized result of 20 independent runs.
The performance analysis across different tasks is presented in Table 3 summarizing all model and training data settings. We found that the QQP and AdvQQP tasks benefits most from the GNC strategy, while the RTE task does not. We believe the reason is that the RTE task is also an entailment classification task, so is not difficult for the baseline self-training method to adapt.
Few-shot + self- self-training performance. We also analyze the experimental results under the com- bined few-shot supervised and self-training settings on different tasks, where we first fine-tune the pretrained entailment classifiers with 12 labeled data examples and then conduct self-training with 132 unlabeled examples. The results of RoBERTa and DeBERTa models are visualized Figure 3 Out of 8 tasks, DeBERTa+GNC is the most stable strategy in general, while RoBERTa+GNC achieves the best performance on the AdvSST2 task. The results indicate that besides the pretrained entailment classifier, the GNC method is also robust for the self-training of few-shot supervised models.
72
Figure 3: Results of self-training few-shot supervised models with DeBERTa and RoBERTa using 12 labeled examples and 132 unlabeled examples.
Geman McClure (GM) and Shifted Geman McClure (SGM) functions. We analyze the perfor- mance of using GM and SGM the robust cost function QNLI and SST2 tasks.
8

---

Under review as a conference paper at ICLR 2023
CH
3.
Figure 4: Analysis of the factors that influence self-training performance and the first-epoch weights calculated by the GNC. P(pseudo label) stands for the average probability of the predicted pseudo labels, while P(real label) stands for the average predicted probability of the auman-generater labels. Pseudo label bias stands for the percentage of the pseudo label in all selected data examples.
The experimental results are shown in Table The results indicate that shifting the optimum point of the GM function can lead to more stable performance, by comparing the mean and minimum performance on QNLI and SST2 tasks. However, the variance of GM performance larger and achieves higher maximum performance the adversarial evaluations. Despite this fact, conclude that the SGM cost function introduces more stable and robust self-training performance.
still
What Makes Self-training Unstable and How Does GNC Help? We randomly selected 20 different train- ing sets of the QNLI task, and then conduct the baseline
QNLI
AdvQNLI
SST2
AdvSST2
Mean
75.73
64.22
$9.04
46.42
Sid
6.13
3:72
3.24
55.41
and GNC self-training on each training set. We plot the relation of self-training performance and related factors as shown in Figure 4 with data pairs and linear regres- sions. From the upper plots, we find that the perfor- mance is roughly related to the pseudo-label accuracy and the pseudo-labeling perplexity. Figure 4 indicates that the confidence of the pseudo-labels affects the self- training performance, and GNC is more robust against the change. Figure I and с suggest that the accuracy pscudo-labeling has more impact. When the accuracy labeling accuracy is high (~ 90%), the baseline and GNC self-training methods achieve similarly high performance. However, when the pseudo-labeling of the baseline self-training method significantly drops, while the GNC self-training performance is still robust even when the labeling accuracy is lower than 50%.
Max
82.39
70.95
92.09
$2.03
Moin
78.67
SGM
Min
Sid
66.08
4.68
60.14
2.9
86.35
1.69
41.89
2.35
Table Comparison of GM and SGM.
accuracy is low, the performance
In Figure a b, and plot the loss weights w; in the first self-training epoch. We find that GNC tends to highlight low-confident data to avoid overfitting as shown in Figurq4]d while it does not highlight all data examples with correct pseudo labels (Figure 4e). Figure 4 indicates that when the pseudo label set of the randomly selected data is imbalanced, the data examples of the majority pseudo labels receives lower loss weights. This can mitigate the bias during self-training.
Additional analysis on data selection and adversarial evaluation data is presented in Appendix C
7 CONCLUSION
We apply graduated non-convexity in robust self-training for natural language understanding. With the Black-Rangarajan Duality, we transfer the optimization of shifted robust cost function to op- timizing a weight loss function, where the loss weights are learned to improve the stability and robustness of self-trained language understanding models. Experimental results summarized from 20 independent runs show that the GNC method improves the performance, stability, and robustness of self- self-trainn training on 4 tasks and corresponding regular and adversarial evaluation corpora. Our label- free method significantly outperforms strong dependdep baselines. Through the analysis and visualization, we found the GNC self self-training strategy is more robust against wrong pseudo-labels, imbalanced training data, overfitting, and adversarial evaluation data.

---

Under review as a conference paper at ICLR 2023
ETHICS STATEMENT
In this work we propose self-training method for language understanding. Traditionally, training a language understanding model needs human annotation on large-scale corpora that contributes to number of annotation jobs. Our method requires much less numan-generann the risk of reducing the number of annotation jobs and pays. On the other hand, since the model learns from its outputs, is still possible for the model to generate risky predictions, although the method proposed in this manuscript helps mitigate the risk.
labels, leading to
REPRODUCIBILITY STATEMENT
Data. We introduce the tasks and corpora we used for training and evaluation in Section 5.1 and Appendix A We mentioned how the data in Section 5.3
Method. We introduce the background of the method in Section the details of our method in Section and present the pseudo-code of our algorithm in Appendix B.2
Hyper-parameter. We describe the key hyper-parameteer of GNC self-training in Section 5.2 and list other regular hyper-parameters in Appendix [.1
Experiments. We describe the experiment results, number of independent runs, and visualized analysis in Section 6 and Appendix to prove the statistical significance.
Source code. We will open-source the source code for reproducing the experiment results. The code is currently hosted anonymously on hitts::/anonyyoont 4open. wcw.e:ese...ccc.ccceoo.ccce GNC-ST-DC8F/ rew./ee.....m.. md
REFERENCES
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. Better fine-tuning by reducing representational collapse. International Conference on Learning Representations, 2020.
Anonymous. Unsupervised few-shot adaptation of entailment classifiers for robust natural language understanding. anonymous preprint under review, 2022.
Max Bartolo. Alastair Roberts, Johannes Welbl, Sebastian Riedel, and Pontus Stenetorp. Beat the ai: Investigating adversarial human annotation for reading comprehension. Transactions of the Association for Computational Linguistics, 8:662-678, 2020.
Max Bartolo, Tristan Thrush, Robin Jia, Sebastian Riedel, Pontus Stenetorp, and Douwe Kiela. Improving question answering model robustness with synthetic adversarial data generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pp. 8830-8848, Online and Punta Cana, Dominican Republic, November 2021. Associa- tion for Computational Linguistics. doi: 10. 10.18653/w12021..co http://w112221.elpmmooo www.lp-mamnnm666
URL https:
7/ oollantologyy.rgg wwwp//ggg..ooooeooo../o.o......e 2021. emn www.pmmminnn......mmmm.....mmamm 696
Michael Black and Anand Rangarajan. On the unification of line processes, outlier rejection, and robust statisties with applications in early vision. International joteraaaaonaoournall of computer vision, 19(1): 57-91, 1996.
Andrew Blake and Andrew Zisserman. Visual reconstruction. MIT press, 1987.
Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher Manning. large anno- tated corpus for learning natural language inference. arXiv preprint /50..:/500005326 2015.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901,, 2020.
10

---

Under review as a conference paper at ICLR 2023
Jiaao Chen, Zichao Yang, and Diyi Yang. MixText: Linguistically-nnoomee interpolation of hid-
den space for semi-supervies text classification. In Proceedings of the 58th Anmual Meet- ing of the Association for Computational Linguistics, pp. 2147-2157, Online, July 2020. As- URL https:
sociation for Computational Linguistics. doi: 10. //aclanthollogy.oom www.hholloo....go2oooooooo att.:/2220..cccmmama. 1944
Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljačić, Shang-Was Wen Li, Wen-tau Yih, Yoon Kim, and James Glass. Diffcse: Difference-based contrastive learning for sentence embeddings. arXiv preprint AXXi:2224.110990 2022.
Kevin Clark, Minh-Thang Luong, Quoe V Le, and Christopher D Manning. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.1055: 2020.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pp. 177-190. Springer. 2005.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 2018.
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better -shot learn-
ers. arXiv preprint arXiv:2012.15722 2020.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simese: Simple contrastive learning of sentence embeddings. arXiv preprint urXiv:2104.088.5 2021.
Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. Ppt: Pre-trained prompt tuning for -shot learning. arXiv preprint arXiv: (09.::2099.04333 04332, 2021.
Junxian He, Jiatao Gu, Jiajun Shen, and Marc' Aurelio Ranzato. Revisiting self- self-training for neural sequence generation. arXiv preprint arXiv: /909. 13788, 2019.
Pengeheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654 2020.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 8018-8022 2020.
Hunter Lang, Monica N Agrawal, Yoon Kim, and David Sontag. Co-training improves prompt- based learning for large language models. In International Conference Machine Learning, pp- 11985-12003. PMLR, 2022.
Teven Le Scao and Alexander Rush. How many data points is prompt worth? In Proceedings of the 2021 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Techmologies, pp. 2627-2636, 2021.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising equuence-to-sequencc pre- training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.134611 2019.
Xiao Liu, Yanan Zheng, Zhengxiao Du Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. arXiv preprint arXiv:2103, 103.85,, 2021.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 1692, 2019.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 1rXiv 2017.
11

---

Under review as a conference paper at ICLR 2023
Hongyin Luo, Shang- Wen Li, Mingye Gao, Seunghak Yu, and James Glass. Cooperative self- training of machine reading comprehension. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pp. 244-257 Seattle, United States, July 2022. Association for Computational Linguis- tics. doi: 10.18653/ 1awp:///1m12222..mmmmaiaa
URL https::tp//..........m //acss://acll wwt.aaoogg..ogg//00000
naacl - mwtmmaainn... 18
Hossein Mobahi and John W Fisher. On the link between gaussian homotopy continuation and convex envelopes. In International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 43-56, Springer, 2015.
Abiola Obamuyide and Andreas Vlachos. Zero-shot relation classification as textual entailment. EMNLP 2018, pp. 72, 2018.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1-67, 2020.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint (Eii:v6606052500 2016.
Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-IJCNLP), pp. 3982-3992, Hong Kong, China, November 2019. Association for Com- putational Linguistics. doi: (0.18653/v1/D199 1410. URL https::/aclantthol.gg//.ooo
D19-1410
Kenneth Rose. Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proceedings of the IEEE, 86(11):2210-2233, 1998.
Timo Schick and Hinrich Schütze. Exploiting cloze-questions for -shot text classification and nat- ural language inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 255-269, 2021.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empiricul methods in natural language pro- cessing, pp. 1631-1642, 2013.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng- Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201. 008339 2022.
James Thore and Andreas Vlachos. Automated fact checking: Task formulations, methods and future directions. arXiv preprint orXiv:1806 800.00687, 2018.
Shoujie Tong, Qingxiu Dong, Damai Dai, Tianyu Liu, Baobao Chang, Zhifang Sui, et al. Ro- bust fine-tuning via perturbation and interpolation from in-batch instances. arXiv preprint arXiv:2205.00633, 2022.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv: (004.0800000450 2018.
Boxin Wang. Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. arXiv preprint arXiv:2111.0228 2021.
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi. Song- fang Huang, and Ming Gao. Towards unified prompt tuning for -hot text classification. arXiv preprint arXiv:2205.n 203.02200 2022.
12

---

Under review as a conference paper at ICLR 2023
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoe Le. Finetuned language models are zero-shot learners. arXiv preprint 21t.::2209.01622, 2021.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 11 1112-1122. Association for Computational Linguistics, 2018. URL
http: / acc web. org/ antho 1 ogy/N1 (oyy/N18-111 8-
Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmentation for consistency training. Advances in Neural Information Processing Systems, 33:6256-6268, 2020.
Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, and Fei Huang. Raise child in large language model: Towards effective and generalizable fine-tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 9514-9528, 2021.
Heng Yang. Pasquale Antonante, Vasileios Tzoumas, and Luca Carlone. Graduated non-convexity for robust spatial perception: From non-minimal solvers to global outeitrrejection IEEE Robotics and Automation Letters, 127-1144,, 2020.
Wenpeng Yin, Jamaal Hay, and Dan Roth. Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat- ural Language Processing (EMNLP-IJCNLP), pp. 3914-3923, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/D19-1ee 1404. URL https:
//aclanthology. httptthhoooooy.ooo1l----111- ottg//9--11000
Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. Word-level textual adversarial attacking as combinatorial optimization. arXiv preprint arXiv:1910.1211 12196, 2019.
Min-Ling Zhang and Zhi-Hua Zhou. Cotrade: Confident co-training with data editing. IEEE Trans- actions on Systems, Man, and Cybernetics, Part B (Cybernetics), 41(6):112-11266 2011.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning, pp- 12697-12706. PMLR, 2021.
Barret Zoph, Golnaz Ghiasi, Tsung- Yi Lin, Yin Cui, Hanxiao Liu, Ekin Dogus Cubuk, and Quoe Le. Rethinking pre- pre-training and self-training. Advances in neural information processing systems, 33: 3833-3845, 2020.
Yang Zou, Zhiding Yu, Xiaofeng Liu, BVK Kumar, and Jinsong Wang. Confidence regularized self-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5982-5991, 2019.
A DATA AND TASK
MultiNLL We use the multi-genre natural language inference (MultiNLI) (Williams et al.) 2018) corpus for pretraining entailment classifiers. The MultiNLI corpus contains 393k sentence pairs that can be classified into 3 categories: entailed, neutral, and contradictory. The entailment relation defined in the MultiNLI task is not strict logical entailment. The textual entailment is defined by "the hypothesis is likely to be true when the premise is true." " The contradiction means "the hypothesis cannot be true when the premise is ture", and when the truth value of the hypothesis cannot be inferred by the premise, the text pair is classified as neutral. We train a classifier to predict the truth value of the constructed suppositions mentioned in Section 4.1 QNLI/AdvQNI QNLI. The question natural language inference (QNLI) corpus contains question- context pairs. The task is predict if the answer to given question is contained in its paired
13

---

Under review as a conference paper at ICLR 2023
passage. The regular validation set of QNLI contains 5.6k question- passage pairs for evaluation. AdvQNLI is the adversarial evaluation set proposed by Wang et al. (2021), containing examples generated by textual adversarial attack. The AdvQNLI evaluation set contains 0.9k text examples.
The adaptation from MultiNLJ to QNLI is a cross-task adaptation. The supposition for the task
Sqntt: The answer to question {q} is entailed by (p}-
QQP/AdvQQP. The Quora question pairs (QQP) task is proposed in Wang et al. (0188) 8). In the QQP task, a model learns to predict two questions are duplicated. In most cases, the task is to predict two questions can be answer by the same answer. The regular evaluation 39 lk question-question pairs and the adversarial set contains 0.4k examples. The adaptation from MultiNLI to QQP a cross-tast task adaptation, an the supposition for the QQP task is
of QQP contains
Sqpp: The answer to {9ı } is entailed by the answer {92)-
RTE/AdvRTE. The recognizing textual entailment (RTE) task is binary (entailment/contraaiccis entailment classification task based in the news domain. Each data example is a claim- claim-passage pair and the task is predicting if the claim is entailed by the paired passage or not. The different between RTE and MultiNI.I that the texts RTE is generally longer than MultiNL.I The regular evaluation set of RTE contains 3k text pairs, and the adversarial evaluation set contains 0.3k data examples. The adaptation from MultiNLI to RTE is cross-domain adaptation and the suppositions are the same as MultiNLI suppositions, as defined in Section 4.1
SST2/AdvSST2. The Stanford Sentiment Treebank (SST-2) is a sentimental analysis corpus that requires model to predict if the attitude of movie review is positive negative. The regular evaluation set contains 1.8k test reviews and the adversarial evaluation set contains 1.4k examples. The adaptation from MultiNLI SST-2 is cross-task adaptation with gap between input for- mats. While the inputs of all tasks described above are text pairs, the inputs of SST-2 are movie reviews without any pair text. To apply the pretrained entailment classifier, construct both label descriptions and suppositions. The construct supposition template is
Ssst2: The movie is good is entailed by the comment {x}
B ADDITION IMPLEMENTATION DETAILS
B. HYPERPARAMETER TUNING
The models pretrained and fine-tuned with the AdamW optimizer Loshchilov & Hutter 2017 In pretrained, the learning rate is set to 5e-6 for BERT and 3e-6 for DeBERTa, and the weight decay weight is set to 4e-6 and the weight decay is 1e-2. The learning rates and weight decay coefficients are fixed across all tasks. We tune the baseline self-training models and found that the best epoch number is 6. In GNC self-training, the model automatically decides the epoch number.
le-5. For the downstream self-training step, the learning rate for both models is
B.2 GNC IMPLEMENTATION
We summarize the process of GNC self-training with pseudo code in Algorithm 1
C ADDITIONAL ANALYSIS
Data selection. Lang et al. (022) have mentioned that selecting high-quality data-pseudo label pairs can benefit self-training. We hereby analyze the effect of model confidence-based based data selection (CS) on all tasks we investigate in this work. The experiment results are presented in Figure
We noticed that under setting where a limited number of unlabeled data is used for self-training, the data selection strategy does not guarantee improvement on all tasks. However, we found the the GNC method can also benefit the data selection strategy all tasks except AdvRTE, improving the average performance of 20 experiments and reduce the risk of getting low performance. On QQP, RTE, and SST2, GNC self-training without data selection is still the most robust strategy.
14

---

Under review as a conference paper at ICLR 2023
Algorithm GNC self-training with shifted Geman McClure (SGM)
Require: Pseudo training set D {00, yo), (I1, (.,,),,,..... ..., (In, yn)) model Mo. 2, E
< 2.8
> update step size of
< 100
i <
number of steps
[wn, Wi..... wn. - [1,1, [1,1,
while μ do
D. Calculate training loss >> Calculate the weighted total loss update model parameters using the total loss calculate evaluation loss of training data
L = E;4 E:4..u) wi
Mr+1 f train(M,, Lt)
ej wwws_fnnssaa..sooooo..oooo
ift = () then
μ
max,e,/ ee/xxe//" /c²
initialize j
else if > 0 then
11 6 u/6
update jr
end if
Hea FHE
D. update
end while
10.0
07.
0111
00.0
oxEies
Figure 5: Ablation study of applying confidence-based data selection (CS) baseline and GNC self-training methods on each task with the DeBERTa model.
Ad Adversarial evaluation analysis. We visualize the experiment results with baseline and GNC self- training on AdvQNLI. The results are shown in 6 As the regular evaluation set, the GNC method is also more robust against high pseudo label noises and labeling perplexities.
0.62
6.00
14
6.0
OLC
0.055
8355
8825 3.50 cats 6.00
855
330 6/5
Gáo
3.55
0.00
0.5
E44
Plssuce
beet
66-
4.5
52
Figure 6: Analysis of 20 experiments AdvQNLI with the baseline and GNC self-training.
15


Reviews:
[{"review_id": "ls7KjEogAm", "reviewer": "Reviewer_py5v", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a self-training method based on graduated non-convexity in order to be robust to outliers with wrong pseudo labels and over-confident training samples. Intuitively the core idea of the method is to penalize these two kinds of samples by assigning low loss weights to them. Experimental results show that the proposed method improves the performance and stability of self-training on many different tasks.", "summary_of_the_review": "Given the strength and weaknesses, I tend to rate the paper as marginally above the acceptance.", "strength_and_weaknesses": "Strength\n\n1. The paper is well-written and the method is clearly presented.\n\n2. Though the main technique is mostly adapted from previous work [1], the use of graduated non-convexity in the self-training setting looks novel to me.\n\n3. The experiments are extensive and the ablation study looks solid.\n\nWeaknesses\n\n1. The empirical improvements compared to baselines are not very stable and to some extent marginal in some datasets.\n\n[1] Heng Yang, Pasquale Antonante, Vasileios Tzoumas, and Luca Carlone. Graduated non-convexity for robust spatial perception: From non-minimal solvers to global outlier rejection. IEEE Robotics and Automation Letters, 5(2):1127–1134, 2020.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "hX4tAv-88se", "reviewer": "Reviewer_uzE6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a graduated-non-convexity-based method to improve the fine-tuning performance on NLP tasks. Based on a gradually convexified loss, the models are assigned different weights to the examples in self-training to mitigate the effect of incorrect labels. Experiments indicate the proposed method outperforms standard self-training.", "summary_of_the_review": "Although the idea in this paper is interesting, the weaknesses make it unsatisfactory for iclr as a top venue. Thus, I tend to reject this work.", "strength_and_weaknesses": "The paper is overall well-written, with clearly stated contributions. Applying the GNC in self-training is an interesting idea. \n\nWeaknesses:    \n1.There seems to be many ad-hoc factors introduced into the proposed algorithm. First, changing the GM to SGM cannot ensure that the Black-Rangarajan Duality still holds. In this sense, it is hard to say the SGM loss and the reweighted classification loss are still related. Where does decreasing μ by μ = μ/2.8 come from? Is it 2.8 is the number with the best performance found with experiments?    \n2.Lacking more comparison with baselines. It is well known that standard self-training is not the perfect solution. There are a lot of works such as FixMatch, which can really improve standard self-training by a large margin. The authors should also include these methods in their comparison.    \n3.The experiment setup section is confusing. Why some tasks have 16 labeled examples while some have 12? The backbones of different methods are also different, make it very hard to compare except the last two rows.     \n4.The authors mentioned running each experiment for 20 times and report the mean. Why not report the standard deviation? Is it the stddev is larger than the margin (I’m inferring this from table 4).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "On34nJdTF2", "reviewer": "Reviewer_ZPmQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose an outlier robust self-training method based on graduated non-convexity (GNC) to mitigate the instability issue in conventional self-training. Specifically, they propose new robust cost functions according to Black-Rangarajan Duality. Experimental results on natural language understanding demonstrate the effectiveness of the proposed method.", "summary_of_the_review": "See above", "strength_and_weaknesses": "Strength:\n\n* This paper considers self-training, which is a very important topic and has potentially wide usage.\n\n* The proposed method is easy to understand, although there are still some presentation issues.\n\n* Experimental results seem to demonstrate the effectiveness of the proposed method.\n\n\nWeaknesses:\n\n* Some parts of the paper are not clear.\n\n  * In Section 4.1, the authors mentioned they pre-train a model on MNLI. How is the pre-training conducted? Do you train a model with 3-way classification loss (entailment, neutral, contradiction) using the provided labels in MNLI? Or do you train a model to predict $s^1_{\\text{mnli}}$?\n\n  * Continuing on the previous comment, how is $s^2_{\\text{mnli}}$ related to $s^1_{\\text{mnli}}$? You mentioned these two are contradictory, could you give a concrete example? Also, how do you use $s^2_{\\text{mnli}}$ in the pre-training stage?\n\n  * What is $\\ell_{\\text{max}}$ in the paragraph below Eq. 8?\n\n\n* Some heuristics seem rather arbitrary.\n\n  * For example, in the paragraph below Eq. 8, the schedule of $\\mu$ is defined as $\\mu \\leftarrow \\mu/2.8$. Could the authors provide some intuition about this $2.8$ factor? Or is this purely based on trial and error? And why is training stopped when $\\mu<1$?\n\n  * Also, in Section 5.2, could the authors provide some intuition about the schedule of $\\epsilon$? In particular the additional hyper-parameter $\\alpha \\in [1.2, 2.8]$?\n\n* The technical novelty and contribution is quite limited. The authors simply introduce an additional hyper-parameter $\\epsilon$ into the GNC method. The design of this hyper-parameter is based on intuition, and absolutely no theoretical guarantee is provided,  (which I think is fine if strong enough empirical evidence is provided). \n\n* The authors mention that “we train BERT and DeBERTa models as the pretrained entailment classifiers”. During the entailment pre-training stage, is the model initialized from pre-trained BERT/DeBERTa? If so, the model already contains information that can help downstream tasks such as SST-2.\n\n* The authors mentioned that “we investigate weakly supervised domain and task adaption of textual entailment models”. The authors could run experiments in a more general setting. For example, directly fine-tune on GLUE where only a small proportion (e.g., 10%) of the labels are considered available. \n\nI will raise my score if the authors can address my concerns and run more experiments.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2jj8d4gJZ_", "reviewer": "Reviewer_HQq2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Facing the problem that self-trained models usually exhibit unstable performance influenced by data distribution and pseudo-label accuracy, this paper firstly applies the graduated non-convexity in self-trainng.\nThe authors firstly formulate the NLU tasks to classification tasks and construct supposition for different tasks. Then the suppositions are fed into the pre-trained model to generate pseudo-labels for each data example. As the core, authors proposed the convexity controlled training. The method achieves great improvement on downstream tasks.\n", "summary_of_the_review": "Please see the comments in Strength And Weaknesses, but I am not an expert in NLP domain, some of my adjustment maybe not correct", "strength_and_weaknesses": "Strengths:\n\n(1) Novelty of firsly introducing the GNC into self-training to achieve robustness;\n\n(2) Easy understanding for the suppositions construtions which is aimed at formulate different tasks as uniform classification task.\n\n(3) Great performance which validates the effectiveness of the method.\n\nWeakness:\n\n(1) why the coefficient μ is decreased by 2.8 in Eq. 8? As the preliminary introduced that \"A more convex iteration provides a guess for the following optimization.\" Thus the setting of the decreasing of coefficient should be discussed.\n\n(2) The robustness against incorrect pseudo-labels, imbalanced training data, overfitting, and adversarial evaluation data have not been clearly verified in the experiments.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject