Entailment as Robust Self-Learner
Year: 2023

Abstract:
Entailment has been recognized as an important metric for evaluating natural language understanding (NLU) models, and recent studies have found that entailment pretraining benefits weakly supervised fine-tuning. In this work, we design a prompting strategy that formulates a number of different NLU tasks as contextual entailment. This approach improves the zero-shot adaptation of pretrained entailment models. Secondly, we notice that self-training entailment-based models with unlabeled data can significantly improve the adaptation performance on downstream tasks. To achieve more stable improvement, we propose the Simple Pseudo-Label Editing (SimPLE) algorithm for better pseudo-labeling quality in self-training. We also found that both pretrained entailmentbased models and the self-trained models are robust against adversarial evaluation data. Experiments on binary and multi-class classification tasks show that SimPLE leads to more robust self-training results, indicating that the self-trained entailment models are more efficient and trustworthy than large language models on language understanding tasks.

heading: 1 Introduction
text: Although achieving state-of-the-art performance in different natural language understanding (NLU) tasks (Devlin et al., 2018; Liu et al., 2019; Yang et al., 2019; Clark et al., 2020; He et al., 2020; Joshi et al., 2020), large-scale pretrained language models still highly depend on human-annotated, taskspecific training corpora for fine-tuning because the self-supervised pretraining objective does not incorporate explicit task-related knowledge. As a result, state-of-the-art language models are still challenged by the lack of adequate fine-tuning data and difficult evaluation examples crafted by adver-
∗ Equal contribution. Correspondence to Hongyin Luo at hyluo@mit.edu. Code and processed data are available at https://github.com/luohongyin/EntST.
sarial attacks or model-in-loop adversarial data annotations (Wang et al., 2021a; Bartolo et al., 2020; Zang et al., 2019; Garg and Ramakrishnan, 2020; Li et al., 2020).
On the other hand, entailment is recognized as a minimal requirement for NLU (Condoravdi et al., 2003). Recent studies have found that entailment learning improves sentence representation (Reimers and Gurevych, 2019a; Gao et al., 2021). However, these models still need fine-tuning with human-annotated training data to handle downstream NLU tasks. The authors of Wang et al. (2021b) found that entailment-based models are also few-shot learners that outperform recent efforts on few-shot NLU. For example, LM-BFF (Gao et al., 2020) proves that entailment learning can significantly improve the data efficiency and adaptation ability of language models.
In this work, we further explore the zero-shot and unsupervised adaptation abilities of entailmentbased models without any human-labeled training corpora on downstream tasks. We first study the zero-shot and unsupervised adaptation abilities of the entailment-based language models. Inspired by recent progress on prompt tuning, we formulate different NLU tasks as contextual entailment (Routley and Meyer, 1973) by constructing task-specific suppositions. The language models are trained to predict the truth value of the constructed suppositions. In zero-shot adaptation experiments, we find this approach significantly outperforms naively concatenating different inputs and labels, proving that the supposition construction method mitigates the distribution gap among different NLU tasks.
We further explore the potential of the unsupervised adaptation ability of entailment-based models. We use the pretrained entailment models to predict the pseudo-labels of unlabeled, task-specific language data. We find that the entailment-based models can be improved with self-training (Blum and Mitchell, 1998) with the automatically annotated
13803
pseudo-labels (He et al., 2019). While the selftraining strategy has been proven effective on different tasks and modalities (Zou et al., 2019; Zoph et al., 2020; Meng et al., 2020; Xie et al., 2020b), a major challenge for self-training is the unstable performance caused by the noisy pseudo-labels. A number of solutions have been proposed to mitigate this issue. The most popular methods are training data selection (Li and Zhou, 2005; Lang et al., 2022) and pseudo-label editing (Shin et al., 2020; Mandal et al., 2020). Recent work also found that simple Dropout (Srivastava et al., 2014) approaches improve contrastive learning (Gao et al., 2021) and speech recognition (Khurana et al., 2021; Dawalatabad et al., 2022).
To combine the benefits of data selection and label editing methods, we propose SimPLE, a simple pseudo-label editting algorithm with simple text augmentation, uncertainty-based data filtering, and majority-based pseudo-labeling. Experiments with different backbone models on binary, multi-class, regular, and adversarial NLU tasks show that our approach makes the following contributions,
• Supposition-based task formulation improves the zero-shot adaptation and robustness against adversarial evaluation data of entailment models across different NLU tasks.
• SimPLE improves the pseudo-labeling accuracy on confident and uncertain training samples, leading to significant improvement over all self-training and pretrained baselines.
• Self-trained, 350M-parameter entailment models without human-generated labels outperform supervised language models with 137B parameters, proving the data and computation efficiency of entailment self-training.

heading: 2 Related Work
text: Language modeling. Task-agnostic, large-scale language models can solve a number of natural language understanding (NLU) tasks (Brown et al., 2020; Raffel et al., 2020; Lewis et al., 2019; Wei et al., 2022a,b). On the other hand, pretraining with annotated training corpora of different natural language tasks also benefits the generalize ability and zero-shot adaptation performance (Sanh et al., 2021). Recent studies have found that textual entailment (Bowman et al., 2015; Williams et al., 2018) is a powerful pretraining task. Entailment models are applied for sentence representation learning
(Reimers and Gurevych, 2019b; Gao et al., 2021), relation extraction (Obamuyide and Vlachos, 2018; Yin et al., 2019), and fact-checking (Thorne and Vlachos, 2018). The authors of Wang et al. (2021b) showed that entailment models can benefit the fewshot learning performance of pretrained language models on NLU tasks. Robustness in Self-training. While most selftraining studies are under the computer vision context (Zoph et al., 2020; Zou et al., 2019), efforts also exist for self-training the latest neural language models, including back translation (He et al., 2019), text augmentation (Xie et al., 2020a; Chen et al., 2020), question-answer synthesis (Bartolo et al., 2021; Luo et al., 2022), and co-training (Lang et al., 2022). However, self-training methods suffer from noisy pseudo-labels. In computer vision, a straightforward solution is obtaining confident pseudo-labels by augmenting input images (Shin et al., 2020; Mandal et al., 2020; Sohn et al., 2020), including shifting, rotating, or adding noise to pixels. However, data augmentation is not as straightforward for natural language if no additional model is used. Instead, some model-level methods can be applied. Zou et al. (2019) proposed regularizing over pseudo-label confidence to avoid overfitting to simple cases, Gao et al. (2021); Khurana et al. (2021) applied dropout to improve the quality of training corpora. Li and Zhou (2005); Lang et al. (2022) applied a graph-based confidence estimation method for removing training samples with uncertain pseudo labels. Difference with previous work. Without any additional language model for text augmentation, we propose a model-level, augmented pseudo-labeling method that improves self-training performance for entailment models. Our method avoids dropping training data and performs more stably than dropout-based methods. Different from previous work on weakly-supervised language understanding with entailment models (Wang et al., 2021b), we do not use any human-generated labels. Our models contain 1/500 trainable parameters compared to the models used in Lang et al. (2022); Sanh et al. (2021).

heading: 3 Entailment Self-training
text: Pretraining. Recent studies have found that entailment-based language models can efficiently adapt to different natural language understanding (NLU) tasks with a limited number of human-
labeled training samples (Wang et al., 2021b; Luo and Glass, 2023). In this work, we find that entailment models can be self-improved without any human-generated labels by constructing suppositions (prompts) that describe the given tasks. Most NLU tasks can be formulated as predicting the truth value of the constructed suppositions that wrap inputs and label descriptions, as shown in Table 1.
By training the entailment model using the MNLI corpus given with the constructed suppositions, the model can be directly adapted to other tasks with relatively high accuracy. We will show that without entailment pretraining, similar performance can only be achieved by 400 times bigger language models. The entailment-based models can be further fine-tuned on unlabeled texts via self-training. We apply different adaptation strategies for binary and multi-class classification tasks. Binary classification. Supposition-based entailment models predict True, Neutral, and False scores for each supposition, corresponding to entail, neutral, and contradictory labels of the MNLI corpus. For binary classification, we ignore the neutral score and calculate only True and False probabilities, and the True/False predicted can be linked to corresponding labels according to the supposition. For example, the SST2 supposition in Table 1 being true means that {x} is a positive movie review. The predicted True/False values are used as pseudo-labels for self-training, Multi-class classification. In binary classification, the model is presented with a single supposition and asked to decide whether it’s true or not. In multi-class classification, the model is presented with a context sentence and multiple labels and is asked to choose the correct label.
To predict the correct answer from multiple options, we propose an entailment score ranking method. First, for each sentence to be classified, we construct a supposition for each label. For example, in an emotion classification task, given the sentence
S, we construct the following suppositions: "I am happy is entailed by S", "I am sad is entailed by S", and "I am shocked is entailed by S". We calculate the entailment probability of each supposition with the entailment model and predict the label associated with the most entailed supposition.
We propose a max-confidence tuning method for self-training. We select the class with the highest entailment score and then record its predicted pseudo-label for further self-training, and ignore other classes. The model does not need to classify each class correctly but merely learns to predict the truth value of its most confident supposition.

heading: 4 Simple Pseudo-label Editing
text: We propose the simple pseudo-label editing (SimPLE) method, a three-step pipeline for generating robust pseudo labels, including augmented pseudolabeling with dropout, uncertain data filtering, and majority-based relabeling. We introduce the details of each step in this section.

heading: 4.1 Simple Augmentation for Pseudo-labeling
text: Because of languages’ discrete and sequential nature, changing a token in a sentence might completely invert its meaning. As a result, unlike straightforward and effective image augmentation processes like FixMatch (Sohn et al., 2020), additional augmentation models are usually needed for text augmentation. Recent studies have found that instead of data-level augmentation, the Dropout mechanism leads to decent embedding-level augmentation. Gao et al. (2021) applied dropout for contrastive sentence representation learning, and Khurana et al. (2021) selected confident pseudolabels by measuring the consistency of a model with the same input data and random dropouts.
As the first step of generating augmented pseudo labels, we run N independent evaluations with random dropout (dropout rate = 0.1) for each input training sample xi and obtain a set of N noisy pseudo-labels.
Yi = {yj = M∗j (xi) | j ∈ [0, N)} (1)
where j stands for the j-th independent evaluation with a dropout model M∗. Meanwhile, we store a set of sequence representations Ei = {e0, e1, . . . , eN−1} of xi collected in each feedforward process. After finishing this step, we collect a set of data, pseudo-label, and embeddings.
C = {(xi, yji , e j i ) | i ∈ [0,M), j ∈ [0, N)} (2)
where M stands for the number of unlabeled training samples, each associated with N pseudo-labels and corresponding hidden states. In total, the augmented method outputs M ∗ N label-embedding pairs for further processing.

heading: 4.2 Uncertainty Estimation
text: Following Li and Zhou (2005) and Lang et al. (2022), we estimate the confidence of all pseudolabels using the SETRED algorithm. The motivation of this algorithm is that training samples with similar embeddings are likely to have the same pseudo-labels. On the other hand, if a training sample is located near samples with different pseudolabels in the embedding space, its own pseudolabel is likely to be uncertain. Using the output data-embedding-label set shown in Equation 2, we can calculate the nearest neighbors of each training sample and estimate the labeling consistency.
To estimate the uncertainty of yu, the pseudolabel of training sample xu, we calculate the Euclidean distances between xu and all other M ∗N− 1 samples using the calculated text embeddings. We construct a set of the top k nearest neighbors of xu, namely N(u). With the nearest neighbor set, an uncertain score of (xu, yu) can be calculated as follows,
Ju = ∑
v∈N(u) I(yu ̸= yj) / (1 + ∥eu − ev∥2) (3)
where I is a binary indicator function, whose value is 1 when yu ̸= yv and 0 otherwise. ∥eu − ev∥2
stands for the Euclidean distance between the embeddings of xu and xv. As a result, Ju would have a higher value when more near neighbors are associated with different pseudo-labels.
To estimate the uncertainty of yu, we compare Ju with a null hypothesis where all pseudo-labels in C except yu are randomly shuffled. After the shuffling, the entire data-label mapping set becomes uncertain. The expectation and variance of Ju after shuffling is
Ev[Ju] = (1− P̂yu) ∑
v∈N(u) 1/(1 + ∥eu − ev∥2)
σ(Ju) 2 = P̂yu(1−P̂yu)
∑
v∈N(u) 1/(1+∥eu−ev∥2)2
The uncertainty can be estimated by verifying the significance of the difference between Ju and the null hypothesis. An uncertainty score can be calculated as
s(u) = Ju − Ev[Ju]
σ(Ju) (4)
With this method, we calculate uncertainty scores for all M ∗ N training samples in C for further processing.

heading: 4.3 Filtering and Relabeling
text: After finishing estimating the uncertainty of each training sample, we sort all training samples in C by their uncertainty scores and remove the 20% most uncertain training samples. The remaining samples are used for relabeling based on majority voting. For example, a training sample xi has
N pseudo-labels [yi0, y i 1, . . . , y i N−1] after the augmented labeling step, and n labels are removed based on the uncertainty scores.
The final pseudo-label of xi is decided by the voting result of the N − n remaining labels. If all generated pseudo-labels of a training sample are removed or there is a tie in the voting, we re-run the labeling process without dropout to get the final pseudo-label. Following this approach, we keep all training samples and, meanwhile, obtain a more robust pseudo-label set.

heading: 5 Experiments
text: Benchmarks. We conduct experiments on popular natural language understanding tasks in the GLUE (Wang et al., 2018) benchmark, including RTE (Dagan et al., 2005), QNLI (Rajpurkar et al., 2016), QQP, SST-2 (Socher et al., 2013), and CoLA (Warstadt et al., 2019). We also assess the robustness of the proposed method against adversarial evaluation sets in the AdvGLUE corpus (Wang et al., 2021a), including Adv-QNLI, Adv-QQP, Adv-RTE, and Adv-SST2. The data in AdvGLUE is created by adding word-level and sentence-level perturbations to the GLUE data, as well as humancrafted examples. For Multi-Classification, we use Copa (Alex Wang, 2019) (which consists of questions paired with two answer choices), Emotion Classification (Elvis Saravia, 2018), Amazon Review (Phillip Keung, 2020) and Ag-News (Xiang Zhang, 2015). More details are shown in Appendix A. Hyper-parameters. We train 350M RoBERTa (Devlin et al., 2018) and DeBERTa (He et al., 2020) models for the language understanding tasks, without using larger language models like GPT-3 (Brown et al., 2020) or T0 (Sanh et al., 2021) that are used for generating pseudo-labels in (Lang et al., 2022). We also use the same hyperparameters across all tasks, attempting to avoid the problems mentioned in Perez et al. (2021). In the entailment pretraining on the MNLI dataset (Williams et al., 2018), we optimize both RoBERTa and DeBERTa models with the AdamW optimizer (Loshchilov and Hutter, 2018). For all tasks and both models, we set ε = 10−6. In the entailment pretraining, we set the weight decay weight to 10−5, and the learning rate for both models is 3e6. During the self-training step, the learning rate of both models on all binary classification tasks is 4e-6 and is 1e-6 on multi-classification tasks, and
the weight decay is constantly 10−2. We run the entailment pretraining for 2 epochs and the selftraining for 6 epochs. In confidence-based labeling, we drop 1/8 data with the lowest confidence. Self-training details. For each binary classification task, we randomly select N = 2000 unlabeled data examples. For each multi-classification task, we randomly select N = 50 unlabeled data examples. To estimate the uncertainty of the pseudolabels in SETRED and SimPLE algorithms, we use the hidden states of the 4th layer from the top of both RoBERTa and DeBERTa language models as the supposition embeddings and measure the uncertainty with 9 neighbors. In SimPLE, we run 7 inferences for each training sample with different dropouts. We train and evaluate the models for each task with 10 independent runs on 2 V100 32G GPUs. Each experiment takes less than an hour. Assessment. We evaluate the performance of our algorithm by comparing the average classification accuracy against baseline methods and the robustness. We describe the term Robustness as follows: in multiple independent experiments, a robust method should achieve high maximum, minimum, and average accuracy against with different backbone model and training data, on different natural language understanding tasks.

heading: 5.1 GLUE and AdvGLUE Tasks
text: The experiment results are shown in Table 2. We compare the adaptation performance of entailmentbased language models and the improvement of different self-training approaches. Compare with supervised baselines. We compare our entailment self-training methods with few-shot fine-tuning baselines. The few-shot baselines, including PET (Schick and Schütze, 2021), LM-BFF (Gao et al., 2020), P-tuning (Liu et al., 2021), PPT (Gu et al., 2021), and UPT (Wang et al., 2022), are based on 350M BERT or RoBERTa backbones. Our pretrained DeBERTa entailment model outperforms the best few-shot baseline (LM-BFF) by 4.5%, and the RoBERTa entailment model outperforms LM-BFF by 1.5%. With self-training, our SimPLE method further improves the model’s performance by a large margin. The RoBERTa performance is boosted by nearly 5% and the average performance of DeBERTa is over 86%, outperforming the best few-shot supervised baselines by 6.9%.
On the other hand, we compare our model with fully supervised RoBERTa/DeBERTa models and
robust training methods, including R3F (Aghajanyan et al., 2020), child tuning (CT) (Xu et al., 2021), and match tuning (MT) (Tong et al., 2022) models, on the AdvGLUE benchmark. We found that the fully-supervised DeBERTa model is the best baseline on the AdvGLUE benchmark. However, our RoBERTa entailment model outperforms all robust training baselines with the same pretrained backbone by over 10%. With SimPLE selftraining, the DeBERTa entailment model achieves the best performance on AdvGLUE, outperforming the fully-supervised DeBERTa model by 2.1% as well as all other baselines.
We found that our pretrained entailment models outperform EFL, the few-shot fine-tuned entailment model based on RoBERTa-large proposed by Wang et al. (2021b). The self-trained models further outperform EFL with larger margins. This indicates the strong adaptation ability introduced by the supposition-based NLU strategy.
Compare with large language models. We found that both zero-shot pretrained and semi-supervised self-trained entailment models outperform the fewshot large language models on QNLI, QQP, and RTE tasks, and achieve significantly higher average accuracy on GLUE. This suggests that our method
is computation-efficient - the models use 1/400 parameters, without human-generated task-specific labels, but achieve better performance than expensive large-scale language models on NLU tasks.
Compare with self-training baselines. By averaging 10 independent evaluations across GLUE and AdvGLUE benchmarks and backbone models, we found that Dropout and SETRED improve baseline self-training performance on a similar level. On average, SETRED outperforms Dropout by 0.5% on 4 experiment settings. On the GLUE benchmark, the SimPLE method improves the model’s performance by 1.5 to 2% on average. The highest improvement boost is on the QNLI tasks, where the SimPLE self-training method outperforms the baseline self-training by 9% and 6% on RoBERTa and DeBERTa respectively. Although the average improvement is not very high, we will show that SimPLE is significantly more robust. The results show that augmenting the pseudo-labels without removing uncertain training samples benefits selftraining, which aligns with our hypothesis.
In general, the experiments on binary classification NLU tasks proved the data and computation efficiency of entailment self-training over different strong baseline models. Furthermore, the SimPLE
algorithm we propose in this work achieves the best average performance, significantly outperforms all baselines on some of the tasks, and meanwhile preserves the robustness of entailment models against adversarial benchmarks.

heading: 5.2 Multi-class NLU Tasks
text: The experiment results on Copa, Emotion, Amazon Review, and Ag News are shown in Table 3. In multi-classification tasks, we present the comparative results of the pretrained entailmentbased language models and the 4 self-training approaches compared in the previous section with binary NLU tasks, including standard self-training, dropout-based re-labeling, SETRED, and SimPLE. The effect of dropout-based augmentation. By merely using dropout, the augmented self-training outperforms the standard normal self-training baseline which keeps all the pseudo-labels in general. This further validates the previous finding that by adding dropout, the models adopt noises that benefit the inference, generate augmented pseudo-labels and mitigate the overfitting problem. The effect of SETRED. By merely using SETRED, the self-training does not see a consistent improvement in performance and even falls behind the pretrained and standard self-trained models that preserve all pseudo labels in some tasks (like AmazonReview). This fact suggests that removing uncer-
tain pseudo-labels can lead the model to overfit confident training samples, thus hurting the selffine-tuning performance. The effect of SimPLE. Table 3 shows that the SimpLE algorithm constantly outperforms all pretrained and self-trained baselines on both backbone models across all multi-class benchmarks, which aligns with the result on the binary NLU tasks. This fact further validates our hypothesis that augmenting the pseudo-labels of uncertain training samples can improve the performance of self-training. Compare with Large Language Models. We notice that our self-trained methods can outperform several large language models. On Emotion and AG News tasks, the pretrained entailment model without self-training can achieve a significant improvement over the GPT-3-175b model, which is 500 times large than the entailment model. This indicates that the entailment-based model is a more efficient and trustworthy option for many natural language understanding tasks.

heading: 6 Analysis
text: Robustness. Besides the mean accuracy of all experiments, we also visualize the results of all independent evaluations of different self-training strategies in Figure 2. We found that SimPLE constantly outperforms other self-training baselines on the regular GLUE benchmark by comparing
mean, maximum, and minimum accuracy. Although DeBERTa performs similarly under different self-training strategies on QQP in terms of average accuracy, there exists a significant gap between the minimal performance of baseline and SimPLE. This indicates that SimPLE is more robust and safer compared with the regular self-training algorithm. The only exception is the DeBERTa model on SST2 - the mean performance of SimPLE is better, but it has a lower minimal performance than the baseline self-training method.
Most models overfit to the training corpora and achieve high accuracy on regular evaluation sets, but perform poorly on adversarial benchmarks (Wang et al., 2021a). As a result, fully supervised models achieve less than 60% accuracy on AdvGLUE. We also investigate if SimPLE hurts the model’s robustness against adversarial evaluation data. We found that, except RoBERTa on AdvQQP, other settings show that the entailment-based models are still robust after SimPLE self-training. As we compared in Table 2, all these results significantly outperform fully-supervised baselines. Pseudo-labeling Accuracy. We show the pseudolabeling accuracy of RoBERTa and DeBERTabased entailment models with different strategies in Figure 3 with 10 independent experiments. The results indicate that the DeBERTa models predict more accurate pseudo-labels in general. On the
other hand, the pseudo-label sets produced by SimPLE with both models are significantly less noisy than the standard and dropout-based labeling methods without removing any uncertain data samples. SETRED achieves the highest labeling accuracy because it drops uncertain samples. The comparison suggests that SimPLE achieves the highest performance because it achieves high pseudo-labeling accuracy on uncertain training samples. Case study. We visualize the hidden states, pseudolabels, and confidence of the training samples in the QNLI tasks calculated by the pretrained DeBERTa entailment model with the SimPLE algorithm in Figure 4. The embedding space is calculated with tSNE (Van der Maaten and Hinton, 2008) using 252 training samples with 252*7=1764 embeddings. Half of them are plotted in the figure. Each training sample is evaluated with 7 different dropouts, and the uncertainty is estimated with 9 neighbors. In
Figure 4, different embeddings of the same training sample are labeled with the same color, while the uncertain cases are marked in black. + and - stand for the truth value of the suppositions. As shown in the figure, most uncertain cases appear around the uncertain circle. We also highlight two training samples with uncertain representations. This phenomenon indicates that the SimPLE algorithm can drop most embeddings of a data sample and edit the voting results of the dropout-based pseudo-labeling method, improving the pseudo-labeling accuracy from 76.5% to 79.2% in this experiment.
We also show that the original pseudo-label set is unbalanced, with 67.1% of all predicted labels being “False”. Although we do not provide any prior knowledge about the label distribution of the task (unknown without human annotation), the SimPLE method mitigates the bias through the uncertain candidate removal process. Figure 4 shows that most uncertain pseudo-labels estimated by SimPLE are “False”, thus the remaining pseudo-labels are more balanced.

heading: 7 Conclusion
text: We show that entailment-based language models can be adapted to different NLU tasks without supervision and achieve robust performance against noisy pseudo-labels and adversarial texts. We design a supposition-based prompting strategy to improve the zero-shot adaptation performance of entailment-based models. To improve the stability of self-training, we propose the SimPLE algorithm for augmented pseudo-labeling. Experiments on binary, multi-class, regular, and adversarial NLU tasks show that the SimPLE self-training strategy significantly outperforms a number of strong baselines, including 400 and 500 times larger language models on both zero-shot and weakly supervised settings, proving the effectivenss of entailment selftraining for efficient and trustworthy natural language understanding systems.
Limitations
Our method utilized pretrained entailed models and adapted them to other domains under zeroshot and self-training settings. There are two limitations that we would like to improve in future work. Firstly, we use human-designed suppositions for each task, which is less automatic than a direct, zero-shot adaptation of the models. Secondly, the self-training on some multi-class classification
tasks is not as high as on binary NLU tasks, indicating the challenge of applying entailment models to multi-choice tasks. We would like to overcome this in the next step.
Ethics Statement
We propose a method that can significantly reduce the financial and environmental cost of language model learning. By reducing the need for data collection and human labeling, our method can effectively protect user and data privacy by avoiding leaking any information while building the training corpora. We found that a medium-sized language model can achieve similar performance as the stateof-the-art large-scale language models, suggesting that we can cost less financially and environmentally during model training and evaluation for comparable performance. However, since we reduced the need for human-labeling efforts, the deployment of the system might decrease the number of data annotation jobs.

heading: A Data Details
text: GLUE/AdvGLUE In this work, we evaluate our method with the GLUE1 and AdvGLUE2 benchmarks. We pretrain our models on MNLI, and evaluate on all other AdvGLUE tasks, AdvQNLI, AdvQQP, AdvRTE, and AdvSST2. we also evaluate the models on the regular versions of these tasks in GLUE. The statistics of the GLUE and AdvGLUE benchmarks are shown in Table 4. Multi-Classification In multi-class classification tasks, we evaluate our method with SuperGlue Copa(Alex Wang, 2019), Emotion Classification(Elvis Saravia, 2018), and Amazon Review(Phillip Keung, 2020). The statistics of these corpora are shown in Table 5.
1https://gluebenchmark.com/ 2https://adversarialglue.github.io/
Reproducibility
Data. We introduce the tasks and corpora we used for training and evaluation in Section 5 and Appendix A. Method. We introduce the difference between our method and previous work in Section 2, the details of our method in Section 3 and 4. Hyper-parameter. We describe the key hyperparameters of self-training in Section 5. Experiments. We describe the experiment results, and number of independent runs in Section 5, and Section 6 to prove the statistical significance.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: The section after conclusion

heading: 3 A2. Did you discuss any potential risks of your work?
text: The section after the Limitation sectioon

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstraction & section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4 to 6, used Huggingface transformers and PyTorch packages.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Yes, section 5
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? The packages are widely used for public research.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? The packages are widely used for public research.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? The evaluation corpora are widely used for public research.
7 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? The packages are widely used for public research.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Appendix
C 3 Did you run computational experiments? Section 5 and 6

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 5
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 5

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 5 and 6

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Sectioon 5
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: There are two limitations that we would like to improve in future work. Firstly, we use human-designed suppositions for each task, which is less automatic than a direct, zero-shot adaptation of the models. Secondly, the self-training on some multi-class classification
tasks is not as high as on binary NLU tasks, indicating the challenge of applying entailment models to multi-choice tasks. We would like to overcome this in the next step