Entailment as Robust Self-Learner (2023)

Abstract: Entailment has been recognized as an important metric for evaluating natural language understanding (NLU) models, and recent studies have found that entailment pretraining benefits weakly supervised fine-tuning. In this work, we design a prompting strategy that formulates a number of different NLU tasks as contextual entailment. This approach improves the zero-shot adaptation of pretrained entailment models. Secondly, we notice that self-training entailment-based models with unlabeled data can significantly improve the adaptation performance on downstream tasks. To achieve more stable improvement, we propose the Simple Pseudo-Label Editing (SimPLE) algorithm for better pseudo-labeling quality in self-training. We also found that both pretrained entailment-based models and the self-trained models are robust against adversarial evaluation data. Experiments on binary and multi-class classification tasks show that SimPLE leads to more robust self-training results, indicating that the self-trained entailment models are more efficient and trustworthy than large language models on language understanding tasks.

## introduction although achieving state-of-the-art performance in different natural language understanding (nlu) tasks @xcite @xcite @xcite , large-scale pretrained language models still highly depend on human-annotated, taskspecific training corpora for fine-tuning because the self-supervised pretraining objective does not incorporate explicit task-related knowledge. as a result, state-of-the-art language models are still challenged by the lack of adequate fine-tuning data and difficult evaluation examples crafted by adver-sarial attacks or model-in-loop adversarial data annotations @xcite @xcite @xcite . on the other hand, entailment is recognized as a minimal requirement for nlu @xcite . recent studies have found that entailment learning improves sentence representation @xcite . however, these models still need fine-tuning with human-annotated training data to handle downstream nlu tasks. the authors of @xcite found that entailment-based models are also few-shot learners that outperform recent efforts on few-shot nlu. for example, lm-bff @xcite proves that entailment learning can significantly improve the data efficiency and adaptation ability of language models. in this work, we further explore the zero-shot and unsupervised adaptation abilities of entailmentbased models without any human-labeled training corpora on downstream tasks. we first study the zero-shot and unsupervised adaptation abilities of the entailment-based language models. inspired by recent progress on prompt tuning, we formulate different nlu tasks as contextual entailment @xcite by constructing task-specific suppositions. the language models are trained to predict the truth value of the constructed suppositions. in zero-shot adaptation experiments, we find this approach significantly outperforms naively concatenating different inputs and labels, proving that the supposition construction method mitigates the distribution gap among different nlu tasks. we further explore the potential of the unsupervised adaptation ability of entailment-based models. we use the pretrained entailment models to predict the pseudo-labels of unlabeled, task-specific language data. we find that the entailment-based models can be improved with self-training @xcite with the automatically annotated pseudo-labels @xcite . while the selftraining strategy has been proven effective on different tasks and modalities @xcite @xcite , a major challenge for self-training is the unstable performance caused by the noisy pseudo-labels. a number of solutions have been proposed to mitigate this issue. the most popular methods are training data selection @xcite and pseudo-label editing @xcite . recent work also found that simple dropout @xcite approaches improve contrastive learning @xcite and speech recognition @xcite . to combine the benefits of data selection and label editing methods, we propose simple, a simple pseudo-label editting algorithm with simple text augmentation, uncertainty-based data filtering, and majority-based pseudo-labeling. experiments with different backbone models on binary, multi-class, regular, and adversarial nlu tasks show that our approach makes the following contributions, task inputs supposition mnli {p, h} h is entailed by p. rte {p, h} h is entailed by p. qnli {t, q} the answer to q is entailed by t. qqp {q1, q2} q1's answer is entailed by q2's answer. sst2 {x} the movie is good is entailed by x. by training the entailment model using the mnli corpus given with the constructed suppositions, the model can be directly adapted to other tasks with relatively high accuracy. we will show that without entailment pretraining, similar performance can only be achieved by 400 times bigger language models. the entailment-based models can be further fine-tuned on unlabeled texts via self-training. we apply different adaptation strategies for binary and multi-class classification tasks. binary classification. supposition-based entailment models predict true, neutral, and false scores for each supposition, corresponding to entail, neutral, and contradictory labels of the mnli corpus. for binary classification, we ignore the neutral score and calculate only true and false probabilities, and the true/false predicted can be linked to corresponding labels according to the supposition. for example, the sst2 supposition in to predict the correct answer from multiple options, we propose an entailment score ranking method. first, for each sentence to be classified, we construct a supposition for each label. for example, in an emotion classification task, given the sentence s, we construct the following suppositions: "i am happy is entailed by s", "i am sad is entailed by s", and "i am shocked is entailed by s". we calculate the entailment probability of each supposition with the entailment model and predict the label associated with the most entailed supposition. we propose a max-confidence tuning method for self-training. we select the class with the highest entailment score and then record its predicted pseudo-label for further self-training, and ignore other classes. the model does not need to classify each class correctly but merely learns to predict the truth value of its most confident supposition. ## simple pseudo-label editing we propose the simple pseudo-label editing (sim-ple) method, a three-step pipeline for generating robust pseudo labels, including augmented pseudolabeling with dropout, uncertain data filtering, and majority-based relabeling. we introduce the details of each step in this section. ## experiments benchmarks. we conduct experiments on popular natural language understanding tasks in the glue @xcite benchmark, including rte @xcite , qnli @xcite , qqp, sst-2 @xcite , and cola @xcite . we also assess the robustness of the proposed method against adversarial evaluation sets in the advglue corpus @xcite , including adv-qnli, adv-qqp, adv-rte, and adv-sst2. the data in advglue is created by adding word-level and sentence-level perturbations to the glue data, as well as humancrafted examples. for multi-classification, we use copa (alex wang, 2019) (which consists of questions paired with two answer choices), emotion classification (elvis saravia, 2018), amazon review (phillip keung, 2020) and ag-news (xiang zhang, 2015). more details are shown in appendix a. hyper-parameters. we train 350m roberta @xcite and deberta @xcite ) models for the language understanding tasks, without using larger language models like gpt-3 @xcite or t0 @xcite that are used for generating pseudo-labels in @xcite . we also use the same hyperparameters across all tasks, attempting to avoid the problems mentioned in @xcite . in the entailment pretraining on the mnli dataset @xcite , we optimize both roberta and deberta models with the adamw optimizer @xcite . for all tasks and both models, we set ε = 10 -6 . in the entailment pretraining, we set the weight decay weight to 10 -5 , and the learning rate for both models is 3e-6. during the self-training step, the learning rate of both models on all binary classification tasks is 4e-6 and is 1e-6 on multi-classification tasks, and the weight decay is constantly 10 -2 . we run the entailment pretraining for 2 epochs and the selftraining for 6 epochs. in confidence-based labeling, we drop 1/8 data with the lowest confidence. self-training details. for each binary classification task, we randomly select @xmath0. for each multi-classification task, we randomly select @xmath1. to estimate the uncertainty of the pseudolabels in setred and simple algorithms, we use the hidden states of the 4th layer from the top of both roberta and deberta language models as the supposition embeddings and measure the uncertainty with 9 neighbors. in simple, we run 7 inferences for each training sample with different dropouts. we train and evaluate the models for each task with 10 independent runs on 2 v100 32g gpus. each experiment takes less than an hour. assessment. we evaluate the performance of our algorithm by comparing the average classification accuracy against baseline methods and the robustness. we describe the term robustness as follows: in multiple independent experiments, a robust method should achieve high maximum, minimum, and average accuracy against with different backbone model and training data, on different natural language understanding tasks. ## analysis robustness. besides the mean accuracy of all experiments, we also visualize the results of all independent evaluations of different self-training strategies in figure 2 . we found that simple constantly outperforms other self-training baselines on the regular glue benchmark by comparing most models overfit to the training corpora and achieve high accuracy on regular evaluation sets, but perform poorly on adversarial benchmarks @xcite . as a result, fully supervised models achieve less than 60% accuracy on ad-vglue. we also investigate if simple hurts the model's robustness against adversarial evaluation data. we found that, except roberta on advqqp, other settings show that the entailment-based models are still robust after simple self-training. as we compared in we also show that the original pseudo-label set is unbalanced, with 67.1% of all predicted labels being "false". although we do not provide any prior knowledge about the label distribution of the task (unknown without human annotation), the simple method mitigates the bias through the uncertain candidate removal process. figure 4 shows that most uncertain pseudo-labels estimated by sim-ple are "false", thus the remaining pseudo-labels are more balanced. ## conclusion we show that entailment-based language models can be adapted to different nlu tasks without supervision and achieve robust performance against noisy pseudo-labels and adversarial texts. we design a supposition-based prompting strategy to improve the zero-shot adaptation performance of entailment-based models. to improve the stability of self-training, we propose the simple algorithm for augmented pseudo-labeling. experiments on binary, multi-class, regular, and adversarial nlu tasks show that the simple self-training strategy significantly outperforms a number of strong baselines, including 400 and 500 times larger language models on both zero-shot and weakly supervised settings, proving the effectivenss of entailment selftraining for efficient and trustworthy natural language understanding systems.