V erify M atch: A Semi-Supervised Learning Paradigm for Natural Language Inference with Confidence-Aware M ix U p (2024)

Abstract: While natural language inference (NLI) has emerged as a prominent task for evaluating a model’s capability to perform natural language understanding, creating large benchmarks for training deep learning models imposes a significant challenge since it requires extensive human annotations. To overcome this, we propose to construct pseudo-generated samples (premise-hypothesis pairs) using class-specific fine-tuned large language models (LLMs) thereby reducing the human effort and the costs in annotating large amounts of data. However, despite the impressive performance of LLMs, it is necessary to verify that the pseudo-generated labels are actually correct. Towards this goal, in this paper, we propose VerifyMatch, a semi-supervised learning (SSL) approach in which the LLM pseudo-labels guide the training of the SSL model and, at the same time, the SSL model acts as a verifier of the LLM-generated data. In our approach, we retain all pseudo-labeled samples, but to ensure unlabeled data quality, we further propose to use MixUp whenever the verifier does not agree with the LLM-generated label or when they both agree on the label but the verifier has a low confidence—lower than an adaptive confidence threshold. We achieve competitive accuracy compared to strong baselines for NLI datasets in low-resource settings.

## introduction natural language inference (nli) @xcite aims to determine the relation between two sentences (referred as premise and hypothesis)whether it is entailment, neutral, or contradiction. nli plays a pivotal role in assessing a model's ability to perform natural language understanding (nlu) and reasoning. the advancement of nli has been fueled, in part, by the creation of large datasets such as snli @xcite , mnli @xcite , and anli @xcite for training massive deep learning models. however, creating a large-scale nli benchmark requires a considerable amount of human effort. this is because human annotators have to generate texts that demand logical inferences. for example, in the creation of the snli and mnli datasets @xcite , human workers receive unlabeled premises and are prompted to generate hypotheses, one per class, for each class label-entailment, neutral, contradiction. similarly, in the creation of the anli dataset @xcite , human annotators receive an unlabeled premise and a target label, and are asked to generate a hypothesis that deceives a model into producing a misclassified prediction of the given target label. in this manner, creating new large-scale nli datasets becomes a burdensome task. hence, the high cost and difficulty of collecting labeled data for nli has driven interest in semi-supervised learning (ssl), which effectively utilizes both labeled and unlabeled data. however, the nature of unlabeled data for ssl on nli is more complex compared to single-sentence classification tasks. this is because one of the sentences in the pair (usually the hypothesis) along with the class label, is missing from the data and requires intensive human annotations as described above. therefore, in order to leverage unlabeled data for ssl on nli, the unavailability of both hypotheses and class labels must be tackled. to overcome this, we propose to leverage large language models (llms) to generate missing hypotheses and to assign initial pseudo-labels where we create readily available unlabeled data for ssl on nli. however, llms may not always generate the most relevant or accurate output. hence, we further propose to leverage pseudo-labeling @xcite to ensure the quality of the generated hypotheses and their assigned labels. pseudolabeling is a widely used semi-supervised learning method that automatically assigns pseudo-labels to unlabeled data and incorporates them into model training. prior research on pseudo-labeling generally employs a pre-defined high threshold for all classes, which assumes pseudo-labels with confidence above the threshold are of high quality and hence beneficial for training while others are of low quality so are discarded @xcite @xcite . thus, this approach results in restricting access to a considerable amount of samples. to address this issue, @xcite propose to use adaptive thresholds for different classes to encourage a model to learn from more diverse samples and achieve better performance in low-resource settings compared to approaches that use a fixed high confidence threshold @xcite . despite their promising results when using flexible thresholds, many pseudo-labeled samples are still discarded. @xcite propose to use all pseudo-labeled samples by assigning lower weights to unconfident pseudo-labeled samples during training. although the diversity of training data increases substantially compared to previous works, there are still erroneous pseudo-labels that enter with high weights in the training set as training progresses. to this end, we propose verifymatch, a semisupervised learning approach, which uses all pseudo-labeled samples in model training where unconfident pseudo-labeled samples are incorporated into training instead of being discarded or used with lower weights during training as in previous works. verifymatch consists of two components: (1) pseudo-generated data construction using large language models (llms), and (2) a verifier that leverages pseudo-labeling to ensure the quality of llm-generated pseudo-labels. in verify-match, the llm pseudo-labels guide the training of the verifier and, at the same time, the verifier determines the veracity of the llm-generated labels. our pseudo-generated data construction produces readily available unlabeled data for semisupervised learning (ssl) on natural language inference (nli). specifically, given a small amount of labeled data, we first fine-tune llms for every class. we then use these class-specific fine-tuned llms for generating hypotheses for a given unlabeled premise along with assigning the initial pseudo-label. by leveraging class-specific finetuned llms, we prevent potential skew or imbalance in the distribution of class labels within pseudo-generated data, thereby ensuring comprehensive coverage of all class labels. for example, given a premise 'a man painting over graffiti', we produce three hypotheses, one for each class, 'entailment,' 'contradiction,' and 'neutral,' by using the corresponding class-specific fine-tuned llm. to ensure the quality of llm-generated hypotheses and their pseudo-labels, our verifier (a task classifier) produces pseudo-labels on sentence pairs and checks them against llm-assigned pseudolabels. if there is disagreement between the labels, we call these "mismatched samples". even when there is agreement, the verifier might be unsure of its prediction (i.e., unconfident on a predicted class), because the sample is ambiguous or possibly mislabeled. we consider these samples as "unconfident samples" both types of samples are then "denoised" by interpolating them with human-annotated labels through mixup @xcite . hence, verifymatch improves the diversity of training data while ensuring its quality. we show competitive performance on various nli datasets in low-resource settings compared to strong baseline methods. our contributions are as follows: semi-supervised learning (ssl) ssl has produced a diverse collection of approaches including self-training (also called pseudo-labeling) @xcite @xcite @xcite @xcite . in general, self-training relies on a fixed high threshold value on model confidence in the pseudo-label class to filter out lowconfidence pseudo-labeled samples (li and @xcite @xcite @xcite @xcite thereby limiting access to a broader range of training samples. to overcome this, chen et al. (2023) proposed to integrate lowconfidence samples by assigning lower weights to them during training. however, this method still may introduce erroneous pseudo-labels with high weights as training iteration progresses. hence, we propose to integrate low-confidence pseudogenerated samples after denoising. 1 https://ai.meta.com/blog/meta-llama-3/ (1) pseudo-generated (pg) data (2) verifying labeled training data mismatched pg data unconfident pg data (2) supervised training (2) unsupervised training (2) mixup final task model task model (verifier) llms figure 1: the overview of our proposed approach: (1) llms construct pseudo-generated (pg) data, and (2) the verifier identifies mismatched pg data and unconfident pg data, to denoise them through mixup while the rest pg data are used via unsupervised training in addition to using labeled data in supervised training to obtain the final classifier. mixup mixup @xcite is a regularizer for neural models by training convexly combining random pairs and their associated labels. many works have empirically noticed regularization effects of mixup that improve performance on deep neural networks @xcite @xcite @xcite @xcite . mixup also has shown effectiveness in ssl for nlp tasks @xcite @xcite . building upon this, we propose to use a mixup approach for ssl to denoise lowconfidence pseudo-generated samples by mixing them with labeled samples. ## proposed approach: verifymatch in this section, we introduce verifymatch, our semi-supervised learning (ssl) approach for natural language inference. verifymatch seamlessly combines two components: 1. pseudo-hypothesis generation and pseudo-label assignment for the unlabeled data using large language models (llm) as one component, and 2. ssl model training with pseudo-labeling as another component. in veri-fymatch, the llm pseudo-labels guide the training of the ssl model and to ensure the quality of pseudo-labels it includes three key elements: a verification step that accounts for the agreement / disagreement of llm-generated and ssl-generated pseudo-labels; adaptive confidence thresholding that leverages the ssl model's confidence and uncertainty in the predictions; and the use of all training samples-no matter how noisy through a mixup data augmentation strategy that mixes in-between labeled and unlabeled samples. ## ablation study mismatched & unconfident pseudo-generated (pg) data to explore the impact of identifying mismatched and unconfident pg data and incorporating them after denoising through mixup, we show the results of verifymatch without leveraging them. these results are obtained by removing l mm and l unconf one at a time in the final training loss eq. ( 7 ). that is, we simply discard mismatched pg data and unconfident pg data, and show results in ## conclusion we proposed verifymatch, which constructs pseudo-generated samples using large language models (llms), and introduced semi-supervised learning (ssl) that acts as a verifier to ensure the quality of pseudo-generated samples for natural language inference. for ssl, we further proposed to identify and incorporate mismatched and unconfident pseudo-generated samples after denoising through mixup, which allows a model to have access to a broader range of training samples. we empirically validate that verifymatch achieves competitive performance compared to strong baselines.