Leveraging Training Dynamics and Self-Training for Text Classification (2022)

Abstract: The effectiveness of pre-trained language models in downstream tasks is highly dependent on the amount of labeled data available for training. Semi-supervised learning (SSL) is a promising technique that has seen wide attention recently due to its effectiveness in improving deep learning models when training data is scarce. Common approaches employ a teacher-student self-training framework, where a teacher network generates pseudo-labels for unlabeled data, which are then used to iteratively train a student network. In this paper, we propose a new self-training approach for text classification that leverages training dynamics of unlabeled data. We evaluate our approach on a wide range of text classification tasks, including emotion detection, sentiment analysis, question classification and gramaticality, which span a variety of domains, e.g, Reddit, Twitter, and online forums. Notably, our method is successful on all benchmarks, obtaining an average increase in F1 score of 3.5% over strong baselines in low resource settings.

## introduction deep learning models have achieved impressive performance in most supervised learning settings. however, supervised approaches struggle when labeled data is scarce, since they often suffer from overfitting @xcite . semi-supervised learning (ssl) is a powerful approach @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite that can overcome this drawback by leveraging unlabeled data, which usually comes in much larger quantities and is easier to collect. a popular category of ssl methods in text classification is self-training @xcite @xcite @xcite , an iterative approach that uses a trained teacher model to produce pseudo-labels for unlabeled examples, then uses these labels to train a student model, followed by repeating the process with the student as a new teacher until a convergence criterion is met. the quality of the pseudo-labels in the teacherstudent framework is an important factor in the selftraining process. in supervised learning, noisy labels are problematic and can negatively impact the generalization performance @xcite especially for deep neural networks, which can attain zero training error on any dataset @xcite . this phenomenon applies to self-training as well since the student model predictions are optimized towards potentially noisy pseudo-labels. to address this drawback, popular self-training methods @xcite mask out examples that the teacher model is not confident about. however, relying only on the teacher's confidence in predictions can be problematic especially if the teacher model is not well calibrated @xcite or has poor performance. in this work, we investigate the impact of the pseudo-label quality over the performance of selftraining methods in text classification and show that designing more sophisticated quality assurance measures for the teacher pseudo-labels leads to an improvement in generalization performance. we, hence, propose a novel self-training method that leverages training dynamics to assess the adequacy of the teacher pseudo-labels. in a nutshell, instead of using only the teacher's current beliefs about an unlabeled example (i.e., the confidence) to decide if an example should be masked or not, our method also analyzes how pseudo-labeled examples behave during training. specifically, we leverage area under the margin (aum) @xcite ) from supervised learning, which captures the divergence between the annotated label and the predicted label during training. aum is calculated as the average difference between the logit corresponding to the gold annotated label and the largest other logit. prior work has shown that a low aum score correlates well with an example being mislabeled. our approach, which we call aum-st, extends aum to unlabeled data and provides a more robust and effective mechanism of identifying noisy pseudo-labels compared to the current approaches based only on confidence. for each unlabeled example, aum-st computes the average logit difference between the teacher pseudo-label and the largest other logit during training. an unlabeled example with low aum indicates that there is a constant tension between its assigned (potentially incorrect) pseudo-label and the hidden true class. our method therefore masks pseudo-labeled examples with low aum. critically, unlike the vanilla aum, where the annotated labels are constant, in aum-st the pseudo-labels are variable and dependent on the teacher network. in each self-training iteration, as the self-training process progresses, the teacher starts generating more qualitative pseudolabels, which are then used to further improve the student. in a way, our aum-st can be viewed as a method to enforce a strict learning curriculum @xcite @xcite , where challenging unlabeled examples are not used until the teacher network is able to produce adequate pseudo-labels for them. to show the effectiveness of our approach, we test it on a diverse range of text classification tasks, ranging from emotion detection and sentiment analysis to gramaticality and question classification. notably, aum-st is extremely effective on all benchmarks in low resource settings, obtaining an average improvement in accuracy over a baseline bert @xcite model of 3.5% using 200 examples per class and 8.3% improvement using as few as 20 examples per class. our contributions are as follows: 1) we introduce a new self-training method for text classification that leverages training dynamics to enforce high-quality teacher pseudo-labels during training. 2) we evaluate our approach and demonstrate its effectiveness on eight text classification tasks in low resource settings. 3) we perform a comprehensive analysis of our approach and show its effectiveness in removing noisy teacher pseudo-labels during training. ## related work we first discuss related work on semi-supervised learning for text classification. second, we zoom in to self-training, a type of ssl that is the core of our aum-st. finally, we discuss approaches of learning with label noise. semi-supervised learning in nlp semisupervised learning has attracted much attention in the nlp community @xcite @xcite @xcite @xcite , since unlabeled data is often much easier to acquire compared to labeled data. for example, @xcite used adversarial perturbations to text in the word embedding space. @xcite used a hierarchy structure to propagate supervision from high-level labels to lower-level labels, while @xcite introduced cross-view training, where a model makes auxiliary predictions only seeing parts of the input text and is trained to match the predictions when given the entire input. @xcite used data augmentations on unlabeled examples and trained the model to output the same predictions when fed clean or augmented versions of the same input. mukherjee and awadallah (2020b) introduced uncertainty estimates into self-training, a particular type of ssl where a teacher and a student model are iteratively trained using labeled and unlabeled data. self-training is the core of our aum-st, hence we detail it further in the next paragraph. self-training our aum-st approach builds upon previous works on self-training @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite , but replaces the traditional confidence thresholding data filtering mechanism with a more effective approach that takes into account the training dynamics of unlabeled examples. self-training is an ssl method where a single model is repeatedly trained on both labeled and unlabeled data, until a convergence criterion is met. the model selects which unlabeled data to train on using its own predictions. concretely, traditional self-training follows these steps: 1) train a teacher model m on a labeled set l. 2) use m to make predictions and obtain pseudo-labels on a set of unlabeled examples u . 3) optionally, filter out unlabeled examples using a criterion. for example, in traditional self-training, unlabeled examples where the teacher confidence is not high enough are ignored. 4) use both the labeled set l and the generated pseudo-labeled set to train a new student model m . 5) continue to step 2 with the student as the new teacher (i.e., m ← m ). learning with label noise several approaches to achieve label noise robustness have been proposed. for example, @xcite proposed adding a noise layer in the neural network architecture, whose parameters can be learned for an accurate correct label estimation. @xcite introduced a curriculum-learning approach that uses learnable data parameters and ranks the importance of examples in the learning process. these parameters are then leveraged to decide the data to use at different training stages. @xcite on the other hand proposed to alter the loss function to make it more robust in the face of label noise. to this end, they introduced peer loss functions, which evaluate predictions on both the samples at hand, as well as carefully automatically constructed peer samples. other approaches designed techniques to accurately identify and eliminate potentially mislabeled instances @xcite . our work builds on the latter approaches; we leverage area under the margin @xcite @xcite and adapt it to our self-training setup. we emphasize that most of these methods for learning with label noise can be adapted to our setting, and these are potential future directions for our work. ## our method in this section, we first provide background information on the vanilla aum @xcite metric. next, we introduce our proposed aum-st and detail the various procedures that we used to improve its performance. ## experiments and results in this section, we first introduce the eight benchmark text classification datasets used to evaluate aum-st ( §4.1). second, we introduce weak and strong baselines ( §4.2) which we compare against our aum-st. next, we detail our experimental setup ( §4.3) and conclude by presenting the results obtained on all datasets in low data regimes ( §4.4). ## conclusion we improve the traditional self-training framework through a novel area under the margin unlabeled example selection technique, and show that our approach is effective in a wide range of text classifica-tion tasks. we studied our approach in various domains (social networks, forums, online platforms) and contexts (movie reviews, medical forum discussions, fact-based questions), and observed that our aum-st outperforms other strong self-training approaches. in the future, we plan to incorporate other approaches of learning under label noise into ssl frameworks such as self-training.