Self-training with Few-shot Rationalization (2021)

Abstract: While pre-trained language models have obtained state-of-the-art performance for several natural language understanding tasks, they are quite opaque in terms of their decision-making process. While some recent works focus on rationalizing neural predictions by highlighting salient concepts in the text as justifications or rationales, they rely on thousands of labeled training examples for both task labels as well as annotated rationales for every instance. Such extensive large-scale annotations are infeasible to obtain for many tasks. To this end, we develop a multi-task teacher-student framework based on self-training pre-trained language models with limited task-specific labels and rationales and judicious sample selection to learn from informative pseudo-labeled examples. We study several characteristics of what constitutes a good rationale and demonstrate that the neural model performance can be significantly improved by making it aware of its rationalized predictions, particularly in low-resource settings. Extensive experiments in several benchmark datasets demonstrate the effectiveness of our approach.

## introduction recent success in several natural language understanding tasks can be attributed to training largescale and complex neural network models. while these models work very well for specific tasks, they offer limited insights into their inner working and are often used as black-box predictors. to address these shortcomings, recent works @xcite @xcite have focused on designing interpretable nlp systems that can explain the model's predictions. a typical approach to study this decision-making process has been via annotating rationales as a short and sufficient part of the input text leading to the specific 1 code available at https://aka.ms/rationalest prediction that can be also used as auxiliary supervision for training. appropriate use of such rationales can improve the downstream task performance as the model learns to focus on the task-relevant parts of the input @xcite . however, a significant resource challenge is to obtain large-scale annotated rationales to train these models as explored in fully supervised setting in recent work @xcite . this requires models to have access to both instance-level task labels as well as token-level binary rationale labels depicting whether a token should be included in the rationale or not. such extensive annotations are infeasible to obtain for many tasks, hence devising models that can effectively exploit a limited number of annotated rationales is utterly important. therefore, our objective is two-fold: (1) improve downstream task performance and (2) improve rationale extraction -with few labeled examples for the downstream task and corresponding rationales. recent works @xcite @xcite on few-shot learning have explored self-training as a mechanism to train neural network models with limited labeled data. these methods usually train a teacher and then a student model to imitate the teacher in turn. they usually assume access to a set of unlabeled instances and use stochastic regularization techniques such as dropout and data augmentation obtained from pseudo-labeled examples. in this work, we leverage self-training as a mechanism to train neural network models with self-generated rationales and task labels over unlabeled data. since pseudo-labeled rationales from the teacher model can be noisy, we show that judicious sample selection to upweight informative examples and downweight noisy ones is beneficial. furthermore, we predict task and rationale labels in a multi-task learning (mtl) setup, where we share parameters between the task objective and the rationale prediction objective. we show that the mtl setup for joint learning is more effective than the decoupled learning, which consists of first extracting rationales and then using them for classification, as explored in some of the prior works. given the paucity of rationale labels, a critical part in the mtl setup is to understand what constitutes a good rationale. we build over insights from prior work @xcite focusing on low-resource settings with access to limited labels via multi-task self-training. to this end, we explore several characteristics of a good rationale in terms of (i) sufficiency such that the extracted rationale is adequate for the model to make its decision; (ii) completeness such that the model is less confident on its predictions if it ignores the rationale text; and (iii) coherency such that the model extracts phrases as rationales rather than isolated disconnected words. in practice, we enforce (i) by matching predictions of the student model with the rationale as input and the teacher model with the full input; (ii) by maximizing entropy in the student predictive distribution when it sees the complementary of the rationale as input; and (iii) by recurring to additional regularization methods. we show that our multi-task joint optimization captures all of the above salient aspects for rationale extraction while improving the downstream task performance. in summary, our contributions are: (a) we develop a multi-task self-training framework to train neural models with limited labels along with extracting rationales as justifications for its predictions. furthermore, we show the impact of judicious sample selection for sample-and token-level re-weighting to learn from informative pseudo-labeled examples during self-training. self-training self-training @xcite @xcite trains a base (teacher) model on limited labeled data and applies them to unlabeled data to generate pseudolabels. the generated pseudo-labels are used for training the student model in an iterative fashion. self-training has demonstrated state-of-the-art performance in several tasks including text classification @xcite and image classification @xcite . we leverage self-training with re-weighting noisy pseudo-labels for both task and rationale extraction in a multi-task learning framework while encoding the desired characteristics of a good rationale. ## experimental setup datasets we evaluate our framework on five different tasks from the eraser benchmark (deyoung et al., 2020). these include movies for sentiment analysis @xcite ), e-snli @xcite for natural language inference, fever @xcite for fact extraction and verification, boolq @xcite awadallah, 2020). we perform early stopping on validation performance for the teacher multi-task training in each loop of self-training. we select the best model based on validation loss from the different self-training iterations. for evidence, we use a hyper-parameter 4 to upweight examples from the minority pseudo-labeled class in the self-training loop to combat class imbalance. baselines we compare our model performance to the following methods for both fully supervised (with access to all training labels) and our few-shot setting with 100 labels per class. (1) bert w/o explanation fine-tunes bert on a set of labeled examples without accounting for rationales. (2) bert with explanation is our multi-task learning setup where the classifiers are trained to predict both task labels and rationales without encoding the rationale characteristics. we also compare against the semisupervised setting from prior work paranjape et al. (2020) that uses 25% annotations for rationales and 100% task labels for each dataset. finally, we compare our method against a fully supervised bert-to-bert model in the eraser benchmark @xcite which first performs rationale extraction followed by downstream task prediction using extracted rationale. ## results overall performance for the fully supervised models, we observe that bert with explanation in our multi-task learning setup improves by 2.6% over vanilla bert, thereby, demonstrating the usefulness of rationales and the effectiveness of multi-task learning. our model performance for downstream tasks and rationales further improves with increasing the amount of labeled data. with 25% labeled training data, our model performs at par with @xcite (which has been trained with 100% task labels) in 3 out of 4 tasks and has comparable performance with fully supervised bert-to-bert baseline. additionally, our model does not require additional user inputs in the form of desired sparsity threshold as in prior work @xcite . in order to validate the effectiveness of sufficiency and completeness loss as a way of integrating rationales in the model in contrast to attention regularization in @xcite , we perform the following experiment. instead of using teacherpredicted rationales, we use the ground-truth rationales. we observe that our model with sufficiency and completeness loss outperforms the prior method of integrating explanations via attention regularization (results in figure 3 in appendix). this may reflect the fact that attention regularization imposes the stricter assumption to have uniform attention on the explanation tokens. we observe the self-training performance to improve on initializing our encoders with pre-trained domain-specific checkpoints. for instance, using biobert checkpoint, we observe a gain of 4.8% over that of bert-base in the evidence dataset. ## conclusion we develop a multi-task self-training framework for rationale extraction focusing on low-resource settings with access to very few training labels. to this end, we build on insights from prior work on the characteristics of a good rationale to encode them via judiciously designed loss functions in our self-training framework. extensive experiments on benchmark datasets show our model to outperform other state-of-the-art methods with access to limited labels. we further demonstrate that the performance of pre-trained language model can be improved by making it aware of the rationales for its decision-making process in both high (fully supervised) and low-resource (few label) settings.