Few Shot Rationale Generation using Self-Training with Dual Teachers (2023)

Abstract: Self-rationalizing models that also generate a free-text explanation for their predicted labels are an important tool to build trustworthy AI applications. Since generating explanations for annotated labels is a laborious and costly process, recent models rely on large pretrained language models (PLMs) as their backbone and few-shot learning. In this work we explore a self-training approach leveraging both labeled and unlabeled data to further improve few-shot models, under the assumption that neither human written rationales nor annotated task labels are available at scale. We introduce a novel dual-teacher learning framework, which learns two specialized teacher models for task prediction and rationalization using self-training and distills their knowledge into a multi-tasking student model that can jointly generate the task label and rationale. Furthermore, we formulate a new loss function, Masked Label Regularization(MLR) which promotes explanations to be strongly conditioned on predicted labels. Evaluation on three public datasets demonstrate that the proposed methods are effective in modeling task labels and generating faithful rationales.

## introduction interpretable nlp has emerged to learn models which explain their predictions through either extractive @xcite or natural language explanations @xcite @xcite . due to higher expressivity of free text, generative self-rationalizing models have gained much research interest. however, the early works assume a fully supervised setup and require a large annotated dataset @xcite . collecting large scale, manual annotations for task labels and corresponding explanations is challenging and expensive. on the other hand, a much larger unlabeled corpora is often available, making semi-supervised approaches like * work done during an internship at amazon few-shot learning @xcite and selftraining @xcite attractive solutions. in the context of self-rationalizing models, @xcite explore few-shot learning, while @xcite seek to improve a supervised labeler by augmenting it with rationale generation. in this work we start from a few-shot setup, assuming only a handful of examples available with their labels and hand-written rationale. we leverage a large unlabeled dataset and self-training techniques to improve over the simple few-shot model. we hypothesize that using only a few examples, learning to generate meaningful explanations jointly with predicting the labels themselves, is a particularly challenging objective and self-training can suffer from a weak initial model. to address this, we propose a novel dual teacher learning approach to learn a self-rationalizing model from the two teacher models in a cascading manner. at first, a predictor model is learned for predicting task labels, and then a rationalizer model is learned to generate an explanation conditioned on an input and the task labels predicted by the predictor model. we iteratively improve both models via self-training. in contrast to learning the joint model directly, the rationalizer model allows for much richer representation learning by moving the label information from decoder to the encoder part, and utilizing the encoder's self-attention mechanism to extract input-label correlations. a stronger few-shot model for rationale generation provides higher quality pseudo labels, consequently making self-training more effective. although the two conditional models (predictor and rationalizer) might be better performing, a single self-rationalizing model is still desirable for practical applications, due to its easeof-maintenance and parameter efficiency for faster inference. we apply principles from knowledge distillation @xcite on the two conditional models to learn a joint model that generates task label and explanation as a single sequence. the teacher models are used for generating pseudo labels on the entire unlabeled dataset. the initial few-shot labeled data and the pseudo labeled dataset are finally combined to train the joint model. faithfulness of explanations is an imperative property for practical applications of interpretability analysis. a model generated explanation is considered faithful if it accurately explains the decision making of the model @xcite . similar to prior study @xcite , we also observe that a free text explanation generated by models might sound plausible, without satisfying the faithfulness criteria of explaining the predicted task label. this motivates us to design a masking based regularization function, masked label regularizer (mlr), to encourage the model to condition on the task label while generating an explanation. mlr is an entropy based constraint that forces the rationalizer model to be maximally uncertain in generating an explanation in absence of label tokens and is used to ensure that the rationalizer model preserves faithfulness through the self-training iterations. to summarize, our contributions are: ## related work prior works on generating free text rationales have explored joint models @xcite as well as several variants of pipeline models @xcite . we also use sequence to sequence models @xcite as our backbone models. while most of the self-rationalizing literature assumes fully supervised setups, star (zelikman et al., 2022) explores an alternate bootstrapping setup where limited rationales are available, but the task labels are present for the whole dataset. we consider the generic and more restrictive setting where only limited annotations are available for both task label and rationale. for limited labeled data scenario, many nlp applications have started reporting success with self-training @xcite @xcite . inspired from these works, we employ self-training to the selfrationalization problem. we introduce a new training framework with two conditional models and using them as teachers in a further distillation step to train the joint model. besides the popular use for model compression, knowledge distillation has also shown superior performance when using the same model architecture and size for both the student and teacher models @xcite , and distilling from multiple teachers @xcite . recently, a work @xcite in computer vision domain has explored using pseudo-labels from multiple teachers to train a joint student model. however, they have multiple specialized teachers trained independently through full supervision, in contrast to the cascading nature of our dual teacher self-training setup. evaluating the quality of free-text rationales is significantly challenging and several works have proposed metrics to evaluate the explanations around fluency and their faithfulness properties @xcite @xcite . a recent work @xcite also tries to imbue faithfulness through a regularizing coefficient. however, they apply the regularizer to perturb the rationale while generating task label. in contrast we use a label masking regularizer to enforce the rationalizer model to generate an explanation which is faithful to the label. ## background we first provide some necessary background on self-rationalizing models and a theoretical outline of self training based learning. self-rationalization: a self-rationalization model tries to learn the joint distribution of output(o) and explanation(e), given an input(i), i.e. p (o, e|i). a common approach is modeling it as a sequence-to-sequence problem and generating the task prediction and the rationale jointly @xcite . input-output format for a self rationalizing joint model is illustrated in figure 1 . the input consists of a task prompt, (e.g. explain nli), and in output sequence the task label is generated first (e.g. contradiction), followed by a separator token (explanation:), and then the free text explanation. during inference, greedy decoding is used to generate the sequence until an eos token is produced. self-training is a type of semi-supervised learning based method, which assumes access to a small labeled dataset (d l ) and a large, unlabeled indomain dataset (d u ). the algorithm progresses iteratively in four steps. first, a teacher model is trained on the labeled dataset (d l ), to obtain θ t . the trained teacher is then used to infer pseudo-labels on d u , generating the pseudolabeled dataset d pl . a student model is then trained on d pl to obtain the θ s . in the next iteration the teacher model is updated with the learned parameters from the student and the process repeats until a convergence criterion is met. ## dual teacher for self-rationalization we combine the strengths of self-training and knowledge distillation to train a self-rationalizing joint model from dual teachers. following sections describe the components, their losses and the learning procedures in more detail. input-output formats of the models are shown in figure 1 , and the overall framework is illustrated in figure 2 . ## results and discussion we evaluate on public datasets for three different tasks. the task requires generation of a task label which describes the relationship between a premise and a hypothesis as entailment/contradiction/neutral, and a free text explanation for the prediction. comve @xcite aims to evaluate if a model can distinguish between sensible and nonsensical statements based on common knowledge. we combine the data from subtask a (validation) and subtask c (generation) for our experiments. ecqa @xcite augments the commonsense qa dataset @xcite with free-text explanations that support the the correct answer choice and refute the incorrect ones. we utilize the explanations for the correct output (positive property) as the explanation. for few-shot settings we sample 100 examples per class for each dataset. the self-training setup leverages the few-shot labeled dataset(d l ) and the rest of the training set as unlabeled dataset(d u ). ## conclusion we study the self-rationalization problem with fewshot labels and demonstrate that self-training is an effective learning paradigm and can significantly reduce the gap between few shot and fully supervised model performance. we present a novel dual teacher learning framework that learns two models for task label prediction and rationale generation through self-training and efficiently distill the knowledge in a single self-rationalizing joint model. with a masking based loss formulation we enforce label-explanation association in the rationalizer, leading to generation of more faithful explanations. we conduct experiments on three public benchmark datasets for free text explanations, and show that the proposed methods are effective in improving task performance while generating accurate and faithful explanations.