Refining Language Models with Compositional Explanations

1 Introduction . With recent advances in model architectures and pre-training techniques , neural language models [ 4 , 31 , 25 ] have achieved impressive results on a broad set of natural language processing ( NLP ) tasks , such as sentiment analysis and hate speech detection [ 3 , 45 ] . However , when a source model ( finetuned on some upstream dataset ) is applied to a target domain with a different data distribution , the model may suffer from poor performance due to some spurious feature patterns learned from the upstream dataset [ 33 , 44 , 6 ] . Moreover , some spurious patterns may cause unintended biases in the downstream tasks , resulting in fairness and trust concerns about the model [ 21 ] . Prior work suggests that humans can identify such spurious patterns through examining the visualized “ heat-map ” ( Fig . 1 ) produced by a post-hoc model explanation algorithm [ 13 ] . As a prominent example , feature attribution methods [ 42 , 14 , 19 ] interpret model prediction on an instance by assigning an importance ( attribution ) score to each input feature ( or token , in the context of NLP tasks ) , which helps uncover an overemphasis or understatement of a specific feature ( e.g. , “ Sweden ” is overemphasized as indication of hate speech , as in Fig . 1 ) . To alleviate these spurious patterns , recent attempts study model regularization methods that update models in a differentiable and incremental fashion , which looks to align feature attribution scores with the “ intended ” scores manually specified by human annotators [ 38 , 35 , 30 ] . For example , attribution scores on overemphasised , unintended tokens are decreased ( to close to zero ) through updating the model weights [ 21 ] . 1Code and data are available at https : //github.com/INK-USC/expl-refinement . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Despite these initial successes , existing model regularization methods have limited capacity in conveying complex human feedback regarding spurious patterns , and limited regularization strength as the regularization term is enforced on only instances associated with human feedback ( while the vast amount of unlabeled data is not leveraged ) [ 38 , 36 , 30 , 47 ] . To pinpoint a spurious pattern precisely , an annotator needs to describe it by composing multiple assertions regarding feature attribution and feature interaction ( e.g. , “ to be a failure ” modifies “ Sweden ” in Fig . 1 ) . However , previous work consider only the former [ 36 , 30 ] and omit the latter when characterizing spurious patterns . Moreover , refining these data-hungry language models often requires large amount of labeled data . To extend the coverage of regularization , one must match ( generalize ) one human feedback to multiple ( unlabeled ) instances in the target domain – i.e. , identify instances that potentially suffer from the same spurious patterns , and then regularize the model with a larger set of instances . To this end , we introduce Refining Language Model with Compositional Explanation ( REMOTE ) , a framework that alleviates spurious patterns of a trained model and addresses the aforementioned limitations , by soliciting complex and compositional explanations from human and refining the model with broadened coverage during regularization ( see Fig . 2 for an overview ) . Firstly , human annotators are shown the post-hoc explanations of the source model ’ s predictions on target domain data ( Fig . 1 ) . They are asked to describe the spurious patterns they find and their suggestions to adjust the importance scores and interactions of features . Secondly , we extract executable first-order logic rules from these human-provided compositional explanations . The execution of the logic rules are decomposed to several atomic operations . We devise softened versions of these operations so the logic rules provide noisy labels and refinement advice to a larger number of instances in the target domain . Lastly , we update model weights according to the suggestions in the explanations , using the enlarged set of instances obtained in the previous step . We highlight two major contributions of the REMOTE framework . First , to the best of our knowledge , REMOTE is the first work that studies gathering feature-level supervision from complex human explanations . Prior work [ 15 , 47 ] has explored producing pseudo labels from explanations ( i.e. , what is the correct label ? ) , while we focus on more concrete feature-level supervision ( i.e. , why is this label correct ? ) with the goal of reducing spurious patterns . Second , we quantify and regularize the interaction between features , in addition to feature attributions used in prior work . This greatly improves the expressiveness of human explanations by supporting more complex rationale that involves more than one feature . We validate our approach on three pairs of datasets in hate speech classification and sentiment analysis . Compared with direct transfer ( evaluate the source model on target domain data ) and other baselines ( distillation and weight regularization ) , we observe notable performance improvements after refining the model with our proposed framework . In addition , we demonstrate that REMOTE can reduce unintended biases on group identifiers in hate speech detection . 2 Related Work . Human-in-the-loop Learning . The idea of bringing human into the learning process to enhance the model has been explored through multiple paths . One direction of prior work is to ask human annotators to label important instances ( i.e . active learning [ 39 ] ) or to determine which model to use [ 9 ] . A more interpretable direction is to associate features with specific labels as a source of supervision [ 32 ] , or let users suggest adjustments of low-level features by examining model explanations [ 24 , 43 , 26 ] . These explanation-related methods are limited to relatively simple models including linear classifiers [ 24 ] and CNN [ 26 ] because the model explanations directly correspond to the low-level features . With model-agnostic post-hoc explanation algorithms that explain predictions of a trained model without interfering with its learned weights [ 42 , 19 ] , our interactive machine learning method enables inspection and feedback to models as complex as BERT . On the other hand , several works [ 15 , 47 , 51 ] argue that natural language explanations rather than numeric labels as human feedback provide a richer and more efficient source of supervision for training . Our method learns from natural language in different setting , where we aim to adapt a trained model to another domain . Another line of work has studied explanation regularization as an interpretable approach to impose prior knowledge into neural networks [ 38 , 35 , 36 , 30 , 7 ] . Compared with them , our work incorporate complicated human knowledge conveyed by natural language and further proposes regularization on feature interaction to better take context into consideration . Model Transfer and Domain Adaptation . Sequential transfer learning , also known as model transfer [ 46 ] , considers a model being trained sequentially over different labeled datasets that are not available at the same time . Existing model transfer methods update the model ( using labeled data from target domain ) with various fine-tuning techniques – e.g. , updating layer weights [ 8 ] , creating learning rate schedule [ 18 ] , adding regularization [ 48 , 28 , 27 ] , and model distillation [ 17 , 37 ] . More recent work also studies transferring models without labeled target data , known as unsupervised model adaptation [ 27 ] , by generating target-like data [ 29 ] , or learning target-specific features [ 27 ] . Our work goes beyond supervision in the form of “ instance-label ” data and considers more complex human feedback on feature attribution and interaction . In another relevant thread , unsupervised domain adaptation ( UDA ) looks to adapt a trained model to a new domain using unlabeled data from the target domain , by updating feature representation to minimize the distribution divergence between domains [ 12 , 11 ] , or updating models to match the distribution statistical moments at different orders [ 41 , 52 , 34 ] . However , UDA methods typically require access to labeled instances from the source domain , which are not available in our problem setting . 3 Model Refinement with Compositional Explanations . We study the problem of refining a source model for better adapting to a new domain ( Sec . 3.1 ) . By presenting the post-hoc explanation heat-map computed on the target data for the source model , we solicit from human annotators compositional explanations that describe what spurious patterns are observed in the instances and how to adjust the feature attribution and interaction scores to alleviate these spurious behaviors ( Sec . 3.2 ) . We aim to generalize the collected explanations to instances in the target domain ( Sec . 3.3 ) and update model weights based on these explanation-generalized instances ( Sec . 3.4 ) . 3.1 Problem Formulation . We consider adapting a text classification model fS trained on some source data DtrainS ( e.g. , a sentiment classifier trained on news articles ) to a new target domain T ( e.g. , tweets ) during a model refinement stage . We focus on a challenging setting [ 28 , 29 ] where the upstream labeled data DtrainS is not available ( e.g. , due to privacy constraints [ 27 ] , or access restriction [ 29 ] ) but unlabeled data in the target domain ( DT ) are readily accessible , during model refinement . This setting also reflects the common practice nowadays , as users may download trained models from public model repositories and look to deploy the model to their own data . Our problem setting is different from ( unsupervised ) domain adaptation [ 11 , 34 ] , where labeled source data ( DtrainS ) is available for making model update . Our work also distinguishes from traditional transfer learning [ 18 , 28 ] which focuses on leveraging labeled data in the form of “ instance-label ” pairs in the target domain [ 18 ] , while we solicit human explanations on model ’ s spurious patterns for adjusting feature attribution and interaction scores . We evaluate model ’ s performance on target data ( DtestT ) and source data ( DtestS ) as the measurement of success . We expect the target performance to be improved , while the source performance to be maximally preserved , after model refinement . In addition , a refined model should no longer rely on spurious patterns ( e.g. , being over-sensitive to group identifiers for hate speech detection ) . Therefore , we report False Positive Rate Difference ( FPRD ) on a synthetic dataset [ 5 ] as a fairness metric . 3.2 Compositional Explanation for Model Refinement . In the following , we define the format of compositional explanations used in our study , and the procedure to solicit these explanations from annotators . Compositional Explanations . Our compositional explanations consist of two parts : spurious patterns and refinement advice . We define a spurious pattern as words or phrases in a sentence , whose attribution/interaction scores do not align with human judgment . Annotators are required to describe a spurious pattern precisely and sufficiently using three types of expressions : ( 1 ) existence of a feature ; ( 2 ) characteristic of a feature ; ( 3 ) relation between features . We list examples for these expressions in Table 1 . Annotators then provide refinement advice , i.e. , their advice to increase or decrease the attribution/interaction scores of features . The annotator also provide a label for the current instance . Compositionality in human explanations can not only improve the precision of matching instances , but enable annotators to describe their observations more flexibly as well . We provide concrete examples of the compositional explanations in Table 2 and “ Compositional Explanation ” block in Fig . 2 . Explanation Solicitation . We first use the source model fS to make predictions on a small set of unlabeled instances randomly sampled from DT and present these post-hoc explanation heatmaps [ 19 ] to human annotators ( see Fig . 1 ) . When shown with an instance with its heat-map , human annotators will : 1 ) read the sentence and provide a label , 2 ) inspect the heat-map , and 3 ) write an explanation if a spurious pattern is found . We estimate the time cost of each step and the ratio of finding spurious patterns , and will elaborate it in Sec . 4.2 . If the annotator identifies a spurious pattern in the heat-map for instance xref and provides an explanation , we refer to xref as the reference instance for the explanation . In this step , we have obtained a set of instances xref along with their raw human-provided explanations e , which we denote as E0 = { ( xref , e ) } .

Summary: The paper deals with the problem of spurious patterns generally learned by a language model. They address this problem by refining the model using compositional explanations. First, they obtain annotation from human workers and then obtain first-order logic from them that map and explain internal features of the model. These updated explanations are used to update the model. They show their results in hate speech and sentiment classification datasets.