Refining Language Models with Compositional Explanations
Authors: Huihan Yao
Ying Chen
Qinyuan Ye
Xisen Jin
Xiang Ren
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: Natural language processing
language model
explanation-based learning
spurious patterns
post-hoc model explanation

Abstract: Pre-trained language models have been successful on text classification tasks, but are prone to learning spurious correlations from biased datasets, and are thus vulnerable when making inferences in a new domain. Prior work reveals such spurious patterns via post-hoc explanation algorithms which compute the importance of input features. Further, the model is regularized to align the importance scores with human knowledge, so that the unintended model behaviors are eliminated. However, such a regularization technique lacks flexibility and coverage, since only importance scores towards a pre-defined list of features are adjusted, while more complex human knowledge such as feature interaction and pattern generalization can hardly be incorporated. In this work, we propose to refine a learned language model for a target domain by collecting human-provided compositional explanations regarding observed biases. By parsing these explanations into executable logic rules, the human-specified refinement advice from a small set of explanations can be generalized to more training examples. We additionally introduce a regularization term allowing adjustments for both importance and interaction of features to better rectify model behavior. We demonstrate the effectiveness of the proposed approach on two text classification tasks by showing improved performance in target domain as well as improved model fairness after refinement.

Refining Language Models with Compositional Explanations
Huihan Yao Ying Chen² Qinyuan Ye Xisen Jin' Xiang Ren
Peeking University TTsnnhuaa University University of Southern California yachuihan@pku. edu. en chenyingl7@maile tsinghua. edu, cn {qinyuany, seinyujny,xisenjiii [llangren@ussee edu
Abstract
Pre-trained language models have been successful on text classification tasks, but are prone learning spurious correlations from biased datasets, and are thus vul- nerable when making inferences in a new domain. Prior work reveals such spurious patterns via post-hoc explanation algorithms which compute the importance of input features. Further, the model is regularized to align the importance scores with human knowledge, so that the unintended model behaviors are eliminated. However, such regularization technique lacks flexibility and coverage, since only importance scores towards a pre-defined list of features are adjusted, while more complex human knowledge such as feature interaction and pattern generalization can hardly be incorporated. In this work, we propose refine learned language model for target domain by collecting humun-provided compositional explana- tions regarding observed biases. By parsing these explanations into executable logic rules, the human-specified refinement advice from a small set of explanations can be generalized to more training examples. We additionally introduce a regular- ization term allowing adjustments for both importance and interaction of features to better rectify model behavior. We demonstrate the effectiveness of the proposed approach two text classification tasks by showing improved performance in target domain as well as improved model fairness after refinemen
Introduction
With recent advances in model architectures and pre-training techniques, neural language models [4] have achieved impressive results on a broad
31 25
natural language processing (NLP) tasks,
such sentiment analysis and hate speech detection tuned on some upstream dataset) applied to a target domain with different data distribution, model may suffer from poor performance due to some spurious feature patterns learned from the upstream dataset [33 []. Moreover, some spurious patterns may cause unintended biases in the downstream tasks, resulting in fairness and trust concerns about the model [21].
3 45
However, when a source model (fine-
Prior work suggests that humans can identify such spurious patterns through examining the visualized "heat-map" (Fig. I produced by a post-hoc model explanation algorithm [3]. As a prominent example, feature attribution methods 42 19] interpret model prediction on an instance by assigning an importance (attribution) score to each input feature (or token, in the context of NLP tasks), which helps uncover an overemphasis or understatement of a specific feature (e.g.. "Sweden" is overemphasized as indication of hate speech, in Fig. []. To alleviate these spurious patterns, recent attempts study model regularization methods that update models fashion, which looks to align feature attribution scores with the "intended" scores manually specified
differentiable and incremental
by human annotators [38 35 30 For example, attribution scores on overemphasised, unintended tokens are decreased (to close to zero) through updating the model weights [21]
Code and data are available at https://gituub.com/KNK WWCiKK-USCeexp-fffnnneeeee
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Hate
Despite these initial successes, existing model regu- larization methods have limited capacity in conveying complex human feedback regarding spurious patterns, and limited regularization strength as the regulariza- tion term is enforced on only instances associated with human feedback (while the vast amount of unlabeled
..eSweenn has been proved to failure...
Sweden has been proved to failure...
....Seeen been proved failure...
Adjust to non non-hatte
Adjust hatee
Non-hate
data is not leveraged) [38] 36 30 47 To pinpoint a
spurious pattern precisely, an annotator needs to de- scribe it by composing multiple assertions regarding feature attribution and feature interaction (e.g., "to be
Figure 1: An illustration of post-hoc model ex- planation heat-map for hate speech detection. A trained hate speech classifier mis- classifies the sentence as non non-hateful, After observing the heat- map, human annotators may suggest that "Swe- den" be adjusted to neutral, and "failure" should contribute more to predicting hate speech.
failure" modifies "Sweden" in Fig. D However,
previous work consider only the former [36 30 and
omit the latter when characterizing spurious patterns. Moreover, refining these data-hungry language models often requires large amount of labeled data. To extend the coverage of regularization, one must match (generalize) one human feedback to multiple (unlabeled) instances in the target domain - i.e. identify instances that potentially suffer from the same spurious patterns, and then regularize the model with larger set of instances.
To this end, introduce Refining Language Model with Compositional Explanation (REMOTE), a framework that alleviates spurious patterns of a trained model and addresses the aforementioned limitations, by soliciting complex and compositional explanations from human and refining the model with broadened coverage during regularization (see Fig. 2 for an overview). Firstly, human annotators are shown the post-hoe explanations of the source model's predictions on target domain data (Fig. [ They are asked to describe the spurious patterns they find and their suggestions to adjust the importance scores and interactions of features. Secondly, we extract executable first-order logic rules from these human- -prvided compositional explanations. The execution of the logic rules are decomposed to several atomic operations. We devise softened versions of these operations so the logic rules provide noisy labels and refinement advice to a larger number of instances in the target domain. Lastly, we update model weights according to the suggestions in the explanations, using the enlarged set of instances obtained in the previous step.
We highlight two major contributions of the REMOTE framework. First, to the best of our knowledge, REMOTE is the first work that studies gathering feature-level supervision from complex human explanations. Prior work [5 47 has explored producing pseudo labels from explanations (i.e., what is the correct label?), while we focus on more concrete feature-level supervision (i.e., why is this label correct?) with the goal of reducing spurious patterns. Second, we quantify and regularize the interaction between features, in addition to feature attributions used in prior work. This greatly improves the expressiveness of human explanations by supporting more complex rationale that involves more than one feature.
We validate our approach three pairs of datasets in hate speech classification and sentiment analysis. Compared with direct transfer (evaluate the source model on target domain data) and other baselines (distillation and weight regularization), we observe notable performance improvements after refining the model with our proposed framework. In addition, we demonstrate that REMOTE can reduce unintended biases on group identifiers in hate speech detection.
2 Related Work
Human-in-the-loop Learning. The idea of bringing human into the learning process enhance the model has been explored through multiple paths. One direction of prior work is to ask human annotators label important instances (i.e. active learning [39 or to determine which model to use [9]. A more interpretable direction is to associate features with specific labels as a source let users suggest adjustments of low-level features by examining model explanations 24 43 26 These explanation-related methods are limited to relatively simple models including linear classifiers (44 and CNN (26 because the model explanations directly correspond to the low-level features. With of a trained model without interfering with its learned weights [42 19 our interactive machine
of supervision 32]
post-hoc explanation algorithms that explain predictions
learning method enables inspection and feedback to models complex as BERT. On the other hand, several works 15 47 5T argue that natural language explanations rather than numeric labels as human feedback provide a richer and more efficient source of supervision for training. Our method

---

3.2 Compositional Explenation
3.3
Inepect Post-Hoc Explanation Heal-maps
(eeausee the word "Sweden" country, Yorlure" is negative. "Jaftrr"
First-Order Logle Rule
..weddn has been proved to be fallure..
score of "Swedin" should decreased. Altrilaation wame of "fatture" chould increnseel
drouriv "Swadin"
e spWords, countryl elsjWerd2, nogsbwc) (8 Wb Word2, 31
fer hai beye proved To te ledire..
be Telox..
interaction scure of "swedin" and "Jothre" should be incrossed.
DerrewnAtt(Weeddi x hemookkr-Wardd))
bocrcack Wordd,,, WordZ)
Unlabeled instance from
target domalin
3.3
Refinement Advice Molny label: hateful Attribution cerew of "Itnttat"" chould be dectwaoed oubiibation "stupid" should The interaction score "Britain" and "stupid" chruld be incrwased.
Model Refinement
mntabliddeen stupid laeporal puini of cacing.
Anethe-reminds that Brizcin's
Neural Model For Geonen alizaation
3.4
Figure 2: Overview for model refinement with explanation regularization. Post-hoc explanation hcat-maps are presented to annotators, and compositional explanations describing the model refinement suggestions are collected (Sec. 3.2). We parse them to first-order rules and match unlabeled data in the target domain with neural model (Sec. 3.3). and use them to refine the model (Sec. (3.4)
learns from natural language in different setting, where we aim adapt a trained model another domain. Another line of work has studied explanation regularization as an interpretable approach to impose prior knowledge into neural networks [38] [3| 36 30 77. Compared with them, our work incorporate complicated human knowledge conveyed by natural language and further proposes regularization on feature interaction to better take context into consideration.
Model Transfer and Domain Adaptation. Sequential transfer learning, also known as model transfer [46), considers model being trained sequentially over different labeled datasets that not available at the same time. Existing model transfer methods update the model (using labeled data from target domain) with various fine-tuning techniques e.g., updating layer weights (8) creating learning rate schedule [18], adding regularization 488] 28 27]. and model distillation [7 37] More recent work also studies transferring models without labeled target data, known as unsupervised model adaptation (27), by generating target-like data (29). or learning purge-sppecifi features [27]. Our work goes beyond supervision in the form of "instance-label" data and considers more complex human feedback on feature attribution and interaction. In another relevant thread, unsupervised domain adaptation ( (UDA) looks to adapt a trained model to new domain using unlabeled data from the target domain, by updating feature representation to minimize the distribution divergence between domains 112 I or updating models to match the distribution statistical moments at different orders 52 34 However, UDA methods typically require access to labeled instances from the source domain, which are not available in our problem setting.
3 Model Refinement with Compositional Explanations
We study the problem of refining a source model for better adapting to new domain (Sec. (Scc.B.I). By presenting the post-hoe explanation heat-map computed on the target data for the source model, we solicit from human annotators compositional explanations that describe what spurious patterns are observed in the instances and how adjust the feature attribution and interaction scores alleviate these spurious behaviors (Sec. 3.2p. We aim to generalize the collected explanations to instances in the target domain (Sec. 3.3 and update model weights based on these ielanattion-eneraliiae
instances (Sec. 3.4).
3.1 Problem Formulation
We consider adapting a text classification model fs trained on some source data Digrain (e.g.. a sentiment classifier trained on news articles) to new larget domain T (e.g., tweets) during a model refinement stage. We focus on a challenging setting (28] 29) where the upstream labeled data Di is not available (e.g., due to privacy constraints (27], or access restriction (29) but unlabeled data in the target domain (Dr) are readily accessible, during model refinement. This setting also reflects the common practice nowadays,
users may download trained models from public model repositories and look to deploy the model to their own data. Our problem setting is different from (unsupervised) domain adaptation III 34 where labeled source data (Dirain) is available for making model update. Our work also distinguishes from traditional transfer learning 18 28 which focuses on leveraging labeled data in the form of "instance-label" pairs in the target domain [8), , while we solicit human explanations on model's 's spurious patterns for adjusting feature attribution and interaction scores.
We evaluate model's performance on target data (D)est) and source data (DVe*t) as the measurement of success. We expect the target performance to be improved, while the source performance to be maximally preserved, after model refinement. In addition, refined model should no longer rely
3

---

spurious patterns (e.g., being over-sensitive to group identifiers for hate speech detection). Therefore, we report False Positive Rate Difference (FPRD) on a synthetic dataset as a faimess metric.
3.2 Compositional Explanation for Model Refinement
In the following. we define the format of compositional explanations used in our study, and the procedure to solicit these explanations from annotators.
Table 1: Three types of expressions for describing spurious patterns.
Compositional Explanations. Our compositional expla- nations consist of two parts: spurious patterns and refine- ment advice. We define a spurious pattern as words or phrases in a sentence, whose attribution/nneeraation scores do not align with human judgment. Annotators are re- quired to describe a spurious pattern precisely and suffi- ciently using three types of expressions: (I) existence of a feature; (2) characteristic of a feature; (3) relation between features. We list examples for these expressions in Ta- [T Annotators then provide refinement advice, i.e., their advice to increase or decrease the attribution/interaation scores of features. The annotator also provide a label for the current instance. Compositionality in human explanations can not only improve the precision of matching instances, but enable annotators to describe their observations more flexibly as well. We provide concrete examples of the compositional explanations in Table 2 and "Compositional Explanation" block in Fig. 2
1. Existence of fenture Example: X is "jews". "panasite"
2. Characteristic of a single feature
Description: Entity type, part-of -p-spe tag, sen- timent label of wonl/phrase.
Example: X is Person eatity. is verb. is positive word.
3. Relation between features
Description: Semantic roles, re-feferenee, dis- lance, elc.
Example: X is the subject of Y. tokens away from Y. X modilies Y.
Explanation Solicitation. We first use the source model fs to make predictions on a small set of unlabeled instances randomly sampled from Dr and present these post-hoe explanation heat- maps [19 to human annotators (see Fig. [T. When shown with an instance with its heat-map, human annotators will: 1) read the sentence and provide a label, 2) inspect the heat-map, and 3) write an explanation if spurious pattern is found. We estimate the time cost of each step and the ratio of finding spurious patterns, and will elaborate it in Sec. 4.2 If the annotator identifies a spurious pattern in the heat-map for instance Tref and provides an explanation, we refer to Iref as the reference instance for the explanation. In this step, we have obtained a set of instances Fref along with their raw human-providee explanations e, which we denote as Eg {(Iret,)))
3.3 Generalizing Explanation in Target Domain
With the collected human explanations, now we detail how to parse the collected natural- catural-language explanations into executable logic rules B > II, and how to execute those logic rules by softening their constraints so that they can be generalized to multiple unlabeled instances in Dr.
Explanation Parsing, Each raw explanation is parsed into first-order logic rule in the form of B + H. Here the description of the spurious pattern is parsed into rule body B and the refinement advice is parsed into rule head H (see "First-Order Logic Rule" in Fig. 2 We parse the raw explanations using a semantic parser based on Combinatory Categorial Grammar (CCG) [53] which can deal with natural language with linguistic variations and thus is friendly to annotators. To tailor the parser to our needs, we define a lexicon str2predicate to map 301 common expressions (e.g., "directly after", "negative word"") to 83 predicates and implement the corresponding operations with atomic modules (described in next paragraph). Anmotators can iteratively modify their explanations or update the lexicon until they make sure their explanations are accepted by the parser. We denote the collection of parsed explanations as E = ((rref, B, H)}. More details are in Appendix
Explanation Generalization with Matching Model G. With E obtained in previous steps, we now aim at generalizing from one explanation (Zref, B, H) to multiple unlabeled instances in Dr. That is, for each unlabeled instance x in Dr, we attempt to match it with each spurious pattern B and the reference instance Fref we have obtained. If the matching is successful, we consider that x may suffer from a spurious pattern similar to B, and will use this instance in the later regularization phase.
For this purpose, we construct an executable matching model G from each rule body B. G is dynamically constructed from three atomic execution units (Individuality Module, Interaction Module and Compositionality Module), following the predicates in B. We introduce these modules as follows.

---

Individuality module is used to output the feature (i.e., word) in the unlabeled instance x that corresponds to a feature gref the reference instance Fref- The module will first search if an exact match of fref exists in Zg. If an exact match does not exist, it will search for words of same type with fref in Zik, including named entity type, constituency parse structure, etc. We leave more details in Appendix E If no feature is found after this step, Individuality module will return None.
Interaction module examines whether the relation between features described in B holds true in the unlabeled instance Fref. Given Zref and two words fref.1. fref.2 in it, we first call Individuality module to find matched 9k.1 9k.2 in 2k, and then use Inter- action module to examine their relation. We use either natural distance (number of words between the two words ( or dependency distance (distance between (ref.1, Gref.2 on the dependency tree), depending on the descriptions given by an- notators. The former is applicable to explanations such as "X is within 3 words before Y", and the lat- ter is applicable to explanations such as "Feature X modifies feature Y". The Interaction module will output True of False, indicating whether the relation described in B is satisfied in Fref-
Table 2: An example of compositional explana-
tion. The explanation characterizes where spuri- pattern is (in the instance), and how the attribu- tion/interaction scores should be regularized.
Information shown to Annofators:
Reference Instance: They prove more distressing than attractive. Information from Heat-mmp: Predicted lahel is positive, Attrhu- tion of distressing is low.
Compositional Explanation
Sparions Pattern: is "distressing. Y is "than". Z is "aftrac- dive". negative. Z positive. X immediately before Y is immediarely before Z.
Nolsy Label: Negative.
Refinement Advice: Attribution score should be increased. Matched Instance
Instamce: Self-fagellatinn is more depressing than entertaining.
Noisy Label: Nogative.
Relinement Advice: is "depressing". Attribution score of X should he increased. Soft-Mutching Details- and "de- pressing" synonyms. "Mitractive" "entertaining" syn- onyms. The semuntic similarity is captured by sottened Individuality module.
Compositionality module is used to execute logic operations (i.e." "AND", "OR") specified in the ex- planation. Based on intermediate output produced by other modules, the module will output True or False, indicating whether B matches with unlabeled instance
Broadening Coverage with Softened Matching. The matching process described above enforces constraints in B strictly, e.g., the word "nice" will be rejected in an Individuality module looking for the word "good". despite their semantic similarity. To broaden the coverage of strict matching, we propose soft matching for each module, which relaxes the rule body and generates a larger number of matched instances to enhance regularization strength.
Table provides a concrete example. Based on the reference sentence "They prove more distressing than attractive", together with its explanation, sentence will be matched if and only if it contains distressing and attractive in strict matching. In contrast, in soft version, distressing can be generalized to depressing and attractive entertaining.
For Individuality module, we allow synonyms, coreference and morphological/spelling variation of 9ref in Fy using neural model. Given sentence 2;; = [w1, u...... Wm], we first encode the sentence 4 and obtain each token's 's contextualized representations [V1, U2, .... um] Given a phrase 9:: [W]1, ..., W[2] in Tg. we apply mean pooling over the token representations to get its phrase representation, i.e., Σ}=, u3/(12 - l1). We compute the representation for gref in an analogous way. The softened individuality module will output set of candidate spans whose representations have the highest cosine similarity to gref's.
with a BERT-Base model
For Interaction module, denote the distance between qk,1 and (x,2 d, and human-specified distance constraint between Gref.1 and fref.2 as dres. Instead of strictly following the distance constraint, we compute a score indicating how close S dref is to being correct: max(1 de poeer +1 220) if d > dref and otherwise.
For Compositionality module, soft logic/ Lukasiewiczt logic (22) operations are used to aggre- gate two intermediate scores produced by other modules, iee, AND(21, 22) = max + 22 0); (=1, 22) min(z1 + 22, 1).
Using the three atomic modules, we are able to generalize from human-provided explanations to more unlabeled instances in Dr. After the matching process, a matched instance 2g will be associated with a noisy label yk. some refinement advice Tk. and a confidence score zд. If strict matching is employed, 2 is set to one. If soft matching is employed, 2₁ is computed from scores produced by each module. We use Dmatchh = G(Dr) = [Z yk. rk. } to denote this set of instances.
5

---

3,4 Learning with Explanatiio--Generllizee Data Objective for Model Update. The learning objective for refining fs is defined as follows.
L = aLLattr +Linter),
where L' is the classification loss using the noisy labels {yk Cattr and Linter regularization terms computed using refinement advice {r*}. and a is the hyperparameter to control the strength of the regularization terms. We discuss the selection of in Sec. 4.4
With strict matching, the noisy labels Cstrict = {ua} and refinement advice Ratriet = {re} are less noisy; with soft matching, the noisy labels Cr and refinement advice Reoft can cover more training data. As noisy labels and refinement advice incorporate different information, it is preferable to decouple the matching process for the two components, such using Reoft with Catrict.
Regularization with Refinement Advice. We denote the attribution and interaction scores produced by the source model before adjustment as e and yp. The refinement advice T' contains set of target scores i or pp.y that suggest how the attribution or interaction scores of specific phrases should be adjusted. Target scores i and Tp.g of phrases p: q regarding class are to decrease the score, and if suggested to increase it. The regularization terms are the squared L distance between current and target importance and interaction scores, summed over all C classes and phrases p E Ti.-
0 if it was suggested
Cattr Σ Σ (p, PErk
Cinter
Σ
- -
(2)
(p-1)E<2
We consider two feature attribution methods for regularization Integrated Gradient 42 and Sampling and Occlusion [19]. Next, we briefly introduce their formulations and our approach to quantify feature interactions.
Importance Attribution Score Computation. Integrated Gradients (IG) computes an importance score of a feature (word) w; as the integrated gradient along the straight line path from an input sentence r and a neutral baseline x' = [w', w2. .... w'm] (e.g.. a sequence of all padding tokens). Formally, the importance score of word w, for label e is written as, & (wx;z) (w; - w()=
f1 (f" (a-(===)))) where (+() is the model prediction score for the class
Ow:
Sampling and occlusion (SOC) assigns importance score of a phrase = [w:,..., wy] (one or sequence of words) as the expectation of the prediction change under context replacement within fix-sized neighboring region 8. We use (x .ts) to denote model prediction when the words in x within å region are replaced with sampled æs; and use (= ; &g; 0p) to further denote the model prediction when the phrase p is replaced with padding tokens 0p. The importance of the 2t) (= ct;; 0p)), phrase in the input & is computed as, (f; .t) [S] Ex
where &s are replaced neighboring words sampled from a replacement set S (e.g., sampled from language model pre-trained on the training set). Specifically, with the neighboring range ō set as O, of (x)
[((x) - f 0p), the explanation is the same as input occlusion (or leave-one-out) algorithm [54) which is the prediction difference between erasing and keeping in the sentence.
Quantifying Feature Interactions. We borrow the definition of interaction from the cooperative game theory [0]. Under this definition, interaction describes how importance of a phrase changes when other words phrases are absent present. Based on the definition, we define the interaction score between two phrases and for predicting a class C as
(p. = o'(p; x)) (f;
where df (p, x) denotes the importance score of p after masking the phrase g from the sentence.
4 Experiments
4.1 Experiment Setup
Datasets. For hate speech detection, we use Stormfront (2) and HatEval [ as upstream datasets, and the Gab Hate Corpus (GHC) (20) as the downstream dataset. Stormfront is corpus collected from the white supremacist website and HatEval is the official Hate Speech dataset from 2001 Our two upstream datasets contain shorter and simpler sentences than those of GHC, which was

---

Table 3: Statistics for explanation solicitation and generalization. For each dataset pair, we report the annotation information (total time, numbers of labels and explanations written in corresponding time, and explanation yield rate). We also provide the number and precision of matched instances in strict and soft version, and the size of negative sampling instances in hate speech detection. Finally, we include size of C cample. the sets of instances with ground truth labels, which we sample from
target domain based on same time cost and use in baseline methods,
Dataset Pair
Tosal Time
labeled]
[e]
Exp. Yield Rate
[strict]
Prec. strict
[roff]
Prec. of soft
[balanced]
sample
HatFval -
mins
212
34
0.751
370
0.692 0.658
1400
Stormfront GHC
94 mins
285
40
14%
237
0.717
278
1600
464
AmazonMusic 2
15 mins
29
62%
1308
0,942
1737
0.917
204
collected from social network with a high rate of hate speech. For sentiment analysis, we first train on AmazonMusic [T6], and apply the model to the Stanford Sentiment Treebank-2 (SST-2) dataset [4)]. Details of the datasets are described in Appendix A
Compared Methods. We consider two variants of REMOTE in the experiments: (i) using only Rooft. and (ii) both Reoft and Cstrict. We include more experiments REMOTE using both Raoft and Czaft in Appendix D Detailed experiment settings about hyper-parameters are included in Appendix A
We compare our method with the following two lines of works. (1) Model transfer methods: 2 regularization [28] places a penalty the difference between the source and the new model parameters. Distillation method (37 adds a loss between prediction of source model and new model to encourage behavior reservation.
(2) Explanation-basee learning methods, including BabbleLabble [15] and NExT [47). use human explanations to generate pseudo-labeled data for model updating. Since our explanation design is different from [15] 47), their matching process is not directly applicable. We adopt our strict and soft matching method to generate pseudo-labeled data and conduct an ablation study (Sec. 4.4) to compare with: fine-tune with strict-matched data Caeriee (as Labblel.abble), and fine-tune with soft-matched data Ceo (as NExT). More experiments compared with other unsupervised model transfer methods [29 are included in Appendix D
All methods other than REMOTE are trained with only labeled instances from Dr. To form fair comparison with the baselines (when annotation time is controlled), we collect set of labeled instances (denoted as Csample) in the following way: we first estimate the amount of instances that cost the same annotation time the time used in collecting the explanations (detailed time cost is discussed in Sec. 4.2); then we randomly sample from Dr this amount of examples along with their ground-truth labels. In addition, we report performance of the source model fine-tuned over the fully labeled Dr set (denoted as Cat)), referred as fine-tune (Can)). As an "oracle method"", it provides a reference on how much space is left for improvement for source model.
Evaluation Metrics. In addition to Fl-scores on source domain and target domain test sets, we also evaluate the hate-speech classifiers the Identity Phrase Templates Test Set [5] (77,000 samples, with 50% of them as "toxic"). We evaluate the unintended biases for identity phrases T by computing the False Positive Rate Difference (FPRD) [5] as Eter |FPR- FPRel, where F FPR, is the false positive rate computed upon the subset of samples that contain the term t. A smaller value in FPRD suggests that the model is less biased.
4.2 Explanation Collection and Generalization
Explanation Solicitation and Time Cost. In Sec. 3.2 we introduce three steps in explanation solicitation - providing label, inspecting the heat-map, writing the explanation. We use SOC [19] a post-hoc explanation algorithm to generate heat-maps for annotators, and estimate the time cost for each step during annotation via our self-designed explanation solicitation interface (Sec. F). It takes on average 12/5/25 seconds for the three steps respectively to collect explanations for hate speech detection, and 4/2/14 seconds for sentiment analysis. We present annotation information in details on three data pairs in Table 3 In hate speech detection, due to difference in toxicity ratios between upstream and downstream datasets, we only collect explanations with noisy label as hateful. We defer the discussion on this choice to the case study on hate/nonhatt-llbeele rules in Sec. 4.4 To balance the label distribution, we randomly sample some instances in target domain as non-hate examples Csalanced, the sizes of which are also included in Table Experiment shows that model performance is not sensitive to the different numbers and sampling strategies for negative samples (see Appendix D). Our explanations are annotated by three CS graduate students with hourly wage as $20. To obtain high-quality explanations, two annotators need to verify the explanations written by the other annotator. Full agreement among annotators happened 90+% of time.
7

---

Table Main Results on three pairs datasets with different language models. We report F1 scores on source domain and target domain with standard deviation, and FPRD on IPTTS as fairness metric for hate speech refine the source model with best F1, and run experiment in three random sceds
task. For each setting, controlling the order of the data during fine-tuning. The annotation time cost of each dataset pair is provided. Best results are bold.
Dataset
HatEval
GIC (94
SST-2
Metrics
Source (?)
Target (*)
FPRD (.)
Source FI(I
Target FI
FPRD (
Source Pl (?)
Target (?)
BERI-Large
model
63.7-0.5
31.4114
19.5
41.91:144
17.1
87
Fine-tune (Cungic)
2.5
40.7+0.6
175.1
49.6++1.
45.0+3.11
24.3
91.4+14
88.5+1.1
12-reg. (Crampes)
58.1=29
4188118
102.2
49.921.8
45.3=199
12.2
90.52:1,1
88.9=1.6
Distilation (Cooagte)
63.1=25
43.3420
132.7
46.542.3
48.81111
23.4
91.910.3
89.420.4
REMOTE (Raors)
61.5-0.2
37.5+0.7
55.5
56.4+20.
45.7=40.9
0.5
92.7=0.11
89.4-0022
REMOTE (Rai Catvie))
62.0=0.4
16.1
15.3
49.0
52.2:10.4
10.0
92.75:0.2
90.3:0022
(Caz)
51.3=5.6
52.510.4
46.0
53.8
142.3
92.520.22
94.410.4
RoBERTie Base
Source (Crampte)
62.7-0.9
10.91199
61.6
57.4111.
39.6:11.2
43.8
9.
87.5.0.99
61.3=3.0
40.8112
185.2
56.7122.
40.311.9
89.020..6
17-1og. (Crampte)
62.4-2.0
42:24:08
292.7
48.8
41,7=00.5
46.2
90.92110 1.0
89.0::0.6
Distillation Casmple)
61.8=14
42.1112
152.5
48.411.6
40.9.10.3
28.1
91.310.5
19.1
REMOTE (Rasri)
61.1=0.6
10.51:.1
57.01110
40.9110.6
36.7
88.5
REMOTE (Rair + Caveee)
57.5=0.9
44.71110
97.8
$7.6:1199
50.1
77.5
91.420.2
89.5
Fine-tune (aa)
51,4=322
50.620.4
263.2
52.244499
50,5#1155
294.0
91.2400.9
95.1=0044
Quality Check of Explanation Generalization. We generalize the collected explanations on unla- beled instances in target domain in strict and soft version. To best utilize the collected explanations, we tune the threshold of confidence score in soft matching process from 0.5 to 0.9 based on the performance on dev sets, and set them as 0.7, 0.55 and 0.6 for Stormfront - GHC, HatEval - GHC and Amazon SST-2 respectively. To prove the effectiveness of our matching neural model, evaluate the matching quality by the precision of noisy labels. The statistics are shown in Table
4.3 Performance Comparison
In comparisons below, we keep the total human annotation time per model fixed, by controlling the number of explanations in REMOTE, and the number of labeled instances in model transfer methods.
Comparison with Model Transfer Baselines. Table 4 shows results on three source target dataset pairs with source models trained using LERT-Large 4 and RoBERTa- Base [3I] REMOTE with Reoft + Cstrict outperforms all other methods that have access Csample with the same annotation time on target performance. This demonstrates that REMOTE is highly label-efficient compared to the model transfer methods.
Comparison between REMOTE variants. REMOTE with just Reaft shows notable target Fl im- provement over the source models among all dataset pairs and language models. On top of that, REMOTE with Ro + Cstriet performs consistently better than with just Ro can conclude that our generated noisy labels sufficiently accurate and important in low-resource situation.
Fairness and Source Domain Per- formance. REMOTE preserves source F1 better than fine-tune (Catt) in most cases. We also observe that REMOTE mitigates unintended bias (as FPRD values are reduced) and simultane- ously achieves target close to the performance of fine-tune (Car).. We attribute this to our design of RE- MOTE which allows annotators to pin- point spurious patterns precisely with compositional explanations, and our feature-level regularization method which refines the model by teaching
Table 5: Significant Test Main Results. We report p pavuu between target Fl of REMOTE and each baseline method on every dataset pair Table 4 The difference is regarded as statically significant when S 0.05.
Hattival GHC
Storufroot
Ammazo
Metrics
P-Natue
Sig or not
P-Nalue
Sig. or not
P-Value
Sig. or not
BERT-Lange
tune (Cosmpte)
yes
0.0163
yes
yes
12-reg (C (Cammple)
0.0274
ves
0.0003
yes 0.2071
no
Distillation (Coomiie)
0.0959
0.0073
0.0252
yos
Pine- cune (Cammpte)
0.0124
Ves 0.0026
yes
0.3297
no
(2-reg (Cammple)
0.0278
0.0012
0.3297
Distillation (Coomgie)
yus
0.0008
yes
80
why an label is correct and reduces unintended biases.
Significance Test. To show the statistical significance between our method and baseline methods, we do an unpaired 1-test between REMOTE and each baseline methods. We report the P-values in Table 5
We found that the improvements brought by REMOTE is statistically significant in most cases.
8

---

Table 6: Effectiveness of model regularization technique on BERT-Base. We compare REMOTE (Reoft + Cstrict) ) (based on IG and SOC) with other methods using Cstrict to show the effect of Reoft-
Dataset
HatBval
GBIC
Sturmfront - GIIC
AmuzonMusic -
SST-2
Metrics
Source Fl (1)
Target PL (+)
FPRD (1)
Source PI(")
Target F1
FPRD (1)
Source F1 (7)
Target F1
Source model
64,2+0.3
29.542.55
115,6
57,2-007
42,1=1155
16.0
91 01.4:0.4
83.5=22.5
(Caenee)
60.3
45.1
42.0=:1.6
49.0=:0.5
59.2
90.740.1
86.0=0.66
1.-rrg (Crena)
62.7+1.1
46.3+0.2
77.1
46.5+00.7
49.9-0.8
86.4
90.7+0.3
86.8-0.6
Distillaion (Catrie)
63.2
16.3:09
65,4
1..42133
49,421.1
49.7
90.6:0055
86.7=1.0
Hine-une (Cooe)
62.4=188
454+20
49,6-4.1
47.9-0.6
164.0
90.4:0099
86.300A
REMITE (R G W.
64,2+0,4
47.2+1.3
129.5
51,44466
49.5-1.1
12.8
91.2+0011
87.0+0.7
REMOTE (Raa Cerree) w. SOC
63.2:10.6
46.62:11
49.0
49,4:119
51.1:116
34.6
91.1:022
07.3:001
Fine-tune (Cuz)
60.0+42.3
51,5+0.9
333.3
46,9=24
$2.9=110
115.0
90.4:0.1
92.9=0044
Et-ccore (%)
50
55
Ditiiiaton
HCNO (Rue Groc)
Fiokkre
52.9
Mad?
55
50
Guod
Fiweire (Culk
30H-72 Narl
465
45
45
40
Sourcs Hrod: 42.1
35
Source node:
35
30
25
10
40.34
20
30
40
50
60
70
80
90
20
30
40
50
60
70
80
Noateer Rides
Time (min)
Time (min)
Figure 4: Performance HatE- val GHC on BERT BERT-Base using different number of explanations.
(a) Storm + GHC
(b) HatEval + GHC
Figure 3: Label Efficiency Study on BERT-Base.
Aeross Different Language Models. We conduct experiments across different language models to study whether improvement brought by REMOTE is consistent. In Table 4 performance for both BERT-Large and RoBERTa- Base shows that incorporating noisy labels and regularization advice can generally help, and it can be widely applied to different pre-trained language models. We also include additional experiments using BERT-
4 and BiLSTM+Attention in Appendix
Study on Label Efficiency. We further analyze the performance trends of different methods when varying the annotation time. Figure shows that with the same amount of annotation time, REMOTE consistently outperforms the fine-tuning and distillation baseline, demonstrating better label efficiency. Within 30 minutes of annotation. REMOTE improves the Fl score on the downstream dataset significantly for HateEval + GHC. This indicates that even a small number of compositional explanations can provide strong supervision towards adapting the model the target domain.
4.4 Performance Analysis
Effectiveness of Explanation Regularization. In Table 6 REMOTE (Rooft + Cstrict) has lower FPRD and higher target Fl scores compared with fine-tuning or performing model transfer using noisy labels (Cstrict or ft) We observe that Ro plays an essential role in reducing unintended biases. As Runft contains specified information regarding identity groups, it enables the model to better capture the context involving identity phrases, and thus improves the faimess of the model. We include comparison between solely using Ratriet and Reoft in Appendix [ Rooft always leads to better performance than Ratrict. With coft-matching, regularization advice applied more instances and thus take more effect.
Effectiveness of Noisy Labels. To show the effectiveness of the noisy labels generated by REMOTE, we further compare the performance among fine-tuning with Commple (labeled in- stances randomly selected from Dr), Cstrict and Czoft (noisy labels generated in different matching versions) on hate speech detection. It shows that models fine-tuned with noisy labels can yield better performance than ground-truth labels. It is partly because that Csampte is ran- domly sampled and does not target cases with spurious patterns. In contrast, Cstrict and Csoft generalize collected explanations to unlabeled instances to create hateful instances, and then ran- domly samples some unlabeled instances from Dy as han-hateful instances. Regarding quantity, the numbers of matched instances are smaller than the size of Csampte. but the sizes of Cstrict or Csoft (generalized hateful instances and sampled non-hateful instances) are larger than Csample- Overall, quality and quantity of data both contribute to the final performance.
Table 7: Comparison on fine-tuning with different datasets. We compare performance of simply fine- tuning the source model BERT-Base with Cramples Cutrict and Czoft on hale speech detection.
Dataset
Starmfront -GCC
Halaal-GHC
Metries
(1)())
FPRD (
(1)
FPRD (
Source model
42.11:1.5
16.0
29.5=2.5
115.6
Fine- (Coompte)
41.0+0.11
302.6
45.6-0.1
20.3
Fine-tune (Carse))
45.1+22.2
49.0-0.5
Fine- tune (Cosp)
45,412..0
57.9
47.9_0.6
164.0

---

Performance Change from Increasing the Number of Explanations. We investigate the perfor- mance in HatEval - GHC setting using different number of explanations. We present the results in Fig. 4 Note that we randomly sample the subset of explanations for three times to reduce variance. We observe that that the performance continuously grows when more explanations are introduced. This indicates that model can learn more diverse knowledge from more explanations. We also conduct the same set of experiments for Stormfroott - GHC setting. Results are included in Appendix D
Ablation Study on Attribution and Interaction. To demonstrate the necessity of regularizing both attribution and interaction, we conduct ablation experiments that regularize them separately. Results in Table (()) show that both attribution and interaction can contribute to performance improvements and bias mitigation. Regularization with both attribution and interaction achieves higher FI score in target domain than regularizing with only one of them.
Table 8: Ablation study on hate speech detection on BERT-Base with REMOTE
(a) Ablation study on hatc/nonhate-labeee rules.
(b) Case study on attribution (a.) and interaction (i.).
Datasel
Stormsfront -GBC
Hateval GHC
Dataset
Sturmfrout .GBC
NatEval--GHt
Metrics
F1
FPRD
F1
FPRD
Metries
FPRD
FPRD
Source Modkl
42.1::1.5
16.0
29.5-2.5
115.6
Source modd
42.1:11.5
16.0
29.5:22.5
115.6
lanbute-labeled rules
43.2+=0.4
9.0
32.1-0.5
175.3
Reg. with a.
44.6-116
7.2
36.8--477
39.8
All riles
43.2+044
8.2
35.7-2.1
173.0
Reg. with
41.2+00.1
8.5
31.8+0.5
170.7
Hate- late-labeled miles
45.7=:1.4
12.6
39.94444
56.2
with
4,
12.6
39.9:4,4
56.2
Case Study on Hate-labeled Rules and Nonhate-labeled Rules for GHC. To reveal the reason of only collecting hate-labeled explanations for GHC, we show the ablation results in Table 8 a)). Note that, here "label" is the noisy label given by human annota- tors. Besides the hate-labeled rules, we also collect nonhate-labeled rules, and conduct experiments with them separately and together. We can see that regularization with only hateful rules has the best performance on the target domain and has low unintended biases.
(%)
10
h6.
355
51. 50.4
SouTee
Sensitivity Study on Regularization Strength. We tune a between 0.003 and 0.03 in the experiments of REMOTE (Rzoft + Cstrict) and report the results in Fig. [5] REMOTE is not very sensitive to a, and
0.03 0.505 0.07 0.81 6.12 0.53 Regularizatinn Strength
Figure 5: Sensitivity of reg. strength Stormfront to GHC.
improves the source model performance with a wide range of values. Results on another datasst pair are included in Appendix D
Discussion on Optimisation Time Cost. The time cost for computing the loss in Eq. (2) depends on the proportion of phrases that are given target scores (by generalizing the human-provine explanations). In our implementation, only the phrases that are given target scores will be traversed when computing the loss. Typically, very few phrases in a matched sentence would have target scores assigned. For example in the HatEval - GHC setting, 291 out of the 370 soft- matched instances have only one phrase/phrase pair with target score assigned. The main time cost comes from computing the importance/interatiion scores in line 264 and Eq 3. With the hardware and training details specified in Appendix it usually takes 50s for one iteration with Rooft loss and 30s without it.
5 Conclusion
In this work, we introduce a novel framework to refine a trained language model with human-provided compositional explanations. To break the bottlenecks of previous explanation regularization methods, our method (I) supports more diverse ways of regularization (i.e., both feature attributions and interactions) and (2) broadens regularization coverage by generalizing explanations to unlabeled instances. We believe our work opens up a new way to communicate human knowledge to model learning and refinement, and explores the possibility of supervising model features, as opposed the traditional paradigm of supervising a model with large-scale labeled examples. Extensive experiments demonstrate that our framework can improve source model's performance on different tasks and reduce unintended bias. Future work may expand on our method by adapting it to more challenging tasks such as reading comprehension and language generation, or by incorporating human feedback in multiple turns in an active learning setting.
10

---

Acknowledgments and Disclosure of Funding
This research is supported part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via Contract No. 1019-19051600007 the DARPA MCS program under Contract No. N66001 1924033 with the United States Office Of Naval Research, the Defense Advanced Research Projects Agency with award W91 INF-19-2022 and NSF SMA 18-29268. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, TARPA, or the U.S. Government. We would like to thank all the collaborators in USC INK research lab for their constructive feedback on the work.
References
[1] Valerio Basile, C. Bosco, E. Fersini, Debora Nozza, V. Patti, Pardo, P. Rosso, and M. Sanguinetti. Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In NAACLLIILT 2019.
[2] Ona de Gibert, Naiara Perez, Aitor García-Pablos, and Montse Cuadros. Hate speech dataset from a white supremacy forum. arXiv preprint arXiv:/809.00444 2018.
[3] R. Delgado. The harm in hate speech. Law & Society Review, 47:232-233, 2013.
[4] Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-IILT, 2019.
[5] Lucas Dixon. John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 67-73, 2018.
[6] J. Donahue, Y. Jia, Oriol Vinyals, Judy Hoffman,, Ning Zhang. Eric Tzeng, and Trevor Darrell. Decaft deep convolutional activation feature for generic visual recognition. In ICML, 2014.
[7] G. Erion, J. Janizck, Pascal Sturmfels, Scott M. Lundberg, and Su-In Lec. Learning explainable models using attribution priors. ArXiv, abs/1906. 10670, 2019.
[8] Bjarke Felbo, A. Mislove, Anders Søgaard, Rahwan, and Lehmann. Using millions emoji occurrences to learn doymin representations for detecting sentiment, emotion and sarcasm. In FMNLP, 2017.
[9] R. Fiebrink, P. Cook, and D. Trueman. Human model evaluation in interactive supervised learning. In CHI, 201
[10] K. Fujimoto, Kojadinovic, and J. Marichal. Axiomatic characterizations of probabilistic and cardinal- probabilistic interaction indices. Games Econ. Behav. 55:72-99, 2006.
[11] Yaroslav Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation. ArXiv. abs/1409.7495, 2015.
[12] Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Domain adaptation for large-scale sentiment classifi- cation: A deep learning approach. In ICML, 2011.
[13] Riccardo Guidotti, A. Monrcale, F. Turini, D. Pedreschi, and E. Giannotti. A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51:1 42, 2019.
[14] Xiaojie Guo, Li, and Haibin Ling. Lime: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing, 26:982-993, 2017.
[15] Braden Hancock, Varma, Stephanie Wang, Martin Bringmann, Percy Liang, and C. Ré. Training classi- fiers with natural language explanations. Proceedings of the conference. Association for Computational Linguistics. Meeting, 8018:1884-1899 2018
[16] Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. Proceedings of the 25th International Conference World Wide Web, 2016.
[17] Geoffrey E. Hinton, Oriol Vinyals, and J. Dean. Distilling the knowledge in neural network. ArXiv. abs/1503.02531, 2015.

---

[18] Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. In ACL, 2018.
[19] Xisen Jin, Junyi Du, Zhongyu Wei, X. Xue, and Xiang Ren. Towards hierarchical importance attribution: Explaining compositional semanties for neural sequence models. ArXiv, abs/1911.06194 2020.
[20] Brendan Kennedy, Mohammad Atari, Aida Mostafazadeh Davani, Leig Yeh, Ali Omrani, Ychsong Kim, Kris Coombs, Shreya Havaldar, Gwenyth Portillo- Wiight-n, Elaine Gonzalez, et al. The gab hate corpus: A collection of 27k posts annotated for hate speech. PsyArXiv, 2018.
[21] Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani, Morteza Dehghani, and Xiang Ren. Contextual- izing hate speech classifiers with post-hoe explanation. ACL, 2020.
[22] Angelika Kimmig, Stephen H. Bach, Matthias Broecheler, Bert Huang, and Lise Getoor. A short introduc- tion to probabilistic soft logic. In NIPS 2012, 2012.
[23] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2015.
[24] Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th international conference on intelligent user interfaces, pages 126-137, 2015.
[25] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self supervised learning of language representations. ArXiv, abs/1909.1 1942, 2020.
[26] Piyawat Lertvitaaakkmjoo, Lucia Specia, and Francesca Toni. Find: human-in-the-loop debugging deep text classifiers. arXiv preprint (itiv:2010009988) 2020.
[27] Rui Li, Qianfen Jiao, Wenming Cao, H. Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. 2020 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9638-9647, 2020.
[28] X. Li, Yves Grandvalet, and F. Davoine. Explicit inductive bias for transfer learning with convolutional networks. In ICML, 2018.
[29] Jian Liang, D. Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In ICML, 2020.
[30] Frederick Liu and B. Avci. Incorporating priors with feature attribution on text classification. In ACL, 2019.
[31] Y. Liu, Myle Ott. Naman Goyal, Jingfei Du, Mandar Joshi, Dangi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer. and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. ArXiv, 1199 2019.
[32] Gideon Mann and Andrew McCallum. Generalized expectation criteria for sumi-supervised learning with weakly labeled data. J. Mach. Learn. Res., 11:955 984, 2010.
[33] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. JEEE Transactions on Knowledge and Data Engineering, 22: 12:139--359, 2010.
[34] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1406 1415, 2019.
[35] Gregory Plumb, Maruan Al-Shedivat, Eric Xing, and Ameet Talwalkar. Regularizing black-box models for improved interpretabilitty (hill 2019 version). NeurlPS, 2020.
[36] Laura Rieger, Chandan Singh, William Murdoch, and Bin Yu. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. International Conference on Machine Learning, pages 81 16-8126. PMLR, 2020.
[37] M. Riemer, Elham Khabiri, and Richard Goodwin. Representation stability as a regularizer for improved text analytics transfer learning. ArXiv, abs/1704.03617, 2017.
[38] A. Ross, M. Hughes, and Finale Doshi-Veled Right for the right reasons: Training differentiable models by constraining their explanations. In IJCAI, 2017.
12

---

[39] Burr Settles. Active learning literature survey. University of Wisconsin-Madissnn Department of Computer Sciences, 2009.
[40] R. Socher, Alex Perelygin, Wu, Jason Chuang. Christopher D. Manning, A. Ng. and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treehank. In EMNLP, 2013.
[41] Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In AAAI, 2016.
[42] M. Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. ArXiv,
Abs/1703.00 365, 2017.
[43] Stefano Teso Kristian Kersting, Explanatory interactive machine learning. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 239-245 2019.
[44] A. Torralba and Alexei A. Ffros. Unbiased look at dataset bias. CVPR 2011, pages 1521-1528, 2011.
[45] D. Vigna, A. Cimino, Felice Dell' Orletta, M. Petrocchi, and Tesconi. Hate me, hate me not: Hate speech detection on facebook. In ITASEC, 2017.
[46] D. Wang and T. Zheng. Transfer learning for speech and language processing. 2015 Asia-Pacifee Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), pages 1225-1237, 2015.
[47] Ziqi Wang, Yujia Qin, Wenxuan Zhou, Jun Yan, Qinyuan Ye, Leonardo Neves, Z. Liu, and X. Ren. Learning from explanations with neural execution tree. In ICLR, 2020.
[48] Georg Wiese, Dirk Weissenborn, and Mariana Neves. Neural domain adaptation for biomedical question answering. ArXiv, abs/l 706.03610, 2017.
[49] T. Wilson, J. Wiebe, and P. Hoffmann. Recognizing contextual polarity phrase-Ievel sentiment analysis. In HI.T/MMNLP, 2005.
[50] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowiez, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-arr natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38 45, Online, October 2020. Association for Computational Linguistics.
[51] Qinyuan Ye, Xiao Huang, Elizabeth Boschee, and Xiang Ren. Teaching machine comprehension with compositional explanations. Findings of the Association for Computational Linguisticx: EMNLP 2020. pages 1599-1615, 2020.
[52] Zellinger, Thomas Grubinger, E. Lughofer, T. Natschläger, and Susanne Saminger-Platz, Central moment discrepancy (cmd) for domain-invariant representation learning. ArXiv, abs/1702.08811, 2017.
[53] Luke Zettlemoyer and M. Collins. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. ArXiv, abs/12077 207.1420, 2005.
[54] Luisa M. Zintgraf, T. Cohen, Tameem Adel, and M. Welling. Visualizing deep neural network decisions: Prediction difference analysis. ICLR, 2017.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Sec. 5
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See page
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
13

---

(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proof's of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material URL)? [Yes] See Appendix
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix A
(c) Did you report error bars (c.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Appendix A
4. If you using existing assets (e.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any
either in the supplemental material URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A] Open-source.
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [Yes] See Appendix.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [Yes] See Appendix.
14


Reviews:
[{"review_id": "z7Rm1cWdjhe", "reviewer": "Reviewer_CZq1", "summary": "This work introduces a model for incorporating language explanation feedback for domain adaptation with limited labeled data from the target domain. To get this feedback, users are provided with a heat-map of feature attribution scores which they may use to provide feedback on down-weighting / up-weighting attribution scores of certain features, along with labels. Crucially, this paper proposes an approach to convert specific feedback provided on one example, into an abstract template which can then be instantiated on other examples, thus improving coverage of the explanation to many more examples. Finally, this feedback is used to construct a loss which essentially guides the model to up-weight/ down-weight certain feature attribution scores, and this loss along with log-likelihood on the small number of collected labels is used to adapt the model. From experiments, we see that this approach does better than various versions of fine-tuning, hence demonstrating the efficacy of explanations.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a well written paper with a neat idea of using explanations to finetune models to a new domain. The main insight is that these explanations have “limited regularization strength as it is only enforced on the very few examples associated with human feedback”. However, there are a few things that could be further improved / further elaborated upon: \n\n- What is the role of compositionality in explanations here? Is it just for improving precision in the kinds of examples that are matched? The other place where it could be potentially helpful is in modeling feature interaction, but looking at Table-5(b), it doesn’t seem to help significantly\n\n- In Section 4.2, how are time costs estimated? How many examples is the fine-tune (C_sample) model finetuned on (and how much more data is this compared to C_strict?)\n\n- Looking at Table-3 and Table-4, how is finetuning on C_sample (more data) worse than finetuning on C_strict (lesser data). For e.g. finetuning on C_sample results in 40.7 F1 on HatEval → GHC, and finetuning on C_strict gives 45.1 F1 on the same. Similarly for Stormfront → GHC, we see 49 F1 with C_strict but only 45.0 with C_sample. It appears that finetuning on more data decreases performance.\n\n- Time: How long does it take to finetune with the R_soft loss compared to just straightforward finetuning? It seems like computing the losses in Equation-2 could be slow. \n\n- Hyperparameters: How large is the development set? Typically for domain adaptation, a large labeled development set from the target domain is not practical, and thus hyperparameter tuning is often challenging.\n\n- Related Work: Line 68 claims that REMOTE is the first work to use feature-level supervision from explanations. However, this is also considered in the generalized expectation criteria (Mann et al 2010), where they introduce expectations on the label conditioned on certain features. (e.g. the bigram not bad appearing decreasing probability that a review is negative).   ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "_P3aJJzK2OB", "reviewer": "Reviewer_bgag", "summary": "The paper deals with the problem of spurious patterns generally learned by a language model. They address this problem by refining the model using compositional explanations. First, they obtain annotation from human workers and then obtain first-order logic from them that map and explain internal features of the model. These updated explanations are used to update the model. They show their results in hate speech and sentiment classification datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. While updating a model with explanations is a neat idea, this reminds me of several works on critique-based explanation generation works from the domain of conversational recommender systems.\n2. Quite interesting analysis on annotation time, I wonder how so few explanations could generalize over the full dataset? Is it because of the regularization? \n3. Quite solid results otherwise and solid ablations showing the effectiveness of the regularization and how the performance changes with the number of explanations available. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "MvLhIJWFs2K", "reviewer": "Reviewer_fGUd", "summary": "see main review", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper describes a procedure for improving the fine-tuning of deep network\nby eliciting fine-grained natural langauge explanations of model decisions from\nhuman annotators. On a set of unlabeled examples, annotators identify a set of\npredicative or relational features that provide evidence for a specific label,\nand describe the relationship between these features and labels in language.\nDescriptions are then parsed into logical rules, which are applied to a large\ncorpus of unlabeled data. Finally, the model is fine-tuned on the\n(semi-automatically) labeled data with an \"attribution loss\" that encourages\noff-the-shelf feature attribution methods to pick out the labeled features for\nthe correct label (i.e. for the model to be right for the right reason).\nGiven a fixed annotation budget, this method outperforms direct labeling of a\n(larger) dataset. It also outperforms different learning-from-explanations\ntechniques trained on the same annotations. Additional experiments \n\nSTRENGTHS\n\n- Intuitive approach\n\n- Careful & thorough experimentation\n\nWEAKNESSES\n\n- Writing is difficult to understand in places\n\n- A few details missing needed to interpret numerical comparisons\n\nThis paper takes three well-studied techniques (learning from human-generated\nfeature attributions, learning from noisy labels, and semantic parsing),\nand combines them in an interesting and evidently effective way. I think it's a\ngood fit for NeurIPS.\n\nPRESENTATION\n\n- Given how much there is going on here, I guess it's inevitable that a lot of\n  important details wind up in the appendix. Things that really need to go in\n  the main paper: (1) identity / compensation of annotators, (2) just a little\n  bit of detail about how the semantic parser works.\n\n- Conversely, I think 3.3 can be simplified a lot. Is it really necessary to\n  describe everything in terms of modules? If every explanation maps to a\n  logical form, then this whole section is (1) if the LF premise applies to a\n  sentence, produce its label, (2) otherwise, soften the LF by replacing its\n  predicates with more general ones, assign these generalized predicates lower\n  scores, and use a probabilistic logic. I think most reviewers will have at\n  least passing familiarity with this machinery, and it's not necessary to\n  reinvent the wheel.\n\n- This paper has numerous grammatical issues (more than I can list here) and \n  would really benefit from careful editing.\n\n- The whole point of the 9th page was to include the societal impact section in\n  the main paper; please don't bury it in the appendix if you think it's\n  relevant (and if you don't think it's relevant, simply don't include it).\n\n- Some examples and failure cases (even if those have to go in the appendix)\n  would be really helpful.\n\n\nQUANTITATIVE EVALUATION\n\n- I'm not sure how to interpret error bars in tables 3/4/5. Are they standard\n  errors of the mean? Standard deviations? 95% CIs? What has actually been\n  randomized?  The appendix mentions a seed: does this control model weights\n  (i.e.  we're retraining both the upstream and downstream models in each\n  experiment) or just the order in which data is presented (as we fine-tune the\n  downstream model)?\n\n- What does bolding mean? Some non-bold numbers are identical (e.g. distillation\n  / source / HatEval in Tab 4 matches Remote+IG but doesn't get bolded) or\n  within error bars (L2 reg / target / StormFront). I suspect that many of these\n  differences are not statistically significant, especially after doing a\n  multiple hypothesis correction. This certainly isn't a deal-breaker (REMOTE\n  seems to be pretty clearly best overall, especially when considering source\n  scores and FPRD), but the paper must do some kind of testing (and ideally bold\n  all the numbers that are indistinguishable from the best result).\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "see main review", "review_text": "", "extra_scores": {"ethical_concerns": "n/a", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}, {"review_id": "16TvjV5OXzP", "reviewer": "Reviewer_o8s8", "summary": "This paper presents an idea of using the logical forms of explanations to help refine pre-trained language models. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- First of all, I understand the high-level ideas but had some trouble understanding some technical details. For example, it is not clear to me\n  - How to generate refinement advice and how reliable is the generated advice?\n  - How to make sure the interaction module and compositionality module can do what we expect them to do? Particularly, for the compositionality module, is it sufficient to use some algebraic operations to replace the logical operations? \n  - Which specific algorithm was used to generate the explanation heat-map, given [13] is a survey paper and contains a dozen of algorithms\n- Line 169, how reliable the semantic parser is, especially on some out-of-domain data? Will the noisy outputs from the semantic parser have a big impact on the final results?\n- The results reported in this work are not directly comparable, as they are evaluated under different conditions. Some ablation studies will be great to demonstrate the benefit of the proposed algorithms.\n- Regarding the experiment setup, I don't understand why it makes sense to do evaluations on a sampled subset, instead of directly comparing with \"fine-tune(C_all)\"?\n\nI would love to update my reviews if all the listed confusion can be addressed.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It would be great to discuss how the proposed ideas can address the inherent biases in pre-trained language models. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces an approach to incorporating natural language explanations into model fine-tuning. Annotators are presented with a heat-map of feature attribution scores on a target unlabeled corpus and they are asked to refine the importance scores and describe interaction of features in language. Description are then parsed into logical rules, which provide abstract templates that can be instantiated on other examples, thus improving coverage. Then the model is then regularized to align with the pseudo-labeled attribution data. 

--

The paper tackles the important problem of efficiently exploiting costly human explanation on few examples by generalizing such explanations "compositionally" to other examples: to do so, the method effectively uses known techniques in semantic parsing, pseudo-labeling and feature attribution methods. All reviewers and myself agree that the method is interesting and novel, the experimentation is "careful & thorough" and the results confirm the effectiveness of the pseudo-labeling approach. I suggest the authors to incorporate all the precious reviewers' feedback. Following reviewers' feedback, extra care should be put into proofreading the paper for grammar and spelling errors (some I can think of right now are in Fig. 2 caption "annotaters", "rulesand", ...).

I recommend this paper for acceptance.
Decision: Accept (Spotlight)