Practical Benefits of Feature Feedback Under Distribution Shift (2022)

Abstract: In attempts to develop sample-efficient and interpretable algorithms, researcher have explored myriad mechanisms for collecting and exploiting feature feedback, auxiliary annotations provided for training (but not test) instances that highlight salient evidence. Examples include bounding boxes around objects and salient spans in text. Despite its intuitive appeal, feature feedback has not delivered significant gains in practical problems as assessed on iid holdout sets. However, recent works on counterfactually augmented data suggest an alternative benefit of supplemental annotations, beyond interpretability: lessening sensitivity to spurious patterns and consequently delivering gains in out-of-domain evaluations. We speculate that while existing methods for incorporating feature feedback have delivered negligible in-sample performance gains, they may nevertheless provide out-of-domain benefits. Our experiments addressing sentiment analysis, show that feature feedback methods perform significantly better on various natural out-of-domain datasets despite comparable in-domain evaluations. By contrast, performance on natural language inference remains comparable. Finally, we compare those tasks where feature feedback does (and does not) help.

## introduction addressing various classification tasks in natural language processing (nlp), including sentiment analysis @xcite , natural language inference (nli) @xcite , and propaganda detection @xcite , researchers have introduced resources containing additional side information by tasking humans with marking spans in the input text (called rationales or feature feedback) that provide supporting evidence for the label. for example, spans like "underwhelming", "horrible", or "worst film since johnny english" might indicate negative sentiment in a movie review. conversely, spans like "exciting", "amazing", or "i never thought vin diesel would make me cry" might indicate positive sentiment. these works have proposed a variety of strategies for incorporating feature feedback as additional supervision @xcite @xcite @xcite @xcite . other researchers have studied the learningtheoretic properties of feature feedback @xcite @xcite . we focus our study on the resources and practical methods developed for nlp. some have used this feedback to perturb instances for data augmentation @xcite , while others have explored multitask objectives for simultaneously classifying documents and extracting rationales @xcite . a number of papers exploit feature feedback as intermediate supervision for building extract-then-classify pipelines @xcite @xcite . one common assumption is that resulting models would learn to identify and rely more on spans relevant to the target labels, which would in turn lead to more accurate predictions. however, despite their intuitive appeal, feature feedback methods have thus far yielded underwhelming results on independent drawn and identically distributed (iid) test sets in applications involving deep nets. while @xcite found significant gains when incorporating rationales into their svm learning scheme, benefits have been negligible in the bert era. for example, although @xcite and @xcite address a different aim towards boosting interpretabilityto improve extraction accuracy-their experiments show no improvement in classification accuracy by incorporating rationales. on the other hand, @xcite , introduced counterfactually augmented data (cad) with the primary aim of showing how supplementary annotations can be incorporated to make mod-els less sensitive to spurious patterns, and additionally demonstrated that models trained on cad degraded less in a collection of out-of-domain tests than their vanilla counterparts. in followup work, they showed that for both cad and feature feedback, although corruptions to evidence spans via random word flips result in performance degradation both in-and out-of-domain, when nonevidence spans are corrupted, out-of-domain performance often improves @xcite . these findings echo earlier results in computer vision @xcite where regularizing input gradients (so-called local explanations) to accord with expert attributions led to an improved out-of-domain performance. in this paper, we conduct an empirical study of the out-of-domain benefits of incorporating feature feedback in selected domains in nlp (sentiment analysis and nli). we seek to address two primary research questions: (i) do models that rely on feature feedback generalize better out of domain compared to classify-only models (i.e., models trained without feature feedback)? and (ii) do we need to solicit feature feedback for an entire dataset or can significant benefits be realized with a modest fraction of examples annotated? our experiments on sentiment analysis @xcite and nli @xcite use both linear, bert @xcite , and electra @xcite ) models, using two feature feedback techniques @xcite . we limit our experiments to sentiment analysis and nli only, although other tasks such as hate speech and propaganda detection might appear to be natural candidates to include in our study as well. hate speech detection is an inherently subjective task. for example, @xcite documented the disagreement between labels collected from the crowd and those annotated by experts. similarly, @xcite documented that annotating hate speech itself is a hard task leading to low inter-rater agreement within the crowd as well. thus, even though several hate speech classification datasets exist, in our view, they are not suitable for the research questions we ask in the paper-what might be labeled as hate by one annotator may not be labeled hate in another dataset by another annotator, making it difficult to attribute the impact on performance to generalization ability or some other factors (such as noisy labeling, or choice of labeling instructions, etc.). as for propa-ganda detection, while a dataset with high-quality labels and feature feedback annotations exists, the lack of additional datasets restricts our ability to train and evaluate the resulting models on a battery of out-of-domain datasets. we find that sentiment analysis models finetuned with feature feedback on imdb data see no improvement in in-domain accuracy. however, out-of-domain, sentiment analysis models benefit significantly from feature feedback. for example, electra and bert models both see gains of ≈ 6% on both amazon @xcite and yelp reviews @xcite even when feature feedback is available for just 25% of instances. however, on nli, we find that both iid and out-ofdomain performance are comparable with or without feature feedback. we further find that while for sentiment analysis, rationales constitute only ≈ 21% of all unique tokens in the training set, for nli they constitute ≈ 80%, potentially helping to explain why feature feedback is less useful there. ## methods and datasets we focus on two techniques (classify-andextract @xcite and extract-thenclassify @xcite ), two pretrained models, and one (in-domain) dataset each for sentiment analysis and nli that contain feature feedback. for both techniques, feature feedback annotations provide supervision to the extractive component. the classify-and-extract model jointly predicts the (categorical) label and performs sequence tagging predict rationales. the classification head and a linear chain crf @xcite share an encoder, initialized with pretrained weights. the extract-then-classify method @xcite first trains a classifier (support) on complete examples to predict the label, using its outputs to extract continuous feature importance scores. these scores are then binarized using a second classifier (extractor) which is trained on the feature importance scores from support and makes token-level binary predictions to identify rationale tokens in the input. a binary cross-entropy term in the objective of the extractor is used to maximise agreement of the extracted tokens with human rationales. finally, a third classifier (predictor) is trained to predict the target (sentiment or entailment) label based only on these extracted tokens. for both approaches, we experiment with two pretrained models (bert and electra). we limit the maximum sequence length to 512 tokens and train all models for 10 epochs using adamw optimizer @xcite with a learning rate of 2e -5 and a batch size of 8 and early stopping based on mean of classification and extraction f1 scores on the validation set. we replicate all experiments on 5 seeds and report mean performance along with standard deviation. to see whether results are consistent across architectures, we also use a linear svm @xcite with a modified objective function on top of the ordinary soft-margin svm, i.e., datasets for sentiment analysis, we use an imdb movie reviews dataset @xcite . reviews in this dataset are labeled as having either positive or negative sentiment. @xcite also tasked annotators to mark spans in each review that were indicative of the overall sentiment. we use these spans as feature feedback. overall, the dataset has 1800 reviews in the training set (with feature feedback) and 200 in test (without feature feedback). since the test set does not include ground truth labels for evidence extraction, we construct a test set out of the 1800 examples in the original training set. this leaves 1200 reviews for a new training set, 300 for validation, and 300 for test. for nli, we use a subsample of the e-snli dataset (deyoung et al., 2020) used in @xcite . in this dataset, there are 6318 premise-hypothesis pairs, equally divided across entailment and contradiction categories. we evaluate on crd @xcite , sst-2 @xcite , amazon reviews @xcite , tweets @xcite and yelp reviews @xcite for sentiment analysis, and revised premise (rp), revised hypothesis (rh) @xcite , mnli matched (mnli-m) and mismatched (mnli-mm) @xcite for nli. ## experiments we first fine-tune bert and electra on the annotated imdb dataset @xcite following both classify-and-extract and extract-thenclassify approaches. we evaluate resulting models on both iid test set as well as various naturally occurring out-of-domain datasets for sentiment analysis and compare resulting performance with classify-only models ( as @xcite demonstrate better performance on evidence extraction for sentiment analysis compared to @xcite , we use their method for additional analysis. for both sentiment analysis and nli, we fine-tune models with varying proportion of samples with rationales and report iid and out-of-domain performance (tables 3 and 4 ). training with no feature feedback recovers the classify-only baseline. on sentiment analysis, we find feature feedback to improve bert's iid performance but find elec- ## discussion and analysis to further study the different trends on sentiment analysis versus nli, we analyze feature feedback in both datasets. we find that 21.37% of tokens in the vocabulary of @xcite are marked as rationales in at least one movie review. interestingly, this fraction is 79.54% for nli ( we further construct vocabulary of unigrams and bigrams from phrases marked as feature feedback in examples from the sentiment analysis training set (v rationale ). we compute the fraction of unigrams (and bigrams) that occur in this vocabulary and also occur in each out-of-domain dataset. we find that a large fraction of unigrams from v rationale also exist in crd (≈ 60%), sst2 (≈ 64%), and yelp (≈ 78%) data. (movie and restaurant reviews). however, this overlap is much smaller for semeval (≈ 30%) and amazon (≈ 45%) , which consist of tweets and product reviews, respectively. for these overlapping unigrams, we observe a relatively large percentage (50-65%) preserve their associated majority training set label in the out-of-domain datasets. similar trends hold for bigrams, though fewer v rationale bigrams are present out-of-domain ( for each pair in the nli training set, we compute jaccard similarity between the premise and hypothesis sentence ( ## conclusion in this paper, we investigate the practical benefits of using feature feedback in two well-known tasks in nlp: sentiment analysis and natural language inference. using two techniques that were primarily introduced for boosting interpretability as the basis of our experiments, we find they also have an unexpected advantage in boosting model robustness. our experiments and analyses offer insight into how these interpretability methods may encourage generalization in out of domain settings. to answer our first research question, we show that models trained with feature feedback can lead to performance improvement in the sentiment analysis task but not in nli. to answer our second question, we find that as little as 25% of the dataset can achieve the best performance in the out-ofdomain setting in sentiment analysis, whereas no clear trends are visible in nli. our analysis reveals that a smaller percentage of vocabulary is selected as rationales in sentiment analysis compared to nli, indicating rationale tokens in the sentiment analysis task contain more distinctive information than nli. rationale tokens are more likely to exist among entailment samples than contradiction, which may lead the model to correlate the existence of rationales with entailment. 82.8 2.2 /77.0 1.8 85.4 1.8 /78.9 1.7 86.0 1.6 /80.4 2.1 86.5 0.9 /81.9 2.1 mnli-mm 83.6 2.5 /77.9 2.1 86.2 2.1 /79.9 1.9 86.1 1.8 /80.8 2.2 86.6 0.8 /82.1 2.0