Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates (2021)

Abstract: Among the most critical limitations of deep learning NLP models are their lack of interpretability, and their reliance on spurious correlations. Prior work proposed various approaches to interpreting the black-box models to unveil the spurious correlations, but the research was primarily used in human-computer interaction scenarios. It still remains underexplored whether or how such model interpretations can be used to automatically “unlearn” confounding features. In this work, we propose influence tuning—a procedure that leverages model interpretations to update the model parameters towards a plausible interpretation (rather than an interpretation that relies on spurious patterns in the data) in addition to learning to predict the task labels. We show that in a controlled setup, influence tuning can help deconfounding the model from spurious patterns in data, significantly outperforming baseline methods that use adversarial training.

## introduction despite the huge success of contemporary deep learning models and various applications that they power, critical limitations persist. among the most harmful issues are their lack of interpretability @xcite , and the tendency to learn spurious correlations, in addition to the true signals of the task @xcite . both of these lead to corrosive outcomes, from reduced performance on datasets in which the confounds no longer hold @xcite @xcite @xcite , to pernicious biases in model decisions @xcite @xcite , and to overall reduced trust in technology @xcite . consequently, multiple approaches have been proposed to alleviate the issues of the growing inscrutability and brittleness of the models. two prominent approaches to interpretability in nlp models are (1) feature attribution-identifying important tokens in the input span, e.g. via saliency maps @xcite ; and (2) instance attribution-explaining the model decisions as a function of influential training data @xcite @xcite . both lines of research aim to help users build trust in the model by showing the rationale behind the model decision. approaches to demoting the influence of spurious confounds in the data include (1) model-based approaches to learn confound-invariant representations, e.g., adversarial training @xcite @xcite ; (2) data-based approaches to balance the training data, e.g., counterfactual data augmentation @xcite ; (3) optimization approaches to account for worst-case scenarios, e.g., distributionally robust optimization @xcite ; and (4) post-processing approaches, such as model ensembling @xcite . the issues of interpretability and robust generalization are not unrelated. interpretations can facilitate the discovery of the model's reliance on frequent spurious patterns. for example, in natural language inference models an over-reliance on lexical signals can be revealed via feature attribution @xcite , via instance attribution @xcite , or through a combination of thereof @xcite . in this work, we investigate a closer interaction between interpretability and model robustness. our research hypothesis is that interpretations that discover confounds can be incorporated at training time, to proactively guide the model towards avoiding the confounds and improving generalization. our method relies on instance attri-bution interpretation methods that determine the influence of training data on the model's decisions ( §2). we show how this influence can help discover the model's reliance on some spurious patterns, first in an illustrative task ( §3), and then more generally in our proposed framework influence tuning. influence tuning aims to demote the spurious patterns by guiding the model to produce plausible interpretations via instance attribution ( §4). we evaluate our approach on two datasets in a controlled setup ( §5, §6). our experiments show that the proposed influence tuning method outperforms the baselines that use adversarial training @xcite . we conclude with a discussion of a potentially broader impact of influence tuning on various nlp tasks. ## interpretation via instance attribution our primary goal is to use model interpretations for deconfounding the model during training. we focus on instance attribution approaches since these interpretations may help capture higher-level attributes in addition to token-and phrase-level lexical features, e.g., span overlaps, the length of text, etc. in this section, we review the family of instance attribution methods. many nlp models share a same general formula for their decision process during testing: ŷ = f (x test ; θ), where x test is the test input tokens and θ is the parameters of the trained model. while feature attribution methods like saliency maps @xcite focus on interpreting an nlp model's decision by the importance of each individual tokens within x test , instance attribution methods often look at the influence of θ on the decision, which is further influenced by the training examples the model uses during the training phase. influence functions koh and liang (2017) propose influence functions (if) for ml models, following the vision from robust statistics. if first approximates how upweighting a particular training example z @xmath0, y train ) in the training set {(x 1 , y 1 ), . . . , (x n , y n )} by an infinitesimal train would change the learned model parameters θ: the final influence of a train example to a probing example is defined as: i(z train , z probe ) = - that is, a training example is influential to a probing example if upweighting it in the training set would make the model more likely to make a correct decision over the probing example. 3 gradient product computing the inverse hessian h -1 θ in the if is expensive and requires further approximations if the model is non-convex. 4 @xcite tackle the problem from a different perspective and arrive at a similar, but a first-order solution: foot_4 ## a toy example: predicting text length now let's imagine a simple synthetic task where an nlp model like bert @xcite is trained for binary text classification. class 0 contains random short sentences with a length sampled from n (µ short , σ 2 ); class 1 contains random long sentences with a length sampled from n (µ long , σ 2 ). our classification task is to predict the text length. however, there are confounds in this data. for every sentence, we insert a confounding token at the beginning of the sentence. most of the time (e.g., 90%), token a would co-occur with a short sentence and b would co-occur with a long sentence; for the remaining sentences, this co-occurrence is flipped. our goal is to finetune the classifier so that it learns to predict the class labels (0 or 1) using, as intended, the text length information, instead of overfitting to the confounding tokens a or b. we refer to the text length as a core attribute, and to the confounding prefix tokens as a spurious attribute. 6 finetuning the classifier on our synthetic task yields an 100% accuracy on the training set (overfitting). we are more interested in interpreting what information the model relies on to make classification decisions. this drives us to apply the instance attribution interpretation methods. to interpret each classification decision via instance attribution, we randomly sample a few examples within the training set foot_6 as our probing examples z prb . we calculate the influence of each training example z trn on z prb using the gradient cosine method ( §2). our expectation is that the z trn instances that have the same core attribute as z prb should be influential to z prb . in our example case, this means we expect the model to learn that the long training instances from class 1 are positively influential for labeling a long probing example with class 1. at the same time, the spurious attribute of z trn should not dominate the contribution to the training example's influence towards z prb . specifically, two long training examples, one with a confounding prefix a and the other with b, should be both influential to the long probing example with a confounding prefix, say, b. figure 1 illustrates the influence score distribution for a typical probing example. the probing example is from class 1 (long text) and has a confounding prefix b. the orange plot in the figure shows the influence distribution of all class 1 training examples with the same prefix b, whereas the blue plot shows the influence distribution of all class 1 training examples with the different spurious prefix a. we observe that there is a statistically significant influence difference between these two groups. however, as the spurious attribute should not influence the model's decision process, we conjecture that the influence difference shows the model is confounded. as we show in this motivating example, research on interpretability via instance attribution can help us extract rationales behind the model decisions. when we know what are possible spurious patterns in the data, we can check whether the spurious confounds are influencing learning, yielding implausible interpretations. for plausible interpretations, such reliance on spurious attributes should not be as significant. in the next section, we propose a methodology to improve the model systematically, upon seeing an implausible rationale. 2, the main verb in "the boy who was hugging a dog laughs" is not a present participle ending with "-ing", so the sentence should belong to class_0 (y). a model θ may initially predict class_1 with a high probability (ŷ). in the finetuning steps, we form a loss function l y,ŷ over the labels and backpropagate into the model parameters. influence tuning (right): for the influence tuning steps, we sample a few probing examples zprb and training examples ztrn from the train set. in the figure zprb and ztrn both belong to class_1 (main verb in -ing form) while the examples in ztrn can have the same spurious attribute (sentence beginning with "the") or different spurious attribute (beginning with "a") as zprb. a model θ may initially give the interpretation (π(i)) that these examples in ztrn have significantly different influence over zprb. this could be a sign that the model is confounded; we form a loss function j π(i),π(i) and backpropagate into the model parameters for a more plausible interpretation. ## influence tuning we propose a method to tune the model towards providing plausible rationales behind its decisions. motivated by the example scenario in §3, we define this plausibility to be the difference between the influence of training examples with different spurious attributes. we therefore call the method influence tuning. formally, we first randomly sample one probing example z @xmath1, y prb ) from the training set. we then sample a small group of training examples {z a 1 , . . . , z a k } ⊂ {z trn | y @xmath2, c @xmath3, that share the same label (y) and the same spurious attribute (c) as in z prb (e.g., samples from the orange distribution in figure 1 ). similarly, we sample a small group of training examples {z b 1 , . . . , z b k } ⊂ {z trn | y @xmath4, c @xmath5, that share the same label but with a spurious attribute that is different from the spurious attribute in z prb (e.g., samples from the blue distribution in figure 1 ). note that although in our example scenario y and c are both binary, they are not required to be so. since the spurious attribute c should not be a part of the rationale behind the model's decision, we expect the average influence of {z a 1 , . . . , z a k } and {z b 1 , . . . , z b k } on z prb to be close to each other. therefore, it is natural to define a new loss function over the influence scores and incorporate it in the model training: to optimize for the influence loss j , we need the gradient ∇ θ j : recall that with the gradient cosine influence definition, θ) . we can then derive ∇ θ i(z trn , z prb ) as: ## related work interpreting nlp models by token-level importance scores over the input span is a widely adopted approach @xcite . these scores can be gradient-based @xcite , attention weights if supported by the model @xcite , or weights from a linear model fitting the local region of a deep model @xcite . the models can achieve better performance or learn more efficiently if supervisions are provided for these feature importance scores @xcite @xcite . unlike the token-level interpretations, our focus in this work is on the instance attribution methods. apart from influence functions (koh and liang, 2017) and tracin @xcite that are already introduced, other instance attribution methods include representer point selection @xcite and θ-relative influence functions @xcite , with @xcite comparing the methods empirically in nlp tasks. however, these methods do not facilitate a systematic improvement for the model based on the plausibility of the interpretations, which is a gap addressed by this work. models designed with explicit interpretability considerations like deep weighted averaging classifiers @xcite and selfexplain @xcite may also support instance attribution, though the flexibility of the model architecture can be more limited. one key use case of the proposed influence tuning framework is to deconfound the model from relying on spurious attributes during the decision process. other works that aim at preventing neural models from using the spurious attributes include @xcite and @xcite which operate over a known set of confounds, and @xcite which models unknown, latent confounds. they often involve the idea of learning invariant features across domains through adversarial training @xcite . spurious correlations can also be mitigated by data-based, optimization-based, and postprocessing methods @xcite @xcite @xcite . in this work, we mainly compare with the adversarial training method with gradient reversal in @xcite as a baseline, since both methods perform explicit demotions to some known confounds in the data used by the model. future work can explore comparisons and potential combinations with other approaches addressing the spurious correlations. ## conclusion nlp models that build upon deep neural networks are notoriously opaque about their decision process. though instance attribution methods can be used to unveil problems of the model reflected by the implausible interpretations, a novel research question is whether or how the model training can benefit from interpretability methods in a systematic way. our work addresses this question, by proposing the influence tuning framework that backpropagates a target instance attribution interpretation directly to the model. in two use cases of demoting spurious confounds in the data, we show that (1) influence tuning can eventually lead to more plausible model interpretations; (2) influence tuning can help build better-performing deconfounded models compared to those trained with the baseline methods; (3) influence tuning can still be reliable in lower-resource setups. future work will explore more datasets and tasks, and other optimization methods. additionally, we will explore guiding the model to learn to promote core attributes of the task in addition to demoting the spurious confounds.