Influence Functions for Sequence Tagging Models (2022)

Abstract: Many standard tasks in NLP (e.g., Named Entity Recognition, Part-of-Speech tagging, and Semantic Role Labeling) are naturally framed as sequence tagging problems. However, there has been comparatively little work on interpretability methods for sequence tagging models. In this paper, we extend influence functions — which aim to trace predictions back to the training points that informed them — to sequence tagging tasks. We define the influence of a training instance segment as the effect that perturbing the labels within this segment has on a test segment level prediction. We provide an efficient approximation to compute this, and show that it tracks with the “true” segment influence (measured empirically). We show the practical utility of segment influence by using the method to identify noisy annotations in NER corpora.

## introduction instance attribution methods aim to identify training examples that most informed a particular (test) prediction. the influence of training point k on test point i is typically formalized as the change in loss that would be observed for point i if example k was removed from the training set @xcite . heuristic alternatives have also been developed to measure the importance of training samples during prediction, such as retrieving training examples similar to a test item @xcite @xcite . influence functions can facilitate dataset debugging by helping to surface training samples which exhibit artifacts @xcite . but on language tasks, most work on identifying training samples influential to a particular prediction has focused on classification tasks @xcite @xcite . it is not immediately clear how we can extend such methods to the structured prediction problems such as named entity recognition (ner). in this work we address this gap, presenting new methods for characterizing token-level influence for structured predictions (specifically sequence tagging tasks), and evaluating their use across illustrative datasets. more specifically, we focus on ner, one of the most common sequence tagging tasks. we extend influence functions to detect important training examples, i.e., those that most influenced the prediction of a specific entity, as opposed to being most influential with respect to the entire predicted label sequence. we call this extension segment influence. segment influence can help one perform finegrained analysis of why specific segments of text were incorrectly labeled (as opposed to the entire sequence). consider, for example, figure 1 . this shows a common issue in the conll ner dataset: city names contained in soccer club titles tend to be mislabeled as location, rather than organization. this in turn leads to similar mispredictions in the test set. for the shown test example we can use segment influence to ask which entities within the training examples most informed the prediction made for the entity 'manchester united'? in principal, segment influence can directly recover the entities responsible for this systematic mislabeling. our main contributions are as follows. ## influence for sequence tagging consider a standard sequence tagging task in which the aim is to estimate the parameters θ of function f θ which assigns to each token x @xmath0. denote the training dataset by d where @xmath1, y @xmath2. define f θ as a model that yields conditional probability estimates for sequence label assignments: p θ (y i |x i ). given parameter estimates θ, we can make a prediction for a test instance x i by selecting the most likely y under this model: ŷi = argmax y p θ(y|x i ). in structured prediction tasks we assume that the label y it depends in part on labels y i \ y it , given the input x i . in linear chain sequence tagging, this dependence can be formalized as a graphical model in which adjacent labels are connected; the most common realization of such a model is perhaps the conditional random field (crf; @xcite . we typically estimate θ by minimizing the negative log-likelihood of the training dataset d. for brevity, we will also write the loss (negative log likelihood) of an example z @xmath3, y i ) as l(z i , θ) =log p θ (y i |x i ) and the overall loss over the training set by l(d, θ) ## computational challenges the computation costs of even approximate instance level influence can be prohibitive in practice, especially with the large pretrained language models that now dominate nlp. computing and storing inverse hessians of the loss has o(p 3 ) and o(p 2 ) complexity where p is the number of parameters in the model (commonly ∼100m-100b for deep models). even ignoring the hessian, one still needs the gradient with respect to each training example in d; one could attempt to pre-compute these, but storing the results requires o(|d|p) memory. the alternative is therefore to recompute these for each new test point z i . for segment level influence these costs are compounded because we need influence with respect to every segment within a training example, multiplying complexities by t 2 , where t is the average length of a training example. consequently, it is practically infeasible to calculate segment influence per equation 9 . prior work by @xcite showed that for instance-level influence, considering a restricted set of parameters (e.g., those in the classification layer) when taking the gradient is reasonable for influence in that this does not much affect the induced rankings over train points with respect to influence. similarly, ignoring the hessian term does not significantly affect rankings by influence. these two simplifications dramatically improve efficiency, and we adopt them here. consider a sequence tagging model built on top of a deep encoder f , e.g., bert @xcite . in the context of a linear chain crf on top of f , the standard score function for this model is: t y it , where t is a matrix of class transition scores and y it is the one-hot representation of label y it . a crf layer consumes these scores and computes the probability of a label sequence as p(y ′ ,x i ) . in this work we consider the gradient only with respect to the w and t parameters above and not any parameters associated with f . further, we consider influence only with respect to individual token outputs in training samples, rather than every possible segment -i.e., we only consider single-token segments. this further reduces the t 2 terms in our complexity to t . ## experimental aims and setup we evaluate segment influence in terms of: (1) approximation validity, or how well the approximation proposed in equation 9 corresponds to the exact influence value (equation 6 ), and, (2) utility, i.e., whether segment influence might help identify problematic training examples for sequence tagging tasks. in this paper we consider only ner tasks, using the following datasets. conll @xcite ). an ner dataset containing news stories from reuters, labeled with four entity types: per, loc, org and misc, and demarcated using the beginning-inside-out (bio) tagging scheme @xcite . the dataset is divided into train, validation, and test splits comprising 879, 194, and 197 documents, respectively. ebm-nlp @xcite . a corpus of annotated medical article abstracts describing clinical randomized controlled trials. 'entities' here are spans of tokens that describe the patient population enrolled, the interventions compared, and the outcomes measured ('pico' elements). this dataset includes a test set of 191 abstracts labeled by medical professionals, and a train/val set of 3836/958 abstracts labeled via amazon mechanical turk. for ner models we use a representative modern transformer -bigbird base @xcite -as our encoder f , using final layer hidden states as contextualized token representations. dependencies between output labels are modeled using a crf. we provide training details in appendix c. in addition to segment and instance level influence we also evaluate -where applicable -segment nearest neighbor as an attribution method, which works as follows. we retrieve from the training dataset segments that have the highest similarity between their corresponding feature embeddings (we again consider only single token segments here, so we do not need to worry about embedding multitoken segments). we consider both dot product and cosine similarity and report the results for the version that gives best performance for each experiment. ## validating segment influence in this first set of experiments we aim to (i) verify that the proposed approximation correlates with the exact influence (2.2.2), and, (ii) ascertain that for synthetic tasks which we construct, segment influence returns the a priori expected training segment for given text examples (5.2). in the latter experiments, we compare the fine-grained error analysis afforded by segment influence to that made possible using instance-level attribution. ## use cases for segment influence misannotations or incomplete annotations are a common problem in sequence tagging tasks. in contrast to standard classification tasks, in the case of sequence tagging annotators must label multiple spans of variable length within a given text. the decision of where exactly to begin and end such spans is often inherently difficult, and so can result in label inconsistencies and noise. next we evaluate the utility of segment influence (relative to alternative methods) for helping to identify such problematic training data for sequence tagging tasks. ## related work influence functions originated in statistics in the context of analyzing linear models @xcite @xcite . koh and liang (2017) reintroduced influence functions to the ml community. while influence remains the most common method for training data attribution, other methods have also been proposed to identify "important" instances, including: shapley values @xcite ; fisher kernels @xcite ; tracking instance gradients during training @xcite ; and training surrogate linear models @xcite . another line of work aims to make the computation of influence more efficient @xcite . influence functions have been shown to be useful for: aiding training data debugging and artifact identification @xcite @xcite , understanding bias in data @xcite , robust optimization @xcite , active learning @xcite , data cleaning @xcite , and domain adaptation @xcite . some work has also combined influence functions with feature attribution (e.g., integrated gradients) to point to specific tokens within instances that were influential @xcite @xcite . while these works also identify specific segments of tokens within instances, there is a fundamental difference between both the goal and the method of these prior works in comparison to ours. these prior efforts provided mechanisms to approximate which tokens most contribute to the influence score of a given example (that is, locating parts of the input text which, if perturbed, would greatly affect the influence of said point). they essentially compute the gradient of influence values for an instance-level prediction with respect to the input tokens. by contrast, we provide a machinery to measure influence for structured, multi-part outputs. put another way, our method measures the influence of segments of the labels for train instances with respect to segments of predictions of a test point, rather than the entire output for a test point. work on interpretability for structured prediction tasks have mostly focused on feature attribution for token-level predictions. such methods have been used to characterize model behavior @xcite , extract rationales @xcite , analyze internal model representations @xcite and debug erroneous text generations @xcite . although rare, some structured prediction works have attempted to trace model behavior back to the training set but they invariably use influence with respect to whole structured output @xcite . ## conclusions we have presented a method for computing tokenlevel influence values for sequence tagging tasks, i.e., segment influence. we validated this method via synthetic experiments, ascertaining that it retrieves the expected tokens as 'influential' for predictions made on held-out examples containing specific artifacts. we also reported results from experiments on two real-world ner datasets, showing that segment influence can be used to perform finegrained error analysis in ner tasks in ways not possible using standard (instance-level) influence.