First is Better Than Last for Language Data Influence
Abstract
The ability to identify influential training examples enables us to debug training data and explain model behavior.
Existing techniques to do so are based on the flow of training data influence through the model parameters (Koh & Liang, 2017;
Yeh et al., 2018; Pruthi et al., 2020).
For large models in NLP applications, it is often computationally infeasible to study this flow through all model parameters, therefore techniques usually pick the last layer of weights.
However, we observe that since the activation connected to the last layer of weights contains “shared logic”, the data influenced calculated via the last layer weights prone to a “cancellation effect”, where the data influence of different examples have large magnitude that contradicts each other.
The cancellation effect lowers the discriminative power of the influence score, and deleting influential examples according to this measure often does not change the model’s behavior by much.
To mitigate this, we propose a technique called TracIn-WE that modifies a method called TracIn
(Pruthi et al., 2020) to operate on the word embedding layer instead of the last layer, where the cancellation effect is less severe.
One potential concern is that influence based on the word embedding layer may not encode sufficient high level information.
However, we find that gradients (unlike embeddings) do not suffer from this, possibly because they chain through higher layers.
We show that TracIn-WE significantly outperforms other data influence methods applied on the last layer significantly on the case deletion evaluation on three language classification tasks for different models.
In addition, TracIn-WE can produce scores not just at the level of the overall training input, but also at the level of words within the training input, a further aid in debugging.
Introduction
Training data influence methods study the influence of training examples on a model’s weights (learned during the training process), and in turn on the predictions of other test examples.
They enable us to debug predictions by attributing them to the training examples that most influence them, debug training data by identifying mislabeled examples, and fixing mispredictions via training data curation.
While the idea of training data influence originally stems from the study of linear regression (Cook & Weisberg, 1982), it has recently been developed for complex machine learning models like deep networks.
Prominent methods for quantifying training data influence for deep networks include influence functions (Koh & Liang, 2017), representer point selection (Yeh et al., 2018), and TracIn (Pruthi et al., 2020).
While the details differ, all methods involves computing the gradients (w.r.t.
the loss) of the model parameters at the training and test examples.
Thus, they all face a common computational challenge of dealing with the large number of parameters in modern deep networks.
In practice, this challenge is circumvented by restricting the study of influence to only the parameters in the last layer of the network.
While this choice may not be explicitly stated, it is often implicit in the implementations of larger neural networks.
In this work, we revisit the choice of restricting influence computation to the last layer in the context of large-scale Natural Language Processing (NLP) models.
We first introduce the phenomenon of “cancellation effect” of training data influence, which happens when the sum of the influence magnitude among different training examples is much larger than the actual influence sum.
This effect causes most training examples to have a higher magnitude of influence score, and reduces the discriminative power of data influence.
We also observe that different weight parameters may have different level of cancellation effects, and the weight parameters of bias parameters and latter layers may have larger cancellation effects.
To mitigate the “cancellation effect” and find a scalable algorithm, we propose to operate data influence on weight parameters with the least cancellation effect – the first layer of weight parameter, which is also known as the word embedding layer.
While word embedding representations might have the issue of not capturing any high-level input semantics, we surprisingly find that the gradients of the embedding weights do not suffer from this.
Since the gradient chain through the higher layers, it thus takes the high-level information captured in those layers into account.
As a result, the gradients of the embedding weights of a word depend on both the context and importance of the word in the input.
We develop the idea of word embedding based influence in the context of TracIn due to its computational and resource efficiency over other methods.
Our proposed method, TracIn-WE, can be expressed as the sum of word embedding gradient similarity over overlapping words between the training and test examples.
Requiring overlapping words between the training and test sentences helps capture low-level similarity, while the word gradient similarity helps capture the high-level semantic similarity between the sentences.
A key benefit of TracIn-WE is that it affords a natural word-level decomposition, which is not readily offered by existing methods.
This helps us understand which words in the training example drive its influence on the test example.
We evaluate TracIn-WE on several NLP classification tasks, including toxicity, AGnews, and MNLI language inference with transformer models fine-tuned on the task.
We show that TracIn-WE outperforms existing influence methods on the case deletion evaluation metric by 4 − 10 × .
A potential criticism of TracIn-WE is its reliance on word overlap between the training and test examples, which would prevent it from estimating influence between examples that relate semantically but not syntactically.
To address this, we show that the presence of common tokens in the input, such as a “start” and “end” token (which are commonly found in modern NLP models), allows TracIn-WE to capture influence between semantically related examples without any overlapping words, and outperform last layer based influence methods on a restricted set of training examples that barely overlaps with the test example.
Preliminaries
Consider the standard supervised learning setting, with inputs x ∈ X , outputs y ∈ Y , and training data
D = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , ... ( x n , y n ) } .
Suppose we train a predictor f with parameter Θ by minimizing some given loss function ℓ over the training data, so that Θ = arg min Θ (cid:80) ni =1 ℓ ( f ( x i ) , y i ) .
In the context of the trained model f , and the training data D , we are interested in the data importance of a training point x to the testing point x ′ , which we generally denote as I ( x, x ′ ) .
2.1 Existing Methods
We first briefly introduce the commonly used training data influence methods: Influence functions (Koh & Liang, 2017), Representer Point selection (Yeh et al., 2018), and TracIn
(Pruthi et al., 2020).
We demonstrate that each method can be decomposed into a similarity term S ( x, x ′ ) , which measures the similarity between a training point x and the test point x ′ , and loss saliency terms L ( x ) and L ( x ′ ) , that measures the saliency of the model outputs to the model loss.
The decomposition largely derives from an application of chain rule to the parameter gradients.
[Equation]
The decomposition yields the following interpretation.
A training data x has a larger influence on a test point x ′ if (a) the training point model outputs have high loss saliency, (b) the training point x and the test point x ′ are similar as construed by the model.
In Section 3.3,we show that restricting the influence method to operate on the weights in the last layer of the model critically affects the similarity term, and in turn the quality of influence.
We now introduce the form of each method, and the corresponding similarity and loss saliency terms.
Influence Functions:
[Equation]
where H Θ is the hessian (cid:80) ni =1 ∇ 2Θ ℓ ( x, Θ) computed over the training examples.
By an application of the chain rule, we can see that Inf ( x, x ′ ) =
L ( x ) S ( x, x ′ )
L ( x ′ ) , with the similarity term S ( x, x ′ ) =
[Equation]
x, Θ
[Equation]
x
[Equation] [Equation]
, and the loss saliency terms L ( x ) =
[Equation]
( x,
[Equation]
f ( x, Θ) .
The work by Sui et al.
(2021) is very similar to extending the influence function to the last layer to satisfy the representer theorem.
Representer Points:
[Equation]
where a ( x, Θ) is the final activation layer for the data point x , λ is the strength of the ℓ 2 regularizer used to optimize Θ , and j is the targeted class to explain.
The similarity term is S ( x, x ′ )
= a ( x, Θ) T a ( x ′ , Θ) , and the loss saliency terms are L ( x )
= 12 λn ∂ℓ ( x, Θ) ∂
f j ( x, Θ) , L ( x ′ )
= 1 .
TracIn:
[Equation]
where Θ c is the weight at checkpoint c , and η c is the learning rate at checkpoint c .
In the remainder of the work, in our notation, we suppress the sum over checkpoints of TracIn for notational simplicity.
(This is not to undermine the importance of summing over past checkpoints, which is a crucial component in the working on TracIn.)
For TracIn, the similarity term is S ( x, x ′ ) =
∇ Θ f ( x, Θ) T ∇ Θ f ( x ′ , Θ) , while the loss terms are L ( x ) =
∂ℓ ( x, Θ) ∂
f ( x, Θ) , L ( x ′ )
=
∂ℓ ( x ′ , Θ) ∂ f
( x ′ , Θ) .
2.2 Evaluation: Case Deletion
We now discuss our primary evaluation metric, called case deletion diagnostics (Cook & Weisberg, 1982), which involves retraining the model after removing influential training examples and measuring the impact on the model.
This evaluation metric helps validate the efficacy of any data influence method in detecting training examples to remove or modify for targeted fixing of misclassifications, which is the primary application we consider in this work.
This evaluation metric was also noted as a key motivation for influence functions (Koh & Liang, 2017).
Given a test example x ′ , when we remove training examples with positive influence on x ′ ( proponents ), we expect the prediction value for the ground-truth class of x ′ to decrease.
On the other hand, when we remove training examples with negative influence on x ′ ( opponents ), we expect the prediction value for the ground-truth class of x ′ to increase.
An alternative evaluation metric is based on detecting mislabeled examples via self-influence (i.e. influence of a training sample on that same sample as a test point).
We prefer the case deletion evaluation metric, as it more directly corresponds to the concept of data influence.
Similar evaluations that measure the change of predictions of the model after a group of points is removed is seen in previous works.
Han et al.
(2020) measures the test point prediction change after 10% training data with the most and least influence are removed, and Koh et al.
(2019) measures the correlation of the model loss change after a group of trained data is removed and the sum of influences of samples in the group, where the group can be seen as manually defined clusters of data.
Deletion curve.
Given a test example x ′ and influence measure I , we define the metrics
DEL + ( x ′ , k, I ) and DEL − ( x ′ , k, I ) as the impact on the prediction of x ′ (for its groundtruth class) upon removing top- k proponents and opponents of x ′ respectively:
[Equation]
where, Θ +; k ( Θ − ; k ) are the model weights learned when top- k proponents (opponents) according to influence measure I are removed from the training set, and c is the groundtruth class of x ′ .
The expectation is over the number of retraining runs.
We expect DEL + to have large negative, and
DEL − to have large positive values.
To evaluate the deletion metric at different values of k , we may plot DEL + ( x ′ , k, I ) and DEL − ( x ′ , k, I ) for different values of k , and report the area under the curve (AUC): AUC - DEL + = (cid:80)
k m k
= k 1 1 m DEL + ( x ′ , k, I ) , and AUC - DEL − = (cid:80) k
m k
= k 1 1 m DEL − ( x ′ , k, I ) .
We note that the case deletion diagnostics is different to the leave-one-out evaluation of Koh & Liang (2017) by two points.
First, leave-one-out evaluation focuses on removing one point, which is more meaningful in the convex regime where the optimization is initialization-invariant.
We consider the leave-k-out evaluation which is closer to actual applications, as one may need to alter more than one training data to fix a prediction.
Second, we consider the expected value of leave-k-out, to hedge the variance caused by specific model states, which was pointed out by Søgaard et al.
(2021) to be a major issue for leave-one-out evaluation (especially when the objective is no longer convex).
Cancellation Effect of Data Influence
The goal of a data influence method is to distribute the test data loss (prediction) across training examples, which can be seen as an attribution problem where each training example is an agent.
We observe cancellation across the data influence attributions to training examples, i.e., the sign of attributions across training examples disagree and cancels each other out.
This leads to most training examples having a large attribution magnitude, which reduces the discriminatory power of attribution-based explanations.
Our next observation is that the cancellation effect varies across different weight parameters.
In particular, when a weight parameter is used by most of the training examples, the cancellation effect is especially severe.
One such parameter is the bias, whose cancellation effect is illustrated by the following example:
Example 3.1.
Consider an example where the input x ∈ R d is sparse, and x i has feature i with value 1 and all other features with value 0.
The prediction function has the form f ( x ) = x · w + b .
It follows that a set of optimal parameters are w i =
y i , b = 0 .
We further assume that the parameter b is initialized to 0 and has never changed during the gradient descent progress.
In this case, it is clear that the bias parameter b is irrelevant to the model (as removing it will not change the model at all).
However, the influence to individual examples caused by the bias b is still non-zero.
This is because even that the sum of gradient for bias b is 0 , (s.t. (cid:80) i ∂L
( f ( x i ) ,y i ) ∂b = 0 ), each individual term ∂L ( f ( x i ) ,y i ) ∂b
= ∂L
( f ( x i ) ,y i )
∂
f ( x i ) is non-zero for most x i .
Note that ∂L ( f ( x i ) ,y i ) ∂b contributes to the influence to data x
i directly, and thus the bias parameter b will contribute to the influence of all the training data.
In the contrary, for each weight variable w j , ∂L ( f ( x i ) ,y i )
∂w j is only non-zero for x j , and thus the weight variable w j only contributes to the influence of one training data x j .
Thus, the bias would affect the influence for more training examples compared to the weights.
The above example illustrates that while the bias parameter is not important for the prediction model (removing the bias can still lead to the same optimal solution), the total gradient that flows through the bias still high.
In fact, we find empirically that the total influence that flows through the bias is larger than that flowing through the weight, since each training example’s gradient will affect the bias but the total contribution will be cancelled out, so the bias will remain 0 .
We also note that even for deep network models that do not have a sparse input, the neurons connected to the weight are often(due to ReLU types of activation functions).
Thus, the gradient of weight parameters is often sparser compared to the gradient of bias parameters, and thus bias parameters would often have stronger cancellations, which we validate empirically.
3.1 Measuring the Cancellation Effect
In the above example, we defined strong cancellation effect when some weight parameters does not change a lot during training (or has saturated in the training process), but the total strength of the gradient of the weight parameters summed over training data is large.
For weights W , we first define two terms ∆ W c and G ( W ) c ,
[Equation]
where ∆ W c measures the norm of weight parameter change between checkpoint c and c + 1 , and
G ( W ) c measures the sum of weight gradient norm times learning rate summed over all training data.
When ∆ W c is small, this means that the weight W may have saturated at checkpoint c , and the weight may not actually affect the model output much (and thus the weight W is not important for this epoch of training).
When G ( W ) c is large, this means that the sum of gradient norm with respect to W c is still large, and the influence norm caused by ∂l ( x i ,y i )
∂W c will also be large.
To measure the cancellation effect, we define the cancellation ratio of a weight parameter W as:
[Equation]
When G ( W ) c is large and ∆ W c is small, this means that a non-important weight W c greatly influenced the total influence norm, which may be only possible if the influence contributed from W c to different examples cancelled each other out.
Applying this interpretation to the cancellation of bias parameters, the intuition is that the bias parameters are not mainly responsible for the reduction of testing example loss change during the training process (since ∆ W c is small).
However, they dominate the total influence strength due to their dense nature ( G ( W ) c is large).
Parameters with high cancellation may not be ideal to the calculation of influences.
3.2 Removing Bias In TracIn Calculation to Reduce Cancellation Effect
To investigate whether removing weights with high cancellation effect really helps improve influence quality, we conducted an experiment on a CNN text classification on Agnews dataset with 87% test accuracy.
The model is defined as follows: first a token embedding with dimension 128 , followed by two convolution layers with kernel size 5 and filter size 10 , one convolution layers with kernel sizeand filter size 10 , a global max pooling layer, and a fully connected layer; all weights are randomly initialized.
The first layer is the token embedding, the second layer is the convolution layer, and the last layer is a fully connected layer.
The model has 21222 parameters in total (excluding the token embedding), in which 102 parameters are bias variables.
We find C ( bias ) to be 16789 , and C ( weight ) to be 2555 , which validates that the bias variables have a much stronger cancellation effect than the weight variables.
A closer analysis shows that G ( bias ) is similar to G ( weight ) ( 627206 and 559142 ), but ∆( bias ) is much smaller than ∆( weight ) ( 0 .
74 and 4 . 37 .)
Even though the bias parameters has a much smaller total change compared to the weight parameters, their impact on the gradient norm (and thus influence norm) is even higher than the weight parameters.
This verifies the intuition in Example 3.1 that the bias parameter has a stronger cancellation effect since the gradient to bias is almost activated for all examples despite the actual bias change being small.
To further verify that the TracIn score contributed by the bias may lower the overall discriminatory power, we compute AUC - DEL + and AUC - DEL − for TracIn and TracIn-weight on AGnews with our CNN model.
The AUC - DEL + for TracIn and TracIn-weight is − 0 .
036 and − 0 .
065 respectively, and the AUC - DEL + for TracIn and
TracIn-weight is 0 .
011 and 0 . 046 .
The result shows that by removing the TracIn score contributed by the bias (with only 102 parameters), the overall influence quality improves significantly.
Thus, in all future experiments, we remove the bias in calculation of data influence if not stated otherwise.
3.3 Influence of Latter Layers May Suffer from Cancellation
As mentioned in Section 1, for scalability reasons, most influence methods choose to operate only on the parameters of the last fully-connected layer Θ last .
We argue that this is not a great choice, as the influence scores that stems from the last fully-connected weight layer may suffer from cancellation effect, as different examples “share logics” in the activation representation of this layer, and have a higher gradient similarity for different examples.
Early layers, where examples have unique logic, may suffer less from the cancellation effect.
We first measure the gradient similarity for different examples for each layer, which is E x a ,x b COS-SIM [ ∂l ( x a ) /∂w, ∂l ( x b ) /∂w ] , where COS-SIM is the cosine similarity.
This measures the expected gradient cosine similarity between two examples.
The expected gradient similarity for testing examples between different layers in the CNN classification are: first 0 .
035 , second 0 .
075 , third 0 .
21 , last 0 .
This verifies that the latter layers in the neural network have more aligned gradients between examples, and thus share more logics between training examples.
We report the cancellation ratio for each of the TracIn layer varaint in Table 1, where
TracIn-first, TracIn-second, TracIn-third, TracIn-last, TracIn-All refer to TracIn scores based on weights of the first layer, second layer, third layer, last layer, and all layers (the bias is always omitted).
As we suspected, early layers suffers less from cancellation, and latter layers suffers more from cancellation.
To assess the impact on influence quality, we evaluate the AUC - DEL + and
AUC - DEL − score for TracIn calculated with different layers on the AGnews CNN model in Tab. 1.
We observe that removing examples based on influence scores calculated using parameters of later layers (with more “shared logic"") leads to worse deletion score compared to removing examples based on influence scores calculated using parameters of earlier layers (with more “unique logic”).
Interestingly, the performance of TracIn-first even outperforms TracIn-all where all parameters are used.
2 We hypothesize that since the TracIn score based on later layers contain too much cancellation,
[Table] [Table]
it is actually harmful to include these weight parameters in the TracIn calculation.
In the following, we develop data influence methods by only using the first layer of the model, which suffers the least from cancellation effect.
Word Embedding Based Influence
In the previous section
[Table]
we argue that using the latter layers to calculate influence may lead to the cancellation effect, which over-estimates influence.
Another option is to calculate influence on all weight parameters, but may be computational infeasible when larger models with several millions of parameters are used.
To remedy this, we propose operating on the first layer of the model, which contains the less cancellation effect since early layers encodes “unique logit”.
The first layer for language classification models is usually the word embedding layer in the case of NLP models.
However, there are two questions in using the first layer to calculate data influence: 1.
the word (token) embedding contains most of the weight parameters, and may be computational expensive 2.
the word embedding layer may not capture influential examples through high-level information.
In the rest of this section, we develop the idea of word embedding layer based training-data influence in the context of TracIn.
We focus on TracIn due to challenges in applying the other methods to the word embedding layer: influence functions on the word embedding layer are computationally infeasible due to the large size (vocab size × embedding_dimension) of the embedding layer, and representer is designed to only use the final layer.
We show that our proposed influence score is scalable thanks to the sparse nature of word embedding gradients, and contains both low-level and high-level information since the gradient to the word embedding layer can capture both high-level and low-level information about the input sentence.
4.1 TracIn on Word Embedding Layer
We now apply TracIn on the word embedding weights, obtaining the following expression:
[Equation]
Implementing the above form of TracIn-WE would be computationally infeasible as word embedding layers are typically very large (vocab size × embedding dimension).
For instance, a BERT-base model has 23 M parameters in the word embedding layer.
To circumvent this, we leverage the sparsity of word embedding gradients ∂ℓ ( x, Θ) ∂
Θ WE , which is a sparse vector where only embedding weights associated with words that occur in x have non-zero value.
Thus, the dot product between two word embedding gradients has non-zero values only for words w that occur in both x, x ′ .
With this observation, we can rewrite TracIn-WE as:
TracIn-WE
[Equation]
where Θ w are the weights of the word embedding for word w .
We call the term ∂ℓ ( x )
∂ Θ w T · ∂ℓ ( x ′ )
∂
Θ w the word gradient similarity between sentences x, x ′ over word w .
[Table]
4.2 Interpreting Word Gradient Similarity
Equation 4 gives the impression that TracIn-WE merely considers a bag-of-words style similarity between the two sentences
[Table]
and does not take the semantics of the sentences into account.
This is surprisingly not true!
Notice that for overlapping words, TracIn-WE considers the similarity between gradients of word embeddings.
Since gradients are back-propagated through all the intermediate layers in the model, they take into account the semantics encoded in the various layers.
This is aligned with the use of word gradient norm
∥ ∂
f ( x ) ∂ Θ w ∥ as a measure of importance of the word w to the prediction f ( x ) (Wallace et al., 2019; Simonyan et al., 2013).
Thus, word gradient similarity would be larger for words that are deemed important to the predictions of the training and test points.
Word gradient similarity is not solely driven by the importance of the word.
Surprisingly, we find that word gradient similarity is also larger for overlapping words that appear in similar contexts in the training and test sentences.
We illustrate this via an example.
Table 2 shows 4 synthetic premise-hypothesis pairs for the Multi-Genre Natural Language Inference (MNLI) task (Williams et al., 2018).
An existing pretrained model (He et al., 2020) predicts these examples correctly with softmax probability between 0 . 65 and 0 .
Notice that all examples contain the word ‘not’ once.
The word gradient importance ∥
∂ f ( x ) ∂ Θ w ∥ for “not” is comparable in all 4 sentences.
The value of word gradient similarity for ‘not’ is 0 .
34 for the pair S1-S2, and − 0 .
12 for S1-S3, while it is − 0 .
05 for
S1-S4.
This large difference stems from the context in which ‘not’ appears.
The absolute similarity value is larger for S1-S2 and S1-S3, since ‘not’ appears in a negation context in these examples.
(The word gradient similarity of S1-S3 is negative since they have different labels.)
However, in S4, ‘not’ appears in the phrase “not only ... but”, which is not a negation (or can be considered as double negation).
Consequently, word gradient similarity for ‘not’ is small between S1 and S4.
In summary, we expect the absolute value of TracIn-WE score to be large for training and test sentences that have overlapping important words in similar (or strongly opposite) contexts.
On the other hand, overlap of unimportant words like stop words would not affect the TracIn-WE score.
4.3 Word-Level Decomposition for TracIn-WE
An attractive property of TracIn-WE is that it decomposes into word-level contributions for both the testing point x ′ and the training point x .
As shown in (4), word w in x contributes to TracIn-WE ( x, x ′ ) by the amount ∂ℓ ( x )
∂ Θ w T · ∂ℓ ( x ′ )
∂
Θ w 1 [ w ∈ x ′ ] ; a similar word-level decomposition can be obtained for x ′ .
Such a decomposition helps us identify which words in the training point ( x ) drive its influence towards the test point ( x ′ ).
For instance, consider the example in Table.
3, which contains two test sentences (T1, T2) and a training sentence S1.
We decompose the score TracIn-WE(S1, T1) and
TracIn-WE(S1,T2) into words contributions, and we see that the word “lazy” dominates TracInWE(S1, T1), and the word “end” dominates TracIn
WE(S1, T2).
This example shows that different key words in a training sentence may drive influence towards different test points.
The featuredecomposition for influence introduces additional interpretability to why two examples are highly influenced.
This is demonstrated in a case study where we cluster difficult training examples based on a normalized TracIn-WE score in Sec.
A.
4.4 An approximation for TracIn-WE
As we note in Sec. 4.1, the space complexity of saving training and test point gradients scales with the number of words in the sentence.
This may be intractable for tasks with very long sentences.
We alleviate this by leveraging the fact that the word embedding gradient for a word w is the sum of input word gradients from each position where w is present.
Given this decomposition, we can approximate the word embedding gradients by saving only the top-k largest input word gradients for each sentence.
(An alternative is to save the input word gradients that are above a certain threshold.)
Formally, we define the approximation ∂ℓ ( x, Θ)
[Equation]
where x i is the word at position i , and x top − k is the set of top-k input positions by gradient norm.
We then propose
[Equation]
Computational complexity Let L be the max length of each sentence, d be the word embedding dimension, and o be the average overlap between two sentences.
If the training and test point gradients are precomputed and saved then the average computation complexity for calculating
TracIn-WE for m training points and n testing points is O ( mnod ) .
This can be contrasted with the average computation complexity for influence functions on the word embedding layer, which takes O ( mnd 2 v 2 + d 3 v 3 ) , where v is the vocabulary size which is typically larger than 10 4 , and o is typically less than 5 .
The approximation for TracIn-We-Topk drops the computational complexity from O ( mnod ) to O ( mno k d ) where o k is the average overlap between the sets of top-k words from the two sentences.
It has the additional benefit of preventing unimportant words (ones with small gradient) from dominating the word similarity by multiple occurrences, as such words may get pruned.
In all our experiments, we set k to 10 for consistency, and do not tune this hyper-parameter.
4.5 Influence without Word-Overlap
One potential criticism of TracIn-WE is that it may not capture any influence when there are no overlapping words between x and x ′ .
To address this, we note that modern NLP models often include a “start” and “end” token in all inputs.
We posit that gradients of the embedding weights of these tokens take into account the semantics of the input (as represented in the higher layers), and enable
TracIn-WE to capture influence between examples that are semantically related but do not have any overlapping words.
We illustrate this in S5-S7 in Tab. 2 via examples for the MNLI task.
Sentence
S5 has no overlapping words with S6 and S7.
However, the word gradient similarity of “start” and “end” tokens for the pair S5-S6 is 1 . 15 , while that for the pair S5-S7 is much lower at − 0 . 05 .
Indeed, sentence S5 is more similar to S6 than S7 due to the presence of similar word pairs (e.g., think and thinks, annoying and boring), and the same negation usage.
We further validate that TracIn-WE can capture influence from examples without word overlap via a controlled experiment in Sec. 5.
Experiments
We evaluate the proposed influence methods on 3 different NLP classification datasets with BERT models.
We choose a transformer-based model as it has shown great success on a series of downstream tasks in NLP, and we choose BERT model as it is one of the most commonly used transformer model.
For the smaller Toxicity and AGnews dataset, we operate on the Bert-Small model, as it already achieves good performance.
For the larger MNLI dataset, we choose the Bert-Base model with 110 M model parameters, which is a decently large model which we believe could represent the effectiveness of our proposed method on large-scale language models.
As discussed in Section 2.2, we use the case deletion evaluation and report the metrics on the deletion curve in Table 4 for various methods and datasets.
The standard deviation for all AUC values all methods is reported in Table 7.
[Table]
Baselines One question to ask is whether the good performance of TracIn-WE is a result that it captures the low-level word information well.
To answer this question, we design a synthetic data influence score as the TF-IDF similarity Salton & Buckley (1988) multiplied by the loss gradient dot product for x and x ′ .
TR-TFIDF can be understood by replacing the embedding similarity of
TracIn-
Last by the TF-IDF similarity, which captures low level similarity.
[Equation]
Toxicity.
We first experiment on the toxicity comment classification dataset (Kaggle.com, 2018), which contains sentences that are labeled toxic or non-toxic.
We randomly choose 50 , 000 training samples and 20 , 000 validation samples.
We then fine-tune a BERT-small model on our training set, which leads to 96% accuracy.
Out of the 20 , 000 validation samples, we randomly choose 20 toxic and 20 non-toxic samples, for a total of 40 samples as our targeted test set.
For each example x ′ in the test set, we remove top- k proponents and top- k opponents in the training set respectively, and retrain the model to obtain DEL + ( x ′ , k, I ) and DEL − ( x ′ , k, I ) for each influence method I .
We vary k over { 10 , 20 , . . .
For each k , we retrain the model 10 times and take the average result, and then average over the 40 test points.
We implement the methods Influence-last, Representer Points,
TracIn-last, TracIn-WE, TracIn-WE-Topk, TracIn-TFIDF (introduced in Sec. G), TracIn-common (which is a variant of TracIn only using the start token and end token to calculate gradient), and abbreviate TracIn with TR in the experiments.
We see that our proposed TracIn-WE method, along with its variants TracIn-WE-Topk outperform other methods by a significant margin.
As mentioned in Sec.
3.3, TF-IDF based method beats the existing data influence methods using last layer weights by a decisive margin as well, but is still much worse compared to TracIn-WE.
Therefore, TracIn-WE did not succeed by solely using low-level information.
Also, we find that TracIn-WE performs much better than TracIn-common, which uses the start and end tokens only.
This shows that the keyword overlaps (such as lazy, end in Table 3) is crucial to the great performance of TracIn-WE.
AGnews.
We next experiment on the AG-news-subset (Gulli, 2015; Zhang et al., 2015), which contains a corpus of news with 4 different classes.
We follow our setting in toxicity and choose 50 , 000 training samples, 20 , 000 validation samples, and fine-tune with the same BERT-small model that achieves 90% accuracy on this dataset.
We randomly choose 100 samples with 25 from each class as our targeted test set.
The AUC - DEL + and AUC - DEL − scores for k ∈ { 10 , 20 , . .
, 100 } are reported in Table 4.
Again, we see that the variants of TracIn-WE significantly outperform other existing methods applied on the last layer.
In both AGnews and Toxicity, removing 10 top-proponents or top-opponents for TracIn-WE has more impact on the test point compared to removingtop-proponents or top-opponents for TracIn-last.
MNLI.
Finally, we test on a larger scale dataset, Multi-Genre Natural Language Inference (MultiNLI)
Williams et al.
(2018), which consists of 433 k sentence pairs with textual entailment information, including entailment, neutral, and contradiction.
In this experiment, we use the full training and validation set, and BERT-base which achieves 84% accuracy on matched-MNLI validation set.
We choose 30 random samples with 10 from each class as our targeted test set.
We only evaluate TracIn-WE-Topk, TracIn-last and TracIn-TFIDF as those were the most efficient methods to run at large scale.
We vary k ∈ { 20 , . .
. , 5000 } , and the AUC - DEL + and AUC - DEL − scores for our test set are reported in Table 4.
Unlike previous datasets, here TracIn-TFIDF does not perform better than TracIn-
Last, which may be because input similarity for MNLI cannot be merely captured by overlapping words.
For instance, a single negation would completely change the label of the sentence.
However, we again see TracIn-WE-Topk significantly outperforms TracIn-Last and TracIn-TFIDF, demonstrating its efficacy over natural language understanding tasks as well.
This again provides evidence that TracIn-WE can capture both low-level information and high-level information.
The deletion curve of Toxicity, AGnews, MNLI is in shown in Fig.
4.5 and Fig.
Toxicity-Roberta.
To additionally test whether our experiment results apply to more modern models, we repeat our experiments on the toxicity dataset with a Roberta model Liu et al.
(2019), while fixing other settings.
We find that the TracIn-WE and TracIn-WE-Topk still significantly outperforms other results.
No Word Overlap.
To assess whether TracIn-WE can do well in settings where the training and test examples do not have overlapping words, we construct a controlled experiment on the Toxicity dataset.
We follow all experimental setting for Toxicity classification with the Bert model, but making two additional changes – (1) given a test sentence x ′ , we only consider the top- 5000 training sentences (out of 50 , 000 ) with the least word overlap for computing influence.
We use TF-IDF similarity to rank the number of word overlaps so that stop word overlap will not be over-weighted.
(2) We also fix the token embedding during training (result when word-embedding is not fixed is in the appendix, where removing examples based on any influence method does not change the prediction), as we find sentence with no word overlaps carry more influence when the token embedding is fixed.
The AUC - DEL + and AUC - DEL − scores are reported in the lower section of Table 4.
We find that
TracIn-WE variants can outperform last-layer based influence methods even in this controlled setting, showing that TracIn-WE can retrieve influential examples even without non-trivial word overlaps.
In
Section 4.5, we claimed that this gain stems from the presence of common tokens (“start”, “end”, and other frequent words).
To validate this, we compared with a controlled variant, TracIn-WENoCommon (TR-WE-NoC) where the common tokens are removed from TracIn-WE.
As expected, this variant performed much worse on the AUC - DEL + and AUC - DEL − scores, thus confirming our claim.
We also find that the result of TracIn-WE is better than TracIn-common (which is TracIn-WE with only “start” and “end” tokens), which shows that the common tokens such as stop words and punctuation may also help finding influential examples without meaningful word overlaps.
Related Work
In the field of explainable machine learning, our works belongs to training data importance (Koh &
Liang, 2017; Yeh et al., 2018; Jia et al., 2019; Pruthi et al., 2020; Khanna et al., 2018; Sui et al., 2021).
Other forms of explanations include feature importance feature-based explanations, gradient-based explanations (Baehrens et al., 2010;
Simonyan et al., 2013; Zeiler & Fergus, 2014; Bach et al., 2015;
Ancona et al., 2018; Sundararajan et al., 2017; Shrikumar et al., 2017; Ribeiro et al., 2016; Lundberg & Lee, 2017; Yeh et al., 2019; Petsiuk et al., 2018) and perturbation-based explanations (Ribeiro et al., 2016; Lundberg & Lee, 2017; Yeh et al., 2019; Petsiuk et al., 2018), self-explaining models (Wang &
Rudin, 2015; Lee et al., 2019; Chen et al., 2019), counterfactuals to change the outcome of the model (Wachter et al., 2017; Dhurandhar et al., 2018; Hendricks et al., 2018; van der Waa et al., 2018; Goyal et al., 2019), concepts of the model (Kim et al., 2018;
Zhou et al., 2018).
For applications on applying data importance methods on NLP tasks, there have been works identifying data artifacts (Han et al., 2020; Pezeshkpour et al., 2021) and improving models (Han & Tsvetkov, 2020, 2021) based on existing data importance method using the influence function or TracIn.
In this work, we discussed weight parameter selection to reduce cancellation effect for training data attribution.
There has been works that discuss how to cope with cancellation in the context of feature attribution:
Liu et al.
(2020) discusses how regularization during training reduces cancellation of feature attribution, Kapishnikov et al.
(2021) discusses how to optimize IG paths to minimize cancellation of IG attribution, and
Sundararajan et al.
(2019) discusses improved visualizations to adjust for cancellation.
Conclusion
In this work, we revisit the common practice of computing training data influence using only last layer parameters.
We show that last layer representations in language classification models can suffer from the cancellation effect, which in turn leads to inferior results on influence.
We instead recommend computing influence on the word embedding parameters, and apply this idea to propose a variant of
TracIn called TracIn-WE.
We show that TracIn-WE significantly outperforms last versions of existing influence methods on three different language classification tasks for several models, and also affords a word-level decomposition of influence that aids interpretability.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] In Sec.
B (c) Did you discuss any potential negative societal impacts of your work?
[Yes] In Sec.
C (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] In Sec.
D 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes]
In Sec.
E (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] Only used public data.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Finding Mislabeling Patterns by Clustering
We first select training data that are classified incorrectly at least 40% during training (with early stopping) after training 20 models in the AGnews dataset, which are more likely to consist of mislabeling examples.
Our goal is to find mislabeled training examples that may consist similar mislabeling patterns.
We hypothesize that two mislabeled training examples that have high influence to each other may be more likely to have a similar mislabeling patterns.
Thus, we define the influence distance between training data as the negative of a scaled data influence.
[Equation]
Thus, if data B is a strong proponent to data A
[Table]
the influence distance would be small.
We then cluster these “difficult to learn” examples by the influence distance
[Table]
and we apply this clustering on AGnews, where the influence distance is calculated by TracIn-WE on a CNN model
[Table]
and use Agglomerative
Clustering with threshold 0 .
8 , which results in 31 clusters with at least 3 elements.
[Table] [Table]
– BOSTON - The New York Yankees and Boston were tied 4-4 after 13 innings Monday night with the Red Sox trying to stay alive in the AL championship series.
– Steady rain Friday night forced major league baseball to postpone Game 3 of the AL championship series between the Boston Red Sox and New York Yankees.
– After Curt Schilling and Pedro Martinez failed to get the Boston Red Sox a win against the New York Yankees in the first two games of the AL championship series – The Boston Red Sox entered this AL championship series hoping to finally overcome their bitter rivals from New York following a heartbreaking seven-game defeat last October.
Cluster
[Table]
– com October 13, 2004, 5:06 PM PT.
This fourth priority‘s main focus has been enterprise directories as organizations spawn projects around identity infrastructure.
– com September 15, 2004, 11:03 AM PT.
This fourth priority’s main focus has been improving or obtaining CRM and ERP software for the past year and a half.
– com October 11, 2004, 11:16 AM PT.
This fourth priority’s main focus has been enterprise directories as organizations spawn projects around identity infrastructure.
Cluster– This fourth priority’s main focus has been enterprise directories as organizations spawn projects around identity infrastructure.
– com September 13, 2004, 8:58 AM PT.
This fourth priority’s main focus has been improving or obtaining CRM and ERP software for the past year and a half.
– com October 26, 2004, 7:41 AM PT.
This fourth priority’s main focus has been enterprise directories as organizations spawn projects around identity infrastructure.
Cluster– The Wall Street rumor mill is working overtime, spinning off speculation about how soon a decision will be announced in the US government lawsuit aimed at blocking Oracle’s proposed takeover of PeopleSoft.–OracleCorp. has extended its $7.7 billion hostile takeover bid for Pleasanton’ PeopleSoft Inc. until Sept. 10.
Redwood City-based Oracle’s previous offer would have expired at 9 pm Friday.
– A director of the Oracle Corporation testified that the company’s $7.7 billion hostile bid for PeopleSoft might not be the final offer.
Cluster
[Table]
– BLOOMFIELD TOWNSHIP, Mich. -
For the first time in three days at the Ryder Cup, there was plenty of red on the scoreboard - as in American red, white and blue... – Europe go into the singles needing three-and-a-half points to win the Ryder Cup.
– BLOOMFIELD TOWNSHIP, Mich. - Staring down Tiger Woods, Phil Mickelson and the rest of the Americans, Europe got off to a stunning start Friday in the Ryder Cup...
We show 5 clusters where we find that the examples in the clusters are clear mispredictions, and report the description of clusters in Tab. 5 and the actual sentences (sometimes abbreviated) in Tab.
We report the most common words in clusters by recording the top- 5 words that contributed to the TracIn-WE score for each pair of sentences, and report the words that are top- 5 in most pair of sentences in the clusters.
We note that while cluster 2 are not necessary mislabels, the combination with cluster 3 shows some repetition and inconsistency issues of the labeling process of Agnews.
Other clusters clearly demonstrate very clear mislabel patterns in AGnews, which can be fixed systemically by humans writing a fixing function, which can be an interesting follow-up direction.
B Limitations of Our Work
While we believe that our claim that “first is better than last for training data influence” is general, we did not test out the method on all data modalities and all types of models, as the computation of deletion score is very expensive.
We note that we have not tested TracIn-
Last for generative tasks, as it is beyond the scope of the paper, and we leave it to future works.
C Potential Social Impact of Our Work
One potential social impact is that one may use the algorithm to adjust training data to effect a particular test point’s prediction.
This can be used for good (making the model more fair), or for bad (making the model more biased).
D Computation
We report the run time for TracIn-WE, TracIn-Sec, TracIn-Last, Inf-Sec, Inf-Last for the CNN text model.
The second convolutional layer has 6400 parameters, last layer has 440 parameters, token embedding layer has 3 . 2 million parameters.
We applied these methods on 50000 training points and 10 test points.
The preprocessing time (sec) per training point is 0 .
003 , 3 . 52 , 0 . 002 , and the cost of computing influence per training point and test point pair (sec) is: 4 · 10 − 4 , 3 · 10 − 5 , 8 · 10 − 6 , 10 − 1 , 2 · 10 − 5 .
Influence function on the second layer is already order of magnitudes slower than other variants, and cannot scale to the word embedding layer with millions of parameters.
For remove and retrain on Toxicity and AGnews, we run our experiments on multiple V100 clusters.
For remove and retrain on MNLI, we run our experiments on multiple TPU-v3 clusters.
For toxicity and AGnews experiment, we need to fine-tune the language model on the classification task for 40 × 6 × 10 × 2 × 10 times, where the fine-tuning takes around 10 − 20 GPU-minute on a V for Bert-Small, and 40 , 6 , 10 , 2 , 10 stands for number of test points, number of methods, removal numbers, proponents/ opponents, and repetition numbers respectively.
On MNLI, we fine-tuned the language model for 19800(30 × 3 × 11 × 2 × 10) times, where fine-tuning MNLI on BERT-Base takes around 320 TPU-minute on a TPU-v3 cluster for Bert-base.
E Licence of Datatset
Toxicity dataset has license cc0-1.0, AGnews dataset has license non-commercial use, and MNLI has license cc-by-3.0.
F Standard Deviation of Experiments
We report the standard deviation of all AUC-DEL value reported in Tab.4.
To calculate each AUCDEL score, we take the average after retraining 10 times.
We could then measure the standard deviation by bootstrapping.
We report the number in Tab. 7, and we see that all methods have similar standard deviations.
[Table]
G A different viewpoint on Issues with Last Layer.
We present our analysis in the context of the TracIn method applied to the last layer, referred to as TracIn-
Last, although our experiments in Section 5 suggest that Influence-Last and RepresenterLast may also suffer from similar shortcomings.
For TracIn-
Last, the similarity term S ( x, x ′ )
= ∇ Θ f ( x, Θ last ) T ∇ Θ f ( x ′ , Θ last ) becomes a ( x, Θ last ) T a ( x ′ , Θ last ) where a ( x, Θ last ) is the final activation layer.
We refer to it as last layer similarity .
Overall, TracIn-last has the following formultation:
[Equation]
We begin by qualitatively analyzing the influential examples from TracIn-Last, and find the top proponents to be unrelated to the test example.
We also observe that the top proponents of different test examples coincide a lot; see appendix G for details.
This leads us to suspect that the top influence scores from TracIn-Last are dominated by the loss salience term of the training point x (which is independent of x ′ ), and not as much by the similarity term, which is also observed by Barshan et al.
; Hanawa et al.
Indeed, we find that on the toxicity dataset, the top-100 examples ranked by TracIn-
Last and the top-100 examples ranked by the loss salience term ∂ℓ ( x, Θ) ∂
f ( x, Θ) haveoverlaps on average, while the top-100 examples by TracIn-
Last and the top-100 examples ranked by the similarity term a ( x, Θ) T a ( x ′ , Θ) have only 22 overlaps on average.
Finally, we find that replacing the last-layer similarity component by the well-known TF-IDF significantly improves its performance on the case deletion evaluation.
In fact, this new method, which we call TracIn-TDIDF, also outperforms Influence-Last, and
Representer-Last on the case deletion evaluation; see Sectionand Appendix G.
We end this section with the following hypothesis.
Hypothesis G.1.
TracIn-Last and other influence methods that rely on last layer similarity fail in finding influential examples since last layer representations are too reductive and do not offer a meaningful notion of sentence similarity that is essential for influence.
We begin by qualitatively examining the influential examples obtained from TracIn-Last.
Consider the test sentence and its top-2 proponents and opponents in Table 8.
As expected, the proponents have the same label as the test sentence.
However, besides this label agreement, it is not clear in what sense the proponents are similar to the test sentence.
We also observe that out of 40 randomly chosen test examples, proponent-1 is either in the top-20 proponents or top-20 opponents for 39 test points.
To further validate that the inferior results from TracIn-
Last can be attributed to the use of last layer similarity, we perform a controlled experiment where we replace the similarity term by a common sentence similarity measure — the TF-IDF similarity Salton & Buckley (1988).
[Equation]
We find that TFIDF performs much better than TracInCP-last and Influence-Last on the Del+ and
Del- curve (see Fig.
This shows that last layer similarity does not provide a useful measure of sentence similarity for influence.
Since TF-IDF similarity captures sentence similarity in the form of low-level features (i.e., input words), we speculate that last layer representations are too reductive and do not preserve adequate
[Table]
[Table]
low-level information about the input, which is useful for data influence.
This is aligned with existing findings that last layer similarity in Bert models does not offer a meaningful notion of sentence similarity Li et al.
(2020), even performing worse than GLoVe embedding.
H A Relaxation to Synonym Matching
While common tokens like “start” and “end” allow TracIn-WE to implicitly capture influence between sentences without word-overlap, the influence cannot be naturally decomposed over words in the two sentences.
This hurts interpretability.
To remedy this, we propose a relaxation of TracIn-WE, called
TracIn-WE-Syn, which allows for synonyms in two sentences to directly affect the influence score.
In what follows, we define synonyms to be words with similar embeddings.
We first rewrite word gradient similarity as
[Equation]
TracIn-WE can then be represented in the following form: TracIn-WE
[Equation]
x
[Equation]
x
[Equation]
which can be seen as the sum of word gradient similarities for matching words in the two sentences.
It is then natural to consider the variant where exact match is relaxed to synonym match:
[Equation]
where Syn( w, w ′ )
= 1 if the cosine similarity of the embeddings of w and w ′ is above a threshold.
We set the threshold to be 0 .
7 in our experiments.
However, this direct relaxation has the caveat that a word w in x may be matched to several synonyms (including itself) in x ′ simultaneously, which is not in the spirit of TracIn-WE where each word should only be matched to at most one word.
To resolve this, we seek an optimal 1:1 match between words between the two sentence that respects synonymy and maximizes influence.
We formulate this in terms of the Monge assignment problem (Peyré et al., 2019) from optimal transport.
For scalability reasons, we operate on the top- k relaxation of TracInWE (Section 4.4).
Let { w 1 , w 2 , ...w k } and { w ′ 1 , w ′ 2 , ...w ′ k } be the top- k words contained in x and x ′ respectively.
Our goal is to find the optimal assignment function m ∈ M : { 1 , ..., k } → { 1 , ..., k } , such that m ( i ) ̸
= m ( j ) for i ̸ =
j where
[Equation]
[Table]
We define the matching cost between w and w ′ to be the negative absolute value of the word gradient similarity, as this allows us to match synonyms with strong positive as well as strong negative influence.
Optimal assignment can be calculated efficiently by existing solvers, for instance, linear_sum_assignment function in SKlearn (Pedregosa et al., 2011).
The final total influence can be obtained by
TracIn-WE-Syn ( x, x ) = − (cid:88)
[Equation]
We report the result for this relaxation in the following table 10, the result for TR-WE-Syn is close to the result of TracIn-WE, hinting that the additional synonym matching is not particular helpful for the deletion evaluation.
I Qualitative Examples
We show qualitative examples of the top-proponents and top-opponents for two random test points on dataset Toxicity (Tab. 11, 12), AGnews (Tab. 13, 14), and MNLI (Tab. 15, 16).
J No word overlap Experiment – More Details
Why Fix Word Embedding: We first start by the conclusion of our observation: many influence methods cannot find training examples that influences a test point without word overlap in the case where word embedding is not fixed.
To support this observation, we show the deletion curve on no word overlap experiment (when word embedding is not fixed during training) in Fig.
4 and the
AUC-DEL score in Tab.
We can see after that removing proponents the Deletion score is actually slightly positive for all methods, and that removing opponents the Deletion score is actually slightly negative for all methods.
This shows that no influence methods is able to find training examples that influence the test point without having word overlaps.
We thus suspect that influence may flow through examples pairs without word overlaps when the embedding is fixed.
The intuition is that if you have two words A and A’, that have the same initial word embedding.
When embedding is not fixed, the embedding of A’ and A may grow apart during training.
However, if the embedding is fixed, the input of A and A’ will always be the same regardless of whether if the training is applied on A and A’.
Based on this intuition, we fix the word embedding during the model training for the no word overlap experiment.
We now show the deletion curve for our experiment on no word overlap (when word embedding is fixed during training) in Fig.
3 (which is omitted from main text due to space constraint).
We observe that although the signal is weak, most methods other than TR-WE-Noc is consistently positive when opponents are removed, and consistently negative when proponents are removed.
As our result of AUC-DEL suggests, TR-WE variants perform the best in this case.
K Other Experiment Details
For Toxicity and AGnews, we use the small-Bert model 3 as our base model and fine-tune on our validation set.
For Toxicity Roberta, we use the standard Roberta-Base 4 .
For MNLI, we use normal Bert models 5 and fine-tune on the validation set.
For checkpoint selection, we follow suggestions in Pruthi et al.
and choose 3 − 5 checkpoints where the loss has not saturated yet.
We follow standard fine-tuning procedures using SGD optimizers with momentum 0 .
9 , and we fine-tune for 10 epochs on AGnews with 2 e − 2 learning rate and fine-tune for 20 epochs on
Toxicity with 2 e − 4 learning rate.
The retraining parameters is fixed during the calculation of deletion curve.
We split the training and validation set randomly ( 50000 training and 20000 validation and 20000 testing) and fix the random seed.
For MNLI, we calculate deletion curve for k ∈ [20 , 40 , 60 , 80 , 100 , 200 , 400 , 600 , 800 , 1000 , 5000] , and we can see from Fig.
1 that removing 60 examples based on TracIn-WE-Syn affects the test point more than removing 5000 examples based on TracIn-Last.
We also clarify that in the context of our work, we refer to the tokens and words interchangeably for presentation simplicity.
In our work, we use the tokenizer that is used along with Bert or Roberta, which contains mostly words but also some word piece.
When using a character-based tokenizer, the usage of “word” would then become characters.
L Targeted Fixing of Misclassifications
We now discuss an application of our influence method in fixing specific misclassifications made by the model.
We propose two means of fixing (a) remove top-k opponents (b) replace the most negatively influential word in each of the top-k opponents by [PAD].
The most influential word may be identified using the word-level decomposition of TracIn-WE; see Section 4.3.
We consider a
BERT model for the toxicity comment classification task Kaggle.com (2018), and randomly chosemisclassifications from the test set with prediction probability in [0 . 3 , 0 . 7] .
For each misclassification, we apply the two approaches mentioned above for various values of k .
For each k , we report the average percentage of the mistake being fixed in 10 rounds of retraining.
We compare TracIn-WE-Topk with Influence-Last.
To identify the most influential word using
Influence-Last, we consider its gradient w.r.t.
to each word, which is suggested in a similar use case by Pezeshkpour et al.
For fix method (a), removing 50 opponents by TracIn-WE-Topk can fix a mistake 73% of the time, while removing 50 opponents by Influence-last can only fix it 33% of the time.
With both methods, the average accuracy of the model after removing 50 examples only drops by 0 .
For fix method (b), removing the most negatively influential word for the topopponents by TracIn-WE-Topk can fix a mistake 70% of the time, while the same for Influence-Last can only fix a mistake 30% of the time.
With both methods, the average accuracy of the model after removing the most negatively influential word in the top- 200 opponents drops by less than 0 .
We show the full fixing curve in the fixing application in Fig 2, where x-axis is the number of training sentence we remove (either full remove or only remove one top key word).
We show that
TrackIn-WE-topk significantly outperforms Influence-last in the targeted fixing application across different number of removal k .
When remove num k = 0 , we see that the fix probability is 0 . 3 , meaning that after direct retrain without removal, the mistake can actually be correctly classified by the model 30% of the time.
[Table] [Table]
soapbox
[Table]
from that user’s page
[Table]
complying
[Table]
topic
[Table]
not assimilating the teaching of your topic
[Table]
You
[Table]
topic
[Table]
Sandstein
[Table]
I doubt this will get through your thick head (it’s not an insult, it’s an opinion based on your response) but the problem is not the issue itself.
It’s that people like you seem to enjoy (whether or not your side gets it right) to discuss, turn, twist and frankly abuse topics like this which are detrimental to the basic goals of Wikis in general and Wikipedia in particular.
As John Stewart said to two hacks; You’re hurting us.
Toxic
Opponent
[Table]
Pss Rex, be sure to DOCUMENT all the things you’ve discovered on the John Kerry page etc.
It’s awesome that you and I INDEPENDENTLY observed (and can corrorborate) virtually the exactsame pattern by these liberals.
Demonizing conservatives; lionizing liberals.
It’s repeated ad infinitum, ad nauseum.
The more proof we have, the easier it will be to persuade all but their fellow brain- dead truth haters to give a red cent to Wikipedia.
And, until WHOLESALE changes are made from the top down, that’s exactly what’s about to happen.
It’s almost like this is the liberal’s religion.
Too bad they’re gonna have to find a church other than Wikipedia to practice their faith, huh?
I’ve heard rumors that my actions are already sending users Hippocrite, Fred Bauder, WoohooKitty,
Kizzle, FVW, Derex and especially the pimply faced 15 year old RedWolf to become so verklempt they don’t know whether to schedule an appointement with their psychiatrist...or their gynecologist.
Big Daddy- PHASE II Dry up the funding (on the road)
Toxic
[Table]
Dear Mr.RHaworth We have relised that your wikipedia page has come under a number of violent attacks We are happy with this as you are an extreme nob
Wikipedia advises that you imediantly kill yourself.
Thank you - Wikipedia Staff
Non
[Table]
Toxic
ProponentPontiac Monrana Karrmann you full of shit ibelive all of the people who know that the montan will return after 2008 and we want ot knwo ehre do you get your info form and can you sohw it and guess what you dont know anythng about the Montana
Non- Toxic.
Proponent
[Table]
You mean my past BLOCK.
The third block was because of your incompetence.
Jesus doesn’t like liars.
[Table]
Opponent
[Table]
" You are by far the most unhelpful, ungracious administrator I have ever had to deal with.
You’re incompetence is displayed in every encounter we have.
Oh, and I’m quite familar with WP:NPA, which you resort to citing whenever you helpful
Toxic
[Table] [Table]
I find Sandstein’s dealing with the Mbz1 phenomenon very pro- fessional.
He removed the soapbox image from that user’s page and also banned you for not complying with your topic ban.
It is you the one who is not assimilating the teaching of your topic ban.
For example.
You are topic banned because you don’t have a professional approach to I-P topic and in general to any topic related to Jews and Judaism.
The most resent example.
When you reported that soapbox you qualified it as antisemitic.
You at least should get informed of what that is.
A neutral approach would be to have called it as soapbox canvasing and that’s it.
You should focus in your pictures which is the thing that you manage to do relatively well.
Once you get into your holly war program of fighting all that in your imagination is an attack to Judaism you simply behave stupidly.
It is those kinds of behaviors the ones that keep bringing hatred to us.
That kind of attitude is, know it, racist, and if you are true to the struggles of the people of Abraham
you above all should regret behaving as a racist.
Once more, focus on your pictures and maybe even Sandstein will take a like on you.
[Table]
of
[Table]
in
[Table]
All the sources
[Table]
are
[Table]
French national press
[Table]
Hey Asspro I left a disparaging remark about you on the helpdesk.
Enjoy.
[Table]
Road Good aRticles are a pain in the ass.
I’ve asked some one involved with the project what needs to be done with the article to take it to GA.
The history needs to be there, especially the early history which is missing.
It also needs more information on the road features, such as intersections, road width, road markings, etc.
The few sources I’ve looked at haven’t included this, and the roads people tend to use Google Maps as a source
[Table]
CALTLAS IF YOU CONTINUE TO RVV IT, I WILL BLANK YOUR TALK PAGE, I WILL BLOCK YOU, I WILL BAN YOU,
AND I WILL KILL YOUCALTLAS IF YOU CONTINUE TO RVV IT, I WILL BLANK YOUR TALK PAGE, I WILL BLOCK
YOU, I WILL BAN YOU, AND I WILL KILL YOUCALTLAS IF YOU CONTINUE TO RVV IT, I WILL BLANK YOUR TALK
PAGE, I WILL BLOCK YOU, I WILL BAN YOU, AND I WILL KILL YOUCALTLAS IF YOU CONTINUE TO RVV IT,
I WILL
[Table]
YOUR
[Table]
PAGE
[Table] [Table]
[Table]
ARSENE WENGER is preparing for outright confrontation with the FA over his right to call Ruud van Nistelrooy a cheat.
Arsenal boss Wenger was charged with improper conduct by Soho Square for his comments after
[Table]
O’Neal
[Table]
various
[Table]
hush money to keep quiet about sexual encounters, Kobe Bryant told law enforcement officers in Eagle, Colorado.
[Table]
AP - Democratic Party officials picked U.S. Rep. William Lipinski’s son Tuesday to replace his father on the November ballot, a decision engineered by
Lipinski after he announced his retirement and withdrew from the race four days earlier.
[Table]
NEW YORK - Investors shrugged off rising crude futures Wednesday to capture well-priced shares, sending the Nasdaq composite index up 1.6 percent ahead of Google Inc.’s much-anticipated initial public offering of stock.
In afternoon trading, the Dow Jones industrial average gained 67.10, or 0.7 percent, to 10,039.93...
[Table]
NEW YORK - Investors bid stocks higher Tuesday as oil prices declined and earnings results from a number of companies, including International
Business Machines Corp. and Texas Instruments Inc., topped Wall Street’s expectations...
[Table]
NEW YORK - Investors bid stocks higher Tuesday as oil prices declined and earnings results from a number of companies, including International
Business Machines Corp. and Texas Instruments Inc., topped Wall Street’s expectations...
[Table]
China protests against a US investigation that could lead a to trade war over China’s cotton trouser trade.
[Table]
A new anti-corruption watchdog for Bangladesh has been welcomed by global anti-graft campaigners
[Table] [Table]
ATHENS, Aug. 19 – Worried about the potential for a terrorist catastrophe, Greece is spending about $
[Table]
5 billion on security for the Olympic Games.
The biggest threats so far?
Foreign journalists and a Canadian guy dressed in a tutu
[Table]
ATHENS (Reuters)
[Table]
A Canadian man advertising an online gam- ing site, who broke security and jumped into the Olympic diving pool, has been given a five-month prison term for trespassing and disturbing public order, court officials say
[Table]
sensational
[Table]
AFP
[Table]
Britain were
[Table]
minnows Slovakia
[Table]
desperately hoping for an elusive gold
[Table]
later
[Table]
week.
[Table]
NEW YORK - Investors shrugged off rising crude futures Wednes- day to capture well-priced shares, sending the Nasdaq composite index up 1.6 percent ahead of Google Inc.’s much-anticipated ini- tial public offering of stock.
In afternoon trading, the Dow Jones industrial average gained 67.10, or 0.7 percent, to 10,039.93...
World
Proponent
[Table]
NEW YORK - Stocks are seen moving lower at the open Wednes- day as investors come to grips with the Federal Reserve hiking its key rates by a quarter point to 1.75 percent.
Dow Jones futures fell 14 points recently, while Nasdaq futures were down 2.50 points and
S P futures dropped 1.80 points...
[Table]
NEW YORK - Stocks were little changed early Wednesday as investors awaited testimony from Federal Reserve Chairman Alan
Greenspan before a House budget panel.
In morning trading, the Dow Jones industrial average was down 0.08 at 10,342.71...
[Table]
Google Saves Kidnapped Journalist in Iraq Google can claim an- other life saved after a kidnapped Australian journalist was freed by his captors in Iraq earlier today.
Freelance journalist John Mar- tinkus was abducted by gunmen on Saturday outside a hotel near the Australian embassy.
Apparently Martinkus was able to convince his captors ...
Sci/Tech google
[Table]
With a 9:15 p.m. curfew imposed because of Hurricane Jeanne, Tampa Bay beat Toronto with 39 minutes to spare.
Hoping to beat the storm, the Blue Jays were scheduled to leave Florida on a charter flight immediately after the loss.
Today’s series finale was canceled because of the hurricane, which was expected to hit Florida’s east coast late yesterday or ... sport ‘.’
[Table] [Table]
has had a head start as far as diving is concerned because it was never banned here (after all, there are no ancient sites to protect)
[Table]
Diving was banned in places other than Mykonos
[Table]
and they have a job in jail and they work that they should
i and this may sound cruel but i do not think that they should be allowed cigarettes i mean they’re in jail for crying out loud what do they need cigarettes for.
Hypothesis: I think cigarettes should be banned in prison.
[Table]
fill in my name and cash it
[Table]