Influence Scores at Scale for Efficient Language Data Sampling
Authors: Nikhil Anand
Joshua Tan
Maria Minakova
Venue: EMNLP 2023 Main (2023)
Keywords: data effiency
data sampling
difficulty metrics
influence scores
pruning
TL;DR: We can train more data-efficient language models by intelligently choosing a subset of training data. We implement this procedure in public models/datasets as well as in an industry-scale setting and document the results/challenges of doing so.

Abstract: Modern ML systems ingest data aggregated from diverse sources, such as synthetic, human-annotated, and live customer traffic. Understanding \textit{which} examples are important to the performance of a learning algorithm is crucial for efficient model training. Recently, a growing body of literature has given rise to various “influence scores,” which use training artifacts such as model confidence or checkpointed gradients to identify important subsets of data. However, these methods have primarily been developed in computer vision settings, and it remains unclear how well they generalize to language-based tasks using pretrained models.

In this paper, we explore the applicability of influence scores in language classification tasks. We evaluate a diverse subset of these scores on the SNLI dataset by quantifying accuracy changes in response to pruning training data through random and influence-score-based sampling. We then stress-test one of the scores – "variance of gradients" (VoG) from Agarwal and Hooker (2022) – in an NLU model stack that was exposed to dynamic user speech patterns in a voice assistant type of setting. Our experiments demonstrate that in many cases, encoder-based language models can be fine-tuned on roughly 50% of the original data without degradation in performance metrics. Along the way, we summarize lessons learned from applying out-of-the-box implementations of influence scores, quantify the effects of noisy and class-imbalanced data, and offer recommendations on score-based sampling for better accuracy and training efficiency.

Influence Scores at Scale for Efficient Language Data Sampling
Nikhil Anand' and Joshua Tan and Maria Minakova Amazon Alexa Al
Abstract
performance can inform data selection strategies that minimize customer privacy risks associated with the collection of training data, estimate the im- pact of the removal of copyrighted or sensitive data, determine mixing strategies to augment monolin- gual and multilingual datasets to improve accuracy, and identify defective subsets of data. At the same time, in cases where it is desirable to train much data possible such as large language models determining the influence of different data instances (both contextually and during pre- training) can help identify failure modes at the level of specific tokens (Grosse et al., 2023), determine the impact of removal of intellectual property, and significantly reduce costs through more efficient model training (Renduchintala et al., 2023).
Modern ML systems ingest data aggregated from diverse sources, such as synthetic, human- annotated, and live customer traffic. Under- standing which examples are important the performance of learning algorithm is crucial for efficient model training. Recently, a grow- ing body of literature has given rise to vari- ous "influence scores," which use training arti- facts such as model confidence checkpointed gradients to identify important subsets of data. However, these methods have primarily been developed in computer vision settings, and it remains unclear how well they generalize to language-basee based tasks using pretrained models.
this paper, we explore the applicability of influence scores in language classification tasks. We evaluate a diverse subset of these scores on the SNLI dataset by quantifying ac- curacy changes in response to pruning training data through random and influence-score based sampling. We then stress-test one of the scores "variance of gradients" (VoG) from Agarwal Hooker (2022) an NLU model stack that was exposed dynamic user speech pat- terns in a voice assistant type of setting. Our experiments demonstrate that in many cases, encoder-basse based language models can be fine- tuned on roughly 50% of the original data with- out degradation in performance metrics. Along the way, we summarize lessons learned from applying out-of-the-box implementations in- fluence scores, quantify the effects of noisy and class-imbalanced data, and offer recom- mendations on score-based sampling for better accuracy and training efficiency.
A growing body of literature in the science of deep learning aims to capture this hierarchy of example importance and has led to prolifera- tion of number of "difficulty" or "influence" scores (e.g., Paul et al. (2021); Agarwal and Hooker (2022); Toneva et al. (2019); Ethayarajh et al. (2022); Garima et al. (2020); Sorscher et al. (2022); Swayamdipta et al. (2020); see App. A for more complete review). These scores use various train- ing artifacts, such as the margin of confidence the variance of loss gradients, rank the rela- tive contribution of each example to model perfor- mance. This ranking of examples can then be used in many downstream tasks that require intelligent data selection, such as pruning datasets while main- taining or even improving model accuracy (Paul et al., 2021; Sorscher et al., 2022; Marion et al., 2023); identifying outliers and misannotations in labeled data (Garima et al., 2020; Ethayarajh al., 2022; Pleiss al., 2020a; Carlini al., 2019; Feld- man and Zhang, 2020); or reweighting/reordering training examples to increase model robustness (Ren et al., 2018; Wu et al., 2021).
1 Introduction
A salient challenge in training wansformer-based models is selecting which examples are most im- portant for learning. Understanding the relative importance of training examples towards model
Apart from a few notable exceptions (Swayamdipta et al., 2020; Ethayarajh et al.,
*These authors contributed equally, correspondence: nkhlanan@emazon. com, j jshtameamazon.coo

---

2022: Marion et al., 2023), influence scores have primarily been developed and demonstrated in the context of image classífication, and relatively little is known about their efficacy in downstream language-bass
requires careful experimentation tune hyperpa- rameters specific each score. Out of the scores we tested, we find that VoG performs best rela- tive to the random- prrunigg baseline, particularly at large pruning fractions. Test aceuracy is mostly maintained after pruning ~455 of the SNLI train- ing data using VoG scores calculated in a "one-shot" fashion, i.e. from single training run, without any score hyperparameter tuning.
tasks. The application of these scores to data selection is further complicated by the fact that during fine-tuning, modern ML systems often ingest vast amount of data that come from multiple sources, such as synthetic, weak signal, live customer data, and human- annotated. Beyond quantifying the efficacy of influence scores in this highly mixed data setting, there is an operational question of the existence of a simple, scalable influence score that can be easily accommodated in a production workflow.
In a large-scale user study performed using the NLU stack, we find that sampling by VoG scores is effective at surfacing training data that is particu- larly efficient for learning. We prune roughly 50% of training data without incurring statistically sig- nificant regressions in key metrics that track NLU errors, relative to baseline model trained with all data.
In this work, we take first pass at answering these questions. First, we benchmark subset of in- fluence scores on the SNLI dataset (Bowman et al.. 2015) in the downstream task of data reduction us- ing a pretrained BERT model (Devlin et al., 2019). Given the task of pruning a language dataset for fine-tuning, are influence scores useful signals for determining optimal data selection strategies? If so, which scores work best? We evaluate these scores against a random sampling baseline, in both noisy and clean data settings.
2 Experiments on SNLI
2.1 Selection of influence scores
We considered five different influence scores (de- scribed in Table 1) to benchmark in data-reduction tasks on SNLI (Bowman et al., 2015), based on the following criteria: first, they should not require extensive computational resources to implement. For example, the score should not require exten- sive ensemble averaging by training many (* 1) copies of "replicate" models to refine the influence measurement of any particular example since many production models can only be trained once in op- erational workflows. Second, the scores should have a natural definition in language models. This excluded some scores that were originally defined in the context of computer vision, such as input- pixel perturbation (Su et al., 2019). We report the implementation details of these scores in App. B. Our experiments SNLI are run BERTSMALL (L 4. II 512, 29.1M parameters), but we comment on the effects of model size in App. B.3.
User speech patterns are constantly evolving due to current events as well as user-system interactions that can be difficult to anticipate. Are influence scores still effective in surfacing data critical for model performance in this dynamic setting? To an- swer this question, we build upon on our initial find- ings on SNLI and implement one influence score ("variance of gradients" "VoG", first presented in Agarwal et al. (2022)) in generic, large-scale NLU model stack commonly found in commercial voice assistants. We present results for existing in-house test data as well as results for live user study in which we leveraged VoG scores for the pur- pose of substantially reducing training data without incurring model-performme
degradation.
2.2 Experimental Setup
Among the five influence scores evaluated on SNLI, most out-of-the-box implementations do not beat a baseline of randomly pruning the dataset, The implementations can be improved to do better than the random-pruning baseline, but this typically
We ran two sets of data-pruning experiments on SNLI to understand the effectiveness of pruning based on the influence scores in Table 1.
In Section 2.3, we describe data-pruning experi- ments on the original SNLI dataset. First, we gen- erated the influence scores in Table for the entire
'Large language models such as GPT-3 and T5 imple- ment some basic data mixing strategies (Brown al., 2020; Raffel al., 2020). Our focus here, however, the setting of using a pretrained model in downstream task.
We report on the results of scores that require a moderate (1)) number of re-runs such as EL2N (Paul et al., 2021). but our main motivation is to determine if there are influence scores that can he used in "onc-shot" setting, using only training artifacts generated from a single run.
FFor example, data not associated with customer interrup- tions in online traffic, which are then pscudo-annotated with
lahels according to the top model hypothesis.

---

Score
Description
2.3 Results on SNLI
VoG (Agarwal and Hooker, 2022)
Variance of gradients of model outputs with respect to the inputs.
Fig. shows test accuracy at the end of training as a function of percent data pruned for each of the five score-based pruning strategies.
EL2N (Paul al., 2021)
Norm the margin of confidence be- tween the model prediction and the one-hot label.
General Findings: For most scores, we found that pruning the hardest examples resulted in mod- els with poorer test accuracy compared pruning the easiest examples. This supports the findings of Sorscher et al. (2022), which hypothesized that hard examples contain critical information about the decision boundaries of classes in larger, less noisy datasets. We also find that out-of-the-box implementations of influence scores with the ex- ception of VoG do not result in test accuracy higher than the random sampling baseline with- out score hyperparameter tuning. For example, for EL2N scores, it is crucial that the scores are computed early during fine-tuning for best results. We explored different implementations and chose those that gave best results for data pruning, while adhering to the criteria listed in Sec. 2.1.
Forgetting Score (Toneva et al.,
How often an example is forgotten i.c. goes from being classified correctly at checkpoint to incorrectly at i + 1.
2019)
PVI (Lthayarajh ct al. 2022)
Fine-grained information-theortia quantity whose expectation value is the amount of usable information (in bits) by the model.
TracIn (Garima et al., 2020)
Influence of any example towards another example s' hy tracking their gradient dot products. We generate the self-influence scores where = s'.
Table I: Description of the influence scores we gener- ated on the SNLI dataset. See App. B.1 for implementa- tion details.
SNLI training data. We then pruned the training data by using the scores sample either "easy" or "hard" examples,* and measured test accuracy for a model trained on the reduced dataset. We com- pared these score-sampling pruning results to prun- ing by random sampling. We defer details of the implementation of influence scores and additional findings to App. B, but note here that we consider two normalization schemes for VoG scores: class- normalization' where the scores normalized with respect to the mean and standard deviation of each class, and dataset-normallzation, with respect to the full training dataset.
VoG: Remarkably, VoG required only a single model training run and no hyperparameter tuning. At 45% of training data removed, pruning class- normalized VoG-easy examples led to a test accu- racy of 0..04+0..200 0.2%%, compared to 85.52+ 14% with all of the data. At smaller pruming fractions ($ 10%), performance is roughly within the mar- gin of error of sampling randomly. We find that sampling dataset-normalized scores generally per- forms worse than normalieddl (84.60+1.500 at 45% easy pruned), which is due to the over- representation of the "contradiction" class (Fig. 2) in the tail. We will revisit the merits of class versus dataset normalization in See 3.
Results on a relatively clean public dataset like SNLI may not always translate results on large, commercial datasets that are noisier and highly class-imbalanced. In Section 2.4, we address this concern by running similar pruning experiments on SNLI with increasing levels of randomly gener- ated label noise. 6 We then computed VoG, TracIn, and PVI scores?, pruned casy/hard examples based on those scores, and compared test accuracy random sampling baseline.
EL2N: Best results were obtained by computing EL2N scores early in training; we found epoch 2 outperformed the random pruning baseline for small to moderate pruning fractions (between 0-25%), but worse beyond that.
EL2N is a margin-based metric, which means that examples drawn from the tail of the EL,2N distribution should lie close to the decision bound- ary between classes (Paul et al., 2021; Sorscher et al., 2022). If that is true, then removing these examples should dissolve the decision boundary between different classes, and account for the drop in test accuracy. We provide some evidence for this in App. B.4 by clustering the t-SNE (Hinton and Roweis, 2002) encoder representations of the train- ing and test data, before and after pruning EL2N-
'Wo use the terminology "casy" and "hard" for pedagog- ical reasons. Strictly speaking, we are running data-prunt batti-runing experiments where examples are sampled from either the head or tail of the score distributions. Frequently, these examples do correspond to what a human would find easy and hard, respectively, but we clarify in Section 2.3 when they do not. "As originally prescribed in Agarwal and Hooker (2022). "Details about the lahel noise are given in App. B.6. 'These scores were chosen in the noisy label setting due to their reported efficacy in surfacing defoctive data.

---

86
VoC (class-norm)
EL2N
PV1
85
test accuracy [%]
84
86
VoG (dataset-norm)
Forget
Tracin
85
84
easy
random
bard
83
0
20
40 0
20
40 0
20
40
training data pruned [%]
Figure Shows the test accuracy on SNLI as a function of training data pruned for different influence scores (pruning easy examples is blue, random in gold, and hard in orange). Points show the mean test accuracy computed over three training runs, and shaded regions indicate the I-a envelope.
Forgetting Scores: We observe consistent im- provements over the random sampling baseline when pruning the least forgotten examples, with a test accuracy of 84.58++0.02% 45% data pruned. However, due the rapid convergence fine- tuning (resulting in most examples having zero forgetting score), forgetting events for the entire training set had to be logged at high cadence (once every 50 training steps), making it challeng- ing to apply in production setting.
0.6
eataiment contodiction neutral
0.4
0.2
cless permalized
0.6
0.4
02
dataset normaloze
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
TracIn: Pruning up to 30% of training data by TracIn scores led to consistent improvement over training on randomly pruned data. Similar to pruning EL2N-hard examples, pruning TracIn-hard examples dissolves the decision boundary between classes. The similarity index 10 of the top 5% of hard examples for these two scores is 0.37 (ver- 0.11 for random sampling). indicating they are roughly sampling the same difficult examples.
VoG score
Figure 2: Distribution VoG scores for SNLI dataset with different normalization schemes.
hard train data.
PVI: We found that beyond a small fraction (5- 10%) of data pruned, pruning by PVI scores generally did not outperform random pruning. Although manual inspection of the top negative- scoring PVI examples showed that this score was effective finding several misannotated examples in SNLI (see App. B.5), the number of such mis- annotations was quite small, and beyond a certain pruning fraction, the test accuracy fell off r
2.4 Results on SNLI with Added Label Noise
Fig. shows the results of our noisy data reduc- tion experiment, where the amount of isotropic label noise was varied from five to 30 percent. We observed that pruning VoG-easy examples outper- formed the random-pruning baseline in all of the
"Sampling examples from the head of PVI score dis- tribution corresponds to "hard" or potentially misannotated examples, while the tail corresponds
"easier" examples. "While outside the scope of this work, the explicit depen- dence on the model inductive hias through the "null model" in the definition of PVI suggests that it may he more effective at
iterative pruning, rather than single-shot pruning where computed only once for the model trained all data. "'Gienn by the Jaccard index for two sets A and B: Lnn
JuB

---

prune VoG easy
prune random
prune TracIn-hard
prune PVI-hard
86
85
84
test accuracy [%)
label noise 5%
10%
86
85
84
15%
20%
86-
85
84
25%
30%
83
training data pruned [%]
Figure 3: Shows the results of pruming VoG-easy (bluc), TracIn-hard (orange), PVI-hard (purple), or randomly selected (gold) examples for noisy SNLI. The amount of added isotropic label noise is given in the orange inset. Points indicate the average of three runs and shaded regions indicate the 1-o envelope.
typically found in large, general-purpose commer- cial voice assistants. This setting poses practical noisy labels. However, somewhat surprisingly, this was likely not because VoG-based selection was pruning the misannotated examples themselves. The similarity index between the easiest VoG exam- ples and all of the introduced misannotated exam- ples in the 30% label-noise setting was only = 0.1 1. Compared to random sampling (<s 0.11), we con- clude that VoG does not do better than random chance at finding misannotated SNLI examples, but does reliably extract more influential examples that partially mitigate the effects of label noise. In all noisy settings, we found that pruning VoG-hard examples did worse than pruning randomly.
noisy settings, for large pruning fractions. In some cases, pruming based VoG scores even clawed back a fraction of the initial accuracy loss due to
challenges often not reflected in public datasets, such as noisy and evolving speech patterss, di- verse vocabularies, dialects, carrier phrases, and nut-of-distribution named entities. As an added challenge, we focus on Japanese-data trained mod- els and datasets to determine if VoG-based influ- ence scoring could function in a lower-resource setting. The citiisticalmmodee component in this setting consisted of generic domain classifier (DC), intent classifier (IC), and named entity recognition (NER) models organized in a coupled, hierarchical manner: single multi-class DC model trained on all domains' data to first predict the domain for given utterance, which then invokes domain- specific joint IC-NER model trained on in-domain dIta Both sets of models were based on dis- tilled Japanese BERT models and VoG scores computed using the procedure given in App. B. Fig. 4 shows the class class-normalize scores for a subset of five domains that differ in the proportion of the overall training data they represent and intent- label complexity within that domain. We observe that smaller domains tend to have higher scores (e.g., HealthAndFitnee vs. HomeAutomation) and more complex domains tend to have higher scores (Shopping vs. Video). In some cases, domains that are similar in size and complexity still exhibit different VoG score distributions (Music vs. Video) which reflects differing influence of domain data
Pruning by TracIn and PVI scores resulted in small but persistent accuracy increase when ~ 5- 10% of the hard data was pruned. In the 30% noise setting, the similarity index between the TracIn- hard and PVI-hard examples and misannotated ex- amples was R 0.11, 1111 0.09, respectively, again indi- cating that the accuracy gains are not due to the removal of defects. The number of instances with PVI score of < 0 (indicating potential mis- annotation) comprises only 6% of the mislabeled data. Nevertheless, it appears beneficial use these scores to prune 5-10% of overall hard data that adversely impacts training in a noisy setting.
3 VoG in the Context of NLU
Given its promising results for data reduction on SNLI, we set out to evaluate VoG an environment
See App. C.I additional context.
As measured by Shannon entropy; see App. C.6.

---

RoulthAsdFitnes
EloncAutomation
Shopping
percent sampled [%]
VoG daet-
stratified rundom
80
VoG class morm
0.20
cum. donsity
1.0
60
prob. density
40
0.15
0.5
20
0.10
HomeAutomation
Music
Knowledge
Notifications
Video
HealthAndFitaess
0.05
-1.0
-0.5
0.0
0.5
1.0
1.5
class normalized VoG score
Figure 4: Probability density of class-normand narmanoomalized VoG scores data used to train in-house models, grouped by ground-truth truth domain.
Figure 5: Reduction of training data using different sampling techniques for three large domains (left), medium domains (middle), and a small domain (right).
due to factors that cannot be casily discerned with- out extensive manual analysis."
Experiments on De-identified Historical Data: We compared the performance of models trained the complete de-identified historical training data versus reduced subset of it. The sampled data was used for fine-tuning of the DC model and the in-domain subset of that sample was used for fine-tuning IC-NER stat models. Each model was trained for the same number of epochs without early stopping.
3.1 Experiment Setup
Here we describe in-house experiments which leverage VoG scores for frugal data selection. We present evaluation results on both existing internal data and live user interaction data to determine the impact of pruning training data. In both sets of experiments, models were trained and evaluated on de-identified, historical user data.
Due to the non-deterministic nature of proba- bilistic sampling, we averaged over three random seeds for each sampling technique, trained models on those samples, and report evaluation results.
Sampling Technique: We benchmarked VoG- based data sampling against random sampling and stratified sampling." 14 In stratified sampling, we sample utterances randomly while preserving the domain distribution of the training data. Fig. 5 shows the relative reduction of few domains' training data when pruning using different sam- pling techniques. Sampling by dataset-normalized VoG scores led to highly non-uniform reductions, as expected since those scores reflect data influence with respect to all domains' training data.
The majority of experiments investigated model performance and efficiency in the context of aggres- sive reduction of training data (roughly 50%). We performed additional experiments on VoG-based sampling techniques in which we pruned roughly 14% of historical data, in order to understand the impact on model performance when targeting less aggressive data-reduction targets.
User Study: In randomized user study, we investigated the impact of pruning roughly half of the training data. The scaled user study exposes the model to unconstrained human speech, which varies (often dramatically) in carrier phrase fre- quency, vocabulary, and named entities distribution compared to public datasets, offering a challeng- ing setting to evaluate the efficacy of VoG-based scoring. To ensure that the most frequent user quests captured, large-scale NLU systems often consist of both statistical models as well as de- terministic model artifacts. In addition, although these statistical models are primarily trained on historical data, they also are trained on additional
For VoG-based sampling, we used probabilistic method where the sampling likelihood was propor- tional to the score (see App. C.3 for details). This results higher likelihood of pruning training data with scores located near the head (low-score por- tion) of the VoG distribution. We computed scores using training checkpoints of baseline model and used those scores as sampling weights to create pruned training dataset used for the candidate model.
We include additional domain-level analysis in App. C.4. "In each case, sampling was performed without replace- ment.

---

in-house synthetic data. Thus, in order to under- stand how our results might generalize produc- tion system, we compared composite NLU models containing both statistical and deterministic com- ponents, trained on both historical and synthetic data. The total training-dati size reduction for the candidate model versus the baseline model was approximately 40%.
pruned data relative the model trained on all data:
AER
Relative ER
ERno-pruning
It is useful define metric that measures accu- racy loss per utterance, relative to the no-pruning baseline. We report relative data-score efficiency, originally proposed by Çano and Bojar (2019):
The user study was performed using an in-house experimentation platform that provided signals of model performance that captured potential degrada- tions to the user experience. Pruning solely by DC VoG scores led to some downstream NER-related regressions for the composite model when evalu- ated on historical data. Therefore, caution, we pruned training data based mod- ified version of DC-model VoG scores. We first estimated NER complexity for each intent using the Shannon entropy slot-label-trail annotations (i.e. all data labeled with a given intent were assigned the same complexity score). The final sampling scores were taken to be the mean of the DC VoG scores and estimated NER complexity scores. For details, see App. C.6.
AER AER/ERno-pee /E/en-Rnnonnnnggggg
ER Ad/dno-pruiing
(1)
where ERoo-puniig and dno-pruning correspond the error rate and number of training instances for the model trained on all data." 16 a values express the ra- tio of relative-change in performance to the relative- change in training data. In our data- putt-pruning setting, Ad is negative. More positíve values of OER indi- cate less model degradation due pruning, and ØER score of zero indicates model- perforaner regression relative to the no-pruning baseline.
extra pre-
We analyzed model performance in the user study using two in-house proprietary metries de- veloped to detect model defects involving user re- quests:
The user study ran for days, with users dis- tributed equally between the baseline and candidate models.
Predicted Defect Rate (PDR): PDR is model- based metric that uses both the system response and user-provided signals (e.g., whether they in- terrupted the device response) as features pro- duce a binary prediction for each request indicat- ing whether that request likely resulted in a user- perceived defect. We also report tail-PDR, which is PDR corresponding to the bottom 40% of user traffic. These less common requests are much less likely to be covered by deterministic components. Unrecoverable Error Rate (UER): This metric tracks cases where the utterance cannot be acted on. This can happen, e.g., if no domain picks up a request with a high enough confidence threshold, or there are no clarifying questions that could help to recover from the failure state.
3.2 Experiment Metries
In experiments on historical data we measured performance on held-out test data in terms of component-wise error rates. We measured do- main and intent classification performance using the beced classification error rates DCER and ICER.
To evaluate slot-filling performance, we mea- sured semantic error rate (SEMER):
# Intent errors + Slot errors
SEMER
Test data + # Slots
Specifically, mcasured F-SEMER, the harmonic mean of SEMER using predicted labels as the refer- ence and SEMER computed on ground-truth labels as the reference; this score balances precision/recall equally. We also report the interpretation error rate IRER, which reflects the rate of any kind of error (slots, intents, domain). For all error-rate metrics, we report the error rate of the model trained on
3.3 Results on De-identified Historical Data
Table 2 shows the results of experiments de- identified historical data, comparing relative-error and relative data-score efficiency metrics for VoG, random, and stratified sampling. Overall, the best
IIn Çano and Bojar (2019), the relative data-score elfi ciency metric was used to evaluate how well model accuracy scaled as the amount of training data was increased. We use use a in slightly different but analogous manner to quan- tify how well model errors are avoided as the training data is pruned using a given sampling technique.
This evaluation data consisted of roughly million test cases that were sampled from user data and subsequently annotated by humans.

---

Samp, technique
Atrain
ADCER
AFSEMER
AICER L AIRER
ODEER t
OFSEMER
OTCER t
oTHER
Random
-52%
6.04%
5.56%
5.26%
4.98%
-11.7
-00788
10.19
-9.64
Stratified
-2%%
5.23%
5,14%
4.87%
4.92%
10.14
-9.66
-9.33
-9.54
VoG-class-nom
-52%
5,48%
5.08%
5.20%
4.40%
-00.53
-9.76
100.01
-8.47
VoG-dset-nnom
-52%
2.94%
3.65%
3.18%
3.34%
5.65
7702
6.11
6.42
VoG-class-nomm
-66%
3.62%
3.77%
3.63%
3.39%
-7.87
-8.18
-7.87
-7.36
VoG--seet-nomm
-46%
1,52%
2.20%
1.72%
2.09% %
-3.25
-4.73
-3.69
4.50
VoG-dset-norm -14%
0.53%
1.02%
0.53%
0.97%
-3.86
-7.44
-3.92
-7.09
Table 2: Comparison of offline-test error-rate and data-error efficiency metrics for models trained on in-house data, sampled according to different techniques. All pairwise comparisons were relative a baseline trained on data. For each technique, report the average change in performance for three models trained independently sampled training data.
performance for 22%-pruned data was obtained by models trained on KuG-daatase-nnom--samiii data, while random sampling was associated with the worst model performance across all evaluation metrics. The stratified-san -taillied---aliingg pruning baseline improved over random sampling, particularly with respect to cammainnclssiiiiation accuracy (ADCER of 5.23% for stratified sampling vs. 6.04% for ran- dom sampling). In fact, except for AFSEMER and AIRER that track slotting errors, models trained on stratifiedsamppled data even slightly outperformed models trained on VoG-cllas-norm-ssappa
Metric
Rel. change
95% CI
UER
- -1.18%
[-2.00%,
-0.35%]
PDR
0.45%
[-0.18% 1.09%]
PDR-tail 0.56%
[0.003%, 1.09%]
Table 3: Comparison of top-level UER, PDR, and PDR- tail metrics for baseline model vs. VoG model candidate that has been trained on roughly half of the live data used for training the baseline model. * Not stat. sig. difference (p 0.16).
data. The experimental results in Table 2 demonstrate the importance of score normalization: models trained on data pruned by normalied VoG scores outperformed models trained on data pruned by class-normalie normalized VoG scores across all evalua- tion metrics considered, for both pruning per- centages. Using dlass-normalized scores as sam- pling weights increased overall relative DCER by roughly 1.9x when pruning 52% and by roughly 2.4x when pruning 46%, compared to when sam- pling from dataset-normalized scores. In App. C.5, we provide a detailed domain-level analysis to understand which data contributed most the improvement associated with pruning by dataset- normalized VoG scores versus class-normalized.
roughly 50% of historical data." 17
The reduced- train-data model surprisingly achieved slight statistically significant improvement in overall UER and with no statistically significant change to PDR. We saw a small but statistically sig- nificant degradation in PDR- -ail, which indicates that this type of aggressive data reduction can lead to regressions for less common requests. We also present a domain-level analysis of these top-line results in App. C.5.
Taken together, these results suggest that our NLU training data is highly redundant, and that comparable performance can be had by training on an intelligently chosen subset of it. While re- gressions in per-domain UER and PDR suggest potential downsides of pruming data based solely on DC-model gradients for all statistical models of hierarchical NLU system, these results neverthe- less confirm the overall practical utility of pruning data by VoG scores in commercial settings.
Table also shows that the efficiency metric for normaeztd VoG-pruning was higher when pruning 46% compared to when pruming 52% or 14%. These findings can be used to help in- fer appropriate pruning targets for given training dataset that minimize the need for historical train- ing data without regressing on model performance.
3.4 User Study Results
17 As discussed in Scction 3.1, the candidate model also included other non-live training data (c.g., synthetic) in model training, which was also the case for the baseline.
Table shows the results of our user study com- paring the baseline model to a model trained on

---

4 Discussion
sive, and empirical metries that can estimate the the influence of examples in-context learning. Our work develops the foundational understand- ing necessary to make progress on that problem by generalizing results from the computer vision field (such as those scores that approximate more computationally expensive influence functions) to language-bel based tasks.
In this work, we initiated study of the application of influence scores in efficient language data sam- pling. First, we benchmarked a diverse subset of influence scores in a data reduction task and found promising results pruning using VoG scores. In the second part, we used these preliminary results on SNLI impetus to scale VoG up to model stack commonly used in commercial voice assis- tants. We provided a detailed account of how score normalization affects final results and again found encouraging results on experiments involving his- torical data using dataset-normalized VoG scores, as well as in a user study. In particular, we did not see any overall regressions when a model trained only ~50% of historical data was deployed.
Limitations
We hope that our in-house experiments provide a useful data point on the practical utility of influence scores, However, we note that we could not experi- ment with the same number of sampling techniques or prune sizes as we did in SNLI experiments due to computational overheads, and acknowledge that our in-house results are not readily reproducible. In addition, the customer data available for model training and experimentation changes frequently. c.g. due to data-cxpiration policies or customer data-deletion requests, which limited our ability to strictly control the training data between all re- lated model-training runs. However, this limitation applied equally to cach experimental sampling tech- nique and only impacted the relative training-data reductions for given pruning fraction by less than 0.01 % for all sampling techniques.
This work mainly focused on data reduction; would be interesting to reverse some of the pre- sented analysis for data mixing/augmentation in order to identify economical ways of surfacing new model data. Our work also focused on supervised settings using BERT architectures; an obvious path forward would be to extend the definition of these scores to model pretraining and to, e.g., decoder- only architectures that are commonly used for large language models (see, e.g., Marion et al. (2023) along similar lines). While this may be difficult to implement at a microscopic level for a corpus of pretraining data such as the Common Crawl, one avenue could be to apply this method at coarse- grained level by grouping together texts by simi- larity. Given the recent results of Sorscher et al. (2022) and Abbas et al. (2023), this suggests path towards training data-efficient large language mod- els that could, principle, outperform empirically observed scaling laws (Kaplan et al., 2020; Hoff- mann et al., 2022).
We also note that the goal of our paper was not to find the absolute, best-performing influence score through extensive score- heprrepayrrrrtttterr tuning. It is highly possible, for example, that the bench- marking mcasurements reported in Fig. I can be refined for better accuracy (though we have aimed to provide a thorough documentation in App. B.1 of our initial pass at score-hyperparamette tuning on SNLI). Instead, our results should be taken as proof-of- concept of the existence and applica- bility of a simple, scalable, one-shot influence score in both public and production language data reduction settings.
Another highly promising application of our re- sults is determining the influence of specific exam- ples in in-context learning. One concrete general- ization of VoG scores to this setting would be to look at variance of model weights (e.g., in attention heads) in specific layers over the length of the input sequence. This could provide an interpretable met- ric for identifying influential contextual examples and failure modes, at the level of specific tokens (Grosse et al. (2023) propose similar methods using influence functions). Given the increased recent interest in this area of research due to concerns over bias, toxicity, and fairness in large language models, there is a critical need for simple, inexpen-
Finally, our public data experiments primarily focused on a controlled setting using the SNLI dataset, which may not generalize to other pub- lic datasets. To address this, conducted the scaled user study which exposed the model to un- constrained human speech, which varies (often dra- matically) in carrier phrase frequency, vocabulary, named entities distribution and other aspects from publicly available datasets such as SNLI.

---

Ethics Statement
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Tiric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish. Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. CoRR. 1is/20055111655
Influence-basee filtering can have disparate impact on predictions for classes with less annotated data. This can increase the likelihood of training data as- sociated with less frequent language patterns being filtered out, which can increase bias in data that then propagates the trained model. We have at- tempted quantify this bias and correct it through, e.g., nommaiiaoon......iionn.....
Acknowledgements
Frion Çano and Ondiej Bojar. 2019. Efficiency met- ries for data-driven models: A text summarization case study. In Proceedings of the 12th International Conference on Natural Language Generation, pages 229-239, Tokyo, Japan. Association for Computa- tional Linguistics.
We are grateful for the helpful discussions and feedback provided by Jason Crowley and Kay Rottmann.
References
Nicholas Carlini, Úlfar Erlingsson, and Nicolas Paper- not. 2019. Distribution density, tails, and outliers in machine learning: Metrics and applications. CoRR, abs/1910. 134.7
Amro Abbas, Kushal Tirumala, Daniel Simig, Surya Ganguli, and Ari S. Morcos. 2023. Semdedup: Data- efficient learning at web-scale through semantic dedu- plication. CoRR, abs/2303.095..
Cody Coleman, Christopher Ych, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang. Jure Leskovec, and Matei Zaharia. 2020. Selection via proxy: Efficient data selection for deep learning. In 8th International Conference on Learning Repre- sentations. ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Chirag Agarwal, Daniel D' sosuuaa, and Sara Hooker. 2022. Esttmating example difficulty using variance of gradients. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Puttern Recognition, pages 10368-10378.
Chirag Agarwal and Sara Hooker. 2022. Estimating example difficulty using variance of gradients. 2022 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 10358-10368.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Papers), pages 4171-4186. Association for Computational Linguistics.
Robert J. N. Baldock, Hartmut Maennel, and Behnam Neyshabur. 2021. Deep learning through the lens of example difficulty. In Advances in Neural In- formation Processing Systems 34: Annual Confer- ence on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual. pages 10876-10889.
Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. 2022. Understanding dataset difficulty with V-usable information. In Proceedings of the 39th International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 5988-6008. PMLR.
Jacob Bien and Robert Tibshirani. 2011. Prototype selection for interpretable classification. The Annals Applied Statistics, 5(4):2403 2424.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics.
Mohsen Fayyaz, Ehsan Aghazadeh, Ali Modarressi, Mo- hammad Taher Pilehvar, Yadollah Yaghoobzadeh, and Samira Ebrahimi Kahou. 2022. BERT on data diet: Finding important examples by gradient-based pruning. CORR, abs/2211.05611
Ronan Le Bras, Swabha Swayamdipta, Chandra Bha- gavatula, Rowan Zellers, Matthew E. Peters, Ashish Sabharwal, and Yejin Choi. 2020. Adversarial filters of dataset biases. In Proceedings of the 37th Inter- national Conference on Machine Learning, ICML. 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 1088-1088. PMLR.
Vitaly Feldman and Chiyuan Zhang, 2020. What neural networks memorize and why: Discovering the long tail via influence estimation. In Advances in Newral Information Processing Systems 33: Annual Confer- ence on Neural Information Processing Systems 2020, NewrlPS 2020, December 6-12, 2020, viriual.

---

Garima, Frederick Liu, Satyen Kale, and Mukund Sun- dararajan. 2020. Estimating training data influence by tracing gradient descent. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20 20, Red Hook, NY, USA. Curran Associates Inc.
Shaojie Jiang, Thomas Wolf. Christof Monz, and Maarten de Rijke. 2020. TLDR: token loss dynamic reweighting for reducing repetitive utterance genera- tion. CoRR, abs/2003.1 1963.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. CoRR. abs/2001.0833
Roger Grosse, Juhan Bae. Cem Anil, Nelson Elhage. Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilé Lukošiūté, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bow- man. 2023. Studying large language model general- ization with influence functions.
Been Kim, Cynthia Rudin, and Julie A. Shah. 2014. The bayesian case model: A generative approach for case-based reasoning and prototype classifica- tion. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Informa- tion Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 1952-1960.
Guy Hacohen, Leshem Choshen, and Daphna Wein- shall. 2020. Let's agree to agree: Neural networks share classification order on real datasets. In Pro- ceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Vir- rual Event, volume 119 of Proceedings of Machine Learning Research, pages 3950-3960. PMLR
Pang Wei Koh and Percy Liang. 2017. Understand- ing black-box predictions via influence functions. In Proceedings of the 34th International Conference Machine Learning. ICML. 2017, Sydney, NSW, Aus- tralia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages PMLR.
Sariel Har-Peled and Akash Kushal. 2007. Smaller core- sets for k-median and k- k-means clustering. Discret. Comput. Geom., 37
Satoshi Hara, Atsushi Nitanda, and Takanori Machara. 2019. Data cleansing for models trained with SGD. In Advances in Neural Information Processing Sys- tems 32: Annual Conference Neural Information Processing Systems 2019, NewrlPS 2019, December 8- 64, 2019, Vancouver, BC, Canada, pages 4215- 4224.
Futong Liu, Tao Lin, and Martin Jaggi. 2021. Under- standing memorization from the perspective of op- timization via efficient influence estimation. CoRR, abs/21 12.08798.
Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Ad- vances in Neural Information Processing Systems 30: Annual Conference on Neural Information Process- ing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4765-4774.
Geoffrey E. Hinton and Sam Roweis. 2002. Stochastic neighbor embedding. In Advances in Neural Infor- mation Processing Systems, volume 15. MIT Press.
Jordan Hoffmann, Sehastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai. Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc Aurelia Guy, Simon Osindero. Karen Si- monyan, Frich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. 2022. Training computc-optimal large language models. CoRR, abs/2203. 15556.
Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzich Fadace, and Sara Hooker. 2023. When less is more: Investigating data pruning for pretraining ilms at scale.
Kristof Meding, Luca M. Schulze Buschoff, Robert Geirhos, and Felix A. Wichmann. 2022. Trivial or im- possible - dichotomous data difficulty masks model differences (on imagenet and beyond). In The Tenth International Conference on Learning Representa- tions, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, and Andrea Frome. 2021. What do com- pressed deep neural networks forget?
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. 2021. Deep learning on a data diet: Find- ing important examples early in training. In Ad- vances in Neural Information Processing Systems, volume 34, pages 20596-20607. Curran Associates, Inc.
Myunggwon Hwang, Yuna Jeong, and Won-Kyung Sung. 2020. Data distribution search to select core- set for machine learning. In SMA 2020: The 9th International Conference on Smart Media and Appli- cations, Jeju. Republic of Korea, September 17 19. 2020, pages 172-176. ACM.
Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, and Kil- ian Q. Weinberger. 2020a. Identifying mislabeled data using the area under the margin ranking. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, viriual.
Angela H. Jiang, Daniel L.-K. Wong, Giulio Zhou, David G. Andersen, Jeffrey Dean, Gregory R. Ganger. Gauri Joshi, Michael Kaminsky, Michael Kozuch, Zachary C. Lipton, and Padmanabhan Pillai. 2019. Accelerating deep learning by focusing on the biggest losers. CoRR, abs/ 190.00762.

---

Geoff Pleiss, Tinyi Zhang, Ethan R. Elenberg, and Kil- ian Q. Weinberger. 2020b. Identifying mislabeled data using the area under the margin ranking. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual.
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannanch Hajishirzi. Noah A. Smith, and Yejin Choi. 2020. Dataset carlography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. EMNLP 2020, Online, November 16-20, 2020, pages 9275- 9293. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with unified text-to-tent trans- former. J. Mach. Learn, Res,, 21:140: 1-140:67.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geof- frey J. Gordon. 2019. An empirical study of example forgetting during deep neural network learning. In International Conference on Learning Representa- tions.
Mengye Ren, Wenyuan Zeng, Bin Yang. and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In Proceedings of the 35th In- ternational Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 4331-4340. PMLR.
Iulia Ture, Ming- wei Chang, Kenton Lee, and Kristina Toutamova, 2019. Well-read students learn better: The impact of student initialization on knowledge distillation. CoRR, abs/ 08962.....
Xiaoxia Wu, Ethan Dyer, and Behnam Neyshabur. 2021. When do curricula work? In 9th International Con- ference Learning Representations, ICLR 2021, Vir- rual Event, Austria, May 3-7, 2021. OpenReview.nec
HSVNS Kowndinya Renduchintala, Krishnateja Kil- lamsetty, Sumit Bhatia, Milan Aggarwal, Ganesh Ra- makrishnan, Rishabh Iyer, and Balaji Krishnamurthy, 2023. Ingenious: Using informative data subsets for efficient pre-training of large language models.
Chih-Kuan Yeh, Joon Sik Kim, Tan En-Hsu Yen, and Pradeep Ravikumar. 2018. Representer point selee- tion for explaining deep neural networks. Ad- vances in Neural Information Processing Systems 31: Annual Conference on Neural Information Pro- cessing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 9311-9321.
Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "why should I trust you?": Explain- ing the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Fran- cisco, CA. USA, August 13-17, 2016, pages 1135- 1144. ACM.
Shoaib Ahmed Siddiqui, Nitarshan Rajkumar, Tegan Maharaj. David Krueger, and Sara Hooker. 2022. Metadata archaeology: Unearthing data sub- sets by leveraging training dynamics. CoRR,
A Review of Influence Scores and Related Work
In this work, we use "influence scoring" as a broad term refer the large body of scientific literature focused on using artifacts of the learning algorithm such as the loss, model confidence, etc. - to determine the relative importance of specific data instances. Many of these methods can be used to determine influential test examples, in addition to training. This review section should not taken to be comprehensive or exhaustive, but rather as a starting point to delve into subtopics in this area of research. We suggest the "Related Works" section in Swayamdipta et al. (2020) for a nice review of these methods as well.
abs/2209.10015.
Yan Song. Prescott Klassen, Fei Xia, and Chunyu Kit. 2012. Entropy-basee training data selection for do- main adaptation. In COLING 2012, 24th Interna- tional Conference on Computational Linguistics, Pro- ceedings of the Conference: Posters, 8-15 December 2012, Mumbai, India, pages 1191-1200. Indian Insti- tute of Technology Bombay.
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari S. Morcos. 2022. Beyond neural scaling laws: beating power law scaling via data pruning, CoRR, abs/2206. 14486.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. 2019. One pixel attack for fooling deep neu- ral networks. IEEE Trans. Evol. Comput., 23(5):828 841.
There are a number of works that aim to use empirically formulated scores to approximate or improve upon influence functions formulas that estimate the impact of training examples on test examples (see. e.g., Koh and Liang (2017) and references therein). TracIn (Garima et al., 2020) is one such example. Similarly, there are a num- ber of methods that center around explainability
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceed- ings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 3319-3328. PMLR

---

and interpretability; e.g., finding representer points by decomposing pre-activation predictions (Ych et al., 2018), methods that aim to extract feature importance (Sundararajan et al., 2017; Lundberg and Lee, 2017), develop reliable models of predic- tions (Ribeiro et al., 2016), and capture learning order in neural networks (Hacohen et al., 2020).
rate of 1 10-4 for the encoder and x 10-3 for the classifier head. The classifier head was a three-layer fully connected neural network with an intermediate dimension of 64 x64, with 10% prob- ability drop-out. We specify our implementations below for the influence scores:
VoG: VoG scores were computed adhering closely to the method described in Agarwal and Hooker (2022), with the exception that the input pixels were replaced with the input embeddings. Gradients were computed at the locations of the ground-truth labels. The pscudo-code given in al- gorithm describes our implementation in full. The computation is split into two steps for clarity: in the first, we compute the gradients of the pre- softmax model outputs at the location of ground truth labels with respect to the outputs of the embed- ding layer, for the desired number of model check- points Ne (we used 10). For each example i, these gradients will be of dimension embedding_ dim), which denote by Gigh where is the checkpoint, or in matrix form G(")
Next, there is body of literature that broadly includes methods that aim to quantify data quality and difficulty. This includes core-set set methods that select an intelligently weighted subset of training data (Hwang et al., 2020; Har-Peled and Kushal, 2007), information-theorete measures of data qual- ity (Song et al., 2012; Ethayarajh et al., 2022), train- ing dynamics based methods to diagnose and map out datasets (Swayamdipta et al., 2020; Agarwal and Hooker. 2022; Siddiqui et al., 2022), and pa- pers that provide empirical and theoretical defi- nitions of dataset difficulty (Meding et al., 2022; Baldock et al., 2021; Sorscher et al., 2022). Simi- larly, previous works have used adversarial filters (Bras et al., 2020) and proxy selection methods (Coleman et al., 2020) to score examples. A differ- ent approach taken by several works is to identify certain rypes of examples such prototypical ex- amples that match human expectations (Kim et al., 2014; Bien and Tibshirani, 201 I), memorized ex- amples (Liu et al., 2021), or outliers and tails in distributions (Carlini et al., 2019).
((e)
0A G
(2)
aE (c)
where A. denotes the pre-softmax model outputs at the location of the ground truth label and E(() de- notes the embeddings. Next, the VoG score for each example can be computed by first computing the gradient means and variances across checkpoints:
There are also several methods that leverage training dynamics to explicitly maintain/improve accuracy and learning efficiency (Hara et al., 2019; Pleiss et al., 2020b; Toneva et al., 2019; Paul et al., 2021; Jiang al., 2019; Fayyaz et al., 2022; Jiang et al., 2020), and those that quantify bias in com- pressed models (Hooker et al., 2021).
1
Σ
P4
No
checkpoints: e
(3)
V
VN. (G)
m.)
In this paper, we could not exhaustively cover each of these scores, but as outlined in Sec. 2.1, we aimed to select sufficiently diverse subset that could plausibly scale to production stack.
The (unnormalized) score e for each example is then given by the mean of V, (that is, we average over the input embeddings, analogous to how the scores were averaged over pixels in Agarwal and Hooker (2022)). The final scores VoG, can be com- puted by normalizing v with respect to either the score mean and standard deviation in each class, as originally prescribed in Agarwal and Hooker (2022), or the score mean and standard deviation for the full datasct:
B Additional Details about SNLI Experiments
B.1 Score Implementations
Our implementation for the scores we tested in Ta- ble aims to mirror the implementations given in the original references as closely as possi- ble. Our experiments were mainly carried out on sEALSMALL (I. = 4, H = 512, 29.1 parame- ters), trained for 10 epochs (with a batch size of 128) using the Adam optimizer with a learning
VoG; 14
Plelas
(class-norm),
Tclass
(4)
U2 plaset
VoG; dssct
(datasct-norm)

---

VoG scores were computed in a "one-shot" manner, using gradients logged from a single training run.
all of the model weights. A key question, then, is which layer the above gradients should be taken with respect to. experimented with two possi- bilities: using the last-layer classifier weights and using the last encoder hidden layer. We found that in both clean and noisy SNLI settings, using the encoder hidden state gave more stable and better results for data pruning, relative to the random sam- pling baseline (this is what and 3). The final scores were computed from the L2 norm of the gradient dot products in eq. Scores were computed using 10 model checkpoints from a single training run, logged every 500 iterations during training.
Algorithm 1 VoG implementation for language data.
load model used for training
gradients G empty dict
for checkpoint in training checkpoints do load m +
used in Figs.
set m to inference
for batch E DataLoader do
x, y < batch
outputs <- mar)
get embedding layer E from set embeddings to retain grad one-hot vector encoding of y compute back. pass on outputs w.r.t Y Glc] [batch] < detached gradients of E zero-out model gradients
Forgetting Scores: Forgetting scores measure the number of times example moves from being classified correctly classified incorrectly, A key hyperparameter we had to tune was the cadence at which forgetting events are computed for each training example. In Toneva et al. (2019), this mea- surement was done at the batch-level granularity that is, forgetting scores were updated each time the example was seen in the minibatch. We found that due to the rapid convergence of fine-tuning, this resulted in too many examples having zero forgetting score. Fig 6 shows the distribution of forgetting scores for two different cadences; we see that the number of zero forgetting events in- creased by approximately 26% when the scores were computed every 500 iterations as opposed to every 50 iterations. To get enough resolution, we compute forgetting scores for the entire training dataset every 50 training iterations for the first 2 epochs of training, averaged over three random seeds in order obtain sufficient precision in the head of the score distribution. For future work, may be best to go to even finer resolution and compute forgetting scores as frequently as possible carly in training, e.g., the first dozen itcrations in training.
end for end for
VoG scores empty dict for batch b E G[-] do
for example i batch do
V Var(G[cc[t)) across checkpoints c
e|i] < mean of V
end for end for return v
TracIn: TracIn scores were computed using eq. given in Garima et al. (2020), reproduced here for convenience:
TracIn(z, 2')
(5)
n& (w Vul(ur z').
The content of the above equation is that TracIn computes score for pairs of examples z, z'. such that high (low) scores correspond to proponents (opponents) to z. 77 denotes the learning rates for checkpoints i 1....,k: A. Gradients are taken with respect model weights at these checkpoints. In our fine-tuning experiments, the learning rates are constant across checkpoints 7) = 1), and thus en- ter overall factor to the scores that can be normalized away.
EL2N: EL2N scores were computed in the man- ner described in Paul al. (2021) by the equation:
EL2N(2) = [[sottmax[(()]] -yls yll
(6)
where coftmax[f(z) indicates the softmax of the model outputs and indicates the one-hot encod- ings of the labels.
Following the original paper. for generating scores for training examples, we compute the self- influence scores i.c. we set z' 2. It not tractable to compute the gradients with respect to
Final EL2N scores were obtained by averaging scores over 10 training runs. Consistent with the findings of Fayyaz et al. (2022), we found that critical that the scores are computed early

---

entaliment contrndiction
5
1.0
cumulatios
4
counts 100
0.5
3-
sampled every 50 iter.
counts
1.5
2
computation ep.
-7
0.2
1
0.5
0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
sampledd every 300 Iter.
EL2N score
2
6
8
10
Eorgetting score
Figure 7: Distribution of EL2N scores for SNLI dataset at two different checkpoints.
Distribution of forgetting scores for SNLI training data at two different sampling frequencies.
Figure 6:
and final scores were obtained from averaging over 10 random seeds. The scores were computed for models trained on all of the data (and subsequent models were trained on pruned data according to those scores). In future work. it would be inter- esting to consider iterative pruning, where scores recomputed for models trained on data pruned using the previous models' scores.
training. We hypothesize that this is due to the rapid convergence of BERT the training set; after only 6 epochs of training BERTsMALL has nearly memorized the training set (achieving close 97% accuracy), which results an EL2N score close to zero for many examples for which the margin is large (i.e. these examples are always learned). This is confirmed in the distribution of EL.2N scores seen in Fig. 7, where there is a greater spread in EL2N scores computed epoch 2 compared epoch ~7, When computed epoch ~7, the standard deviation of the scores corresponding to 50% of the head of the distribution (i.e. the easiest examples) is on the order of 10 10 roughly the mean standard deviation for individual scores between 10 re-runs, so the precision with which we can measure the score of an individual example is roughly the order of 1/vI0 10 10-2 > 10-4 This
B.2 Probabilistic Sampling vs. Hard Cut-Off in SNLI
Each influence score provides ranking of exam- ples that orders their importance. We considered two different strategies for selecting data once the scores are computed: hard cut-off and probabilis- tie sampling. For the hard cut-off method, we only retain examples with scores above a certain threshold (c.g., to prune 30% of the "casy" data, we would prune the 30% of examples corresponding to the head of the score distribution). The probabilis- tic method relaxes this condition, and each example has a chance of being retained with probability equal the softmax of its score. We used the probabilistic sampling method in two cases: first, in sampling from forgetting scores since this was a discrete score with a vast majority of examples sharing an example score of 0. Therefore, setting hard cut-off would have removed all of these ex- amples. Second, used probabilistic sampling for dataset-normalized VoG scores, since pruning from the tail with hard cut-off resulted in too many examples from the "entailment" class being removed (see Fig. 2). For our in-house experiments on customer data, we opted for linear probabilistic
calculation means that in order resolve the easiest examples correctly for scores computed late in training, one would need to average EL2N scores over an increased number training re-runs.
PVI: Pointwise V-information (PVT) scores from Ethayarajh et al. (2022) were computed using eq. 4 of their paper, reproduced here:
(a + y) = - log, (v) + log, f (v) (7)
The scores require fine-tuning a "null" model, de- noted by f[0] (v) that is trained empty null inputs. f(r][() denotes the model fine-tuned training data. Both models were trained for epochs (we find that empirically this is approxi- mately when the V-information 18
and EL2N scores become correlated. This offers another explanation for why EL2N scores have be to computed carly in training the amount of usable bits of information decreases over the course of model training.
18 IItterestingly, we also find that at late training times, PVI

---

sampling instead of softmax sampling (described in Sec. C.3).
et al. (2022), it is hypothesized that this happens be- cause these examples are support vectors critical in forming the decision boundary between classes and removing them does not result in usable representa- tions of the test data. Phrased differently, we have seen that most training examples can be removed without dramatically impacting test accuracy; the converse of this statement is that small number of training examples have outsized impact on test accuracy. We can visualize this explanation in the case of EL2N scores, which are explicitly defined to be the marginal distance between the model predictions and the one-hot encoded labels.
B.3 Impact of Model Size
We investigated the effect of model capacity on pruning by VoG-sampling. Fig. 8 shows test accuracy versus percent of SNLI training data VoG-score
pruned, for both class-normallize sampling and random sampling. for BERTSMALL and BARE (L = 12, H 768, 110.1M pa- rameters) (Ture et al., 2019). The BERT en- coder was trained using the Adam optimizer with a learning rate of 0.9 x 10-4, along with 3-layer classifier head with an intermediate layer dimen- sion of 256 and learning rate of 0.95 10-3, with 10% drop-out probability.
In Fig. 9, the left subplot shows the t-SNE rep- resentation of the SNLI training data, with five percent of the most difficult EL2N examples high- lighted in red. The bulk of these difficult exam- ples lies on the decision boundary between entail- ment and neutral classes. When none of the data is pruned, the center plot shows the t-SNE rep- resentation of the SNLI test data, comprised of three well-defined clusters. When the most diffi- cult EL2N examples are removed from the training set, we see that the representation the test data (rightmost subplot) is comprised of less defined clusters of roughly uniform density. In particular, the boundary between contradiction and neutral classes almost completely disappears, indicating that the model cannot resolve the differences be- tween the two classes well in the no-pruning scenario.
Aside from having a somewhat larger spread in final test accuracy, we see that the rough qualitative effect of the larger architecture is an overall shift in accuracy for each of the sampling methods. At 45% of the data pruned, sampling by VoG-easy on BERT BASE has test accuracy of 06.70--..8% compared to BERTSMALL which had 85.0440.22% This provides some encouraging evidence that VoG- based pruning is useful for performance-effiienn sampling of training data across different BERT models.
89
oasy curden hand
BERT base
sEnl
#7
best accurucy
B.5 Score Distributions and Examples in SNLI
86-
Score distributions for SNLI are shown in Fig. 10. Examples from the head and tail of each of these distributions is given in Tables 11 through 15.
84-
0
10
20 30
40
B.6 Noisy Data-Reduction Experiment Details
training data pruned [%]
For experiments on SNLI with added noise, we chose experiment using VoG, TracIn, and PVI scores. These were selected because VoG outper- formed other metrics in the non-noisy data reduc- tion setting, and TracIn and PVI due to their poten- tial efficacy in identifying misannotated data.
Figure 8: Shows test accuracy versus percent of SNLI training data pruned by VoG and random sampling, for both BERT and BERTBASE. Each data point rep- resents the mean over three training runs while shaded region shows the 1-a envelope.
Noisy versions of the datasets were created by shuffling of labels in an isotropic manner, which meant there was a chance (roughly 30%) that any given label would not flip. Therefore, the label noise quoted Fig. should be taken as an upper
B.4 Encoder Representations of Scored Data In our data pruning plots (Fig. 1), we observed a drop in test accuracy when pruning hard exam- ples for most of the influence scores. In Sorscher

---

SNLI train
SNLI best [no praning)
SNLI test (5% E12N tail pruned)
GANE 10.
contradiction entallment
neutral
BL2N 5%
15 -6
10
15
-5
5 15
SNE First
Figure 9: t-SNE plots of EL2N for the training data with 5% the EL2N highlighted red (left), test data before pruning the tail 5% of EL.2N examples (center), and test data after pruning.
bound to the true number of misannotations' 19 as quantified Table 4.
by named entity recognition (NER) to resolve en- tities (such as city names, times, etc.) within the utterance. Each BERT-based statistical model was trained on spoken-form Japanese data. Our ex- periments focus the fine-tuning stage of model training, performed in-house data.
% noise added
# misannotations
% of train data
5
18433
3.35
10
36693
6.67
15
55083
10.01
In our experiments we computed VoG scores for the DC model, but we measure the performance impact of doing so for the composite hicrarchical model, including the impact on the accuracy downstream IC and NER models.
20
73724
13,40
25
91535
16.63
30
110189
20.02
Table 4: Shows the number of misannotations for each version of noisy SNLI data, determined by comparing against the version of SNLI with added label noise. The last column shows the percent of mislabeled train- ing data.
C.2 Distribution of Types of Training Data
Model training data is collated from multiple, varied sources (e.g., synthetic, -nmaan-anottted, weak-signal). Data from different sources may have different label-noise distributions and class distributions, which in turn can impact influence scores computed on that data. Table 5 shows the distribution historical and synthetic data in-
C Additional Details about In-House Experiments
C.1 Overview of Common Commercial NLU Systems
house data.
Domain
% total
% historical
% synth.
The natural language understanding (NLU) com- ponent of common commercial systems consist of deterministic systems that cover the most critical and frequently occurring utterances (e.g., "stop!"), while other utterances are covered by multiple sta- tistical ("stat"") models organized in a hierarchical fashion. For example, an utterance spoken to conversational assistant not covered by determin- istic artifacts will typically first be classified to an appropriate domain by BERT-based domain clas- sifier (DC) model, then a specific intent within that domain by intent classifiers (IC) models, followed
Music
18%
89%
11%
Home Automation
12%
89%
11%
Knowledge
11%
85%
15%
Notifications
7%
86%
14%
Video
6%
74%
25%
Shopping
5%
93%
Health & Fitness
<1%
4%
96%
Overall
100%
81%
19%
Table 5: Distribution of data sources used for training the model used for in-house experiments.
In in-house experiments, training-data pruning was performed on de-identified historical user data. In the experiments on historical data, this was the only training data used; models were fine-tuned
19 In historical data, we often do not have an exact count of the misannotations, hut only a rough estimate of the overall noise.

---

VoG (class-norm)
VoG (dset-norm)
2
1
0
4
0
4
3
3
counts / 10s
EL2N
PVI
0
0.0
0.5
1.0
forgetting scores
Tracin
2-
2
2
4
6
0.0
0.2
0.4
0.6
influence score
Figure 10: Shows the distribution of influence scores in Table. computed for SNLI data.
solely on user data. In the user study experiments, stat-model training data was performed on histor- ical data as well as additional supplemental data (e.g., resampled, synthetic, etc.).
in Appendix B.2.
The motivation behind this was that we did not have robust characterization of the noise in our customer data and found that softmax sampling was too aggressive in downsampling easy-to-learn utter- ances (and perhaps retaining too many noisy, hard- to-learn examples). In order to preserve larger fraction of these easiest utterances, sampling ap- proach was used where the probability of sampling was directly proportional to the VoG score. This was accomplished by linearly transforming normal- ized VoG scores VoGnorm (including negative and positive values) to the range [e, 11. The positive- transformed scores were them normalized by divid- ing by the sum of all trantfomed scores in order produce sampling probabilities. This type of transformation aims to preserve the rela- tive ratios between old and new values that existed pre-transformation. That is, the VoG score Utterance was twice the VoG score of Utterance B, the sampling probability for Utterance A will be approximately twice the sampling probability for Utterance B.
This difference was motivated by practical con- cerns. For some smaller (e.g. newly introduced, or low traffic) NLU domains, the amount of histori- cal data available for model training was small in size (less than 20 instances). This combined with domain-wise data imbalance led regression in that subset of domain. That was fine in offline anal- yses (since it applied to all pruning conditions) but unacceptable for exposure to real users.
Due to the cost and operational overheads in- volved with running the user study, could not try the same number of sampling techniques as we did for the historical data experiments. In our user study, we tested our most promising technique from the historical data experiments, sampling by dataset-normalizer VoG scores, and compared rel- ative to the bascline model.
C.3 Filtering Train Data via Score- Weire-Weighted Sampling
Finally, in order filter training data by sam- pling scores we first decide on proportion of data to prune. This in turns determines the number training examples to sample via weighted random
Our approach for filtering in-house data based on influence scores used a slightly different approach than the softmax probabilistic sampling described

---

sampling without replacement. See Figure 12 for an example of 50% train-data reduction using this method.
normalized VoG scores for the same subset of do- mains shown in Figure 4.
HoultilkssPittwa Homm:Automation
Music Stopping
Video
C.4
VoG Distributions In-House Data: Additional details
0.30-
1.0
0.25
Dum. densily
densily
0.16
0.20
0.5
0.14
0.15
prob. donsity
0.12
0.0
0.10
0.100
0.08
0.05
0.06
0.00
0.5
1.0
1.5
2.0
2.5
0.02-
detaset-normallzedd VoG score
0.00
Figure 13: Dataset-normalized VoG scores for historical data downsampled to train candidate models, grouped by ground trunc domain. (Top) Cumulative probability- density distribution. (Botiom) Probability-densii distri- bution for the same data, using a score bin size of 0.05.
-1.0
-0.5
0.0
0.5
2.5
3.0
class-normalized VoG score
Figure II: VoG scores for 10.9 million de-identified historical data instances downsampled to train candidate models.
VoG scores appear to capture more than just the influence of a domain's data related to the size that domain's 's data, but also align with the intuition that training data associated with domains that have more complex domain definitions provide more of challenge for the model to predict correctly, and that training on these challenging examples is likely influence the learned model parameters than training on easy examples would. For example, while HomeAutomation and HealthAndFinees training data similar in intent- label diversity (2.3 bits of entropy for both), they differ greatly in training-data representation (12% vs. < <1%). Intuitively, we would expect smaller do- mains such as HealthAndFitness to exhibit higher average VoG scores than larger domains such as HomeAutomation, which we indeed find. The me- dian normalzed VoG score for Home tion was -0.31 compared to a median of -0.22 for HealthAndFiiness)) As shown in the top of Fig. 4, larger proportion of Humemutomation training instances were associated with negative VoG scores than for HealthAndFitness (roughly 80% vs. 60%).
0.25-
live training data
original
0.20
retained after pruning
prob. density
0.15
0.10
0.05
0.00
1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
dolaset--ormmalized VoG Score
Figure 12: Datasct-normalize VoG scores in the orig- inal training data vs. for the -52%-reduced subset ob- tained via VoG-score-bused sampling. VoG scores for the reduced-size subset have been re-normalized ac- cording the mean and std of the reduced-size subset scores.
Fig. 11 shows the distribution of VoG scores for the subset of historical training data used to train the statistical models, which comprises a majority of the overall training data. Compared to the SNLI VoG distribution in Fig. 2, the VoG distribution of the historical data looks roughly similar, but has more power in the low-scoring, "easy" bins. While in historical data 72% of dlas-normalized scores were less than 0, for SNLI only 66% of scores were less than O.
While Shopping and Video constitute similar proportions of the training data (6% 5%), the intent-label distribution for Shopping exhibits greater diversity than the intent-label distribution for Video (2.5 vs. 1.6 bits), indicating more com- plex and difficult prediction task. This difference in intent diversity appears to be reflected class- normalized VoG scores; Shopping scores tend to
Fig. 12 shows the distribution of data re- tained using dataset-normalizer VoG scores. Fig- ure 13 shows the VoG distribution using dataset-

---

be higher (more positive) than Video scores (me- dian of -0.30 in Shopping vs. -0.39 in Video). As shown in Fig. 4, VoG scores for Video data were more densely located in the low-scoring, "easy" re- gion; 37% of Video vs. 32% of Shopping data were associated with VoG scores below -0.5.
Domain
Median
Mean
Std
Health & Fitness
1.17
1.52
1.63
Video
0.08
0.86
2.00
Knowledge
-0.12
0.11
0.83
Shopping
-0.13
0.15
0.93
Music
-0.34
0.10
1.27
In some cases, it can be difficult to reason about the relative influence of one domain's ss data on trained model compared another domain's 's data. For example, in our real-world setting, the Mu- sic and Video domains capture similar NLU func- tionality (media playback) and are associated with comparable intent-label entropy estimates (1.5 vs. 1.6 bits). Unlike for HealthAndl'itness both Mu- sic and Video out-represent the majority of other domains in the training data (Music is the second largest domain 18%, while Video is the eighth largest 6%). In this case, which domain provides redundant or extraneous data not needed in order to maintain model performance? Appeals to intuition may fail us here, but VoG scores can still be helpful. As shown in Fig. 4, we find that Video training data contains a larger proportion of low-influence data instances than Music (37% of Video vs. 2% of Mu- sic of VoG scores were less than -0.5), potentially signaling the existence of redundant duplicate Video training instances.
Notifications
-0.44
-0.29
0.54
Home Automation
-0.45
-0.24
0.69
Table 7: Dataset-normalized VoG scores by domain, for subset of domains in in-house training data. Across all domains, the mean score 0 and the standard deviation I.
In our exper- iments on de-identified historical data, increased representation of smaller domains when sampling by normaee-d scores translated im- proved DC and IC recall. For HealthAndFitness, dataset-norm VoG sampling was associated with ADCER of 4% vs. 29% for class-norm sampling. A primary contributor to improved DC and IC re- call were improvements in Video, which under dataset-normalized sampling increased in training- data representation by relative 43% but under class- normalized sampling decreased representation by relative 1%
Experiments on Historical Data:
VoG-score summary statistics for de- de-identified historical training data used to train the candidate model are shown in Table 6 (normalized by class) and in Table 7 (by dataset).
For domains such as Automatonn and No- tifications that decreased in representation when sampling by dataset-norm scores, models trained on dataset-normalized VoG scores were associated with improved DC performance and comparable downstream NLU performance compared to class- norm models where those domain's ''s training-daie representation actually increased.
Domain
Median
Mean
Std
Health & Fitness
-0.22
0.0
1.0
Notifications
-0.27
0.0
1.0
Knowledge
-0.28
0.0
1.0
User Study: We analyzed the per-domain results to understand which domains/intents contributed to the observed top-level UER and PDR relative changes (Table 9).
Shopping -0.30
0.0
1.0
Home Automation
-0.31
0.0
1.0
Music
-0.35
0.0
1.0
Video
-0.39
0.0
1.0
A primary contributor the observed top-level UER improvement were Video-related requests. Video UER decreased by relative -1 11.6%. Post- experiment deep dives show that for the VoG model requests that previously were classified as Video were now classified Music or Knowledge. We saw that Music traffic slightly increased (rel. +1%) without an associated increase in Music UER, sug- gesting the majority of requests newly interpreted as Music the VoG model could by served by the Music domain. On the other hand, Music PDR increased by relative 2.6%, which was a primary
Table Class-norm VoG scores by domain, subset of domains in in-house training data. Within each class (domain), the mean score is O and the standard deviation is 1.
C.5 Dewein-Level Analysis of In-Houee House Experiments
Per-domain offline results from the user study ex- periment are shown in Table 8. Per-domain user study results are shown in Table 9.

---

Dotaset-norm
Class-normi
Domain
Atrain
ADCER
AFSEMER
AICER
AIRER
Atrain
ADCER
AFSEMER
AICER
AIRER
Music
0%
4%
-3%
1%
-3%
5%
5%
Knowledge
27%
12%
4%
12%
<0%
6%
Vidco
43%
-1%
2%
1%
1%
-1%
9%
4%
7%
4%
Shopping
18%
-11%
4%
-8%
4%
-5%
-3%
6%
-1%
5%
Notifications
-37%
-2%
2%
2%
2%
-7%
22%
2%
0%
1%
HomcAutomation
-29%
-6%
1%
-3%
2%
-16%
4%
-8%
3%
HealthAndFFiness
84%
4%
3%
6%
-6%
29%
35%
Table 8: Comparison per-domain offline-test relative-error-rate metrics for models trained on historical data, sampled according nommaioomllized dataset-normalized VoG scores. Results were averaged over three training
runs.
Domain
Atraffic
AUER
APDR
PDR (rel. +3.0%).
Music
1.0%
2.1%
2.6%
C.6 Value-Value-Bi Trail and Intent-Label Entropy
Knowledge
5.3%
0.6%
0.7%
Video
9.0%
-11.6%
1.9%
In the experiment on in-house data, VoG scores were measured with respect to DC model gradients, which can overlook training utterances that are in- fluential especially for training IC-NER models. In order surface training utterances influential for not only DC model but also IC-NER model training, we combined DC Vog scores with slot-label-tre entropy estimates, which provide a coarse coarse-grained grained intent-level estimate of NER difficulty.
Shopping -2.6%
12.8%
4.0%
Notifications
1.2%
6.6%
3.0%
HomeAutomation -111%
-5.0%
-2.9%
Table 9: Relative change to UER, PDR, and domain traf- fic for subset of domains in the the user study, compar- ing baseline model to a version of model trained only 50% of historical training data. Model results reflect metrics for the full NL.U system comprised of statistical non-statin sta-scatiiiaal model components. These relative-change results were not compared using hypoth- esis tests and may not be statistically significant.
As an example, consider the customer re- quest "play music by Howard Shore please", which has the annotation "Play|Other music| by 10ther [ooard]AAtistMoe Shore ArtistName Alease/Other" We define the slot-label-trail as the sequence of slots labels and non-Other slot-label values for given anno- tation. The slot-label-trail for this example would be "Other Other Other Howard ArtistName Shore | ArtistName Other". The frequency each such slot-label-trait found in the training data is used to compute Shannon entropy estimates for each intent, given by:
contributor the observed increase in top-level PDR. The simultaneous decrease in Music UER but increase in Music PDR suggests that Music domain could provide some kind of interpretation for request, but that overall those interpretations were associated with slight increase in defect rate (e.g. defects related to searching for the wrong artist name, or searching for album rather than artist).
The user-study impact on VoG-based pruning on the performance of individual domains was not solely determined by whether or not a given do- main's 's representation was increased/decreesee in training data. Both Home-uutomation and Notifi- cations increased in their training-date data representa- tion (Figure 5), however observed opposite- direction impacts in metrics for those domains. Home-Automation was associated with reduced traf- fie (rel -1.1%), UER (rel. -5.0%), and PDR (- 2.9%); whereas Notifications was associated with increased traffic (rel. 1.2%), UER (rel. 6.6%) and
Hintene =- E [log, P(t)], toP
(8)
where P(t) is based on label-trail frequencies found in training data, and T is all label trails in a given intent. Eq. 8 is the formula for com- puting the Shannon entropy of the distribution of laaitttioo--abl-eeeeeee
trails in the training data and is computed separately for each domain-intent combination. Since the log is computing using base 2, resulting entropy estimates are measured units of bits.
The intent-label entropy can be calculated at

---

domain level in similar manner. These results summarized in Table 10.
Domain
Intent-label catropy (bits)
Stot-trai entropy (bils)
Music
1.5
14.8
Knowledge
14.5
Video
Shopping
2,5
12.9
Netifications
10.2
Automatioo
Health & Fitness
2.3
6.1
Table 10: Domain-level intent-label entropy calculations and intent-level slot-label-trail, for subset of domains in historical data. For intent-level entropy, the weighted average across intents is reported per domain.
We used the same approach to calculating sam- pling scores from entropy estimates as we did the case of converting VoG scores to sampling scores. For a given utterance, the final sampling score was the average of the VoG-based and sampling scores. weight. In offline tests, we found that this modification to sampling scores helped slightly improve composite model performance as measured by SEMER (rel. -1.39%) and IRER (rel. -1.5%), which was reflected in per-domain im- provements for the largest- Music and Video intents. This came at the expense of a slight in- crease in Shopping slotting errors on Shopping offline tests (rel. +6).%)

---

Fremise
Hypolheeis
Gold Label
Score Measurement
YoG (clas-nurmalize)) Tail (hardest)
An African woman standing near well filling a neur hut
A woman fesching water from well beside an African Hut with
1ol
with thusched roof.
struw roof.
A man black is applanding runner weaning a red jenee and the number
The man running
lel
People siting down by a strcam of water
They are enjoying the strcam outdoces
lof
colebrating big win he opened botlle
man is celebrating sports team win alcohol.
squinted out all in the air.
skier skinng stoey slope.
person onjoying gravity
friction
1o4
velocity down an incline.
beurded
black and black cap stands young
The mun does
shuve.
1s4
girl who is inside large brown open containcr.
A in blue sweater sits person's shoulders and carrics piowheel.
girl enjoying with topy
shoulders
lel
girl in wilkk shin plays pool
wommnn grean skirt in her.
Tol
A group three people pose for picture with smile while mnother
Three guys are stulling while fourth running away.
l64
sits aside without smillng.
man working on tickel
two women stand
man is giving out tickets.
VoG mormlized Head (easiest)
black and gray walks sidloailk fence and a row of hedges. Two women singing stage while in plays
-0,8801
The three people riding skis
C
-0,880
guilar.
A conciruction crew orange wests working train tracks.
The crew is cating dinncr scaside restaurant.
C
-0,880
woman wearing a black helmet niiig on bike
man is sleeping alons in bod.
C
-0,880
black and white dog with spotted face running through dirt fiel..
dog sleeping his bed by the firsplace.
C
-0.880
Different people are walking sidewalk, in both directions, in
People are siting on a couch watching ev.
C
-0,879
front orange canopy.
Four dogs stand in the snow.
cal is sleeping on the fridge.
C
-0.880
male is standing on a pitching ball.
woman singing shower.
-0,880
A backpacker points to the snowcapped mountains as be stands on mick plain.
woman laying in her hospital bed.
C
-0,880
A woman in welsuit in surfing in the occon.
A woman skeeping in her bed home.
C
-0.879
YoG (datusct-normallzed) Tail (hardest)
MMA fighter in whire shirt with Black and white shorts practices a kick the gym.
An MMA lighter is using punching bag.
C
24.921
Glossy red apples lay stacked a metal bowl. A woman and girl are sitting tile flour behind a wonden rack weaving.
There are a bunch of pears on a plate.
C
26.670
The indy and her duughter are riding honeluck
C
41,494
An elderly whise man playing mumper into microphone.
An elderfy man plays yellow drums.
C
25,417
Two people with hats orunge vesis are working.
The people with the
yellow vests on well.
48.367
Two women are drinking from yellow cups and laughing.
The two women in the comer are drinking from red cups and laughing.
45.679
A group of people, of all ages. listening to concert being per. fornee
single muscian entettiits divense andience
E
36,737
practitioner
teongger is weating gray hooded top and some red beads arcund
The teenager is wearing rod beaded neckhace.
E
32/429
nock.
"The cat is squinting
There are cats playing with yarn.
C
28.089
boy skaseboard grandin down handrail.
boy does bike trik park.
C
27.373
VoG (dataset- normalized) Head (caslest)
A boy in comonllage ponts and his ball lying in fromt a blue An asinn child sits on the ground cating from bowl outside a hut sunny day.
boy byyng on the sidowall nest the car.
-0,714
An asian child sits on the ground cating from bowl inside the but
-0.14
Two men are using two horse to help with furm work.
The men are larm
40.14
Three people skydiving, one of them is sticking there longue out
The people skydlving are scared and shaking.
00.14
making a silly
A zirl in blue sweater sits on person's shoulders and carrics a pinwheel.
girl enjoying with topy on a person's 's shoulders
-0,714
Young boy with long blond-hair running with large houncy hall
The bay is playing a game with the other two people.
40714
had two people playing piggyback chising him.
Min is demonstating a student.
Man showing student how paint
0.714
wearing helmet riding his bike down mikk terrain
chasing after The adults are by the beach.
00714
Three young adulis building a large sand cante at the beach.
.0714
girl monkey
She climbed up.
.0.14
Table 1I:
Shows the highest and lowest scoring VoG examples from SNLI Gold labels are denoted by (C)ontradiction, (E)entailment, (N)eutral, or which indicates that annotators could not come to consensus on the label. A numerical cutoff of 10$ was used to truncate high VoG scores.

---

Premise
Hypothesis
Labe Measurement
EL2N Tail (hardest)
two teams playing rugby. there seems to be more of one team than the other in the picture.
Two teams are sitting inside watching movie.
1.412196
A group of men
trash bags across
Two women swimming ocean.
1.412198
beach, nearby pickup muck and balldozer.
Children playing on merry-go-tomme on a chilly The children are sleeping
1,412257
day.
A man is working laptop computer in open A man surting the web the couch. cafe.
1,412444
Two men are practicing a dance routine while their Two mon are sleep
1,412286
Iriead captures camera.
A man with red beard pushing shopping cart A dog sits inside shopping cart. on busy street.
1,412446
A person doing jump their snowboard over
A person is sleeping
1.416697
orange and white coution sign.
A girl in pink stands in from of Mud coffiee truck.
A group of boys playing marbles.
N
1.412732
Three ice skaters round curner.
The skaters are sleeping.
1.412452
A
Lemmal are at table
Two cais lable.
1.412773
EL,2N Head (easiest)
brown and white dog jumping over red, yellow and white pole
cal sleeps pillow
0.000666
A dog with a
running lield.
Cals sleeping on porch.
0,000657
A dog runs lield with its mouth open.
Two cals sleep in bed.
0,000600
A large brown dog is running Uhrough a grass
A cal sleeps inside.
0.0006646
hackyard.
A brown dog croaching and looking up field of grass.
is inside slooping
0.000630
Two dags are fighting over a red Frishee outside. A buy wearing a red sweater runs along a colorful beach.
Two cats sleep on shelf.
0.000610
A girl sleeps couch in TV.
0.0006600
Two dogs playing with small blue bull in a grassy twa cals are sleeping inside.
0.000663
Two dogs field brown grass.
cats sitting couch A cat sits in the living room.
0.0006665
A beow and white dog runs through a grassy area.
0.000668
Table 12: Shows the highest and lowest scoring EL2N examples from SNLL
Premise
Hypothesis
Gold Label
Score Measurement
PVI Tail
A lage crowd of people are outside und a big sign
The crowd marches in front of the sign.
3.094221
"AIDS K" in the background among
Girl hangs up midair by two hungee cords. A blond woman tips a young blond girl upside- down.
A GIRL. is SWINGING FROM A ROPE.
3,095381
The woman holding the girl upside down over the pol.
3.180583
A man gathering hay a borse drawn wagon. A rugby player wearing white short and shirt being tackled by another player wearing blue short and
Three borses are pulling wagon covered bay
3,120966
A rugby player wearing white shoet and shirt being tackled by another player wearing blue shorts and tree trunks
3.123245
trunks; teammaate of he player being tackled is coming up behind the tackled player.
The young buy sleds down the bill in the snow. A mun in a older woman in white jacket playing crocher.
A boy at the top of snow covered bill.
3,183291
juckel sanding
of an
man was playing crochel with two women.
3.322716
A croad of people at market near highway.
There are people the marker
3.187976
A untiquated soldier stands in salute holding a rifle.
A statue of soldier with a rifle.
3.221569
Five people on their yard with of them
Two people are climbing tree behind three other people.
3,283761
climbing a ladder to tree in the background.
PVI Head (potentially ambignous or misnnnotated)
A group of people sitring oundoors. Three dogs, black one brown, are playing
The people inside.
-11,263250
Three cats, two black brown, are playing in a grass field.
-13.6309976
grass field.
Lacrosse players struggling for control the hall. A bunch people sitting outside a building at night.
Nohody is control the ball.
12.107733
dogs were fighting get bone each other
14.055565
A male and female are at table with drink.
Two cats are a table.
12.030568
A group of men in a blue our driving on the track.
One woman is uriving the blue car.
.11.397809
A ina jumps into u walkway.
set up on public
lemale leaps bed public walkway.
-11.016566
People a marketplace selling watermelons
Dogs at a marketplace selling watermelons.
1106557009
Three people are using a net on beach.
Three people are sitting on a bench.
-11 0000006611
A little boy sits on the tail of fake alligator.
The sitting on a fake alligator.
-10.811694
Table 13: Shows the highest and lowest scoring PVI examples from SNLI

---

Premise
Hypothesis
Gold Label
Score Measurement
Forget Scores Tail (hardest)
A man with white shirt. hrown shoes, and blae The man is fully clothed. weading books.
6.67
people piles
The woman is sitting in the bost being rowed by The woman sitting on the beach.
6.67
A lady wheelchair the corner of
There people waiting street. barefoot by
6.67
and man in white shirt about to cross the strect.
carrying the beach.
walking barefoot
6.67
Two mon sit next to each other on the water a The boat is floating.
6.67
Todiler wearing sirplane shirt, stroller with
The toddler riding stroller.
7.0
large beads.
A woman with glasses is looking at a gray laptop
Three people are having lunch an upscale restaurant.
7.33
with two other people table yellow
Lady wearing green work helmet and holding bag.
The woman wearing straw hat.
7.0
women Scottish garb play happipes
People traditional clothes and play hagpipes
runner appronches the street.
This man fir and well toned running enthusiast.
This man likes to run.
7.67
Forget Scores Head (casiest)
with black typing laptop
computer.
computer.
A woman wearing a blue shirt typing on a laptop.
Somsone typing.
0.0
black walking along pier.
woman outside.
0.0
black walking along pier.
walking A typing in the computer.
A wernn with black hair lyping laptop computer.
0.0
married woman
types.
single moiher run.
A with black hair and jewelry her left hand and arm typing keyboardd
A woman hair.
0.0
In white walled gymmasium, shirlless male handstand the parallel
In the school's gymnasium, gymnast performs a handstand that perfect by the rating him.
0.0
gymnast
another man watches him the background.
A married woman in blue and black rypes.
A woman works on her novel. woman wearing a ryping on laptop. Someone replacing battery on laptop.
0.0
Table 14: Shows the least and most forgotten examples from SNLI.

---

Hypothesis
Measnrement
TraeIn Scores Tail (hurdest)
Three people playing rock music, one of them is wearing a hat and a black pants with black 1-shirt singing playing guilar.
Out of three people playing music, one with black shirt and playing piano.
3.30
A lady giving presentation all femaae class.
There
the
3.30
A man gray shirt drilling into silver can. people stage
A lady drilling a can. human two boys ride skateboard
3.32
buckground.
grvund.
Three lemale bikers travel swillyy through school zone.
4.20
object looking young girls that sit from him table that is outside.
holding object, not looking
Two tecnage girls, one smiling. Group of A guy bike verticul namp with grasey, hilly remain hehind him.
A girl not smiling. There
3.62
business meeting.
business meeting
3.63
A mam inside.
3.51
sky appears clear.
nothing
sky
TracIn Scores Head (easlest)
A bicyelist rounds tarn, followed by a camera- A man is driving car around the roundabout man.
0.00
A young his bike.
bicycling jumping up
man cating the park.
0.00
The racing dog has a muzele and weating striped The dog sleep ouisside.
0.00
jersey &.
sparing.
process laking
Nobedy sparing.
0.00
down the other.
tennis player hits the ball.
A tennis player swimming across the ocean.
0.00
holding two loe cream cones
The person is sitting in car.
0.00
walks through the park.
A man with big backpack walking through grass trail up hill.
A woman with big backpuck is walking through grass trail
0.00
yellow jumpesuit working. A construction worker walks down the street, while others are at work.
Nobody working.
A construction worker silting his
0,00
wearing protective gear including met, jumping rock.
paralyzed hospital.
Table 15: Shows the highest and lowest TracIn examples from SNLI.


Reviews:
[{"review_id": "27ypf3OuFY", "reviewer": "Reviewer_g8MK", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the review and constructive comments!\n  \nIndeed, the standard approach to pre-training large language models involves training on all available data, which is simple though costly. However, there are formidable data-selection challenges for language models that influence scores can help to address, such as using scores to flag data most influential for various failure modes (e.g., toxic and biased response generation). Other practical application challenges that influence scores could be useful for include: assessing the influence of specific training data examples on the response stability with respect to prompt modifications; assessing the model-performance impact of copyrighted material in the training data; measuring the influence of sensitive private information present in big data lakes on model output; and quantifying the potential of model output surfacing private customer data that has been memorized by that model (we note this work by Grosse et al. along similar lines: [https://arxiv.org/abs/2308.03296](https://arxiv.org/abs/2308.03296)). We plan to explicitly incorporate these motivations in a paper revision, as well as a technical discussion on fruitful directions we see in applying influence scoring to LLMs.\n  \nIn addition, while LLMs are groundbreaking, many real-life NLU systems, including voice assistants and fast response/detection systems, require task understanding, latency and precision stability for which LLMs alone may not be an appropriate architectural choice. For example, low-resource languages may necessitate more classical model architectures, for which careful data selection becomes paramount. In other cases, standard LLM training approaches may be too costly or resource intensive, leading to the need for modeling approaches that are trained on specific, high-quality subsets of data (see, e.g., Gunasekar et al., [https://arxiv.org/abs/2306.11644](https://arxiv.org/abs/2306.11644)).  \n  \nWe share your concern about the need for realistic evaluation of influence-scoring data selection approaches we experimented with. This is why we also conducted a large-scale user study evaluating our techniques in an uncontrolled speech online A/B setting. Statistical analysis on the results of this large-scale user study allowed us to empirically validate the impact of the explored influence-scoring techniques on real-life, unconstrained data that reflect practical challenges often not reflected in public datasets, such as noisy and evolving speech patterns, diverse vocabularies, dialects, carrier phrases, and named entity frequencies that deviate from what was provided to a model at training time. Even under these challenging evaluation conditions and under aggressive training-data reduction we still saw the utility of influence scores for data pruning. We will enhance the discussion to make this scaled user study more prominent as well.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is generally well-written, and clear in terms of what it tries to investigate (problem formulation) and methodology. The study (on SNLI) looks decent and it provides some potentially useful insights to the community.", "reasons_to_reject": "I am writing questions and reasons to reject together below:\n\n1) In line 037-039: \"A common challenge in fine-tuning transformer-based models is selecting which examples are most important for learning\"---is it really a challenge anymore? Cheery picking examples might be infeasible when it comes to training larger models such as LLMs.\n\n2) One of the concerns I have with this paper is its impact, the study has been carried out in a controlled setting (SNLI dataset, etc.) and thus inferences made may not generalize well for a wider range of problems (datasets and experimental settings).", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper explores the applicability of training dataset pruning via (five) influence scores in language classification tasks. It tries to find answers to several questions such as how useful the influence scores are for determining optimal data selection strategies, which scores work best, and evaluating these scores against a random sampling baseline in noisy and clean data settings.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HIqXJ4JFNI", "reviewer": "Reviewer_ABYD", "summary": "", "questions": "A: Can we extend this technique to selecting which in-context examples are most likely to give the best performance?\nB: is it possible to come up with some variants of influence scores to decide which parts of the pretraining corpus is important? the problem here  would be that we don’t have any labels.\nC: Are these findings transferable to multilingual models?\nD: How sensitive are influence scores to pretraining schemes and pretraining corpora?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments and positive feedback!  \n\nIt would indeed be ideal to train on as much clean data as possible. However, problems arise not only due to raising training costs, but also because of mounting privacy and intellectual property concerns, customer data deletion requests, capacity constraints of human annotators, and governmental legislation that sets data retention policies (which can lead to very uneven data losses across classes when data expires), or legislation that defaults customers to opt-out of data submission (creating group biases in the available training data). Our goal was, in part, to understand how empirical data-scoring metrics could be used to surface, quantify, diagnose, and ultimately mitigate these risks. For example, given the risk of losing a certain percent of training data due to an data-expiration policy, it is paramount to develop empirical metrics that can surface the most critical and impactful utterances to expose annotators to.  \n  \nSecond, we note that we chose a very aggressive target of a 50% reduction in training data in our online experiments. A regression on less common requests is indeed a cause for concern, but we aimed to highlight the efficacy of our method on overall metrics in a fairly extreme “proof of concept” case. In practical applications, we believe that our method would perform better via iterative instrumentation with gradual reductions and iterative scoring of training data, which is supported by results from the 14%-reduction experiment.  \n  \nRegarding your questions: \n  \nA: We agree that applying this technique to in-context learning would be an exciting next step! We will include further discussion on implementing VoG scores contextually in LLMs/decoder-only architecture in our camera-ready version, but note that one potential approach would be to look at variance of model weights (e.g., in attention heads) in specific layers across the input sequence. This could offer an interpretable metric for pinpointing influential contextual examples, at the level of specific tokens (see, e.g., Grosse et al. [https://arxiv.org/abs/2308.03296](https://arxiv.org/abs/2308.03296) along similar lines). Given rising concerns over bias, toxicity, and fairness in language models , we believe there is a critical need for straightforward, cost-effective metrics that can estimate in-context influence. Our work develops the foundational understanding necessary to make progress on that problem by generalizing results from the computer vision field (such as those scores that approximate more computationally expensive influence functions) to language-based tasks at scale for the first time.\n\nB: Adapting these scores to model pretraining through a self-supervised setup would be an interesting line of investigation (for example, assessing the impact of removal of copyrighted pretraining data). One approach would be to compute an ensemble of the VoG score over all of the model outputs, rather than the output at a specific ground truth \"label\". However, applying this to an entire corpus of pretraining text would be computationally infeasible, so some degree of coarse-graining or clustering would be required to obtain representative samples from different categories (such as Books, Wikipedia, etc.) as estimators for the influence of that category.\n\nC: We believe these findings should generalize to multilingual setups, since the only ingredients are training artifacts (e.g., gradients, losses, etc.) that should, in principle, convey the relevant signal to determine data influence regardless of the specifics of the overall training data distribution. That said, most empirical influence scores do reflect some information about the model inductive bias and some component of the training-data distribution. For certain multilingual setups where training artifacts used to produce scores were derived from training data differing greatly from the to-be-scored data, the mismatch in these two sets of data may preclude or reduce the usefulness of the resulting scores for some downstream applications.\n\nD: While we cannot definitively answer without additional investigation, we can comment that it is challenging to assess the quality and nature of pretraining during finetuning (when we compute these scores). In this sense, we don't expect the efficacy of influence scores to be sensitive to the precise details of pretraining. However, there is a component of inductive bias reflected in these scores (for instance, in appendix B, we see a shift in accuracy and pruning curves resulting in the change in inductive bias coming from increasing model capacity.) It would be interesting to make this precise in future work — how much of dataset difficulty comes from pretraining and/or model inductive bias vs. information extracted about the data distribution?", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Clearly written and easy to read.\n* Extensive set of experiments with a strong focus on using the techniques in production .\n* Validates empirical results comprehensively for applying techniques to the system in production with both offline and online A/B tests.\n* These influence score techniques would allow practitioners to identify the subset of the training data that is most important to performance and annotate more samples the belong to such categories. ", "reasons_to_reject": "No major reasons to reject, but I have some general concerns about the techniques discussed.\n\n* This seems to solve the problem of deciding which examples are important in hindsight which concerns applicability of the technique to a system in production. When even small % differences in downstream performance can have material impact on company revenue, would they not prefer to train models on as much data as possible? Identifying which samples are easy or hard once they have been labelled can help determine which type of examples we need more annotations of, but the results showing regression on less common requests is concerning for people who make the decision to deploy the model into production.", "questions_to_authors": "A: Can we extend this technique to selecting which in-context examples are most likely to give the best performance?\nB: is it possible to come up with some variants of influence scores to decide which parts of the pretraining corpus is important? the problem here  would be that we don’t have any labels.\nC: Are these findings transferable to multilingual models?\nD: How sensitive are influence scores to pretraining schemes and pretraining corpora?", "paper_topic_and_main_contributions": "The main contributions of the paper are:\n\n* Systematically benchmarked the applicability of a set of influence score techniques commonly applied in CV to an NLP task(SNLI).\n* Benchmarking a subset of influence scores for data pruning on the SNLI dataset, and finding that out of the scores tested, VoG (variance of gradients) performed best compared to a random sampling baseline. \n* Applying VoG to a NLU system in production and showing that training on a subset of data selected by the VoG scores did not lead to regression on key metrics.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0xNXNtE1gL", "reviewer": "Reviewer_CqZv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your comments and feedback!  \n\nA primary goal of our paper was to build upon existing influence-scoring literature in computer vision by applying influence scores to core NLU tasks for the first time. Hence we focus our evaluations on inherently different types of influence scores, for the purpose of selecting training data in a large-scale, real-life, language-based application. We believe this fills a gap in current influence-scoring literature, provides useful data points to practitioners, and can serve as a foundation for future experimentation.  \n  \nOur conclusions about the utility of influence scores for language data sampling were based on both SNLI evaluations as well as performance evaluations in a large-scale, online A/B user study, where downstream tasks included not only domain classification but also intent and slotting performance for a wide variety of NLU-related tasks. The scaled user study exposes the model to unconstrained human speech, which varies (often dramatically) in carrier phrase frequency, vocabulary, named entities distribution and other aspects from publicly available datasets such as SNLI. We dedicate a separate section to the real-life user study and showcase how influence-score based data selection can be effective even in an aggressive scenario of 50% reduction of training data, without loss of net accuracy.  \n  \nTo the best of our knowledge, our work is the first to formally investigate influence scores at scale for language data and model architectures. We are excited about the potential for the research community to build upon our findings and extend experiments to new types of model architectures or language tasks. We will include specific recommendations in the paper discussion on how to expand usage into LLMs/decoder-only architectures to estimate influence of contextual examples.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well-written and easy to follow.\n- The authors provide extensive experimental results on influence scores, thereby contributing a valuable resource to the NLP community.", "reasons_to_reject": "My primary concern revolves around the major conclusion of the paper, which appears to rely solely on the performance of BERT encoder on the SNLI dataset. This limited scope raises questions about the generalization of the findings to other datasets or tasks.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents a evaluation of influence scores for data reduction in language classification tasks. The authors benchmark five easy-to-implement influence scores on the Natural Language Inference (NLI) task, considering both noisy and clean data settings. Among the evaluated influence scores, they find that VoG performs best relative to the random-pruning baseline. Building upon these findings, the authors scale VoG to a real-world setting.\n\n\n-------------Post-rebuttal---------------\n\nI have read the author's response, and I think they have addressed my concerns. Therefore, I will improve my score.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper focuses on the use of influence scores to select important examples for fine-tuning transformer-based models. There is a consensus on the clarity and readability of the paper. The paper is well-written and easy to follow. Reviewers also agree that the paper receives praise for its extensive set of experiments, which includes both offline and online A/B tests, and using the techniques in a production environment is a strong aspect of the study. This practical orientation is seen as valuable to the NLP community. Besides, the proposed influence score techniques are seen as potentially valuable for practitioners. They can help identify subsets of training data that are most important for model performance, facilitating better annotation strategies.

Although the reviewers do not have any major concerns, they question the relevance of the challenge mentioned in the paper's introduction, specifically regarding the selection of important examples for fine-tuning transformer-based models. They argue that this might not be a significant challenge anymore, particularly with larger models like LLMs. The paper should address this concern by providing more context and justification for this challenge. One reviewer expresses their concern about the limited scope of the study, as it focuses on a controlled setting using the SNLI dataset. They worry that the inferences drawn from this specific dataset may not generalize well to a wider range of problems, datasets, and experimental settings. The paper should acknowledge this limitation and discuss its implications.

In conclusion, the paper is commended for its clarity, experimental rigor, and potential utility for practitioners in the NLP community. However, it needs to address concerns related to the relevance of the challenge and the generalizability of its findings. Addressing these concerns and providing more context could strengthen the paper's overall contribution.
Decision: Accept-Main