Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics (2020)

Abstract: Large datasets have become commonplace in NLP research. However, the increased emphasis on data quantity has made it challenging to assess the quality of data. We introduce Data Maps—a model-based tool to characterize and diagnose datasets. We leverage a largely ignored source of information: the behavior of the model on individual instances during training (training dynamics) for building data maps. This yields two intuitive measures for each example—the model’s confidence in the true class, and the variability of this confidence across epochs—obtained in a single run of training. Experiments on four datasets show that these model-dependent measures reveal three distinct regions in the data map, each with pronounced characteristics. First, our data maps show the presence of “ambiguous” regions with respect to the model, which contribute the most towards out-of-distribution generalization. Second, the most populous regions in the data are “easy to learn” for the model, and play an important role in model optimization. Finally, data maps uncover a region with instances that the model finds “hard to learn”; these often correspond to labeling errors. Our results indicate that a shift in focus from quantity to quality of data could lead to robust models and improved out-of-distribution generalization.

## introduction the creation of large labeled datasets has fueled the advance of ai @xcite and nlp in particular @xcite . the common belief is that the more abundant the labeled data, the higher the likelihood of learning diverse phenomena, which in turn leads to models that generalize well. in practice, however, out-of-distribution * work done at the allen institute for ai. all definitions are with respect to the roberta-large model. the modal group in the data is formed by the easy-to-learn regions. for clarity we only plot 25k random samples from the snli train set. fig. 8b in app. §c shows the same map in greater relief. drawing analogies from cartography, we propose to find coordinates for instances within the broader trends of a dataset. we introduce data maps: a model-based tool for contextualizing examples in a dataset. we construct coordinates for data maps by leveraging training dynamics-the behavior of a model as training progresses. we consider the mean and standard deviation of the gold label probabilities, predicted for each example across training epochs; these are referred to as confidence and variability, respectively ( §2). fig. 1 shows the data map for the snli dataset @xcite constructed using the roberta-large model @xcite . the map reveals three distinct regions in the dataset: a region with instances whose true class probabilities fluctuate frequently during training (high variability), and are hence ambiguous for the model; a region with easy-to-learn instances that the model predicts correctly and consistently (high confidence, low variability); and a region with hard-to-learn instances with low confidence, low variability, many of which we find are mislabeled during annotation . 1 similar regions are observed across three other datasets: multinli @xcite , winogrande @xcite and squad @xcite , with respect to respective roberta-large classifiers. we further investigate the above regions by training models exclusively on examples from each region ( §3). training on ambiguous instances promotes generalization to ood test sets, with little or no effect on in-distribution (id) performance. 2 our data maps also reveal that datasets contain a majority of easy-to-learn instances, which are not as critical for id or ood performance, but without any such instances, training could fail to converge ( §4). in §5, we show that hard-to-learn instances frequently correspond to labeling errors. lastly, we discuss connections between our measures and uncertainty measures ( §6). our findings indicate that data maps could serve as effective tools to diagnose large datasets, at the reasonable cost of training a model on them. locating different regions within the data might pave the way for constructing higher quality datasets., and ultimately models that generalize better. our code and higher resolution visualizations are publicly available. 3 ## mapping datasets with training dynamics our goal is to construct data maps for datasets to help visualize a dataset with respect to a model, as well as understand the contributions of different groups of instances towards that model's learning. intuitively, instances that a model always predicts correctly are different from those it almost never does, or those on which it vacillates. for building such maps, each instance in the dataset must be contextualized in the larger set. we consider one contextualization approach, based on statistics arising from the behavior of the training procedure across time, or the "training dynamics". we formally define our notations ( §2.1) and describe our data maps ( §2.2). ## data selection using data maps data maps reveal distinct regions in datasets; it is natural to wonder what roles do instances from different regions play in learning and generalization. we answer this empirically by training models exclusively on instances selected from distinct regions, followed by standard in-distribution (id), as well as out-of-distribution (ood) evaluation. our strategy is straightforward-we train the model from scratch on a subset of the training data selected by ranking instances based on the different training dynamics measures. 7 we hypothesize that ambiguous and hard-to-learn regions could be the most informative for learning, since these examples are the most challenging for the model @xcite . we compare these two settings to roberta-large models trained on data subsets, selected using several other methods. all subsets considered contain 33% of the training data (to control for the effect of train data size on performance). baselines the two most natural baselines are those where all data is used (100% train), and where a 33% random sample is used (random). our second set of baselines considers subsets which are the most easy-to-learn for the model (highconfidence), and those that the model is most decisive about (low-variability), which comprises a mixture of easy-to-learn and hard-to-learn examples. we also consider baselines based on highand low-correctness. finally, we also compare with our implementation of the following methods for data selection from prior work (discussed in §7): forgetting @xcite , aflite (lebras et al., 2020), al-uncertainty @xcite , and al-greedyk @xcite . ## role of easy-to-learn instances data maps uncover ambiguous regions, small subsets from which lead to improved ood performance, with minimal degradation of id performance ( §3). we next investigate how performance is affected as we vary the size of the ambiguous subsets. we retrain our model with subsets containing the top 50%, 33%, 25%, 17%, 10%, 5% and 1% ambiguous instances of winogrande (fig. 3 , left and center). large ambiguous subsets (25% or more) result in high id and ood performance. surprisingly however, for smaller ambiguous subsets (17% or less), the model performs at chance level, despite random restarts. 11 in contrast, a baseline that randomly selects subsets of similar sizes is able to learn (while naturally performing worse as data decreases, eventually failing at 1%). this indicates that ambiguous instances alone might be insufficient for learning. given that the model barely struggles with easyto-learn instances (by definition), we next replace some ambiguous examples with easy-to-learn examples in the 17% most ambiguous subset. interestingly, replacing just a tenth of the ambiguous data with easy-to-learn instances, the model not only successfully learns, but also outperforms the random selection baseline's id performance (fig. 3 right). this indicates that for successful optimization, it is important to include easier-tolearn instances. however, with too many replacements, performance starts decreasing again; this trend was seen in the previous section with the high-confidence baseline (tab. 2). ood performance shows a similar trend, but matches or is worse than the baseline. selection of the optimal balance of easy-to-learn and ambiguous examples in low data regimes is an open problem; we defer this exploration to future work. ## detecting mislabeled examples crowdsourced datasets are often subject to noise attributed to incorrect labeled annotations @xcite @xcite , which may lead to training models that are not representative of the task at hand. recent studies have shown that over-parameterized neural networks can fit the incorrect labels blindly @xcite , which might hurt their generalization ability @xcite . for large datasets, identifying mislabeled examples can be prohibitively expensive. our data maps provide a semi-automated method to identify such mislabeled instances, without significantly more effort than simply training a model on the data. we hypothesize that hardto-learn examples-those with low confidencemight be mislabeled, as has also been suggested in prior work @xcite . to verify this hypothesis, we design an experimental setting where we artificially inject noise in the training data, by flipping the labels of 1% of the training data for winogrande. motivated by our qualitative analysis (tab 1), we select the candidates for flipping from the easy-to-learn regionthis minimizes the risk of selecting already mislabeled examples. we retrain roberta with the partly noised data, and recompute confidence and variability of all instances. fig. 4 shows the training dynamics measures, before and after re-training. flipped instances move to the lower confidence regions after retraining, with some movement towards higher variability. this indicates that perhaps the hard-to-learn region (low confidence) of the map contains other mislabeled instances. we next explore a simple method to automatically detect such instances. ## training dynamics as uncertainty measures we introduced data maps, and used training dynamics measures as coordinates for data points in §2. we now take a closer look at these measures, and find intuitive connections with measures of uncertainty. when a model fails to predict the correct label, the error may come from ambiguity inherent to the example (intrinsic uncertainty), but it may also come from the model's limitations (often referred to as model uncertainty). 14 to understand how examples contribute to a dataset, it is important to separate these two sources of error. we start by studying the relationship between intrinsic uncertainty and our training dynamics measures. human agreement can serve as a proxy for intrinsic uncertainty. we estimate human agreement using the multiple human annotations available in snli 's development set. 15 for each annotator, we compute whether they agree with the majority label from the other four, breaking ties randomly and then averaging over annotators. 16,17 fig. 5 visualizes the relation between our training dynamics measures (confidence and variability) and human agreement, averaged over the examples. we observe a strong relationship be13 in preliminary experiments, retraining winogrande after removal of noise did not yield a large difference in performance, given the relatively small amount of noise. these relations are further supported by previous work, which showed that deep ensembles provide well-calibrated uncertainty estimates @xcite @xcite . generally, such approaches ensemble models trained from scratch; while ensembles of training checkpoints lose some diversity @xcite , they offer a cheaper alternative capturing some of the benefits @xcite . future work will involve investigation of such alternatives for building data maps. ## related work our work builds data maps using training dynamics measures for scoring data instances. loss landscapes @xcite are similar to training dynamics, but also consider variables from the stochastic optimization algorithm. @xcite also use training dynamics to find train examples which are frequently "forgotten", i.e., mis-classified during a later epoch of training, despite being classified correctly earlier; our correctness metric provides similar discrete scores, and results in models with better performance. variants of such approaches address catastrophic forgetting, and are useful for analyzing data instances @xcite . prior work has proposed other criteria to score instances. aflite @xcite ) is an adversarial filtering algorithm which ranks instances based on their "predictability", i.e. the ability of simple linear classifiers to predict them correctly. while aflite, among others @xcite , advocate removing "easy" instances from the dataset, our work shows that easy-to-learn instances can be useful. similar intuitions have guided other work such as curriculum learning @xcite and self-paced learning @xcite where all examples are prioritized based on their "difficulty". other approaches have used training loss @xcite @xcite , confidence @xcite , and meta-learning @xcite , to differentiate instances within datasets. perhaps our measures are the closest to those from @xcite ; they propose prediction variance and threshold closeness-which correspond to variability and confidence, respectively. 18 however, they use these measures to reweight all instances, similar to sampling effective batches in online learning @xcite . our work, instead, does a hard selection for the purpose of studying different groups within data. our methods are also reminiscent of active learning methods @xcite @xcite , such as uncertainty sampling @xcite which selects (unlabeled) data points, which a model trained on a small labeled subset, has least confidence in, or predicts as farthest (in vector space, based on cosine similarity) @xcite . our approach uses labeled data for selection, similar to core-set selection approaches @xcite . active learning approaches could be used in conjunction with data maps to create better datasets, similar to approaches proposed in @xcite . for instance, creating datasets 18 they also consider confidence intervals; our preliminary experiments, with and without, yielded similar results. with more ambiguous examples (with respect to a given model) could make it beneficial for ood generalization. data error detection also involves instance scoring. influence functions (koh and liang, 2017), forgetting events @xcite , cross validation @xcite , shapely values @xcite , and the area-under-margin metric @xcite have all been used to identify mislabeled examples. some approaches avoid hard examples altogether @xcite to reduce fit to noisy data. our use of training dynamics to locate mislabeled examples involves minimal additional effort beyond training a model on the dataset. ## conclusion we presented data maps: an automatic method to visualize and diagnose large datasets using training dynamics. our data maps for four different datasets reveal similar terrains in each dataset: groups of ambiguous instances useful for high performance, easy-to-learn instances which aid optimization, and hard-to-learn instances which often correspond to data errors. while our maps are based on roberta-large, the methods to build them are model-agnostic (app. §c.1). our work shows the effectiveness of simple training dynamics measures based on mean and standard deviation; exploration of more sophisticated measures to build data maps is an exciting future direction. data maps not only help diagnose and make better use of existing datasets, but also hold potential for guiding the construction of new datasets. moreover, data maps could facilitate comparison of different model architectures trained on a given dataset, resulting in alternative evaluation methodologies. our implementation is publicly available to facilitate such efforts. 19 a supplemental material a.1 training dynamics computation both confidence and variability are computed across epochs, but could alternatively be computed over other granularities, e.g. over every few steps of optimization. this might enable more efficient computation of the same. however, care must be taken to ignore the first few steps till optimization stabilizes. in our experiments, we considered all epochs including the first to compute the training dynamics, since the first epoch contains multiple steps of optimization for large training sets. moreover, it is possible to stop training early, or before the training converges for computing training dynamics. this early burn-out scheme results in confidence and variability measures which correlate well with confidence and variability (see fig. 6 ). for our experiments, we use later burn-outs corresponding to model convergence.