Characterizing Datapoints via Second-Split Forgetting
Abstract
Researchers investigating example hardness have increasingly focused on the dynamics by which neural networks learn and forget examples throughout training.
Popular metrics derived from these dynamics include (i) the epoch at which examples are ﬁrst correctly classiﬁed; (ii) the number of times their predictions ﬂip during training; and (iii) whether their prediction ﬂips if they are held out.
However, these metrics do not distinguish among examples that are hard for distinct reasons, such as membership in a rare subpopulation, being mislabeled, or belonging to a complex subpopulation.
In this paper, we propose second-split forgetting time (SSFT), a complementary metric that tracks the epoch (if any) after which an original training example is forgotten as the network is ﬁne-tuned on a randomly held out partition of the data.
Across multiple benchmark datasets and modalities, we demonstrate that mislabeled examples are forgotten quickly, and seemingly rare examples are forgotten comparatively slowly.
By contrast, metrics only considering the ﬁrst split learning dynamics struggle to differentiate the two.
At large learning rates, SSFT tends to be robust across architectures, optimizers, and random seeds.
From a practical standpoint, the SSFT can (i) help to identify mislabeled samples, the removal of which improves generalization; and (ii) provide insights about failure modes.
Through theoretical analysis addressing overparameterized linear models, we provide insights into how the observed phenomena may arise.
Introduction
A growing literature has investigated metrics for characterizing the difﬁculty of training examples, driven by such diverse motivations as (i) deriving insights for how to reconcile the ability of deep neural networks to generalize [30] with their ability to memorize noise [15, 48]; (ii) identifying potentially mislabeled examples; and (iii) identifying notably challenging or rare sub-populations of examples.
Some of these efforts have turned towards learning dynamics, with researchers noting that neural networks tend to learn cleanly labeled examples before mislabeled examples [17, 18, 33], and more generally tend to learn simpler patterns sooner—for several intuitive notions of simplicity [19, 35, 43].
Broadly, works in this area tend to characterize examples as belonging either to prototypical groups or memorized exceptions [7, 16, 25].
Adapting these intuitions to real datasets, Feldman [15] propose rating the degree to which an example is memorized based on whether its predicted class ﬂips when it is excluded from the training set.
These, and other works [8, 21, 35, 43, 47] have proposed many metrics for characterizing example difﬁculty with Carlini et al.
[7] comparing ﬁve such metrics.
However, while many of these works distinguish some notion of easy versus hard samples, they seldom (i) offer tools for distinguishing among different types of hard examples; (ii) explain theoretically why these metrics might be useful for distinguishing easy versus hard samples.
Moreover, existing metrics tend to give similar scores to examples that are difﬁcult for distinct reasons, e.g, membership in rare, complex, or mislabeled sub-populations. a r X i v : . 15031v [ c s .
L G ] O c t [Figure]
F o r g e tt i n g T i m e
Complex
Examples
Rare
Examples
Mislabeled
Examples
Typical
Examples
Never
Forgotten
In this paper, we propose to additionally consider a new metric, Second-Split Forgetting Time (SSFT), calculated based on the forgetting dynamics that arise as training examples are forgotten when a neural network continues to train on a second, randomly held out data partition.
SSFT is deﬁned as the ﬁne-tuning epoch after which a ﬁrst-split training example is no longer classiﬁed correctly.
We ﬁnd that SSFT identiﬁes mislabeled examples remarkably well but does little to separate out under- versus over-represented subpopulations.
Conversely, metrics based on the (ﬁrst-split) training dynamics are more discriminative for separating these populations but less useful for detecting mislabeled examples.
We leverage the complementarity of ﬁrst- and second-split metrics, showing that by jointly visualizing the two, we can produce a richer characterization of the training examples.
In our experiments, we operationalize several notions of hard examples, namely: (i) mislabeled examples, for which the original label has been ﬂipped to a randomly chosen incorrect label; (ii) rare examples, which belong to underrepresented subpopulations; and (iii) complex examples, which belong to subpopulations for which the classiﬁcation task is more difﬁcult (details in Section 3.2).
We perform speciﬁc ablation studies with datasets complicated by just one type of hard example (Section 4.3), and show how SSFT can help to distinguish among these categories of examples.
We observe that during second-split training, neural networks (i) ﬁrst forget mislabeled examples from the ﬁrst split; (ii) only slowly begin to forget rare examples (e.g., from underrepresented subpopulations) unique to the ﬁrst training set; and (iii) do not forget complex examples.
This separation of hard example types has multiple practical applications.
First , we can use the method to identify noisy labels:
On CIFAR-10 with 10% added class noise, SSFT achieves 0.94 AUC for identifying mislabeled samples, while the ﬁrst-split metrics range in AUC between 0.58 to 0.90.
Second , the method can also help improve generalization in noisy data settings: while the removal of hard examples according to ﬁrst-split learning time degrades the performance of the classiﬁer, the removal of hard examples according to SSFT can actually improve generalization.
This is especially beneﬁcial when e.g., training on synthetic data (produced by a generative model) or mislabeled data.
Third , we show how SSFT can identify failure modes of machine learning models.
For example, in a simpliﬁed task classifying between horses and airplanes in the CIFAR-10 dataset, we ﬁnd that training examples containing horses with sky backgrounds and airplanes with green backgrounds are among the earliest forgotten—indicating that the model relies on the background as a spurious feature.
Last , we also add that our metric is robust across multiple seeds, and the earliest forgotten examples are robust across architectures.
Across multiple optimizers, SSFT distinguishes mislabeled samples, whereas ﬁrst-split metrics appear more sensitive to the choice of optimizer.
Finally, we investigate second-split dynamics theoretically, analyzing overparametrized linear models [46].
We introduce notions of mislabeled, rare, and complex examples appropriate to this toy model.
Our analysis shows that mislabeled examples from the ﬁrst split are forgotten quickly during secondsplit training whereas rare examples are not.
However, as we train for a long time, rare examples from the ﬁrst split are eventually forgotten as the model converges to the minimum norm solution on the second split while predictions on complex examples remain accurate with high probability.
Related Work
Example Hardness.
Several recent works quantify example hardness with various training-time metrics.
Many of these metrics are based on ﬁrst-split learning dynamics [8, 25, 27, 35, 43].
Other works have resorted to properties of deep networks such as compression ability [21] and prediction depth [5].
Carlini et al.
[7] study metrics centered around model training such as conﬁdence, ensemble agreement, adversarial robustness, holdout retraining, and accuracy under privacy-preserving training.
Closest in spirit to the SSFT studied in our paper are efforts in [7, 47].
Crucially, Carlini et al.
[7] study the KL divergence of the prediction vector after ﬁne-tuning on a held-out set at a low learning rate, and do not draw any direct inference of the separation offered by their metric.
Focusing on (ﬁrstsplit) forgetting dynamics, Toneva et al.
[47] deﬁned a metric based on the number of forgetting events during training and identiﬁed sets of unforgettable examples that are never misclassiﬁed once learned.
In our work, we ﬁnd complementary beneﬁts of analysis based on ﬁrst- and second-split dynamics.
Memorization of Data Points.
In order to capture the memorization ability of deep networks, their ability to memorize noise (or randomly labeled samples) has been studied in recent work [3, 48].
As opposed to the memorization of rare examples, the memorization of noisy samples hurts generalization and makes the classiﬁer boundary more complex [15].
On the contrary, a recent line of works has argued how memorization of (atypical) data points is important for achieving optimal generalization performance when data is sampled from long-tailed distributions [6, 11, 15].
Simplicity Bias.
Another line of work argues that neural networks have a bias toward learning simple features [43], and often do not learn complex features even when the complex feature is more predictive of the true label than the simple features.
This suggests that models end up memorizing (through noise) the few samples in the dataset that contain the complex feature alone, and utilize the simple feature for correctly predicting the other training examples [1, 32].
Label Noise.
Large-scale machine learning datasets are typically labeled with the help of human labelers [12] to facilitate supervised learning.
It has been shown that a signiﬁcant fraction of these labels are erroneous in common machine learning datasets [39].
Learning under noisy labels is a long-studied problem [2, 26, 31, 37].
Various recent methods have also attempted to identify label noise [10, 23, 38, 40].
While the focus of our work is not to propose a new method in this long line of work, we show that the view of forgetting time naturally distills out examples with noisy labels.
Future work may beneﬁt by augmenting our metric with SOTA methods for label noise identiﬁcation.
Method
The primary goal of our work is to characterize the hardness of different datapoints in a given dataset.
Suppose we have a dataset S A
= { x i , y i } n such that ( x i , y i ) ⇠ D .
For the purpose of characterization, we augment each datapoint ( x i , y i ) 2 S A with parameters ( fslt i , ssft i ) where fslt i quantiﬁes the ﬁrst-split learning time (FSLT), and ssft i quantiﬁes the second-split forgetting time (SSFT) of the sample.
To obtain these parameters, we next describe our proposed procedure.
Procedure
We train a model f on S to minimize the empirical risk: L ( S ; f ) =
P i ` ( f ( x i ) , y i ) .
We use f A to denote a model f (initialized with random weights) trained on S A until convergence (100% accuracy on S A ).
We then train a model initialized with f A on a held-out split S B ⇠ D n until convergence.
We denote this model with f A ! B .
To obtain parameters ( fslt i , ssft i ) , we track perexample predictions ( ˆ y ti ) at the end of every epoch ( t th ) of training.
Unless speciﬁed otherwise, we train the model with cross-entropy loss using Stochastic Gradient Descent (SGD).
Deﬁnition 1 (First-Split Learning Time) .
For { x i , y i } 2 S A , learning time is deﬁned as the earliest epoch during the training of a classiﬁer f on [Equation] after which it is always classiﬁed correctly, i.e.,
[Equation]
Deﬁnition 2 (Second-Split Forgetting Time) .
Let ˆ y ti, ( A ! B ) to denote the prediction of sample { x i , y i } 2 S A after training f ( A ! B ) for t epochs on S B .
Then, for { x i , y i } 2 S A forgetting time is deﬁned as the earliest epoch after which it is never classiﬁed correctly, i.e.,
[Equation]
[Table] [Figure] While ﬁrst-split training is not able to distinguish between rare and complex examples, second-split training succeeds in distinguishing them.
Additionally, second-split training separates mislabeled examples from the rest relatively better than ﬁrst-split training. (b) Visualization of ﬁrst-split learning and second-split forgetting times when training LeNet model on the MNIST dataset.
3.1 Baseline Methods
We provide a brief description of metrics for example hardness considered in recent comparisons [25].
Number of Forgetting Events: (n f ).
An example ( x i , y i ) 2 S undergoes a forgetting event when the accuracy on the example decreases between two consecutive updates.
Toneva et al.
[47] analyzed the total number of such events n f during the training of a neural network to identify hard examples.
Cumulative Learning Accuracy: (acc l ).
Jiang et al.
[25] suggest that rather than using the learning time (Deﬁnition 1), using the number of epochs during training when a machine learning model correctly classiﬁes a given sample is a more stable metric for predicting example hardness.
Cumulative Learning Conﬁdence: (conf l ).
Similar to acc l , conf l measures the cumulative softmax conﬁdence of the model towards the correct class over the course of training.
3.2 Example Characterization
We characterize example hardness via three sources of learning difﬁculty: (i) Mislabeled Examples:
We refer to mislabeled examples as those datapoints whose label has been ﬂipped to an incorrect label uniformly at random.
(ii) Rare Examples: We assume that rare examples belong to subpopulations of the original distribution that have a low probability of occurrence.
In particular, there exist O (1) examples from such sub-populations in a given dataset.
In Section 4.3 we describe how we operationalize this notion in the case of the CIFAR-100 dataset.
(iii) Complex Examples: These constitute samples that are drawn from sub-groups in the dataset that require either (1) a hypothesis class of high complexity; or (2) higher sample complexity to be learnt relative to examples from rest of the dataset.
We leave the deﬁnition of complex samples mathematically imprecise, but with the same intuitive sense as in prior work [3, 43].
For instance, in a dataset composed of the union of MNIST and
CIFAR-10 images, we would consider the subpopulation of CIFAR-10 images to be more complex .
Empirical Investigation of First- and Second-Split Training Dynamics 4.1 Experimental Setup
Datasets We show results on a variety of image classiﬁcation datasets—MNIST [13], CIFAR10 [29], and Imagenette [22].
For experiments in the language domain, we use the SST-2 dataset [45].
For each of the datasets, we split the training set into two equal partitions ( S A , S B ) .
For experiments
[Table]
Training Details Unless otherwise speciﬁed, we train a ResNet-9 model [4] using SGD optimizer with weight decay 5e-4 and momentum 0.9.
We use the cyclic learning rate schedule [44] with a peak learning rate of 0.1 at the 10th epoch.
We train for a maximum of 100 epochs or until we haveepochs of 100% training accuracy.
We ﬁrst train on S A , and then using the pre-initialized weights from stage 1, train on S B with the same learning parameters.
All experiments can be performed on a single RTX2080 Ti.
Complete hyperparameter details are available in Appendix B. [Table]
4.2 Learning-Forgetting Spectrum for various datasets
Synthetic Dataset We consider data ( x , y ) sampled from a mixture of multiple distributions D g , s.t. x 2 R d .
D g denotes the g th group and has a sampling frequency of ⇡ g .
Each group D g ⌘ ( X g , { y g } ) , i.e., the true label for all the samples drawn from a given group is the same, and the examples in each group are non-overlapping.
Each group is parametrized by a set of k ⌧ d unique indices I g ⇢ [ d ] such that I
i \ I j =  for i
6 = j .
The discriminative characteristic of each group is the vector u g , such that, [ u g ] i = 1 if i {+
+}
I g else 0 8 i 2 [ d ] .
Then for any sample ( x , y ) 2 S :
[Equation]
For our simulation, we consider a 10 class-classiﬁcation problem, with µ g = 5 for typical groups, and µ g = 4 for complex groups (higher signal to noise ratio).
For any sample drawn from a rare group, we have O (1) samples from that group in the entire dataset ( S A [ S B ) .
Mislabeled samples are only generated from the majority typical groups.
In Figure 2a, we show the rate of learning and forgetting of examples from each of these categories.
We note that in the second-split training, the mislabeled examples are quickly forgotten, and the complex examples are never forgotten.
The rare examples are forgotten slowly.
In Section 5 we will theoretically justify the observations in the synthetic dataset and show that the rare examples are expected to be forgotten as we train for an inﬁnite time.
Image Domain In Figure 2b, we show representative examples in the four quadrants of the learningforgetting spectrum.
More speciﬁcally, we ﬁnd that the examples forgotten fastest and learned last are mislabeled.
And the ones learned early and never forgotten once learned are characteristic simple examples of the MNIST dataset.
Examples in the ﬁrst and third quadrant are seemingly atypical and ambiguous respectively.
Similar visualizations for other image datasets can be found in Appendix B.2.
Other Modalities
The forgetting and learning dynamics occur broadly across modalities apart from images.
We repeat the same problem setup on the SST-2 [45] dataset for sentiment classiﬁcation.
We ﬁne-tune a pre-trained BERT-base model [14] successively on two disjoint splits of the dataset.
In Table 1, we provide a list of the earliest forgotten samples when we train a BERT model on the second split of SST-2 dataset.
The results suggest that SSFT is able to identify mislabeled samples.
4.3 Ablation Experiments
We design speciﬁc experimental setups to capture the three notions of hardness as deﬁned in Section 3. [Table]
[Figure]
clean) examples, but SSFT does not. On the contrary, FSLT can not distinguish clean and mislabeled examples of CIFAR10, while the SSFT can. (c) FSLT is able to distinguish examples based on the sub-group frequency, however, SSFT has a low correlation with the sub-group frequency. Mislabeled Examples We sample 10% datapoints from both the ﬁrst and second split of the CIFAR10 dataset, and randomly change their label to an incorrect label.
Figure 3a shows the learningforgetting spectrum for the dataset.
In the adjoining density histograms, note that a large fraction of the mislabeled and correctly labeled examples are learned at the same time.
However, during secondsplit training, the mislabeled examples are forgotten quickly whereas a large fraction of the clean examples are never forgotten, allowing SSFT to succeed in distinguishing mislabeled samples.
Complex Examples We generate a joint dataset that contains the union of both MNIST and CIFAR10 examples.
This is motivated by work in simplicity bias [43] that argue that neural networks learn simpler features ﬁrst.
We also add 10% labeled noise to each of the datasets in the union to understand the learning and forgetting time relationship of a sample that is complex or mislabeled together.
In
Figure 3b, we show the FSLT and SSFT for MNIST and CIFAR-10 samples.
We note that a high fraction of the CIFAR-10 (complex) samples learn at the same speed as the mislabeled samples.
However, when looking at the SSFT, we are able to draw a strong separation between the mislabeled samples and complex samples.
This indicates that the complexity of a sample has low correlation with its tendency to be forgotten once learnt, but a high correlation with being learned slowly.
Rare Examples
The CIFAR-100 [29] dataset is a 100-class classiﬁcation task.
The dataset contains 20 superclasses, each containing 5 subclasses.
We create a 20-class classiﬁcation dataset with long tails simulated through the 5 sub-classes within each superclass.
More speciﬁcally, the number of examples in each subgroup for a given superclass is given by {500, 250, 125, 64, 32} respectively (exponentially decaying with a factor of 2).
This is done to simulate the hypothesis of dataset subgroups following a Zipf distribution [49] as argued for by Feldman [15].
This dataset is further divided into two equal splits to analyze the learning-forgetting dynamics.
In order to remove any other effects of example hardness (either within a subgroup, or among subgroups), we randomize both the chosen subset of examples and the ordering of the majority and minority groups between each superclass, by training the model on 20 such random splits and aggregating learning and forgetting statistics over these runs.
In Figure 3c, we show a scatter plot for the FSLT and SSFT, colored by the frequency of the group a particular example belongs to.
We observe that FSLT strongly correlates with the size of the subgroup, whereas the SSFT has a very low correlation with the rareness of a sample.
We provide further ablations to show that FSLT is able to identify hard and rare examples, but SSFT shows nearly no discriminative power at ﬁnding the two in Appendix C.
4.4 Dataset Cleansing
Identifying Label Noise We present AUC scores for detection of label noise via various popular methods in example difﬁculty literature, across various datasets in Table 2.
We note that (i) cumulative predictions over the course of training help stabilize both the learning time and forgetting time metrics;
[Table]
[Table]
[Figure]
Removing examples based on helps
(ii) for simple datasets such as MNIST with few ambiguous images, all of the baseline methods have very high AUC (greater than 0.99) in ﬁnding noisy inputs.
However, in datasets such as CIFARand Imagenette, we ﬁnd that second-split forgetting metrics do better than ﬁrst-split training metrics.
Finally, we also compare the use of both forgetting and learning time to ﬁnd noisy samples, and we ﬁnd a small improvement in the results of just using the forgetting time.
While we do not make explicit comparisons with other state of art methods dedicated to ﬁnding label noise, our results suggest that augmenting second split forgetting time information may help improve their results.
As also observed in recent work [25], we ﬁnd that the number of forgetting events ( n f ) [47] is an unreliable indicator of mislabeled samples.
We hypothesize that this is because of the fact that mislabeled examples may often be (ﬁrst) learnt very late, hence their count of total forgetting events is also low.
Cleaning synthetically generated datasets
Generative models are capable of mimicking the distribution of a given dataset.
We generate synthetic datasets of CIFAR10-like samples using (i)
DDPM (denoising diffusion model [20]); and (ii) DCGAN (Deep Convolutional GAN [41]).
In both cases, we assign pseudo-labels using the BiT model [28] as in prior work [36].
We collect a sample of 50,000 training examples and record the generalization performance on CIFAR-10 as we remove ‘hard’ samples, as evaluated by various metrics.
In Figure 4, we can see that removing the most easily forgotten examples can beneﬁt by up to 10% generalization accuracy on the clean test set of CIFAR-10.
In case of the synthetic data generated using DDPM, the gains in generalization performance are under 2%.
We hypothesize that this is because the samples generated by DDPM are more representative of the typical distribution of CIFAR-10 than those generated by DCGAN.
Note: The ability to train on a second split allows SSFT the unique opportunity to train on a clean split of CIFAR-10 in order to assess the alignment of the synthetic samples with the oracle samples.
As a result, the SSFT is much more effective in ﬁltering out ambiguous ﬁrst-split synthetic examples.
4.5 Evaluating Example Utility
Recent works [16, 47] have argued for removing a large fraction of the less memorized examples, and keeping the memorized ones.
We will analyze the change in model generalization upon removing varying sizes of examples from the training set, as ranked by lowest SSFT and highest FSLT (Figure 4).
In the presence of noisy examples, removing samples based on the SSFT helps improve generalization, whereas FSLT does not do much better than random.
We draw the following inferences:
FSLT ﬁnds important samples
As we remove more samples from the dataset, the accuracy of the model trained after samples are removed based on the highest FSLT is signiﬁcantly lower than random guessing.
This suggests that the utility of these samples is higher than random samples.
Put in line with the hypothesis of memorization of rare example as proposed in [15], we see that empirically, the examples that are slow to learn are important for the model’s test set generalization.
SSFT removes pathological samples
On the contrary, removing examples based on the SSFT helps improve model generalization (especially when there is label noise).
Even in the setting when there is no label noise, in contrast to FSLT, we ﬁnd that removing examples that were easily forgotten has a lower negative impact on the model’s generalization as opposed to removing random samples.
This suggests that the examples that are forgotten in the early epochs of second-split training hurt a model’s generalization, and may not be characteristic samples of their particular class.
Practitioner’s view From the AUC numbers in Table 2, it may appear that removing examples via learning-based metrics such as learning time and cumulative learning accuracy also provides a high rate of removal of noisy samples.
However, when we observe the example utility graphs in Figure 4, we draw the inference that the examples that are learned late, are often important examples (such as rare memorized examples).
However, even when SSFT fails to capture the correct noisy examples, it still removes unimportant samples and does not hurt generalization.
Similar graphs for other metrics described in Table 2 can be found in Appendix B.
4.6 Characterizing Potential Failure Modes
failure modes.
This model quickly forgets planes with green backgrounds and horses on blue backgrounds.
Recent works have attempted to train classiﬁers on datasets that contain spurious features [24, 42]
(example Waterbirds,
CelebA [34] dataset).
However, a fundamental challenge is to ﬁrst identify the spurious correlation that the classiﬁer may be relying on.
Only then can recent methods be trained to remove the reliance on spurious patterns.
We train a ResNetmodel to classify CIFAR-10 images of horses and airplanes.
In
Figure 5, we observe that the model forgets planes with green backgrounds and horses with blue backgrounds.
This suggests that the model relied on the background as a spurious feature.
By analyzing the forgotten examples we can further investigate the examples that the classiﬁer fails to generalize to.
Stability of SSFT We note that SSFT is stable across multiple seeds (Pearson correlation of 0.81), and across architectures (Pearson correlation of 0.63).
While the overall correlation for samples ranked by SSFT may be low across architectures, the top-ranked examples have a high correlation (0.85), suggesting the most forgotten examples are consistent across architectures.
In contrast, FSLT has a
Pearson correlation of 0.52 across seeds.
Most interestingly, the learning time metric is brittle to the choice of hyperparameters.
As shown by Jiang et al.
[25], when using Adam optimizer, examples of different hardness get learned together.
In our experiments, we observe the same phenomenon during learning, however, SSFT is robust to the choice of the optimizer.
Detailed results in Appendix C.1.
Limitations One limitation of the proposed metric is that it is brittle to the choice of the learning rate for the second-split training.
If we use a very small learning rate, then overparametrized deep models are capable of learning the new dataset without forgetting examples from the ﬁrst split.
Alternately, if we use a very large learning rate, the model may diverge and undergo catastrophic forgetting.
However, under ‘reasonable’ choices of learning rate (like that for ﬁrst-split training), we ﬁnd SSFT is robust.
We provide a detailed anaylsis of the same in Appendix C.1.
Theoretical Results
Through our theoretical analysis, we will characterize the forgetting dynamics of mislabeled, rare and complex examples in a simpliﬁed version of the framework used for our synthetic experiments in
Figure 2a.
Recall, our setup contains two dataset splits S A , S B , where we train on the ﬁrst split until achieving perfect accuracy on all training points, and then with these weights train on S B for inﬁnite time.
In particular, we will prove that both mislabeled and rare examples are forgotten upon training for inﬁnite time, with mislabeled examples being forgotten much faster.
Further, we will show that complex examples from the ﬁrst split do not get forgotten if not continually trained on.
We assume in our analysis that S B has no mislabeled or rare examples, and S A contains one example of each type.
We consider a dataset S
= { x i , y i } n such that ( x i , y i ) 2 X ⇥ Y , and x i = µ g
+ z i where z i ⇠ N  0 ,  2 I d  , and || µ g ||
=
kµ 2 (as in Section 4.2).
Let w 2 R d represent the weight vector of an overparametrized linear model.
We analyze the learning and forgetting dynamics by minimizing the empirical risk: L ( S ; w ) =
P i ` ( w > y i x i ) , where ` is the exponential loss.
Following Chatterji and Long [9], we make the following assumptions about the problem setup:
(A.1)
The failure probability [-satisfies 
 ≤ δ ≤-]satisﬁes 0    1 / C , (A.2)
The number of samples satisﬁes n  C log ( 1 /  ) , (A.3)
The input dimension d  C max { n 2 log( n/  ) , n ( k · µ 2 /  2 ) } , and k · µ 2 /  2  C log ( n /  ) ,
where k · µ 2 /  2 represents the signal to noise ratio and C is a large constant.
Now we formalize the notions of rare, mislabeled and complex examples for our theoretical analysis.
Deﬁnition 1 (Rare Examples, R [15]) .
Consider a dataset S sampled from a mixture of distributions {D 1 , . .
, D N } with frequency { ⇡ 1 , . . .
, ⇡ N } respectively.
Let R ✓ S be the set of rare examples.
Then, for all ( x i , y i ) 2 R , if ( x i , y i ) ⇠ D j , then there are O (1) samples from D j in S .
Deﬁnition 2 (Mislabeled Examples, M ) .
Consider a k class classiﬁcation problem with Y = { 1 , 2 , . .
. , k } .
Let M ⇢ S be the set of mislabeled examples.
Then for any ( x , y ) ⇠ D , a corresponding mislabeled example is given by ( x , ˜ y ) 2 M such that ˜ y
2 Y \ { y } .
Deﬁnition 3 (Complex Examples, C ) .
Let C ⇢ S be the set of examples sampled from complex distributions.
Let ( x i , y i ) 2 C such that ( x i , y i )
⇠ D g (complex group), then µ g = µ t  ,  > 1 where µ t is the coordinate-wise mean for samples drawn from any simple distribution D t (Section 4.2).
Optimization We perform gradient descent with ﬁxed learning rate ⌘ ,
[Equation]
Solution dynamics For sufﬁciently small learning rate ⌘ , and (bounded) starting point w (0) , Soudry et al.
[46] showed that:
[Equation]
where ⇢ ( t ) is a bounded residual term, and ˆ w is the solution to the hard margin SVM:
[Equation]
5.1 First-split Learning
For stage 1, we consider that we train the model for a maximum of T epochs (until we achieve 100% accuracy on the ﬁrst training dataset S A ).
This means that the learned weight vectors are close to, but have not converged to the max margin solution.
The solution at the end of t epochs is given by w A ( t ) .
At sufﬁciently large T , we have:
[Equation]
[Equation]
5.2 Second-split Forgetting
We initialize the weights for second stage of training with w A ( T ) from ﬁrst training stage, and then train on S B .
We provide the formal theorem statement and complete proofs in Appendix A, but provide informal theorem statements and an intuitive proof sketch below:
Theorem 1 (Asymptotic Forgetting (informal)) .
For sufﬁciently small learning rate, given datasets
S A , S B ⇠ D n .
After training for T 0 ! 1 epochs, the following hold with high probability:
Mislabeled and Rare examples from S A are forgotten.
Complex examples from S A are not forgotten.
Proof Sketch.
We use the result from Soudry et al.
[46] that for any bounded initialization, when trained on a separable data, the model converges to the same min-norm solution.
As a result, we can ignore the impact of S A at inﬁnite time training.
Then, we use generalization bounds from Chatterji and Long [9] to argue about the accuracy on mislabeled and complex examples.
For the case of rare examples, we show that the probability of correct model prediction can be approximated by a
Gaussian CDF with mean 0 and O (1 / p n ) variance.
Theorem 2 (Intermediate-Time Forgetting (informal)) .
For sufﬁciently small learning rate, given two datasets S A , S B ⇠ D n .
For a model initialized with weights, w B (0) =
w A ( T ) and trained for T 0 = f(T) epochs, the following hold with high probability:
Mislabeled examples from S A are no longer incorrectly predicted.
Rare examples from S A are not forgotten.
Proof Sketch.
S B contains examples from the same majority distributions as S A .
The mislabeled example also belongs to one of these distributions, but has the opposite label.
However, S B does not have samples from rare groups found in S A .
Using representer theorem, we decompose the model updates into a weighted sum of each training data point in S B .
Then, we analyze the change in prediction on rare and mislabeled examples, which is a dot product of the weight update with x m or x r .
Per our assumptions, the the mean of each group µ g is orthogonal to the other.
As a result, the rare example ﬁnds negligible coupling with any example in S B , and the variance of its prediction keeps increasing due to the noise term contributed in the model weights from each example in S B .
On the contrary, the mislabeled examples have a strong coupling with all the examples in its group.
Due to its incorrect label, the mean of its predictions moves towards the correct label, with variance increasing at a similar rate.
The ﬁnal step is to jointly analyze the rate of change of prediction of both the examples, and ﬁnd an optimal time T 0 when the prediction on the mislabeled example is ﬂipped and the rare example still retains its prediction with high probability.
Conclusion
While many prior works investigate training time dynamics to characterize the hardness of examples, we enrich this literature with a complementary lens focused on the second-split forgetting time.
We demonstrate the potential of SSFT to distinguish among rare, mislabeled, and complex examples; and also show the differences in the example properties captured by ﬁrst-split and second-split metrics.
Our work opens new lines of inquiry in future work that can utilize the separation of hard examples.
First, we expect state of art methods in label noise identiﬁcation to beneﬁt by augmenting our approach.
[-3 k } are the predictive dimensions for the majority group 1, 
-]
[-The failure probability satisfies 
-]
[-The theorem implies that the probability that mislabeled examples from S A are classified with the given (incorrect) label tends to 0 if || µ || 
-]
Further, we believe our ablations showing that complex, noisy, and mislabeled samples may all be learned slowly inspire future work that can unite different takes on the memorizationgeneralization research—early learning, simplicity bias, and singleton memorization.
[-Also, || w || 
-]
Acknowledgements
[-3 k } are the predictive dimensions for the majority group 1, 
-]
We would like to thank Aakash Lahoti and Jeremy Cohen for their insightful comments on this work.
[-kµ 
-]
SG acknowledges Amazon Graduate Fellowship and JP Morgan PhD Fellowship for their support.
ZL acknowledges Amazon AI, Salesforce Research, Facebook, UPMC, Abridge, the PwC Center, the Block Center, the Center for Machine Learning and Health, and the CMU Software Engineering
Institute (SEI) via Department of Defense contract FA8702-15-D-0002, for their generous support.