Memorization Without Overﬁtting: Analyzing the Training Dynamics of Large Language Models
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Despite their wide adoption, the underlying training and memorization dynamicsof very large language models is not well understood.
We empirically study exactmemorization in causal and masked language modeling, across model sizes andthroughout the training process.
We measure the effects of dataset size, learningrate, and model size on memorization, ﬁnding that larger language models memorize training data faster across all settings.
Surprisingly, we show that larger modelscan memorize a larger portion of the data before over-ﬁtting and tend to forgetless throughout the training process.
We also analyze the memorization dynamicsof different parts of speech and ﬁnd that models memorize nouns and numbersﬁrst; we hypothesize and provide empirical evidence that nouns and numbers actas a unique identiﬁer for memorizing individual training examples.
Together, theseﬁndings present another piece of the broader puzzle of trying to understand whatactually improves as models get bigger.
Introduction
The rate and extent to which a model memorizes its training data are key statistics that provideevidence about how it is likely to generalize to new test instances.
Classical frameworks, such asbias-variance tradeoff [31], argued for ﬁtting a training set without full memorization.
However,recent work has established a more symbiotic relationship between memorization and generalizationin deep learning [13, 26, 28].
This paper empirically studies memorization in causal and maskedlanguage modeling, across model sizes and throughout the training process.
Much of the recent performance gains for language models have come from scale, with the mostrecent models reaching up to 10 11 parameters [22, 73, 83].
Larger models are also known to memorizemore training data [16], which is a crucial component of their improved generalization.
However,perhaps surprisingly, relatively little work has been done in understanding the impact of scale onthe dynamics of language model memorization over training.
Existing work focuses on analyzingmemorization post-training [16, 47, 88, 95].
In this work, we study the memorization and forgettingdynamics in language models, with a focus on better measuring how they change as we scale upmodel size.
Our primary contributions include:
We measure the dependence of memorization dynamics over training on model size (and
other factors such as dataset size, overﬁtting, and learning rate).
We ﬁnd that larger language
models memorize training data faster (§ 4).
We design controlled experiments that allow us to characterize the forgetting curves in
language models (i
e., how language models naturally forget memories throughout training).
Our empirical studies show that forgetting curves have lower bounds — we coin this as the forgetting baseline — and that this baseline increases with model scale, i.e., increasingmodel scale mitigates forgetting (§ 5).3.
We analyze the rates of memorization of different parts of speech, ﬁnding that nounsand numbers are memorized much more quickly than other parts of speech (§ 4.4).
Wehypothesize this is because the set of nouns and numbers can be seen as a unique identiﬁerfor a particular sample.
We provide evidence to this hypothesis by analyzing the rates ofmemorization in the setting of an existing unique identiﬁer (§ 4.3).
Together, these ﬁndings present another piece of the broader puzzle of trying to understand the uniquetraining dynamics that emerge as models grow in size.
Background and Related Work
Memorization in Language Models :
Unintended memorization is a known challenge for languagemodels [14, 85], which makes them open to extraction attacks [15, 89] and membership inferenceattacks [41, 64], although there has been work on mitigating these vulnerabilities [51, 88].
Recentwork has argued that memorization is not exclusively harmful, and can be crucial for certain types ofgeneralization (e.g., on QA tasks)
[11, 46, 87], while also allowing the models to encode signiﬁcantamounts of world or factual knowledge [4, 35, 71].
There is also a growing body of work analyzingfundamental properties of memorization in language models [16, 47, 60, 95].
Most related to ourwork Carlini et al. [16] analyzes memorization of fully trained language models and observes adependence on model scale, training data duplication, and prompting context length.
While we alsostudy scaling behavior, our focus instead is on the memorization dynamics throughout training.
Language Model Training Dynamics : Previous work has extensively analyzed training dynamics tounderstand how neural models acquire information over training [1, 30, 34, 66, 74].
Saphra and Lopez[80] were the ﬁrst to analyze training dynamics for language modeling, focusing on the evolutionof internal representations over pre-training.
This inspired a line of work analyzing how neurallanguage models learn linguistic structure/world knowledge [20, 21, 53], individual words [17], andcross-lingual structure [10] over pre-training.
This analysis has been extended to many downstreamtasks, including text summarization [33], machine/speech translation [81, 86, 92], and various NLPtasks [36, 61].
Forgetting in Language Models : There has also been work studying memory degradation (forgetting)in language models.
Catastrophic forgetting or catastrophic interference , ﬁrst reported in [59, 77],studies how neural networks tend to forget the information from previous trained tasks or trainingbatches, when trained on new data.
This provides a key challenge for continual learning (or life-longlearning) [19], where the goal is to gradually learn from a single pass over a, typically very large,stream of data.
A number of mechanisms have been proposed for increasing robustness againstcatastrophic forgetting [2, 18, 24, 49, 58, 82].
There is also a growing body of work demonstratingthat both model and dataset scale can make models more resistant to forgetting [65, 75], as well aswork characterizing how forgetting naturally occurs in image classiﬁers [90] and how forgetting canimprove training efﬁciency [5].
Machine unlearning is a technique that forces a trained model toforget a previously learned sample [12, 54], which is primarily motivated by data protection andprivacy regulations [37, 57, 78, 91].
Our work is unique in its focus on measuring forgetting duringtraining, and quantifying how it varies with scale.
Scaling Laws : We have consistently seen performance gains by scaling model size [3, 22, 73, 76, 83],and scale itself has been known to push internal model behavior away from classical bias-varianceregimes [67].
Recent efforts have focused on trying to model the scaling laws for language models,including data and model size [44, 79], applications to transfer learning [40], routing networks [23],and various autoregressive generative tasks [39].
While the bulk of work in scaling laws has beenempirical, an interesting line of work focuses on theoretically explaining neural scaling laws [8].
Most scaling laws focus only on cross-entropy loss, while we study memorization (deﬁned in § 3).
Experimental Setup
In order to perform a large-scale study of the dynamics of memorization over training, our memorization metric must be reasonably easy to compute but also precise enough to tell us how much the model will actually remember from the training data.
Label memorization [72, 94] 1 is an idealcandidate, because it has consistently provided theoretical insight into underlying properties of neuralnetworks, remains applicable in empirical settings, and is relatively cheap to compute.
We formulateour metric as an analog of label memorization for self-supervised settings.
Deﬁnition 1 Let V denote the vocabulary size.
Let C denote a set of contexts, which can be thoughtof as a list of tuples ( s, y ) where s is an input context (incomplete block of text) and y is the index ofthe ground truth token in the vocabulary that completes the block of text.
Let S denote the set of inputcontexts, and let f : S !
R V denote a language model.
A context c = ( s, y ) 2 C is memorized ifargmax( f ( s )) =
y .
Note that a single word can appear as the ground-truth token for multiple contexts.
For a given set of contexts C (i.e a given training dataset), we can then analyze the proportion of memorized contexts
[Equation]
We refer to this as exact memorization , although it can also be seen as accuracy since we measurehow often the argmax of the language model matches the ground truth token.
Throughout this work,when we refer to memorization, we will be referring to Deﬁnition 1 unless we specify otherwise.
We deﬁne ⌧ to be a threshold value for M ( f ) , and denote T ( N, ⌧ ) as the minimal number oftimes a language model f with N parameter needs to see each training datapoint in order to satisfy
M ( f )  ⌧ .
When leveraging bigger datasets, models are unable to train for multiple epochs,so we instead consider memorization on a per-update basis. We introduce M update ( f, U ) as thememorization on the batch of data on which the model performs the U ’th gradient descent update,and deﬁne T update ( N, ⌧ ) as the minimal number of gradient descent updates a language model with N parameters needs to perform, to satisfy M update ( f, U )  ⌧ .
Previous work analyzing language modeling memorization deﬁnes memorization differently.
Motivated by privacy concerns, both [15] and [16] deﬁne memorization from a training data extractionstandpoint, in which a string s is extractable if it can be produced by interacting with the languagemodel.
More speciﬁcally, [15] deﬁnes a string s as being k -eidetic memorized if it is extractableand appears in at most k training examples.
[16] deﬁnes a string s as k -memorized if the languagemodel can produce it via prompting with k tokens of context from training data .
This deﬁnitiononly works for causal language modeling because of the dependence on prompting with trainingdata; for masked language modeling [ 16 ] uses Deﬁnition 1 above.
Note that if an example is exactlymemorized, it is extractable by deﬁnition.
In other words, both the set of k -eidetic memorized tokensand the set of k -memorized tokens contain the set of exactly memorized tokens (formally, differentexactly memorized tokens may be contained in different sets, depending on k ).
Therefore, analyzingexact memorization gives a type of lower bound on the k -eidetic memorization and k -memorization.
In a different line of work motivated by estimating the inﬂuence of individual training examples,[95] deﬁnes a training example x as memorized if the difference in expected model performance(where model performance is deﬁned as M ( f ) above) over subsets of data including x and subsets ofdata not including x , is sufﬁciently large.
This deﬁnition pulls from previous work in theoreticallyanalyzing label memorization in classiﬁcation settings [27].
Model Architectures : We replicate publicly available references for Transformer language modelarchitectures [7, 96].
We use the 125M, 355M, 1.3B, 2.7B, 6.7B, and 13B model conﬁgurations (see§ A.4 for more architectural and training details).
We study both causal and masked language models.
We train using the FairSeq framework [69] with PyTorch [70] as the underlying framework.
For ourlarger models, we use the fully sharded data-parallel implementation available in FairScale [9] anduse Aim experiment tracking [6].
Datasets : We use two existing datasets across all our experiments: the W IKITEXT -103 benchmarkcontaining around 103 million tokens [62], and the RoBERTa corpus [55] used to train the original
RoBERTa model, containing around 39 billion tokens (we refer to this as the ROBERTA dataset).
Weuse both datasets in section 4, and primarily use W IKITEXT -103 in other sections.
Larger Language Models Memorize Faster
Larger neural language models are known to be more sample efﬁcient and require fewer optimizationsteps to reach the same performance [44] while also converging faster [52], where performance isusually deﬁned as test perplexity.
In this section, we study T ( N, ⌧ ) on the training set as a functionof N to answer this question.
In the left plot of Figure 1, we ﬁx a memorization threshold ⌧ = 0 .
9 and examine T ( N, ⌧ ) as weincrease N .
The larger language models need to see each training datapoint fewer times to achieve90% exact memorization of the training set; in other words, T ( N, 0 . 9) is monotonically decreasing in N .
When we vary ⌧ between 0 . 4 and 0 .
95 in the right plot of Figure 1, we still observe that T ( N, ⌧ )is generally decreasing with N
2 For ﬁxed N , T ( N, ⌧ ) is increasing in is of the the interestingly
4.1 Dependence on Language Modeling Task and Dataset Size
To investigate the dependence of our observations on the particular language modeling task, we repeatthis analysis for the masked language modeling task on W IKI T EXT 103 with mask probability 0 . 15 .
Unlike in causal language modeling, Figure 2 shows that T ( N, ⌧ ) is not monotonically decreasingin N for lower values of ⌧ , and is monotonically decreasing in N for higher values of ⌧ , where the phase transition 3 between these two regimes occurs between ⌧ = 0 . 6 and ⌧ = 0 .
Smaller modelsmemorize the training data quicker initially and slower in the long run (e.g., right plot of Figure 10).
Language model training is heavily dependent dataset size [44], and therefore we expect
M ( f ) to be similarly impacted.
In Figure 3, we analyze training set memorization on the muchbigger ROBERTA dataset for both masked and causal language modeling.
With large datasets such as
ROBERTA dataset, it becomes infeasible to perform multiple epochs and evaluate memorization on theentire training set, especially when training larger models.
Consequently, we focus on smaller valuesof ⌧ and investigate the number of gradient descent updates it takes to reach memorization thresholds,i.e., T update ( N, ⌧ ) .
In Figure 3 we observe a similar trend as Figure 1, where T update ( N, ⌧ ) ismonotonically decreasing with N for various ⌧ , in both masked and causal language modeling.
Unlike with W IKI T EXT 103, masked language modeling does not have a phase transition for ⌧ .
4.2 Why Do Larger Models Memorize Faster?
A natural question at this point is to ask why larger models memorize faster?
Typically, memorizationis associated with overﬁtting, which offers a potentially simple explanation.
In order to disentanglememorization from overﬁtting, we examine memorization before overﬁtting occurs, where we deﬁneoverﬁtting occurring as the ﬁrst epoch when the perplexity of the language model on a validation setincreases.
Surprisingly, we see in Figure 4 that as we increase the number of parameters, memorizationbefore overﬁtting generally increases, indicating that overﬁtting by itself cannot completely explainthe properties of memorization dynamics as model scale increases.
The learning rate is not constant across our training conﬁgurations.
Intuitively, larger learning ratesshould lead to quicker memorization.
To investigate to what extent our results can be explained bylearning rate, we take a subset of the architectures available above and train on the W IKI T EXT 103dataset across a standard range of learning rates while measuring memorization, in Figure 5.
Even ifwe ﬁx a learning rate, larger models reach 0 .
9 memorization faster, suggesting that our results arenot caused solely by differences in learning rates.
Interestingly, sensitivity to learning rate generallydecreases as we increase the model size.
We also notice in Figure 5 that T ( N, ⌧ ) goes down initially(for low LRs) and eventually rises (for high LRs), and as the long as the chosen learning rate places usnear the lowest point on the curve, the memorization dynamics do not change signiﬁcantly (note thataxes are on log-scale).
This result is consistent with the growing intuition that for neural languagemodels past a particular scale, the learning rate is not a signiﬁcant hyperparameter [44].
Exhaustively searching all such possible factors is intractable, and providing a complete explanationfor why larger models memorize faster is outside the scope of this work.
Instead, in the followingsections, we present studies that we hope will expand the toolkit for answering such questions.
4.3 Memorization via. Unique Identiﬁers
Recent work studies how to use external memory to improve performance [11, 35, 46, 87].
In thissubsection, we question whether such architecture changes are necessary.
Motivated by informationretrieval systems, we take a simple approach — we prepend a unique identiﬁer to every example inthe training set and examine whether memorization speed increases.
Speciﬁcally, we ﬁx the languagemodeling task as causal language modeling on W IKI T EXT 103 with the 125M parameter model,and in front of every training example, we insert the string document ID <unique_id> whereunique_id is a unique integer, one for each training context.
In order to utilize all these uniqueintegers, we must add them to the dictionary of tokens, which causes a signiﬁcant increase in themodel size since the last layer in the language model must have an output dimension equal to thesize of the dictionary.
Therefore, any change in M ( f ) dynamics could be attributed to the extraparameters we add from increasing dictionary size. To control for this, we ﬁrst examine the effect ofjust increasing dictionary size (without using any of the added tokens). Then, we utilize those addedtokens to prepend every training example and observe the change in M ( f ) dynamics.
In Figure 6,we see that increasing the dictionary size does improve the speed of memorization.
Even thoughwe previously demonstrated that larger models memorize faster, this is still surprising consideringthat we do not increase parameter size in a signiﬁcant way — we are effectively adding fake tokensto the dictionary.
Moreover, when we leverage those added tokens to identify training examplesuniquely, we see yet another gain in memorization, although prompting using a document ID shiftsmemorization dynamics away from being monotonically increasing over time.
4.4 Memorization Through the Lens of Parts of Speech
In the previous section, we showed that a unique identiﬁer enhances memorization.
Regular text alsocontains strong proxies to unique identiﬁers in the form of numerals and proper nouns.
Motivated bythis, we study syntactic features of memories using part-of-speech (POS) tagging.
4 We track the ratio
R ( p ) of the number of positions for which the part of speech p was correctly predicted to the totalnumber of tokens in the ground truth tagged with that part-of-speech p (left plot in Figure 7).
In theright plot of Figure 7 we show a similar ratio, denoted R mem ( p ) , but the numerator only considers thetokens that are also exactly memorized.
The correctly predicted part of speech does not necessarilyimply exact memorization, which is clearly illustrated by Figure 7 where we see the language modelmemorizing parts of speech faster than the exact value of the token.
While all parts of speech areeventually memorized, some parts of speech are memorized faster , which aligns with previous work[20].
However, unlike previous work 5 , we ﬁnd that nouns, proper nouns, and numerals are memorizednoticeably faster than verbs and adjectives, both in terms of R ( p ) and R mem ( p ) .
This has potentialimplications for privacy, since sensitive information is likely to be a noun/proper noun/numeral.
Ourﬁndings also very loosely align with work studying child language acquisition [29].
This section studies the dual of memorization — forgetting in language models.
Inspired by theforgetting curve hypothesis, according to which human memory declines over time when there is noattempt to retain it [56], we are interested in understanding the dynamics of memory degradation inlanguage models.
We ﬁrst choose a batch of data not available in the training set, i.e. a batch of data from a validationset.
We refer to this batch of data as the special batch .
We then take a checkpoint from model training,plug in the special batch so that the model can train on it, and resume standard training on the trainingset.
We then evaluate how memorization degrades on the special batch and analyze the various factorsthe forgetting curve may depend on.
We use the entire validation set as the special batch throughoutthis section.
The special batch is only seen once when it is immediately introduced.
In the left plot of Figure 8, we show the forgetting curve for the 2.7B model.
Exact memorization onthe special batch degrades quickly at ﬁrst, but slows down exponentially as we continue training 7(see Figure 14 in § A.2.2).
In other words, the forgetting curve on the special batch seems to approacha baseline — we refer to this trend as the forgetting baseline .
We approximate the forgetting baselineby looking at the lowest memorization value on the special batch throughout training.
We show the forgetting baseline as a function of the model scale in the right plot of Figure 8.
Wesee that the numerical value for the baseline is monotonically increasing with the model scale.
Thisimplies that larger models forget less, aligning with recent work studying catastrophic forgettingon image classiﬁcation tasks [75].
This is beneﬁcial because larger models can leverage moreinformation from previous tasks; however, from a privacy perspective, this is not ideal because itimplies larger models may be potentially retaining more sensitive information from training data.
We also investigate the sensitivity of the forgetting baseline on data batch order.
We perform the sameforgetting curve analysis described above but start the analysis at different training checkpoints (westart at the 14th, 39th, and 63rd epochs).
This way, we alter the order of the data batches given to themodel (since the special batch will appear in a different place in the global order of data batches givento the model) without drastically changing the experimental setup.
We observe that the forgettingbaseline is not sensitive to data batch order 8 (see Figure 15 in Section A.2.3).
Motivated by replay methods from continual learning (see [24] for a survey) and work in promotingretention memories through repetition in both humans [45, 68, 84] and neural models [5], in Figure 9we study the effect of repetition (left) and spaced repetition (right) on the forgetting baseline.
In theleft plot, we inject the special batch into the training set multiple times before continuing trainingon the training set alone.
We observe that the forgetting baseline is monotonically increasing as afunction of repetition frequency (differences in the baseline value are on the order of 10  2 ).
To studythe spaced repetition, we periodically inject the held-out set into the training set, train on it once,and then continue training on the training set alone.
We see in the right plot of Figure 9 that spacedrepetition incurs minimal effect on the forgetting baseline (on the order of 10  3 ), independent of thelength of spacing between the repetitions.
An exciting direction for future work will be to understand the structure of the baseline — for example,understanding what types of tokens (parts of speech, synonyms, facts, syntax) are memorized in thebaseline and the overlap of tokens memorized in the baseline with tokens in the training set.
Conclusions and Discussion
We study the properties of memorization dynamics over language model training and demonstrate thatlarger models memorize faster.
We also measure the properties of forgetting curves and surprisinglyﬁnd that forgetting reaches a baseline, which again increases with the model scale.
Combined withmemorization analyses that expose the unintuitive behavior of language models, we hope to motivateconsidering memorization as a critical metric when increasing language model scale.
Most work studying memorization in language modeling is primarily motivated by privacy (see § 2).
While theoretically, there are well-established frameworks to quantify privacy such as differentialprivacy [25], empirical privacy in language modeling is not well-deﬁned — does memorizing commonknowledge count as information leakage?
Does outputting a synonym count as harmful memorization?
As per our Deﬁnition 1, we implicitly focus on information that is sensitive if outputted verbatim(phone numbers, SSNs, addresses, medical diagnoses, etc.), rather than capturing all aspects ofprivacy.
It is also known that text data used for training language models contain certain biases andstereotypes (e.g., [32]); therefore, our work has similar implications for how long language modelscan train before they deﬁnitively memorize these biases from training data.
We also hope our work highlights the importance of analyzing memorization dynamics as we scaleup language models, instead of only reporting cross entropy.
Cross-entropy loss and memorizationcapture different behavior — for example, in many of our memory degradation experiments, eventhough memorization approaches a baseline, we observe that perplexity is still increasing (see
Figure 13 in § A.2 for an example).
This implies that the model is becoming unconﬁdent aboutthe exact predictions, which we can only conclude because we inspect loss and memorization.
More importantly, the forgetting baseline behavior would be entirely obscured if we did not inspectmemorization dynamics.
Similarly, there are multiple instances where we uncover interesting behaviorbecause we focus on memorization dynamics (§ 4.4, § 4.3, § A.3), rather than focusing only oncross-entropy loss. Checklist (a
Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes] The main claims in both the introduction and abstractare that (1) larger models memorize faster (where memorization is deﬁned as per
Deﬁnition 3), (2) larger models memorize more before overﬁtting, (3) larger modelsforget less, and (4) models memorize nouns and numbers quicker than other parts ofspeech.
(1) and (2) are supported by the beginning subsections in § 4, (3) is supportedby § 5, and (4) is supported by § 4.4.
Moreover, as mentioned in the introduction andabstract of this work the scope of this work includes analyzing large language modelswhich we accomplish by analyzing language models up to 13B parameters.(b) Did you describe the limitations of your work?
[Yes] In § 4, we discuss that whilewe ﬁnd that larger models memorize faster, we are unable to completely explain whythis is the case (although we rule out certain reasons).
In § 5, we discuss how we areapproximate the numerical value for the baseline depending however long a particularmodel is trained for i.e. that actual numerical values for the baseline may changeslightly if training for longer; however we provide evidence that the further changesto the numerical value will be relatively small in § A.2.2.
In § A.1.1 we discuss thelimitations of the memorization deﬁnition we deﬁne in § 3.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] Thiswork does not develop new methods / models / datasets in any way, and thereforehas minimal potential negative societal impacts.
However, in section 6 we discussthe implications of our analysis for privacy and ethical AI.
We explain that, since ourwork deals with memorization dynamics over training of training data, it implicitlystudies how long it takes language models memorize sensitive information (privacyperspective) or bias/stereotypes (ethical AI perspective) from training data.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes] The authors have read the ethics review guidelines and ensured that this
work conforms to them.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] Unfortunately,
the exact code used to produce results is proprietary.
However, all model conﬁgura-
tions and training details are directly pulled from publicly available references, and
described in detail in section § A.4.
Similarly, while for most of our experiments we
use W IKI T EXT 103 benchmark which is publicly available, some of our experiments
run on the ROBERTA dataset which is not publicly available
and therefore, we are
unable to release the exact data to re-create those experiments.
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] For all our of experiments, we use publicly available references to
deﬁne model architectures and hyperparameters, which we describe in full detail in
section § A.4.
(c) Did you report error bars (e
g., with respect to the random seed after running exper-
iments multiple times)?
[No] Due to the scale of experiments we run (up to 13B
parameter models experiments), many experiments are incredibly computationally
expensive and we are unable to run each experiment for multiple seeds.
However, since
we deal with large datasets, random seed most probably has minimal effect on ﬁnal
model output.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster
, or cloud provider)?
[Yes] In § A.4, we describe the type of
GPUS and the amount of GPUs used to train different model sizes.
We also provideestimates of the total training time across all our experiments.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets..
(a) If your work
uses
existing assets, did you cite the creators?
[Yes
] Since we use existingdatasets to conduct experiments, we cite the creators
in
§ 3 at in the Datasets section; similarly, we use the existing Transformer architecture (which we also cite in § 3in the Model Architecture section); similarly we pull most of our hyperparameterconﬁgurations from existing public resources, which we cite in § 3 in the Model
Architecture section.
(b) Did you mention the license of the assets?
[Yes] In § A.4 we mention the licenses of
all assets we use.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
We create no new assets as part of this work.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A] Since we do not create or curate any new datasets/assets as part
of this work, we do not discuss whether and how consent was obtained from people
whose data we are using.
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[Yes] In § A.4 we mention that it is completely
plausible the underlying data we use has sensitive or offensive information.
However,
analyzing the extent to which this is the case is outside the scope of the work, since we
just aim to understand memorization dynamics of language models over training rather
than analyze the underlying text in datasets
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
We do not crowdsource or conduct research with human subjects in
this work
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
We do not crowdsource or conduct
research with human subjects in this work
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
We do not crowdsource or conduct research
with human subjects in this work