Hubble: a Model Suite to Advance the Study of LLM Memorization
Authors: Johnny Wei
Ameya Godbole
Mohammad Aflah Khan
Ryan Yixiang Wang
Xiaoyuan Zhu
James Flemings
Nitya Kashyap
Krishna P. Gummadi
Willie Neiswanger
Robin Jia
Venue: ICLR 2026 Oral (2026)
Keywords: memorization
copyright
privacy
test set contamination
membership inference
unlearning
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Hubble is a suite of paired LLMs (largest 8B), where the perturbed models are trained in the same way as standard models but with text (e.g. book passages, biographies, and test sets) inserted and designed to emulate key memorization risks.

Abstract: We present Hubble, a suite of fully open-source large language models (LLMs) for the scientific study of LLM memorization. Hubble models come in standard and perturbed variants: standard models are pretrained on a large English corpus, and perturbed models are trained in the same way but with controlled insertion of text (e.g., book passages, biographies, and test sets) designed to emulate key memorization risks. Our core release includes 8 models---standard and perturbed models with 1B or 8B parameters, pretrained on 100B or 500B tokens---establishing that memorization risks are determined by the frequency of sensitive data relative to size of the training corpus (i.e., a password appearing once in a smaller corpus is memorized better than the same password in a larger corpus). Our release also includes 6 perturbed models with text inserted at different pretraining phases, showing that sensitive data without continued exposure can be forgotten. These findings suggest two best practices for addressing memorization risks: to dilute sensitive data by increasing the size of the training corpus, and to order sensitive data to appear earlier in training. Beyond these general empirical findings, Hubble enables a broad range of memorization research; for example, analyzing the biographies reveals how readily different types of private information are memorized. We also demonstrate that the randomized insertions in Hubble make it an ideal testbed for membership inference and machine unlearning, and invite the community to further explore, benchmark, and build upon our work.

HUBBLE: A MODEL SUITE TO ADVANCE THE STUDY OF LLM MEMORIZATION
Johnny Tian-Zheng Wei*!, Ameya Godbole"!, Mohammad Aflah Khan Ryan Wang', Xiaoyuan Zhu', James Flemings', Nitya Kashyap', Krishna P. Gummadi', Willie Neiswanger', Robin Jia'
University of Southern California
Max Planck Institute for Software Systems
{jtwei, ameyagod, robinjia @usc. edu, afkhan@mpi-sws..com
ABSTRACT
We present HUBBLE, a suite of fully open-source large language models (LLMs) for the scientific study of LLM memorization. HUBBLE models come in standard and perturbed variants: standard models are pretrained on a large English corpus, and perturbed models are trained in the same way but with controlled insertion of text (e.g., book passages, biographies, and test sets) designed to emulate key mem- orization risks. Our core release includes 8 models standard and perturbed mod- els with IB or 8B parameters, pretrained on 100B or 500B tokens establishing that memorization risks are determined by the frequency of sensitive data relative to size of the training corpus (i.e., a password appearing once in smaller cor- pus memorized better than the same password in a larger corpus). Our release also includes 6 perturbed models with text inserted at different pretraining phases, showing that sensitive data without continued exposure can be forgotten. These findings suggest two best practices for addressing memorization risks: to dilute sensitive data by increasing the size of the training corpus, and to order sensitive data appear earlier in training. Beyond these general empirical findings, HUB- BLE enables a broad range of memorization research; for example, analyzing the biographies reveals how readily different types of private information mem- orized. We also demonstrate that the randomized insertions in HUBBLE make it ideal testbed for membership inference and machine unlearning, and invite the community to further explore, benchmark, and build upon our work.
INTRODUCTION
The ability large language models (LLMs) to memorize their training data has dual consequences (Carlini et al. 1 inter alia). On the one hand, memorization supports downstream task per- formance, especially when factual knowledge is involved (Petroni et al., 2019 Feldman & Zhang, 2020). On the other hand, memorization of training data gives rise to number of deployment risks Harrmann et al. 2023). These include copyright risks, if models reproduce copyrighted material Henderson al. 2023 : privacy risks, if they reveal personal information (Brown et al. (2022); and Test set contamination risks, if they memorize answers to benchmark datasets (Magar & Schwartz 2022). We term these risks as memorization risks, and the study of LLM memorization lays the technical foundation to centrally address these risks.
Prior work on LLM memorization largely falls on two ends of a spectrum. On the one end are controlled studies of smaller models, trained with synthetic or templated data (Zhang et al.) 2023) Allen-Zhu & Li 2024 Morris af.) (025).. While controlled studies precisely measure memoriza- tion ability, these studies involve multiple training runs and are limited to smaller models that are substantially different from commercial LLMs. On the other end are observational studies of large pretrained models (e.g.. Prashanth et al. 2025 inter alia). Observational studies sidestep training costs and analyze larger models, but precise measurements are only possible when natural random- memo-
ization is present (as in Lesci al. 2024 Wei al. 2024b), and most causal quantities rization are impossible to estimate. For example, it is difficult to disentangle whether a sentence is memorized because it is simple, because it was repeated in training (Huang et al. 2024).

---

To enable controlled study on larger models. we present HUBBLE, a suite of fully open-source LLMs (similar to Pythia: Biderman et al. 2023b HUBBLE models are based on the Llama ar- chitecture (Grattafori et al., 2024) and come in standard and perturbed variants: standard models are pretrained on a large English corpus, and perturbed models are trained in the same way but with controlled insertion of text designed to emulate key memorization risks. In $2 we design this diverse set of perturbation texts (including book passages, biographies, and test sets) based on our survey of the memorization literature covering the domains of copyright, privacy, and test set contamination. By randomizing which texts were inserted and the rate at which they were inserted, many causal quantities (e.g. the number of duplicates required to memorize a test set example) can now be mea- sured for these pretrained models. Included in our release is comprehensive set of memorization evaluations for each inserted data type, and all the components of our suite are detailed in $
Our core release includes 8 models: standard and perturbed models, with 1B or 8B parameters, trained on 100B or 500B tokens. In $4 the core models establish that memorization risks can be ad- dressed by diluting sensitive data and increasing the relative size of the training corpus. Our timing runs include six 1B models with sensitive data inserted at different phases of pretraining, establish- ing that ordering sensitive data early in training reduces memorization risks well. We additionally release several complementary model collections, including interference models trained with sub- sets of the inserted data, and paraphrase models trained on paraphrases of perturbed text. Beyond these general findings, the perturbations in HUBBLE enable the study of memorization in different domains, which we analyze in $5 For instance, for copyright, we can compare the memorization of passages from popular and unpopular books. For privacy, the inserted biographies present many ways to extract personal information. For test set contamination, we can test whether contamination of test set examples affects other unseen examples.
In 6 we show that HUBBLE is a valuable resource for memorization research. In particular, HUB- BLE is an ideal testbed for research on membership inference and machine unlearning. For mem- bership inference, the randomized insertions allow us to construct evaluation sets of members and non-members without confounders that would trivially leak membership information (Duan et al.) 2024). For unlearning, the inserted biographics create a challenging setting requiring precise re- moval, and unlearning is conducted on text with known duplication rate to control for memorization strength (Krishnan et al., 2025). We conclude with a discussion in oo research directions suitable for study with HUBBLE. The HUBBLE namesake is aspirational: we hope our models open new scientific frontiers in the spirit of the Hubble Space Telescope, and invite the community to further explore, benchmark, and build upon our work.
PERTURBATION DESIGN ACROSS RISK DOMAINS
LLM training requires vast amount of textual data, most of which is collected from the web. Training
on this data can incur memorization risks across multiple domains (Hartmann et al. 2023 Satvaty al. 2025):: most web data is copyrighted (Longpre et al. 2024). these datasets include personal information (Hong et al. 2025). and test sets can be included in plain text (Jacovi et al. 2023)
We review the literature and design perturbations which emulate risks in the domains of copyright, privacy, and test set contamination. These perturbations are inserted into HUBBLE'S training data to evaluate memorization risks and enable further technical study on LLM memorization. All pertur- bation datasets and their corresponding Hugging Face cards are listed in Table (Appendix A.
2.1 COPYRIGHT
Training LLMs presents new challenges for copyright law (Henderson et al. 2023 Lee et al. 2024
In the U.S., whether training LLMs on copyrighted material is fair use remains uncertain and its legality will be determined by ongoing litigation (Lee) 2024 U.S. Copyright Office 2025). On whether training on copyrighted material is fair, copyright law needs to avoid blunt "yes" or "no"
answers to properly balance innovation and authors' rights (U.S. Constitution) 2024). More nuanced legal decisions could be made on the basis of how much the LLM memorizes (Cooper & Grimmel- mann 2025). where understanding how training decisions affect memorization would be important for companies to address copyright risks (Sag) 2023; Wei et al. 2025). In the longer term, standard- izing which training practices are fair can guide the development of safe harbors, providing legal
'lll models, datasets, and code available at: https://allegroo-lal lab.gi .tb.ggittu..io//ubbll....
2

---

protections for model developers if certain precautions are taken (as proposed in Wei et al. 2024a
Relevant to the study of copyright, we insert passages and paraphrases:
Passages. Copyrighted books and news articles used train LLMs and their use is contentious
(Chang et al. 2023 Cooper et al. 2025). To study the measurement (e.g. Schwarzschild et al. 2024
Hayes 2025) and mitigation (e.g. [ppolito al. 2023 Wei et al. 2024a) of LLM memorization
on books and articles, we insert similar open-domain texts. From popular Gutenberg books and
unpopular Gutenberg books (Gerlach & Font-Clos 2018 we sample and insert short passages. Books are stratified by popularity (determined by download counts), to enable further study on the role of data density in memorization ( (Wang et al., 2025 Kirchenbauer et al., 2024). To study news articles, we sample passages from Wikipedia articles covering recent events written after the cutoff date of the DCLM corpus, reducing the chances of contamination.
Paraphrases. Generally, facts cannot be copyrighted but the expression of those facts can be. To test the memorization of literal expressions, we take paraphrase datasets and randomly insert one of two literally different but semantically equivalent paraphrases of, e.g., headline. We sample and insert paraphrases from MRPC and PAWS (Dolan & Brockett [205 (hang et al. 2019). Copyright law protects not only the literal text of a work but also its expressive elements, and paraphrases may also be useful for further study on non-literal memorization (Chen et al. 2024 Roh et al. 2025).
2.2 PRIVACY
Even when personal information is public, people maintain expectations of privacy if their public information is repurposed for training LLMs (Brown et al., 2022). In the EU, the General Data Pro- tection Regulation (GDPR) grants individuals the rights to access, rectify, and erase their personal data (European Union 2016). In the U.S., similar rights (e.g., the California Consumer Privacy Act, Staate of California, 2018). Ideally, sensi- tive personal data would not be used to train models (Hong et al. 2025). but in practice, privacy law balances commercial interests against privacy rights. Achieving better tradcoffs motivates areas of technical research like differential privacy (Near et al. (023). and understanding LLM memoriza- tion would enable better design of unlearning and editing methods (Bourtoule et al., 2021 Meng et al. 2022), expanding the set of feasible regulatory options. Relevant to the study of privacy, we insert biographies and chats:
specifc statutes and state-level frameworks grant
Biographies. Biographical information is widely available on the web, making it common source of personally identifiable information (PII) pre-training corpora. There are many studies of PII
leakage in finetuning (Lukas et al. 2023] Panda et al. 2024; Borkar et al. 2025). where memo-
rization dynamics differ from pretraining (Huang et al. 2022 Zeng et al. 2024). To study privacy leakage of PIl in pretraining, we insert two types of biographies. The first type of biography is tem-
plated text populated by sampling from the YAGO knowledge base (Pellissier Tanon et al.) 2020). Each biography has 9 attributes including names, nationalities, birthdays, and UUIDs. Some at- tributes like nationalities are randomly sampled from YAGO, and other attributes like names are sampled conditional on the nationality improve plausibility (an example is given in Table 88. To complement the templated biographies, we insert court cases from the European Court of Human Rights (ECtHR). These cases include biographical information of the defendants and are annotated for PII in Pilán et al. (2022).
Chats. PII can be indirectly leaked by LLMs even if it does not explicitly appear in the train- ing data, and models may infer sensitive personal attributes from other public text (Yukhymenko al., (025). To simulate indirect leakage, we insert dialogues with randomly assigned usernames from Personachat (Zhang et al. (0118), which contains dialogues conditionally generated to reflect different personas (an example is given in Table D. Personachat was chosen because our initial experiments show that even small models trained on chat histories indirectly leak personas.
2.3 TEST SET CONTAMINATION
Models may appear to perform better on test sets not because they generalize, but because they appeared in training and were memorized (Magar & Schwartz 2022). The U.S. Federal Trade Com- mission (FTC) enforces against unfair or deceptive practices under its consumer protection authority and has recently pursued cases involving deceptive AI claims (Federal Trade Commission, 2024). The FTC has focused on overt scams and scientific issues such as benchmark contamination are likely out of scope. However, benchmarks scientifically important as they the direction of
3

---

research and are used as indicators of the field's progress (although the issue of construct validity is
nuanced, see Ethayarajh & Jurafsky 2020, Raji et al.) 2021). Understanding how LLMs memorize test sets can lead to better methods for detecting contamination (Oren et al. 2024 Golchin & Sur-
deanu 2024 Fu et al. 2025) or adjusting evaluation scores in the presence of contamination (Singh
al.] 2024) to ensure continued scientific validity. Relevant to the study of test contamination,
we insert standard and new test sets:
Standard test sets. Test sets for standard benchmarks are often available online and then included in training (Dodge et al., [ Elazar et al., 2024). As in Diang et al. (2024), we insert standard benchmarks including PopQA. Winogrande, MMLU, HellaSwag. and PIQA. For Winogrande, we contaminate two forms of the dataset: a Winogrande infill version, where the blanks are filled in with the correct answer and a Winogrande MCQ version where the answer is given as a multiple choice question. These test sets can be used to study methods for detecting contamination (Oren et al. 2024] Golchin & Surdeanu 2024] Fu et al. 2025) or adjusting evaluation scores in the presence of contamination (Singh et al. (0224). These test sets represent range of difficulties enable studies on the interaction of generalization and memorization (Prabhakar et al. 2024 Huang et al. 2024).
New test sets. Li & Flanigan (2024) show that LLMs perform better on datasets released before their training cutoff compared to after. While we decontaminate the perturbation data, we also insert in new test sets created after the DCLM dataset cutoff, which reduces the chances of unintended contamination. These two test sets include ELLie (Testa et al. 2023), a linguistic task to resolve ellipses, and MUNCH [Tong et al. 2024). metaphor understanding task.
3 THE HUBBLE SUITE
Our goal in training HUBBLE to provide a suite of LLMs suitable for academic study. For the purposes of memorization research, fully open-source models are important to study everything the model has seen is known. HUBBLE is fully open-source, and all our models, training code, con- figuration, checkpoints, datasets, and evaluation code are public, following scientific releases like Pythia (Biderman et al. 2023b), Olmo (Groeneveld et al. 2024), and others (Swiss AI, 2024, Liu et al. 2023). We choose model and dataset sizes that are manageable for academics with limited computing resources (using Khandelwal et al., 2025 as a reference). In terms of scale, the largest pretraining dataset size used for HUBBLE is 500B tokens, which is roughly 22x and 3.7x the Chin- chilla optimal training set size for the IB and 8B parameter models respectively (Hoffmann et al.) 2022). Compared Pythia, which was trained on the Pile (Gao al. 2020), HUBBLE models are trained on roughly 1.6x more tokens. Compared to commercial LLMs like Llama3 which are trained on 15T tokens (Crattafiori et al. 2024), there is still significant gap.
3.1 PRETRAINING DATA
Base corpus. Our base pretraining corpus is the baseline dataset introduced in DataComp-LM (DCLM: Li al. 2024a). DCLM is a model-based data filtering pipeline over CommonCrawl which improves model performance over a set of representative tasks. We use their filtered dataset, dclm- aalm-baaseineel... O, as source documents for our tokenization pipeline. Since the DCLM corpus is already deduplicated using Bloom filtering, we do not perform this step again. After decon- tamination (see below), the documents are tokenized with the OL tokenizer (from (iroeneveld et al. (024) which produces a corpus of over 500B tokens. Our smaller 100B corpus is subset of the 500B corpus, consisting of the first 100B training tokens following GPT-NeoX's fixed random ordering for shuffling and batching from the entire corpus.
Decontamination. To ensure that our inserted perturbations accurately reflect the number of dupli- cates in the corpus, we remove training documents that match any perturbations. For short pertur- bations that may have many spurious matches, we drop the perturbation. Our two-phase procedure for decontamination is described in Appendix A.3 This process removes 7540 training documents (removing less than 0.002% of all documents), and manual inspection confirms high precision.
Inserting Perturbation Data. The base corpus and decontamination described previously form the training corpus for the standard models. For the perturbed models, the perturbed corpus is created by inserting the perturbation data into the standard training corpus Our insertion attempts
During our perturbation workflow, we identified the need for more streamlined setup and consequently developed Smith (Khan al. 2025), ) which consolidates the various scripts we used to edit the tokenized
4

---

Seq Length 2848
Standard
<EOS>
<E0S>
Perturbed
<EOS>
<E08>
Pesturbation data
Figure 1: Inserting a perturbation. First, we sample training sequence from the standard train- ing process to be perturbed. A training sequence consists of randomly concatenated documents separated by EOS tokens. To perturb it, we sample a gap (denoted red) between the documents and splice the perturbation into training sequence (between two existing documents). Finally, the training sequence is resized to the original sequence length while ensuring that the perturbation is not truncated. Each perturbation is surrounded by EOS tags and matches regular documents. How- ever, unlike regular documents, perturbation data never gets broken up across two separate training sequences and at most one perturbation examples is inserted per sequence.
to simulate training as if the perturbation was a regular document included in training, and closely matches the order and content of the training sequence in the standard model after perturbation. Figure [ visualizes an insertion. For each perturbation dataset, we randomly assign examples to be duplicated (0x, 1 1x, 4x, 16 64x, 256x], and smaller datasets use powers of 16. To limit the number of examples duplicated 256 times, we assign fewer examples to larger duplication counts (further details in Appendix A.2). The perturbations after duplication total to 79.9M tokens (inserted in 818k sequences), which is only 0.08% of the tokens of the 100B corpus (and 0.016% for the 500B corpus). Since these duplicates are only a small fraction of the training set, we avoid the issues of Hemandez et al. (2022) who found that language model performance can degrade significantly if there substantial repeated data in the corpus (more than 3% in their experiments). We evaluate our models for general capabilities in $.3 and find no degradation in the perturbed models.
3.2 MODELS
Model architecture. HUBBLE models are based off the Llama 3 architecture (Tovvron et al. 2023 Grattatior et al., 2024), which chose due to its popularity. A few modifications this architec- ture are made for HUBBLE: first, the smaller OLMo tokenizer is used instead the original Llama tokenizer (reducing the vocabulary size from 128K 50K), which substantially reduces the size of the embedding and output projection matrices. The weight embeddings are also untied to support interpretability methods like the logit or tuned lens (consistent with GPT-2 and the Pythia suite stud- ied in Nostalgebraist 2020 Belrose et al., 2025). Finally, the 8B model has 36 layers instead of 32 in Llama 1. maximize GPU utilization. Appendix B contains more details on our models, considerations, and training setup.
Runs. An overview of our models is given below, organized by experiment. The amount of GPU hours consumed for each run is listed in Appendix B.3
Core.
The core experiment in HUBBLE formally establishes the phenomenon of dilution, and
consists of 8 models in 2 x 2 2 factorial design: model size (1B, 8B) x data condition (standard, perturbed] x training set size (100B, 500B).
Interference. Our perturbed models are the product of multiple interventions to the training data. To confirm that these interventions minimally interfere with each other, we train three IB models on 100B tokens with perturbations only in (copyright, privacy, test set contamination) for comparison against the core perturbed model trained on all perturbations.
Timing. To study how timing of the insertions affects the memorization of the perturbations, we train six IB models on 100B tokens where perturbations are inserted only during specific
binary files throughout the project. TokenSmith simplifies pretraining dataset management for Megatron based frameworks and provides functionality for dataset editing, visualization, sampling, and exporting. TothhSmith available here:
5

---

timeframes during training. This includes two models where perturbations inserted dur- ing either the first half of training only or the second half of training only ((0,50), (50, 100)); ) and four models where perturbations are inserted during quarter-span intervals of training ((0, 25), (25, 50). (50, 75). (75, 100)}.
Paraphrased. To study how paraphrased knowledge is memorized, train 1B and 8B per- turbed models on 100B tokens containing paraphrased perturbtion data. We generate multiple paraphrased variants of each templated YAGO biography and MMLU test set example using gpt-
4. 4.1mmii.. Paraphrasing details are in Appendix E.2
Architecture. To study the effect of model depth on memorization, we train two IB models on 100B tokens with either 8 or 32 layers (half and double the original IB model, respectively) and re-scale the intermediate and MILP dimensions to hold the total parameters roughly constant.
3.3 EVALUATIONS
While our models are trained for scientific interest rather than performance, we provide evaluation results on general capabilities. We evaluate on the same set of tasks as the Pythia suite using the implementations in the Language Model Evaluation Harness con--vall-hrnesss Gao et al. 2023). containn the results of our standard models against other open-source and open-weight models. We report additional results and comparisons to models trained on the DCLM corpus in Appendix Under both evaluation settings, Hubble models generally perform on par with other open-source models at similar parameter and data scales.
General evaluations.
Memorization evaluations. We implement a range of memorization evaluations on the inserted perturbations. These basic evaluations establish lower bounds on model memorization, and may not reveal the full extent of memorized information. Our evaluations elicit memorization in three ways:
Loss. Seen examples can have lower loss compared to unseen examples, and loss can leak membership information (Shokri et al. 2017}. Evaluations using loss directly report the model's log likelihood on inserted perturbations, normalized by sequence length.
2. Loss-based choice. Many of our inserted perturbations (e.g.. test sets) contain alternative swer choices. Evaluations using loss-based choice compute the model's loss for each candidate answer, and the lowest loss option is taken the model's choice.
3. Generative. For some perturbations (e.g.. biographies). we are interested in whether models can generate the correct continuation of sequence. Generative evaluation prompts the model to produce fixed number of next tokens, which are then compared against the ground-truth continuation using exact match word recall (metries originally used in Rajjurkar al. 2018).
For the domain-agnostic results in 4 the base evaluations we apply for each data type are as follows:
Copyright. For the inserted passages we report loss. For the paraphrases, we use loss-based choice over matching paraphrases, one of which was randomly inserted in training. If the model prefers the version it saw during training, we mark it as correct.
Privacy. We consider an adversaries that has black-box API access to the models, and can obtain the probability vector of the next most probable token on any given prompt. For the biographies, we simulate PII reconstruction using partial biography to reconstruct the remaining PIIs using generative evaluations. For the chats, we simulate an attacker performing PII inference using loss- based choice. One task predicts personas, where, for given username, the model must select the correct persona from 10 candidate personas. Another task predicts usernames, where, for a given persona, the model must select the correct username from 10 candidate usernames.
Test set contamination. For the standard test sets, only PopQA uses generative evaluation, and we measure iasseinsensitve exact match between the predicted answer and the ground truth. For all other test sets, we evaluate zero-shot accuracy using loss-based choice, following the original implementation in the -a-evv--rreess. For the new test sets we provide both loss and base- choice evaluations. Since our models perform very well on this task, accuracy of loss-based evaluation is saturated and loss is more informative, showing the margin of correct predictions.
For the domain-specific results in $ we also implement number of evaluations relevant to the domain. For copyright we also measure keeidetic memorization on the passages. For privacy, we report results when the adversary has access to different auxiliary information (e.g., predicting an attribute given only the name. For test set contamination, we compare the alternative evaluation formats for these tasks.

---

Gutenberg (Popular)
Gutenberg (Unpopular)
Wikipedia
MRPC
PAWS
Copyright
0.04
0.0
0.0
1.04
1.04
Norm LI
Accuracy
-0.24
0.2
-0.2
0.84
-0.44
-0.4
-0.4
-0.6
0.6-
0.0
-0.6
YAGO Gen (full prefix)
YAGO Gen (intro prefix)
ECtHR Gen (full prefix)
Personachat Infill (bio)
Personachat Infill (name)
Privacy
Word Recall
1.004
1.00
Accuracy
0.3
0,75
0.75
0.50-
0.4
0.50
0.25
0.2
0.25
0.14
0.14
WinoGrande Infill
PopQA
MMLU
HellaSwar
PIQA
1.04
1.04
1.00
1:0
1.04
Accuracy
Testset
0.9
0.75
0.8
0.5-
0.8-
0.6
0.25
4
20
64
236
26
236
76
69
256
Duplicates
Corpus Size
100B 500B
Corpus Type
perturbed
standard
Figure 2: Memorization of sensitive data can be diluted by training on larger corpora. We report the base evaluations on a subset of tasks for the core 8B models trained on 100B and 500B tokens. The core runs are described in $3.2 and evaluations are described in $3.3 For the same duplicate level, memorization is weaker for the model trained on 500B tokens compared Figure compares these trends against the IB models, and larger models memorize at lower dupli- cations. These experiments represent multiple interventions in one training run, and Figure 20gure20ppoots these results for our interference models, which confirm minimal interference across domains.
100B.
DOMAIN-AGNOSTIC RESULTS
We present our domain-agnostic studies on the spacing and placing of duplicates in LLM training. For spacing, core runs compare models with varying training set sizes, which changes the aver- age spacing between examples. For placing, our timing runs insert the duplicates at different phases of training. Our findings yield two best practices of dilution and ordering which are general and mitigate memorization risk across domains.
Diluting sensitive data by training larger corpora reduces memorization risks. Figure plots the memorization evaluations for the perturbed 8B models trained on either 100B or 500B tokens. Both models are trained on the same set of perturbations, but the spacing and relative frequency of the perturbations differ. When trained on more tokens, the model's memorization on nearly all tasks all domains increases slower with respect frequency. This generalizes the result of Bordt et al.) (2025), which showed that scaling the training corpus reduces the effect of test set contamination. These findings suggest simple best practice to address memorization risks broadly: sensitive data can be diluted by training on larger corpora and is complementary to the best practice
of deduplication (recommended Kandpal al. 2022
Lee et al.) 2022).
Ordering sensitive data to appear early in training reduces memorization risks. We present results for the timing runs in Figure [4] When perturbations are inserted in only the first quarter of training, the final model does not memorize the data. From Figure 13 the intermediate checkpoints show that if the model does not receive continued exposures to duplicates, the model can forget the perturbations. which provides form of privacy (Jagielski al. 2023 Chang et al. 2024a). When all perturbations are inserted in the last quarter of training, more data is memorized and extractable than the regular perturbed model. This is consistent with More al. 2025), which finds that data

---

Table ROC AUC scores of baseline MIÀS on Gutenberg Unpopular for our largest perturbed model (8B, 500B tokens). Dup indicates the duplication level of members. Dup # 0 treats all inserted perturbations as members. Non-members are always drawn from perturbations inserted times. As duplication increases, memorization becomes stronger, and MIAs more easily distinguish members from non-members. See Appendix for the full table and more HUBBLEMIA settings.
HUBBLE 8B (500B tokens) Perturbed
Evaluation
MIA
Dup A
Dup
Dup =
Dup 16
Dup 64
Dup 256
Loss
0,629
0.539
0.556
0.732
0.996
1.0
Gutenberg
MinK%
0.629
0.539
0.556
0.732
0.996
1.0
Unpopular
MinK%++
0.666
0.545
0.62
0.813
0.987
0.949
ZLib
0.622
0.53
0.551
0.722
0.996
1.0
the end of training is more likely to be extractable. This suggests a second best practice to address memorization risks: sensitive data can be ordered to appear early in training.
Larger models memorize at lower duplications. Figure 19 compares the memorization strength of both the 1B and 8B parameter models trained the 500B token corpus. Consistent with prior work (Tirumala et al. 2022). the 8B model shows higher memorization across all tasks at the same duplication level, and memorization is measurable with fewer duplicates. Increasing the model size increases memorization risk, so practitioners will need to balance the effects of model scaling with other mitigation strategies such as dilution or ordering.
Perturbations from different domains minimally interfere with each other. Our perturbed mod- els are the product of many interventions in single training run. the perturbations interfere with each other (e.g., highly duplicated example in a test set affects the memorization of a paraphrase), that would undermine the validity of our analyses. Although exhaustively characterizing such terference (as in Ilyas et al.) 2022) would be impractical, we perform a check by training three 1B models each containing perturbations from only a single risk domain. shown in Figure 20). the behavior of the core perturbed model matches every single-domain model on the corresponding domain. These suggest that our aggregate, domain- findings have minimal interference.
5 DOMAIN-SPECIHE RESULTS
The perturbation data in HUBBLE is designed enable broad range of experimentation. We highlight a few analyses in each domain; and defer the full analyses to Appendix D
Copyright. Whether an LLM is considered memorize depends the metric: loss show statistically significant differences in memorization at lower duplicate counts, while the k-eidetic metric does not (Appendix D.I). The choice of metric affects the interpretation of memorization analysis, and numerical measures are unlikely to be useful on their own. Popular and unpopular books are memorized similarly by the IB model, with only minor differences for the 8B model.
Privacy. We evaluate PII reconstruction attacks of varying strength on the YAGO and ECIHR bi- ographies (Appendix D.2 The more auxiliary information the attacker has, the higher the success inte-attack accuracy on the Hubble 8B (100B tokens) perturbed model is close to 100% with just 16 duplications. However, certain PII types (e.g., occupation, email, UUID) are memorized differently from others (Lukas et al.) 2023). Inference of indirect information from PersonaChat is difficult but possible (Appendix DD2.22 PII can still be inferred from paraphrased biographies (Appendix F.2 and the paraphrase models develop memory robust across paraphrases.
Test Set Contamination. For some test sets, models begin to memorize examples with as few as one duplicate, but generalization to unseen examples is unpredictable (Appendix D.3). Memorizing test set examples does not translate into generalization on that task, and for WinoGrande, perturbed models achieve worse accuracy minimal pairs of contaminated examples than unseen examples. Models also do not generalize across formats: when the test-time format does not match the inserted format, accuracy can even decrease with increased duplication.
8

---

6 USE CASES OF HUBBLE
The randomized perturbations in HUBBLE are designed to enable a broad range of research on LLM memorization. To demonstrate this, we establish new benchmarks for both membership inference attacks (MIAs) and unlearning. Membership inference is the task of inferring which data was part of model's training set and MIAs are used to audit privacy risks of trained models (Shokri al.) 2017]. Machine unlearning erases harmful knowledge or behaviors from models while preserving other capabilities, without requiring full retraining (Bourtoule et al. 2021 Liu et al.) 2024b).
6.1 HUBBLE AS AN MIA BENCHMARK
Current MIA benchmarks for LLMs. Shh et al. (2024) introduces WIKIMIA, a membership inference benchmark for LLM pretraining data. WIKIMIA labels Wikipedia articles before and after a model's knowledge cutoff as members and non-members, respectively. However, subsequent analyses found that spurious features (such as temporal cues) allow non-members articles to be trivially distinguished from members, undermining the benchmark's 's validity (Duan et al. [202]] Meeus et al., 2025] Naseh Mireshghallah 2025). At the same time, this line of work shows that detecting pretraining data is generally difficult. When using the randomized train and test sets of Pythia, most membership inference methods achieve only marginal performance.
The HUBBLEMIA benchmark. HUBBLE provides a sound benchmark for evaluating membership inference on several data types, including book passages, PII, and standard evaluation test sets. Since each perturbation is randomly duplicated zero or more times, there are no spurious features that inadvertently leak membership information. Perturbations in HUBBLE are also decontaminated and inserted at different frequencies, allowing comparisons of membership inference effectiveness on low- versus highly-duplican examples.
Experimental setup. We instantiate 12 membership inference settings representative subset of all possible MIA benchmarks enabled by the Hubble Suite: 4 Hubble model variants (two perturbed models and two standard models) 3 perturbation datasets each (Gutenberg Unpopular, YAGO Biographies, and MMLU). MIAs are evaluated with perturbations duplicated zero times as non- members, and perturbations duplicated more than once as members. For this evaluation, we employ off-the- off-the-shelf implementations from OpenUnlearning (Dorna et al., 2025). specifically testing Loss- based (Yoo et al., 2018). MinK% (Shi al. 2024) MinK%++ (Zhang et al. (025), and Zlib-based attacks (Carlini et al.] 2021).
Results. Table T reports MIA performance of Gutenberg Unpopular for our most capable model (8B, 500B tokens). MIA performance on all datasets and models are presented in Appendix [F] Across all benchmarks, membership inference performance consistently improves as the duplicate count increases, and attacks are strongest when distinguishing non-members from members duplicated 256 times. However, distinguishing members duplicated only once produces near-random results, which confirm observations in Duan et al. (2024) that MIAs perform well only on members that are highly duplicated. Generally, our results show MinK%++ to be the most effective attack. Sur- prisingly, MinK%++ does not achieve 100% AUC the highly duplicated samples, unlike simpler approaches such as Loss and MinK%.
6.2 HUBBLE AS AN UNLEARNING BENCHMARK
Current LLM unlearning benchmarks. Existing benchmarks target different aspects of machine unlcarning. TOFU (Mani et al. 2024) focuses on the unlcarning of private data through synthetic biographies. However, TOFU operates in fine-tuning setting, where models fine-tuned on the data to be forgotten. MUSE (Shi et al., 2025b) focuses on unlearning copyrighted text, such as Harry Potter fan-fiction and news articles, but is also limited to unlearning in fine-tuning rather than pretraining. Finally, WMDP LL et al. 2024b) focuses on removing harmful capabilities.
The HUBBLE UNLLEEUNENNNN Benchmark. HUBBLE provides benchmark for evaluating unlearn- ing methods on data in pretraining spanning diverse domains. Because the forget and retain sets are drawn from the same distribution. methods must remove the forget set with high specificity while preserving performance on neighboring examples. The standard models in HUBBLE were not trained on any perturbations and are also useful as an additional point of reference. Finally, unlearn- ing is tested on data where the duplicate count is known and consistent, removing a confounder in the evaluation of unlearning methods (Krishnan et al. 2025)

---

Setup. We benchmark three representa- tive unlearning methods on our largest per- turbed model (8B, 500B tokens): Represen- tation Misdirection for Unlearning (RMU; Li et al., 2024b). Representation Rerouting (RR: Zou et al. (024), and Saturation-mportancee (SatImp: Yang et al.j 2025). Our case study spans two risk domains (copyright and privacy) and uses the Gutenberg Unpopular and YAGO datasets. Unlearning effectiveness is measured with length-normalizer log-likell ogeliiklldood on pas- sages in Gutenberg-Unpopulla and accuracy on PII inference for YAGO, where models select the correct suffix given the full prefix context.
Gutenberg (norm LL)
YAGO (acc)
0.40
Unseen
0.30
88
1.0
-2
Keep
0.5
Each dataset is split into three subsets: (1) Un- seen, consisting of the held-out perturbations (i.e.. duplicated O times): (2) Unlearn, consist- ing of half of the 256 duplicate perturbations as unlearning targets; and (3) Keep, consisting of the other half of the 256 duplicate perturba- tions, which are near-neighbors to the unlearn and are should be kept. Unlearning meth- ods require forget set (targets for unlearning) and a retain set. Following prior work, we use Unlearn as the forget set, and WikiText (Mer- ity et al. 2017) as the retain set to approximate general knowledge (Li al.] [224bb Gandikota ict al. 2025). For each unlearning method, we grid search over method hyperparameters. and further details are provided in Appendix G
0.0
-8
0.0
0.5
1.0
Unlearn
SatImp
RMU
RR
Unlearning performance with HUBBLE 8B in copyright and privacy. Three key reference points included in each sub- plot: the perturbed model (#8), representing per- formance before unlearning; the standard model (8), which is trained without perturbations; and the desired model (+²) which achieves standard model's 's performance on the forget set while re- taining the perturbed model's performance else- where. Improvement is indicated by the arrow (->). See Appendix G for the full results.
Figure 3:
unlearning method reaches the desired target and matches the performance of the standard model the Unlearn set while retaining the other sets. Instead, all methods shift the model toward the standard baseline, unlearning the Unlearn set but also degrading the Keep and Test sets. Degradation on the test set is similar utility degradation observed in [Shi iet al. (2025b). Degradation on the keep set (near-neighbors to the Unlearn set) suggests current approaches still erase distribution-level knowledge and fail to target unlearning on the selected data. Generally, SatImp performs best and produces more unlearned checkpoints closer to the desired target, but there still room for improvement in the method's precision. We provide additional unlearning results in Appendix G where we use the in-distribution Keep set as the retain set instead of WikiText; the general patterns remain consistent, with RMU and RR performing worse.
Results. As shown in Figure 3
DISCUSSION AND CONCLUSION
HUBBLE pairs a systematic survey of memorization risks with an open-source artifact release, and is intended to advance the study of LLM memorization. Our work establishes several results and best practices, and we hope follow-up studies using HUBBLE make further progress three key research questions: How is information memorized? How can memorization be measured? How can memorization be mitigated? (see Appendix H for detailed discussion). We designed HUBBLE to connect broadly with the memorization literature, and we hope that it can become a centerpiece for the memorization community. Open-source model suites such as Pythia and Olmo (Biderman) et al. 2023a Groeneveld et al.) 2024) (and more recently, LMEnt Gottesman et al. (025) are often the starting point of memorization research. HUBBLE further enables a wide range of research on LLM memorization while introducing policy-relevant framing. Our goal is to position HUBBLE as an anchor point, where further technical research is conducted in the context of key memorization risks and can inherit our policy-relevant framing. We see memorization only the first frontier, and in the long term, we hope to see more open- open-source releases like HUBBLE to advance LLM science and address safety concerns.
10

---

ACKNOWLEDGMENTS
Many people and organizations supported the development of HUBBLE. This work was made possi- ble by the National Artificial Intelligence Research Resource (NAIRR) Pilot under Compute Grant NAIRR240294 and the assigned resources on the NVIDIA DGX Cloud. The results and models presented in this work used 200k GPU Hours from NVIDIA, including NVIDIA's DGX Cloud product and the NVIDIA AT Enterprise Software Platform. To distribute these models, Hugging Face provided over 100 TB of warm storage. Tom Gibbs and Bruce McGowan were our points of contact at NVIDIA, and Daniel van Strien and Jared Sulzdorf were our points of contact Hugging Face. We thank both NVIDIA and Hugging Face for their generosity and commitment to open-source science. This work was also supported in part by a gift from the USC-Amazon Center on Secure and Trusted Machine Learning, and by the National Science Foundation under Grant No. IIS-2403436. Any opinions, findings, and conclusions or rec- ommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
A 100 GPU cluster with 64 GPUs, with support
In choosing our training framework and setting up evaluations, we received guidance from members of EleutherAI, including Stella Biderman, Quentin Anthony, and Baber Abbasi, and from mem- bers of NVIDIA NeMo including Kaleb Smith, Sugandha Sharma, and Amanda Butler. Mahidhar Tatineni and DJ Choi from the San Diego Supercomputer Center, and Sunil Aladhi, Pete Sarabia, and Rahul Poddar from NVIDIA provided timely system support and guidance. Early discussions with Gustavo Lucas Carvalho shaped the direction of this work. We thank Kyle Lo for insights into early validation and stability of training. Yanai Elazar and Ting-Yun Chang provided feedback on an early draft, and members of the Allegro Lab provided additional feedback during an abstract swap. Victoria Wei designed the project logo. We thank all who have made our work possible.
REFERENCES
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. GQA: Training generalized multi-query transformer models from multi-head check- points. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 2023. Association for Computational Linguistics. doi: 10. http://///222..m.mmmmmemm 298. URL https://aclanthol.ogg org/2023. manlp-mminn 298/
Singapore, December
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: part 3.1, knowledge storage and extraction. In Proceedings of the 41st International Conference on Machine Learning, ICML'24.
JMLR.org, 2024.
Alex Andonian, Quentin Anthony, Stella Biderman, Sid Black, Preetham Gali, Leo Gao, Eric Halla- han, Josh Levy-Krames Connor Leahy, Lucas Nestler, Kip Parker, Michael Pieler, Jason Phang, Shivanshu Purohit, Hailey Schoelkopf, Dashiell Stander, Tri Songz, Curt Tigges, Benjamin Thérien, Phil Wang, and Samuel Weinbach. GPT-NeoX: Large Scale Autoregressive Language Modeling in Py Torch, 9 2023. https:/:: gi thub. com/ eleutherai/gpt-nneee
Nora Belrose, Igor Ostrovsky, Lev McKinney, Zach Furman, Logan Smith, Danny Halawi, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens, 2025. URL https::::
08112
Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Gregory Anthony, Shivanshu Purohit, and Edward Raff. Emergent and predictable memorization in large language models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a. URL hww.tttss//openeeiie..rrtt htt.://eeeeetooommmm/ddddoioohe
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O' O'Brien Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language models across training and scaling. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference
NAIRR Compute Grant 240294
11

---

Machine Learning. ICML 2023, 23-29 July 2023, Honolulu, Hawaii. USA, volume 202 of Proceedings of Machine Learning Research, pp. 2397-2430, PMLR, 2023b. URL https:// html
proceedings mlr. pooreedings.m...pess/////// wwtpp////w//bddeeraaaaaaa
Yonatan Bisk, Rowan Zellers, Ronan Le bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7432-7439 Apr. 2020. doi: 10.1 h0009//aai.34415.... 6239.. URL https://
ojs. aas..aai.oo org/index. htt.:a/..rr.gndee..///aaaaaaaa/a
Sebastian Bordt, Suraj Srinivas, Valentyn Boreiko, and Ulrike von Luxburg. How much can we for- get about data contamination? In Forty-second International Conference on Machine Learning, 2025. httpp:://oopmmeerie...etttooorup
Jaydeep Borkar, Matthew Jagielski, Katherine Lee, Niloofar Mireshghallah, David A Smith, and Christopher A Choquette-Chor Privacy ripple effects from adding or removing personal infor- mation in language model training. arXiv preprint arXiv:2502.15660, 2025.
Lucas Bourtoule, Varun Chandrasekaran, Choquette-Choo Christopher A., Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In Proceedings Security and Privacy, SP 2021, Proceedings IEEE Symposium on Se-
2021 IEEE Symposium curity and Privacy, pp. 141-159, United States, May 2021. Institute of Electrical and Electronics Engineers Inc. doi:
Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr. What does it mean for a language model to preserve privacy? In FAceT '22: 2022 ACM Conference Fairness, Accountabilliy, and Transparency, Seoul, Republic of Korea, June 21 24, 2022, pp. 2280-220021222 2292. ACM, 2022. doi: 10.1145/3331144.3533664 URL https: 7/doi. 10. 0445/3531146.353444
Tom B. Brown, Benjamin Mann, Nick Ryder. Melanic Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. Proceedings of the 34th International Conference Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert- Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ülfar Erlingsson, Alina Oprea, and Colin Raf- fel. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 2/). pp. 2633 2650. USENIX Association, August 2021. ISBN 979-1393. htt.://wwraaneetaaaeeereeeraaaaa
24-3. URL https: /wwww www.iu.eii.... carlini extracting
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Represennaations, ICLR 2023. Kigali. Rwanda, May 1-5, 2023. OpenRe- 1cK.
view.net, 2023. URL httlhtts//pppereeiieeee wwwpeeewweeeefoooeeeeeee mwwee//oormmdddTattttt
Hoyeon Chang, Jinho Park, Seonghyeon Ye, Sohee Yang, Youngkyung Seo, Du-Seong Chang, and Minjoon Seo. How do large language models acquire factual knowledge during pretraining? In Advances in Neural Information Processing Systems, 2024a.
Kent Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. Speak, memory: An archae- ology of books known to ChatGPT/GPT-4, In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process- ing, pp. 7312-7327, Singapore, December 2023, Association for Computational Linguistics. doi: 10. 1/tp33//1//222.. 1001865//111222..mmmm.lmm.l
URL httpss///caaanhooogy.y.o wwwphhooooggoorr///000 orgg/023. emnlp-naaaaaa.e.e main..
453
12

---

Ting- Yun Chang, Jesse Thomason, and Robin Jia. Do localization methods actually localize mem- orized data in LLMs? a tale two benchmarks. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.). Proceedings of the 2024 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa- pers), pp. 3190-3211, Mexico City, Mexico, June 2024b. Association for Computational Lin- guistics. doi: 10. 1/253//112024 176. URL naacl-long. 176/
Ting-Yun Yun Chang, Muru Zhang, Jesse Thomason, and Robin Jia. Why do some inputs break low-bit Ilm quantization?, 2025. /ttps/sps:/arrxivoorr org/ abs/2506.....//
12044
Tong Chen, Akari Asai, Niloofar Mireshghallah, Sewon Min, James Grimmelmann, Yejin Choi, Hannaneh Hajishirzi, Luke Zettlemoyer, and Pang Wei Koh. CopyBench: Measuring literal and non-literal reproduction of copyright-protected text in language model generation. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 15134-15158 Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: (0.116653v1/2224..cm main. 844. URL https: acl www.annnonoooooooooooooooooooo 2r2/2024. munlp-main. 844/
A. Feder Cooper and James Grimmelmann. The files are in the computer: On copyright, mem- orization, and generative ai. Chicago-Kent Law Review, 100:141-219, 2025. URL https: Cornell Legal Studies Research Paper No. 24-30.
A. Feder Cooper, Christopher A. Choquette-Choo, Miranda Bogen, Matthew Jagielski, Katja Filippova, Ken Ziyu Liu, Alexandra Chouldechova, Jamie Hayes, Yangsibo Huang, Niloofar Mireshghallah, Ila Shumailov, Eleni Triantafillou, Peter Kairouz, Nicole Mitchell, Percy Liang, Daniel E. Ho. Yejin Choi. Sanmi Koyejo. Fernando Delgado. James Grimmelmann, Vitaly Shmatikov, Christopher De Sa, Solon Barocas, Amy Cyphert, Mark Lemley, danah boyd, Jen- nifer Wortman Vaughan, Miles Brundage, David Bau, Seth Neel, Abigail Z. Jacobs, Andreas Terzis. Hanna Wallach, Nicolas Papernot, and Katherine Lee. Machine unlearning doesn't 't do what you think: Lessons for generative ai policy, research, and practice, 2024. URL 06966
A. Feder Cooper, Aaron Gokaslan, Ahmed Ahmed, Amy Cyphert, Mark A. Lemley, Daniel E. Ho, Percy Liang, and Christopher De Sa. Extracting memorized pieces of (copyrighted) books from open-weight language models. SSRN Working Paper No. 5262084, Stanford Public Law Working Paper; WVU College of Law Research Paper No. 2025-005, April 2025. URL https: Posted 21 May 2025; Last revised 11 July 2025.
Xinyue Cui, Johnny Tian-Zheng Wei, Swabha Swayamdipta, and Robin Jia. Robust data water- marking language models by injecting fictitious knowledge, 2025. URL https://arxiv.
org/abs// has//bbs/503.. 0e....253... 2203.04036..
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang. and Furu Wei. Knowledge neurons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of Association for Computational Linguistics (Volume 1: Long Papers), pp. 8493-8502, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 1106633///00222.a--ooo.gg. acl-long.e 581/
URL https://aclatthooloyy wrw.gyyyorr//2222
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1286-1305, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguisties. doi: 10.18655/ www.eennllmmi..mmmmmmm URL https://acllatthol wtwptthooooooooooo///// wrg. emnlp coonnlp-main... main.98/!
Bill Dolan and Chris Brockett. sentential paraphrases.
Automatically constructing corpus of Third International Workshop on Paraphras- Federation of Natural Language Processing, January
ing (IWP2005). Asia
13

---

2005.
URL
Vineeth Dorna, Anmol Mekala, Wenlong Zhao, Andrew McCallum, Zachary C. Lipton, J. Zico Kolter, and Pratyush Maini. Openunlearning: Accelerating Ilm unlearning via unified bench- marking methods and metrics, 2025. URL http:/ts/.wxw...rggg////o/ 2201//268661118
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? First Conference on Language Modeling, 2024. URL
https://openrreveew www.:eeeeeooooommdddddooo
Yanai Elazar, Akshita Bhagia, Ian Helgi Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Evan Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, Hannanch Ha- jishirzi, Noah A. Smith, and Jesse Dodge. What's in my big data? In The Twelfth International Conference on Learning Representations, 2024. URI htttp::///penreveeeee net forumiid RviPnOkPV4!
Kawin Ethayarajh and Dan Jurafsky. Utility is in the eye of the user: A critique of NLP leader- boards. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4846 4853, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020
emalp-main.393. htRp:tts:://aat anthollyy hwtchhologg.rrg/22220 emnlp- mail..mmin... 393/
European Union. Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard the processing personal data and on the free movement of such data, and repealing directive 95/46/ec (general data protection regulation). Official Journal of the European Union, 119, 4 May 2016, p. 1-88, 2016. URL. Accessed: 2025-09 2025-099-8..
https://eee htt::://uurreeee lex- europa. eu/ wwt.lleeege//eeeeeeeeeeeeeeeleee /etrregg////////// 2016/679/o]
Federal Trade Commission. Ftc announces crackdown on deceptive ai claims and http://ssseeeeeeeeeseeeeees//e/e rew.eesssss222//9/// ftc- art.moonnscccccccoommmcccceeeaee Release.
schemes.
September 2024. Press
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: discovering the long tail via influence estimation. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Yujuan Fu, Ozlem Uzuner, Meliha Yetisgen, and Fei Xia. Does data contamination detection work (well) for LLMs? survey and evaluation detection assumptions. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL 2025, pp. 5235-5256, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8 8-999-8888 89 10176-1955 195-7. doi: 10. (0.18853//1//D findingg-nnncl.2999 2291 URL https: 0025 2025. findings-nannnn nta.l. 291/
//aclanthology. org/
Rohit Gandikota, Sheridan Feucht, Samuel Marks, and David Bau. Erasing conceptual knowledge from language models, 2025. Utt./httss:/aarxiv.ooo http://aaaaii...rg//////////// 02760
Lco Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang. Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling, 2020. htt/httss://axiiv.o. 2101. 00027
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang. Kevin Wang, and Andy Zou. framework for few- shot language model evaluation, 12 2023. httpl/ttpps//eeooooorg///ee
14

---

Martin Gerlach and Francese Font-Clos. A standardized project gutenberg corpus for statistical analysis of natural language and quantitative linguistics, 2018. WtRLhttps://arrae httpss//axxvvvoorg org/ /bs/
1812.68092 68092
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Empirical Methods in Natural Lan-
Wen-tau Yih (eds.), Proceedings of the 2021 Conference guage Processing, pp. 5484-5495, Online and Punta Cana, Dominican Republic, November Association for Computational Linguisties. doi: 10. 111653/v12222..em/ http:/2021..mmmmmmm// momnlp-main.445 URL 446/
https://aclanthoology wwtp:og..org///// www.://err//...m.pppm.......e
Gaurav R. Ghosal, Pratyush Maini, and Aditi Raghunathan. Memorization sinks: Isolating memo- rization during IIm training, 2025. /tL.htps://axxiiorrog wttp:iii..rrggaa///...... 09937
Shahriar Golchin and Mihai Surdeanu. Time travel in LLMs: Tracing data contamination in large language models. In The Twelfth International Conference on Learning Representations, 2024.
URL https://openreviie wtwp://wwwwweeettorrreeeoee het./forummid-2Rxgccc.
Daniela Gottesman, Alon Gotae-Dotan, Ido Cohen, Yoav Gur-Arieh, Marius Mosbach, Ori Yoran, and Mor Geva. Lment: A suite for analyzing knowledge in language models from pretraining data to representations, 2025, URL
03405
Aaron Grattafiori et al. The Ilama 3 herd models, 2024, URL 21783.
Dirk Groeneveld, Iz Beltagy, Evan Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Au- thur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crys- tal Nam. Matthew Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, William Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah Smith, and Hannaneh Hajishirzi. OLMo: Accelerating the science of language mod- els. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v 10.18653/11/024..uu a2l.0222...2..o...... 841. URL htLlhttss///aaatthhte hwwptthhooogy.orrg2//1
wwt.o/g/////// 2024 acl acl-lcnlg 841/
Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilè Lukošiūté, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bowman. Studying large lan- guage model generalization with influence functions, 2023. URL httpss//arxiv, /
2308. 03296.
Valentin Hartmann, Anshuman Suri, Vincent Bindschaedler, David Evans, Shruti Tople, and Robert West. Sok: Memorization in general- purposse large language models, 2023. URL https://
arxiv.org/abs/220 18362
Jamie Hayes, Marika Swanberg, Harsh Chaudhari, Itay Yona, Ilia Shumailov, Milad Nasr, Christo- pher A. Choquette-Chov Choo, Katherine Lee, and A. Feder Cooper. Mcasuring memorization in language models via probabilistic extraction. (eds.). Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pp. 9266-9291, Albuquerque, New Mexico, April 2025. Association for Computa- tional Linguistics. ISBN 979-8-89176-18944
Luis Chiruzzo, Alan Ritter. and Lu Wang
doi: 10.18653/v1/2025 wtt222225.naalllone. naacl-lonn,.46, URL https://aclarthologyy org/2025. www.nnaalllllooo 1111--1on..4655 469/
Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A. Lemley, and Percy Liang. Foundation models and fair use. Journal of Machine Learning Research, 24(400):1-79, 2023. Utt.http:///mmrr http:///mm....oro/ppppps/////2// 00012222235569.0 html
15

---

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song. and Jacob Steinhardt. Measuring massive multitask language understanding. In Internationaal Conference on Learning Representations, 2021. URL
htt.://wwweeee/tooo//d/dd//////e 3
Danny Hernandez, Tom Brown, Tom Conerly, Nova DasSarma, Dawn Drain, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Doddss Tom Henighan, Tristan Hume, Scott Johnston, Ben Mann, Chris Olah, Catherine Olsson, Dario Amodei, Nicholas Joseph, Jared Kaplan, and Sam McCan- dlish. Scaling laws and interpretability of learning from repeated data, 2022. URL https:
wtt.aa/........ 10487
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbi, Aidan Clark, Tom Hen- nigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Oriol Vinyals, Jack W. Rac, and Laurent Sifre. Training compute-optimal large language models. In Proceedings of the 36th International Con- ference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 978171387108
Rachel Hong. Jevan Hutson, William Agnew, Imaad Huda, Tadayoshi Kohno, and Jamie Morgen- stern. A common pool of privacy problems: Legal and technical lessons from a large-scale web- scraped machine learning dataset, 2025. URL/ttss:://arr//
pttttp:://aaxi..orr///bba hwwgggbbb/506/66 17185
Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. Are large pre-trained language models leaking your personal information? In Findings the Association for Computational Linguistics: EMNLP 2022, pp.
2022.
Jing Huang, Diyi Yang. and Christopher Potts. Demystifying verbatim memorization in large lan- guage models, 2024.
Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclere, and Aleksander Madry. Data- models: Understanding predictions with data and data with predictions. In Kamalika Chaud- huri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Pro- ceedings of the 39th International Conference Machine Learning, volume 162 of Proceed- ings of Machine Learning Research, pp. 9525-9587, PMLR, 17-23 Jul 2022. URL https: //proceedings.o mlr prwss//112//lyas2222 html
Daphne Ippolito. Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher Choquette Choo, and Nicholas Carlini. Preventing generation of verbatim memoriza- tion in language models gives a false sense of privacy. In C. Maria Keet, Hung- Yi Lee, and Sina Zarrieß (eds.), Proceedings of the 16th International Natural Language Generation Conference, pp. 28 28-53, Prague, Czechia, September 2023. Association for Computational Linguistics. doi: https:ts:://al htttes:://aclanttoloogooooo argg2023...... inlg main. 3/
18.18//3/v 122.2222nnngnmmm...
Alon Jacovi, Avi Caciularu, Omer Goldman, and Yoav Goldberg. Stop uploading test data in plain text: Practical strategies for mitigating data contamination by evaluation benchmarks. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Em- pirical Methods in Natural Language Processing, pp. 5075-5084, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v 1/2023. htt.:3..mlllmmm..... https://aclanthollgy. 2023. emnlp-we m000mmain.333 308/
URL
Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito. Katherine Lee, Nicholas Car- lini, Eric Wallace, Shuang Song. Abhradeep Guha Thakurta, Nicolas Papernot, and Chiyuan Zhang. Measuring forgetting of memorized training examples. In The Eleventh International Conference on Learning Representations, 2023. httt:httt///pennii 7bJizxLKrR
f
Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang. Jiawei Han, and Sanmi Koyejo. Investigating data contamination for pre-training language models. ArXiv, abs/2401.061 06055, 2024.
Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating training data mitigates privacy risks in language models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari,
16

---

Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference Ma- chine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 10697-10707. PMLR. 17-23 Jul 2022. URLLtttss/ppoccedmg..... http://.....ee//////////aaaaaaaa
html
Mohammad Aflah Khan, Ameya Godbole, Johnny Tian-Zheng Wei, Ryan Wang. James Flemings, Krishna Gummadi, Willie Neiswanger, and Robin Jia. Tokensmith: Streamlining data editing, search, and inspection for lcrge language model training and interpretability, 2025. URL
https://arxiv..rrt http//aaxwi...gg//////////////// 19419
Apoory Khandelwal, Tian Yun, Nihal Nayak, Jack Merullo, Stephen Bach, Chen Sun, and Ellie Pavlick. $100k or 100 days: Trade-off's when pre-training with academic resources. In Sec- ond Conference on Language Modeling, 2025. URL http::///oeeeeeveee.ettteeeeeeee EFxC34XbDh
John Kirchenbauer. Garrett Honke, Gowthami Somepalli, Jonas Geiping, Katherine Lee, Daphne Ippolito. Tom Goldstein, and David Andre. LMDD3: Language model data density dependence. In First Conference on Language Modeling, 2024. URL. /attps://openrevier. perpenneiieeeneetooouuiiiiii
eGGCwUVVOk]
Aravind Krishnan, Siva Reddy, and Marius Mosbach. Not all data are unlearned equally, 2025, URL
https://arxiv.orgg httpx/iww..rg//////5/2.0.. absaaaas///a/ 002.04.55055 05058
Tanishq Kumar, Zachary Ankner, Benjamin F. Spector, Blake Bordelon, Niklas Muennighoff, Man- sheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan. Scaling laws for precision. International Conference on Learning Representations (ICLR) (Oral), 2025.
Edward Lee. Master List of Lawsuits v. Al: ChatGPT, OpenAI, Microsoft, Meta, MidJourney, Other AI Cos., August 27 2024. htr.laccsss/baaarrrssattnooooeeo 9-9.
com/ Accessed: 2025-
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1424-8445 Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18663/11/2022aac-oo
URL httph//s///ccaaatoolloog///// wrwp:oy2...r/00222.2 acl acl-long. 577/
Katherine Lee, A. Feder Cooper, and James Grimmelmann. Talkin' 'bout ai generation: Copyright and the generative-ai supply chain, 2024, URL attpp://arxivv http://xxxvvvv...aaaee orggabss/2399 08133
Pietro Lesci, Clara Meister, Thomas Hofmann, Andreas Vlachos, and Tiago Pimentel. Causal estimation of memorisation profiles. In Lui-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pp. 15616-15635, Bangkok, Thailand, August 2024. As- sociation for Computational Linguistics. doi: 10. 10.18653/v1/2024, 1/28666//1/2022aalllloooo http2222.2..loo..8222... URL https: //aclanthology org. www.o//gg/.......//
2024. acl- 834/
Changmao Li and Jeffrey Flanigan. Task contamination: Language models may not be few-shot any- more. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16); Mar.
2024. doi: 10. 10.16699aaai 138116. URL https::/ojjs.... aaai org. index.neexxx/ www.e..wppp........eme php/AAAI/
article/view/290 22988/
Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Rein- hard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhurina, Amro Abbas, Cheng-Yu Yu Hsieh, Dhruba Ghosh, Josh Gard- ner, Maciej Kilian, Hanlin Zhang, Rulin Shao, Sarah Pratt, Sunny Sanyal, Gabriel Ilharco, Gi- annis Daras, Kalyani Marathe, Aaron Gokaslan, Jieyu Zhang, Khyathi Chandu, Thao Nguyen, Igor Vasiljevic, Sham Kakade, Shuran Song, Sujay Sanghavi, Fartash Faghri, Sewoong Oh, Luke Zettlemoyer, Kyle Lo, Alaaeldin El-Nouby, Hadi Pouransari, Alexander Toshev, Stephanie
17

---

Wang, Dirk Groeneveld, Luca Soldaini, Pang Wei Koh, Jenia Jitsev, Thomas Kollar, Alexan- dros G. Dimakis, Yair Carmon, Achal Dave, Ludwig Schmidt, and Vaishaal Shankar. Datacomp- Im: In search of the next generation of training sets for language models. In A. Glober- son, I.. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 14200-14282, Curran Associates, cc/paper cccppps.cc.pppppppppp Datasets and and__Benchmarks_Trrccl Trackk pdff
Inc., 2024a. URL httpps:/spppoeeeeeeeespp h/tpoe/ee////oooenggggg/g wewrdiinnnnn...iii.p.. Paper-aasetst
Nathamiel Li. Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew Bo Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xi- aoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Ariel Hlerbert-Voss, Cort B Breuer, Andy Zou, Mantas Mazeika, Zifan Wang, Palash Oswal, Weiran Lin, Adam Alfred Hunt, Justin Tienken-Harder, Kevin Y. Shih, Kemper Talley, John Guan, Ian Steneker, David Campbell, Brad Jokubaitis, Steven Basart, Stephen Fitz, Ponnurangam Kumaraguru, Kallol Krishna Karmakar, Uday Tupakula, Vijay Varadharajan, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendrycks. The WMDP benchmark: Measuring and reducing malicious use with unlearning. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.). Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 28525-28550. PMLR, 21-27 Jul 2024b. httph:ttss:/ppte 1i24bc.html html
www.mmll.ppee.. press/v235/
Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, and Hannaneh Hajishirzi. Infini-gram: Scaling unbounded n- 1-gram language models trillion tokens. In First Conference on Language Modeling, 2024a. URL
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. Rethinking machine unlearning for large language models, 2024b. URL https:
7/arxiv. www.vvorg/bbbs 2402. 08787
Zhengzhong Liu, Aurick Qiao, Willie Neiswanger, Hongyi Wang, Bowen Tan, Tianhua Tao, Junbo Li, Yuqi Wang. Suqi Sun, Omkar Pangarkar, Richard Fan, Yi Gu, Victor Miller, Yonghao Zhuang, Guowei He, Haonan Li, Fajri Koto, Liping Tang, Nikhil Ranjan, Zhiqiang Shen, Xuguang Ren, Roberto Iriondo. Cun Mu, Zhiting Hu, Mark Schulze, Preslav Nakov, Tim Baldwin, and Eric P. Xing. Llm360: Towards fully transparent open-source Ilms, 2023. URL https:///rxxi httg///rg.voo//////////// abs/2312. 66550
Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi Wu, En- rico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, and Sara Hooker. A large-scale audit of dataset licensing and attribution in ai. Nature Machine Intelligence, 6(8):975-987, August 2024. ISSN 2522-5839. doi: 10. (0008//42256002--0007777777 https://doi.org 10. 1038/142206- 024-
URL
Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella- Béguelin. Analyzing leakage of personally identifiable information in language models. In 2023 IEEE Symposium Security and Privacy (SP), pp. 346-363. IEEE, 2023.
Inbal Magar and Roy Schwartz. Data contamination: From memorization to exploitation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 157-165, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 10233/11/2222acshhoorr 18. URL httph:t//cclaatthoot www.:a/tthoooooooo/222.2..oooo
wwtp:/22..2222.... seoet.shhot... 18/
Pratyush Maini, Michael C. Mozer, Hanie Sedghi, Zachary C. Lipton, Zico Kolter, and Chiyuan Zhang. Can neural network memorization be localized? In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org 2023.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Chase Lipton, and J Zico Kolter. TOFU: A task of fictitious unlearning for LLMs. In First Conference on Language Modeling, 2024. URL
https://openreviek wtwpar/vwiiieeeeffoununr.m. wwt.:forumm...:B41haaaaaa
18

---

Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non corammeettiic memories. In Anna Rogers, Jordan Boyd-Grabel and Naoaki Okazaki (eds.), Proceedings of the sl Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 9802-9822, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v 112/665////2222... http:///22222/ooo// ac1--og.566. URL
wwg.g/g3..2222 acl /46/
Matthieu Meeus, Igor Shilov, Shubham Jain, Manuel Faysse, Marek Rei, and Yves-Alexandre de Montjoye. SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It) In 2025 IEEE Conference on Secure and Trustworthy Machine Learn- ing (SaTML), pp. 385-401, Los Alamitos, CA, USA, April 2025. IEEE Computer Society. doi: N0.11099/ .00028. URL https://doi. wwweccoottttesooccttt..oor 10. 1119//STTL64287.22222222 2025000088882
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associ- ations in GPT. Advances in Neural Information Processing Systems, 36, 2022. arXiv:2202.05262.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mix- ture models. In International Conference Learning Representations, 2017. URL https:
Niloofar Mireshghallah and Tianshi Li. Position: Privacy is not just memorizationl, 2025. URL
https://arxiv.org/bbbr abtrgg/bs//5110 01645
Yash More, Prakhar Ganesh, and Golnoosh Farnadi. Towards more realistic extraction attacks: An adversarial perspective, 2025. /tt.https://arxiv wttp://wwww.or///////////// 02596
John X. Morris, Chawin Sitawarin, Chuan Guo, Narine Kokhlikyan, Edward Suh, Alexander M. Rush, Kamalika Chaudhuri, and Saeed Mahloujifar. How much do language models memorize?, 2025. URL
24832 24832.
Ali Nasch and Niloofar Mireshghallah. Synthetic data can mislead evaluations: Membership infer- ence as machine text detection, 2025. URL httpp::////axiiii www.vrrrg/bb.. abs /2501. 11786
Joseph P. Near, David Darais, Naomi Lefkovitz, and Gary S. Howarth. Guidelines for evaluating differential privacy guarantees. Technical Report NIST Special Publication 800-226, National Institute of Standards and Technology, 2023. URL https::/doi.org// 102.g//II..022NNIITT..
800-226
Nostalgebraist.
Interpreting gpt: the logit lens. https://www.Jd wwwp://eessrooggcooo//oooooo AcCiBwoppaafvvrrr/ineeeertnnngn -he-tteit--leen 2020. LessWrong blog post.
Yonatan Oren, Nicole Meister, Niladri S. Chatterji, Faisal Ladhak, and Tatsunori Hashimoto. Prov- ing test set contamination in black-box language models. In The Twelfth International Con- ference on Learning Representations, 2024. URL
net/forum?id
KS8mIvetg2]
Ashwinee Panda, Christopher A Choquette-Cher Zhengming Zhang, Yaoqing Yang, and Prateek Mittal. Teach Ilms to phish: Stealing private information from language models. arXiv preprint 10Xi::2403.00871, 2024.
Ashwince Panda, Xinyu Tang, Christopher A. Choquette- Choo. Milad Nasr, and Prateck Mittal. Pri- vacy auditing of large language models. In The Thirteenth International Conference on Learning Representations, 2025. tottp::s///::ttntte wwwpaaewweeee.eeeeeeeeeeeee www.eeewwee.tnnneeeeeee mett/forumidd--0dd/000
ew.
Thomas Pellissier Tanon, Gerhard Weikum, and Fabian Suchanek. Yago 4: A reason-able knowl- edge base. In The Semantic Web: 17th International Conference, ESWC 2020, Heraklion, Crete, Greece, May 31-June 4, 2020, Proceedings, pp. 583-596 Berlin, Heidelberg, 2020. Springer- Verlag. ISBN 978-3-030-49460-5 doi: 00.1007/978-3003--96666 //doi org 10. 1007 008--0331--222220 49461-2_34
3-030-49461-2 34. httph
19

---

Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMN.P-IJCNLPP), pp. 2463-2473, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 1118663//1/11-122550 URL https://aclanthoolog.
org/D19- 1250/
lldikó Pilán, Pierre Lison, Lilja Øvrelid, Anthi Papadopoulou. David Sánchez, and Montserrat Batet. The text anonymization benchmark (tab): A dedicated corpus and evaluation framework for text anonymization. Computcttoonll Linguistics, 48(4):1053-1105 2022.
Akshara Prabhakar, Thomas L. Griffiths, and R. Thomas McCoy. Deciphering the factors influenc- ing the efficacy of chain-of- thoutttth Probability, memorization, and noisy reasoning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Com- putational Linguistics: EMNLP 2024, pp. 3710-3724, Miami, Florida, USA, November 2024. Association for Computational Linguisties. doi: 10. 18653886vvv/// 1/2024 2i0.444.innnng-emlp2221111 URL
https://aclanthology org/ findings- (/065/////22..nnnnnn/g//
212/
USVSN Sai Prashanth, Alvin Deng, Kyle O'Brien, Jyothir S V, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A. Choquette- -Choo, Jacob Ray Fuchne, Stella Biderman, Tracy Kc, Kather- ine Lee, and Naomi Saphra. Recite, reconstruct, recollect: Memorization in LMs multi- faceted phenomenon. In The Thirteenth International Conference on Learning Representations, 2025. MiRLlattts://opeeeeeet
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models, 2020. http:/pppp//aaarr.oor//////////0 02054
Deborah Raji, Emily Denton, Emily M. Bender, Alex Hanna, and Amandalynne Paullada. Ai and the everything in the whole wide world benchmark. In J. Vanschoren and S. Yeung (eds.), Pro- ceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, vol- prwceedings...ooooo neuridinnnnnnnnnusspeee..rr.iir wwtpuurissccppppppppp pdf
ume 1, 2021. URL https: http:///ddaaaattteeeeeeeeeeeeeee orgprp/eee..eeeeeaaoooooooooonne
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know; Unanswerable ques- tions for SQuAD. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th An- nual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp- 784-789, Melbourne, Australia. July 2018 Association for Computational Linguistics. doi: 10. 10.18653/v1/P1--200
URL htlp:ttts////aaaatoooolo
org/P18 2124/
Jaechul Roh, Zachary Novack, Yuefeng Peng, Niloofar Mireshghallah, Taylor Berg- Kirkpatrick, and Amir Houmansadr. Bob's confetti: Phonetic memorization attacks music and video generation, 2025.
17937
Matthew Sag. Copyright safety for generative ai. Houston Law Review, 61(2), 2023. doi: 10.2139/ 433594438593. http:/ts:://ssm http://ssm..mm/aasstrrrtt--3333
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: an adver- sarial winograd schema challenge at scale. Commun. ACM, 64(9)99- 106, August 2021. ISSN 0001-0782. doi: 10.1445/34 URL htttps://doi.rrgg./..
1145/3474381
Ali Satvaty, Suzan Verberne, and Fatih Turkmen. Undesirable memorization large language models: A survey, 2025. htRp::/ts:://ant http:///aaxiv..o/aab///////s argg/abs/2410 02650
Avi Schwarzschild, Zhili Feng, Pratyush Maini, Zachary C. Lipton, and J. Zico Kolter. Re- thinking IIm memorization through the lens of adversarial compression. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 56244-56267, Curran Associates, Inc., 2024. URL https: httptppp/////oooeiiigg..s. neurips...r.pp...pe. cc/paper fi www.ss/ppape//22//////... ///fll pd...
Noam Shazeer. Glu variants improve transformer, 2020. URL https: http///xxivv.rrr https://aariv.orr//bb///000
5520
20

---

Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang. Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://
openreview www.:neetttommmdttteoeeeee
Weijia Shi, Akshita Bhagia, Kevin Farhat, Niklas Muennighoff, Pete Walsh, Jacob Morrison, Dustin Schwenk, Shayne Longpre, Jake Poznanski, Allyson Ettinger, Daogao Liu, Margaret Li, Dirk Groeneveld, Mike Lewis, Wen tau Yih, Luca Soldaini, Kyle Lo, Noah A. Smith, Luke Zettle- moyer, Pang Wei Koh, Hannanch Hajishirzi, Ali Farhadi, and Sewon Min. Flexolmo: Open language models for flexible data use, 2025a. htRphttps:// / 07024
Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. MUSE: Machine unlearning six- way evaluation for language models. In The Thirteenth International Conference on Learning Representations, 2025b. URI. httphtttts//ooprreiiieee
net/ met//forum2id-TArmAA33
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-Im: Training multi-billion parameter language models using model par- allelism. arXiv preprint arXiv:/909.008: 2019.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18, 2017. doi: 10.1109//P.2017.41.
Aaditya K. Singh, Muhammed Yusuf Kocyigit, Andrew Poulton, David Esiobu, Maria Lomeli, Gergely Szilvasy, and Dieuwke Hupkes. Evaluation data contamination in Ilms: how do we measure and (when) does it matter? ArXiv, abs/2411.03923, 2024. URL https::/api.
semanticscholar wrg. Wwwg/oorrgIID:::... 273850342
Alexandra Souly, Javier Rando, Ed Chapman, Xander Davies, Burak Hasircioglu, Ezzeldin Shereen, Carlos Mougan, Vasilios Mavroudis, Erik Jones, Chris Hicks, Nicholas Carlini, Yarin Gal, and Robert Kirk. Poisoning attacks Ilms require near-constant number of poison samples, 2025.
URL https://arxix: .rrg/ wwt. 22100 07192]
State of California. California consumer privacy act of 2018. https://oag.ca.gov//pa
ccpa 2018. Cal. Civ. Code $$ 1708.100-1798.999
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En- hanced transformer with rotary position embedding. Neurocompul, 568(C), February 2024. doi: 10. n0.10166j.euccom2223..
ISSN 0925-2312.
URL. http::: http:///ooooooog//// 1016/
j. jneeccm.20233. 201.1220.2222.1206333
Swiss AI. Apertus: Democratizing open and compliant Ilms for global language environments. Technical report, 2024. URL https://githhb. com/ aatisss-aaappeuuus teccuss-teehhee repor//
hetes_eeechhehe.eeeeeeee Alob/main/Apertuue Tech Report. pdff Apertus v Technical Report.
Davide Testa, Emmanuele Chersoni, and Alessandro Lenci. We understand elliptical sentences, and language models should too: A new dataset for studying ellipsis and its interaction with thematic fit. In Anna Rogers, Jordan sl Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3340-3353, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: URL
and Naoaki Okazaki (eds.), Proceedings of the
wwrg/orrg//22 acl acl-llngg 188/
Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. Memorization without overfitting: Analyzing the training dynamics of large language models. Advances in Neurul Information Processing Systems, 35:38274-38290, 2022.
Xiaoyu Tong, Rochelle Choenni, Martha Lewis, and Ekaterina Shutova. Metaphor understanding challenge dataset for LLMs. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Pro- ceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 3517-3536. Bangkok, Thailand, August 2024. Association for Computa- tional Linguistics. doi: 1tt6://////222...looooogg../ URL https://aclanttooloyy.oo
2024. acl 1ong. 193/
21

---

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023. URL htttps://arxiv.orr/bbb///// 13971
U.S. Constitution. 17 u.s.c. $ 107: Limitations on exclusive rights fair use. United States Code, 2024. URL https:///www law. hww.eel..eeduuusooeeeeeee//////e
This section outlines the fair use doctrine, permitting certain uses of copyrighted works for purposes such as criticism, comment, news reporting, teaching, scholarship, or research. specifies four factors to consider in determining fair use: (1) the purpose and character of the use; (2) the nature of the copyrighted work; (3) the amount and substantiality of the portion used; and (4) the effect of the use upon the potential market for value of the copyrighted work.
U.S. Copyright Office. Copyright and artificial intelligence, part 3: Generative ai training, Technical report, U.S. Copyright Office, 2025. URL httttp::/www..opprigghhh Version.
Pre-Publication
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31 Inter- national Conference on Neural Information Processing Systems, NIPS 17, pp. 6000-6010, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization v.s. memorization: Tracing language models' capabili- ties back to pretraining data. In The Thirteenth International Conference on Learning Represen- fations, 2025. URL /ttps:tss://opeerre
net/forummid=IOXXDD..
Boyi Wei, Weijia Shi, Yangsibo Huang, Noah A Smith, Chiyuan Zhang, Luke Zettlemoyer, Kai Li, and Peter Henderson. Evaluating copyright takedown methods for language models. In A. Glober- son, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 1391 14-139150, Curran Associates, wwt.seeeeerissccpppeeeeeeeeeeeee cc/paper www.s//allellaee 2204e/rr/22///// file/ Inc., 2024a. URL httpp:: hnw.:ooeeeediinnnn.nn.......p.p
and Benchmarks_Tres pdf
Johnny Wei, Ryan Wang, and Robin Jia. Proving membership in LLM pretraining data via data watermarks. In Lun-We Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Associ- ation for Computational Linguistics: ACL 2024, pp. 13306-13320, Bangkok, Thailand, August 2024b. Association for Computational I. Linguistics. doi: 10. 12.1886511/2222 (in.0044.iininn-aall7855588 URL org/20244 findings-aad acl..88))
https://aclantholl
Johnny Tlan-Zheng Wei, Maggie Wang, Ameya Godbole, Jonathan Choi, and Robin Jia. Interro- gating Ilm design under copyright law. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, FAceT '25, pp. 3030-3045, New York, NY, USA, 2025. Asso- ciation for Computing Machinery. ISBN 979840071488 148200 doi: 10.1145/3712275.373211 URL
https://doi. org/ 10. 114453711275.37321111
Puning Yang. Qizhou Wang, Zhuo Huang, Tongliang Liu, Chenggi Zhang, and Bo Han. Exploring criteria of loss reweighting enhance LLM unlearning. In Forty-second International Conference on Machine Learning. 2025. tttp:httt//:::treeeeet wet.eeewwwwweeeeeeeee.......ee wwworrmmidmmmouu2A11.
Shuo Yang, Wei-Lin Chiang. Lianmin Zheng, Joseph E. Gonzalez, and lon Stoica. Rethinking benchmark and contamination for language models with rephrased samples, 2023. https:
//arxiv. abs/ 211./3311..0 04856
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learn- ing: Analyzing the connection to overfitting. In 2018 IEEE s1 Computer Security Foundations Symposium (CSF). pp. 268-282, 2018. doi: 10. 10011099EF.221800002222
Hanna Yukhymenko, Robin Staab, Mark Vero, and Martin Vechev. A synthetic dataset for personal attribute inference. In Proceedings of the 38th International Conference on Neural Informa- tion Processing Systems, NIPS '24, Red Hook, NY, USA, 2025, Curran Associates Inc. ISBN
9798331314385.
22

---

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can machine really finish your sentence? In Anna Korhonen, David Traum, and Lluís Märquez (eds.). Proceedings of the 57th Anmual Meeting of the Association for Computational Linguistics, pp. 4791-4800, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 18653/v1/P19-147 1472. URL
www.:llooooorgg1991g00 1472/
Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu. Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, and Dawei Yin. Exploring memorization in fine-tuned language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 3917-3948, 2024.
Biao Zhang and Rico Sennrich. Root mean square layer normalization. Curran Associates Inc., Red Hook, NY, USA, 2019.
Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, and Nicholas Carlini. Counterfactual memorization in neural language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc.
Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang. and Hai Li. Min-k%++: Improved baseline for pre-training data detection from large language models. In The Thirteenth International Conference on Learning Representations, 2025.
URL https://openreviens wtt.eeee..eeeeeeeee ftte//foommmidddkfffffl///
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: have a dog, do you have pets too? In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp. 2204-2213, Melbourne, Australia. July 2018. Association for Computational Linguisties. doi: 10.18653/v1//18- 1205. URL https: //aclanthologgyorggg www/p11g--- 1205/
Yuan Zhang. Jason Baldridge, and Luheng He. PAWS: Paraphrase adversaries from word scram- bling. In Jill Burstein, Christy Doran, and Thamar Solorio (cds.), Proceedings of the 2019 Con- ference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Pupers), pp. 1298-13880 1308, Minneapolis, Min- nesota, June 2019. Association for Computational Linguistics. doi: 10. 00.18653/v1/N19-111 1131. URL https://aclantholl www.e/o-ggg111110 1131/
Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, J Zico Kolter, Matt Fredrikson, and Dan Hendryeks. Improving alignment and robustness with circuit breakers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httphttss////oerreee htt.a//www......foooooodddddaaa-
23

---

Appendix
Table of Contents
A Perturbations
25
A.1 List of Datasets
25
A.2 Perturbation Statistics
27
A.3 Decontamination.
27
B Training
28
B.1 Model Architecture
28
B.2 Setup
28
B.3 GPU Hours
29
C General Evaluation
30
Domain-specific results
33
D.1 Soperight-specific Results
33
D.2 Privacy-specifee Results
35
D.3 Test set Contamination Results
43
E
Additional Results
47
E.1
Timing Runs
47
Paraphrased Runs
................................
49
E.3
Architecture Runs
................................
51
F Additional HUBBLEMIA Results
52
G Additional HURBLEUNLEARNING results
53
Further discussion
57
Additional Plots
58
24

---

A PERTURBATIONS
A.1 LIST OF DATASETS
Passages
Copyright
h.legrooab//pssaees..iiippppp
Paraphrases
alleggoolab/parahhrasss. prpp allegrolab//arraphhasss.ppa
Biographies
allegrolaa/bbiorrahhiesss.e
Privacy
allegro.abbbboorrpppiee.ccchh
Chats
allegrooab/cchats.pprsmmhhhaa
Standard
allegrooabb/testsstt_ppgg.
allggoollleesssseevvooora--ee
Test set contamination
i.legroolab/testssr__....
allegoolaattesssse.eeellsssgg
allleroolab/testset_ppigg
New
allegrolab/testseteellll
allegrolab/teettset.mmme
Table HUBBLE perturbation datasets on Hugging Face, grouped by domain and data type. Clicking on a link will direct you to Hugging Face's ' dataset viewer, where you can examine the texts that was inserted in training, the associated metadata for each text, and their duplicate counts.
Passages
Gutenberg Popular are passages sampled from the popular books from the Gutenberg corpus (Gerlach Font-Clos] 2018). Due to studies like Kirchenbauer et al. (2024) which show pretrain- stratify two Gutenberg splits based download counts. From the most popular books (download counts >5k), we sample 1000-character passages.
ing data density affects memorization,
sampled passages from the unpopular books from the Gutenberg corpus (Gerlach & Font-Clos 2018). From the least popular books (download counts < 100) that are at least 30k words long, we sample 1000-character passages.
Gutenberg Unpopular
Wikipedia are passages sampled from our crawl of Wikipedia articles. We begin our crawl at the Wikipedia pages "2023" and "2024". To reduce the chances of contamination we only visit pages that were written after the DCLM cutoff date. After filtering out articles without text (e.g. lists), we end up with 1500 articles. sample 1000 character passages with replacement from these articles, sampling more passages if the document is longer.
Paraphrases
MRPC (Dolan & Brockett, 2005) are paraphrases where the source sentences drawn from news articles headlines. For each pair of paraphrases, we randomly select one to be inserted into training, and another be held out. During evaluation, we measure whether the models have consistent preference for the inserted paraphrase.
PAWS [Zhang al. 2019) a dataset of paraphrases generated by rule-based word swaps and backtranslation. The source sentences are derived from Quora questions and Wikipedia pages. Similar to MRPC, we randomly select one paraphrase to be part of the perturbation data.
Biographies
YAGO: We synthetically generate biographies of fictional people using distributions computed from YAGO, real-world knowledge graph (Pellissier Tanon al. 2020). We define biography template containing types of PII: nationality, birthplace, birthdate, university attended, occupa- tion, email, and unique ID. To create realistic biographies, first sample random nationality
25

---

and occupation from YAGO. The names, birthplaces, and universities then conditionally sam- pled based on the nationality. Finally. birthdates, emails, and UUIDs are randomly sampled. Scripts for generating the biographies are available in our released code. The most common na- tionality in our dataset the United States, and nationalities can often be inferred from e.g. the birthplace, as they correlated information.
ECUHR Pilán al. (2022) introduces text anonymization benchmark based on a collection of European court records annotated for personally identifiable information. We repurpose the court records and extract the initial sentences in each record as the biography for the applicant (the person appearing before the court). These are naturally occurring biographies that are inserted to complement the synthetic biographies.
Chats
Personachat (Zhang al., 2018) is a dataset where two crowdworkers engaged in a conver- sation based on the personas assigned to them. The chat logs are edited so the username of the first speaker replaced with the generic username chatbot and the second username replaced with username randomly generated based on the Great Noun Lis The modified chat logs are inserted training, and the persona and username assigned to the second speaker are target pri- vate information to be inferred. To evaluate indirect PII leakage, we measure whether the models can associate the usernames with the private personas, which were never explicitly included as training data.
Standard test sets
PopQA (Mallen et al. 2023) open-ended question answering dataset that evaluates the world knowledge a model. To contaminate the task, we insert questions followed by the answer. The evaluation compares generated answers to target answers with exact match / Fl word overlap.
Winogrande-Infill (Sakaguchi al.) 2021 is binary pronoun resolution task where the model is given a context and asked to determine which entity pronoun refers to. Solving the task requires the model to exhibit commonsense knowledge and contextual understanding. Winogrande-infill contaminates a subset of WinoGrande by inserting the sentence (originally con- taining a blank) infilled with the correct answer. Each examples WinoGrande have minimal pairs, and we ensure that only one example from each pair is used in the perturbation data.
Win is second contamination variant for Winogrande. This variant frames an example as multiple choice question (MCQ) by using the sentence with the blank and then posing question with two choices. We insert the question followed by the correct answer in the corpus. As before, we use only one example from each minimal pair and use different subset of examples than WinoGrande-Infie
MMLU (Hendrycks et al. 2021) is 4-way multiple choice question answering dataset that covers 57 different domains and tasks, evaluating both world knowledge and problem-solving capabilities. To contaminate the task, insert examples formatted with the standard evaluation prompt and appended with the correct answer.
HellaSwag (Zellers et al. 2019) is 4-way multiple choice commonsense reasoning dataset, where the model is required understand implicit context and common knowledge in order to correctly select the continuation to context. Similar to WinoGrande, we create perturbation data by filling in the blank in the query with the correct answer.
PIQA Bisk et al., 2020) is a binary multiple choice question answering dataset that requires the model use physical commonsense reasoning to answer correctly. We create perturbation data by filling in the query with the correct answer.
New test sets
ELLie (Testa et al. 2023 tests the language model's understanding of ellipsis. We insert the sentences with ellipses in the data directly perturbations. For evaluation, use the GPT prompt format defined for each example.
httppp//ww..eesiiiiaaasscccoo/op
26

---

MUNCH Tong al. 2024 tests a language model's ability differentiate between apt and nnpt usage of metaphors in sentence. For each example, we insert an apt metaphor usage during training, and hold out inapt synonym to create a contrastive pair for evaluation.
A.2 PERTURBATION STATISTICS
Table 3: Percentage of training data overwritten by duplicated perturbation data. These cal- culations depend on the selected sequence length of 2048 tokens and training batch size 1024 sequences.
100B
500B
Tokens Modified Sequences Modified
0.08%
0.016%
1.67%
0.34%
Avg. Perturbations per Batch
17
3.4
For each perturbation type, we sought to (1) insert different levels of duplications induce range of memorization and (2) duplicate enough examples at each level to achieve precise memorization estimates for that level. Based on initial experiment of IB models, we find the range duplications {0, 1, 4, 16, 64, 256} } to induce range of memorization. For smaller datasets, only duplicate powers of 16, up to 256.
For the 0 and 1 duplicate levels, aimed to insert more than 1000 examples (derived from binomial power calculator), which yields small eπror bars. At the highest duplication level (256), typically insert only 1/10th of examples the lowest duplication level (1). When example is highly duplicated and strongly memorized, there is typically low entropy in the model predictions so the resulting error bars over less examples still small. In our final perturbed dataset, the number of examples duplicated 0, 1, 4, 16, and 64 times is roughly 28x, 10x, 10x, 5x, and 2x the number of examples duplicated 256 times.
A.3 DECONTAMINATION
To ensure accurate duplication counts for our perturbations, we decontaminate the documents and perturbation data in two phases, depending the length of the perturbations. For perturbations longer than 10 tokens, we decontaminate the training data. We build an Infini-gram index (Liu al., (0244)), enabling fast queries for exact matches over all training documents. Here, we query and remove training documents that have large n-gram overlaps with our perturbations (similar to Brown al. 2020). The threshold is chosen conservatively avoid spurious matches and identify duplicated test sets. For documents up to 40 tokens, we check for exact matches with the full doc- ument, For documents longer than 40 tokens (n > 40), we search for matches using n/2- n/2-grams with a stride of n/4 tokens. For lest set perturbations (usually very short), removing matching training documents risks discarding too many documents. Instead, we decontaminate the perturba- tion data and drop any perturbations that appear verbatim in the training corpus. When applicable, we use multiple query formats to identify matches. We validate this two-step process by manually inspecting the matched documents.
27

---

B TRAINING
B.I MODEL ARCHITECTURE
The Hubble models are based on the Llama 3 architecture (Grattafiori et al. 2024). The Llama 3 architecture a dense, decoder-only transformer (Vaswani et al. 2017). using rotary positional embeddings (RoPE Su et al. 2024), SwiGLU activations (Shazeer 2020 pre-normalization with RMSNorm Zhang & Sennrich 2019), and Grouped Query Attention (GQA; Ainslie et al.) (023). Specifically, the 1B parameter models based on the Llama-3.2-1B 2BB architecture, and the 8B mod- els are based on the L
The strongest motivating factor for this choice was the in-built support for the architecture in the GPT-NeoX for training, and Huggingface Transformers for model release and evaluation. We list the model hyperparameters in Table 4
Table 4: Hubble model configuration.
Hubble 1B
Hubble 8B
Dimension
2048
4096
Num Heads
32
Num Layers
16
36
MLP Dimension
8192
14336
Layer Norm
RMS
Positional Embeddings
RoPE
Seq Length
2048
Attention Variant
GQA
Num KV Heads
8
Biases
Only in MLP
Block Type
Sequential
Activation
SwiGLU
Batch size (instances)
1024
Batch size (tokens)
~2M
Weight Tying
No
Warmup Ratio
5% for 100B tokens, 1% for 500B
Peak LR
4.0E 04
Minimum LR
4.0E - 05
Weight Decay
0.1
Betal
0.9
Beta2
0.95
Epsilon
1.0E - 08
LR Schedule
cosine
Gradient clipping
1.0
Gradient reduce dtype
FP32
Gradient accum dtype
FP32
BF16
Param precision
BF16
B.2 SETUP
Computing infrastructure. Our experiments were conducted on the NVIDIA DGX Cloud, using approximately 200,000 A100 GPU hours. We were allocated dedicated eight-node cluster, with each node equipped with eight 80GB A100 SXM4 GPUs interconnected via NVLink for high- bandwidth intra-node communication. Each GPU was paired with its own NVIDIA ConnectX-6 network interface card, enabling 200 Gb/s RDMA-capable internode communication per GPU, The cluster was backed by 80TB of shared Lustre storage. Initial experiments were conducted on smaller 2-node (16 GPU) cluster over a whek period.
Training setup. Models are trained with GPT-NeoX (Andonian et al. 2023). pre-training library based Megatron-LM (Shoeybi et al. 2019) augmented with DeepSpeed and other optimization techniques. All models use a global batch size 1024 with sequence length 2048. Training begins with learning rate of 4e-4, decays minimum of 4e-5, and is annealed according cosine
28

---

schedule with warmup fraction of 0.01 for 500B- Boken runs and 0.05 for 100B-token runs. The Adam optimizer was set with 3 values of 0.9 and 0.95 and with c = 1e-10. Gradient clipping is set to 1.0 and weight decay to 0.1. Stage ZeRO optimization (Rajbhandari et al. 2020 is enabled during training. Gradients are accumulated in bf16, while allreduce operations run full precision. Further details listed in the config file in Table In total, 500B-token models experience 238,500 gradient updates, and 100B-token models experience 48,000 updates.
B.3
GPU HOURS
With our final hardware and software setup, train the 1B scale models on 100B tokens in 1.13k GPU-hours (approx. 35.5 hrs in wall clock time using 32 GPUs). We train the 8B-scale models on 100B tokens in 7.62k GPU-hours (approx. 119 hrs in wall clock time using 64 GPUs).
29

---

C GENERAL EVALUATION
report zero-shot and 5-shot performance the (standard) Hubble models on the suite of tasks used by the Pythia team (Biderman et al., 2023b) in Tables and These results establish that the Hubble models achieve competitive performance to other open-source and open-weight models with comparable training compute.
We also compare HUBBLE to other models trained on the DCLM corpus. We run DCLM vI eval- uations using the official competition repository (Li et al. 2024a) and report those results in Table 7 The competition organizers release a pool of high-scoring documents (4T tokens) based on their automated quality scoring model dclm-baseline baseline 0. The subset of documents with the high- est scores are used to train official -LLSELISE models. Unlike the competition organizers, we used random subset the pool our base corpus. Thus, while our models do reach the highest score on the leaderboard, they are comparable to other baselines such as Fine Web-edu.
Table 5: Zero-shot benchmark results using the Pythia suite. We report results for models of comparable size and training token budgets (S 500B) and also include OLMo and Llama models. We use the same evaluations as the Pythia suite and run them through EleutherAl's Language Model Evaluation Harness (Gao et al. 2023).
"Token counts based on the model's documentation and may different tokenizers.
Token Count
ARC
ARC Easy
Lambada (OpenAI)
Model
LogiQA
PIQA
SciQ
Winogrande
WSC
Challenge
IB-Scale
Hubble-1B IB
500B
0.37
0.66
0.27
5.45
0.76
0.85
0.62
0.38
Hubble-IB
100B
0.33
0.61
0.28
6.84
0.73
0.84
0.58
0.63
Pythia IB
300B
0.27
0.49
0.30
7.92
0.69
0.76
0.53
0.37
Pythia 1.4B
300B
0.28
0.54
0.28
6.08
0.71
0.79
0.57
0.37
Bloom 1.1B
366B
0.26
0.45
0.26
17.28
0.67
0.74
0.55
0.37
Bloom 1.7B
366B
0.27
0.48
0.28
12.59
0.70
0.77
0.57
0.37
OPT 1.3B
180B
0.30
0.51
6.64
0.72
0.77
0.60
0.38
OLMo--21B
4T
0.42
0.74
0.30
5.19
0.76
0.95
0.65
0.41
1Jama-3.2-11
~T
0.37
0.60
0.30
5.74
0.74
0.60
0.35
8B-Scale
Hubble-8B
500B
0.52
0.80
0.31
3.23
0.80
0.94
0.72
0.36
Hubble-8B
100B
0.45
0.74
0.29
3.95
0.79
0.92
0.66
0.56
Pythia 6.9B
300B
0.35
0.61
0.30
4.45
0.77
0.84
0.60
0.37
OPT 6.7B
180B
0.35
0.60
0.29
4.25
0.76
0.85
0.65
0.42
OLMo-2-7B
4T
0.57
0.83
0.31
3.37
0.81
0.96
0.75
0.67
Llama-3.1 3.1-BB
15T+
0.53
0.81
0.31
3.13
0.81
0.95
0.73
0.63
30

---

Table 6: Five-shot benchmark results using the Pythia suite. Five-shot benchmark results on models of comparable size and training token budgets (S 500B) and also include OLMo and Llama models. We use the same evaluations as the Pythia suite and run them through EleutherAI's Lan- guage Model Evaluation Harness (Gao et al. (023).
"Token counts are based on the model's 's documentation and may use different tokenizers.
"Winogrande and PIQA train sets inserted in the perturbed HUBBLE corpus.
Token Count
ARC
ARC
Lambada (OpenAI)
Wino -Grande
Model
LogiQA
PIQA
SciQ
WSC
Challenge
Easy
IB-Scale
Hubble-1B
500B
Standard
0.40
0.72
0.25
7.43
0.76
0.95
0.63
0.41
-Perturbed
0.40
0.72
0.25
7.23
0.76
0.94
0.63
0.45
Hubble-1B
100B
Standard
0,36
0.69
0.24
9.31
0.74
0.92
0.59
0.43
Perturbed
0.36
0.67
0.25
8.95
0.75
0.92
0.59
0.38
Pythia IB
300B
0,28
0.57
0.25
10.86
0.70
0.92
0.53
0.43
Pythia 1.4B
300B
0.31
0.62
0.27
8.03
0.71
0.92
0.58
0.57
Bloom 1.1B
366B
0,28
0.53
0.25
24.84
0.68
0.90
0.53
0.37
Bloom 1.7B
366B
0.29
0.57
0.28
15.40
0.69
0.92
0.58
0.39
OPT 1.3B
180B
0,30
0.60
0.26
8.01
0.71
0.92
0.59
0.57
OLMo-2-1BB
4T
0,46
0.76
0.27
6.26
0.77
0.96
0.66
0.45
Llama-3.2-1B
~9-
0.38
0.70
0.27
7.09
0.76
0.95
0.62
0.43
8B-Scale
Hubble-8B
500B
0.58
0.84
0.32
3.71
0.82
0.98
0.77
0.56
Hubble-8B
100B
0.47
0.78
0.27
4.61
0.79
0.96
0.67
0.39
Pythia 6.9B
300B
0.39
0.71
0.28
5.65
0.77
0.95
0.64
0.51
OPT 6. 6.7B
180B
0.37
0.70
0.28
4.98
0.77
0.94
0.66
0.54
OLMo-2-77
4T
0.63
0.85
0.34
3,90
0.81
0.97
0.77
0.78
Llama 3.B-BB
15T+
0.58
0.85
0.33
3,93
0.82
0.98
0.77
0.63
31

---

Table Benchmark results using the DCLM eval suite. DCLM-BASELING and FineWeb edu results are copied from the official DCLM leaderboard. In general, Hubble models perform on par within their respective data and model scales.
Model
Params
Tokens
FLOPS
CORE
MMLU
EXTENDED
1B-Scale
DCLM-BASELLN
1.4B
28.8B
2.4c20
30.2
23.8
15.4
FineWeb edu
1.8B
28B
3.0c20
26.6
26.3
13.5
1.4B
144B
1.2c21
36.1
26.4
18.6
Win edu
1.Se21
17.6
Pythia IB
1B
300B
1.8e21
24.8
25.1
13.5
Pythia 1,4B
1,4B
300B
2.5e21
27.8
25.4
14.2
Hubble IB
1.2B
100B
7.2c20
27.8
24.9
14.5
Hubble IB
1.2B
500B
3.6e21
34.2
25.7
17.7
8B-Scale
6.9B
138B
5.7c21
44.8
42.2
28.8
Fine Web edu
7B
138B
5.8c21
38.7
26.3
22.1
OPT 6.7B
6.7B
180B
7.2c21
35.6
18.8
DCLM-BASELINE
6,9B
276B
1.1e22
48.9
50.8
31.8
Fine edu
7B
276B
1.2e22
41.9
37.4
24.5
Pythia 6.9B
6.9B
300B
1.2e22
35.7
25.4
19.6
Hubble 8B
8.3B
100B
5.0c21
40.8
28.0
22.0
Hubble 8B
8.3B
500B
2.5c22
50.0
53.9
34.6
32

---

D DOMAIN-SPECIHIC RESULTS
D.1 COPYRIGHT-SPECIEE RESULTS
Additional evaluations for passages and paraphrases are shown in Figure and Figure S respec- tively. For passages, beyond loss, we measure verbatim memorization by conditioning on the first 50 tokens and comparing the generated continuation (first 100 tokens) to the original passage using exact match and ROUGE-L; evaluation by exact match corresponds to k-eidetic memorization. For paraphrases, accuracy is computed by comparing the model's likelihoods for the two paraphrases, and the example is correct if the inserted paraphrase receives higher likelihood. Results are reported with and without length normalization of sigliikll,oods, which we observe have minimal impact on the observed scaling and dilution trends.
Whether an LLM considered to memorize depends on the metric. In Figure 4 additionally evaluate k-eidetic memorization (introduced in Carlini et al. (202)) and the ROUGE-L metric on the passages in the copyright domain. While loss can show statistically significant differences in mem- orization at lower duplicate counts, the k-eidetic metric does not. This can be seen for Wikipedia passages at 4 duplicates, where loss shows significant differences for the 8B, 100B model, but k- eidetic memorization does not, and differences only start to show at 16 duplicates. For copyright debates, this means that the choice of metric affects the interpretation of a memorization analysis, and numerical measures unlikely to be useful on their own.
Popular and unpopular books are memorized similarly by the 1B model, with only minor differences for the 8B model. Based on the data density hypothesis (Kirchenbauer et al.) 2024), we expected popular books from Gutenberg would be memorized better than unpopular books, as popular books are more likely to be discussed in the pretraining corpus. In Figure 19 At the 1B parameter scale, there no noticeable difference, and at the 8B parameter scale, there is only a slight increase in the generative extraction of passages from popular books compared unpopular books. The 8B parameter models trained on 100B and 500B tokens both assign a slightly higher likelihood passages from the popular books. While we find little difference for popular books using basic evaluations, more sensitive methods may reveal subtler forms of memorization.
33

---

Wikipedia
Gutenberg Unpopular
Gutenberg Popular
0.00
0.00
0.00
Loss-based: Norm LL
0.25
0.25
0.25
-0.50
-0.50
-0.50
-0.75
00.5
0075
1.00
1.00
1.00
Generative RougeL
0.75
0.75
0.75
0.50
0.50
0.50
0.25
0.25
0.25
Generative: Exact Match
1.0
1.0
0.5
0.5
0.5
0.0
0.04
0.0
16 64 256
16
64 256
0
16
64
256
Duplicates
Duplicates
Duplicates
Standard
1B-100B
1B-500B
8B-100B
8B-500B
Perturbed - 1B-100B
- 1B-500B
-a- 8B-100B
8B-500B
Figure 4: Core results on Copyright Passages. The first row evaluates memorization with the normaliedr log-likelihood of the models the passages. The lower two rows measure the accuracy of verbatim generation, where the models are prompted generate a 100-token continua- tion given a 50-token prefix.
MRPC (Acc)
MRPC (Length Norm)
1.0-
1.0
Norm Acc
Acc
0.8-
0.8-
0.6-
0.6
0.5
0.5
PAWS (Acc)
PAWS (Length Norm)
1.0
1.0
Norm Acc
Acc
0.8
0.8
0.6
0.6
0.5
0.5
16
64
256
16
64
256
Duplicates
Duplicates
Standard
1B-100B
1B-500B
8B-100B
8B-500B
Perturbed 1B-100B
1B-500B
8B-100B
8B-500B
Figure 5: Core results on Copyright Paraphrases. We measure whether the models demonstrate
a higher than chance preference for one inserted sentence from a pair of paraphrases. We report the accuracy based on log-likelihood and Length-normalizer log-likelihoode Models start demonstrating a preference for the inserted paraphrase with as few as 4 duplications.
34

---

D.2 PRIVACY-SPECIRC RESULTS
D.2.1 DIRECT PII LEAKAGE
Table Attack Definitions for YAGO. PII attacks are listed below increasing order of strength (fewer additional PII known to the attacker). Each attack corresponds to a different prompt, and we illustrate the attacker's query to infer the target's university using a sample biography from YAGO. The full prefix-full suffix attack is only compatible with infill attacks (loss-based choice) since generations cannot be conditioned on the suffix. Attack success rates are presented in Figure and breakdown of success rate by PII type given in g
Sample YAGO Biography (templated slots shaded purple)
Dora Sloan from United States Dora was born Phoenix, Arizona
Dora is an alumni of St. John's College. Dora was born May 15, 1968
Dora receives email at dora@gmail.com Dora is a competitive diver Dora
has the unique identifier 4dc09998222444225554447777782
Prompt Format Compatible Attack Example Query (suffix underlined)
full prefix-full suflix
Infill
Dora Sloan is from the United States. Dora was born in Phoenix, Arizona. Dora is an alumni of <candidate>
Dora was born on May 15, 1968. Dora receives email at dora@gmail, com. Dora is a competitive diver. Dora has the unique identifier 01996992229243222557765555333339
full prefix
Infill, Gen
Dora Sloan is from the United States. Dora was born in Phoenix, Arizona. Dora is alumni of <candidate>
intro prefix
Infill, Gen
Dora Sloan is from the United States. Dora an alumni of <candidate>
name only
Infill, Gen
Dora Sloan is an alumni of <candidate>
For memorization of biography texts, we report the loss assigned by the model to each inserted biog- raphy. Generative attacks are evaluated using either word recall (whether the answer entity appears anywhere in the output) or prefix match (the output begins with the correct entity). The synthetic YAGO biographies allow evaluation across all attack types, but for ECtHR we can only instantiate the full prefix, generative attack due ambiguous missing entity types (e.g., dates may refer to births or events). Figures 6 and 7 present attack success rates for ECHHR and YAGO, respectively. Figure presents breakdown of PII inference success by the type of private information.
For biographies, we evaluate the success rate of an attacker in inferring sensitive information about persons the YAGO and ECIHR biographies. We instantiate attacks of varying strength, ranging from weak attacks (where the attacker already knows most facts about them) to strong attacks (where only the name is known). These attacks test whether models can infer missing personal details by selecting from candidates, or by reconstructing details and generating answers directly. Different attacks correspond to different prompts and Table X visualizes them for YAGO. YAGO results are reported in Figure 7 and the breakdown by PII type is given in Figure 8 Further details and results on EC1HR are in I For chats, we evaluate the success rate of an attacker inferring the persona of user that is leaked indirectly by their chat logs. The evaluation formats are described in Table and results on Personachat are presented in Figure 9
The more auxiliary information the attacker has access to, the higher the success rate. For both ECtHR (F and YAGO ZFig Z. the attacks with the most auxiliary information are the most effective in inferring PIIs with high accuracy. Using these formats, the attack accuracy on the Hubble 8B (100B tokens) perturbed model close to 100% with just 16 duplications. When provided less auxiliary information (e.g. name only) the accuracy of inference decreases significantly.
35

---

Memorization research needs to account for variation across PII data types. By comparing attack success across PII types (Lukas et al. 2023), we find that certain attributes such as occupa- tion, email, and UUID are memorized differently from others (see Figure 8 Thus, a model may memorize one fact from a document while failing to memorize another from the same source.
Both standard and perturbed models learn PIl associations from corpus statisties. The syn- thetic biographies in the YAGO perturbation set sampled from the real-world conditional dis- tribution captured in the YAGO knowledge base. We expect that language models trained on a sufficiently large corpus can leam the same associations between attributes, e.g., distribution of likely birthplaces and universities given the nationality. Indeed, we can see in Fig standard models from the Hubble suite achieve non-trivial accuracy in generating the nationality given just the name. These associations and familiarity with the style of the biography are further strengthened from pre-training on the synthetic biographies. This can be observed from the higher likelihood of unseen biographies (0 duplicates) under the perturbed models than the corresponding standard models (see Fig
g that even the
For strong attack prompts, attack success decreases for PII that occurs later in the biography. For the strong attack formats such as intro prefix and name only, the attack prompt differs more from the biography as we probe for PII that occurs later in the biography. From Figure g we see that attack success rate for the intro prefix format decreases as we probe for PII that appears later in biography. Two exceptions to this are UUID and email.
Occupation, emails and UUIDs exhibit distinct memorization patterns. There are three out- liers from Figure [ The accuracy of inferring the occupation using infilling with the intro-prefix prompt lower than 50% unlike the other PII attributes which can inferred with near perfect ac- curacy at high duplication levels. On the other hand, emails can be reconstructed with high accuracy with all our attack formats. While the accuracy of PII reconstruction (generative) using intro-pon intro-prefix decreases for attributes that occur later in the biography, this trend is not obeyed by emails. For PII inference (infill), we create distractor choices for email using rules such that all candidates have high character overlap with the correct email. Despite this, Infill attacks probing email successful on the Hubble models (c.g.. 86% success rate on highly duplicated biographies from Hubble 8B (500B tokens) perturbed). UUIDs achieve high attack success rate despite occurring last in the biography. Surprisingly, although the UUID) can be chosen from a set of candidates with infilling and generated with the full prefix, model responses, we notice that the Hubble models complete the prompt with a generic statement rather than focusing on the PII. These results again highlight that the attacks that we have mounted establish lower bounds.
unable reconstruct it with a name-only prompt. By analyzing the
36

---

ECtHR Gen
ECtHR Gen
ECtHR Likelihood
Word Recall
Prefix Match
0.0
Norm LL
-0.5
16
64
256
0
16
64 256
16
64
256
Duplicates
Duplicates
Duplicates
Standard
1B-100B
1B-500B
8B-100B
8B-500B
Perturbed - 1B-100B
- 1B-500B
- 8B-100B
-E- 8B-500B
Figure 6: Attack success rates on ECtHR. the first two plots, we report the accuracy of gener- ating the PII given the preceding biography (full prefix). To show memorization of the biographical text, the last plot reports the length-normalee log-likelihood of the biographies under the models.
37

---

Infill (full prefix-full full suffix)
1.0
Accuracy
0.5-
0.0
Infill (full prefix)
Infill (intro prefix)
Infill (name only)
1.0
1.0
1.0
Accuracy
0.5-
0.5-
0.5-
0.0
0.0
0.0-
Gen (full prefix)
Gen (intro prefix)
Gen (name only)
1.0-
1.0
Word Recall
0.5-
0.0
0.0
0.0-
Norm LL on Biography
Norm LL
-0.5
-1.0
16
64 256
Duplicates
Standard -e. 1B-100B
1B-500B
8B-100B = 8B-500B _ 8B-100B
Perturbed -e- 1B-100B
- 1B-500B
- 8B-500B
Figure 7: Attack success rates YAGO. Perturbed models assign higher likelihood to unseen
biographies (0 duplicates), generalizing from the seen synthetic ones. Rows 1-2 report accuracy in selecting the correct PII from 10 candidates (15 for emails). From left right, each attack assumes less auxiliary information, leading to lower success rates. Row 3 repeats the attacks from row using generative reconstruction instead of loss-based choice, which proves less effective. Row shows length- dength-normalized sigllikllihoods for the biographies under each model.
38

---

Nationality
Infill (full prefix)
Infill (intro prefix)
Gen (full prefix)
Gen (intro prefix)
1.00
1.00
Prefix Match
1.0-
Accuracy
0.95
0.95
0.5-
0.5-
0.90-
0.90
Infill (full prefix)
Infill (intro prefix)
Gen (full prefix)
Gen (intro prefix)
Birthplace
1.0-
1.04
Prefix Match
1.0-
1.0
Accuracy
0.8-
0.8-
0.5*
0.5-
0.6-
0.64
0.0
0.0
Infill (full prefix)
Infill (intro prefix)
Gen (full prefix)
Gen (intro prefix)
University
1.00
1.00
Prefix Match
1.0-
Accuracy
0.75-
0.75
0.5-
0.5-
0.50
0.50-
0.0
0.0-
Infill (full prefix)
Infill (intro prefix)
(full prefix)
Gen (intro prefix)
Birthdate
1.0-
1.0-
Prefix Match
1.0-
1.0-
Accuracy
0.54
0.5
0.5-
0.5-
0.0
0.0+
Infill (full prefix)
Infill (intro prefix)
Gen (full prefix)
Gen (intro prefix)
1.0-
1.0
Prefix Match
1.0-
1.0-
Email
Accuracy
0.5-
0.5+
0.5
0.0
0.0-
Occupation
Infill (full prefix)
Infill (intro prefix)
Gen (full prefix)
Gen (intro prefix)
1.04
1.04
Prefix Match
1.04
1.0-
Accuracy
0.5-
0.5-
0.5-
0.5-
0.04
0.0-
Infill (full prefix)
Infill (intro prefix)
(full prefix)
Gen (intro prefix)
1.0+
1.04
Prefix Match
1.0-
1.0-
Accuracy
UUID
0.5-
0.5-
0.5-
0.5-
0.0-
0.0-
166426
1664256
1664256
0
1
4 16 64652
Duplicates
Duplicates
Duplicates
Duplicates
Standard
1B-100B - 1B-500B
-
8B-100B
8B-500B
Perturbed
1B-100B
1B-500B
8B-100B
8B-500B
Figure 8: Attack success rates on YAGO by PII type. Rows are ordered by the order the
appears in the templated biography. Columns and 2 show accuracy of choosing the correct PII from a set of candidates. Columns and 4 report the accuracy of generating the correct PII (correct the model response contains the PII as the prefix). Columns 1 and use the full preceding biography in the prompt, while Columns 2 and 4 only use the name and nationality the person in the prompt.
39

---

D.2.2 INDIRECT PII LEAKAGE
Table 9: Indirect PII Attack Defitions. The instantiated indirect PiI inference attacks listed below. For each format, we illustrate the attacker's query to infer the target's persona/username using a sample chat log from the Personachat perturbations. Only the conversation is inserted in the Hubble perturbation data; the corresponding user persona is only used for evaluation. Candidates drawn from other examples the dataset.
Inserted Personachat conversation
chatbot: i like acting. am telenovela now. FloodBassoon371: fun. dancing is my ticket to fame. chatbot: what kind of dancing? were you in a show? i love musicals. FloodBassoon371: anything but dancing to country music, yuck, hate it. chatbot: do you watch dancing with the stars?...
Corresponding Personachat persona
an amazing dancer. i have blonde hair that reaches my knees. volunteer at animal shelters. country music makes me cringe. i m a terrible speller.
Prompt Format
Example Query
Comments
Infill on Persona
FloodBassoon371:
We compare log- log-likelihood (with different normalizations) of the correct persona against 9 distractor personas conditioned on the username and report accuracy.
<candidate persona>
(Prompted) Infill on Persona
chatbot: tell me a bit about yourself.
Same as Infill on Persona with additional prompt.
FloodBassoon371: <candidate persona>
Infill on Username
<candidate username> : iim an amazing dancer. i have blonde hair that reaches...
We compare log- log-likelihood (with different normalizations) of the persona given the correct username against likelihood given (9) distractor usernames and report accuracy.
(Prompted) Infill on Username
chatbot: tell me a bit about yourself.
Same as Infill on Username with an additional prompt.
<candidate username> : iim an amazing dancer. i have blonde hair that reaches...
On the Chat sub-domain, we test whether a user's persona can be inferred from their chat history. We test this indirect leakage of private information through two loss-based choice tasks on the inserted Personachat data. In the first task, Infill on Persona, we test the models' accuracy on selecting the correct persona conditioned randomly from the other personas in the perturbation data). In the second task, Infill on Username, we whether the model can accurately select the correct username given the persona (distractor usernames are randomly drawn from the perturbation data). We illustrate the attacks in Table 9 For completeness, we also report the loss of the chat history and persona under the core models. We report findings in Figure 9
the username from set of 10 personas (distractors
drawn
Inference of indirect information is difficult but possible. The details of our analysis on Per- sonaChat in $D22.2 The accuracy our attacks is close to random guessing when asked to choose between the persona choices given the username (Infill on Persona). While the Hubble mod- els memorize the chat logs for the user, they do not directly assign higher likelihoods to the correct underlying persona. However, the username of the chat can be inferred when the attack is reversed, ie., prompting the model to identify the username corresponding to a given persona. In the best case, for the 8B perturbed Hubble model (100B tokens), Prompted Infill on Username achieves
40

---

accuracy of 34% on chats duplicated 64 times. This shows that, again, any memorization evaluations only lower bound on what is memorized.
Models assign lower likelihood signed to the persona by the Hubble models decreases the strength of memorization of the chat history increases (i.e., with lower dilution). This effect is more prominent for the IB parameter models than the 8B parameter models.
persona when memorizing chats. The log-likelihood as-
41

---

Personachat Norm LL on Chat
Personachat
Norm LL on Persona
0.0
0.0
perplexity
Norm LL
00.5
-0.5
-1.0-
-1.-
Personachat Infill on Persona
Personachat Infill on Persona
Personachat
Infill on Persona
acc. mutual info
0.2
0.15-
acc moom
0.15-
acc
0.10-
0.10-
0.1-
0.05
0.05
Personachat (Prompted) Infill on Persona
Personachat
Personachat (Prompted) Infill on Persona
(Prompted) Infill on Persona
acc m..muuuallll infoa
0.2
0.2
acc norm
0.15-
acc
0.10
0.14
0.05-
Personachat
Personachat Infill Username
Infill on Username
0.3
0.3
acc
0.24
0.1
0.1
Personachat (Prompted) Infill Username
Personachat (Prompted) Infill on Username
0.4-
0.4-
acc
0.2-
0.2-
0.0-
0.0
16
64 256
64
256
Duplicates
Duplicates
Standard --- 1B-100B
= 1B-500B
8B-100B
= 8B-500B
Perturbed 1B-100B
1B-500B
8B-100B
8B-500B
Figure 9: Core results on Personachat. Row reports the length-normalized log-likelihood of the inserted chat and the underlying persona under the different Hubble models. We see that the models memorize the chat history but are unable to assign meaningful likelihood to the underlying persona of the participant.
Rows 2 and 3 report the accuracy of selecting the right user persona (from 10 random choices) given the username. Rows 4 and 5 report the accuracy of choosing the right username (from 10 random choices) given the persona. Rows and 5 perform the same tests as rows 2 and 4 (respectively) but use an additional chat-style template.
42

---

D.3 TEST SET CONTAMINATION RESULTS
In this section, we report alternative metrics for each of the contaminated testsets. For PopQA, we report Fl score Rajpurkar et al. (2018) in addition to the Exact Match (accuracy). For ELLie, we run both generative evaluation (measured using exact match accuracy) and report the normal- ized log- log-liicl on the inserted perturbations. For all Infill-based tasks (WinoGrande-Inan Infill, Hel- laSwag, PIQA, MUNCH), we report accuracy using alternative normalization schemes: acc directly compares the conditional log-likelihoog of each choice, acc norm compares the conditional log- likelihood of each choice normalized by the byte-length of the choice, and acc_ acc_mutual _info pares the conditional log-likeldhood of each choice after subtracting the unconditional log- lig-likelihood of just the choice. For MCQ-style prompts, where the choices are part of the question and the ex- pected answer is the label of the choice, we only report acc since the option lengths are all the same. We report the performance on PopQA, HellaSwag, MMLU, and PIQA in Figure [T] We report the performance on different WinoGrande formats in Figure 12 Finally, we report performance on the new test sets, MUNCH and ELLie, in Figure T0
mutuul_info com-
Models begin to memorize test set examples with as few one duplicate, but generalization to unseen examples is unpredictable. From Figure [1] we see that the Hubble perturbed mod- els trained on 100B tokens show an increase in accuracy on PopQA, HellaSwag, and PIQA with just 1 instance of contamination. However, memorizing test set examples does not translate into generalization on that task: perturbed models show no improvement over standard models when trained on contaminated tasks (reflected in model performance on 0 duplicates), aside from small improvements on PopQA and under certain settings of HellaSwag. In fact, model performance on unseen examples degrades for WinoGrande and few settings of HellaSwag. For WinoGrande (see Figure T2). we find that perturbed models achieve worse accuracy on minimal pairs of contaminated examples than unseen examples. Likewise, the paraphrased model fails to answer MMLU questions which were contaminated with paraphrases of that question. We hypothesize that pretraining on handful of contaminated test examples is not enough to generalize the task, leading only to memorization.
For WinoGrande, models do not generalize across formats and have worse accuracy on con- taminated examples in a new format than on unseen examples. We inserted two variants of WinoGrande, one in the standard infill (cloze) format, and another in the MCQ format, where op- tions are presented with the question and the model has to generate the correct option. In Figure 12 we report the model accuracy when the test time format does not match the inserted format. For examples inserted with the MCQ format, when tested on the infill format, the perturbed model accuracy even decreases with increased duplication.
Models do not generalize from contaminated examples to the corresponding minimal pairs. For each example in WinoGrande, there is paired minimal example where the answer is flipped. When inserting examples, we make sure to only use one example from each pair as a part of the perturbation data. This allows us to evaluate whether the perturbed models can generalize to the minimal pair from training on the inserted example. Our results on WinoGrande show that the models.
MUNCH is solved by standard models. From Figure I0 we see that both standard and perturbed models achieve very high accuracy on MUNCH. Each MUNCH example consists of two sentences, one of which is the original, valid sentence, and the other is modified by swapping one word from the original sentence for an inappropriate synonym. The task is to identify which sentence is meaningful and valid. Our core models are all competent at language modeling and thus can solve the task with high accuracy (> 96%). Even so, we see increased accuracy with perturbed models on the examples that are duplicated more than 16 times.
ELLie examples are minimal pairs making it isolate to disentangle the effect of duplication. EL is a task that tests whether language models can understand sentences with ellipsis. From Figure TO we see that the standard model achieve near O accuracy on the task. On the other hand, perturbed models achieve accuracy greater than 50% even on examples that were never duplicated.
On further analysis, we realized that the examples in ELLie are minimal pairs 5 When we insert the
$ Many examples in EL contain the same first sentence hut different query sentences (the second sen- tence). Thus, they passed our deduplication check.
43

---

ELLIE (Gen)
ELLIE
Exact Match
1.0-
Norm LI
-1.0
0.5-
-1.5
0.0
Munch
Munch
1.00
1.00
acc norm
acc
0.98
0.95
0.96-
16
256
16
256
Duplicates
Duplicates
Standard
1B-100B
1B-500B
8B-100B
8B-500B
Perturbed 1B-100B
1B-500B
8B-100B
- 8B-500B
Figure 10: Core results on ELLie and MUNCH.
examples our corpus, examples with the same first sentence were put in different duplication bins, e.g., of all the examples with the same core sentence, some examples were sometimes duplicated times and other examples were duplicated times. Thus, we see that models achieve high accuracy on examples duplicated 0 times. This invalidates the use of ELLie for studying dilution.
44

---

PopQA
PopQA
1.0-
1.0-
Exact Match
0.5-
F1
0.5-
0.0
0.0
HellaSwag
HellaSwag
HellaSwag
1.00-
1.00
acc mutual info
1.00-
norm
a0c
0.75
0.75-
0.75
0.50-
0.50-
0.50-
MMLU
1.0-
0.5
PIQA
PIQA
PIQA
1.0
1.0-
acc mutual info
1.0-
acc. noom
acc
0.8-
0.8-
0.8-
0.6-
0.6-
0.6
16
64
256
16
64 256
0
16
256
Duplicates
Duplicates
Duplicates
Standard
1B-100B
1B-500B
8B-100B
8B-500B
Perturbed 1B-100B
1B-500B
- 8B-100B
- 8B-500B
Figure 11: Core results on Test Sets (Part 1). Results for PopQA, HellaSwag, MMLUU and PIQA using different variants of accuracy measurement.
45

---

WinoG Infill
WinoG Infill (paired)
WinoG Infill
1.00
1.00
1.004
acc. norm
a0c
0.75-
acc
0.75-
0.75-
0.50
0.50
0.50-
WinoG MCQ-on-Inine
WinoG MCQ-on Infill (paired)
WinoG MCQ-on-Infill
0.6
0.6
acc. mutual info
0.6
ace
0.5-
0.5-
0.5-
WinoG MCQ
WinoG MCQ (paired)
WinoG MCQ
1.0
1.0-
acc m.t.uutallll.. info
1.0-
acc
0.5
0.5-
0.5-
0.0
0.0
0.0
WinoG Infill-on-MCQ
WinoG Infill-on-MCO (paired)
WinoG Infill-onMMCO
0.8-
acc norm
acc
ace
0.6-
0.6-
0.6-
16 64 256
0
1
4
16 64 256
0
16
64 256
Duplicates
Duplicates
Duplicates
Standard 1B-100B
1B-500B
8B-100B
8B-500B
Perturbed - 1B-100B - 1B-500B
- 8B-100B
- 8B-500B
Figure 12: Core results and variants on WinoGrande. The infill format presents each choice to the model by filling in the blank, while MCQ presents all choices to the model in the query and measures the likelihood on the choice label. Rows and evaluate accuracy on duplications inserted with the Infill format. Rows 3 and 4 evaluate accuracy MCQ format. Column 2 reports accuracy on the minimal pairs the inserted examples. Rows and 4 use the Infill format for evaluation while rows 2 and 3 use the MCQ format for evaluation.
duplications inserted with the
46

---

E ADDITIONAL RESULTS
E.1 TIMING RUNS
To study how memorization evolves over training, evaluate memorization intermediate check- points every 2,000 steps up to 48,000. We also include Timing runs to analyze forgetting. Figure 13 reports normalized log- eeglikklihood on Wikipedia passages and accuracy MRPC paraphrases, each inserted 256 times. Across all four Timing runs, both metrics rise as duplicated data are encountered, peak once all perturbations have been seen, and then decay.
Wikipedia
MRPC
0.0
1.0-
-02
0.8-
Norm LL
Accuracy
-0.4
0.6
-0.6
0.8
0.4
20000
40000
20000
40000
Training Steps
Training Steps
standard
InsertRange(O550)
InsertRange(25,50) InsertRange(0225) InsertRange(50.75)
perturbed
Figure 13: Forgetting curves for the intermediate checkpoints of Timing runs. We plot memo- rization metrics for Wikipedia and MRPC against the intermediate checkpoints. We report results the subset of examples duplicated 256 times. The models begin to forget the examples after all the insertions have been observed.
47

---

Wikipedia
Gutenberg (Popular)
Gutenberg (Unpopular)
0.0
0.0
Norm LL
-0.25-
0.5
-055
Copyright
-0.50
MRPC
PAWS
1.00-
1.0
Accuracy
0.75-
0.8-
0.50-
YAGO Gen (full prefix)
YAGO Gen (intro prefix)
ECtHR Gen (full prefix)
1.0-
0.6
1.0
Word Recall
0.4
0.5-
0.5
Privacy
0.2
Personachat Infill (bio)
Personachat Infill (username)
0.3-
Accuracy
0.15-
0.2
0.10-
0.1
0.05
PopQA
WinoGrande: Infill
WinoGrande: MCQ
1.0-
1.0-
1.00-
Accuracy
0.75-
0.54
0.8-
0.50-
Testset
0.0-
MMLU
HellaSwag
PIQA
1.0-
1.00-
1.0-
Accuracy
0.75
0.5-
0.8-
0.50-
4
16
64
256
16
64
256
16
64
256
Duplicates
standard
injectrange(0550) injectrange(50,100)
(0jectrange(0.255) (2jec))ange(25.50)
injectrange(50.75) injectrange(75.100)
perturbed
Figure 14: Evaluation on the InsertRange models. Models that were trained perturbations only in the early stages of training have lower performance on the memorization tasks than models trained perturbations in the late stages of training. InsertRange (x,y) denotes a model trained on a corpus with perturbations inserted batches between x% and y% of training.
48

---

E.2 PARAPHRASED RUNS
Two perturbed models (IB and 8B parameters) are trained on 100B tokens with the same pertur- bation data as the core perturbed model but with two data sets paraphrased: MMLU and YAGO Biographies. To prepare the data for the paraphrased runs, we construct paraphrased variants of the YAGO biographies and MMLU test set with gpt-4. 1-mini. Unless otherwise noted, gener- ation uses temperature=1 and top p=1. For each original perturbation example to be inserted, we obtain as many paraphrases as its required duplication count. The datasets are paraphrased as follows:
MMLU paraphrases. We follow the paraphrasing instruction of Yang et al. (2023). When a para- phrase query is declined by 4pt-4. 1-mini APT's safety filter, use gemini --.5-flashh Sf-fash-litt with the same parameters.
YAGO paraphrases. We adopt the diverse-style watermarking generation instructions from Cui et al. (2025). Each paraphrase is checked with string-matching validator ensure all biograph- ical attributes are preserved. A paraphrase is accepted only if every attribute appears. We follow the procedure until we obtain the required number of valid paraphrases.
PII still be inferred from paraphrased biographies. In Figure 15 the high accuracy of PII recontruction and inference indicates the paraphrase model has not just memorized a fixed string; instead, it generalizes unseen queries for the PII, and this knowledge remains retrievable (similar to the retrievability observed in AlllenZZhu & Li 2024). The accuracy of strong name-only attacks higher on the 3B-parameter paraphrase model than on the original perturbed model at high dupli- cation levels, indicating that models trained on paraphrases develop stronger semantic memory than the verbatim memory formed from training on exact duplicates. Personachat also shows the model's ability to retrieve memorized information in new contexts, and models can infer a user's persona based on the memorized chat logs (although the accuracy is low).
PII can be leaked from paraphrased biographies with loss-based choice and generative evalua- tions. The weakest attacks, which assume that the attacker has access to all PII about person except one fact, are successful models trained with paraphrased biographies. However, they have lower effectiveness than extracting the facts from the model that was trained on the original biographies. PII can be extracted with 100% accuracy from the core 8B perturbed model using the full prefix and full suffix MCQ format. This accuracy drops to 89% when extracting PII from the paraphrase model. Surprisingly, when using stronger attacks (attacker has access only the persons name), PIT is more accurately extractable from the 8B model trained on paraphrased biographies compared to the core models. However, this finding depends on the format of the attack and scale; generative evaluations cannot extract PII from the IB paraphrased model.
Models cannot generalize from paraphrased MMLU to the original examples. We find that both models (IB and 8B parameters) obtain random accuracy on the MMLU MCQ evaluations when trained on paraphrased versions of the examples.
49

---

YAGO Infill (full prefix-full suffix)
1.04
Accuracy
0.5-
YAGO Infill (full prefix)
YAGO Infill (intro prefix)
YAGO Infill (name only)
1.0-
Privacy
Accuracy
0.75
0.75-
0.5-
0.50-
0.50-
YAGO Gen (full prefix)
YAGO Gen (intro prefix)
YAGO Gen (name only)
1.0-
1.0
1.0-
Word Recall
0.5-
0.5-
0.5-
4
16
64
256
0
4
16
64
256
16
64
256
Duplicates
Duplicates
Duplicates
MMLU
1.0-
Testset
Accuracy
0.5-
0
16
64 256
Duplicates
('1B', 'standard') (1B', 'perturbed') ('1B', 'perturbed (paraphrase)")
('8B', 'standard') - ('8B', 'perturbed') - ('BB', 'perturbed (paraphrase)")
Figure 15: Performance of Hubble perturbed models trained paraphased insertions. The models do not generalize from paraphrased examples seen in training to the original examples. How- ever, PII be reconstructed from models trained paraphrased biographies, with stronger attacks.
50

---

E.3 ARCHITECTURE RUNS
Wikipedia
Gutenberg (Popular)
Gutenberg (Unpopular)
0.0
0.0
Norm LL
-0.25
-0.5-
-0.5-
Copyright
-0.50
MRPC
PAWS
1.04
1.04
Accuracy
0.8-
0.6-
0.6-
YAGO Gen (full prefix)
YAGO Gen (intro prefix)
ECtHR Gen (full prefix)
1.0
0.6-
1.0
Word Recall
Word Word Recall
Word Recall
0.4-
0.5-
0.5-
Privacy
0.2
Personachat Infill (bio)
Personachat Infill (username)
0.3
Accuracy
0.15-
0.2-
0.10
0.05-
0.1-
PopQA
WinoGrande: Infill
WinoGrande: MCQ
1.0
1.04
1.0
Accuracy
0.8-
0.5-
0.8-
0.6-
Testset
0.0
0.6-
-0
MMLU
HellaSwag
PIQA
1.00-
Accuracy
0.75
0.5-
0.8-
0.50
64 256
16
64
256
16
64
256
Duplicates
Architecture Corpus Type
Base
2x Depth perturbed
.5x Depth
standard
Figure 16: Deeper models memorize slightly more than shallower models. We train three 1B- parameter models with 16, and 32 layers, adjusting width to keep total parameters constant (se 1.2B). All models are pre-trained on 100B tokens. As shown in Figure [6, the deeper (narrower) model memorizes slightly more than the base 16-layer model, while the shallower (wider) model memorizes less.
51

---

F ADDITIONAL HUBBLEMIA RESULTS
instantiate variants MIA benchmarks using the Hubble suite, using models and 3 pertur- bation datasets (passages from Gutenberg Unpopular, biographies from YAGO, and contaminated examples from MMLU). As discussed both the and unseen sets, serving only reference point i.e. method should achieve snteerttha-randdooe accuracy this setting.
[ the standard models use entirely unseen data for
Table 10: ROC AUC scores of baseline MIAS for the HUBBLE 8B (500B tokens) perturbed model. Dup indicates the duplication level of members. Dup & treats all inserted perturbations as members. Non-mem me-mmemerr are always drawn from perturbations inserted 0 times. As duplication increases, memorization becomes stronger, and it becomes easier for membership inference attacks (MIA) to distinguish between members and non non-members.
HUBBLE 8B (500B tokens) Perturbed
Evaluation
MTA
Dup #
Dup
Dup= =
Dup =
Dup = 64
Dup 256
Loss
0.629
0.539
0.556
0.732
0.996
1.0
Gutenberg
MinK%
0.629
0.539
0.556
0.732
0.996
1.0
Unpopular
MinK ++
0.666
0.545
0.62
0.813
0.987
0,949
ZLib
0.622
0.53
0.551
0.722
0.996
1.0
Loss
0.692
0.538
0.652
0.897
1.0
1.0
Yago
MinK%
0.692
0.537
0,651
0.896
1.0
1.0
Biographies
MinK %
0.714
0.571
0.686
0.892
0.995
0.983
ZLib
0.676
0.524
0.633
0.872
1.0
Loss
0.673
0.529
0.628
0.857
1.0
1.0
MinK%
0.672
0.529
0.626
0.854
MMLU
MinK%
0.743
0.58
0.731
0.943
0.994
0.986
ZLib
0.644
0.523
0.593
0.775
0.993
0.999
Table 11: Membership inference performance on various benchmarks with Hubble 1B Per- turbed. The Dup values indicate the composition of the seen set: for example, Dup * means the attack compares all seen data against unseen data, whereas Dup K means the attack compares unseen data against data that was included exactly K times the seen set.
Hubble IB Perturbed (500B tokens)
Evaluation
MIA
Dup #
Dup =
Dup=
Dup= 16
Dup 64
Dup 256
Loss
0.552
0.52
0.504
0.552
0.73
0,999
Gutenberg Unpopular
MinK%
0.552
0.52
0.504
0.552
0.729
0.999
MinK +++
0.575
0.513
0.53
0.605
0.825
1.0
ZLib
0.543
0.511
0.497
0.533
0.729
1.0
Loss
0.606
0.506
0.557
0.696
0.928
1.0
Yago
MinK%
0.606
0.506
0,556
0.695
0,927
1.0
Biographies
MinK ++++
0.615
0.509
0.565
0.715
0.947
1.0
ZLib
0.596
0.499
0.551
0.679
0.899
1.0
Loss
0.557
0.499
0.524
0.575
0.748
1.0
MinK
0.557
0.5
0.524
0.575
0.747
1.0
MMLU
MinK
0.605
0.522
0.556
0.681
0.887
0.996
7Lih
0.548
0.502
0.521
0.556
0.67
0.998
52

---

Membership inference performance on various benchmarks with Hubble 8B Stan- dard. The Dup values indicate the composition of the seen set: for example, Dup * 0 means the attack compares all seen data against unseen data, whereas Dup means the attack compares unseen data against data that was included exactly K times in the seen set.
Table 12:
Hubble 8B Standard (500B tokens)
Evaluation
MIA
Dup #
Dup =
Dup =
Dup =
Dup 64
Dup = 256
Loss
0.507
0.522
0.486
0.495
0.54
0.545
Gutenberg Unpopular
MinK%
0.507
0.522
0.486
0.495
0.54
0.545
+++ +++
0.504
0.517
0.493
0.499
0.484
0,543
ZLib
0.497
0.514
0.48
0.474
0.535
0.544
Loss
0.499
0,489
0,499
0.519
0,486
0,516
Yago
MinK%
0,499
0.489
0.499
0.519
0.487
0.516
Biographies
MinK%
0.503
0.5
0.503
0.507
0.505
0.505
ZLib
0.495
0.479
0.5
0.523
0.481
0.495
Loss
0.502
0.506
0.503
0.512
0.459
0.476
MinK%
0.502
0.506
0,503
0.512
0.458
0.476
MMLU
MinK% ++
0.506
0.51
0.505
0.514
0.497
0.45
ZLib
0.501
0.505
0.504
0.506
0,463
0.495
G ADDITIONAL HUBBLEUNLEARNING RESULTS
Below are the detailed byperparameters for each method:
Hyperparameter
RMU
RR
SatImp
Training type
Layer FT 5, 6.
LoRA FT
Full FT
Layers Targets
10, 20 (transform all) 16 0.05
LoRA Rank / Dropout
LoRRA a
Alpha (a)
100, 1000, 10000
0.01, 0.1,
Steering coefficient
5, 50,00,00 500
B1, 32
(5. 6),
Learning rate
5e-5, le-5, 5e-4
5e-5, 1e-4, 5e-4, le-3
1e-5, 5e-5, le-4
Effective batch size
8
16
Epochs
4, 8
4,
Sample max length
512
256
256
Table 13: Grid search configurations for unlearning methods. Each method is tuned over the listed hyperparameters. RMU and RR involve partial fine-tuning, while SatImp uses full fine-tuning.
53

---

We provide the full scale unlearning results for Gutenberg in Figure 17 and YAGO in Figure 18
Gutenberg (norm LL) Retain: Keep
Gutenberg (norm LL)
Retain: Keep
0.0
-0.5
-1.0
-15
Test
Keep
-35
-9
Unlearn
Unlearn
SatImp
RMU
RR
SatImp
RMU
RR
Gutenbery (norm LL) Retain: Wikitext
Gutenberg (norm LL) Retain: Wikitext
0.0
-2.5
-6
-7.5
Tost
-10.0
-12
-14
-15.0
-16
-17.5
-18
-20
-15
-10
-5
-20
-15
-10
-5
Unlearn
Unlearn
SatImp
RMU
RR
SatImp
RMU
Figure 17: Unlearning results on Gutenberg Unpopular. Unlearning results using (fut-of-domain, unseen) Wikitext (lower row) and (in-domain, seen) Keep set (upper row) as the retain sets. None of the unlearning methods simultaneously achieve the target behavior on both the seen Keep set (left column) and the unseen Test set (right column).
54

---

YAGO (acc) Retain: Keep
YAGO (acc) Retain: Keep
0.9
0.35
0.34
0.7
Test
0.6
0.5
0.32
0.4
8
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Unlearn
Unlearn
Sallmp
RMU
SatImp
RMU
RR
YAGO (acc) Retain: Wikitext
YAGO (acc) Retain: Wikitext
1.0
0.35
0.8
Keep
0.25
Test
0.15
0.2
0.10
0.2
Unlearn
Unlearn
SatImp
RMU
SatImp
RMU
RR
Figure 18: Unlearning results on YAGO biographies. Unlearning results using (fut-of-domain, unseen) Wikitext (lower row) and (in-domain, seen) Keep set (upper row) as the retain sets. None of the unlearning methods simultaneously achieve the target behavior on both the seen Keep set (left column) and the unseen Test set (right column).
55

---

56

---

H FURTHER DISCUSSION
How is information memorized? Understanding how transformers memorize is a basic scientific question that has been studied extensively in the literature (Geva et al. 2021; Dai et al. [202]] among
others). A better understanding of the mechanisms model memorization can inform the design of knowledge editing or unlearning techniques (Meng et al. 2022). Another practical application is in separating out knowledge from model parameters and enabling the responsible use of data (Shi al. 2025a Ghosal al. 2025). With the perturbations in HUBBLE, interpretability studies can analyze wide range of causal effects and control for factors such as the duplication rate or timing of an inserted text. The randomness in the perturbation data (e.g.. the synthetic biographies) may also be useful as canaries to probe whether knowledge is localized to certain parameters (Maini et al.) 2023 Chang et al. 2024b). Finally, the released checkpoints enable the study of how memorization evolves throughout training (Biderman et al. 2023a Chang et al. (024a)..
How can memorization be measured? For debates around copyright and privacy, there is need for more intuitive and robust memorization metrics (Schwarzschild et al. 2024] as an example). HUBBLE perturbations span diverse data types that enable the development new metrics, and the controlled insertions can validate these measurements (the same property that makes HUBBLE solid benchmark for membership inference). Measuring memorization is closely related to privacy auditing, both aim detect whether model reveals information about specific training exam- ples; borrowing intuitions from differential privacy, such as bounding sensitivity, may be useful here (Panda al. 2025). For number of tasks within HUBBLE, model performance reflects combina- tion of both memorization and generalization (Feldman & Zhang (222)), and isolating memorization effects may require advanced attribution methods (Ilya et 2022 Grosse et al. 2023).
How can memorization be mitigated?? HUBBLE establishes two best practices-dilutionn and ordering- for mitigating memorization. HUBBLE'S perturbation data is designed emulate memo- rization risks across domains, and the models provide testbed for evaluating new mitigation strate- gies. One direction to explore whether quantization can generally reduce memorization risks as well (Chang et al. 2025 Kumar et al. 2025). Because memorization and data poisoning both rely on how models internalize specific examples, advances mitigation may also reduce poisoning vulner- abilities; for instance, ordering has been found to influence the strength of poisoning attacks (Souly (ct al., 2025). Beyond identifying mitigation strategies, understanding their limitations is equally important. Best practices such dilution may reduce memorization but may not fully eliminate all copyright or privacy concerns (Cooper et al. 2024 Mireshghallah Li 2025).
57

---

I ADDITIONAL PLOTS
Wikipedia
Gutenberg (Popular)
Gutenberg (Unpopular)
0.00
0.0
0.0
Norm LL
0.25
-0.5
Copyright
-0.50
MRPC
PAWS
1.00
1.0
Accuracy
0.75+
0.6
0.50
YAGO Gen (full prefix)
YAGO Gen (intro prefix)
ECtHR Gen (full prefix)
1.04
1.0
Word Recall
0.4-
0.5-
0.5-
Privacy
0.2
Personachat Infill (bio)
Personachat Infill (username)
Accuracy
0.15-
0.2+
0.10
0.1-
0.05-
PopQA
WinoGrande: Infill
WinoGrande: MCQ
1.0-
1.04
1.0
Accuracy
0.8-
0.54
0.8
0.6-
Testset
0.0
MMLU
HellaSwag
PIQA
1.0-
1.0
1.0
Accuracy
0.8-
0.9-
0.5-
0.64
-
0.8-
16
64
256
16
64
256
16
64
256
Duplicates
Model Size
Corpus Type
perturbed
standard
Figure 19: Larger models memorize at lower duplicates. When trained on the same 500B-token corpus, the 8B parameter perturbed model memorizes model. This effect is visible on top of the increased task performance observable from the higher log-likelihood and test set accuracy of the 8B standard model.
data than the 1B parameter perturbed
58

---

Wikipedia
Gutenberg (Popular)
Gutenberg (Unpopular)
0.00
0.0
0.0
Norm LL
0.25
00.5
0.5
Copyright
-0.50
MRPC
PAWS
1.004
1.004
Accuracy
0.75+
0.75-
0.50
0.50-
YAGO Gen (full prefix)
YAGO Gen (intro prefix)
ECtHR Gen (full prefix)
1.0
0.6
1.0
Word Recall
0.4-
0.54
0.5+
Privacy
Personachat Infill (blo)
Personachat Infill (username)
Accuracy
0.15-
0.2-
0.10-
0.1-
PopQA
WinoGrande: Infill
WinoGrande: MCQ
1.00-
Accuracy
0.75-
0.54
0.8-
0.50
Testset
0.0
MMLU
HellaSwag
PIQA
1.0
Qwrenk
1.00
1.0
Accuracy
0.75-
0.5-
0.50
4
16
64
256
16
64
256
4
16
64
256
Duplicates
standard
perturbed
testset lesl
copyright__oo only
privacy_onn
Figure 20: The perturbed model matches the behavior of domain-specific models on the respec- tive set of evaluations. The perturbed model matches the copyright.onl/ model in memorizing the copyright passages and paraphrases, privacy biographies and chat, and testset only model memorizing the testsets. Thus, the perturbed model can be used to study individual domains despite being jointly trained on all three domains.
model in generating memorized PII from
59


Reviews:
[{"review_id": "yzpQe3orcM", "reviewer": "Reviewer_m7QY", "summary": "This paper proposes HUBBLE, a comprehensive, open-sourced suite of LLMs and data to explore the memorization of pretrained LLMs. More specifically, it consists of 8 pairs of standard models and perturbed models with 1B or 8B and pretrained on 100B or 500B tokens. The authors conducted a series of experiments on the HUBBLE, which show conclusions consistent with the existing studies on LLM memorization and also provide new insights on how to mitigate the memorization of sensitive data. Moreover, the experimental results also show the effectiveness of HUBBLE to build benchmarks for use cases of the membership inference attacks and machine unlearning.", "questions": "The questions are listed as follows.\n1. In Figure 1, do the authors have any explanation for why dilution does not work well on the data samples duplicated 256 times?\n2. For the experiments on the impact of the ordering of sensitive data in the pretraining, is the total number of sensitive data and their duplication times kept the same for all settings? For the setting with perturbed data across all four quarters, a specific sensitive data may be used in all four stages. However, for the setting with perturbed data only in the first quarter, will the same sensitive data be repeatedly used four times for fair comparison?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The strengths of the paper are listed as follows.\n1. HUBBLE will be fully open-sourced upon publication, which is important for the researchers to reproduce its results and leverage the suite for their own study in LLM memorization.\n2. The design of HUBBLE is reasonable and useful. The pair of standard models and perturbed models enables a fair and clean comparison when studying the mechanism of LLM memorization.\n3. The open-source data and checkpoints also make the quantitative analysis of LLM memorization feasible. The unavailability of checkpoints or training data is a big bottleneck for the study of LLM memorization.\n4. HUBBLE provides an insightful categorization of the potential risk caused by LLM memorization, which reflects the deep understanding of this research area and the practical safety demands in real-world applications.\n5. The evaluation is comprehensive and solid. The numerical results, observations and conclusions drawn from HUBBLE are reasonable, convincing, and also consistent with the existing works in LLM memorization.", "weaknesses": "The major concern of HUBBLE is that it only covers the LLMs with 1B and 8B. Therefore, some observations from HUBBLE may not be extended to large LLMs, which usually have a higher memorization rate and are more widely adopted by commercial applications. However, for the academic research of LLM memorization, the two model sizes are sufficient.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uzwWS0uQTC", "reviewer": "Reviewer_HMNX", "summary": "This work introduces a suite of (not yet) open sourced models specifically for study of memorization in LLMs. HUBBLE consists of standard and perturbed model pairs trained on the same data with the latter containing controlled insertions of synthetic or real-world sensitive data.\n\nOn standard NLP benchmarks, HUBBLE models perform comparably to other open-weight models (e.g., Pythia) at similar scale, confirming training stability. \n\nThe authors train 8 primary models (1B and 8B parameters, 100B and 500B tokens) and 6 auxiliary models with variations in insertion timing, frequency, and architecture. Experiments on these trained models show two primary trends:\n\nDilution effect: memorization risk decreases when sensitive data are diluted within larger corpora.\nOrdering effect: placing sensitive data early in pretraining reduces long-term memorization.\nModel size effect: Larger models (8B) memorize faster and at lower duplication counts, aligning with previous scaling trends.\nMinimal interference: Perturbations from distinct domains (privacy, copyright, contamination) do not interfere significantly.\n\nThis work also opens up the possibility for controlled benchmarks for MIA attacks and unlearning", "questions": "- Please detail if there are a set of directions which could be explored using these models apart from the standard memorization and unlearning tasks for which this study is directly intended for", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The strengths of this work are listed as follows:\n\n- This work would help form any following work overcome one of the primary drawbacks in this field, where experiments had to be either painfully small scale or had to rely on somewhat unknown data patterns of models trained without a focus on memorization studies\n- The predictable and standard nature of the perturbations added to the training data would help any further study conduct clear A/B tests on this study\n- I really like the detailed nature of the experiments especially the domain specific results", "weaknesses": "- This is an (computationally)expensive study and ideally more patterns could/should have been embedded into the training data to make the most out of it. One interesting direction might've been understanding the relation between the distance between paraphrased samples and the degree to which they're memorized\n\n- Test set insertions focus on benchmarks like MMLU and HellaSwag, whereas contamination in real corpora can be indirect or paraphrastic.\n\n- The work isolates memorization metrics from general model utility. While HUBBLE provides general evaluation benchmarks, it does not deeply analyze how memorization mitigation (via dilution or ordering) affects downstream performance or factual recall.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "f2MLurHDwY", "reviewer": "Reviewer_XuhG", "summary": "This paper's main contributions are as follows:\n1. Hubble, a suite of open-source pretrained LLMs for memorization research\n2. A memorization study using those models\n3. Benchmarks for membership inference and unlearning using Hubble models\n\n**Hubble model suite**: The core Hubble suite consists of 8 models that span the combinatorial space of 1B vs. 8B parameters, 100B vs. 500B pretraining tokens, and with vs. without added \"perturbations\". Perturbations are documents that are injected into the base corpus (DataComp-LM); they are designed to allow studies of copyright, privacy, and benchmark contamination. Memorization can be studied by comparing pairs of models with and without added perturbations. The Hubble suite contains further \"ablation models\", measuring effects of data ordering, interference between data domains, paraphrasing, and architecture choices. All models use a modified Llama 3.X architecture and training is fully open-source (with known datasets and ordering).\n\n**Memorization study**: The paper performs a broad memorization study on the Hubble models. Most notably, the study confirms that memorization increases with data duplication, that larger models memorize more, and that data appearing earlier in training is memorized less. Furthermore, training on a few samples of a benchmark's test set does not necessarily improve performance on that benchmark. The authors also verify that simultaneously including multiple data domains at once does not lead to interference effects between domains.\n\n**Membership inference and unlearning benchmarks**: Lastly, the paper proposes to use the Hubble models as a benchmark for membership inference and unlearning. The authors perform a small case-study for both types of benchmarks by comparing a few representative methods. They find that none of the evaluated methods manage to unlearn target knowledge and simultaneously preserve capabilities on non-target data.", "questions": "1. How are duplication counts sampled? Is it completely uniformly at random, or are the numbers balanced (beyond using 256 duplicates less often)? From Footnote 3 on p. 4, I believe it to be the former, but I could not find an exact statement.\n2. What exactly are the held-out perturbations (e.g., perturbations with duplicate count 0 in Section 5.1)? Will those be published with the results of this paper in an easily-accessible format? I believe that this is an crucial point for benchmarking membership inference; other researchers might theoretically be able to generate such a held-out set themselves, but this requires non-trivial engineering effort that the authors of Hubble already did.\n3. Similarly, what are the sizes of the member and non-member sets in the membership inference benchmarks?\n4. How difficult would it be to extend Hubble to post-training (e.g., instruction tuning)? This might be a highly useful extension, given that there are even fewer benchmarks to measure memorization in post-training.\n5. From my understanding, different repetitions and number of training tokens change the relative _frequency_ of perturbations. This terminology is mentioned consistently throughout the paper, except for the beginning of Section 4. That part of the paper also mentions the _spacing_ between perturbations. Is the spacing something that is effectively controlled? Or does the beginning of Section 4 just assume that spacing and frequency are negatively correlated? In the latter case, I would either only mention frequency or empirically validate that lower frequency indeed yields uniformly larger spacing between perturbations.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "**Highly useful model suite**: The Hubble model suite is highly useful for memorization research, and hence serves as a more contemporary test bed compared to, say, Pythia. For one, the 8 core models allow studying the causal effects of adding certain types of data in a broad range of domains. The authors choose domains and datasets well, and they demonstrate that the models are useful to measure things beyond pure verbatim memorization.\n\nThe model ablations (\"Runs\" on p. 4) are also noteworthy. For one, they allow the authors to verify their setup (e.g., that there is no interference between domains; Figure 20). In addition, due to their design, I can see the model ablations being useful for future research (even beyond memorization).\n\n**Broad memorization study and benchmarks**: The authors not only propose a useful model suite, but also perform a plethora of studies on those models. This includes confirmation of already established phenomena (e.g., data is memorized less if it is seen earlier in training), but also new insights (e.g., test set contamination does not necessarily improve benchmark scores, or that common unlearning methods fall short of their goal). Due to the breadth of the memorization study, and the additional benchmarks, I think those contributions are already significant in isolation.\n\n**Polished and clear presentation**: Overall, the paper seems polished; it is mostly well-organized and easy to understand. The authors provide a lot of detail and transparency on their training and evaluation procedures. In the memorization study, findings that confirm existing observations/beliefs are contextualized with the relevant related work. I also like Figure 4; it intuitively illustrates how the base corpus is modified to include perturbations.\n\n**Insights and tools for model training**: The insights from this paper could be useful beyond memorization research. The authors mention that their research resulted in additional tools that could benefit future research (Footnote 2 on page 4). Additionally, assuming the full training pipeline code is documented well, it might even help other researchers train similar suites of models.", "weaknesses": "While this is an overall high-quality paper, there is one potential major issue related to decontamination between the base training corpus and perturbations. I am happy to raise my score if those issues are addressed.\n\n**Potential contamination between base corpus and perturbations**: The decontamination step of Hubble's data pipeline focuses on high precision (i.e., only discarding samples that clearly overlap the base corpus and perturbations). However, false negatives (i.e., contamination that is not detected) are much more harmful for memorization research, as they can taint the results for the use cases in this paper (studying memorization, MIA/unlearning benchmarking).\nFor one, the authors verify matches manually; however, this can only detect false positives and not false negatives. Moreover, decontamination for perturbations (of non-trivial length) only uses exact matches of at least 20 tokens. Hence, it is possible that non-trivial overlaps between perturbations and base corpus exist (e.g., with slightly different formatting).\nI thus argue that, to ensure results of memorization studies are valid, this requires further discussion and a potential empirical study of shorter overlaps between base corpus and perturbation sequences.\n\n**Membership inference benchmark is incomplete**: The membership inference case-study (Section 5.1 and Appendix F) exclusively uses the ROC AUC score as a metric. This is well-known to be misleading and obfuscating important details [(Carlini et al., 2021)](https://arxiv.org/abs/2112.03570). Instead, I recommend reporting the TPR at a low fixed FPR (e.g., 1%), and providing full ROC plots.\nAdditionally, I could not find the size of the member and non-member sets mentioned. However, this information is crucial to assess the resolution of the reported metrics (small set sizes might yield high variance in reported scores).\n\n**Relatively short pretraining compared to \"real-world\" models**: Hubble is only trained on 0.5T tokens, compared to 4T and 9-15T tokens for OLMo and Llama 3.X models of similar sizes. The authors are transparent about this. Yet, the insertion timing results have to be viewed relative, and Hubble might not allow studying memorization due to emergent behaviors from very long pretraining. On the flip side, while Hubble models are obviously not SotA on standard benchmarks, they seem to at least often perform on-par with similarly-sized Llama 3.1/3.2 (but not OLMo 2) models. I understand that this limitation is unavoidable due to the large cost of pretraining.\n\n**Minor feedback on figures**:\n1. Most plots only use a single hue for all lines and distinguish them by brightness (see, e.g., Figures 1 and 2). I think the plots would be easier to read and compare if they used very different colors, especially in plots with many lines.\n2. I found it hard to process Figure 1, particularly since it contains many small plots for many settings. At that point in the paper, I would find it easier to have a high-level figure that highlights an aggregate or a subset of results, and defer the full Figure 1 to the appendix.\n3. In contrast, Section 4.1 provides a lot of interesting insights, but defers most plots to the appendix. The paper could be easier to read if the main matter showed (a subset of) the results discussed there (to avoid jumping between sections) and deferred detailed plots (such as Figure 1) to the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ClXpw6iRWo", "reviewer": "Reviewer_vHGJ", "summary": "This paper presents a set of open-source large language models (LLMs) called Hubble, which is designed for the scientific study of LLM memorization. The Hubble models spans 1 billion to 8 billion parameters and are trained on corpora of 100 billion to 500 billion tokens. Importantly, the models come in pairs: standard models trained on a large English corpus and perturbed models that include controlled insertions of sensitive text (like book passages, biographies, and test sets) to simulate memorization risks. During the training process, the authors conduct experiments to understand how the insertion of sensitive data affects memorization. Their analysis reveals that (1) diluting sensitive data by increasing the training corpus size can reduce memorization risks, (2) inserting sensitive data earlier in the training process can help mitigate these risks, (3) larger models memorize with lower duplications and (4) perturbations from different domains minimally interfere with each other. As for the use cases, the authors demonstrate that the Hubble models can be used to evaluate membership inference and machine unlearning methods, providing a controlled environment for further research in these areas.", "questions": "Can you share your comments on the weaknesses above?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- This paper is a fully open-source contribution to the field of LLM memorization, providing a suite of models that can be used for later research.\n- The training methodology is faithfully described, and the analysis of the results is thorough and valid, without overclaiming as far as I can tell.\n- The amount of experiments and analysis is impressive.", "weaknesses": "I am not fully convinced that the published models will add much value to the research community. In general, when people pick a model, they tend to pick commercial ones like LLaMA or Qwen, therefore, it is likely that the Hubble models will be used in special cases as discussed in Section 5. I am not an expert in MIA, so I cannot judge the value of the models in that context. However, I am not sure if what was discussed in Section 5.2 is a desirable benchmark for unlearning. The Huddle benchmark randomly separates the 256-duplicated set into a Unlearn set and a Keep set, meaning that these two sets follows the same distribution. As a result, we see in Figure 3 that unlearning methods fail to reduce the memorization of the Unlearnset without hurting the performance on the Keep set. However, in real unlearning tasks, the forget set and the retain set are usually from different distributions, and unlearning algorithms are usually *expected* to generalize the forget set and remove related but not identical information. From this perspective, the Hubble benchmark might be too strict compared to many unlearning tasks.\n\nBut overall, I think this is a solid paper with a good amount of experiments and analysis, and I recommend acceptance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)