Memorization: A Close Look at Books (2025)

Abstract: To what extent can entire books be extracted from LLMs? Using the Llama 3 70B family of models, and the “prefix-prompting” extractiontechnique, we were able to auto-regressively reconstruct, with a very high level of similarity, one entire book (Alice’s Adventures in Wonderland) from just the first 500 tokens. We were also able to obtain high extraction rates on several other books, piece-wise. However, these successes do not extend uniformly to all books. We show that extraction rates of books correlate with book popularity and thus, likely duplication in the training data. We also confirm the undoing of mitigations in the instruction-tuned Llama 3.1, following recent work (Nasr et al., 2025). We further find that this undoing comes from changes to only a tiny fraction of weights concentrated primarily in the lower transformer blocks. Our results provide evidence of the limits of current regurgitation mitigation strategies and introduce a framework for studying how fine-tuning affects the retrieval of verbatim memorization in aligned LLMs.

## introduction large language models (llms) can memorize their training corpus, and this capability grows along with model scale, prompt length, and the extent of data duplication within the training set @xcite . such capability makes llms susceptible to extraction attacks, through which adversaries can retrieve sensitive information, including personally identifiable details like phone numbers and email addresses, directly from model outputs @xcite . this vulnerability raises privacy and security concerns, especially given that organizations that develop llms frequently incorporate copyright-protected content into their training datasets. unauthorized disclosure of copyrighted material during extraction attacks could expose these companies to legal risks and lawsuits @xcite . consequently, to mitigate these vulnerabilities, companies have implemented rigorous safeguards @xcite , including data deduplication, content filtering, alignment techniques, and output validation mechanisms, to prevent verbatim text regurgitation and unintended disclosure of sensitive information from deployed llms. recent research has shown that alignment processes do not entirely eliminate memorization in production-scale llms. specifically, new extraction methodologies, such as divergence attacks and fine-tuning-based extraction, can partially undo built-in regurgitation mitigations, therefore exposing memorized training content @xcite . @xcite the extraction of textual excerpts from fine-tuned gpt-3.5-turbo models and further examined how memorization manifests across pretrained and instruction-aligned models. in this paper, we investigate the extraction of entire books from llama 3 pretrained, and from llama 3.1 models, both pretrained and instruction-tuned. for the instruction-tuned model, we employ nasr et al.'s sft-based technique. books are important, because they are at the center of several copyright litigation cases. 1 they are also technically interesting targets to extract, because they tend to be long and unique. an ideal extraction method would be able to auto-regressively extract entire books from an llm trained on them given just their first n tokens. while such extraction method does not [yet] exist, we were able to autoregressively extract a version of "alice's adventures in wonderland" from llama 3 pretrained that closely resembles the original. we were also able to obtain high reconstruction rates, although not auto-regressively, for many more books, with several llama models. moreover, we show how the popularity of books present in the training data, and therefore the likelihood of their duplication, affects their memorization by llama. we conduct an analysis of memorization by examining the piece-wise reconstruction rates of full-length books sourced from project gutenberg, cross-referencing the results with the number of ratings in goodreads. the following summarizes our experiments and findings: nasr et al.'s sft technique in both pretrained and instructiontuned llama 3.1 models, including varying number of training samples. main results: the technique does not improve the extraction rates on the pretrained model, but it significantly improves those rates on the instruction-tuned model. nevertheless, as already reported in nasr et al.'s work, those rates are still lower than the extraction rates of the baseline pretrained model. ## analysis of weight updates in this section, we focus on analyzing the weight updates introduced by the lora fine-tuning process on the baseline llama model. by design, lora only updates certain layers of the original network. moreover, due to the compressed nature of the lora formalism -where the rank of the learned adaptation matrices is usually much smaller than the full rank of the underlying weight matrices -only a subset of the parameters within those layers are effectively modified. this raises two natural questions: how large is the fraction of the original weights that receive significant updates, and how are these updates distributed across the different layers of the network? to address these questions, we focus on our sft-1000 trained lora model as a representative case. our trained lora models were free to modify layers of the following modules of the llama transformer blocks (q, k, v and o), at the self-attention and feedforward mlp blocks (gate, up, down). figure 15 provides a schematic overview of the llama transformer architecture, highlighting the locations where our lora adapters are integrated. to carry out our analysis, we begin by reconstructing the weight update matrices for all layers that could be modified by the lora adapters. we note that the llama 3.1 70b instruct model has 80 stacked transformer blocks containing all the aforementioned modules. thus, since our lora training was applied across the full model depth, we reconstruct 560 weight update matrices. as typical in lora, each of the layers we decided to train adaptors gets a pair of low-rank matrices a and b of shapes @xmath0, where d in and d out are the input and output dimensions of the original llama weight matrix @xmath1. the rank we use for our lora adaptors is @xmath2. to study where significant updates take place, we must perform the reconstruction of the weight update w update matrix from lora's a and b trained matrices. this reconstruction is straightforward: the full-rank weight update is simply w @xmath3, where α is the lora scaling factor and r is the lora rank hyperparameter. this update matrix has the same shape as the original weight matrix, i.e. w @xmath4, and represents the effective change that would be applied to the base model if the lora adapters were merged back into the baseline llama model. since lora's central idea is training these low-rank projections while keeping the original model weights frozen, w update captures exactly what lora is trying to inject into the base model after the supervised fine-tuning process, which is precisely what we want to discover. nevertheless, it is misleading to analyse w update directly, since what really matters is the impact of the update in the original network weights, and not the absolute values of these updates: a small absolute ∆ value of the update might actually cause a huge impact if the original neuron weight was tiny, while a large ∆ might be insignificant if the original weight was already huge. thus, we further construct relative update matrices, i.e.,: where ⊘ denotes a hadamard division (which is just an element-wise division for matrices of equal dimensions, as here), and abs denotes that the matrix has all its elements in absolute, positive values. figure 5 shows a histogram built from the concatenated set of values of all 560 w rel matrices. the distribution clearly reveals that the vast majority of updates are relatively small in magnitude when compared to the original weights. only about ∼ 14% of the original weights are receiving a boost greater than only 1%, and a mere ∼ 0.15% are updated by more than 100%. these results suggest that only sparse and highly localized updates are sufficient to make the instruct network start remembering documents that were used in its training set. naturally, this result raises the follow-up question of how these few significant updates are distributed across the entire llama network. the top panel of figure 6 shows that these updates are heavily concentrated at the earliest transformers instead of the significant updates being applied more uniformly throughout the entire network. this pattern is similar regardless of whether we examine the self-attention blocks or the multilayer perceptron (mlp) blocks. nevertheless, although the evolution of the update fraction along the network is similar for both types of blocks -with a predominant concentration of significant updates in the early layers -we observe that the self-attention layers receive approximately seven times more updates (in fraction) than the mlp layers in these early transformers. these findings suggest that early layers play a central role in adapting the model, while later layers require minimal changes to help the network remember its training data. moreover, even at those early layers, the fraction of weights of the original network that needs to be changed is sparse. ## conclusion our results demonstrate that modern large language models, particularly the llama 3 family, retain substantial amounts of memorized content from the training corpus. we find that both autoregressive generation, and passage-wise reconstruction are sensitive to a book's likely presence in the training data, with stronger reconstruction observed for more popular or widely reviewed texts. instruction tuned models like llama 3.1 exhibit reduced memorization by default, but we show that targeted fine-tuning can partially mitigate this suppression. this effect is most pronounced in the lower layers of the network, where small updates appear to undo alignment caused suppression. more broadly, our study introduces a scalable framework for measuring memorization across models and training stages. by combining behavioral evaluations with an analysis in the change of the weights of these models, we uncover corre-lations between memorization, training exposure, and popularity, offering insight into what factors make a model remember, and when that memory can be accesed or suppressed.