Understanding Verbatim Memorization in LLM s Through Circuit Discovery (2025)

Abstract: Underlying mechanisms of memorization in LLMs—the verbatim reproduction of training data—remain poorly understood. What exact part of the network decides to retrieve a token that we would consider as start of memorization sequence? How exactly is the models’ behaviour different when producing memorized sentence vs non-memorized? In this work we approach these questions from mechanistic interpretability standpoint by utilizing transformer circuits—the minimal computational subgraphs that perform specific functions within the model. Through carefully constructed contrastive datasets, we identify points where model generation diverges from memorized content and isolate the specific circuits responsible for two distinct aspects of memorization. We find that circuits that initiate memorization can also maintain it once started, while circuits that only maintain memorization cannot trigger its initiation. Intriguingly, memorization prevention mechanisms transfer robustly across different text domains, while memorization induction appears more context-dependent.

## introduction large language models (llms) have demonstrated remarkable capabilities across a broad spectrum of applications @xcite @xcite . despite these impressive advancements, researchers have identified various challenges with these models, with memorization emerging as an important issue. memorization in llms refers to the model's tendency to store and reproduce exact phrases or passages from its training data when prompted with appropriate contexts. memorization capabilities have both positive and negative aspects. on one hand, deliberate memo-rization enables models to store facts, concepts, and general knowledge that enhance their performance in tasks like question answering and information retrieval @xcite @xcite . on the other hand, undesirable memorization creates several problems: privacy risks when models expose personal information, security issues when they reveal passwords or credentials, copyright concerns when they reproduce protected content, and biases reflecting their training data. furthermore, memorization complicates model transparency and interpretability, making it difficult to determine whether specific outputs reflect generalization or memorized content. interpretability research has provided important insights into memorization in llms, revealing that memorization is distributed across model layers with distinct patterns: early layers promote tokens in the output distribution, while upper layers amplify confidence @xcite . studies have demonstrated that memorized information shows distinct gradient patterns in lower layers and is influenced by attention heads focusing on rare tokens @xcite . however, there exists a promising approach for understanding neural networks that has yet to be fully applied to memorization: the circuits framework @xcite . this framework, which seeks to reverse-engineer model behavior by localizing it to subgraphs of the model's computation graph, has gained significant traction in interpretability research by providing mechanistic explanations of how models accomplish specific tasks @xcite . most state-of-the-art automatic circuit discovery algorithms rely on patching techniques, which require contrastive datasets with clean and corrupted examples @xcite @xcite . creating such datasets is challenging for open-ended tasks like memorization. as a result, circuit discovery has mostly been applied to small, well-defined tasks like indirect object identification (ioi) @xcite , greater-than comparisons @xcite , or factual knowledge recall @xcite . these simpler tasks make circuit discovery more tractable, but leave open the question of how to apply similar techniques to understand more complex behaviors like memorization. in this paper, we introduce an approach to studying memorization circuits in language models trained on the pile dataset. we focus specifically on the wikipedia subset of the pile, as its clean, well-structured content makes it easier to analyze when manually creating our contrastive datasets. using this data, we identify highly memorized sentences and precisely locate the divergence points where the model transitions from memorization to generation. our contributions include: our findings help us better understand how memorization works in language models and may lead to improved ways to control this behavior. ## methodology our analysis focused on wikipedia subset from the pile dataset @xcite to identify memorized content, we used memorization score @xcite . after scoring all samples, we retained only those with memorization scores of exactly 1.0, representing perfect memorization. this created a base of 4047 samples for our contrastive datasets. ## experiments we define memorization token (t mem ) as the token that appears in memorized continuations, i.e. "ground truth" token from the pre-training dataset. predicted token (t pred ) refers to the token that the model predicts with highest probability (i.e. the argmax token of an unpatched model). see ## results as shown in ## conclusion in this work, we identified and analyzed circuits responsible for memorization behaviors in language models through targeted circuit discovery methods. the branch comparison task appeared easier to disentangle than the memorization decision task, requiring only 14 edges (0.04% of the model) compared to 141 edges (0.43%) for similar faithfulness levels. however, we should be cautious about this finding, as our results in section 5 show that these small circuits have limited generalizability across tasks. the extremely small circuit size raises questions about whether we've captured the complete memorization mechanism or merely identified a critical but incomplete component. our cross-task generalization results reveal a clear pattern: memory decision circuits work well for branch comparison tasks, but branch comparison circuits perform poorly on memory decision tasks. this suggests that memory decision circuits contain components that can both detect memorizable content and control its usage, while branch comparison circuits mainly handle continuation with less ability to make initial memorization decisions. cross-corpus experiments reveal that memorization prevention mechanisms transfer across different datasets, while memorization induction appears more context-dependent. this suggests that different text domains may trigger distinct mechanisms that collectively manifest as memorization behavior, with the ability to prevent memorization being more generalizable than the ability to induce it. our experiments show that removing memorization from already-memorized samples (noising approach) is easier than inducing memorization in non-memorized samples (denoising approach). interestingly, this observation contradicts findings from @xcite , who demonstrated that memorized continuations are harder to corrupt than non-memorized ones. in their experiments, they showed that even when targeting the top 0.1% of gradient-implicated weights, memorized passages resisted modification while maintaining their distinctive patterns. this difference likely comes from our focus on finding specific computation paths rather than using gradient methods to find important weights. one hypothesis is that memorization works through two different mechanisms: a "trigger" circuit that decides when to use memorized content (which we found), and a more spreadout storage system throughout the model (which they found). the trigger circuit can be easily disrupted, while the stored information itself is harder to change. this suggests that memorization safeguards might work better by targeting these trigger mechanisms rather than trying to remove the stored information.