From Teacher to Student: Tracking Memorization Through Model Distillation (2025)

Abstract: Large language models (LLMs) are known to memorize parts of their training data, raising important concerns around privacy and security. While previous research has focused on studying memorization in pre-trained models, much less is known about how knowledge distillation (KD) affects memorization.In this study, we explore how different KD methods influence the memorization of fine-tuned task data when a large teacher model is distilled into smaller student variants.This study demonstrates that distilling a larger teacher model, fine-tuned on a dataset, into a smaller variant not only lowers computational costs and model size but also significantly reduces the memorization risks compared to standard fine-tuning approaches.

## introduction the rapid scaling of large language models (llms) has led to growing concerns about their ability to memorize and potentially reproduce sensitive training data. while earlier work has largely focused on describing memorization in llms through qualitative analysis @xcite , more recent research has introduced a quantifiable framework that evaluates memorization based on a model's ability to recall training examples verbatim when prompted @xcite , but crucially focused only on pre-trained models and their original training datasets. these foundational studies left open critical questions about memorization during fine-tuning -a common practice by which models are tuned to downstream tasks @xcite . fine-tuning is particularly risky because it tends to employ specialized, possibly sensitive data, e.g., medical records, proprietary data @xcite @xcite . in contrast to pre-training data that is generally broad and public, fine-tuning data is smaller and more specific, fine-tuning datasets are smaller and more targeted, making memorization both more likely and more dangerous. addressing this gap, recent work by @xcite systematically investigates memorization and privacy risks in domain-specific llms. their findings confirm that fine-tuned models, especially those trained on domain-specific corpora, are significantly prone to memorizing and potentially leaking sensitive content. building on these findings, this study examines how knowledge distillation affects memorization in student models. as large language models grow in capability and size, knowledge distillation (kd) @xcite has emerged as a critical technique to reduce their computational demands, where we train a small student model with supervision from a large teacher model. this technique, which compresses knowledge from large "teacher" models into smaller "students", was originally developed for efficiency, its impact on memorization remains unexplored, particularly in the fine-tuning context. this study bridges this gap by systematically studying how different distillation methods affect memorization when transferring knowledge from a finetuned teacher to smaller students. our experiments demonstrate that distillation not only achieves its traditional benefits of reduced model size and computational costs, but also serves as an effective privacy-preserving technique by considerably decreasing memorization while preserving task performance. this dual advantage makes distillation particularly valuable for deploying llms in privacy-sensitive settings. ## methodology 2.1 defining memorisation to study how memorization persists or changes through model distillation, we adopt a definition of memorization based on the framework introduced by @xcite , adapted for instructionfollowing tasks. given an instruction-contextresponse tuple (p,c,s) from our fine-tuning dataset d, where p is the instruction (prefix),c is the context and s is the target response, we define: memorization criterion: a model f is said to have memorized response s if, when prompted with instruction p and context c using greedy decoding, it generates s' such that: the algorithm below formalizes the measurement of memorization over a dataset d: ## limitations and future work this study is limited by the use of only a single dataset (dollyeval) due to computational constraints,which may not fully capture diverse memorization behaviors across tasks. the memorization analysis was conducted with a fixed 50-token window (for the memorisation fraction part), while examining varying sequence lengths could yield more comprehensive insights. future work should validate these results across different model architectures beyond gpt-2 and incorporate additional metrics like perplexity and bleu for a more complete evaluation of model behavior and memorization patterns. ## conclusion this study establishes knowledge distillation as a powerful technique for addressing the privacy challenges of deploying large language models. our findings reveal that distillation not only fulfills its original objective of model compression and computational efficiency, but also plays a critical role in mitigating the privacy risks posed by memorization. while larger models and direct fine-tuning were associated with higher memorization and inflated training set rouge scores,the student models produced via distillation consistently demonstrated lower memorization fractions. these insights suggest that distillation offers a dual benefit by enabling deployability in a resource constrained settings while simultaneously enhancing privacy by reducing a model's tendency to memorize confidential or sensitive data. future work should explore how to optimize the privacy-utility tradeoff further and extend this to several other architectures and domains.