Title: Distillation Robustifies Unlearning
Venue: NeurIPS 2025
Accepted: yes
Keywords: machine unlearning

robust unlearning

relearning attacks

AI safety

alignment

distillation
Abstract: Current LLM unlearning methods are not robust. A few steps of finetuning can revert their effects. We begin by showing that this is true even for an idealized form of unlearning: training to imitate a model that was never trained on unwanted information. This shows that training a model can drastically modify its input-output behavior while leaving its underlying capabilities intact. In light of this dynamic, we show our main result. Training a randomly initialized student on the outputs of an unlearned model transfers behaviors while leaving latent capabilities behind. In short, distillation robustifies unlearning. Based on this result, we propose Unlearn-Noise-Distill-on-Outputs (UNDO), a scalable method that distills an unlearned model into a noised copy of itself. UNDO introduces a tunable tradeoff between compute cost and robustness, establishing a new Pareto frontier on synthetic language and arithmetic tasks. At its strongest setting, UNDO matches the robustness of a model retrained from scratch with perfect data filtering while using only 60-80% of the compute and requiring only 0.01% of the pretraining data to be labeled. We also show that UNDO robustifies unlearning on the more realistic Weapons of Mass Destruction Proxy (WMDP) benchmark. Since distillation is widely used in practice, incorporating an unlearning step beforehand offers a convenient path to robust capability removal.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the problem that most machine-unlearning techniques for large LMs only suppress dangerous capabilities that are to be unlearned, leaving them latent and easy to re-learn with light fine-tuning. The methods presented in the paper are motivated by a simple observation: if you first apply any standard unlearning method to suppress the unwanted behavior and then distill that model into a randomly initialized student, the harmful capability is largely stripped away while the desired skills are retained. They demonstrate this across two unlearning tasks, arithmetic and the WMDP weapons benchmarks, showing that their Unlearn-Noise-Distill-on-Outputs (UNDO) procedure (which adds controlled parameter noising to trade compute for robustness -- i.e., taking a weighted average between a randomly initialized and the pretrained model) closes much of the gap to an expensive \"retrain-from-scratch with perfect data filtering\" oracle (training only on the \"clean\" subset of the data). UNDO consistently sits on the pareto frontier of the utility-unlearning curve, with distilled models relearning harmful tasks no faster than the oracle while preserving most of the original utility.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is clearly written. The objective is well defined, the problem which is the focus of this paper is interesting, and the presented method is motivated by experiments that show very clear differences in unlearning rate between randomly initialized and pretrained models. \n\nMost of the experiments are carried out in relatively simplified settings, where the forget set is either an entire language or the distinction between two mathematical operators (addition and multiplication). In such a case, the distinction between the examples to be unlearned and the retain set is very significant, as they differ in domain. This contrasts with usual unlearning work that focuses on very specific concepts. Fig 8 reports results on WMDP, but does not present the utility-unlearning curve. What would it look like in this case?\n\nI am not sure the forget-utiltiy comparison in Fig7 is completely fair. If I understand correctly, the baselines just experience unlearning, while UNDO experiences unlearning+retraining on the retain set. It can be the case that this retraining is responsible for better retain-set accuracy. Can it be the case that it’s mainly this retraining, and not the noise, that explains the better tradeoff? Did you try to ablate the noising step? \n\nRobustness is measured via gradient-based relearning attacks (finetuning on the forget set). It would be beneficial to also compare against jailbreaking attacks in inference time.\n\nQUESTIONS:\nIn Fig7a, how come many methods do better than the filtering gold standard?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the critical shortcoming of existing machine unlearning methods for large language models, which often only achieve \"behavioral suppression\" rather than true removal of undesired capabilities. The authors demonstrate that these suppressed capabilities can be easily recovered through adversarial finetuning. The core contribution is the finding that distilling an unlearned \"teacher\" model into a new, randomly initialized \"student\" model can robustify the unlearning process, effectively transferring desired behaviors while discarding the latent, undesired capabilities. Building on this insight, the paper introduces Unlearn-Noise-Distill-on-Outputs (UNDO), a practical method that creates a tunable trade-off between unlearning robustness and computational cost by partially noising the teacher model's weights before distillation. Through extensive experiments on language modeling, arithmetic reasoning, and the Weapons of Mass Destruction Proxy (WMDP) benchmark, the authors show that their approach significantly improves resilience to relearning attacks compared to existing methods, establishing a new Pareto frontier for the trade-off between model performance and unlearning robustness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper tackles a highly significant and timely problem in LLM safety. The distinction between merely suppressing behavior and genuinely removing a capability is a central challenge for deploying safe models. This work offers a practical and seemingly effective method to bridge this gap, which is of great interest to the community.\n\nThe experimental evaluation is exceptionally thorough and well-designed. The authors construct a compelling narrative, starting by demonstrating the fundamental limitations of output-matching for robust unlearning. They then systematically validate their core hypothesis - that distillation robustifies unlearning - across multiple unlearning techniques and tasks. The proposed UNDO method is analyzed in detail, clearly illustrating the compute-robustness trade-off. The comparison against strong, relevant baselines under increasingly powerful relearning attacks is convincing and highlights the superiority of the proposed approach. Finally, validating the method on a realistic benchmark like WMDP strengthens the claims. The appendices are remarkably detailed, ensuring high potential for reproducibility.\n\nThe paper is very well-written, clearly structured, and easy to follow. The central idea is simple yet powerful, and it is communicated effectively through concise prose and informative figures. The logical flow from problem identification to solution proposal and validation makes the paper a pleasure to read.\n\nWhile its constituent parts (unlearning, distillation) are not new, the core insight - that distillation can act as a selective filter to remove latent capabilities from an unlearned teacher - is novel and impactful. The generalization of this idea into the UNDO framework, which offers a practical knob to tune the trade-off between cost and robustness, is an original and useful contribution. The initial experiment showing that oracle-matching is insufficient for robustness is a valuable finding that helps clarify the nature of the problem for the field.\n\nThe paper contains clear ablation studies and multiple experiments supporting the main hypotheses and also answering potential questions from the readers. For instance, the plots showing effect of noise levels on robustness was very useful.\n\nThe paper uses well known and recent benchmarks to measure effects of the method (e.g. MMLU for retain performance and WMDP for forget performance).\n\nCode will be released on a public GitHub repository, which is clearly stated in the main body of the paper. The paper also lists detailed hyperparameter settings. Both of these enhance reproducibility of this work and its usefulness to the community.\n\nIf the basic observation holds at larger scales, it offers a practical path for open-weight release: distillation is already part of most deployment pipelines, so adding “unlearn then distill” could become standard practice. That is a meaningful contribution to LLM safety.\n\nWeaknesses:\n\nThe primary weakness, which the authors rightly acknowledge, is the high computational cost of distillation. While the paper frames this as a manageable trade-off and notes that distillation is already part of many development pipelines, the cost may still be a significant barrier to adoption, especially for organizations without access to massive compute resources or for applications where frequent unlearning is required.\n\nA key practical limitation is the observed drop in performance on the retain task when applying UNDO to a public model without access to its original pretraining data. This currently limits the \"off-the-shelf\" applicability of the method and suggests that its effectiveness is still somewhat dependent on having access to high-quality, representative data for the distillation phase.\n\nThe paper provides strong empirical evidence that distillation robustifies unlearning but offers a less detailed explanation for why this is the case. The intuition that desired behavior is transferred while latent capabilities are \"left behind\" is compelling but high-level. A deeper analysis - perhaps investigating the information bottleneck properties of distillation in this context or examining changes in the model's internal representations - could further strengthen the paper's contribution, though this may be extensive enough for future work.\n\nThe selection of models and model sizes is very limited.\n\nQUESTIONS:\nDo the authors have preliminary results for a wider variety of model families and larger model sizes? It would be interesting to see if robustness degrades with scaling.\n\nWMDP is still a proxy. Have you tried latent knowledge probes rather than behavioral scores?\n\nThe performance drop on the MMLU retain task for the WMDP experiments is a key practical hurdle. Do you have other ideas on how to mitigate this issue without needing the original pretraining data?\n\nThe paper argues that UNDO is more efficient than the \"gold standard\" of data filtering with full retraining. Could you provide a more concrete, quantitative comparison of the computational cost (e.g., total FLOPs) between UNDO and full retraining for one of the smaller-scale experiments, such as the language task? This would help readers better contextualize the claimed cost savings.\n\nThe proposed UNDO method introduces noise into the weight space of the teacher model before distillation. Have you considered or experimented with alternative methods of \"damaging\" the teacher to prevent the transfer of latent information?\n\nThe effect of perturbation levels (alpha) seem to vary based on task (e.g. arithmetic vs language). Do you have an intuition for why this is the case? Does this suggest that the optimal alpha is highly task-dependent, and can you offer any guidance for how a practitioner might select a suitable value for a new task without performing an expensive hyperparameter sweep?\n\nSome references are broken (showing as ??) - please fix these.\n\nLIMITATIONS:\nYes. The authors have adequately addressed the limitations of their work in a dedicated paragraph and in the appendix. They are upfront about the significant computational cost of distillation and the challenge of maintaining retain performance when the original pretraining data is unavailable. The discussion is fair and provides important context for the results.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a way to improve robustness of machine unlearning methods (which aim to remove certain harmful knowledge/capabilities). The problem: current unlearning methods produce models that can still elicit the harmful behavior on finetuning. The paper presents evidence of this problem in Sec 3.\n\nThe paper then proposes a way to improve the robustness of any existing unlearning method (i.e. plug-and-play), at the cost of additional compute. The key insight is to use distillation on unlearned models. This method is called Unlearn-and-Distill, presented in Sec 4.\n\nThe paper introduces a way to control the trade-off between additional compute and unlearning robustness. This method is called UNDO (Unlearn-Noise-Distill-on-Outputs), presented in Sec 5. It also compares with other unlearning methods in Sec 6.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n**S1**. The paper makes an interesting and novel observation that distillation robustifies unlearning. This approach, called Unlearn-and-Distill, gives advantage over simple unlearn baselines (Figure 4).\n\n**S2**. The paper introduces an approach which allows for trading between compute and robustness (UNDO, Figure 5).\n\n**S3**. The paper demonstrates that UNDO extends the pareto-optimal frontier on the tradeoff between quality retention (for harmless cases) and unlearning (for harmful cases). Figures 7 and 8 demonstrate competitive performance of UNDO over baselines.\n\n**S4**. The paper is well written: It has a good structure that follow nicely, and explanations are clear to understand.\n\n**Weaknesses**\n\nThe paper does not present much intuition/explanation of why distillation is effective at improving the robustness of unlearning methods. At a high-level, distillation seems to perform unlearning + \"mixing\" the unlearned weights with noise. Are there ways to investigate why this works?\n\nRelatedly, are there other ways to \"mix\" the unlearned weights with noise instead of distillation? Perhaps they can be cheaper in compute?\n\nQUESTIONS:\n**Q1**. In the formulation in Section 2, the setup requires a retain set. Can unlearning work without a retain set, and are there algorithms in the literature? I ask this because practically, one might want to retain all capabilities except a few harmful ones. Thinking ahead, in this case, can distillation provide an advantage on existing unlearning methods? \n\n**Q2**. W.r.t lines 169-173, what is the dataset used for distillation? Is it the full training dataset?\n\n**Q3**. W.r.t lines 204-206, I'm unclear about whether this claim has been shown empirically, because your approach (line 100 and around) requires the entire training set to be partitioned into the retain and forget sets.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents an investigation into the limitations of current machine unlearning techniques, particularly targeting performance with respect to relearning attacks, where a model quickly recovers forgotten capabilities through finetuning. The authors argue that conventional unlearning often suppresses undesired behaviors rather than eliminating them, leaving latent representations intact due to factors like plasticity and representational entanglement. To address this, the authors propose Unlearn-and-Distill, a method where a model is first unlearned and then used to train a freshly initialized student model through knowledge distillation, effectively discarding residual capabilities. Building on this idea, they introduce UNDO (Unlearn-Noise-Distill-on-Outputs), a more flexible and computationally efficient extension.\n\nThe authors empirically validate their methods across multiple domains, including language modeling, arithmetic reasoning, and a high-stakes Weapons of Mass Destruction Proxy (WMDP) benchmark. The results show that UNDO shows robustness against relearning attacks compared to existing unlearning methods such as MaxEnt, SAM, and RepNoise. Experiments demonstrate that increasing the noise perturbation parameter α consistently strengthens resistance to relearning, with even modest values yielding improvements. Moreover, the paper shows that UNDO achieves favorable retain-forget trade-offs and remains resilient under strong adversarial finetuning (e.g., 500-step attacks, also highlighted in Figure 7).\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* The paper systematically demonstrates that standard unlearning methods leave residual capabilities that can be quickly relearned (similar findings are also revealed in other recent works [1, 2, 3, 4]). Intriguingly, the paper introduces a controlled methodology (referred to as UNDO) that allows interpolation between noising and full reinitialization, with some empirical improvements in resisting relearning across multiple domains (language, arithmetic, WMDP), that may be helpful for future works on similar lines. \n\n* While distillation has been used for compression or privacy, this work is somewhat novel in explicitly using it as a means to destroy/degrade/suppress latent capacities that contribute to fast relearning (making the idea itself unique and less explored in the unlearning community [though some initial works like [5] do explore a teacher-student approach]). This reframing is both conceptually and technically significant, as it connects behavioral unlearning with representation-level damage via noising, followed by reconstruction under behavioral constraints, making it a good unlearning mechanism for the community to explore in future works. Making it align with the unlearning community by using similar benchmarks would definitely benefit the overall quality/utility of this work.\n\n**Weaknesses**\n\n* One of the notable weaknesses of this work is that it evaluates robustness primarily through forget/retain set performance and relearning speed. However, the effects of perturbation and distillation on general downstream performance (e.g., OOD generalization, calibration, fairness) remain unexplored. Since the proposed UNDO method introduces large-scale noising and re-alignment, understanding its unintended side effects on broader capabilities becomes imperative. It would be good to add other evaluation metrics that are used in unlearning [also refer to questions for clarification details regarding other evaluation metrics]\n\n* Another major weakness is that the paper uses a completely different setting than the existing works on machine unlearning, making it difficult to compare/validate/check how the proposed unlearning fits the existing set of works. It would be good to add some results on benchmarks like open-unlearning, using datasets like TOFU for a better alignment with the unlearning research community. \n\n* On the method side, the current method distills only output distributions (via forward KL), potentially ignoring subtle representational changes. It remains unclear whether internal activations, gradients, or alignment signals are also preserved or transformed in ways that might make the model vulnerable to alternative elicitation strategies. Some works like [1, 2, 3, 4] show challenges in measuring unlearning and how the model’s knowledge remains intact even after unlearning. Since it is difficult to compare the proposed work using different datasets, it remains challenging to rely on the results for the applicability of the proposed learning method in other settings. \n\n[1] https://arxiv.org/abs/2410.16454\n\n[2] https://arxiv.org/abs/2406.13356\n\n[3] https://arxiv.org/abs/2411.15477\n\n[4] https://arxiv.org/abs/2402.16835\n\n[5] https://arxiv.org/abs/2310.02238\n\nQUESTIONS:\n* While the paper emphasizes that the mixing coefficient $\\alpha$ largely controls robustness and computation cost, the role of the noise scale $\\beta$ seems less clearly justified. It would be good if the authors could provide more detailed insights for this parameter in practive. For instance, are there conditions or dataset properties under which certain beta values outperform others? \n\n* The paper evaluates robustness under several adversarial relearning scenarios, but the extent and variety of these attacks could be expanded (and would never be exhaustive). This raises some questions regarding the guarantee of unlearning, which remains a challenge in general. It would be good if the authors could comment on similar lines, explaining the extent and limitations of the current setting.\n\n* The experiments focus on retain and forget sets to measure unlearning effectiveness, but it is unclear how the method affects generalization to unrelated downstream tasks or out-of-distribution data. It would be good if the authors provide evaluations or discussions on whether the UNDO method preserves overall model utility and fairness across a wider task spectrum. Additionally, evaluation using richer and more comprehensive metrics such as Verbatim Probability, ROUGE variants, Model Utility, Forget Quality, TruthRatio, and Membership Inference Attack (MIA) strength, as used in recent benchmarks (e.g., Open-Unlearning), could further help validate the claims made in the paper.\n\nLIMITATIONS:\nYes, the authors clearly mention some of the major limitations of their approach, particularly around the distillation and the challenges of applying UNDO in settings where access to the original pretraining data is limited (e.g., Gemma-2-2B). These limitations make the proposed approach difficult to apply to all the open-weight LLMs where pretraining data is available.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}