Title: Elastic Robust Unlearning of Specific Knowledge in Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: LLM Unlearning; Preference Optimization; Unlearning Robustness
Abstract: LLM unlearning aims to remove sensitive or harmful information within the model, thus reducing the potential risk of generating unexpected information. However, existing Preference Optimization (PO)-based unlearning methods suffer two limitations. First, their rigid reward setting limits the effect of unlearning. Second, the lack of robustness causes unlearned information to reappear. To remedy these two weaknesses, we present a novel LLM unlearning optimization framework, namely Elastic Robust Unlearning (ERU), to efficiently and robustly remove specific knowledge from LLMs. We design the elastic reward setting instead of the rigid reward setting to enhance the unlearning performance. Meanwhile, we incorporate the refusal feature ablation into the unlearning process to trigger specific failure patterns for efficiently enhancing the robustness of the PO-based unlearning methods in multiple scenarios. Experimental results show that ERU can improve the unlearning effectiveness significantly while maintaining a high utility performance. Especially, on the WMDP-Bio benchmark, ERU shows a 9\% improvement over the second-best method, and maintains 83\% performance even under 1,000 sample fine-tuned retraining attacks, significantly better than the baseline method.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Elastic Robust Unlearning (ERU), an optimization framework for robust and effective unlearning in Large Language Models. ERU introduces Elastic Reward Setting to balance reference-based and reference-free reward signals, providing flexible and adaptive optimization during unlearning. It utilizes Refusal Feature Ablation, which simulates worst-case perturbations during unlearning to defend against knowledge recovery preemptively. The paper formulates unlearning as a max-min optimization problem, where the inner loop simulates adversarial conditions, and the outer loop removes harmful knowledge.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. ERU models the unlearning process as a robust optimization problem with Elastic Reward Setting to overcome the rigidity.\n\nWeaknesses:\n1. The proposed method incorporates a simplified adversarial training component, following a prior approach. However, it does not compare it to more advanced or state-of-the-art adversarial training techniques, instead of AdvNPO, to show the rationale for using Refusal Feature Ablation over others. Such comparisons are important for isolating the benefits of integrating adversarial mechanisms into the framework. Additionally, training time is typically not the primary concern in most machine unlearning scenarios.\n\n2. The current ablation study lacks a variant that isolates the contribution of the Elastic Reward component. Evaluating the model's performance with only Elastic Reward (and without other components) would help quantify its individual impact and clarify the source of improvements.\n\n3. The paper does not report statistical significance tests to support claims of performance improvement. Without such analysis, it is difficult to determine whether the observed gains are meaningful or within the margin of variability.\n\nQUESTIONS:\nCould the authors compare the proposed method against other sophisticated adversarial training techniques under the same Elastic Reward settings? This would help isolate the value of the framework's adversarial component and demonstrate the rationale and whether its integration offers a meaningful improvement\n\nCould the authors conduct an ablation study that evaluates the method using only the Elastic Reward component? This would clarify its individual contribution to overall performance and better support the design choices.\n\nCould the authors perform statistical significance tests to demonstrate that the proposed method yields significant improvements in unlearning effectiveness while preserving model utility as claimed? This would strengthen the empirical claims and enhance the credibility of the reported gains.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Elastic Robust Unlearning (ERU), a novel framework for improving the effectiveness and robustness of large language model (LLM) unlearning. Addressing limitations in existing preference optimization (PO)-based methods, ERU introduces two key innovations: an elastic reward mechanism that enhances unlearning flexibility, and refusal feature ablation, which induces targeted failure modes to boost robustness against unlearned knowledge relearning. Experimental results demonstrate that ERU achieves superior unlearning performance while preserving model utility, outperforming prior methods in a number of benchmark datasets and tasks.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. This work provides a detailed analysis of the shortcomings of existing LLM unlearning approaches.\n\n2. The proposed methods smartly balance the weights of reference-based and reference-free policy optimization during unlearning.\n\n3. Adopting the refusal feature ablation can greatly reduce the computation cost compared to using regular adversarial training to prevent unlearned knowledge relearning.\n\n4. The paper is well-organized, and it is easy for the readers to capture the main ideas.\n\n## Weaknesses\n\n1. The refusal feature ablation (RFA) is directly adopted to enhance the robustness of elastic policy optimization. Dedicated design is required for the scenarios of LLM unlearning. For example, the D_harmful and D_harmless can be constructed by the forget or retained datasets. In real-world scenarios, unlearning may be employed to delete outdated information and users' private information.\n\n2. More details of balancing the inner and outer optimization should be provided. For max-min bi-level optimization, the optimization pace impacts the final performance a lot.\n\n3. There is no ablation study of detaching the refusal feature ablation from ERU. The balancing effect between reference-based and reference-free optimization offered by the elastic reward margin needs validation.\n\n4. The robustness of defending against adaptive attacks should be discussed and validated. That is to say, if the adversaries have the prior knowledge that the target LLM has been protected by RFA and they adopt some dedicated measures to break up the protection before conducting unlearned knowledge relearning, how will ERU perform, how long, and how strong will the unlearning effect last?\n\nQUESTIONS:\nPlease see the above weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Elastic  Robust Unlearning (ERU), a novel framework designed to remkove specific knowledge from LLM more effectively and more robustly than the PO-based unlerarning algorithms like DPO, NPO, or the variants of NPO. They have two key designs: 1. the elastic reward setting, by using a reference model value combined between the uniforms distribution and the original model's output. This is in contrast with the rigid reward setting in prior works that either uses a uniform distribution as the reference value or use the original model's prediction probability. 2. They apply a refusal feature ablation (RFA)-based adversarial training procedure during the training process to simulate adversarial removal of the refusal featurein the model's hidden activation and formulate the robust unlearning as a max-min optimization problem. The experiments on multiple unlearning benchmarks, such as TOFU, WMDP, MUSE, show that ERU outperforms PO-based algorithms by a good margin.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper evaluates ERU across multiple axes—effectiveness, utility preservation, and robustness. Experimental results are favorable across multiple benchmarks compared to the most widely used methods, such as GA and NPO, or their variants. This is convincing and shows the effectiveness of the designed algorithm.\n\n2. The combination of the RFA-based adversarial training is novel and improved the robustness of unlearning algorithms. Incorporating RFA into the unlearning loop is a clever way to approximate worst-case adversarial perturbations without costly inner-loop PGD. By randomly ablating learned refusal features in the residual streams with probability p, ERU achieves robustness comparable to latent-space adversarial training but at a lower computational cost.\n\nWeakenesses and Questions:\n\n1. The novelty of this elastic reward setting: Given that ERU’s elastic reward formulation collapses to the rigid reference (only using uniform distribution or only using reference models) when alpha = 0 or alpha = 1, why should elastic and rigid reward methods be considered fundamentally distinct classes rather than simply two points on the same spectrum? In particular, since every rigid scheme can be recovered by fixing \nalpha at an extreme, what principled argument or theoretical insight supports treating ERU as a qualitatively new paradigm?\n\n2. You did some ablation studies about the effects of two main components of ERU: the elastic reward design and the RFA-based adversarial training in the appendix, but this ablation is only on the RWKU dataset for comparing the effectiveness. I wonder how they perform on the other three datasets.\n\n3. Figure 6 (PS: It does not have a caption) shows the effect of alpha and shows that setting alpha away from zero will improve the performance. It seems that the optimal alpha highly depends on the model (what about across datasets? What are the optimal alpha?) Is there anyway to explain this? Do you have any more intuition about what the hyperparameter alpha is and what it controls? And the most importantly, in what case would you expect a larger optimal alpha?\n\nQUESTIONS:\nSee the Strength and Weakness Section.\n\nLIMITATIONS:\nSee the Strength and Weakness Section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Elastic Robust Unlearning. This method combines an \"elastic reward\" which regularizes the unlearning algorithm by adding a parameter to vary the influence of the reference model, with the technique of refusal feature adversarial training to make the model more robust to relearning and jailbreaking attacks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The method comprehensively covers unlearning effectiveness as well as robustness, incorporating recent techniques to improve performance on both.\n- The evaluation thoroughly covers several benchmarks that are currently state of the art.\n- The proposed method of an \"elastic\" reward is interesting and intuitive, to interpolate between the reference-based and reference-free setting.\n\nWeaknesses:\n- The results seem generally inconclusive. ERU does not do uniformly better than other methods across benchmarks, and as such it seems to be another point on the forget/retain tradeoff curve rather than pushing the Pareto frontier forward.\n- The elastic reward is interesting but the reasoning for it seems fairly heuristic. NPO itself is a version of regularized gradient ascent, and the elastic reward seems to be an alternative regularizer that incorporates the reference model. The arguments for why this should lead to better unlearning seem to be based on a fairly hand-wavy explanation of training dynamics (Appendix B). As such it's hard to predict when this method will be effective.\n- Some results seem cherry picked: why is ERU regularized not included in Table 1? Why is WMDP evaluated only on bio and cyber, but not chemistry? Why is TOFU not included in Table 2? Additionally, presenting forget and retain in separate tables rather than side by side makes it difficult to determine which methods perform the best.\n\nQUESTIONS:\n- Why is TOFU not included in the utility analysis in 4.4? It does not seem to appear in the appendix either.\n- The ablation study in table 8 is a useful study but the results are presented in a confusing way. Why is unlearning effectiveness evaluated on a different dataset compared to unlearning robustness? Ideally the ablation would be on a single consistent benchmark.\n- How do the regularized ERU models (in table 2) perform on the forget tasks (table 1)? This is key to understanding the forget/retain tradeoff - otherwise the comparisons are apples and oranges.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-09"}