Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning
Authors: Yicheng Lang
Yihua Zhang
Chongyu Fan
Changsheng Wang
Jinghan Jia
Sijia Liu
Venue: ICLR 2026 Poster (2026)
Keywords: LLM Unlearning
Robustness
Zeroth-Order Optimization
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: How optimizers influence LLM unlearning robustness

Abstract: Large language model (LLM) unlearning aims to surgically remove the influence of undesired data or knowledge from an existing model while preserving its utility on unrelated tasks. This paradigm has shown promise in addressing privacy and safety concerns. However, recent findings reveal that unlearning effects are often *fragile*: post-unlearning manipulations such as weight quantization or fine-tuning can quickly neutralize the intended forgetting. Prior efforts to improve robustness primarily reformulate unlearning objectives by explicitly assuming the role of vulnerability sources.  In this work, we take a different perspective by investigating the role of the *optimizer*, independent of unlearning objectives and formulations, in shaping unlearning robustness. We show that the "*grade*" of the optimizer, defined by the level of information it exploits, ranging from zeroth-order (gradient-free) to first-order (gradient-based) to second-order (Hessian-based), is tightly linked to the resilience of unlearning. Surprisingly, we find that downgrading the optimizer, such as using zeroth-order methods or compressed-gradient variants (*e.g.,* gradient sign-based optimizers), often leads to stronger robustness. While these optimizers produce noisier and less precise updates, they encourage convergence to harder-to-disturb basins in the loss landscape, thereby resisting post-training perturbations. By connecting zeroth-order methods with randomized smoothing, we further highlight their natural advantage for robust unlearning.  Motivated by these insights, we propose a *hybrid optimizer* that combines first-order and zeroth-order updates, preserving unlearning efficacy while enhancing robustness. Extensive experiments on the MUSE and WMDP benchmarks, across multiple LLM unlearning algorithms, validate that our approach achieves more resilient forgetting without sacrificing unlearning quality.

Published as a conference paper at ICLR 2026
DOWNGRADE TO UPGRADE: OPTIMIZER SIMPLIFICA- TION ENHANCES ROBUSTNESS IN LLM UNLEARNING
Yicheng Lang'
Yihua Zhang'
Chongyu Fan'
Changsheng Wang'
Jinghan Jia' 1 Sijia Liu 1.2 'The OPTML Lab, Dept. CSE, Michigan State University
MIT-IBM Watson Al Lab, IBM Research
ABSTRACT
Large language model (LLM) unlearning aims to surgically remove the influence of undesired data or knowledge from an existing model while preserving its utility on unrelated tasks. This paradigm has shown promise in addressing privacy and safety concerns. However, recent findings reveal that unlearning effects are often fragile: post-unleamiig manipulations such as weight quantization or fine-tuning quickly neutralize the intended forgetting. Prior efforts improve robust- ness primarily reformulate unlearning objectives by explicitly assuming the role of vulnerability sources. In this work, take different perspective by investi- gating the role of the optimizer, independent of unlearning objectives and formu- lations, in shaping unlearning robustness. We show that the "grade" of the opti- mizer, defined by the level of information it exploits, ranging from zeroth-order (gradient-free to first-order (gradient -bsedd) to second-order tightly linked to the resilience of unlearning. Surprisingly, we find that down- grading the optimizer, such as using zeroth-order methods or compressed--raddee gradient variants (e-g., gradient sign-based optimizers), often leads to stronger robustness. While these optimizers produce noisier and less precise updates, they encourage convergence harder-to-disturb basins in the loss landscape, thereby resisting post-training perturbations. By connecting zeroth-order methods with random- ized smoothing, we further highlight their natural advantage for robust unlearn- ing. Motivated by these insights, we propose a hybrid optimizer that combines first-order and zeroth-order updates, preserving unlearning efficacy while enhane- ing robustness. Extensive experiments on the MUSE and WMDP benchmarks, across multiple LLM unlearning algorithms, validate that our approach achieves more resilient forgetting without sacrificing unlearning quality. Code is available at https: / thub. com.//.coo////o /OP Graill-rroupp. Bareeaar____pp i zer.
flessian-basede is
INTRODUCTION
Large language models (LLMs) have demonstrated remarkable capabilities in natural language un- derstanding and generation across diverse applications (Achiam et al., 2023; Touvron et al., 2023; Yang et al., 2025a). However, their pre-training on massive data corpora raises growing concerns about safety, privacy, and trustworthiness (Mazeika et al., 2024; Li et al., 2024; Liu et al., 2025; Huang et al., 2024). LLMS may inadvertently reproduce copyrighted content (Eldan & Russinovich, 2023; Shi al., 2024), expose personally identifiable information (Staab et al., 2023; Yao et al., 2024a), or generate harmful instructions (Barrett et al., 2023; Li et al., 2024). To address these risks, LLM unlearning has emerged as promising direction, aiming to remove the influence of unde- sired data, knowledge, and associated model capabilities without incurring the cost of retraining the entire model and preserving the model's general utility (Yao et al., 2024b; Fan et al., 2024; Zhang et al., 2024a; Zhuang et al., 2025; Reisizadeh et al., 2025; O'Brien et al., 2025).
Despite recent progress in developing LLM unlearning algorithms that achieve both effective for- getting and utility preservation (Yao et al., 2024b; Zhang et al., 2024a; Fan et al., 2024; Li et al., 2024; Jia et al., 2024a), ensuring robust unlearning remains a significant challenge. Unlearning per- formance quickly deteriorate under post-unlest unle-anlennning weight perturbations. Prior work shows that fine-tuning on even a small set of forgotten samples or semantically related texts can substantially reverse unlearning effects (Lynch et al., 2024; Hu et al., 2024), while model compression techniques

---

Published as a conference paper at ICLR 2026
such quantization may also resurface erased content (Zhang et al., 2024d). Furthermore, when unlearned models are adapted to downstream tasks via fine-tuning, their unlearning guarantees often degrade (Wang et al., 2025a).
Existing research on robust LLM unlearning has primarily focused on problem-level reformulations or algorithm-level modifications, often assuming specific vulnerability source and tailoring the unlearning method accordingly. For instance, Fan et al. (2025) cast robust unlearning as a min-max problem against relearning-indueed perturbations and adapt sharpness-aware minimization (SAM) (Foret et al., 2020) to strengthen robustness. Tamirisa et al. (2024) propose tamper-resistant un- learning via meta-learning, modeling the attacker as weight-tamperin adversary. Similarly, Wang et al. (2025a) leverage invariant risk minimization (IRM) (Arjovsky et al., 2019) to regularize un- learning against degradation from irrelevant fine-tuning. While effective, these approaches rely on customized changes to unlearning objectives, thereby modifying the underlying optimization al- gorithm itself. In contrast, the role of the base optimizer, independent of any problem-wise and algorithm-level modifications, in shaping unlearning robustness remains largely unexplored. No- tably, even heuristic optimizer adjustments, such as increasing the learning rate, have been observed to improve robustness against weight quantization (Zhang al., 2024d), hinting at deeper connec- tion. This raises the central research question of this work:
How does the choice of optimizer inftuence the robustness of LLM unlearning, and what optimizers can improve robustness without sacrificing unlearning effectiveness?
To address this question, we introduce the concept of optimizer grade for LLM unlearning, defined by the level of gradient information utilized by an optimizer. The first-order (FO) gradient-based Adam optimizer (Kingma& & Ba, 2014), widely adopted in LLM unlearning (Shi et al., 2024; Li et al., 2024; Jia et al., 2024a; Fan et al., 2024; Zhang et al., 2024d), represents a "high-grade" op- timizer. In contrast, down-graded alternatives reduce the precision of gradient information. For example, graddiet-compressee quantize gradients into low-bit representations, while zeroth-order (7O) optimizers rely solely on finiteddifference estimates of objective values, serving as gradient-free counterparts to FO methods (Chen al., 2023; Liu al., 2020; Zhang et al., 2024c). Although these optimizers reduce gradi- ent fidelity, they remain principled and wwweerencceguurantten general optimization tasks, including LLM unlearning.
methods such as signSGD and signAdam (Bernstein et al., 2018)
making them suitable for solving
From the perspective of optimizer grade, a key finding of our work is that downgrading optimizers can unexpectedly enhance unlearning robustness. We provide both technical rationale and empirical evidence showing a clear link between optimizer grade and robustness grade. In particular, ZO optimizers, while less precise in unlearning effectiveness, exhibit strong robustness against weight tampering. Building on this insight, we propose Hybrid optimizer that integrates FO and ZO methods within a unified framework, combining the robustness of ZO with the optimization accuracy of FO. In summary, our contributiosn are listed below.
We present the first systematic study of optimizer choice in LLM unlearning, showing that down- grading the optimizer (via quantized or zeroth-ordes order updates) can improve robustness against weight tampering. We also provide a rationale: downgraded optimizers introduce higher optimization noise tolerance, making unlearned models more resilient to post-unlcarning weight perturbations.
We propose FO-ZO hybrid optimization, a unified framework that integrates FO and ZO optimiz- ers, combining ZO-induces induced robustness with FO-driven unlearning effectiveness.
We validate our findings through extensive experiments across diverse unlearning tasks and meth- ods, demonstrating consistent link between optimizer grade and unlearning robustness.
RELATED WORKS
LLM unlearning. L.M unlearning aims to remove memorized data or specific model behavior from pretrained LLMs (Liu et al., 2025; Fan et al., 2024; Maini et al., 2024; Jia et al., 2024a; Shi et al., 2024). Its applications span copyright protection (Shi et al., 2024: Eldan & Russinovich, 2023), privacy preservation (Wu et al., 2023; Lee et al., 2024; Kuo et al., 2025), and the removal of harmful abilities (Li et al., 2024; Lang et al., 2025; Zhou et al., 2024; Tamirisa et al., 2024)(Wan)) et al., a). Most existing approaches forgetting while retaining general utility (Yao et al., 2024b; Li et al., 2024; Zhang et al., 2024a; Fan et al., 2024; Jia et al., 2024a; Reisizadeh et al., 2025; Yang et al., 2025b) (Wang et al., b:a). Com-
fine-tuning based, employing regularized optimization promote
2

---

Published as a conference paper at ICLR 2026
plementary lines of work perform unlearning at inference time without altering model parameters, including in in-context unlearning (Thaker et al., 2024; Pawelczyk et al., 2023) and intervention-baxe based decoding strategies (Liu et al., 2024; Suriyakumar et al., 2025; Deng et al., 2025; Bhaila et al., 2025; Wang et al., 2025b).
Robustness of LLM unlearning. Recent studies have shown that unlearned LLMs remain vulner- able to both input-level and weight-level "perturbations" (Hu et al., 2024; Lynch et al., 2024; Lucki al., 2024; Fan et al., 2025). Input-space perturbations, such as in-context examples or adversarial prompts/jailbreak, can still elicit forgotten information from the model (Lucki et al., 2024; Sinha al., 2025; Yuan al., 2025). Weight-space perturbations include quantization, which can resur- face memorized data (Zhang et al., 2024d). relearning on forgotten or semantically similar data (Hu et al,,, 2024; Che et al., 2025; Lynch et al., 2024), and irrelevant downstream fine-tuning that reverses unlearning effects (Wang et al., 2025a). To enhance robustness, several algorithmic defenses have been proposed. Tamper-resistant safeguards leverage meta-learning to anticipate weight tampering (Tamirisa et al., 2024), while latent adversarial training improves resilience in the representation space (Sheshadri et al., 2024). Fan et al. (2025) cast robust unlearning as a min-max optimization problem and apply sharpness aware minimization (SAM) and smoothness-indus variant risk minimization (IRM) has been employed to mitigate vulnerabilities from irrelevant fine- regularization, such as Jensen-Shannol diver-
techniques. In-
tuning (Wang al., 2025a), and divergence- gence, has also been introduced to strengthen robustness (Singh et al., 2025). Beyond optimization strategies, other works explore robust data filtering and pre- pre-training methods resist harmful weight tampering (O' O''Brinn et al., 2025).
Optimization for LLM unlearning. The LLM unlearning problem is typically formulated as an optimization task, making it natural to study through the optimization lens. A notable example is Jia al. (2024b), who introduced second-order unlearming (SOUL) by linking iiffuence-functionn based unlearning (Koh & Liang, 2017) with the second order optimizer Sophia (Liu et al., 2023), thereby enhancing forgetting performance via iterative influence removal. Similarly, Reisizadeh et al. (2025) leveraged bi-level optimization to balance unlearning effectiveness and utility retention, while Fan max robust optimization to improve resilience. Despite these advances, the role of optimizer grade in shaping unlearning robustness has received little attention. In partic- ular, ZO optimization (Liu et al., 2020; Nesterov & Spokoiny, 2017; Duchi et al., 2015; Ghadimi & Lan, 2013), which estimates gradients from function evaluations and finite differences (avoid- ing backpropagation), has not been studied for LLM unlearning. Initial efforts only applied ZO to non-LLM settings, such memory-efficient unlearning (Zhang al., 2025a) and graph unlearning (Xiao et al., 2025), primarily for computational efficiency. Similarly, ZO has also been explored for memory--fficient fine-tuning of LLMs (Malladi et al., 2023; Zhang al., 2024c: Tan et al., 2025; Mi et al., 2025). In this work, we instead examine ZO from a robust unlearning perspective, showing that, even as highly degraded form of optimization, it can enhance the resilience of LLM unlearning against weight tampering.
et al. (2025) adopted
tampering
3 PRELIMINARIES AND PROBLEM STATEMENT: OPTIMIZER "GRADE" VS. UNLEARNING ROBUSTNESS
LLM unlearning setup. L.M unlearning refers to the process of selectively erasing the influence of specific data or knowledge (and the associated model behaviors) from trained model, while preserving its overall usefulness. The aim is to make the model "forget" undesired content (e.g., private, copyrighted, or harmful information) without the cost of retraining from scratch and without impairing its performance on unrelated tasks.
Formally. LLM unlearning is typically cast as regularized optimization problem involving two competing objectives: a forget loss (ft), which enforces the removal of the undesired data/knowl- edge, and retain loss (C,), which preserves the model's general utility. The forget loss is evaluated on the forget dataset Dr using an mnlearnig-sspeiffi objective, while the retain loss is computed on the retain dataset D, using standard objectives such as cross-entropy or KI. divergence (Maini et al., 2024). This yields the optimization problem (Liu et al., 2025):
minimize (r() + (0
where A > 0 is regularization parameter that balances unlearning effectiveness (captured by 4 against utility retention (captured by f,). In (1), the choice of the unlearning objective & determines the specific unlearning method applied to solve the problem. For instance, if & -f,, then gradient
3

---

Published as a conference paper at ICLR 2026
descent optimization effectively leverages the gradient difference between prediction losses on Dr and D, to promote forgetting. This approach is referred to as Gradient Difference (GradDiff) (Liu et al., 2022: Yao et al., 2024b). Alternatively, if 4 is defined via the Direct Preference Optimiza- tion (DPO) (Rafailov et al., 2023) objective by treating the forget data in Dj exclusively as negative samples, then the resulting regative-sample-onff formulation leads to the Negative Preference Opti- mization (NPO) method (Zhang et al., 2024a; Fan et al., 2024) for solving (I). Furthermore, if & is cast as min-max objective against worst-case perturbations (aimed at enhancing unlearning robust- ness, as will be discussed later), then the resulting forget loss corresponds to the Sharpness- Aware Minimization (SAM) objective, giving rise to the -AMebased robust unlearning (Fan et al., 2025).
To evaluate unlcarning performance, we primarily adopt the MUSE benchmark (Shi et al., 2024), which targets copyrighted information removal. MUSE consists of two subsets: unlearning book contents from Harry Potter ("books corpus", MUSE-Boook) and unlearning BBC News articles ("news corpus", MUSE--Nwws) Performance is assessed using three metrics: verbatim memorization on the forget set (D;; VerbMem), knowledge memorization on the forget set (Dr; KnowMeme), and knowledge memorization on the retain set (D,; KnowMem). Unlearning is conducted on two fine-tuned models: ICLM-7B trained on the books corpus and LLaMA2-7B trained on the news corpus. We focus on MUSE because it jointly covers
unlearning evaluation (captured by VerbMem) and knowledge-centric unlearning evaluation (captured by KnowMem). In addition, we also include experiments on the WMDP (Li et al., 2024) and TOFU (Maini et al., 2024) benchmarks in the additional experiment section.
Unlearning robustness challenge. Once a model has been unlearned to erase undesired informa- tion, it is crucial that the forgetting effect remains stable. In other words, the model should be robust post-unlearning against both intentional and unintentional weight perturbations. In this work, we focus on two representative forms of weight tampering studied in LLM unlearning: relearning ar- tacks (Hu et al., 2024; Fan et al., 2025; Deeb & Roger, 2024). which represent intentional perturba- tions aimed at restoring forgotten knowledge, and weight quantization (Zhang et al., 2025b), which reflects unintentional perturbations introduced by model compression.
Relearning attacks exploit data samples that follow the forget data distribution, for example, subsets of Dr (Fan et al.. 2025; Hu et al., 2024) or retain data D, drawn from the same distribution as Pr (Deeb & Roger. 2024). These samples are used to update the unlearned model and test whether the resulting weight perturbations (denoted as 5) can undo the effects of unlearming in (1). thereby resurfacing the forgotten information. Formally, the relearning attack can be expressed as
minimize Ereeeu (0 + Drekum)
where Pn denotes the unlearned model from (1) and Dreteam is the relearn dataset. Unless specified otherwise, we set Dreleam as a subset of DD. Following (Fan et al., 2025). relearn is instantiated by fine-tuning the unlearned model for a fixed number of steps, e.g. 100, which we denote as "Relearn100". Different from relearning attacks, quantization compresses the full-precision weights of the unlearned model into lower precision by reducing the number of bits used to represent them. As shown in (Zhang et al., 2025b), although quantization is benign compression technique, it can unintentionally undermine unlearning by shifting parameters toward regions in the loss landscape that resurface forgotten knowledge.
The "grade" of an optimizer: Motivation for its link to unlearning robustness. Prior work has begun examine the optimizer's influence on LLM unlearning. Here, we use the term optimizer to refer to the objective-anoosic optimization method employed to solve the unlearning problem in (I). For instance, first-order gradient-basee methods such as Adam (Kingma & Ba, 2014) can be used to implement multiple unlearning approaches like GradDiff and NPO. It has been shown in (Jia et al., 2024c) that the choice of optimizer can impact unlearning effectiveness. For example, second-order optimizers such as Sophia (Liu et al., 2023) closely connect to influence function-based unlearning (Koh & Liang, 2017; Jia et al., 2024c), which estimates and removes the effect of specific training data model. However, prior work has examined the optimizer's 's role in shaping unlearning robustness against weight perturbations like relearning attacks and quantization.
In this work, we introduce a fresh perspective by examining the notion of "optimizer grade" and its relationship to the grade of unlearning robustmess. By "optimizer grade", we refer to the level of (descent) information optimizer leverages to guide the optimization trajectory converging toward a (locally) optimal solution. We can differentiate the optimizer grade based on the order of gradi- ent information an optimizer exploits. For instance, zeroth-order (ZO) optimization methods (Liu
4

---

Published as a conference paper at ICLR 2026
Pre-unlearn
Sophia
Adam
8-bit Adam
1-bit Adam
O before
after
100
50-
40
1g
VerbMemm on
VeroMemm
Krownomm
15
KrowMom
20
10
40
Ire-oningin Sopoia Adam
B-ot adom
1-stt Adam
KnowMem 50
10
tre-ssind
Saphss
Adam
8-bit Asam 1 Adam
KnowMem
(b) MUSE-Books: Performance on Dr and D, Figure 1: Unlearning performance under 4-b weight quantization using NPO on MUSE with different opti- mizers (Sophia. Adam, 8-bit Adam, and 1-bit Adam). Performance is measured by unlearning effectiveness (VerbMem and KnowMem on Dj, left plots in cach sub-figure) and utility (KnowMem on Dr, right plots in each sub-figure). "Pre-unleamn" represents the target model to conduct un unlearning, and "before Q" (the cirele) and "after Q" (the diamond) represent the unlearned models before and after 4-bit weight quantization. (a) Unlearning on MUSE-News. (b) Unlearning on MUSE-Books.
(a) MUSE-News: Performance on Dj and D,
et al., 2020), which approximate gradients through finite differences of objective function values, can be regarded as downgrade of first-order (FO) methods; FO methods, in turn, down- grade of second-order (SO) methods. Furthermore, even within the same order, the optimizer grade can vary depending whether the gradient information is compressed. A well-known example is gradient sign-based FO optimization, such as signSGD (Bernstein et al., 2018), which represents downgrade of standard SGD. Therefore, we focus on optimizer grades from two perspectives: (a) inter-order, comparing zeroth-, first-, and second-order methods; and (b) intra-order, contrast- ing compressed versus uncompressed gradient information within the first order. The problem of interest can thus be formulated as: How does the optimizer grade affect unlearning robustness?
An interesting and, as we will show later, insightful conclusion is that a downgraded optimizer can in fact lead to upgraded unlearning robustness. We motivate it by comparing unlearning robustness under 4-bit weight quantization (via GPTQ (Frantar et al., 2022)) across optimizers of varying or- ders, using NPO on the MUSE benchmark. The optimizers include the SO optimizer Sophia, the FO optimizer Adam, and its downgraded gradient-compressed variants: 8-bit Adam (with 8-bit gra- dient compression) (Dettmers et al., 2022) and 1-bit Adam (with 1-bit gradient compression, also known as signAdam) (Wang et al., 2019). As shown in Fig. 1. before quantization ("before Q"), the unlearning performance of downgraded optimizers (8-bit Adam and 1-bit Adam) is comparable to that of full-precision Adam and Sophia, as indicated by similar VerbMem, KnowMem on DD and KnowMem on D.. However, when the unlearned models are subjected to 4-bit quantization for post-unlearning robustness assessment, the unlearning performance of FO Adam and SO Sophia is substantially worse compared to their downgraded optimizer counterparts (e.g. I-bit Adam), as ev- idenced by increases in VerbMem and KnowMem on Dj. By comparison, the SO optimizer Sophia shows the weakest robustness on Dj after quantization, even worse than the FO Adam. This high- lights clear interplay between optimizer grade and robustness. Focusing on utility measured by KnowMem on D,, we observe that quantized unlearned models gain utility, whereas the original pre-unlearned model suffers utility drop after quantization. This occurs because quantization can partially revert the unlearning effect, thereby casing the tradcoff between forgetting on Dr and re- tention on D,, which in turn boosts utility.
DOWNGRADING THE OPTIMIZER UPGRADES UNLEARNING ROBUSTNESS
Optimizer downgrade via gradient compression. Let denote the descent direction used in the 1-th update of a FO optimizer, with the update rule given by 0++1 O, 7/m, where 1) > 0 denotes a learning rate. For Adam, m, corresponds to the momentum term (i.e, moving average of adaptive gradients) (Reddi et al., 2018), while for SGD, m, is simply the gradient of the objective function. 'The gradient compression replaces the full-precision gradient with a quantized version, obtained through a quantization operator Q(;; N) using the gradient's 's N-bit representation:
0 0, 1-72mm;; N); And if N = 1, then Q(m;; 1) sign)m:).
(3)
where sign(x) denotes the element-wise sign of the vector x. The SGD variant of (3) with N 1 corresponds to signSGD (Bernstein et al., 2018). Similarly, the Adam variants with N = 8 and N = 1 give rise to 8-bit Adam (Detteers et al., 2022) and signAdam (Wang et al., 2019), respectively. It is also worth noting that gradient compression reduces the information available in the descent step (3), yet it still suffices to guarantee convergence of the optimization (Bernstein et al., 20189 2018). As shown in Fig. 1, gradient compression improves unlearning robustness compared to its
5

---

Published as a conference paper at ICLR 2026
uncompressed counterpart under post-unlearning weight quantization. This effect can be explained from (3): When a gradient compression based optimizer is used for unlearning, it naturally improves tolerance weight perturbations. the quantization operator Q(-) effectively acts "denoiser", mapping perturbed weights onto the same discrete bit values.
Optimizer downgrade via Zo gradient estimation and its link to randomized smoothing. The observation that gradient compression yields tolerance to weight perturbations suggests broader principle: if an optimizer inherently tolerates noise, it may also enhance robustness when applied to unlearning. Following this principle, downgrading from FO 70 optimization can also im- prove robustness, since 70 methods estimate gradients via finite differences of objective function values, while still enjoying provable convergence guarantees (Liu et al.. 2020). Formally, the ZO approximation of the FO gradient fx( for objective function f(x) is given by
1 Σ fxx lui)) f(x p(u)) Q
f(()
(4)
24
where [u:}=1 are random direction vectors (e.g., sampled uniformly from the unit sphere), and u > 0 is the perturbation size used for finite differences. As shown theoretically in (Liu et al., 2018), the ZO gradient estimator is an unbiased estimator (4) of the gradient of smoothed version of the original objective function,
[x(x) == E., [[[x nu)], with Vx,(()) = E.,IV (x)).
(5)
where the expectation taken over the random direction vector u. Therefore, employing a ZO gradient estimation-base optimizer is equivalent to solving randomized smoothing (RS) (Cohen et al., 2019) of the original problem (Liu et al., 2020), where V (x) serves as stochastic gradient estimate of the smoothed objective. is clear from (5) that RS inherently incorporates random noise into the optimization process. Indeed, minimizing RS-type unlearning objective (with FO optimizer) has been shown to improve unlearning robustness (Fan et al., 2025).
There exist many variants of ZO optimization methods. For LLM unlearning we emphasize two choices. First, sampling random vectors from the unit sphere distribution rather than a Gaussian yields more stable unlearning by reducing gradient estimation variance (Ma & Huang, 2025). Sec- ond, we adopt the AdaZO optimizer (Shu et al., 2025), a state-of-the-ar method that further reduces variance and improves convergence. Unless otherwise specified, ZO refers AdaZO.
Adam
signSGD
signAdam
RS
Zo
O before Q + after Q
40-
Know/Mem on
60
Verbakemn
KnowWeth
20
Verblkem
20
10
15
Adam
Mignsaa
signsddam
x
10
15 20 25
Adam
Mgpsco
agnddam
2o
KnowMem on D.
KnowMem on Dr4
(a) GradDiff quantization
(b) NPO quantization
Unleam
Releam110
100
50
31.8
740
on
60
VerbMem
KnownWem
50
Knowllem
50
30
30
20-1
10
20
550
signadamiss MS 70
550 550 sign samass
70
(c) GradDiff relearning
(d) NPO relearing
Figure 2: On MUSF-Books, (a-b): Unlearning performance under 4-hit weight quantization using GradDiff and NPO with different optimizers (Adam, signSGD. signAdam. (FO) RS, ZO method). The figure format is consistent with Fig. 1. (c-d): Unlearn performance with relearning 100 steps ("Relearn100"). using GradDiff and NPO with different optimizers.
Enhanced unlearning robustness to weight quantization via downgraded optimizers. Extend- ing Fig. I by incorporating additional downgraded optimizers beyond Adam (including signSGD, RS, and AdaZO). Fig. 2(a-b) reports the initial unlearning performance on MUSE-Bood Books ('before Q"" and the performance under 4-bit weight quantization ("after Q"), using GradDiff and NPO as the unlearning methods. Consistent with Fig. 1, the 1-bit compressed optimizers signAdam and signSGD improve quantization robustness compared to Adam. Likewise, the first-order RS-based

---

Published as a conference paper at ICLR 2026
optimization also achieves both effective unlearning before quantization and improved robustness after quantization. The ZO optimizer, viewed as the ZO downgrade of RS, shows more nuanced behavior. Prior to quantization, ZO exhibits weaker unlearning for both GradDiff and NPO, it yields higher VerbMem and KnowMem on Dj and lower KnowMem on D,- However, after quanti- zation, 70 demonstrates remarkably strong robustness: it attains substantially lower VerbMem and KnowMem on Dj than other methods. This pattern holds across both GradDiff-- and NPO-based un- learning. The tradeoff is that ZO yields the weakest utility, reflecting its downgraded optimization accuracy. As will be shown later, we can leverage ZO's s robustness benefits to improve FO-based unlearning via hybrid approach that integrate ZO with FO.
Zo exhibits stronger robustness than other optimizers against relearning. Fig. 2(c-d) shows unlearning robustness under relearning attacks. Among first-order downgraded optimizers, FO RS performs the best, with lower VerbMem and KnowMem on Dr after 100 relearning steps ("Re- learn for both GradDiff and NPO, consistent with literature on smoothness optimization (Fan et al., 2025). In contrast, signAdam and signSGD show only occasional gains over Adam. The most notable improvement comes from ZO, which consistently yields the lowest VerbMem and KnowMem Dj across both unlearning methods. Results robustness to relearning (Fig. 2(c- d)) and weight quantization (Fig. 2(a-b)) highlight the distinctive advantage of the downgraded ZO optimizer over RS, mundiett-oopresseed FO, and standard FO. We hypothesize that ZO guides un- learning into a different optimization basin, yielding distinct dynamics and greater robustness.
VerbMern on D,
KnowMem on
KnowMem on D,
50
50
50
50
25
25
25
25
0.25 0.5
0.75
0.75
0.25
0.25
0.5
0.75
(signSGD)
(Adam)
(signAdam)
(Adam)
(RS)
(Adam)
(20)
(Adam)
Figure 3: Linear mode connectivity (LMC) between downgraded optimizers (signSGD, signAdam, RS, and ZO) and Adam on MUSE- Books, using NPO.
To validate the distinctiveness of Zo optimizers, we use linear mode connectivity (LMC) (Frankle al., 2020; Qin et al., 2022; Lubana al., 2023; Pal al., 2025) to compare converged unlearning solutions from two optimizers. LMC assesses whether two unlearned models can be connected by linear interpolation in parameter space. Formally, for 0, and 02, LMC holds if the unlearning met- ric (e.g., KnowMem on D2)) of ((a) o0, + (1 a)02 remains consistent a E [0, varies. Fig. 3 shows LMC between models unlearned with downgraded optimizers and Adam. Gradient- compressed optimizers (signSGD, signAdam) display clear connectivity with Adam: VerbMem on Dr, KnowMem on Dt, and KnowMem on D, remain stable across interpolation, indicating conver- gence to the same basin. In contrast, ZO lacks LMC with Adam, implying convergence a separate basin supporting its distinctive unlearning and robustness.
5 BEST OF BOTH WORLDS: LLM UNLEARNING VIA HYBRID OPTIMIZATION
As indicated by 3, FO and 70 optimizers converge to different basins: FO yields stronger unlearning but limited robustness, whereas ZO offers weaker unlearming before quantization and relearning yet greater robustness to weight perturbations, due to the perturbation tolerance of its gradient estimation and optimization. This raises the question of whether integrating Zo into FO can achieve both effective unlearning and robustness beyond the standard FO optimizer.
Recall that ZO is inherently noisier than FO due to gradient estimation variance (4), which limits optimization efficiency (Liu et al., 2020). To address this, we propose a hybrid FO-ZO method ("Hybrid")) FO optimization (Adam by default) is applied to the pre-unlearned model 0 for N steps, producing 0x: then 70 optimization (AdaZ0 by default) continues for another N steps to obtain 02N. This alternation repeats, ending ZO round, so the final model is OAN for k alternating rounds.
Rationale behind FO-ZO hybrid: A follower-leader game. In the proposed hybrid strategy, the alternation between FO and ZO naturally integrates their optimization effects. This can be viewed as two-player game: the high-grade FO optimizer acts player that solves the unlearning problem with high precision, while the ZO optimizer introduces noise, effectively solving a random- smoothing objective that enhances tolerance weight perturbations. However, find that starting
7

---

Published as a conference paper at ICLR 2026
Pre Pre-urleamm
Adam
SAM
sign-Adam
Hybrid
O
before
after
Verbieem on
on
50
60
55
50
40
30
Pre me-unioa Roam
SAM signtou
mybra
Pre
SAM
signadamy
My brin
KnowMem
KnowMern
(a) GradDiff quantization
(b) NPO vs. quantization
Unicarn
Relcam100
100
46.2
on Dy
40
VerbMern
70
KnowWen
50
20
20
20-
05
32
00
06
98
0,5
00
3.5
88
A00m
vggadamm Hyloid
Adam
SAM
signddamn
Hyand
Adamt
SAM
ugnddam
Hylrld
Adam
SAM
Mgnadain
Hybrld
(c) GradDiff relearning
(d) NPO relearinn
Figure 4: (a-b): Unleaming performance before and after 4-bit quantization on MUSE-Books using GradDiff
and NPO with optimizers Adam, SAM, Aigm and Hybrid FO-ZO. (c-d): GradDiff and NPO on MUSE- Books under different optimizers against "Relearn 100" (100 relcaming steps). The figure format follows Fig. 2.
with the FO optimizer and ending with the ZO optimizer yields stronger unlearning robustness and a more stable optimization process, consistent with our design goal. The rationale behind the hybrid schedule is that this two-player game can be viewed leader-follower game (also known bi- level optimization) (Zhang et al., 2024b). Since unlearning robustness is the primary goal, the 70 optimizer should be treated the "leader." Mcanwhile, the FO optimizer, as a high-grade optimizer with stronger unlearning effectiveness, acts as the "follower," providing a high-quality initialization for ZO and reducing the variance introduced by ZO gradient estimation.
Hybrid optimization achieves both strong unlearning effectiveness and robustness. In Fig. 4, we show that the "Hybrid" optimizer demonstrates superior robustness to both weight quantiza- tion (Fig. 4(a-b)) and relearnngg (Fig. 4(c-d)), outperforming gradient-compressed signAdam, stan- dard Adam, and SAM (sharpness-awe) aware minimization with explicit robust design). As shown in Fig. 4(a-b), before quantization, Hybrid achieves superior unlearning effectiveness, evidenced by the lowest VerbMem and KnowMem scores on Df, while preserving utility as measured by Know on Dr.. This stands in sharp contrast to the ZO optimizer in Fig. 2, where robustness gains come at the cost of utility loss. After quantization, Hybrid maintains consistent robustness benefits, with utility drops similar to those of the original model. Notably, its robustness gains even surpass SAM, de- spite SAM's explicit robustness design in the unlearning objective. Fig. 4(c-d) further demonstrates Hybrid's robustness against relearning. For both GradDiff- and NPO-based unlearning, Hybrid achieves substantially lower VerbMem and KnowMem after Relearn100.
6 ADDITIONAL EXPERIMENTS
In this section, we provide additional experiments validating the link between optimizer grade and unlearning robustness grade, including evaluations on WMDP (Li et al., 2024), TOFU (Maini al., 2024), and supportive experiments for our proposal.
Experiment setups. We further evaluate on the WMDP benchmark, which tests harmful knowl- edge removal via LLM unlearning. Following the robustness protocol in (Fan et al., 2025), we fine-tune unlearned models on a small subset of forget samples for varying epochs. Experiments use Hoppyr-7BB-bes
with two stateful unlearning algorithms: representation misdirection for unlearning (RMU) (Li al., 2024) and NPO. Baselines include Adam, signAdam. ZO, and SAM, compared against our Hybrid. Unlearning effectiveness is measured by test accuracy on WMDP- Bio, while utility is measured by accuracy on MML.U (Hendrycks et al., 2020); effective unlearning corresponds to low WMDP-Bir and high MMLU accuracy. We further validate the proposed Hy- brid method on the TOFU benchmark (Maini et al., 2024), designed for fictitious unlearning on a synthetic QA dataset. Using NPO under the forget10 scenario, the goal is to erase memorization of fictitious authors. The target model is LLaMA2-77 7B finettuned on the dataset corpus. Evaluation uses three metrics: (i) Probability on D; (Prob.), (ii) ROUGE-L on D; (Rouge), and (iii) Model util- ity (MU), aggregating memorization on D,, real authors, and world knowledge. Effective unlearning corresponds to low Prob./Roue and high MU.
8

---

Published as a conference paper at ICLR 2026
*
Pre-unleam
Adam
SAM
signAdam
Zo
Hybrid
Acc on WMDP (1)
Acc on WMDP (.1
Acc on WMDP (
Acc on WMDP
ACc on MMLU (T)
Acc on MMLL
Epochs
Epachs
(a) RMU w/o relearn
(b) NPO w/o relearn
(c) RMU vs. relearn
(d) NPO vs. relearn
Figure 5: Unlearning performance and relearning robustness of RMU and NPO on WMDP-BBL -Bi using different
optimizers (Adam, signAdam. ZO. SAM, and Hybrid). Relearning is conducted by fine-tuning the unlearned model on 40 forget data samples across multiple epochs. (a) Unlcarning effectiveness and utility retention of RMU without relearning; (b) NPO without relcarning; (c) RMU across different relcarning epochs; (d) NPO across different relcarning epochs.
Experiment results on WMDP. As WMDP unlearning is vulnerable to relearnnng attacks Fan et al. (2025), we investigate the role of optimizers before and after such attacks. Relearning is simulated by fine-tuning the unlearned model on 40 forget samples across epochs. Fig. 5 shows WMDP and MMLU accuracy for RMU and NPO (a-b), and robustness under relearning (c-d). The proposed Hybrid consistently outperforms baselines in both settings, notably surpassing SAM-despite its explicit robustness design-whie while retaining comparable or superior unlearning effectiveness before relearning. Another notable observation is that when robustness against relearning is not considered, the ZO optimizer appears inferior to other methods in Fig. 5(a-b), owing to its high optimization variance from ZO gradient estimation, consistent with the MUSE results in Fig. 2. However, once relearning taken into account, the robustness benefit of ZO becomes evident in Fig. 5(c-d), even surpassing Hybrid at larger relearning epochs. This again confirms our key finding that downgrading the optimizer can enhance the robustness of unlearning.
Prob.
Rouge
MU 1
10
Original Retrain
99.8
63.2
50
(1)
14.8
39.9
61.3
Roune
Adam
0.9
0.0
53.2
20
20
ZO
30.4
41.7
50.3
Hybrid
Hybrid
Hybrid
1.8
61.5
50
100
100
Steps
Steps
(a) Unlearn performance on TOFU
(b) Prob. with relearning.
(c) Rouge. with releaming
Figure 6: Unlcarning performance and robustness of NPO using Adam, ZO, and Hybrid optimizer on TOFU under the forget10 scenario. (a) Unlcarning effectiveness of NPO before relearning with different optimizers, evaluated by probability (Prob.), ROUGE-L (Rouge), and model utility (MU). Here, "Original" denotes the pre-unlearned target model, while "Retrain" refers to the model trained solely on the retain dataset, provided by TOFU. (b-c) Robustness against relearning, showing Prob. and Rouge. against increasing relearning steps.
Experiments on TOFU. Fig. presents the NPO-based unlearning performance on TOFU before and after releaming using different optimizers (Adam, ZO, and Hybrid). As shown in Fig. 6(a). Hy- brid consistently matches or outperforms Adam, achieving stronger unlearning effectivenees with lower Prob. and Rouge. and higher MU. In contrast, 70 delivers weaker unlearning prior relearn- ing. However, Fig. 6(b-c) highlights the robustness advantage of ZO and Hybrid over Adam under relearning. as both maintain lower Prob. and Rouge values with increasing steps. Notably, Hy- brid provides the best overall trade-off, combining effective unlearning with resilience to relearning, outperforming Adam and enjoying ZO's robustness.
A Ablation studies on hybrid optimization. We conduct additional experiments on MUSE-Books to provide further justification for the optimizer scheduler design Hybrid optization, detailed in Sec.5. As shown in Fig. 7, changing the switch step N does not materially affect unlcarning's 's ro- bustness against quantization and relearning. Besides, Fig. presents the performance where Hybrid optimization has different steps N for FO and ZO. We can see that allocating an equal number of FO and ZO updates (denoted as FO- 20, ZO= 20) achieves the best balance between unlearning effectiveness and robustness. This outcome is consistent with our method design grounded in the followerr game (Sec. Assigning more FO than ZO steps (e.g., FO 20, Zo= 10) results in decline in unlearning robustness because the "leader" component (ZO updates responsible for steering the model toward robustness) becomes weaker than the "follower" (FO updates that em- phasize high-precision unlearning but do not explicitly promote robustness). Conversely, assigning

---

Published as a conference paper at ICLR 2026
Adam
Hybrid (N=20)
Hybrid (N=5)
Hybrid (N-100)
O before Q
after Q
Uricam
Unicam
Vorakem
Kroofeem
KrrumMamm
214
KnowWlam on
50-22
No000
A0017
10r33
100
20
10r3
1w:100
(a) NPO vs. quantization MUSE-Books
(b) NPO vs. relearning MUSE-Books
Figure 7: (a) NPO-based unlearn performance and quantization robustness of Hybrid optimization with switch steps 20, 5 and 100 (e.g.. "Hybrid (N 20)" represents Hybrid optimization where the FO and ZO optimizer switch every 20 steps). (b) Relearning robustness of Hybrid optimization with different switch steps.
Adam
Hybrid (F0-20.20-20)
Hybrid (F0-10.20-20)
Hybrid (FO-20.20-10)
before
after Q
UMioam
VoldMon
KnowMam on D
10x2 Ressaneet 2001200000
Adon
60x20.0042
70x30
20-50
10 10
Adm
10x20 201200
70.30 2D-10
10-35.00-1
(a) NPO vs. quantization on MUSE-Books
(b) NPO vs. relearning on MUSE-
Figure 8: (a) NPO-based unlearn performance and quantization robustness of Hybrid optimization where FO and ZO have different steps (e.g.. "FO= 10, Zo- 20" represents optimization with Adam for 10 steps and ZO for 20 steps in each round), evaluated MUSE-Book (b) Relearning robustness of Hybrid optimization with different FO and 70 steps.
more ZO than FO steps (e.g., FO= 10, ZO= 20) slightly reduces unlearning effectiveness, since the "follower" (FO) becomes weaker to provide the high-fidelity updates needed to maintain strong unlearning performance in a non-relearnig evaluation setting.
Other ablation studies. In Appx. E, we further validate the robustness of Hybrid by conducting relearning experiments on both Dj and D, for different numbers of steps, and additionally include general utility (i.e., model capabilities that should be preserved but are not explicitly tested in un- learning benchmarks) evaluation for the optimizers discussed in this study. As detailed in the ap- pendix, Hybrid demonstrates consistent robustness both Dj and D, and lower-grade optimizers do not necessarily compromise general utility. Additionally, our Hybrid method is highly efficient and does not incur additional runtime cost.
CONCLUSION
To enhance the robustness of LLM unlearning against post-unlearinng weight tampering (e.g.. re- learning attacks and weight quantization), we investigate the role of optimizer design and demon- strate that downgrading the optimizer can improve robustness. This reveals a novel connection between optimizer grade and unlearning robustness. Among downgraded optimizers, zeroth-order (ZO) methods show weaker unlearning performance (when weight tampering is not considered) but substantially greater robustness compared to first-order (FO) optimizers for unlearning. Build- ing on this insight, we propose a FO-ZO hybrid optimization strategy that augments standard FO unlearning with ZO updates. achieving both strong unlearning effectiveness and enhanced robust- ness. Extensive experiments across multiple datasets validate the benefits of this approach. We refer readers to Appx. A-C for discussions on limitations, ethics statement, and LLM usage.
ACKNOWLEDGEMENT
This work was supported in part by the National Science Foundation (NSF) CISE Core Program Awards IIS 210-2207052 and IIS- (S0-5542,3, the NSF CAREER Award IIS IIS-2338068, the Cisco Research Award, the Amazon Research Award for AI in Information Security, the Open Philanthropy Re- search Award, the Schmidt Sciences Research Award, and the Center for AI Safety (CAIS) Compute Award.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303, 08774, 2023.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: 002993, 2019.
Clark Barrett, Brad Boyd, Elie Bursztein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, et al. Identifying and mitigating the security risks of generative ai. Foundations and Trends@ in Privacy and Security, 6(1):1-52, 2023.
Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, and Animashree Anandkumar. signsgd: Compressed optimisation for non-convex problems. In International conference on machine learning, pp. 560-569. PMLR, 2018
Karuna Bhaila, Minh-Hao Van, and Xintao Wu. Soft prompting for unlearning in large language models. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Con- ference of the Nations of the Americas Chapter of the Association for Computational Linguis- ties: Human Language Technologies (Volume 1: Long Papers), pp. 1046-4056 Albuquerque, New Mexico, April 2025. Association for Computational Linguisties. ISBN 979-8-89176- 189- URL https: / acl
6. doi: 10.18653/v 10//6/////222.../ naacl-long.20 204/.
Zora Che, Stephen Casper, Robert Kirk, Anirudh Satheesh, Stewart Slocum, Lev E McKinney, Rohit Gandikota, Aidan Ewart, Domenie Rosati, Zichu Wu, et al. Model tampering attacks enable more rigorous evaluations of Ilm capabilities. arXiv preprint 1XXi::2522.05200 2025.
Aochuan Chen, Yimeng Zhang, Jinghan Jia, James Diffenderfer, Jiancheng Liu, Konstantinos Parasyris, Yihua Zhang, Zheng Zhang, Bhavya Kailkhura, and Sijia Liu. Deepzero: Scaling up zeroth-order optimization for deep model training. arXiv preprint
2023.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv: arXiv:1803.05455 2018.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. international conference on machine learning, pp. 1310-1320. PMLR, 2019.
Aghyad Deeb and Fabien Roger. Do unlearning methods remove information from language model weights? arXiv preprint arXiv:2410.08822 08827, 2024.
Zhijie Deng, Chris Yuhao Liu, Zirui Pang, Xinlei He, Lei Feng, Qi Xuan, Zhaowei Zhu, and Jiaheng Wei. Guard: Generation time Ilm unlearning via adaptive restriction and detection. arXiv preprint arXiv:2505.133/ 13312, 2025.
Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 8-bit optimizers via block-wise quantization. 9th International Conference on Learning Representations, ICLR, 2022.
John C Duchi, Michael I Jordan, Martin J Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE Transactions on Information Theory, 61(5):2788-28) 28088 2015.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning for lims. 2023.
Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. Sim- plicity prevails: Rethinking negative preference optimization for Ilm unlearning. arXiv preprint arXiv:2410.07163, 2024.
Chongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, and Sijia Liu. Towards Ilm unlearning resilient to relearning attacks: sharpness-awar aware minimization perspective and beyond. arXiv preprint 2.Xi:2502.00374 2025.

---

Published as a conference paper at ICLR 2026
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-awart minimiza- tion for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020.
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode con- nectivity and the lottery ticket hypothesis. International Conference on Machine Learning, pp. 3259-3269. PMLR. 2020.
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210. 2tt.::2210./723333 /7323, 2022.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 2024. URL https::///// wwt.oooo..gg//eceoeeeeeeo
Saced Ghadimi and Guanghui Lan. Stochastic first-and zeroth methods for nonconvex stochas- tic programming. SIAM journal on optimization, 23(4))2341-2365 2013.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint 101i:20009003000, 2020.
Shengyuan Hu, Yiwei Fu, Zhiwei Steven Wu, and Virginia Smith. Jogging the memory of unlearned model through targeted relearning attack. arXiv preprint arXiv:2406.1333 2024.
Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao. Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:2401.0556. 2024.
Jinghan Jia, Jiancheng Liu, Yihua Zhang. Parikshit Ram, Nathalie Baracaldo, and Sijia Liu. Wa- gle: Strategic weight attribution for effective and modular unlearning in large language models. Advances in Neural Information Processing Systems, 37:55620-55644 2024a.
Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. SOUL: Unlocking the power of second-order optimization for LLM unlearning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Pro- ceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 4276-4292, Miami, Florida, USA, November 2024b. Association for Computational Linguis- tics. doi: 10. 11118////w/2222emmmmllmooo emn.ppmmain 245/
URL https: http///aaaanntataaaaeea or2//024.
Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. SOUL: Unlocking the power of second-order optimization for LLM unlearning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 4276 Miami, Florida, USA, November 2024c. Association for Computational Linguistics.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885-1894, PMLR, 2017.
Martin Kuo, Jingyang Zhang, Jianyi Zhang, Minxue Tang, Louis DiValentin, Aolin Ding, Jing- wei Sun, William Chen, Amin Hass, Tianlong Chen, et al. Proactive privacy amnesia for large language models: Safeguarding pii with negligible impact model utility. arXiv preprint arXiv:2502.1 17591, 2025.
Yicheng Lang, Kehan Guo. Yue Huang, Yujun Zhou, Haomin Zhuang, Tianyu Yang, Yao Su, and Xiangliang Zhang. Beyond single-value metrics: Evaluating and enhancing LLM unlearning with cognitive diagnosis. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher
12

---

Published as a conference paper at ICLR 2026
Pilehvar (eds.), Findings of the Association for Computational Linguisties: ACL2025,, pp. 21397- 21420, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8- 89176-256-5. doi: 10.18653/v1/200 1/2333/11222.../// 1/22:....n22...... wwwdiings-al.11000 URL https://aclanthot org/2025.
act. . 1102/.
Dohyun Lee, Daniel Rim, Minseok Choi, and Jaegul Choo. Protecting privacy through ap- proximating optimal parameters for sequence unlearning in language models. arXiv preprini 0rXiv:2406.14091, 2024.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403. 0.2010,,,1,,.. 2024.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv22099007958, 2021.
Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. In Conference on Lifelong Learning Agents, pp. 243-254. PMLR, 2022.
Chris Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. Large language model unlearning via embedding embeeddingcoorupted prompts. Advances in Neural Information Processing Systems, 37:1 18198- 118266, 2024.
Hong Liu, Zhiyuan Li, David Hall, Percy Liang, and Tengyu Ma. Sophia: A scalable stochastic second-order optimizer for language model pre-training. arXiv preprint arXiv:2305. 14342, 2023.
Sijia Liu, Bhavya Kailkhura, Pin- Yu Chen, Paishun Ting, Shiyu Chang, and Lisa Amini. Zeroth- order stochastic variance reduction for nonconvex optimization. Advances in neural information processing systems, 31, 2018.
Sijia Liu, Pin-Yo Yu Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III. and Pramod K Varshney. A primer on zeroth-order optimization signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, 37(5):43-54 2020.
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pp. 1-14, 2025.
Ekdeep Singh Lubana, Eric J Bigelow, Robert Dick, David Krueger, and Hidenori Tanaka. Mech- anistic mode connectivity. In International Conference on Machine Learning, pp. 22965-23004. PMLR, 2023.
Jakub Lucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, and Javier Rando. An adversarial perspective on machine unlearning for ai safety. arXiv preprint 1801 2024.
Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Menell. Eight meth- ods to evaluate robust unlearning in llms. arXiv preprint ..81: 2024.
Shaocong Ma and Heng Huang. Revisiting zeroth-order optimization: Minimum-variane two-point estimators and directionally aligned perturbations. In The Thirteenth International Conference on Learning Representations, 2025.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Lipton, and Zico Kolter. Tofu: A task of fictitious unlearning for 1lms. arXiv preprint arXiv:2401.06121, 2024.
Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems, 36:53038-53077 2023.
Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint 2rXiv:2402.04244, 2024.
13

---

Published as a conference paper at ICLR 2026
Zhendong Mi, Qitao Tan, Xiaodong Yu, Zining Zhu, Geng Yuan, and Shaoyi Huang. Kerzoo: Kernel function informed zeroth-order optimization for accurate and accelerated Ilm fine-tuning. arXiv preprint arXiiv2505. 18886, 2025.
Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17(2):227-566, 2017.
Kyle O'Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, and Stella Biderman. Deep ignorance: Filter- ing pretraining data builds tamper-resistant safeguards into open-weight 1lms. arXiv preprint arXiv:2508. 06601,, 2025.
Soumyadeep Pal, Changsheng Wang, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. Llm unlearning reveals a stronger-than-eppeee coreset effect in current benchmarks. arXiv preprint vrXiv:2504.1018. 2025.
Martin Pawelezyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearning: Language models as few shot unlearners. arXiv preprint arXiv:2310.07579, 2023.
Yujia Qin, Cheng Qian, Jing Yi, Weize Chen, Yankai Lin, Xu Han, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Exploring mode connectivity for pre-trained language models. arXiv preprint arXiv:2210.1 14102, 2022.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728-5374) 2023.
Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of adam and beyond. In International Conference on Learning Representations, 2018. URL net/ fiet/fore Eorum? 1i
Hadi Reisizadeh, Jinghan Jia, Zhiqi Bu, Bhanukiran Vinzamuri, Anil Ramakrishna, Kai- Chang, Volkan Cevher, Sijia Liu, and Mingyi Hong. Blur: A bi-level optimization approach for Ilm unlearning. arXiv preprint vrXiv:2506.0816 2025.
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadficld-Menell, et al. Latent adver- sarial training improves robustness to persistent harmful behaviors in 1lms. arXiv preprint arXiv:2407.15549 2024.
Weijia Shi, Jaechan Lec, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah Smith, and Chiyuan Zhang. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint AxKV:22077.064000 2024.
Yao Shu, Qixin Zhang, Kun He, and Zhongxiang Dai. Refining adaptive zeroth-order optimization at ease. arXiv preprint 25Xiv:2502.01014, 2025.
Naman Deep Singh, Maximilian Müller, Francesco Croce, and Matthias Hein. lasts: Utility presevig,,,,,,,,, robust, and almost irreversible forgetting in llms. arXiv preprint 0rXiv:2509.02820. 2025.
Unlearning that
Yash Sinha, Manit Baser, Murari Mandal, Dinil Mon Divakaran, and Mohan Kankanhalli. Step- by-step reasoning attack: Revealing' erased"knowwi knowledge large language models, arXiv preprint arXiv:2506. 17279, 2025.
Robin Staab, Mark Vero, Mislav Balunović, and Martin Vechev. Beyond memorization: Violating privacy via inference with large language models. arXiv preprint arXiv:2310.07298, 2023.
Vinith M Suriyakumar, Ayush Sekhari, and Ashia Wilson. Ued: Unlearning in 1lms via contrastive decoding. arXiv preprint arXiv:2506.12009 2025.
Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, et al. resisaatt safeguards for open-weight lims. arXiv preprint arXiv:2408.0076/ 2024.
14

---

Published as a conference paper at ICLR 2026
Qitao Tan, Jun Liu, Zheng Zhan, Caiwei Ding, Yanzhi Wang. Xiaolong Ma, Jaewoo Lee, Jin Lu, and Geng Yuan. Harmony in divergence: Towards fast, accurate, and memorryefficient zeroth-order Ilm fine-tuning, arXiv preprint arXiv:2502.03304, 2025.
Pratiksha Thaker, Yash Maurya, Shengyuan Hu, Zhiwei Steven Wu, and Virginia Smith. Guardrail baselines for unlearning in Ilms. arXiv preprint arXiv:2403.03329, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307. 0928088 2023.
Changsheng Wang, Chongyu Fan, Yihua Zhang, Jinghan Jia, Dennis Wei, Parikshit Ram, Nathalie Baracaldo, and Sijia Liu. Rethinking unlearning for large reasoning models. ICML 2025 Workshop Machine Unlearning for Generative Al,
Changsheng Wang. Yihua Zhang, Jinghan Jia, Parikshit Ram. Dennis Wei, Yuguang Yao, Soumyadeep Pal, Nathalie Baracaldo, and Sijia Liu. Invariance makes Ilm unlearning resilient even to unanticipated downstream fine-tuning. arXiv preprint arXiv:2506.013399 2025a.
Dong Wang, Yicheng Liu, Wenwo Tang, Fanhua Shang, Hongying Liu, Qigong Sun, and Licheng Jiao. Signadam++: Learning confidences for deep neural networks. 2019 International Con- ference Data Mining Workshops (ICDMW), pp. 186-195. IEEE, 2019.
Yaxuan Wang, Chris Yuhao Liu, Quan Liu, Jinglong Pang. Wei Wei, Yujia Bao, and Yang Liu. Dragon: Guard Ilm unlearning in context via negative detection and reasoning. arXiv preprint Stti::25/1.057880 2025b.
Yue Wang, Qizhou Wang, Feng Liu, Wei Huang, Yali Du, Xiaojiang Du and Bo Han. Gru: Mit- igating the trade-off between unlearning and retention for 1lms. In Forty-second International Conference Machine Learning, b.
Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. DEPN: Detecting and editing privacy neurons in pretrained language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empir- ical Methods in Natural Language Processing, pp. 2875 2886, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18653/v1/2 wwtp2/2mmmmmmmainn.... https: fttps://aclantholl wttpg/orgg/22203 http//222.emmlpp....... emnlp ma 174/.
URL
Yang Xiao, Ruimeng Ye, Bohan Liu, Xiaolong Ma, and Bo Hui. Efficient knowledge graph unlearn- ing with zeroth-order information. arXiv preprint (tti::2508..40111 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025a.
Puning Yang, Qizhou Wang, Zhuo Huang, Tongliang Liu, Chengqi Zhang, and Bo Han. Exploring criteria of loss reweighting to enhance 11m unlearning. arXiv preprint arXiv:2505. 1953, 2025b.
Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. A survey large language model (Ilm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, 4(2): 100211, 2024a.
Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning, The Thirty-eight Annual Conference on Neural Information Processing Systems, 2024b.
Hongbang Yuan, Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robust- ness in large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 25769-25777. 2025.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a ma- chine really finish your sentence? arXiv preprint arXiv: /00::090500000 2019.
15

---

Published as a conference paper at ICLR 2026
Ci Zhang, Chence Yang, Qitao Tan, Jun Liu, Ao Li, Yanzhi Wang. Jin Lu, Jinhui Wang, and Geng Yuan. Towards memory-efficient and sustainable machine unlearning on edge using zeroth-order optimizer. In Proceedings of the Great Lakes Symposium VLSI 2025, pp. 227-232, 2025a.
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catas- trophic collapse to effective unlearning. arXiv preprint 1.Xiv:22404.08866 2024a.
Yihua Zhang. Prashant Khanduri, loannis Tsaknakis, Yuguang Yao, Mingyi Hong, and Sijia Liu. An introduction bilevel optimization: Foundations and applications in signal processing and machine learning. IEEE Signal Processing Magazine, 41(1):38-59) 2024b.
Yihua Zhang, Pingzhi Li, Junyuan Hong, Jiaxiang Li, Yimeng Zhang, Wenging Zheng, Pin- Yu Chen, Jason D Lee, Wotao Yin, Mingyi Hong, et al. Revisiting zeroth-order optimization for memory-efficient Ilm fine-tunings A benchmark. arXiv preprint 2xXiv:2402.11592 2024c.
Zhiwei Zhang. Fali Wang. Xiaomin Li, Zongyu Wu, Xianfeng Tang, Hui Liu, Qi He. Wenpeng Yin, and Suhang Wang. Catastrophic failure of Ilm unlearning via quantization. arXiv preprint arXiv:2410. 16454, 2024d.
Zhiwei Zhang, Fali Wang, Xiaomin Li, Zongyu Wu, Xianfeng Tang, Hui Liu, Qi He, Wenpeng Yin, and Suhang Wang. Catastrophic failure of LLM unlearning via quantization. In The Thirteenth International Conference on Learning Representations, 2025b. URL UtLhtttp:///ppmme //pp////eeene.e... net/ Eorum? wwd.d=lHeeeaammmm ammz.
Xin Zhou, Yi Lu, Ruotian Ma, Yujian Wei, Tao Gui, Qi Zhang, and Xuanjing Huang. Making harmful behaviors unlcarnable for large language models. In Lun- Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp.
Bangkok, Thailand, August 2024. Association for Computational Linguis- http://......inns-.eeeee aadings-acl.611. URL https: / or2/2024.
tics. doi: 10.18653/w/20244
findings- acl. 611/.
Haomin Zhuang, Yihua Zhang, Kehan Guo, Jinghan Jia, Gaowen Liu, Sijia Liu, and Xiangliang Zhang. SEUF: Is unlearning one expert enough for mixture-of-expen LLMs? Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8664-8678, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-25144 doi: 10111865/1//222..oo acl-long 424, URL https: / /aclannthooog.. org/2025 acl long. 424/.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A LIMITATIONS
While we conduct comprehensive experiments and in-depth analysis to show the role of optimizers in robust LLM unlearning, certain limitations persist in our study. There are other optimizers we did not include in our study, e.g., Muon optimizer and the Shampoo optimizer. Also, our methods and insights could be extended to relevant and important fields, such as safety alignment, which we did not include this work. Additionally, there ncedsstudy on whether the downgrade of optimizers improves robustness in general.
B ETHICS STATEMENT
The datasets used in this paper are from publicly available sources and do not contain sensitive or private information. Our research focuses on the LLM unlearning, which erases private or harmful data memorization in LLMs and enhances LLM safety. By studying optimizer design and integrating hybrid optimization, we further improve the robustness of unlearning. making it less vulnerable to most-unlearning weight tampering.
C LLM STATEMENT
In this paper, the sole purpose LLMs is to assist with improving the fluency of the paper, such refining the grammar. At point did the language model contribute to research ideas to the generation of original content.
D DETAILED EXPERIMENT SETUP
Settings on MUSE. For the first-order (FO) optimizers (Adam, gradient-compressed Adam, and RS) on both MUSE-Books and MUSE-News, we fix / in NPO to 0.1, and tune the learning rate in the range [5e-6, 1e-5]. On MUSE-Booke we perform unlearning for I epoch and tune the retain loss coefficient À for GradDiff and NPO in {1.0, 10.0, 20.0, 50.0} via grid search. On MUSE- News, we conduct 5 epochs unlearning, saving checkpoints per epoch, and select checkpoint with the best retain performance the final model.
For the zeroth-order (ZO) methods and Hybrid, we also fix 3 in NPO to 0.1 and make the same grid search for A. We tune the learning rate via grid search in [le-5, 5e-5] and conduct 1000 steps of unlearning, checkpointing every 100 steps to select the model with the best retain performance. In Hybrid, switch the optimizer every 20 steps MUSE-Books and every 50 steps MUSE- News.
Settings on WMDP. For both NPO and RMU using FO optimizers, perform 150 unlearning steps. For NPO, we fix to 0.1 and tune the hyperparameters via grid search: learning rate e [5c-6, 1c- 1c-5) 5) and À (1.0, 2.5). For RMU, we follow the default settings proposed in Li et al. (2024).
For NPO and RMU using ZO and Hybrid, perform 400 unlearning steps and checkpoint ery 100 steps, selecting the model with the best utility. We tune the learning rate via grid search in (1e-5, 5e-5] and employ the same values in the FO setting. For Hybrid, switch the optimizer every 20 steps.
Settings TOFU, We fix 3 in NPO to 0.1 and tune {1.0, 2.5). For the FO setting, fix the learning rate to le-5. For ZO and Hybrid, we tune the learning rate via grid search [1e-5, 5e-5). For Hybrid, switch the optimizer every 20 unlearning steps.
17

---

Published as a conference paper at ICLR 2026
Adam
SAM
Hybrid
100
150
120
200
200
VeriMMem on Dy
VeriMMem Dy
100
120
VeroMem
VeraMem
50
330
63
30
25
15 (+)
38
3:
40
6 (r(r)
35
(-]
Dj
Dj (r)
(a) GradDiff. relearn on D)
(b) NPO. relear D2
(c) GraDifff relear en D,
(d) IPO. relearn D.
Figure A1: Robustness of Grad and NPO on MUSE Books against relearning on Dr and D, across different numbers of relearning steps. The initial unlearned models at "step 0" are obtained using Adam, SAM, and Hybrid optimizers, respectively.
E ABLATION STUDIES.
Additional experiments on bybrid optimization. We evaluate the robustness of the proposed Hybrid optimizer under two relearning settings: using the forget set Dr and the retain set D,. While earlier experiments considered the worst-case robustness scenario with D; relearning samples, our results show that Hybrid maintains robustness even when the relearning set is drawn from D,, demonstrating its resilience beyond the worst-case setting. Fig. A1 shows that Hybrid consistently outperforms Adam and SAM, achieving lower KnowMem and VerbMem on Dj across the relearn- ing path. Moreover, Hybrid not only surpasses SAM with its explicit robustness design against relearning attacks but also demonstrates stable resilience when fine-tuned on D,.
General utility evaluation. We employ the following benchmarks and the An-evall an--evallaaaesss harness library (Gao et al., 2024) to evaluate the general utility of the unlearned models. These benchmarks target different aspects of reasoning, factuality, and commonsense competence:
Hellaswag (Zellers et al., 2019) measures a model's ability to perform commonsense reasoning in everyday situations. It presents context and several possible sentence completions. High performance indicates strong capability in narrative understanding and choosing contextually ap- propriate continuations.
TruthfulQA (Lin et al., 2021) evaluates the truthfulness and factual of model responses.
ARC-Challenge (Clark et al., 2018) focuses on scientific question answering at the level of ele- mentary and middle school multiple-choice exams.
As shown in Table. A1, models trained with down-graded optimizers and Hybrid etain competitive performance relative to the upgraded FO optimizer, confirming that lower lower-grade grade optimizers do not necessarily compromise general utility.
Table A1: General utility evaluation of different optimizers across unlcarning benchmarks and methods. The values the table are the average of hellaswag, truthfulga and evaluation scores.
Optimization Methods
Dataset
Pre-unlearn
Method
Adam
signAdam
SAM
Zo
Hybrid
GradDiff
32.6
32.9
28.1
26.4
30.1
MUSE-Books
36.2
NPO
27.2
28.5
27.6
32.9
25.2
GradDiff
37.0
37.7
37.4
36.5
36.7
MUSE-News
41.9
NPO
38.8
40.9
32.7
37.0
37.3
RMU
53.4
49.1
52.9
41.5
48.9
WMDP
53.3
26.7
Run time evaluation. As shown in Table. A2, downgraded optimizers such ZO and Hybrid achieve comparable wall-clock time to standard first-order optimizers. While ZO uses multiple function evaluations, the absence of backward passes and the limited number of unlearning iterations make its cost practically similar. We also observe that downgraded optimizers are significantly efficient than the robust-optimization baseline SAM, whose sharpness-aware updates require
18

---

Published as a conference paper at ICLR 2026
Table A2: Run time (in minutes) of different optimizers for GradDiff and NPO-based unlearning MUSE,
Optimization Method
Dataset
Unlearning Ohjective
Adam
SignSGD
signAdam
SAM
20
Hybrid
GradDiff
15.2
14.8
15.0
15.4
30.6
18.9
17.7
MUSE-Beoks
NPO
18.9
17.7
17.8
18.1
35.9
22.8
21.9
GradDiff
33.2
31.1
32.6
35.9
85.7
40.8
41.7
MUSE-Nows
NPO
39.3
38.5
39.0
39.8
81.3
40.9
39.3
an expensive inner loop. In contrast, our Hybrid method alternates between FO and ZO updates without introducing any nested optimization, preserving efficiency while improving robustness.
F ADDITIONAL RESULTS
We show the unlearning performance with quantization and relearning for GradDiff and NPO on MUSE-News, using the down-graded optimizers, in Fig. A2. LMC of NPO on MUSE-News is shown in Fig. A3. We further show the performance of Hybrid on MUSE-NewS with NPO and GradDiff, against relearning and quantization, in Fig. A4.
For both the study downgraded optimizers and hybrid optimization, the experiment results on MUSE-News aligned with MUSE-Books: For instance, shown in Fig. A2(a-b), ZO achieves best performance with 4-bit quantization ("with Q"). Fig. A2(c-d) further demonstrates the ro- bustness of ZO against relearning, where ZO with both GradDiff and NPO acheives the lowest KnowMem and VerbMem on Dj after relearning 100 steps.
Adam
signSGD
signAdam
RS
Zo
before Q
+ after Q
Know
KocasMamm
Adoth
Spriso
A0S0
SensGD
Growitem
(a) GradDiff quantization
(b) NPO quantization
Relear1000
Krr
winadom
3900360
Sry000
Aigi 85
wgpddon 85
(c) GradDiff vs. relearning
(d) NPO vs. relearning
Figure A2: Unlearning performance and robustness using GradDiff and NPO on MUSE-News with different optimizers (Adam, signSGD, signAdam, (FO) RS, 70 method). (a-h) shows unlcarning's robustness against 4-bit quantization, and (c-d) shows unlearning's robustness against relearning 100 steps ""Relearn100")
VerbMem
KnowMem on D,
KnowMem on D,
50
50
0.25
0.5
0.25
0.5
(signSGD)
(Adam)
(signAdam)
(Adam)
(RS)
(Adam)
(20)
(Adam)
Figure A3: Linear mode connectivity (LMC) between downgraded optimizers (signSGD, signAdam, RS, and ZO) and Adam on MUSE-News, using NPO unlcarning.
19

---

Published as a conference paper at ICLR 2026
The effectiveness of hybrid optimization is also demonstrated MUSE-NeWE as Fig. A4 illus- trates. Across GradDiff and NPO, Hybrid yields unlearn performance on par with Adam. Espe- cially with the NPO algorithm, Hybrid shows clear robustness advantage against both quantization (Fig. A4(b) and relearning (Fig. A4(d)) compared to the baseline optimizers (e.g., Adam and SAM).
signAdam
Hybrr
ahter
VeltHam
d
oiglaaen mosse
93 33
opoadan meperd
KnowMem
KnowMem
(a) GrafDiff quantization
(b) NPO quantization
Relear1100
KosuNemon
KnrosNarm
sggbddem inpoldd
SAI
nguldam
Indldd
Nomm
syntdam
sigeecao
(c) GradDiff vs. relearning
(d) NPO vs. relearning
Figure A4: Unlearinn performance before and after 4-bit quantization using GradDiff and NPO on MUSE-News with the optimization methods: Adam, sharpness-awan minimization (SAM), signAdam and hybrid FO-ZO optimization (Hybrid). (c-d): GradDiff and NPO with different optimizers against relearning 100 steps. The figure format is consistent with Fig. 2.
Methed
VerbMem (3)
KnowMem (1)
Retain (T)
Unility
Wrl
W/
Wio
Wy
Truhful
ARC Chaillnge
Pre-unlearn
36.8
21,4
NPO signSGID
22.2
NPO sigm
30.7
25.8
35.9
52.5
23.6
33.2
28.5 8
NPO wi RS
0.0
34.5
23.4
54.6
49.9
23.4
31.1
28.2
NPO SAM
o.
30.0
0.0
22.5
35.2
46.0
23.6
207
26 7
NPO w/ 20
20.7
16.2
23,9
16.4
36.6
21.1
18.5
41.7
35.5
NPO w/ Hybxid
0
54.5
38.5
23.8
28.4
23.5
GraDiff
60.5
5.9
31.6
46.0
51.0
22.4
41.7
33.6
GradDidt
2.81
29.2
33.0
Gmdliifff signAdam
GradDift
GradDiif or
21.4
42.1
GradDiff wi
11.5
9.2
4.0
263
20.3
36.7
27.4
GraDDaft w/ Hybrid
48.5
38.2
24.1
30.8
24.2
Table A3: Unlearning evaluation and general utilities on MUSE-Books, using GradDiff and NPO.
Methed
VerbMem (j)
KnowMem (1)
Retain (T)
Lisiity
Wio atk
W/
W/o atk
W/
Truthful QA
Hellaswag
ARC- Challenge
Pre-unlearn
64.0
54.4
26.9
NPU SignSGID
43.5
37.9
Aigm
45.6
35.3
28.9
40.0
NPO RS
5.0
29.5
43.4
46.9
43.3
45.7
26.8
53.0
37.0
NPO SAM
0.
19.5
42.0
47.6
35.2
42.0
26.1
42.7
20.4
NPO w/ ZO
21.3
17.3
39.2
26.3
41.2
29.0
23.5
$0.7
36.8
NPO w/ Hybeld
8.9
8.4
35.9
32.0
38.9
34.0
26,4
49.7
35.7
GodDiif
5.9
26.2
30,9
48.2
51.5
26.9
56
42.7
GradDeff sqnSGD
27.2
49.5
26.1
57.2
GidiDiff
43.1
CrddDit
GradDur
51.3
GradfTir
29.6
15.8
2.6
36.6
GrdDiff Ilybrid
9.2
19.0
25.7
40.0
29.2
33.8
23.3
52.2
35.9
Table A4: Unlearning evaluation and general utilities on MUSE-News, using GradDiff and NPO.
20


Reviews:
[{"review_id": "MpCgsYPprI", "reviewer": "Reviewer_dqWt", "summary": "The paper investigates how optimizers of different “grades”—namely 0th-order, 1st-order, and 2nd-order—affect the process of machine unlearning. The authors observe that lower-grade optimizers, while less effective at unlearning, are more robust to relearning. Building on this insight, they propose a hybrid optimizer that combines 0th- and 1st-order methods to achieve a balance between unlearning effectiveness and robustness. Experiments conducted on MUSE and WMDP support the findings.", "questions": "See Weaknesses, and also\n1. Lines 177–180 appear to be repeated\n1. Line 366: Should $k$ be even?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* A timely and relevant exploration of the relationship between optimizer order and unlearning dynamics.\n* The finding that lower-grade optimizers are more robust to relearning is intriguing and adds nuance to our understanding of unlearning mechanisms.\n* The paper examines multiple optimizer variants, including SignedSGD, SignedAdam, and low-bit quantization, providing a richer empirical picture.", "weaknesses": "* While the observation that lower-grade optimizers are robust to relearning is interesting, the underlying cause may not be adequately analyzed. The discussion already hints that the robustness could stem from higher noise levels in these optimizers, which also naturally explain their weaker unlearning effects. If so, the “grade” of the optimizer might be a proxy variable rather than the true causal factor—making noise the key driver of robustness.\n* The discussion of computational cost is largely absent. If noise is indeed the mechanism behind robustness, it could impose significant efficiency costs on the unlearning process. Consequently, the proposed hybrid approach may also entail higher cost. Since cost–performance trade-offs are crucial for practical deployment, this omission weakens the empirical contribution.\n* The evaluation is limited to only three datasets, which feels insufficient for a paper with primarily empirical claims. Broader validation would help establish generality and robustness of conclusions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nJVXKo2zj7", "reviewer": "Reviewer_bZEQ", "summary": "This paper explores an underexplored dimension of LLM unlearning: the impact of the optimizer on the robustness of forgetting. While most prior work has focused on modifying unlearning objectives, this study reveals that the \"grade\" of the optimizer from zeroth-order (gradient-free) to second-order (Hessian-based) influences the resilience of unlearning to post-hoc operations like quantization or fine-tuning. The authors find that lower-grade optimizers (e.g., zeroth-order or compressed-gradient optimizers) improve robustness by converging to flatter, harder-to-disturb basins in the loss landscape. Zeroth-order methods are naturally linked to randomized smoothing, which enhances resilience against perturbations. Based on these insights, the authors propose a hybrid optimizer combining first- and zeroth-order updates to strike a balance between unlearning efficacy and robustness.", "questions": "Please refer to the Weaknesses section for a detailed discussion.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper focuses on robust unlearning in the optimization process, showing a fresh research insight.\n\n2. The proposed hybrid strategy effectively balances precise convergence (first-order) with robust basin discovery (zeroth-order).", "weaknesses": "1. The assessment of utility preservation is limited. Benchmarks such as MMLU should be included to evaluate general knowledge retention, and for the ToFU dataset, real-author and world-fact evaluations should be considered to better reflect practical unlearning scenarios.\n\n2. The paper does not examine how varying the number of unlearning steps affects performance. Analyzing different unlearning durations would clarify the stability and convergence behavior of the proposed method.\n\n3. No experiments are provided to study how different step counts between the two optimization methods influence results. Moreover, there is no theoretical justification for using the same number of steps (N) for both optimizers. A rationale or ablation study would strengthen the methodological soundness.\n\n4. The proposed approach primarily combines two existing optimizers without introducing substantial conceptual innovation.\n\n5. The robustness claim is insufficiently supported. To substantiate it, the method should be evaluated under membership inference and adversarial attack settings, which are standard in assessing unlearning and privacy resilience.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tts08QhfZJ", "reviewer": "Reviewer_dWKo", "summary": "This paper presents a new perspective on robust large language model (LLM) unlearning—the process of selectively erasing specific knowledge from models while maintaining their general capabilities. Prior studies have largely focused on modifying the unlearning objectives or introducing explicit robustness regularizers (e.g., sharpness-aware or meta-learning methods). In contrast, this work investigates the role of the optimizer itself in shaping the robustness of unlearning outcomes, asking whether the optimizer’s complexity—or “grade”—affects how resilient forgetting remains under post-unlearning perturbations such as relearning attacks and weight quantization.", "questions": "Kindly please see the drawbacks above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "The authors are obviously the experts in the field. The draft is well structured, insightful, and easy to read. Although the current draft has some minor limitations, I will definitely improve my score when they are solved. \n\nThis paper Introduces “optimizer grade” as a new dimension for understanding unlearning dynamics, shifting focus from objective design to optimizer behavior.\n\nThis paper demonstrates that weaker or noisier optimizers (e.g., sign-based, ZO) can improve robustness. It is an impactful finding that challenges conventional assumptions about optimization fidelity.\n\nThe proposed FO–ZO hybrid method is straightforward to implement, and can be integrated with existing unlearning systems without redesigning objectives.", "weaknesses": "Some previous works, such as those combined with IRM and SAM, can be viewed as using higher order gradients, which are shown to be more robust to attacks. They somehow conflict the opinion of this paper, instead showing that lower order methods are better. Could the authors explain further about these different conclusions?\n\nDue to the use of momentum, Adam is not strictly a first-order approach but between first and second orders. From my view, only vanilla SGD is first ordered. So, maybe SGD should also be adopted for comparison. \n\nA weird thing is that, the second order approaches are mentioned in the abstract, but not involved in the main context. A remember some papers have claimed the use of natural gradient for LLM unlearning by assumption diagonal Hessian. I am not sure if they have released their codes, but as a paper that specifically focuses on optimizers, they should be involved in this paper. \n\nIt is good to see the authors define unlearning as erasing specific data under the condition in preserving the overall performance. Under this condition, comparing unlearning strength across optimization method is reasonable and fair to me. However, in practice, strict preservation is hard. The results in Figs 1-6 only report unlearning metrics, but I do not know how the retention performance is affected. So, the results are not convincing to me. A simple counterexample to show the results are not reliable is that, when we set an extreme large learning rate for GA, the model will erase all its knowledge. In this case, the model cannot recover any knowledge to be unlearned. However, in this case, we cannot say using large learning rate will make the model more robust. On the other side, it is hard to tune for the equal level of retention, so I suggest you to further report the results after UWC [1], which uses model mixing to align retention performance. \n\n[1] Unlearning with Control: Assessing Real-world Utility for Large Language Model Unlearning\n\nIn Fig 1, did the authors adopt the same learning rate across methods, or using the best hyper parameters across trials? Also, it is interesting to discuss what ensure a fair comparison across methods, e.g., the same learning rate or the same gradient magnitude. \n\nHow to ensure the conclusion “a downgraded optimizer can in fact lead to upgraded unlearning robustness”, drawn from NPO, is general across a wide range of methods?\n\nThe related works can benefit from more recent and milestone works, such as [2-8].\n\n[2] Rethinking Unlearning for Large Reasoning Models\n\n[3] BLUR: A Bi-Level Optimization Approach for LLM Unlearning\n\n[4] DRAGoN: Guard LLM Unlearning in Context via Negative Detection and Reasoning\n\n[5] GRU: Mitigating the Trade-off between Unlearning and Retention for Large Language Models\n\n[6] Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond\n\n[7] Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning\n\n[8] Rethinking machine unlearning for large language models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r98KuIGN6u", "reviewer": "Reviewer_psAf", "summary": "This paper investigates how optimizer design affects the robustness of LLM unlearning, introducing the concept of optimizer grade to compare high-order and downgraded optimizers. It finds that downgraded optimizers (e.g., signSGD, zeroth-order methods) lead models to flatter and more stable minima, improving post-unlearning robustness. A hybrid optimizer is further proposed, achieving superior resistance to relearning and quantization attacks across multiple benchmarks.", "questions": "See weaknesses section.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Introduces a novel perspective by analyzing optimizer grade as a key factor in unlearning robustness.\n\n2. Provides strong empirical evidence across diverse benchmarks and unlearning methods.\n\n3. Proposes a simple yet effective Hybrid FO–ZO optimizer with clear performance gains.", "weaknesses": "1. The paper explains the robustness gains mainly through intuition about flatter minima but lacks a deeper theoretical or quantitative analysis to support this claim.\n\n2. All experiments are limited to the unlearning setup, so it remains unclear whether the proposed idea generalizes to other fine-tuning or robustness tasks.\n\n3. The study does not provide details on the computational overhead of zeroth-order or hybrid optimization, leaving questions about practical scalability.\n\n4. The Hybrid FO–ZO optimizer design appears largely empirical; the choice of alternation schedule is not well-motivated or systematically analyzed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)