Venue: NAACL 2025

# Unlearning as multi-task optimization:

A normalized gradient difference approach with an adaptive learning rate

 Xiaomeng Jin

UIUC

xjin17@illinois.edu &Zhiqi Bu

Amazon AGI

zhiqibu@amazon.com &Bhanukiiran Vinzamuri

Amazon AGI

vinzambu@amazon.com &Anil Ramakrishna

Amazon AGI

aniramak@amazon.com &Kai-Wei Chang

Amazon AGI & UCLA

kwchang.cs@gmail.com &Volkan Cevher

Amazon AGI & LIONS EPFL

volkan.cevher@epfl.ch &Mingyi Hong\({}^{{\dagger}}\)

Amazon AGI & University of Minnesota

mhong@umn.edu

Equal contribution. Work done during Xiaomeng Jin's internship at Amazon. Corresponding author: zhiqibu@amazon.com Concurrent positions as an Amazon Scholar and as a faculty at the corresponding institutes. This paper represents the work performed at Amazon.



we study the rich literature of multi-task methods that seeks the Pareto optimality of two tasks (e.g., IMTL Liu et al. (2021), GradNorm Chen et al. (2018a), RLW Lin et al. (2021), PCGrad Yu et al. (2020), and scalarization Boyd and Vandenberghe (2004)), and design an approach specifically for the LLM unlearning problem.

Footnote 1: [https://github.com/tensorflow-deep-learning-for-the-Learning-of-the-MLM-unlearning-and-exploiting-inductive-learning-of-the-machine-learning-of-the-machine-learning-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-](https://github.com/tensorflow-deep-learning-for-the-Learning-of-the-MLM-unlearning-and-exploiting-inductive-learning-of-the-machine-learning-of-the-machine-learning-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-the-machine-to-learn-of-the-machine-to-learn-) 

optimizing two distinct learning objectives simultaneously. The key challenge is how to balance the trade-off among objectives during the optimization procedure by modifying the per-task gradients (e.g. PCGrad Yu et al. (2020), RLW Lin et al. (2021), IMTL Liu et al. (2021)). Several recent works have studied Pareto frontier and optimally in context of NLP tasks (e.g., multi-lingual machine translation Chen et al. (2023); Xin et al. (2022) and NLP fairness Han et al. (2023)). While optimization is often treated as a black-box tool in NLP research, studying optimization provides deeper insights and inspires new algorithms.4

Footnote 4: An example is the Baum-Welch algorithm, originally proposed to estimate the parameters of HMMs and applied in speech recognition in 1970s before it was recognized as an instance of the EM algorithm Dempster et al. (1977). It was later identified as a special case of a broader class of convex-concave optimization (CCCP) Yvile and Rangarajan (2001). This connection inspired new designs, such as the unified EM Samdani et al. (2012).

## 3 Unlearning as multi-task optimization

This section casts machine unlearning as a multi-task optimization (MTO), specifically the two-task optimization problem. Let the retain set be denoted by R and the forget set by F, with \(L_{\text{R}}\) and \(L_{\text{F}}\) representing the corresponding cross-entropy losses for language modeling. We are interested in finding

\[\arg\min_{\theta}L_{\text{R}}(\mathbf{\theta})\quad\cap\quad\arg\max_{\theta}L_{ \text{F}}(\mathbf{\theta}), \tag{1}\]

where \(\mathbf{\theta}\) represents the model parameters.

There might not be a solution that simultaneously achieves both objectives in Eq. (1). For LLMs, the unlearning solutions generally exhibit a trade-off between performance in R and F (_cf._, Figure 1). To forget F, one may unavoidably undern general knowledge such as grammar rules on F, which can sacrifice the performance on R.

In MTO, Pareto optimality is used to characterize the trade-offs between multiple objectives. In layperson's terms, if \(\theta\) is Pareto optimal, it is impossible to improve \(L_{\text{R}}\) or \(L_{\text{F}}\) without worsening the other. Formal definition is in below:

**Definition 1** (Pareto optimality in unlearning).: _For two models \(\mathbf{\theta}\) and \(\mathbf{\theta^{\prime}}\), if \(L_{\text{R}}(\mathbf{\theta})\geq L_{\text{R}}(\mathbf{\theta^{\prime}})\) and \(L_{\text{F}}(\mathbf{\theta})\leq L_{\text{F}}(\mathbf{\theta^{\prime}})\) with at least one inequality being strict, then \(\mathbf{\theta}\) is dominated by \(\mathbf{\theta^{\prime}}\). A model is Pareto optimal if it is not dominated by any other models._

In the remainder of this section, we will discuss current unlearning methods in a unified MTO framework and analyze their Pareto optimality. Building on this, we then propose a dynamic scalarization approach tailored to LLM unlearning.

### Static linear scalarization

A popular MTO method is scalarization, which addresses MTO by optimizing the linear scalarization problem (LSP). This method combines multiple tasks into a single, reweighted task:

\[\text{LSP}(\mathbf{\theta};c)=c\cdot L_{\text{R}}(\mathbf{\theta})-(1-c)\cdot L_{ \text{F}}(\mathbf{\theta}), \tag{2}\]

where \(c\) is fixed. At iteration \(t\), the gradient of LSP, \(\mathbf{g}_{\text{static}}(\mathbf{\theta}_{t};c)\), lies within the linear span of per-task gradients as shown in Figure 2 (yellow area):

\[\mathbf{g}_{\text{static}}(\mathbf{\theta}_{t};c)\!\!=\!\!\frac{\partial\text{LSP}}{ \partial\mathbf{\theta}_{t}}\!\!=\!\!cg_{\text{R}}(\mathbf{\theta}_{t})\!-\!(1\!\!-\! c)\mathbf{g}_{\text{F}}(\mathbf{\theta}_{t}). \tag{3}\]

Then, the corresponding update rule by the (stochastic) gradient method is

\[\mathbf{\theta}_{t+1}=\mathbf{\theta}_{t}-\eta_{t}[c\cdot\mathbf{g}_{\text{R}}(\mathbf{\theta} _{t})-(1-c)\cdot\mathbf{g}_{\text{F}}(\mathbf{\theta}_{t})].\]

**Remark 3.1**.: _We term the static linear scalarization as the **extended GDiff** in this work. Some existing methods are special cases of extended GDiff. For example, Gradient Descent (GD) on retaining set is equivalent to extended GDiff with \(c=1\). Gradient Ascent (GA) on forgetting is equivalent to that with \(c=0\), and vanilla GDiff Liu et al. (2022) set \(c=0.5\) (i.e., equally weighted)._

A nice property of linear scalarization is the Pareto optimality at the convergence of convex models Boyd and Vandenberghe (2004)), which we state in Lemma 2 (proof in Appendix G) for the static \(c\) and later extend to Theorem 3 for the dynamic \(c_{l}\) in Section 3.2.

**Lemma 2** (restated from Xin et al. (2022)).: _For any \(0<c<1\), the model \(\mathbf{\theta}_{\text{LSP}}^{*}(c)=\textit{argming}_{b}\text{LSP}(\mathbf{\theta};c)\) is Pareto optimal._

Figure 2: Gradient space in 2-dimension. \(\mathbf{g}_{\text{F}}\) is the forgetting gradient and \(\mathbf{g}_{\text{R}}\) is the retaining gradient, each with a perpendicular dashed line. Yellow area is the linear span (Eq. (3)) by scalarization. Green area is positively correlated to \(\mathbf{g}_{\text{R}}\) and negatively correlated to \(\mathbf{g}_{\text{F}}\) by Eq. (6), whereas NGDiff always stays within this green area at each iteration.

 

Lemma 2 suggests5 that we can sweep through \(c\in[0,1]\) and construct the Pareto frontier after sufficiently long training time (e.g., the blue dotted line in Figure 6 in Appendix A). However, while any \(c\) leads to a Pareto optimal point, the solution may be useless: e.g., perfect memorization on \(({\rm R},{\rm F})\) that fails to unlearn is also Pareto optimal. Next, we investigate different choices of c by extending the static scalarization in (3).

Footnote 5: We note that Lemma 2 is only applicable to the global minimum of LSP, which is not always achievable. While this result has its limitations and requires empirical validation, it provides guidance for algorithm design.

### Dynamic scalarization

Static scalarization uses a constant \(c\) in (3). However, we can extend it to use different scalars at different iteration:

\[\mathbf{\theta}_{t+1}=\mathbf{\theta}_{t}-\eta_{t}\mathbf{g}_{\text{UN}}(\mathbf{ \theta}_{t};c_{t}),~{}\text{where}\] \[\mathbf{g}_{\text{UN}}(\mathbf{\theta};c_{t}):=c_{t}\cdot\mathbf{g}_{\text{R }}(\mathbf{\theta})-(1-c_{t})\cdot\mathbf{g}_{\text{F}}(\mathbf{\theta}). \tag{4}\]

It is worth noting that instead of defining \(\mathbf{\theta}^{*}=\operatornamewithlimits{argmin}_{\theta}\text{LSP}\) at the loss level, we can define it at the gradient level based on the stationary condition of the training dynamics, i.e., \(\mathbf{g}_{\text{UN}}(\mathbf{\theta}^{*})=\mathbf{0}\).

Several unlearning and MTO methods can be viewed as special cases of Eq. (4):

1. Gradient descent (GD on R), \(c_{t}=1\)
2. Gradient ascent (GA on F), \(c_{t}=0\)
3. Gradient difference (vanilla GDiff), \(c_{t}=0.5\)
4. Loss normalization (LossNorm), \(\frac{c_{t}}{1-c_{t}}=\frac{L_{\text{F}}}{L_{\text{R}}}\)
5. RLW Lin et al. (2021), \(c_{t}=\frac{c+1}{e^{\lambda_{i}}+e^{\lambda_{j}}}\) with \(\lambda_{i}\sim N(0,1)\)
6. PCGrad Yu et al. (2020), \(\frac{c_{t}}{1-c_{t}}=1+\frac{\mathbf{g}_{\text{T}}^{\top}\mathbf{g}_{\text{R}}}{\|\bm {g}_{\text{U}}\|^{2}}\)
7. IMTL-G Liu et al. (2021), \(c_{t}=\mathbf{g}_{\text{T}}^{\top}(\frac{\mathbf{g}_{\text{T}}}{\|\mathbf{g}_{\text{F}}\|} -\frac{\mathbf{g}_{\text{R}}}{\|\mathbf{g}_{\text{R}}\|})/(\mathbf{g}_{\text{F}}-\mathbf{g}_{ \text{R}})^{\top}(\frac{\mathbf{g}_{\text{F}}}{\|\mathbf{g}_{\text{F}}\|}-\frac{\mathbf{ g}_{\text{R}}}{\|\mathbf{g}_{\text{R}}\|})\)

Despite the different designs of \(\{c_{t}\}\), we show in Theorem 3 (proof in Appendix G) that all \(\mathbf{\theta}^{*}(\{c_{t}\})\) are Pareto optimal following Lemma 2, including our NGDiff to be introduced in Section 4.2.

**Theorem 3**.: _For any \(\{c_{t}\}\) with \(0\leq c_{t}\leq 1\) that converges as \(t\to\infty\), the model \(\mathbf{\theta}^{*}(\{c_{t}\})=\lim_{t\to\infty}\mathbf{\theta}_{t}\) in (4) is Pareto optimal._

## 4 Unlearning with normalized gradient difference

While Theorem 3 shows the Pareto optimality of \(\mathbf{\theta}^{*}\) as \(t\to\infty\), it does not shed insight on the convergence through intermediate steps \(\mathbf{\theta}_{t}\). Put differently, although many MTO and unlearning methods are all Pareto optimal upon convergence, they may converge to different Pareto points at different convergence speeds. Therefore, it is important to understand and control the algorithm dynamics to maintain high performance for R throughout the training. Specifically, the dynamics are determined by the choices of \(\mathbf{g}_{\text{UN}}\in\mathbb{R}^{d}\) and \(\eta_{t}\in\mathbb{R}\) in Eq. (4).

In this section, we propose to use gradient normalization for \(\mathbf{g}_{\text{UN}}\) and automatic learning rate for \(\eta_{t}\), so as to achieve stable convergence, effective unlearning, high retaining utility, without manually tuning the learning rate.

### Loss landscape of unlearning

Applying the Taylor expansion on Eq. (4), we can view the local landscapes of loss \(L_{\text{R}}\) and \(L_{\text{F}}\) as quadratic functions, where \(L_{\omega}(\mathbf{\theta}_{t+1})-L_{\omega}(\mathbf{\theta}_{t})=-\eta_{t}\mathbf{g}_{ \omega}^{\top}\mathbf{g}_{\text{UN}}(c_{t})+(\eta_{t}^{2}/2)\mathbf{g}_{\text{UN}}^{ \top}\mathbf{H}_{\omega}\mathbf{g}_{\text{UN}}+o(\eta_{t}^{2}),\) (5)

where \(\omega\) is either R or F.

Here \(\mathbf{H}_{\omega}=\frac{\partial^{2}I_{\text{FN}}}{\partial\mathbf{\theta}^{2}}\) is the Hessian matrix, which _empirically_ gives \(\mathbf{g}_{\text{UN}}^{\top}\mathbf{H}_{\omega}\mathbf{g}_{\text{UN}}>0\) and renders \(L_{\text{R}}\) and \(L_{\text{F}}\) locally and directionally convex along the gradients. This allows the existence of a minimizing learning rate to be characterized in Section 4.3. We visualize the loss landscape of Phi-1.5 Li et al. (2023a) model on an unlearning benchmark, TOFU dataset Maini et al. (2024) in Figure 3 and observe that the quadratic functions in Eq. (5) are well-fitted in most iterations.

### Normalized gradient difference

In order for \(L_{\text{F}}\) to increase as well as \(L_{\text{R}}\) to decrease, we want to construct \(\mathbf{g}_{\text{UN}}\) such that

\[\mathbf{g}_{\text{R}}^{\top}\mathbf{g}_{\text{UN}}(c_{t})\geq 0\geq\mathbf{g}_{\text{F}}^{ \top}\mathbf{g}_{\text{UN}}(c_{t}). \tag{6}\]

To satisfy Eq. (6) , we propose a normalized gradient difference method (NGDiff) to dynamically

Figure 3: Loss values of retaining and forgetting sets with different learning rates. Markers are \(L_{\text{R}}(\mathbf{\theta}_{t}-\eta\mathbf{g}_{\text{R}})\) and \(L_{\text{F}}(\mathbf{\theta}_{t}-\eta\mathbf{g}_{\text{F}})\) estimated by Phi-1.5 on _TOFU_ at step 10. The curves are fitted as quadratic functions.

 

set \(c_{t}=\frac{1/||\mathbf{g}_{\mathbb{R}}||}{||\mathbf{g}_{\mathbb{R}}|+1/||\mathbf{g}_{ \mathbb{F}}||}\Longrightarrow\mathbf{g}_{\text{NGDiff}}(\mathbf{g}_{\mathbb{R}},\mathbf{ g}_{\mathbb{F}}):=\frac{\mathbf{g}_{\mathbb{R}}}{\|\mathbf{g}_{\mathbb{R}}\|}-\frac{\mathbf{g}_{ \mathbb{F}}}{\|\mathbf{g}_{\mathbb{F}}\|}\). In words, we normalize the retaining and forgetting gradients6

Footnote 6: We illustrate in Appendix B that NGDiff is critically different and simpler than GradNorm.(Chen et al., 2018a).

We analyze NGDiff as follows. First, we show the condition in Eq. (6) is satisfied at all iterations in the following lemma (proof in Appendix G):

**Lemma 4**.: \(\mathbf{g}_{\mathrm{NGDiff}}(\mathbf{g}_{\mathbb{R}},\mathbf{g}_{\mathbb{F}})\) _satisfies Eq. (6) for any \(\mathbf{g}_{\mathbb{R}}\in\mathbb{R}^{d}\) and \(\mathbf{g}_{\mathbb{F}}\in\mathbb{R}^{d}\)._

In Theorem 5 (proof in Appendix G), we leverage Lemma 4 to claim that the local loss improvement under appropriate learning rate, which will be implemented adaptively in Section 4.3.

**Theorem 5**.: _Consider \(\mathbf{\theta}_{t+1}=\mathbf{\theta}_{t}-\eta\mathbf{g}_{\mathrm{NGDiff}}\)._

_(1) Unless \(\mathbf{g}_{\mathbb{R}}\) is exactly parallel to \(\mathbf{g}_{\mathbb{F}}\), for any sufficiently small learning rate \(\eta\), there exist two constants \(\epsilon_{\mathrm{R},1}=o(\eta),\epsilon_{\mathrm{F},1}=o(\eta)\) such that_

\[L_{\mathrm{R}}(\mathbf{\theta}_{t+1})-L_{\mathrm{R}}(\mathbf{\theta}_{ t})<\epsilon_{\mathrm{R},1};\] \[L_{\mathrm{F}}(\mathbf{\theta}_{t+1})-L_{\mathrm{F}}(\mathbf{\theta}_{t})> \epsilon_{\mathrm{F},1}.\]

_(2) If additionally \(\mathbf{g}_{\mathrm{NGDiff}}^{\top}\mathbf{H}_{\mathbb{R}}\mathbf{g}_{\mathrm{NGDiff}}>0\) and \(\mathbf{g}_{\mathrm{NGDiff}}^{\top}\mathbf{H}\mathbf{g}_{\mathrm{NGDiff}}>0\), then for any learning rate \(0<\eta<\frac{2\mathbf{g}_{\mathbb{R}}^{*}\mathbf{g}_{\mathrm{NGDiff}}}{\mathbf{g}_{\mathrm{ NGDiff}}^{\top}\mathbf{H}_{\mathbb{R}}\mathbf{g}_{\mathrm{NGDiff}}}\), there exist two constants \(\epsilon_{\mathrm{R},2}=o(\eta^{2}),\epsilon_{\mathrm{F},2}=o(\eta^{2})\) such that_

To interpret Theorem 5, we view \(\epsilon\approx 0\) as \(\eta\) is generally small (say \(\eta\sim 10^{-4}\) in our experiments), and hence, NGDiff is optimizing on R and F simultaneously. Visually speaking, Lemma 4 constrains NGDiff's gradient to stay in the green area in Figure 2 unless \(\mathbf{g}_{\mathbb{F}}\parallel\mathbf{g}_{\mathbb{R}}\), whereas other methods do not explicitly enforce Eq. (6) and may consequently harm the retaining utility.

```
1:for\(t=1,2,...\)do
2:---NGDiff----
3: Compute \(L_{\mathrm{R}}(\mathbf{\theta}_{t})\) by a forward pass on R
4: Compute \(\mathbf{g}_{\mathbb{R}}(\mathbf{\theta}_{t})\) by backward propagation
5: Compute \(L_{\mathrm{F}}(\mathbf{\theta}_{t})\) by a forward pass on F
6: Compute \(\mathbf{g}_{\mathrm{F}}(\mathbf{\theta}_{t})\) by backward propagation
7: Construct \(\mathbf{g}_{\mathrm{NGDiff}}=\mathbf{g}_{\mathbb{R}}/\|\mathbf{g}_{\mathbb{R}}\|-\mathbf{g}_{ \mathbb{F}}/\|\mathbf{g}_{\mathbb{F}}\|\)
8:--AutoLR---
9if\(t\bmod 10==0\):then
10: Compute \(L_{R}^{\pm}=L_{\mathrm{R}}(\mathbf{\theta}_{t}\pm\eta\mathbf{g})\) by two forward passes on R
11: Fit the quadratic function in Eq. (5) from \((-\eta,0,\eta)\rightarrow(L_{\mathrm{R}}^{-},L_{\mathrm{R}},L_{\mathrm{R}}^{+})\)
12: Derive the optimal learning rate \(\eta_{t}^{*}\) by Eq. (7) and set \(\eta=\eta_{t}^{*}\)
13: Update \(\theta_{t+1}=\theta_{t}-\eta\mathbf{g}_{\mathrm{NGDiff}}\)
```

**Algorithm 1** Learning-rate-free NGDiff 

Experiments

### Setup

**Dataset** We evaluate the empirical performance of our proposed method on the two following datasets (see more dataset details in Section F.1):

_Task of Fictitious Unlearning (TOFU)_[25]. _TOFU_ consists of 20 question-answer pairs based on fictitious author biographies generated by GPT-4 [3]. In our experiments, we use the _forget10_ (\(10\%\) of the full training set) as the forgetting set and _retain90_ (\(90\%\) of the full training set) as the retaining set.

_MUSE-NEWS_[26]. This dataset consists of BBC news articles [8] published since August 2023. We use its _train_ split to finetune a target model, and then the _raw_ set, which includes both the forgetting and retaining data, for the target model unlearning. Finally, the _verbmem_ and _knowmem_ splits are used to evaluate the unlearned model's performance.

**Unlearning methods** We compare NGDiff with 4 baselines. The first baseline method is the target model without any unlearning, while the remaining three are the state-of-the-art unlearning methods.

_No-unlearn_. We fine-tune the base model on the full training data. Subsequent unlearning approaches are then applied on _No-unlearn_.

_Gradient Difference (GDiff)_[7]. GDiff (see Sec. 3.2) applies static linear scalarization with \(c=0.5\) in MTO. For a thorough comparison, we also include the extended GDiff method, with \(c=0.1\) or \(c=0.9\).

_Loss Normalization (LossNorm)_. As discussed in Section 3.2, this approach computes and normalizes the forget loss and retain loss separately, with the overall loss being \(L_{\text{R}}/|L_{\text{R}}|-L_{\text{F}}/|L_{\text{F}}|\).

_Negative Preference Optimization (NPO)_[11]. NPO is adapted from Direct Preference Optimization (DPO) [26] and uses preference optimization [12] with the loss: \(L_{\text{NPO},\beta}(\mathbf{\theta})=\)

\[-\frac{2}{\beta}\mathbb{E}_{\text{F}}\Big{[}\log\sigma\Big{(}-\beta\log\frac{ f(S,w)}{f_{\text{No-unlearn}}(S,w)}\Big{)}\Big{]}, \tag{8}\]

where \(S\) is randomly sampled from F, \(\beta>0\) is the inverse temperature, \(f\) is the unlearned model, and \(f_{\text{No-unlearn}}\) is the model before unlearning.

Foundation ModelsWe test multiple LLMs: LLaMA2-7B [12], Phi-1.5 [8], Falcon-1B [13], GPT2-XL [14] and Mistral-7B [15]. They are pre-trained and then fine-tuned on datasets in Section 5.1, with AdamW optimizer and are carefully tuned (Appendix F).

### Evaluation Metrics

Following the existing work [26], we evaluate the unlearning performance based on model's output quality. We expect a good performance should satisfy the following requirements:

**No verbatim memorization**. After the unlearning, the model should no longer remember any verbatim copies of the texts in the forgetting data. To evaluate this, we prompt the model with the first \(k\) tokens in F and compare the model's continuation outputs with the ground truth continuations. We use ROUGE-L recall scores for this comparison, where a lower score is better for unlearning.

**No knowledge memorization**. After the unlearning, the model should not only forget verbatim texts, but also the knowledge in the forgetting set. For the _MUSE-NEWS_ dataset, we evaluate knowledge memorization using the _Knowmem_F split, which consists of generated question-answer pairs based on the forgetting data. Similar to verbatim memorization, we use ROUGE-L recall scores.

**Maintained model utility**. An effective unlearning method must maintain the model's performance on the retaining set. We prompt the model with the question from R and compare the generated answer to the ground truth. We use ROUGE-L recall scores for these comparisons. Additionally, we evaluate the model using the Truth Ratio metric. We use the _Retain10-perturbed_ split from _TOFU_, which consists of five perturbed answers created by modifying the facts in each original answer from R. The Truth Ratio metric computes how likely the model generates a correct answer versus an incorrect one, where a higher value is better.

### Main Results

The results for _Verbatim memorization (Verbmem)_, _Model utility (Utility)_, _TruthRatio_, and _Knowledge memorization (Knownem)_ using different unlearning methods are presented in Table 1, 2 as well as \(6\) in Appendix. We evaluate these metrics using _TOFU_ and _MUSE-NEWS_ across LLMs.

In summary, our NGDiff consistently achieves the superior performance across all models on both datasets. In stark contrast, the baseline unlearning methods (1) either effectively forget R by reducing _Verbmem_ and _Knowmem_ but fail maintain 

the _Utility_ and _TruthRatio_, such as GDiff with \(c\leq 0.5\), NPO; (2) or cannot unlearn F on Phi-1.5 and Mistral-7B, such as LossNorm and GDiff with \(c=0.9\). We highlight that the effectiveness of these unlearning methods are highly model-dependent and dataset-dependent, unlike NGDiff.

For the _TOFU_ dataset, we observe that some unearning methods fail to unlearn the forget data effectively. For example, GDiff-0.9 and LossNorm do not unlearn effectively when applied to Phi-1.5, Llama2-7B and Mistral-7B. In fact, GDiff-0.9 has \(80\%\sim 100\%\)_Verbmem_ and LossNorm has \(>40\%\)_Verbmem_ on Phi-1.5. However, they are effective on Falcon-1B and GPT2-XL, even though these models have similar sizes (\(\approx 1\)B parameters) to Phi-1.5. On the other hand, some methods fail to preserve the model utility after unlearning. For example, GDiff-0.1 has close to 0 _Utility_ on Phi-1.5, Falcon-1B, GPT2-XL and Llama2-7B; similarly, NPO also experiences a significant drop in _Utility_ on Phi-1.5 model, Falcon-1B and GPT2-XL, but not so on Llama2-7B. In contrast, our NGDiff remains effective in unlearning F and maintaining R across the models. In addition, NGDiff achieves the best _TruthRatio_ on all models except Llama2-7B (which is still on par with the best), indicating

\begin{table}
\begin{tabular}{l|p{36.1pt}|r|r r r r r r} \hline \hline \multirow{2}{*}{**Base Model**} & \multicolumn{1}{c|}{**Metric**} & \multicolumn{6}{c}{**Method**} \\  & & _No-unlearn_ & _GDiff-0.9_ & _GDiff-0.5_ & _GDif-0.1_ & _NPO_ & _LossNorm_ & _NGDiff_ \\ \hline \multirow{3}{*}{Phi-1.5} & Verbemm \(\downarrow\) & \(1.000\) & \(0.805\) & **0.027** & **0.000** & **0.000** & \(0.432\) & **0.024** \\  & Utility \(\uparrow\) & \(1.000\) & \(0.992\) & \(0.308\) & \(0.000\) & \(0.000\) & **0.752** & **0.747** \\  & TruthRatio \(\uparrow\) & \(0.385\) & \(0.205\) & \(0.216\) & \(0.221\) & \(0.179\) & \(0.214\) & **0.353** \\ \hline \multirow{3}{*}{Falcon-1B} & Verbemm \(\downarrow\) & \(1.000\) & **0.041** & **0.001** & **0.000** & **0.017** & **0.055** & **0.021** \\  & Utility \(\uparrow\) & \(1.000\) & **0.434** & \(0.305\) & \(0.000\) & \(0.114\) & \(0.521\) & \(0.428\) \\  & TruthRatio \(\uparrow\) & \(0.408\) & \(0.237\) & \(0.244\) & \(0.217\) & \(0.184\) & \(0.252\) & **0.354** \\ \hline \multirow{3}{*}{GPT2-XL} & Verbemm \(\downarrow\) & \(1.000\) & **0.029** & **0.001** & **0.000** & **0.031** & **0.022** & **0.046** \\  & Utility \(\uparrow\) & \(0.999\) & \(0.381\) & \(0.250\) & \(0.000\) & \(0.136\) & \(0.376\) & **0.792** \\  & TruthRatio \(\uparrow\) & \(0.412\) & \(0.186\) & \(0.278\) & \(0.133\) & \(0.179\) & \(0.196\) & **0.399** \\ \hline \multirow{3}{*}{Llama2-7B} & Verbemm \(\downarrow\) & \(1.000\) & \(0.810\) & **0.011** & **0.000** & \(0.709\) & **0.010** & **0.002** \\  & Utility \(\uparrow\) & \(1.000\) & **0.851** & \(0.324\) & \(0.000\) & \(0.682\) & \(0.264\) & **0.724** \\  & TruthRatio \(\uparrow\) & \(0.490\) & \(0.340\) & **0.364** & \(0.161\) & \(0.329\) & \(0.329\) & \(0.334\) \\ \hline \multirow{3}{*}{Mistral-7B} & Verbemm \(\downarrow\) & \(1.000\) & \(1.000\) & \(0.945\) & \(0.410\) & \(0.385\) & \(0.259\) & **0.009** \\  & Utility \(\uparrow\) & \(1.000\) & **0.999** & **0.944** & \(0.517\) & \(0.341\) & **0.925** & **0.996** \\ \cline{1-1}  & TruthRatio \(\uparrow\) & \(0.344\) & **0.345** & **0.366** & **0.374** & **0.364** & **0.358** & **0.379** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance on _TOFU_ dataset _(_forget10/retain90_)_ with different unlearning methods and models. We define success as the model being able to reduce _Verbatim memorization_ to below 0.1 or maintain at least 70% of the _Model utility_ and the _TruthRatio_ compared to the _No-unlearn_, with successful cases highlighted in bold. NGDiff achieves success in most cases.

\begin{table}
\begin{tabular}{l|c|c c c c c c c} \hline \hline \multirow{2}{*}{**Base Model**} & \multicolumn{1}{c|}{**Metric**} & \multicolumn{6}{c}{**Method**} \\  & & _No-unlearn_ & _GDiff-0.9_ & _GDiff-0.5_ & _GDiff-0.1_ & _NPO_ & _LossNorm_ & _NGDiff_ \\ \hline \multirow{3}{*}{Llama2-7B} & Verbemm \(\downarrow\) & \(0.561\) & \(0.555\) & **0.043** & **0.004** & **0.000** & \(0.388\) & **0.036** \\  & Knowmem \(\downarrow\) & \(0.755\) & \(0.717\) & **0.287** & **0.000** & **0.000** & **0.514** & **0.455** \\  & Utility \(\uparrow\) & \(0.646\) & **0.641** & \(0.275\) & \(0.000\) & \(0.000\) & **0.506** & **0.556** \\ \hline \multirow{3}{*}{Mistral-7B} & Verbemm \(\downarrow\) & \(0.578\) & \(0.177\) & **0.000** & **0.000** & \(0.113\) & \(0.196\) & **0.098** \\  & Knowmem \(\downarrow\) & \(0.416\) & **0.257** & **0.000** & **0.000** & \(0.343\) & \(0.293\) & **0.165** \\ \cline{1-1}  & Utility \(\uparrow\) & \(0.411\) & **0.339** & \(0.000\) & \(0.000\) & **0.316** & **0.343** & **0.354** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Results on the _MUSE-NEWS_ dataset. We boldface the entries where unlearning successfully reduces _Verbatim memorization_ to below 0.1, reduces _Knowledge memorization_ to less than 70% of _No-Unlearn_, or maintains at least 70% of the Utility compared to _No-Unlearn_. With the exception of NGDiff, most unlearning approaches exhibit a significant trade-off between forgetting and utility.

\begin{table}
\begin{tabular}{l|c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{6}{c}{**TOFU** (without \(\rightarrow\) with AutoLR)} \\  & & _Verbmem_\(\downarrow\) & _Utility_\(\uparrow\) & _TruthRatio_\(\uparrow\) \\ \hline
**No-unelearn** & \(1.00\) & \(1.00\) & \(0.39\) \\
**GDiff c=0.9** & \(0.81\to 0.20\) & \(0.99\to 0.42\) & \(0.21\to 0.31\) \\
**GDiff c=0.5** & \(0.03\to 0.00\) & \(0.31\to 0.03\) & \(0.22\to 0.30\) \\
**GDiff c=0.1** & \(0.00\to 0.00\) & \(0.00\to 0.00\) & \(0.22\to 0.23\) \\
**NPO** & \(0.00\to 0.00\) 

that the model's answers remain factually accurate for questions in the retaining data.

For the _MUSE-NEWS_ dataset, NGDiff also outperforms the baseline methods on Llama2-7B and Mistral-7B models by achieving a lower _Verbmem_ and a higher _Utility_. The _Knowmhem_ results indicate that NGDiff not only unlearns the verbatim copies of the forgetting texts, but also successfully removes the associated knowledge. While the model capacities of Phi-1.5 and Falcon-1B are smaller, limiting their ability to learn knowledge effectively after fine-tuning on the full dataset, as shown in Table6, NGDiff still performs well.

To further illustrate the performance of our proposed method during the training, in addition to the last iterate results, we plot the ROUGE scores and loss terms during the unlearning process in Figure4. We apply the extended GDiff, LossNorm, and NGDiff methods, to the Phi-1.5 model using the _TOFU_ dataset. While GDiff with \(c=0.5\) and \(c=0.7\), and NGDiff are effective in unlearning, only NGDiff preserve the model utility above \(75\%\) ROUGE score. A closer look at the second and the fourth plots of Figure4 shows that NGDiff exhibits the fastest and most stable convergence on F while maintaining a low retaining loss \(\leq 0.1\).

### Ablation Study

**Effectiveness of NGDiff**. In our experiments, we utilize the automatic learning rate scheduler (AutoLR) for NGDiff method. To investigate the impact of NGDiff alone, we compare all methods with or without AutoLR in Table3. With AutoLR or not (where we use manually tuned learning rates), NGDiff, GDiff (\(c=0.1\) or \(0.5\)) and NPO can effectively unlearn in terms of _Verbmem_. However, among these four methods, NGDiff uniquely retains a reasonable _Utility_ between \(60\sim 75\%\), while other methods retains only \(0\sim 30\%\)_Utility_. A similar pattern is observed in terms of _TruthRatio_ as well. Overall, NGDiff significantly outperforms other baseline methods with or without AutoLR.

**Impact of automatic learning rate**. To evaluate the impact of AutoLR scheduler, we see in Table3 all methods exhibit an increase in the _TruthRatio_ metric and a decrease in _Verbmem_, though with some loss in the _Utility_. For instance, _LossNorm_ benefits significantly from AutoLR with \(\approx 20\%\) decrease in _Verbmem_, and NGDiff increases its retaining _Utility_ and _TruthRatio_ by \(>22\%\). We specifically demonstrate the impact of AutoLR on NGDiff in Figure5. Without AutoLR, the model's performance is highly sensitive to the static learning rates: when \(\eta=10^{-5}\), the model fails to unlearn F as indicated by the low loss and high ROUGE score; in contrast, when \(\eta=10^{-4}\), there is a significant

Figure 4: Comparison of unlearning methods on TOFU. The figures show the ROUGE scores and loss terms during unlearning process with different methods, which includes GDiff, LossNorm, and NGDiff. We observe that NGDiff effectively unlearns the forgetting data while maintaining the performance on the retaining data.

Figure 5: Comparison between AutoLR and different learning rates on NGDiff. The figures show the ROUGE scores and loss values during the unlearning process on TOFU dataset using Phi-1.5 model. We observe that AutoLR outperforms the static learning rates with better model utility and more stable convergence.

 

drop in ROUGE score on the retain data, falling from 100% to around 50%. However, with the AutoLR scheduler, we observe a steady reduction in the _Verbmem_ (with the ROUGE forget close to 0 at convergence) while maintaining high utility (the ROUGE retain is 0.747, which is 19.5% higher than the best results without AutoLR).

## 6 Conclusion and Discussion

We formulated the machine unlearning problem as a two-task optimization problem and proposed a novel unlearning method NGDiff based on normalized gradient difference and automatic learning rate adaption. By leveraging insights from multi-task optimization, NGDiff empirically improves forgetting quality while maintaining utility. We hope this paper helps establish a connection between LLM unlearning and multi-task optimization, and inspires further advancements in this field.

### Limitations

Like other machine learning approaches in NLP, while our goal is to remove the influence of specific documents from LLMs, complete removal cannot always be guaranteed. Therefore, caution should be exercised when applying the proposed unlearning techniques in practical applications as unlearned LLMs can still potentially generate harmful or undesired outputs.

There are several technical alternatives that we did not explore in this paper due to its scope and limited resources. For example, other learning-rate-free methods could potentially be adapted as alternatives to the GeN approach used in this work. Additionally, other multi-task optimization methods could be applied to machine unlearning. However, scaling these approaches to the level of LLMs could be challenging, and are left as future work.

Finally, we mainly examined NGDiff's effectiveness on LLM unlearning in this paper with two benchmark datasets, TOFU and MUSE. To show its generalizability, we provide an additional example to apply the algorithm to computer vision tasks (see Appendix D). However, it would be desirable to test NGDiff on other modalities beyond NLP and CV applications.

## Ethical Consideration

We acknowledge that our work is aligned with the _ACL Code of the Ethics_8 and will not raise ethical concerns. We do not use sensitive datasets/models that may cause any potential issues/risks.

Footnote 8: [https://www.aclweb.org/portal/content/acl-code-ethics](https://www.aclweb.org/portal/content/acl-code-ethics)

#### Acknowledgments

We would like to thank Shankar Ananthakrishnan, Fabian Triefenbach and Jianhua Lu from Amazon AGI Foundations team for providing feedback on this paper and supporting this work.

## References

* Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamad Anaddat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. Cited by: SS1.
* L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot (2021)Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 141-159. External Links: Document Cited by: SS1.
* S. Boyd and L. Vandenberghe (2004)Convex optimization. Cambridge university press. Cited by: SS1.
* Z. Bu and S. Xu (2024)Automatic gradient descent with generalized newton's method. arXiv preprint arXiv:2407.02772. Cited by: SS1.
* N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, et al. (2021)Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633-2650. Cited by: SS1.
* J. Chen and D. Yang (2023)Unlearn what you want to forget: efficient unlearning for LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, pp. 12041-12052. External Links: Link, Document Cited by: SS1.
* K. Chen, Z. Wang, B. Mi, W. Liu, S. Wang, X. Ren, and J. Shen (2024)Machine unlearning in large language models. arXiv preprint arXiv:2404.16841. Cited by: SS1.
* L. Chen, S. Ma, D. Zhang, F. Wei, and B. Chang (2023)On the pareto front of multilingual neural machine translation. ArXivabs/2304.03216. Cited by: SS1.
* S. Chen, Y. Zhang, and Q. Yang (2021)Multi-task learning in natural language processing: an overview. ACM Computing Surveys. Cited by: SS1.

 

Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. 2018a. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In _International conference on machine learning_, pages 794-803. PMLR.
* Chen et al. (2018b) Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. 2018b. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. _Preprint_, arXiv:1711.02257.
* Chen et al. (2020) Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. 2020. Just pick a sign: Optimizing deep multitask models with gradient sign dropout. _Preprint_, arXiv:2010.06808.
* De Cao et al. (2021) Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 6491-6506, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Defazio and Mishchenko (2023) Aaron Defazio and Konstantin Mishchenko. 2023. Learning-rate-free learning by d-adaptation. _Preprint_, arXiv:2301.07733.
* Dempster et al. (1977) A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. _Journal of the Royal Statistical Society: Series B_, 39:1-38.
* Desideri (2012) Jean-Antoine Desideri. 2012. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. _Comptes Rendus Mathematique_, 350(5-6):313-318.
* Eldan and Russinovich (2023) Ronen Eldan and Mark Russinovich. 2023. Who's harry potter? approximate unlearning in llms.
* Gupta et al. (2021) Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, and Chris Waites. 2021. Adaptive machine unlearning. _Advances in Neural Information Processing Systems_, 34:16319-16330.
* Han et al. (2023) Xudong Han, Timothy Baldwin, and Trevor Cohn. 2023. Fair enough: Standardizing evaluation and model selection for fairness research in NLP. In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_, pages 297-312, Dubrovnik, Croatia. Association for Computational Linguistics.
* He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep residual learning for image recognition. _Preprint_, arXiv:1512.03385.
* Hu et al. (2021) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. _Preprint_, arXiv:2106.09685.
* Iygi et al. (2023) Maor Iygi, Oliver Hinder, and Yair Carmon. 2023. Dog is sgd's best friend: A parameter-free dynamic step size schedule. _Preprint_, arXiv:2302.12022.
* Jang et al. (2023) Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajaungen Logeswaran, and Minjoon Seo. 2023. Knowledge unlearning for mitigating privacy risks in language models. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 14389-14408, Toronto, Canada. Association for Computational Linguistics.
* Javaloy and Valera (2022) Adrian Javaloy and Isabel Valera. 2022. Rotograd: Gradient homogenization in multitask learning. _Preprint_, arXiv:2103.02631.
* Jiang et al. (2023) Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. _arXiv preprint arXiv:2310.06825_.
* Khaled et al. (2024) Ahmed Khaled, Konstantin Mishchenko, and Chi Jin. 2024. Dowg unleashed: An efficient universal parameter-free gradient descent method. _Preprint_, arXiv:2305.16284.
* Krizhevsky et al. (2009) Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images.
* Li et al. (2024) Yige Li, Hankun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. 2024. Backdoorllm: A comprehensive benchmark for backdoor attacks on large language models. _Preprint_, arXiv:2408.12798.
* Li et al. (2023a) Yuanzhi Li, Sebastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023a. Textbooks are all you need ii: phi-1.5 technical report. _arXiv preprint arXiv:2309.05463_.
* Li et al. (2023b) Yucheng Li, Frank Geurin, and Chenghua Lin. 2023b. Avoiding data contamination in language model evaluation: Dynamic test construction with latest materials. _arXiv preprint arXiv:2312.12343_.
* Lin et al. (2021) Baiijiong Lin, Feiyang Ye, Yu Zhang, and Ivor Tsang. 2021. Reasonable effectiveness of random weighting: A litmus test for multi-task learning. _Transactions on Machine Learning Research_.
* Liu et al. (2022) Bo Liu, Qiang Liu, and Peter Stone. 2022. Continual learning and private unlearning. In _Conference on Lifelong Learning Agents_, pages 243-254. PMLR.
* Liu et al. (2024a) Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. 2024a. Conflict-averse gradient descent for multi-task learning. _Preprint_, arXiv:2110.14048.
* Liu et al. (2021) Liyang Liu, Yi Li, Zhanghui Kuang, J Xue, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang. 2021. Towards impartial multi-task learning. In _iclr_.
* Liu et al. (2024b) Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Xiaojun Xu, Yuequang Yao, Hang Li, Kush R Varshney, et al. 2024b. Rethinking machine unlearning for large language models. _arXiv preprint arXiv:2402.08787_ 

Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varshney, Mohit Bansal, Sammi Koyejo, and Yang Liu. 2024c: Rethinking machine unlearning for large language models. _Preprint_, arXiv:2402.08787.
* Maini et al. (2024) Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. 2024. Tofu: A task of fictitious unlearning for llms. _arXiv preprint arXiv:2401.06121_.
* Mishchenko and Defazio (2024) Konstantin Mishchenko and Aaron Defazio. 2024. Prodigy: An expeditiously adaptive parameter-free learner. _Preprint_, arXiv:2306.06101.
* Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. _Preprint_, arXiv:2203.02155.
* Pawelczyk et al. (2024) Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. 2024. In-context unlearning: Language models as few shot unlearners. In _ICML_.
* Penedo et al. (2023) Guilherme Penedo, Quentin Malaric, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobebidi, Baptiste Pannier, Ebesam Almzrouei, and Julien Launay. 2023. The refinedweb dataset for falcon llm: Outperforming curated corpora with web data, and web data only. _Preprint_, arXiv:2306.01116.
* Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. _OpenAI blog_, 1(8):9.
* Rafailov et al. (2024) Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. _Preprint_, arXiv:2305.18290.
* Samdani et al. (2012) Rajhans Samdani, Ming-Wei Chang, and Dan Roth. 2012. Unified expectation maximization. In _Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 688-698, Montreal, Canada. Association for Computational Linguistics.
* Shi et al. (2024) Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A Smith, and Chiyuan Zhang. 2024. Muse: Machine unlearning six-way evaluation for language models. _arXiv preprint arXiv:2407.06460_.
* Si et al. (2023) Nianwen Si, Hao Zhang, Heyu Chang, Wenlin Zhang, Dan Qu, and Weiqiang Zhang. 2023. Knowledge unlearning for llms: Tasks, methods, and challenges. _Preprint_, arXiv:2311.15766.
* The New York Times (2023) The New York Times. 2023. Exhibit j.
* Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. _arXiv preprint arXiv:2307.09288_.
* Ullah et al. (2021) Enayat Ullah, Tung Mai, Anup Rao, Ryan A Rossi, and Raman Arora. 2021. Machine unlearning via algorithmic stability. In _Conference on Learning Theory_, pages 4126-4142. PMLR.
* Wallace et al. (2021) Eric Wallace, Tony Zhao, Shi Feng, and Sameer Singh. 2021. Concealed data poisoning attacks on NLP models. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 139-150, Online. Association for Computational Linguistics.
* Wang et al. (2020) Ziriui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. 2020. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. _Preprint_, arXiv:2010.05874.
* Wu et al. (2024) Xiaobao Wu, Liangming Pan, William Yang Wang, and Anh Huan Luu. 2024. Akew: Assessing knowledge editing in the wild. _Preprint_, arXiv:2402.18909.
* Xin et al. (2022) Derrick Xin, Behrooz Ghorbani, Justin Gilmer, Ankush Garg, and Orhan Firat. 2022. Do current multi-task optimization methods in deep learning even help? _Advances in neural information processing systems_, 35:13597-13609.
* Yu et al. (2020) Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. 2020. Gradient surgery for multi-task learning. _Advances in Neural Information Processing Systems_, 33:5824-5836.
* Yuille and Rangarajan (2001) Alan L Yuille and Anand Rangarajan. 2001. The concave-convex procedure (cccp). In _Advances in Neural Information Processing Systems_, volume 14. MIT Press.
* Zhang et al. (2024a) Dawen Zhang, Pamela Finckenberg-Broman, Thong Hoang, Shidong Pan, Zhenchang Xing, Mark Staples, and Xiwei Xu. 2024a. Right to be forgotten in the era of large language models: Implications, challenges, and solutions. _Preprint_, arXiv:2307.03941.
* Zhang et al. (2024b) Ruizqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024b. Negative preference optimization: From catastrophic collapse to effective unlearning. _arXiv preprint arXiv:2404.05868_.
* Zhang et al. (2023) Zhihan Zhang, Wenhao Yu, Mengxia Yu, Zhichun Guo, and Meng Jiang. 2023. A survey of multi-task learning in natural language processing: Regarding task relatedness and training methods. In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_, pages 943-956, Dubrovnik, Croatia. Association for Computational Linguistics 



## Appendix A Preliminary evidence

In our preliminary experiments, we observe two key issues preventing the standard methods from being practically applied. First, balancing retaining and forgetting losses is difficult. In Figure 6, we observe a trade-off between the performance on R and F, where some methods fail to unlearn F (points in the upper-right corner of the left figure), and some do not maintain utility in R (points in the bottom-left corner of the left figure). The blue dotted line in Figure 6 further illustrates the trade-off in GDiff by sweeping a hyper-parameter \(c\in[0,1]\), which is used to balance the losses on the forgetting and retaining data (see Eq. (3)). Picking an appropriate \(c\) to balance the two terms is often challenging. Secondly, the optimization methods for unlearning are usually sensitive to the learning rate. As illustrated in Figure 7, even for the same algorithm, various learning rates lead to substantial changes in the ROUGE scores and loss values, making the unlearning methods unstable and difficult to use in practice.

## Appendix B Comparing NGDiff with GradNorm

```
1:Initialize the scalaring coefficients \(w_{\text{R}}(\mathbf{\theta}_{0})=1\) and \(w_{\text{F}}(\mathbf{\theta}_{0})=1\)
2:Pick value for \(\alpha>0\) and pick the weights \(\mathbf{\theta}_{\text{LS}}\) (the last shared layer of \(\mathbf{\theta}_{t}\))
3:for\(t=1,2,...\)do
4: Compute retaining loss \(L_{\text{R}}(\mathbf{\theta}_{t})\) by one forward pass
5: Compute retaining gradient \(\mathbf{g}_{\text{R}}(\mathbf{\theta}_{t,s})=\nabla_{\mathbf{\theta}_{\text{LS}}}L_{ \text{R}}\)
6: Compute forgetting loss \(L_{\text{F}}(\mathbf{\theta}_{t})\) by one forward pass
7: Compute forgetting gradient \(\mathbf{g}_{\text{F}}(\mathbf{\theta}_{t,s})=\nabla_{\mathbf{\theta}_{\text{LS}}}L_{ \text{F}}\)
8: Compute Loss \(L(\mathbf{\theta}_{t})=w_{\text{R}}(\mathbf{\theta}_{t})L_{\text{R}}(\mathbf{\theta}_{t})+ w_{\text{F}}(\mathbf{\theta}_{t})L_{\text{F}}(\mathbf{\theta}_{t})\)
9: Compute \(\bar{\mathbf{g}}(\mathbf{\theta}_{\text{LS}})\) by averaging \(\mathbf{g}_{\text{R}}\) and \(\mathbf{g}_{\text{F}}\)
10: Compute GradNorm loss \[L_{GN}(\mathbf{\theta}_{t})=|\mathbf{g}_{\text{R}}-\bar{\mathbf{g}}\times[r_{\text{R}}( t)]^{\alpha}|_{1}+|\mathbf{g}_{\text{F}}-\bar{\mathbf{g}}\times[r_{\text{F}}(t)]^{ \alpha}|_{1}\]
11:Compute GradNorm gradients \(\nabla_{\mathbf{w}_{\text{R}}}L_{GN}\) and \(\nabla_{\mathbf{w}_{\text{F}}}L_{GN}\in\mathbb{R}\)
12:Compute the full gradient \(\nabla_{\mathbf{g}_{t}}L\)
13:Update \(v_{\text{R}}(\mathbf{\theta}_{t})\to w_{\text{R}}(\mathbf{\theta}_{t+1})\) and \(w_{\text{F}}(\mathbf{\theta}_{t})\to w_{\text{F}}(\mathbf{\theta}_{t+1})\) using \(\nabla_{\mathbf{w}_{\text{F}}}L_{GN}\) and \(\nabla_{\mathbf{w}_{\text{F}}}L_{GN}\)
14: Update \(\mathbf{\theta}_{t+1}=\mathbf{\theta}_{t}-\eta\nabla_{\mathbf{\theta}_{t}}L\)
15: Renormalize \(w_{\text{R}}(\mathbf{\theta}_{t+1})\) and \(w_{\text{F}}(\mathbf{\theta}_{t+1})\) so that \(w_{\text{R}}(\mathbf{\theta}_{t+1})+w_{\text{F}}(\mathbf{\theta}_{t+1})=2\)
```

**Algorithm 2** NGDiff

We compare the GradNorm algorithm Chen et al. (2018b) with our proposed method, NGDiff. We highlight some steps of GradNorm in red to indicate the differences than NGDiff:

* NGDiff sets the scaling coefficient as \(1/||\mathbf{g}_{\text{R}}||\) and \(1/||\mathbf{g}_{\text{F}}||\), while GradNorm uses gradient descent to learn these coefficients as \(w_{\text{R}}\) and \(w_{\text{F}}\).
* NGDiff is model-agnostic while GradNorm contains specific designs for multi-task architecture. In unlearning, there are **2 data splits** (i.e., F and R) and each data split defines one task. Hence all model parameters are shared. However, in the original form of GradNorm, there is **1 data split** on which multiple tasks are defined (can be more than 2). Hence the model parameters are partitioned into [shared layers, task 1 specific layers, task 2 specific layers].
* NGDiff computes the full per-task gradients whereas GradNorm only computes the last shared layer's gradients.
* NGDiff requires 2 back-propagation at each iteration but GradNorm requires 3 (2 for per-task gradients, 1 for \(\nabla_{\theta}L\)), which may translate to more training time for large models.
* GradNorm introduces additional hyperparameters that can be difficult and costly to tune, and may cause instability of training if not

Figure 6: Loss values and ROUGE scores on the forgetting and retaining data from the _TOFU_ dataset using different unlearning methods on the Phi-1.5 language model. We apply the _extended GDiff_ with various coefficients (see (3), \(0\leq c\leq 1\)) and connect the results with a blue dashed line. We denote MTO methods as different markers, and use a grey dashed line to represent the loss of random guess.



properly tuned. These hyperparameters include \(\alpha\) and two learning rates to update \(w_{\text{R}}\) and \(w_{\text{F}}\) in Line 13 of Appendix B. In contrast, NGDiff is hyperparameter-free when equipped with GeN (AutoLR).
* NGDiff are theoretically supported by Theorem 5, while the choice of hyperparameters and the use of a heuristic \(r_{i}(t)\) by GradNorm may require further justification. Here \(r_{i}(t)=\widetilde{L}_{i}(\mathbf{\theta}_{t})/\mathbb{E}_{\text{task}}[ \widetilde{L}_{i}(\mathbf{\theta}_{t})]\) is the "relative inverse training rate" of task \(i\), where \(\widetilde{L}_{i}(\mathbf{\theta}_{t})=L_{i}(\mathbf{\theta}_{t})/L_{i}(\mathbf{\theta}_{0})\), \(i\in\{\text{F},\text{R}\}\).

In summary, NGDiff is remarkably simpler and more well-suited than GradNorm for unlearning, with stable performance and theoretical ground.

## Appendix C Details related to GeN

### Brief introduction of GeN

GeN (Bu and Xu, 2024) is a method that sets the learning rate for any given gradient \(\mathbf{d}\) as

\[\eta_{\text{GeN}}=\frac{\mathbf{G}^{\top}\mathbf{d}}{\mathbf{d}^{\top}\operatorname{ \mathbf{H}}\mathbf{d}}\]

where \(\mathbf{G}\) is the gradient and \(\mathbf{H}\) is the Hessian matrix of some loss \(L\). One only needs to access the scalars \(\mathbf{G}^{\top}\mathbf{d}\) and \(\mathbf{d}^{\top}\mathbf{H}\mathbf{d}\), without computing the high-dimensional \(\mathbf{G}\) and \(\mathbf{H}\) (or Hessian-vector product). To do so, two additional forward passes are needed: given a constant (say \(\xi=0.001\)), we compute \(L(\mathbf{\theta}+\xi\mathbf{d})\) and \(L(\mathbf{\theta}-\xi\mathbf{d})\). Then by curve fitting or finite difference as demonstrated below, we can estimate up to arbitrary precision controlled by \(\xi\):

\[\mathbf{G}^{\top}\mathbf{d}\approx\frac{L(\mathbf{\theta}+\xi\mathbf{d})-L(\mathbf{\theta}- \xi\mathbf{d})}{2\xi}\]

and

\[\mathbf{d}^{\top}\mathbf{H}\mathbf{d}\approx\frac{L(\mathbf{\theta}+\xi\mathbf{d})-2L(\mathbf{ \theta})+L(\mathbf{\theta}-\xi\mathbf{d})}{\xi^{2}}\]

Notice that the regular optimization requires 1 forward pass and 1 back-propagation; GeN requires in total 3 forward passes and 1 back-propagation. Given that back-propagation costs roughly twice the computation time than forward pass, the total time increases from 3 units of time to 5 units. Nevertheless, GeN needs not to be applied at each iteration: if we update the learning rate every 10 iterations as in Remark 4.2, the total time reduces to \(3+2/10=3.2\) units, and the overhead is less than 10% compared to the regular optimization.

### Adapting GeN to unlearning

Naively applying GeN to the unlearning will result in

\[\eta_{\text{GeN}}=\frac{\mathbf{G}^{\top}\mathbf{g}_{\text{UN}}}{\mathbf{g}_{\text{ UN}}^{\top}\mathbf{H}\mathbf{g}_{\text{IN}}}\]

which minimizes the loss over all datapoints, in both F and R. This is against our goal to maximize the forgetting loss. We must consider the learning rate separately for F and R, as shown in Appendix G (Proof of Theorem 5). When both losses have a convex curvature in Figure 3, the optimal learning rate is only well-defined for \(L_{\text{R}}\) and we do not claim to maximize \(L_{\text{F}}\). In other words, if we minimize \(L_{\text{R}}\), we get to worsen \(L_{\text{F}}\) (though not maximally); if we choose to maximize \(L_{\text{F}}\), we will use infinite learning rate that also maximizes \(L_{\text{R}}\). Therefore, our learning rate in (7) only uses R instead of the whole dataset.

## Appendix D Computer Vision Experiments

To demonstrate the effectiveness of unlearning across other modalities, we also evaluate our method on the image classification task. Specifically, we choose the CIFAR-10 and CIFAR-100 dataset (Krizhevsky et al., 2009) and train a ResNet-50 (He et al., 2015) model from scratch. For the CIFAR-10 dataset, we sample 500 images from the class \(dog\) as the forgetting data, and use images

Figure 7: ROUGE scores and loss values during unlearning with vanilla GDiff (equally weighted), under different learning rates to which the unlearning performance is highly sensitive.

 

from the remaining 9 classes as the retaining data. For the CIFAR-100 dataset, we sample 500 images from the class \(bed\) as the forgetting data, and use images from the remaining 99 classes as the retaining data. After training, the initial forget data accuracy is \(0.926\), and the retain data accuracy is \(0.956\) on the CIFAR-10 dataset. The initial forget data accuracy is \(0.745\), and the retain data accuracy is \(0.750\) on the CIFAR-100 dataset. Then we apply different unlearning methods to the trained models. As shown in Table 4, all methods successfully reduce the forget accuracy to \(0\). However, the retaining accuracy of _NGDiff_ remains the highest, which shows its effectiveness in preserving the model utility in image classification tasks.

## Appendix E Other Related Works

Machine unlearningMachine unlearning is oftentimes viewed as a continual learning approach, that removes specific data points after a model has been trained to memorize them. Such removal is light-weighted in contrast to re-training, especially when the forgetting set is much smaller than the retaining. In addition to the methods already introduced in Section 3.2 (namely GA, GDiff and NPO), other methods include SISA Bourtoule et al. (2021), influence functions Ullah et al. (2021), differential privacy Gupta et al. (2021) and so on. However, these methods could be difficult to scale on large models and large datasets due to the algorithmic complexity. To our best knowledge, this is the first work that formulate the unlearning problem as a two-task problem, which can be solved by a number of well-known MTO methods.

Multi-task optimizationMTO is a paradigm where one model is trained to perform multiple tasks simultaneously, so as to significantly improve the efficiency in contrast to training multiple models, one for each task. The key challenge of MTO is the performance trade-off among tasks, where the multi-task model is worse than single-task model if trained on each task separately. Therefore, the core idea is to balance different tasks by modifying the per-task gradients, e.g. with normalization LossNorm and NGDiff), PCGrad Yu et al. (2020), RLW Lin et al. (2021), IMTL Liu et al. (2021), MGDA Desideri (2012), CAGradLiu et al. (2024a), GradVaccine Wang et al. (2020), GradDrop Chen et al. (2020), RotoGrad Javaloy and Valera (2022), etc.

Learning-rate-free methodsParameter-free or learning-rate-free methods automatically set the learning rate scheduler without the hyperparameter tuning, which is computationally infeasible for LLMs, e.g. LLAMA2 pre-training uses 3 hyperparameters just for the learning rate: warmup steps, peak learning rate, and minimum learning rate. At high level, there are two approaches to learning-rate-free methods.

On one hand, GeN Bu and Xu (2024) leverages the Taylor expansion and convex-like landscape of deep learning, which is applicable for the general purpose, even if the gradient is modified like in the unlearning.

On the other hand, methods like D-adaptation Defazio and Mishchenko (2023), Prodigy Mishchenko and Defazio (2024), DoG Ivgi et al. (2023), DoWG Khaled et al. (2024) are based on the convex and G-Lipschitz conditions: \(L({\bar{\theta}}_{T})-L({\mathbf{\theta}}_{*})\leq\frac{|{\theta}_{0}-{\theta}^{2} }{2iT}+\frac{{Q}^{2}}{2}\) where \({\theta}_{*}\) is the unknown minimizer of \(L\) and \({\bar{\theta}}_{T}\) is an averaging scheme of \(\{{\theta}_{0},...,{\theta}_{T}\}\). With the same theoretical foundation, these methods propose different ways to approximate the initial-to-final distance \(|{\theta}_{0}-{\theta}_{*}|\). There are two main issues to apply these methods on the unlearning. Firstly, the assumption of \(G\)-Lipschitz is hard to verify and the minimizer \({\theta}_{*}\) is not well-defined in multi-objective (see our discussion on Pareto optimality under Lemma 2). Secondly, the optimal learning rate \(\frac{|{\theta}_{0}-{\theta}_{*}|}{G\sqrt{T}}\) is defined in a manner to minimize the

\begin{table}
\begin{tabular}{c|c c|c c} \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c|}{**CIFAR-10**} & \multicolumn{2}{c}{**CIFAR-100**} \\  & _Forget Acc_ \(\downarrow\) & _Retain Acc_\(\uparrow\) & _Forget Acc_\(\downarrow\) & _Retain Acc_\(\uparrow\) \\ \hline
**No-unlearn** & \(0.926\) & \(0.956\) & \(0.745\) & \(0.750\) \\ \hline
**GDiff c=0.9** & \(0.000\) & \(0.817\) & \(0.000\) & \(0.664\) \\
**GDiff c=0.5** & \(0.000\) & \(0.830\) & \(0.000\) & \(0.609\) \\
**GDiff c=0.1** & \(0.000\) & \(0.825\) & \(0.000\) & \(0.667\) \\
**LossNorm** & \(0.000\) & \(0.753\) & \(0.000\) & \(0.432\) \\
**NGDiff** & \(0.000\) & **0.931** & \(0.000\) & **0.701** \\ \hline \end{tabular}
\end{table}
Table 4: Results of _Forget Acc_ and _Retain Acc_ using different unlearning methods on the CIFAR-10 dataset. Compared to other baseline methods, _NGDiff_ has the best performance on the model utility.

 

loss, whereas MTO methods operate on the gradient level. Hence MTO is incompatible to such parameter-free methods given that we cannot derive a corresponding loss (e.g. there exists no \(L_{\text{NGDiff}}\) such that \(\frac{\partial L_{\text{NGDiff}}}{\partial\theta}=\mathbf{g}_{\text{NGDiff}}\)).

## Appendix F Experiments

### Datasets

To evaluate the empirical performance of our proposed method, we experiment on the following datasets in Table 5.

* [leftmargin=0pt]
* _Task of fictitious Unlearning (TOFU)_Maini et al. (2024). This dataset consists of question-answer pairs based on fictitious author biographies generated by GPT-4 Achiam et al. (2023). Initially, predefined attributes, such as birthplace, gender, and writing genre, are assigned to 200 distinct authors. GPT-4 is then prompted to generate detailed information about each author. Following the synthesized data, 20 question-answer pairs are created for each fictitious author. The dataset is then divided into distinct datasets: the retaining set and the forgetting set. In our experiments, we use the _forget10_ and _retain90_ split, which excludes 10% of the original dataset.
* _MUSE-NEWS_Shi et al. (2024). This dataset consists of BBC news articles Li et al. (2023b) from August 2023. It includes seven subsets of news data: _raw_, _verbmem_, _knowmem_, _privileak_, _scal_, _sust_, and _train_. We utilize the _train_ split to finetune a target model, and then the _raw_ set, which includes both the forget and retain data, for the target model unlearning. Then, we use _verbmem_, _knownmem_ split to evaluate the unlearned model's performance.

### Evaluation Metrics

Following the existing work Shi et al. (2024), we evaluate the unlearning performance based on the quality of outputs from the model after unlearning. We expect a good performance should satisfy the following requirements:

**No verbatim memorization** We evaluate this metric by prompting the model with the first \(l\) tokens of the news data in the forget set and compare the model's continuation outputs with the ground truth continuation. Specifically, for each input \(x\in F\), we choose \(x_{[l]}\) as input, and compare the output \(f(x_{[l]})\) with the ground truth continuation \(x_{[l+1:]}\) with the ROUGE-L recall score:

\[\begin{split}&\textit{Verbmem}(f,F)\\ &=\frac{1}{\|F\|}\Sigma_{x}~{}\text{ROUGE-L}(f(x_{[l]}),x_{[l+1; ]})\end{split} \tag{9}\]

**No knowledge memorization** To evaluate this metric, we use the generated question-answer pair based on each example \(x\in F\). We prompt the model with the question part \(q\) and compare the output answer \(f(q)\) to the ground truth answer \(a\) using ROUGE-L recall scores:

\[\textit{Knowmem}(f,F)=\frac{1}{\|F\|}\Sigma_{x}~{}\text{ROUGE-L}(f(q),a) \tag{10}\]

**Maintained model utility** An effective unlearning method should also maintain the model's performance on the retain data. For the _MUSE-NEWS_ dataset, we use the _Knowmem\({}_{r}\)_ split, which consists of the generated question-answer pairs based on the retain data. For the _TOFU_ dataset, we prompt the model with the question from the retain set and compare the generated answer with the ground truth. We use ROUGE-L recall scores for evaluation:

\[\textit{Utility}(f,R)=\frac{1}{\|R\|}\Sigma_{x}~{}\text{ROUGE-L}(f(q),a) \tag{11}\]

Additionally, we evaluate the model using the _Retain10-perturbed_ split from the _TOFU_ dataset. It consists of five perturbed answers for each original answer, keeping original template but modifying the facts. We compute the Truth Ratio metric, which compares the likelihood of the model generating a correct answer versus an incorrect one for each question in the retain set. A higher Truth Ratio indicates better model utility that effectively remembers knowledge from the retain data.

### Hyper-parameter Settings

To finetune a targeted model with the full dataset, we use the optimizer Adam with a learning rate of \(\eta=\{10^{-5},2*10^{-5}\}\), a training batch size of {16, 32}, and train 25 epochs for all language models. For the unlearning process, we use the optimizer Adam with a learning rate \(\eta=\{10^{-5},5*10^{-5},10^{-4}\}\), and train 15 epochs for all unlearning 

methods. For the Phi-1.5, Falcon-1B, and GPT2-XL models, we perform full-model parameter tuning. For the Llama2-7B and Mistral-7B models, we apply the LoRA training method (Hu et al., 2021) with \(rank=8\).

### Other unlearning results

More results on _MUSE-NEWS_ dataset with Phi-1.5 model and Falcon-1B model are in Table 6.

### Visualization of learning rate scheduling

We monitor the gradient norms and the learning rate in Figure 8 for Algorithm 1. We observe that the automatic learning rate is indeed effective, picking up from \(5e-5\) to around \(2e-4\), and that NGDiff assigns a smaller learning rate to the forgetting gradient, not perturbing the model too much to maintain the high utility on the retaining set.

## Appendix G Proofs

In this section, we provide the proofs of all the lemma and theorems in this paper.

**Lemma 2** (restated from (Xin et al., 2022)).: _For any \(0<c<1\), the model \({\mathbf{\theta}^{*}_{LSP}}(c)=argmin_{\theta}\text{\emph{LSP}}({\mathbf{\theta};c})\) is Pareto optimal._

Proof of Lemma 2.: We show that the solution of LSP cannot be a dominated point, and therefore it must be Pareto optimal. Consider a solution \({\mathbf{\theta}^{*}}=\operatorname*{argmin}_{\theta}\text{\emph{LSP}}({\mathbf{\theta} ;c})\), and suppose it is dominated by some \({\mathbf{\theta}^{\prime}}\), i.e. \(L_{\text{F}}({\mathbf{\theta}^{*}})\leq L_{\text{F}}({\mathbf{\theta}^{\prime}}),L_{ \text{R}}({\mathbf{\theta}^{*}})\geq L_{\text{R}}({\mathbf{\theta}^{\prime}})\) with at least one inequality being strict. This contradicts that \({\mathbf{\theta}^{*}}\) is minimal as \(cL_{\text{R}}({\mathbf{\theta}^{*}})-(1-c)L_{\text{F}}({\mathbf{\theta}^{*}})>cL_{ \text{R}}({\mathbf{\theta}^{\prime}})-(1-c)L_{\text{F}}({\mathbf{\theta}^{\prime}})\). 

**Theorem 3**.: _For any \(\{c_{t}\}\) with \(0\leq c_{t}\leq 1\) that converges as \(t\to\infty\), the model \({\mathbf{\theta}^{*}}(\{c_{t}\})=\lim_{t\to\infty}{\mathbf{\theta}_{t}}\) in (4) is Pareto optimal._

Proof of Theorem 3.: Let \(c=\lim_{t}c_{t}\), then Eq. (4) gives that \({\mathbf{g}}_{\text{UN}}({\mathbf{\theta}_{t}})=c_{t}{\mathbf{g}}_{\text{R}}({\mathbf{\theta} _{t}})-(1-c_{t}){\mathbf{g}}_{\text{F}}({\mathbf{\theta}_{t}})\to cg_{\text{R}}({\mathbf{ \theta}^{*}})-(1-c){\mathbf{g}}_{\text{F}}({\mathbf{\theta}^{*}})={\bf 0}\) as \(t\to\infty\). Note \({\mathbf{\theta}^{*}}(\{c_{t}\})\) is equivalent to the LSP solution \({\mathbf{\theta}^{*}_{\text{LSP}}}(c)=\operatorname*{argmin}_{\theta}\text{\emph{ LSP}}({\mathbf{\theta};c})\) as the latter has the same stationary condition, which is Pareto optimal by Lemma 2. 

**Lemma 4**.: \({\mathbf{g}}_{\text{NGDiff}}({\mathbf{g}}_{\text{R}},{\mathbf{g}}_{\text{F}})\) _satisfies Eq. (6) for any \({\mathbf{g}}_{\text{R}}\in\mathbb{R}^{d}\) and \({\mathbf{g}}_{\text{F}}\in\mathbb{R}^{d}\)._

Proof of Lemma 4.: We firstly show \({\mathbf{g}}_{\text{R}}^{\top}{\mathbf{g}}_{\text{UN}}\geq 0\) for \({\mathbf{g}}_{\text{UN}}={\mathbf{g}}_{\text{NGDiff}}\). We write

\[{\mathbf{g}}_{\text{R}}^{\top}{\mathbf{g}}_{\text{NGDiff}} ={\mathbf{g}}_{\text{R}}^{\top}\left(\frac{{\mathbf{g}}_{\text{R}}}{\|{ \mathbf{g}}_{\text{R}}\|}-\frac{{\mathbf{g}}_{\text{F}}}{\|{\mathbf{g}}_{\text{F}}\|}\right)\] \[=\|{\mathbf{g}}_{\text{R}}\|-\frac{{\mathbf{g}}_{\text{R}}^{\top}{\mathbf{g}} _{\text{F}}}{\|{\mathbf{g}}_{\text{F}}\|}\geq\|{\mathbf{g}}_{\text{R}}\|-\frac{\|{\mathbf{ g}}_{\text{R}}\|\|{\mathbf{g}}_{\text{F}}\|}{\|{\mathbf{g}}_{\text{F}}\|}\] \[=0\]

where the inequality is the Cauchy-Schwarz inequality. Similarly, \({\mathbf{g}}_{\text{F}}^{\top}{\mathbf{g}}_{\text{UN}}\leq 0\) easily follows. 

**Theorem 5**.: _Consider \({\mathbf{\theta}_{t+1}}={\mathbf{\theta}_{t}}-\eta{\mathbf{g}}_{\text{NGDiff}}\)._

_(1) Unless \({\mathbf{g}}_{\text{R}}\) is exactly parallel to \({\mathbf{g}}_{\text{F}}\), for any sufficiently small learning rate \(\eta\), there exist two constants \(\epsilon_{\text{R},1}=o(\eta),\epsilon_{\text{F},1}=o(\eta)\) such that_

\[L_{\text{R}}({\mathbf{\theta}_{t+1}})-L_{\text{R}}({\mathbf{\theta}_{t}})<\epsilon_{ \text{R},1};\]

\[L_{\text{F}}({\mathbf{\theta}_{t+1}})-L_{\text{F}}({\mathbf{\theta}_{t}})>\epsilon_{ \text{F},1}.\]

_(2) If additionally \({\mathbf{g}}_{\text{NGDiff}}^{\top}{\mathbf{H}}_{\text{R}}{\mathbf{g}}_{\text{NGDiff}}>0\) and \({\mathbf{g}}_{\text{NGDiff}}^{\top}{\mathbf{H}}_{\text{F}}{\mathbf{g}}_{\text{NGDiff}}>0\), then for any learning rate \(0<\eta<\frac{2{\mathbf{g}}_{\text{R}}^{\top}{\mathbf{g}}_{\text{NGDiff}}}{{\mathbf{g}}_{ \text{NGDiff}}^{\top}{\mathbf{H}}_{\text{R}}{\mathbf{g}}_{\text{NGDiff}}}\), there exist two constants \(\epsilon_{\text{R},2}=o(\eta^{2}),\epsilon_{\text{F},2}=o(\eta^{2})\) such that_

\[L_{\text{R}}({\mathbf{\theta}_{t+1}})-L_{\text{R}}({\mathbf{\theta}_{t}})<\epsilon_{ \text{R},2};\]

Figure 8: Gradient norms and learning rates during the unlearning on _TOFU_ dataset using NGDiff and Phi-1.5 model. The AutoLR scheduler assigns a smaller learning rate to the forgetting gradient, effectively preserving model utility on the retaining set.

 

\[L_{\text{F}}(\mathbf{\theta}_{t+1})-L_{\text{F}}(\mathbf{\theta}_{t})> \epsilon_{\text{F,2}}.\]

Proof of Theorem 5.: Applying (5) with \(\mathbf{g}_{\text{NGDiff}}\) gives

\[L_{\text{R}}(\mathbf{\theta}_{t+1})-L_{\text{R}}(\mathbf{\theta}_{t})\] \[=-\eta g^{\top}_{\text{R}}\mathbf{g}_{\text{NGDiff}}+\frac{\eta^{2}}{ 2}\mathbf{g}^{\top}_{\text{NGDiff}}\mathbf{H}_{\text{R}}\mathbf{g}_{\text{NGDiff}}+o( \eta^{2}) \tag{12}\]

For part (1), note that Lemma 4 gives \(\mathbf{g}^{\top}_{\text{R}}\mathbf{g}_{\text{NGDiff}}>0\) unless \(\mathbf{g}_{\text{F}}\parallel\mathbf{g}_{\text{R}}\). Hence for any \(\eta>0\), we have

\[L_{\text{R}}(\mathbf{\theta}_{t+1})-L_{\text{R}}(\mathbf{\theta}_{t})=- \eta g^{\top}_{\text{R}}\mathbf{g}_{\text{NGDiff}}+o(\eta)<o(\eta)\]

and similarly for \(L_{\text{F}}\).

For part (2), now that \(\mathbf{g}^{\top}_{\text{NGDiff}}\mathbf{H}_{\text{R}}\mathbf{g}_{\text{NGDiff}}>0\), we have

\[-\eta g^{\top}_{\text{R}}\mathbf{g}_{\text{NGDiff}}+\frac{\eta^{2}}{ 2}\mathbf{g}^{\top}_{\text{NGDiff}}\mathbf{H}_{\text{R}}\mathbf{g}_{\text{NGDiff}}<0\] \[\Longleftrightarrow 0<\eta<\frac{2\mathbf{g}^{\top}_{\text{R}}\mathbf{g}_{ \text{NGDiff}}}{\mathbf{g}^{\top}_{\text{NGDiff}}\mathbf{H}_{\text{R}}\mathbf{g}_{\text {NGDiff}}}\]

and similarly

\[-\eta\mathbf{g}^{\top}_{\text{F}}\mathbf{g}_{\text{NGDiff}}+\frac{\eta^{2}}{2}\mathbf{g}^ {\top}_{\text{NGDiff}}\mathbf{H}_{\text{F}}\mathbf{g}_{\text{NGDiff}}>0\Longleftarrow 0<\eta\]

We complete the proof by substituting the inequalities into (12). 

**Remark G.1**.: _There is a computational overhead to use GeN, as it requires additional forward passes to estimate \(\eta^{*}_{t}\). Nevertheless, we only update the learning rate every 10 iterations so that the overheads is amortized and thus negligible._

Proof of Remark G.1.: We extend our original complexity analysis in Remark 4.2 and provide quantitative analysis of computation overheads in FLOPs (floating point operations). Specifically, assume it takes around N FLOPS to perform one forward pass on one example and 2N FLOPS to backpropagate. The basic GDiff requires roughly 6BTN FLOPS to run with batch size B and total number of iterations T, because it needs 1 forward and 1 backward for the retain set and another for the forget set. Our learning-rate-free NGDiff requires about (6BTN+4BTN/10), hence a 6.6% increase to unlearn the data. We assure that the extra forward passes do not add memory burden, because they are in gradient-free mode (e.g. under torch.no_grad() mode). All in all, NGDiff is almost as efficient as GDiff and other unlearning methods, as the target unlearning corpus is usually relatively small. This has been empirically observed by our large-scale (up to 7B) model unlearning experiments. 

## Appendix C

\begin{table}
\begin{tabular}{c|l|c c c c c c c} \hline \hline \multirow{2}{*}{**Base Model**} & \multicolumn{3}{c|}{**Metric**} & \multicolumn{8}{c}{**Method**} \\  & & _No-unlearn_ & _GDiff-0.9_ & _GDiff-0.5_ & _GDiff-0.1_ & _NPO_ & _LossNorm_ & _NGDiff_ \\ \hline \multirow{3}{*}{Phi-1.5} & Verbmem \(\downarrow\) & \(0.018\) & \(0.000\) & \(0.012\) & \(0.000\) & \(0.000\) & \(0.012\) & **0.004** \\  & Utility \(\uparrow\) & \(0.372\) & \(0.277\) & \(0.061\) & \(0.000\) & \(0.000\) & \(0.061\) & **0.001** \\  & Knowmum \(\downarrow\) & \(0.030\) & \(0.000\) & \(0.002\) & \(0.000\) & \(0.000\) & \(0.002\) & **0.023** \\ \hline \multirow{3}{*}{Falcon-1B} & Verbmem \(\downarrow\) & \(0.204\) & \(0.132\) & \(0.000\) & \(0.000\) & \(0.000\) & \(0.126\) & **0.000** \\  & Utility \(\uparrow\) & \(0.386\) & \(0.214\) & \(0.000\) & \(0.000\) & \(0.000\) & \(0.142\) & **0.025** \\ \cline{1-1}  & Knowmum \(\downarrow\) & \(0.232\) & \(0.078\) & \(0.000\) & \(0.000\) & \(0.000\) & \(0.130\) & **0.087** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Results of _Verbetatim memorization, Model utility_, and _TruthRatio_ on _MUSE-NEWS_ dataset with different unlearning methods on Phi-1.5, and Falcon-1B models. Lower _Verbmem_ along with higher _Utility_ and _TruthRatio_ indicate a more superior unlearning strategy.