# Improving Unlearning with Model Updates Probably Aligned with Gradients

Virgile Dine, Teddy Furon, Charly Faure

► To cite this version:

Virgile Dine, Teddy Furon, Charly Faure. Improving Unlearning with Model Updates Probably Aligned with Gradients. AISec 2025 - 18th ACM Workshop on Artificial Intelligence and Security, Oct 2025, Taipei, Taiwan. pp.4889-4891, 10.1145/3733799.3762975. hal-05247290

HAL Id: hal-05247290

https://hal.science/hal-05247290v1

Submitted on 8 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Improving Unlearning with Model Updates
Probably Aligned with Gradients

Virgile Dine*¹, Teddy Furon¹, and Charly Faure²

¹Centre Inria de l'Université de Rennes, France

²AMIAD, France

Abstract

We formulate the machine unlearning problem as a general constrained optimization problem. It unifies the first-order methods from the approximate machine unlearning literature. This paper then introduces the concept of feasible updates as the model's parameter update directions that help with unlearning while not degrading the utility of the initial model. Our design of feasible updates is based on masking, i.e. a careful selection of the model's parameters worth updating. It also takes into account the estimation noise of the gradients when processing each batch of data to offer a statistical guarantee to derive locally feasible updates. The technique can be plugged in, as an add-on, to any first-order approximate unlearning methods. Experiments with computer vision classifiers validates this approach.

# 1 Introduction

Machine learning models are integrated into many real-world applications. Since these models contain artifacts of potentially sensitive training data, this raises concerns about data confidentiality and user privacy. The ability to remove specific training data from a model has emerged as a key mechanism to enforce, for instance, the "right to be forgotten" promoted by the European GDPR law [17] or the "right to erase" in the Canadian CPPA legislation [29]. Approximate machine unlearning aims to find efficient mechanisms, avoiding the cost of learning a new model from scratch over the training dataset deprived of the sensitive data.

Privacy is not the only application of machine unlearning. It has been proven useful as a defense against backdoor attacks by annihilating the influence of the poisoned training data [45, 38], or to improve fairness by removing data that induce biases in the training set. Another scenario is the derivation of a restricted public model from a powerful private model learned on some sensitive data [12]. The accuracy of the public model should be on par with or slightly degraded compared to the private model. On the other hand, the model should not leak information about the sensitive training data. The procedure used by an attacker to infer whether a specific data record was included in the training set is known as a Membership Inference Attack (MIA) [33].

This mechanism inherently provokes tension between the goal of forgetting some training data and the non-desired consequence of unlearning the model's capabilities. Indeed, the term *Machine*

<footnote>*contact author: virgile.dine@inria.fr
paper accepted to AISec'25 co-located with the 32nd ACM Conference on Computer and Communications Security</footnote>
*Unlearning* is somewhat misleading: it unlearns some training data, meaning forgets, not its ability to perform a given task (coined as *Catastrophic Forgetting* in machine unlearning).

This work assumes that the unlearner is also the creator of the initial classifier. It means that the unlearner knows the initial model and all its parameters, the dataset to retain, and the dataset to forget. The unlearner aims to find an update of the initial model's parameters that erases the influence of the forget dataset.

Our first contribution formalizes machine unlearning as a constrained optimization problem: forgetting the data while sustaining the model's utility. This generalizes first-order unlearning methods through a single framework. A second contribution introduces the concept of feasible update as a sound elementary solution to the constrained optimization problem. We propose a general procedure to craft feasible updates based on masks, carefully selecting the parameters to be updated. These are presented as add-ons to be plugged into the existing unlearning methods covered by our framework. A third contribution takes into account the processing per batch of data. A simple statistical model relating the gradient computed over a batch to the gradient computed over the whole dataset gives birth to a statistical guarantee that the mask selects the appropriate parameters. The appendix contains the proofs of our propositions. To summarize, this work provides a theoretical foundation for masking unlearning methods like saliency unlearning [8]. Our approach is validated over 360 experimental configurations combining several add-ons, unlearning methods, classifiers, datasets, types of unlearning scenarios, and sizes of the portion of data that must be unlearned.

# 2 Related Work

## 2.1 Machine Unlearning

This work deals with centralized machine unlearning, which starts with a regular neural network classification model and obtains a model sharing the same architecture but with updated parameters. It excludes techniques that prepare the unlearning actually before the learning by resorting to specific data structures and model architectures, like split unlearning techniques [11, 42, 13, 10, 5, 2]. These techniques indeed target exact unlearning, which makes the output of the unlearned model statistically indistinguishable from the output of the model that never saw the forget data during its training [41].

On the contrary, our proposal pertains to the approximate unlearning strategy and, more specifically, to fast first-order methods that compute one or more gradients to proceed a model's parameters update. This excludes the more demanding second-order methods that compute Hessian matrices [14].

Among the first-order method family, one can find the following basic schemes: Fine-Tuning (FT) [40] where the model is further refined using only the remaining dataset $D_{\text{retain}}$, Gradient Ascent (GA) [35] which attempts to undo learning by applying gradient ascent on the forget dataset $D_{\text{forget}}$, and Successive Random Labeling (SRL) [12] which fine-tunes the model over the whole dataset $D_{\text{train}}$ but with random labels for data belonging to $D_{\text{forget}}$. Slightly more elaborated are bi-objective schemes such as Negative Gradient Plus (NGPlus) [24], a mixed version of gradient ascent and fine-tuning that performs a gradient ascent over $D_{\text{forget}}$ with a descent over $D_{\text{retain}}$. It also encompasses methods considering the gradient of functions that are not classification losses. For example sparse-MU [19], which integrates the $\ell_1$-norm of the model parameter as a penalty. SCalable Remembering and Unlearning unBound (SCRUB) [24], minimizes (resp. maximizes) the Kullback-Leibler divergence between the predicted probabilities of the initial and the unlearned models over $D_{\text{retain}}$ (resp. $D_{\text{forget}}$).
Finally, our work is inspired by the Saliency Unlearning method (SalUn) [8], which enforces the sparsity of the parameter update by a binary mask. It amounts to selecting some parameters to be updated and leaving the remaining ones untouched. The selection is driven by the amplitude of the components of the gradient of the classification loss over $D_{forget}$. Our work proposes a theoretical framework for designing more explainable masks.

Interested readers may find many more references in the following surveys [32, 41, 39].

## 2.2 Membership Inference Attack

A Membership Inference Attack (MIA) discloses whether a given piece of data belongs to the training set of a model. In other words, whether the model saw this piece of data during its training. The evaluation of an unlearning technique uses an MIA to verify that the data of $D_{forget}$ are no longer deemed as training data.

MIAs are usually intensive because they compute shadow models, i.e. a set of models trained with and without this piece of data in order to model how the membership impacts the outputs of a classifier like [33] or the LiRa attack [4]. U-LiRa [15] is an attack more adapted to machine unlearning that distinguishes data unlearned or not in the training set. Lightweight alternative MIAs (dedicated to unlearning or not) refuse the cost of the shadow models' computation. They take decisions based on simple statistics from the model output [17, 42, 25, 1].

Even though U-LiRa is more effective, the experiments reported in [15] reflect that the MIAs lead to similar evaluations: when U-LiRa measures that one unlearning method is better than another, another MIA generally leads to the same conclusion. Our paper proposes versatile add-ons that can be plugged into many unlearning techniques. Our objective is to show the impact due to our add-ons. Therefore, we can resort to simple MIAs to measure a relative improvement w.r.t. the baseline technique.

## 2.3 Gradient Agreements

This article frames unlearning as a constrained optimization problem. Similarly to Multi-Task Learning with two tasks, the issue is to combine two gradients (w.r.t. the model parameters) into a single parameter update vector. Some well-known aggregations are IGA [22], or PCGrad [44] and CAGrad [27] that modify gradients by projecting one onto another. Some recent proposals induce a sparse aggregation. The AND masking strategy [30] verifies whether all gradients point in the same direction and updates a network parameter only if there is full agreement among them. This masking approach is similar to using a logical AND operation in gradient directions. The idea was improved [31] to tackle real-world problems where the collected data differs with respect to some environment. Our work rediscovers the AND formulation but from a different rationale stemming from the KKT conditions (see Prop. 1).

# 3 Problem Statement

## 3.1 Notations

Datasets are noted with a calligraphic font. For a given function $F$, $\mathbb{E}_{s \sim D}[F(s)]$ is the expectation of the random variable $F(s)$ when $s$ follows a uniform distribution over the finite set $D$. In the same manner, $\mathbb{P}_{s \sim D}[F(s) = y]$ is the probability that the output of $F$ equals a given $y$.
We denote by $u \odot v \doteq (u_i v_i)_i$ the vector corresponding to the term-by-term product of two vectors $u = (u_i)_i$ and $v = (v_i)_i$. We write $u \le 0$ (resp. $\ge 0$) when the vector $u$ verifies $\forall i, u_i \le 0$ (resp. $\ge 0$). This enables us to use inequalities directly on vectors when it makes sense.

For the sake of simplicity, we note $F \odot G \doteq s \mapsto F(s) \odot G(s)$ and equations are written function-wise whenever possible, i.e. $F = G$ when $\forall s, F(s) = G(s)$, or $M$ for a model $M_\theta$ if there is no ambiguity.

## 3.2 Exact and Approximate Unlearning

Consider a trained classifier $M_{\theta_0}: \mathcal{X} \to \mathcal{Y}$ predicting labels in the set $\mathcal{Y}$ from observations in the input space $\mathcal{X}$. Supervised learning found the model's parameters $\theta_0$ by optimizing a loss function $L$ over a training set $\mathcal{D}_{\text{train}} \subset \mathcal{X} \times \mathcal{Y}$:

$$ \theta_0 \in \arg\min_{\theta \in \Theta} \mathbb{E}_{(x,y) \sim \mathcal{D}_{\text{train}}} [L(M_{\theta}(x), y)]. \quad (1) $$

Data forgetting consists in unlearning some portion $\mathcal{D}_{\text{forget}} \subset \mathcal{D}_{\text{train}}$ of the data while conserving performance as if the model had been trained over the set to retain $\mathcal{D}_{\text{retain}} = \mathcal{D}_{\text{train}} \setminus \mathcal{D}_{\text{forget}}$. Then exact unlearning amounts to finding the parameters $\theta^*$ of a new model as if it was trained from scratch without the problematic data $\mathcal{D}_{\text{forget}}$ as follows:

$$ \theta^* \in \arg\min_{\theta \in \Theta} \mathbb{E}_{(x,y) \sim \mathcal{D}_{\text{retain}}} [L(M_{\theta}(x), y)]. \quad (2) $$

Approximate unlearning methods consist of low-cost modification of the trained parameters $\theta_0$ so that the unlearned model is close to the exact one (2), which has never seen the set $\mathcal{D}_{\text{forget}}$.

## 3.3 Usual Evaluation Metrics

In real-world unlearning scenarios, the parameters $\theta^*$ of the exact model is unknown, preventing any measure of closeness in the parameter space $\Theta$. Indeed, it remains to be defined how the unlearned model should mimic the behavior of the exact one, $M_{\theta^*}$. This is usually measured by common accuracy metrics [19] such as:

$$ \mathbb{P}_{(x,y) \sim \mathcal{D}} [M(x) = y], \quad (3) $$

with $\mathcal{D} = \mathcal{D}_{\text{forget}}$ for the Unlearning Accuracy (UA); $\mathcal{D} = \mathcal{D}_{\text{retain}}$ for the Retain Accuracy (RA); and $\mathcal{D} = \mathcal{D}_{\text{test}}$, a test set with empty intersection with $\mathcal{D}_{\text{train}}$, for the Test Accuracy (TA).

Another important criterion is the attack success rate of MIAs. Verifying if the binary decision rule $b$, trained to output 1 if the piece of data $x$ is deemed as belonging to the training set and 0 otherwise, has a good accuracy over the forget set:

$$ \text{MIA} \doteq \mathbb{E}_{(x,y) \sim \mathcal{D}_{\text{forget}}} [b(M, x)]. \quad (4) $$

The lower, the better. If this score is low, the decision rule considers the forgotten data as test data. If it is high, the decision rule considers the forgotten data as part of the training data. A value near 1/2 means that the decision rule cannot assign the whole forgotten data to be part of the training or test data.

Last but not least, the Run Time Efficiency (RTE) measures the duration of the unlearning procedure.

Our experimental protocol introduces two other metrics (see Sect. 6.2) that are rarely used in the literature because they require the computation of the ideal model $M_{\theta^*} (2)$.
# 4 General Framework for First-Order Methods

This section proposes a framework that encompasses the bi-objective first-order unlearning methods proposed in the literature and that leads to the concept of feasible update.

## 4.1 Optimization Problem

Let $M_{\theta_0}$ be the initial model (1) trained to minimize the loss $L$ over the dataset $\mathcal{D}_{\text{train}}$, and $\mathcal{D}_{\text{forget}} \subset \mathcal{D}_{\text{train}}$ the dataset to be forgotten. We model the unlearning as a constrained optimization problem:

$$ \begin{array}{ll} \min_{\theta \in \Theta} & U(\theta) \\ \text{s. t.} & C(\theta) \le 0, \end{array} \qquad (5) $$

where the functional $U$ drives the unlearning objective while $C$ encodes the preservation of the classification performance. These functions are not always, but typically, defined as:

$$ U(\theta) \doteq \mathbb{E}_{(x,y) \sim \mathcal{D}_U} [L_U(M_{\theta}, x, y)], \qquad (6) $$

$$ C(\theta) \doteq \mathbb{E}_{(x,y) \sim \mathcal{D}_C} [L_C(M_{\theta}, x, y) - L_C(M_{\theta_0}, x, y)], \qquad (7) $$

where $\mathcal{D}_U$ and $\mathcal{D}_C$ are two datasets, and $L_U$ and $L_C$ are two losses. The constraint means that the parameter $\theta$ shall not deteriorate the performances of the initial model, at least over the subset $\mathcal{D}_C$. The formulation (5) is general as it encompasses the first-order methods introduced in Sect. 2.1:

* FT (Fine-tuning): set $L_U = L$ with $\mathcal{D}_U = \mathcal{D}_{\text{retain}}$, and $C$ the null function (no constraint),

* GA (Gradient Ascent): set $L_U = -L$ with $\mathcal{D}_U = \mathcal{D}_{\text{forget}}$, and $C$ the null function (no constraint),

* NGPlus (Negative Gradient Plus): set $L_U = -L$ with $\mathcal{D}_U = \mathcal{D}_{\text{forget}}$, and $L_C = L$ with $\mathcal{D}_C = \mathcal{D}_{\text{retain}}$,

* SRL (Successive Random Labeling): set $L_U = L$ over a special dataset $\mathcal{D}_U$ where $y$ is replaced by a random label when $(x, y) \in \mathcal{D}_{\text{forget}}$, and $L_C = L$ with $\mathcal{D}_C = \mathcal{D}_{\text{retain}}$,

* $\ell_1$-sparse MU: set $L_U = \|\theta\|_1$ with $\mathcal{D}_U = \mathcal{D}_{\text{retain}}$, and $L_C = L$ with $\mathcal{D}_C = \mathcal{D}_{\text{retain}}$,

* SCRUB: set $L_U = -\text{KL}(M_{\theta_0} \| M_{\theta})$ with $\mathcal{D}_U = \mathcal{D}_{\text{forget}}$, and $L_C = \text{KL}(M_{\theta_0} \| M_{\theta}) + \gamma \cdot L$ with $\mathcal{D}_C = \mathcal{D}_{\text{retain}}$.

## 4.2 KKT Condition at Equilibrium

A classical result in constrained optimization allows us to relate the gradients of the unlearning objective and constraint functions.

**Proposition 1** (Necessary Condition at Equilibrium). In the optimization problem (5), the KKT conditions impose the necessary condition at the equilibrium $\bar{\theta}$:

$$ \nabla U \odot \nabla C(\bar{\theta}) \le 0. \qquad (8) $$

The Prop. 1 is easily interpreted: at a local minimum $\bar{\theta}$ there is no longer a direction in the parameter space to improve $U$ while maintaining performance $C$.
## 4.3 Feasible Update

This section first gives a desirable property of an update of the model parameter $\theta \leftarrow \theta + \eta\Delta$, and then proposes a design.

**Definition 1.** The update direction $\Delta \in \Theta$ is said to be feasible if it is negatively aligned with both gradients $\nabla U(\theta)$ and $\nabla C(\theta)$:

$$ \langle \Delta, \nabla U(\theta) \rangle \le 0 \quad \text{and} \quad \langle \Delta, \nabla C(\theta) \rangle \le 0. \qquad (9) $$

The goal is simple: such an update, up to the first order, minimizes the loss function $U$ while decreasing the constraint $C$.

**Proposition 2** (Feasible update's Guarantee). Any feasible update (9) ensures to fulfill the constraint while improving the minimization of the problem (5). That is, for any feasible $\Delta$, there exists a sufficiently small learning rate $\eta > 0$ so that starting at $\theta_0$ one has

$$ U(\theta_0 + \eta\Delta) \le U(\theta_0) \quad \text{and} \quad C(\theta_0 + \eta\Delta) \le 0. \qquad (10) $$

Our design for a feasible update is based on two parts: a special weighting of gradients and an aggregation Agg. Following the necessary condition 1, we define a selection of the parameters to be updated. This takes the form of a boolean mask:

$$ m_{\text{AND}}(\nabla U, \nabla C)_i \doteq \begin{cases} 1 & \text{if } \partial/\partial\theta_i U \cdot \partial/\partial\theta_i C > 0, \\ 0 & \text{otherwise.} \end{cases} \qquad (11) $$

The interpretation is also simple: the mask indicates the parameters that we can safely update. Up to the first order, a change of one of these parameters modifies both $U$ and $C$ in the same way, whether it is an increase or a decrease.

From the construction of the mask, it is easy to design a feasible update by combining both gradients. We define a sign-invariant function $\text{Agg}: \mathbb{R} \times \mathbb{R} \to \mathbb{R}$. The following construction is feasible as it ensures a negative correlation with the gradients:

$$ \Delta_{\text{AND}} \doteq -m_{\text{AND}} \odot \text{Agg}(\nabla U, \nabla C), \qquad (12) $$

with an abuse of notation where the equation is function-wise (i.e., $\theta$ is omitted for the sake of simplicity) and where function Agg is applied component-wise. As for the choice of this sign-invariant function, we propose:

$$ \text{Agg}(x, y) = \alpha x + \beta y, \quad (\alpha, \beta) \in \mathbb{R}_+^2, \qquad (13) $$

$$ \text{Agg}(x, y) = \underset{z \in \{x, y\}}{\operatorname{argmin}} |z|. \qquad (14) $$

The last suggestion is indeed the aggregation choice made in the Mask-Small-Gradients (MSG) method [25] to compute the random re-initialization before the fine-tuning phase.

Our technique belongs to the first-order methods family that makes a small parameter shift. Additionally, its amplitude can be quantified.

**Proposition 3** (Vicinity of $\theta_0$). Denoting the number of parameters selected by any mask $m$ by $\|m(\nabla U, \nabla C)\|_0$ and the update direction $\Delta \doteq -m \odot \text{Agg}(\nabla U, \nabla C)$. Then the updated parameters $\theta = \theta_0 + \eta\Delta(\theta_0)$, is close to the original parameter: $\forall q \ge 1$

$$ \|\theta - \theta_0\|_q \le \eta \|m(\nabla U, \nabla C)(\theta_0)\|_0^{1/q} \|\Delta(\theta_0)\|_{\infty}. \qquad (15) $$
<figure><img src="image_2.png" /><figcaption>Figure 1: Masking procedure to ensure agreement between losses with Agg (14) in pink and (13) in blue ($\alpha = 0.8, \beta = 0.5$).</figcaption></figure>

Moreover, if the update is based on the absolute min function (14):

$$ \|\theta - \theta_0\|_q \le \eta \|m(\nabla U, \nabla C)(\theta_0)\|_0^{1/q} \|\nabla C(\theta_0)\|_{\infty}. \quad (16) $$

It turns out that if $\mathcal{D}_C \subset \mathcal{D}_{\text{train}}$ then $\|\nabla C(\theta_0)\|_{\infty} \approx 0$ because the constraint is usually the classification loss over the data to be retained, i.e. data already seen during the training of $M_{\theta_0}$. Besides, experimentally $\|m_{\text{AND}}(\nabla U, \nabla C)(\theta_0)\|_0$ happens to decrease so that the update is at each step a slight change of the parameters. As models are Lipschitz w.r.t. those parameters [37, 9], the output of the updated model is never drastically different than the output of the initial model.

# 5 Processing with Batches

The previous section explains an add-on technique, assuming that the unlearning procedure has access to functions $U$ and $C$ and their gradients. This section keeps the same optimization problem (5) but without access to the true gradients $\nabla U$ and $\nabla C$ and we use new notations to clarify this distinction. This illustrates the case where $U$ and $C$ are averages of classification losses over large datasets. Machine learning practitioners typically address this issue by computing gradients over batches.
## 5.1 Probabilistic Modeling of Gradients

We consider a classic statistical model: computed over a random batch, the gradient $\hat{g}_U(\theta)$ (resp. $\hat{g}_C(\theta)$) is a noisy estimation of the true gradient $g_U(\theta) \doteq \nabla U(\theta)$ (resp. $g_C(\theta) \doteq \nabla C(\theta)$):

$$ \hat{g}_U = g_U + N_U, \text{ with } N_U \sim \mathcal{N}(0; \Sigma_U), \\ \hat{g}_C = g_C + N_C, \text{ with } N_C \sim \mathcal{N}(0; \Sigma_C), \qquad (17) $$

where $N_U$ and $N_C$ are two independent Gaussian random vectors.

The estimation noise may induce a modification of some parameters that the true gradients do not recommend (see Fig. 2). Our conservative strategy is to make the selection process by the mask (11) as certain as possible. This amounts to evaluating the probability of a correct selection of the i-th parameter, based on the noisy components $\hat{g}_{U,i}$ and $\hat{g}_{C,i}$ of the gradients observed at a given batch.

<figure><img src="image_3.png" /><figcaption>Figure 2: Masking wrong modification direction caused by noisy gradient, with Agg (13) for the updates ($\alpha = 0.8, \beta = 0.5$).</figcaption></figure>

**Proposition 4 (Gradient’s Probable Alignment).** Given the statistical model (17) and a batch over which the empirical gradients take the value $\hat{g}_U$ and $\hat{g}_C$, the selection of the i-th parameter is a good
choice with probability:

$$ \mathbb{P}[g_{U,i}g_{C,i} > 0 | \hat{g}_U, \hat{g}_C] = \phi_{U,i} \cdot \phi_{C,i} + (1 - \phi_{U,i})(1 - \phi_{C,i}), \quad (18) $$

where the vector $\phi_U \doteq \Phi(\hat{g}_U/\sqrt{\mathrm{diag}(\Sigma_U)})$ (idem for $\phi_C$) and the operators are taken component-wise. $\Phi$ is the cumulative distribution function of a standard Gaussian: $\Phi(x) \doteq \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{x} \exp(-\frac{t^2}{2}) dt$.

Figure 3 shows the empirical distribution of the probability (18) computed over all the components $\theta_i$ for a given batch. Roughly speaking, there is one quarter of the parameters where the signs of the true gradient components likely disagree (small probability), one quarter where they likely agree (large probability), and one half where nothing is certain (probability around 1/2). This latter case happens typically when at least one of $\hat{g}_{U,i}$ or $\hat{g}_{C,i}$ takes a small value compared to the associated standard deviation, so that $\phi_{U,i}$ or $\phi_{C,i}$ is close to 1/2, and so is the probability (18). The figure shows that, conditioned on the observation of a batch, those different regimes are evenly represented.

An implementation issue is the evaluation of the standard deviation of the estimation noises $N_{U,i}$ and $N_{C,i}$. A simple solution is to retrieve the standard deviation of the i-th component of the gradient per datum included in the batch. This quantity is indeed computed efficiently as the stacked moving average second-order momentum in the PyTorch implementation of the Adam optimizer [21].

<figure><img src="image_4.png" /><figcaption>Figure 3: Kernel density estimation and frequencies over values. Distribution of the agreeing probabilities $\mathbb{P}[g_{U,i}g_{C,i} > 0 | \hat{g}_U, \hat{g}_C]$ for all the parameters of a VGG16 during an unlearning epoch step. Dotted vertical line at $p = 1/2$.</figcaption></figure>
## 5.2 New Masking Strategy

In this section, the update is applied on the empirical gradients, but leveraging on Prop. 4 for designing new masks dedicated to batch processing. A first idea is as follows:

$$ \Delta_{\text{PROB}} \doteq -m_{\text{PROB}} \odot \text{Agg}(\hat{g}_U, \hat{g}_C) \quad \text{with} \quad (19) $$

$$ m_{\text{PROB}}(\hat{g}_U, \hat{g}_C)_i \doteq \begin{cases} 1 & \text{if } \mathbb{P}[g_{U,i}g_{C,i} > 0 | \hat{g}_U, \hat{g}_C] > p, \\ 0 & \text{otherwise.} \end{cases} \quad (20) $$

where $p \in (0, 1)$ is a level of certainty. In a nutshell, we modify a parameter if and only if we are certain that the update is locally feasible (see Fig. 2).

The novelty over saliency masking is threefold. First, SalUn selects parameters where the gradient ascent is confident but implicitly wrongly assumes that all parameter variances are equal. By introducing variance, we can achieve more precision. Second, we replace the arbitrary saliency threshold parameter with an interpretable quantile. Third but not least, SalUn requires the computation of a third gradient to compute the masking while our techniques recycle the already computed gradients.

**Proposition 5 (Properties of $m_{\text{PROB}}$).** The mask $m_{\text{PROB}}$ (20) equals the mask $m_{\text{AND}}$ (11) applied on the empirical gradients at a given batch in the following cases:

1. The level of certainty is set to $p = 1/2$,

2. All variances $\Sigma_{U,(i,i)}$ and $\Sigma_{C,(i,i)}$ tend to 0 and $p \in (0, 1)$.

## 5.3 The Focus Vector

The level of certainty $p$ is a hyperparameter whose setting is ad hoc. An alternative is a probabilistic mask where each component is randomly sampled according to a Bernoulli distribution $\mathcal{B}$:

$$ \Delta_{\text{BER}} \doteq -m_{\text{BER}} \odot \text{Agg}(\hat{g}_U, \hat{g}_C) \quad \text{with} \quad (21) $$

$$ m_{\text{BER},i} \sim \mathcal{B}(f_i) \quad \text{where } f_i \doteq \mathbb{P}[g_{U,i}g_{C,i} > 0 | \hat{g}_U, \hat{g}_C]. \quad (22) $$

This probabilistic selection leads, in expectation over $(\hat{g}_U, \hat{g}_C)$, to a weighting of the update acting as a focus on the parameters more likely to contribute to solving the optimization problem (5):

$$ \begin{aligned} \mathbb{E}[\Delta_{\text{BER}} | \hat{g}_U, \hat{g}_C] &= -\mathbb{E}[m_{\text{BER}} \odot \text{Agg}(\hat{g}_U, \hat{g}_C) | \hat{g}_U, \hat{g}_C] \\ &= -f \odot \text{Agg}(\hat{g}_U, \hat{g}_C), \end{aligned} \quad (23) $$

with $f = (f_i)_i$ the focus vector. Again, by the proof of Prop. 5 Case 2, this tends almost surely, when the variance goes to zero, to the update (12) applied to the batch gradients. By continuity for low standard deviations, one understands that this update is a good direction for decreasing both $U$ and $C$.

It explains our final design for the update:

$$ \Delta_{\text{F}} \doteq -f \odot \text{Agg}(\hat{g}_U, \hat{g}_C). \quad (24) $$

Knowing that the scalar product of two random vectors concentrates around zero in high-dimensional space [6], we state a feasible theoretical guarantee for this last version of the update.
<figure><img src="image_5.png" /><figcaption>Figure 4: Heatmap of the probability $P[g_{U,i}g_{C,i} > 0 | \hat{g}_U, \hat{g}_C]$ with respect to coordinates $(\phi_{U,i}, \phi_{C,i}) \in (0, 1)^2$. Hatched areas correspond to a probability lower than $p$, provoking a masking by (20) - $p = 0.45$ in this example.</figcaption></figure>

**Proposition 6 (Focus update's Theoretical Guarantee).** Let the aggregation Agg be the linear positive combination (13) of the gradients with $\alpha, \beta > 0$. Suppose that the measured gradients $\hat{g}_U$ and $\hat{g}_C$ are in expectation scalarly independent: $E[\langle \hat{g}_U, \hat{g}_C \rangle] = 0$. Then, the update $\Delta_F = -f \odot (\alpha \hat{g}_U + \beta \hat{g}_C)$ is feasible in expectation, and for any sufficiently small learning rate $\eta > 0$:

$$E[U(\theta_0 + \eta \Delta_F(\theta_0))] \leq U(\theta_0) \qquad (25)$$

$$E[C(\theta_0 + \eta \Delta_F(\theta_0))] \leq 0. \qquad (26)$$

We verified experimentally the hypothesis $E[\langle \hat{g}_U, \hat{g}_C \rangle] = 0$. The empirical values of $\langle \hat{g}_U, \hat{g}_C \rangle$ are initially about $\pm 10^{-5}$ and continue decreasing down to $\pm 10^{-10}$ during the unlearning procedure.

**Proposition 7 (Properties of the Focus Vector $f$).** The term-by-term multiplication by the focus vector $f$ corresponds to

1. the mask $m_{AND}$ (11) applied on the empirical gradients when the variances $\Sigma_{U,(i,i)}$ and $\Sigma_{C,(i,i)}$ tend to 0, $\forall i$,

2. halving the learning rate when the variances $\Sigma_{U,(i,i)}$ and $\Sigma_{C,(i,i)}$ tend to $+\infty$, $\forall i$.

Prop. 7 shows that the focus vector add-on is a soft trade-off between the AND mask when the gradient estimation noise is small and a slowdown of the learning rate when the variance is too high.
**Algorithm 1 FOCUS VECTOR FOR UNLEARNING**

**Require:** Objective and constraint dataset and loss $\mathcal{D}_U$, $\mathcal{D}_C$ and $L_U$, $L_C$; batch size $B$; aggregation function Agg; initial parameters $\theta_u \leftarrow \theta_0$; learning rate $\eta$; stability term $\epsilon$;

**Ensure:** Updated model parameters $\theta_u$ after unlearning

1: **for** $e = 1, 2, \dots$ **do** ▷ Epoch loop

2: **for** $b = 1, 2, \dots$ **do** ▷ Batch loop

3: Sample batches

4: $(x_i^U, y_i^U)^B_{i=1} \sim \mathcal{D}_U, (x_i^C, y_i^C)^B_{i=1} \sim \mathcal{D}_C$

5: Compute gradients

6: $\hat{g}_U \leftarrow \frac{1}{B} \sum_{i=1}^B \nabla L_U(M_{\theta_u}(x_i^U), y_i^U)$

7: $\hat{g}_C \leftarrow \frac{1}{B} \sum_{i=1}^B \nabla L_C(M_{\theta_u}(x_i^C), y_i^C)$

8: Retrieve second-order statistics

9: $\Sigma_U \leftarrow \text{AdamMoments}(L_U)$

10: $\Sigma_C \leftarrow \text{AdamMoments}(L_C)$

11: Compute per-parameter focus

12: $\phi_U \leftarrow \Phi(\hat{g}_U / \sqrt{\text{diag}(\Sigma_U) + \epsilon})$

13: $\phi_C \leftarrow \Phi(\hat{g}_C / \sqrt{\text{diag}(\Sigma_C) + \epsilon})$

14: $f \leftarrow \phi_U \odot \phi_C + (1 - \phi_U) \odot (1 - \phi_C)$

15: Compute update direction

16: $\Delta_F \leftarrow -f \odot \text{Agg}(\hat{g}_U, \hat{g}_C)$

17: Update parameters

18: $\theta_u \leftarrow \text{Optimize}(\theta_u, \Delta_F, \eta)$

19: end for

20: end for
return $\theta_u$

# 6 Experiments

## 6.1 Experimental Setup

All the experiments code is available at https://github.com/owl1996/UnlearningFocusVector and Sect. 6.3 only presents snapshots of this extensive experimental body.

**Architectures and Datasets.** Our experimental protocol focuses on image classification tasks with the datasets CIFAR-10 [23] and SVHN [28], and the deep neural network architectures ResNet18 model [16] and VGG16 [34].

**Unlearning Scenarios.** The protocol considers two scenarios about the dataset $\mathcal{D}_{\text{forget}}$ to be forgotten:

* $\mathcal{D}_{\text{forget}}$ is a random fraction of the whole training data,

* $\mathcal{D}_{\text{forget}}$ is a random fraction of a specific class.

For the first scenario, random forgetting, $\mathcal{D}_{\text{forget}}$ represents 5%, 10%, or 50% of the entire training data. For the second scenario, in class forgetting, $\mathcal{D}_{\text{forget}}$ represents 10%, 40%, or 75% of a class data.
<table><caption>Table 1: Comparison of mask-based add-ons after 10 epochs of 10% random data forgetting. Upper part: SRL - Cifar10 - VGG16. Bottom part: NGPlus - SVHN - ResNet18.</caption><thead><tr><th>Methods</th><th>MIA entropy</th><th>rUA</th><th>TA</th><th>RA</th><th>UA</th><th>FID</th></tr></thead><tbody><tr><td>SRL</td><td>0.89 ± 0.01</td><td>4.62 ± 0.87</td><td>92.77 ± 0.12</td><td>99.24 ± 0.16</td><td>97.78 ± 0.39</td><td>93.21 ± 0.40</td></tr><tr><td>SRL - SalUn</td><td>0.89 ± 0.01</td><td>4.70 ± 0.78</td><td>92.80 ± 0.14</td><td>99.25 ± 0.19</td><td>97.85 ± 0.32</td><td>93.36 ± 0.41</td></tr><tr><td>SRL - AND</td><td><strong>0.69 ± 0.01</strong></td><td>2.30 ± 0.47</td><td>92.18 ± 0.08</td><td>98.76 ± 0.21</td><td>95.45 ± 0.52</td><td>91.81 ± 0.41</td></tr><tr><td>SRL - PROB</td><td><strong>0.76 ± 0.02</strong></td><td><strong>0.45 ± 0.36</strong></td><td>91.69 ± 0.16</td><td>98.71 ± 0.22</td><td>93.61 ± 0.74</td><td>90.74 ± 0.47</td></tr><tr><td><strong>SRL - F</strong></td><td><strong>0.77 ± 0.03</strong></td><td><strong>0.41 ± 0.50</strong></td><td>91.71 ± 0.15</td><td>98.75 ± 0.23</td><td>93.57 ± 0.34</td><td>90.60 ± 0.14</td></tr><tr><td>NGPlus</td><td>0.94 ± 0.01</td><td>3.13 ± 0.76</td><td>94.75 ± 0.54</td><td>99.74 ± 0.06</td><td>97.98 ± 0.77</td><td>94.24 ± 0.74</td></tr><tr><td>NGPlus - SalUn</td><td>0.93 ± 0.02</td><td>2.67 ± 1.37</td><td>94.53 ± 0.91</td><td>99.73 ± 0.01</td><td>97.53 ± 1.42</td><td>93.90 ± 1.20</td></tr><tr><td>NGPlus - AND</td><td><strong>0.54 ± 0.02</strong></td><td>-46.14 ± 7.45</td><td>75.99 ± 3.39</td><td>88.57 ± 2.96</td><td>48.67 ± 7.72</td><td>47.67 ± 7.91</td></tr><tr><td>NGPlus - PROB</td><td><strong>0.88 ± 0.03</strong></td><td><strong>0.43 ± 1.34</strong></td><td>93.18 ± 1.29</td><td>99.53 ± 0.11</td><td>95.24 ± 1.55</td><td>92.24 ± 1.75</td></tr><tr><td><strong>NGPlus - F</strong></td><td><strong>0.89 ± 0.02</strong></td><td><strong>0.14 ± 0.80</strong></td><td>93.42 ± 0.74</td><td>99.56 ± 0.04</td><td>94.99 ± 0.94</td><td>92.25 ± 1.06</td></tr></tbody></table>

**Unlearning Methods** Saliency techniques and our updating procedures AND (12), PROB (19), and F (24) are coined *add-ons* because they are generics and they can be applied to different unlearning methods (defined by their losses, constraints, and datasets) introduced in Sect. 4.1. Our experimental protocol considers SRL, NGPlus, and SCRUB.

**Benchmark** For each method, a baseline is the ideal model trained from scratch in the same way as the initial model but without the forget set $D_{forget}$. We consider the baselines Mask-Small-Gradients (MSG) [25] and Convolution-Transpose (CT) [20], in addition to the following cited in Sect. 2.1: Fine-Tuning (FT) [40], SalUn [8] and SCRUB [24]. Of note, MSG and CT participated in the NeurIPS'2023 Machine Unlearning competition [36]. Moreover, FT, MSG and CT are evaluated as the strongest baselines in [3] due to their consistent performances across different datasets. In particular, MSG and CT show robustness against powerful per-sample MIAs like U-LiRA. SalUn is similar to our approach as it masks the least salient weights of the gradient ascent over the forget set. It follows the same spirit of our add-ons and is thus a competitor.

**Hyper Parameters** The batch size is 256 for any dataloader, both for training and unlearning. We train the initial and the exact ideal models for 100 epochs with an initial learning rate set to $10^{-1}$. For unlearning, the initial learning rate $\eta$ equals $10^{-4}$. The linear combination (13) takes $\alpha = 0.05$ for the gradient of loss $L_U$ and $\beta = 0.95$ for the gradient of loss $L_C$. By default, the PROB add-on sets the quantile parameter $p = 0.3$. SalUn [8] uses a median absolute value mask; for SCRUB, we let the additional loss with ponderation $\gamma = 1$ (see Sect. 4.1).

**Multiple Runs** For each given setting, the initial model, the unlearning procedure, and the exact ideal model are computed several times with different random seeds to measure means and standard deviations reported in the figures and tables.

## 6.2 Evaluation Metrics.

Our evaluation essentially follows the criteria outlined in [19] as discussed in Sect. 3.3:

* RTE: Run Time Efficiency (in seconds),

* MIA: Membership Inference Attack,
* TA: Test Accuracy,

* RA: Retain Accuracy,

* UA: Unlearning Accuracy.

Our protocol train five different support vector classifiers as decision rules b for the MIA (4), each exploiting different data [18, 43, 26, 1, 7]. The different training features used for the MIA are as follows: correctness (binary variable indicating whether the model well classifies the input data), confidence (probability score of the assigned class), logits (the logits output by the model), entropy (the entropy of the output of the model), mix entropy (the mix entropy of the logits). For simplicity, we do not show all five MIA metrics, but report the performance of the ones that achieve the best average score across comparisons: entropy and mix entropy.

We introduce two other metrics for the unlearning evaluation. Most works consider that a low the UA is a good indicator but it forces the unlearned model to misclassify the data to be forgotten. The problem is that these data are regular testing data for an ideal model $M_{\theta^*}$ which may still classify them correctly. We define the Relative Unlearning Accuracy (rUA) as:

$$ \text{rUA} \doteq \mathbb{P}_{(x,y)\sim\mathcal{D}_{\text{forget}}} [M(x) = y] - \mathbb{P}_{(x,y)\sim\mathcal{D}_{\text{forget}}} [M_{\theta^*}(x) = y]. \quad (27) $$

A good forgetting of the data expects rUA to be close to zero. When it is strongly positive, the unlearned model still classifies the data to forget too well, while the ideal model may have a lower accuracy over these data. When it is strongly negative, the procedure is too brutal, provoking catastrophic forgetting of the model. We define the Fidelity score (FID) (a.k.a. consistency [41]):

$$ \text{FID} \doteq \mathbb{P}_{(x,y)\sim\mathcal{D}_{\text{forget}}} [M(x) = M_{\theta^*}(x)]. \quad (28) $$

It measures how well the unlearned model mimics the behavior of the ideal model. When the ideal model fails because too much data was forgotten, entailing a loss of the model's utility, this metric gauges whether the unlearned model makes the same mistakes.

These new metrics are applicable only to laboratory experiments, in the sense that they cannot be directly observed in practice, as the ideal model is never fully realized. Yet, they offer a better understanding of what is expected from an unlearning.

## 6.3 Results

According to our study, when keeping the unlearning method and the scenario fixed (class-wise or random), the change of data or model architecture has no impact: the order of the add-ons techniques across all the metrics is conserved. Then, to avoid redundancy and excessive figures, we only report the non-intersecting part of the configurations to exhibit the different regimes of unlearning on which we test our add-ons.

### 6.3.1 Comparison of Masks

Table 1 reports the metrics of mask-based add-ons for two unlearning methods, SRL and NGPlus. It includes no masking (baseline version of SRL and NGPlus), SalUn [8], and our variants AND (12), PROB (19), and F (24). Our variants bring better MIA and rUA metrics for a moderate price to pay in terms of TA, RA, and FID. One exception is NGPlus-AND, which does not perform well. It illustrates the drawback of this add-on when applied to batch processing: an agreement on the sign of the noisy batch gradient components may not lead to such an agreement on the true gradient.
<figure><img src="image_6.png" /><figcaption>Figure 5: Comparison of add-ons (SalUn, AND, PROB and F) over SRL method. Evolution of the metrics over the number of epochs. Cifar10 – VGG16 – 50% Random forgetting</figcaption></figure>

Hence, the mask AND may select parameters incompatible with a locally feasible update. The add-ons PROB and F correct this erroneous behavior.

This phenomenon is also visible in Fig. 5 comparing the add-ons over SRL. The simple AND add-on is more unstable, while F and PROB add-ons are a good trade-off between efficiency and efficacy. In each case, the three add-ons we introduce are those that lower the MIA and absolute rUA with few steps of unlearning.

As for RTE, our add-on is ≈ 22% slower than the baseline method it is applied to. Of note, SalUn [8] masks the parameters with the least salient weights of the gradient ascent over the forget set. It requires the computation of a third gradient. This is already available in the NGPlus method, so for this particular trivial method SalUn is as fast as our add-on. Yet, it is not the case in SRL and SCRUB methods, for which SalUn is ≈ 55% slower than the baseline.

### 6.3.2 Generalization w.r.t. the Unlearning Methods

Fig. 6 shows the main benefit of the focus vector add-on F (24): The rUA goes more rapidly down to zero, no matter the method we plug it into (NGPlus, SRL, or SCRUB). As for the MIA, the focus vector add-on is always under the initial method score curve. The price to pay is moderate: a slight loss on the TA and a small increase on the RTE.

### 6.3.3 Differences between SRL, SCRUB, and NGPlus

Across the multiple settings of our experimental benchmark, we notice that NGPlus may have a behavior different than SRL and SCRUB. Fig. 7 shows a class-wise forgetting setup where all add-ons lead to similar results. One or two steps of unlearning is sufficient here and indeed recommended to avoid destroying the accuracy of the specific class due to the gradient ascent, which results in stronger MIA.
<figure><img src="image_7.png" /><figcaption>Figure 6: Benefit of the focus vector add-on F (24). Evolution of the metrics over the number of epochs for three methods (SRL, NGPlus, and SCRUB). Cifar10 - VGG16 - 10% Class-wise forgetting</figcaption></figure>

### 6.3.4 Benchmark

Table 2 overall shows that our add-ons are competitive in unlearning efficacy compared to the baselines. The MIA performs poorly over the ideal model, but we are interested in whether the add-on lowers the MIA's performance. Indeed, our add-ons always lower the initial unlearning method's score. MSG and CT look resilient to MIA at early epochs because they consist in initially jeopardized a portion of the model parameters.

Yet, this initial modification also pulls down the other metrics rUA and TA as shown in Fig. 8 and 9. Those figures show the evolution of the metrics along the epochs, enabling us to compare the efficacy and efficiency of the methods. The remaining epochs try to recover this parameter randomization in CT and MSG. This slowly improves rUA and TA but also favors the MIA. On the contrary, our solutions converge faster to a lower rUA. Also, the MIA is less efficient. We noticed that MSG needs a more careful setting of the threshold across the different architectures than the 30% proposed by the authors. Our solutions are always faster at decreasing the absolute value of rUA while stabilizing the test accuracy right from the first epochs.

To summarize, the relative unlearning accuracy metric quickly reaches zero with our solutions. The competitors need about twice as much time to reach this level of rUA and comparable accuracies. It shows that, in practice, our solutions improve efficiency.

## 7 Discussion and Limitations

Our experimental protocol considers 360 configurations: 5 add-on possibilities (none, salient, AND, PROB and focus F) over 3 unlearning methods, 2 model architectures, 2 datasets, and 2 types of forgetting scenarios with 3 ratios. In general, our experimental results, not presented here due to a lack of space, confirm our observations over the architectures, the types of forgetting scenarios, and the datasets. The ratio has an impact on the number of epochs. As for the methods, NGPlus
<figure><img src="image_8.png" /><figcaption>Figure 7: Comparison of add-ons (SalUn, AND, PROB and F) over NGPlus method. Evolution of the metrics over the number of epochs. NGPlus - SVHN - ResNet18 - 10% Class-wise forgetting</figcaption></figure>

is dominated by SRL or SCRUB. NGPlus forces an augmentation of the gradient over the forget set $D_{forget}$, which destroys the model capabilities along the epochs. A big loss on the forget set means that the model has to mostly equivocate on data that must be treated as such as if they belong to the test set. This deludes basic membership inference attacks, showing they are not powerful enough. When the model systematically gives a wrong answer on some data, unlearning is detectable. We recommend using NGPlus during one or two epochs only because the gradient ascent leads to unstable behavior of the resulting model.

The saliency masking of SalUn is justified in [8] by the argument of sparsity but does not take into account the variances while computing the mask. Our paper gives two other justifications: feasible update and the noisy estimation of the gradients when processing by batch. By combining these two arguments, we promote the idea that masking should prevent hazardous update directions. Our focus vector is based on a simple statistical model that enable to derive the probability of a right parameter selection. Finally, we show that the focus vector improves the performance of unlearning algorithms both in efficiency and efficacy. It requires fewer steps to reach an acceptable rUA while conserving the accuracies and lowering the score of population-based MIA.

The theoretical analysis assumes that the measured gradients on batches are a noisy representation of a true gradient with a simple Gaussian model. Measured gradients certainly follow a more complex distribution. However, the experiments already show the benefit of this simple assumption. The last limitation of our work concerns the MIA metric. There exist more powerful membership inference attacks specifically in unlearning, such as U-LiRa, but they require an unreasonable amount of computation power. We chose to verify that the unlearned model lowers the performance of some basic MIAs compared to the initial model. We expect that this also holds for U-LiRa knowing that basic attacks generally follow the same tendencies as U-LiRa as shown in [15].
<table><caption>Table 2: Benchmark - Epoch 10 - Cifar10 - VGG16 - 10% Random Forgetting</caption><thead><tr><th>Methods</th><th>MIA entropy</th><th>rUA</th><th>TA</th><th>RA</th><th>UA</th><th>FID</th></tr></thead><tbody><tr><td>Initial</td><td>0.91 ± 0.01</td><td>5.82 ± 0.84</td><td>92.66 ± 0.10</td><td>98.63 ± 0.27</td><td>98.73 ± 0.27</td><td>93.00 ± 0.56</td></tr><tr><td>Ideal</td><td>0.84 ± 0.01</td><td><b>0.00 ± 0.00</b></td><td>92.53 ± 0.16</td><td>99.56 ± 0.05</td><td>92.91 ± 0.71</td><td>100.0 ± 0.00</td></tr><tr><td>FT</td><td>0.91 ± 0.01</td><td>5.15 ± 0.88</td><td>92.75 ± 0.15</td><td>99.18 ± 0.14</td><td>98.3 ± 0.39</td><td>93.44 ± 0.52</td></tr><tr><td>MSG</td><td><b>0.76 ± 0.02</b></td><td>-3.20 ± 1.13</td><td>86.65 ± 0.56</td><td>93.07 ± 0.84</td><td>89.96 ± 0.66</td><td>88.42 ± 0.69</td></tr><tr><td>CT</td><td><b>0.65 ± 0.08</b></td><td>-16.01 ± 0.37</td><td>76.87 ± 0.33</td><td>80.45 ± 0.27</td><td>77.14 ± 0.37</td><td>77.51 ± 0.53</td></tr><tr><td>SRL</td><td>0.89 ± 0.01</td><td>4.62 ± 0.87</td><td>92.77 ± 0.12</td><td>99.24 ± 0.16</td><td>97.78 ± 0.39</td><td>93.21 ± 0.40</td></tr><tr><td>SalUn</td><td>0.89 ± 0.01</td><td>4.70 ± 0.78</td><td>92.80 ± 0.14</td><td>99.25 ± 0.19</td><td>97.85 ± 0.32</td><td>93.36 ± 0.41</td></tr><tr><td>NGPlus</td><td>0.91 ± 0.01</td><td>3.34 ± 0.53</td><td>92.71 ± 0.05</td><td>99.17 ± 0.13</td><td>96.49 ± 0.24</td><td>92.85 ± 0.52</td></tr><tr><td>SCRUB</td><td>0.91 ± 0.01</td><td>5.60 ± 0.74</td><td>92.90 ± 0.15</td><td>99.18 ± 0.17</td><td>98.5 ± 0.18</td><td>93.18 ± 0.63</td></tr><tr><td>SCRUB - SalUn</td><td>0.90 ± 0.01</td><td>5.62 ± 0.74</td><td>92.92 ± 0.04</td><td>99.12 ± 0.18</td><td>98.53 ± 0.2</td><td>93.13 ± 0.66</td></tr><tr><td>SCRUB - AND</td><td>1.00 ± 0.00</td><td>4.30 ± 0.66</td><td>92.24 ± 0.19</td><td>98.83 ± 0.23</td><td>97.21 ± 0.31</td><td>92.57 ± 0.78</td></tr><tr><td><b>SCRUB - F</b></td><td>0.90 ± 0.02</td><td>5.21 ± 0.82</td><td>92.66 ± 0.17</td><td>99.09 ± 0.18</td><td>98.12 ± 0.23</td><td>93.04 ± 0.65</td></tr><tr><td>SRL - AND</td><td><b>0.69 ± 0.01</b></td><td>2.30 ± 0.47</td><td>92.18 ± 0.08</td><td>98.76 ± 0.21</td><td>95.45 ± 0.52</td><td>91.81 ± 0.41</td></tr><tr><td>SRL - PROB</td><td><b>0.76 ± 0.02</b></td><td><b>0.45 ± 0.36</b></td><td>91.69 ± 0.16</td><td>98.71 ± 0.22</td><td>93.61 ± 0.74</td><td>90.74 ± 0.47</td></tr><tr><td><b>SRL - F</b></td><td><b>0.77 ± 0.03</b></td><td><b>0.41 ± 0.50</b></td><td>91.71 ± 0.15</td><td>98.75 ± 0.23</td><td>93.57 ± 0.34</td><td>90.60 ± 0.14</td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>Figure 8: Comparison of baselines with add-on F over SRL. Evolution of the metrics over the number of epochs. Cifar10 – VGG16 – 10% Random Forgetting</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>Figure 9: Comparison of baselines with add-on F over NGPlus. Evolution of the metrics over the number of epochs. SVHN – ResNet18 – 10% Random forgetting</figcaption></figure>
References

[1] Martin Bertran, Shuai Tang, Aaron Roth, Michael Kearns, Jamie H Morgenstern, and Steven Z Wu. Scalable membership inference attacks via quantile regression. *Advances in Neural Information Processing Systems (NeurIPS)*, 36:314–330, 2023.

[2] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In *IEEE symposium on Security and Privacy (SP)*, pages 141–159, 2021.

[3] Xavier F Cadet, Anastasia Borovykh, Mohammad Malekzadeh, Sara Ahmadi-Abhari, and Hamed Haddadi. Deep unlearn: Benchmarking machine unlearning. *arXiv preprint arXiv:2410.01276*, 2024.

[4] Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In *2022 IEEE symposium on security and privacy (SP)*. IEEE, 2022.

[5] Chong Chen, Fei Sun, Min Zhang, and Bolin Ding. Recommendation unlearning. In *Proceedings of the ACM web conference 2022*, pages 2768–2777, 2022.

[6] Eungchun Cho. Inner product of random vectors. *International Journal of Pure and Applied Mathematics*, 56(2):217–221, 2009.

[7] Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label- only membership inference attacks. In *International Conference on Machine Learning*, pages 1964–1974. PMLR, 2021.

[8] Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. In *The Twelfth International Conference on Learning Representations*, 2024.

[9] Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George Pappas. Efficient and accurate estimation of lipschitz constants for deep neural networks. *Advances in Neural Information Processing Systems (NeurIPS)*, 32, 2019.

[10] Sanjam Garg, Shafi Goldwasser, and Prashant Nalini Vasudevan. Formalizing data deletion in the context of the right to be forgotten. In *Annual International Conference on the Theory and Applications of Cryptographic Techniques*. Springer, 2020.

[11] Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, and Stefano Soatto. Mixed-privacy forgetting in deep networks. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 792–801, 2021.

[12] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, 2020.

[13] Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 35, 2021.

[14] Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. In *Proceedings of the 37th International Conference on Machine Learning*, ICML’20, 2020.
[15] Jamie Hayes, Ilia Shumailov, Eleni Triantafillou, Amr Khalifa, and Nicolas Papernot. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. In *Conf. on Secure and Trustworthy Machine Learning (SaTML)*, 2025.

[16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In *Proceedings of the IEEE conference on computer vision and pattern recognition*, pages 770–778, 2016.

[17] Chris Jay Hoofnagle, Bart van der Sloot, and Frederik J. Zuiderveen Borgesius. The european union general data protection regulation: what it is and what it means*. *Information & Communications Technology Law*, 28:65 – 98, 2019.

[18] Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. Membership inference attacks on machine learning: A survey. *ACM Computing Surveys (CSUR)*, 54(11s):1–37, 2022.

[19] Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsity can simplify machine unlearning. *Advances in Neural Information Processing Systems*, 36, 2023.

[20] Toshi K. and Marvel World. Neurips 2023 machine unlearning discussion, 2023.

[21] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. *arXiv preprint arXiv:1412.6980*, 2014.

[22] Masanori Koyama and Shoichiro Yamaguchi. Out-of-distribution generalization with maximal invariant predictor. 2020.

[23] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

[24] Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. *Advances in Neural Information Processing Systems (NeurIPS)*, 36, 2023.

[25] Doun Lee. Neurips 2023 machine unlearning discussion, 2023. Accessed on January 31, 2024.

[26] Shuhao Li, Yajie Wang, Yuanzhang Li, and Yu-an Tan. l-leaks: Membership inference attacks with logs. *arXiv preprint arXiv:2205.06469*, 2022.

[27] Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. *Advances in Neural Information Processing Systems (NeurIPS)*, 34:18878–18890, 2021.

[28] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In *NIPS workshop on deep learning and unsupervised feature learning*, volume 2011, page 4. Granada, 2011.

[29] House of Commons of Canada. Consumer privacy protection act.

[30] Giambattista Parascandolo, Alexander Neitz, Antonio Orvieto, Luigi Gresele, and Bernhard Schölkopf. Learning explanations that are hard to vary. In *International Conference on Learning Representations*, 2021.
[31] Soroosh Shahtalebi, Jean-Christophe Gagnon-Audet, Touraj Laleh, Mojtaba Faramarzi, Kartik Ahuja, and Irina Rish. Sand-mask: An enhanced gradient masking strategy for the discovery of invariances in domain generalization. CoRR, abs/2106.02266, 2021.

[32] Thanveer Shaik, Xiaohui Tao, Haoran Xie, Lin Li, Xiaofeng Zhu, and Qing Li. Exploring the landscape of machine unlearning: A comprehensive survey and taxonomy. IEEE Transactions on Neural Networks and Learning Systems, pages 1–21, 2024.

[33] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP). IEEE, 2017.

[34] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations, ICLR, 2015.

[35] Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Understanding factors influencing machine unlearning. 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pages 303–319, 2021.

[36] Eleni Triantafillou, Fabian Pedregosa, Jamie Hayes, Peter Kairouz, Isabelle Guyon, Meghdad Kurmanji, Gintare Karolina Dziugaite, Peter Triantafillou, Kairan Zhao, Lisheng Sun Hosoya, Julio C. S. Jacques Junior, Vincent Dumoulin, Ioannis Mitliagkas, Sergio Escalera, Jun Wan, Sohier Dane, Maggie Demkin, and Walter Reade. Neurips 2023 machine unlearning challenge, 2023.

[37] Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018.

[38] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. 2019 IEEE Symposium on Security and Privacy (SP), pages 707–723, 2019.

[39] Weiqi Wang, Zhiyi Tian, Chenhan Zhang, and Shui Yu. Machine unlearning: A comprehensive survey. arXiv preprint arXiv:2405.07406, 2024.

[40] Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine unlearning of features and labels. ArXiv, abs/2108.11577, 2021.

[41] Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. Machine unlearning: A survey. ACM Comput. Surv., 56(1), August 2023.

[42] Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. Arcane: An efficient architecture for exact machine unlearning. In IJCAI, volume 6, page 19, 2022.

[43] Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268–282. IEEE, 2018.

[44] Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems (NeurIPS), 33:5824–5836, 2020.

[45] Shaobo Zhang, Yimeng Pan, Qin Liu, Zheng Yan, Kim-Kwang Raymond Choo, and Guojun Wang. Backdoor attacks and defenses targeting multi-domain ai models: A comprehensive review. ACM Computing Surveys, 57(4):1–35, 2024.
# Appendix: Proofs of the Propositions

## Proof of Proposition 1

*Proof.* The first-order Karush-Kuhn-Tucker condition provides a constraint on a candidate solution $\bar{\theta}$: there exist $\mu \ge 0$ s. t.

$$\nabla U(\bar{\theta}) + \mu \cdot \nabla C(\bar{\theta}) = 0.$$

Then, by multiplying component-wise by $\nabla C(\bar{\theta})$:

$$\nabla U(\bar{\theta}) \odot \nabla C(\bar{\theta}) + \mu \cdot \nabla C(\bar{\theta}) \odot \nabla C(\bar{\theta}) = 0.$$

We have $(\nabla C(\bar{\theta}) \odot \nabla C(\bar{\theta}))_i = (\partial/\partial_{\theta_i} C(\bar{\theta}))^2 \ge 0$, so that

$$\nabla U(\bar{\theta}) \odot \nabla C(\bar{\theta}) = -\mu \cdot \nabla C(\bar{\theta}) \odot \nabla C(\bar{\theta}) \le 0.$$

□

## Proof of Proposition 2

*Proof.* Knowing that $C(\theta_0) = 0$ by definition (7) and since $\langle \Delta, \nabla U(\theta_0) \rangle < 0$ and $\langle \Delta, \nabla C(\theta_0) \rangle < 0$ because the update is feasible, then, up to the first order in $\eta$, both functions $U$ and $C$ decreases along the direction $\Delta$.
□

## Proof of Proposition 3

*Proof.* The norm $\|m(\nabla U, \nabla C)(\theta_0)\|_0$ is the number of non-zero components, each of them being lower in absolute value to $\|\Delta(\theta_0)\|_{\infty}$:

$$\|\theta - \theta_0\|_q = \|\eta\Delta(\theta_0)\|_q = \eta \left( \sum_i |\Delta(\theta_0)_i|^q \right)^{1/q} \\ \le \eta \|m(\nabla U, \nabla C)(\theta_0)\|_0^{1/q} \|\Delta(\theta_0)\|_{\infty}.$$

If $\text{Agg}(x, y) = \underset{z \in \{x, y\}}{\operatorname{argmin}} |z|$, one has both $\text{Agg}(x, y) \le |x|$ and $\text{Agg}(x, y) \le |y|$ so that $\|\Delta(\theta_0)\|_{\infty} \le \|\text{Agg}(\nabla U, \nabla C)(\theta_0)\|_{\infty} \le \|\nabla C(\theta_0)\|_{\infty}$. □

## Proof of Proposition 4

*Proof.* Knowing that the $i$-th component of the empirical gradient over a given batch equals $\hat{g}_{U,i}$, we have

$$\phi_{U,i} \doteq \mathbb{P}[g_{U,i} > 0 | \hat{g}_{U,i}] = \mathbb{P}[N_{U,i} < \hat{g}_{U,i} | \hat{g}_{U,i}] = \Phi \left( \frac{\hat{g}_{U,i}}{\sqrt{\sum_{U,(i,i)}}} \right).$$

The same holds for $\phi_{C,i}$. The estimation noises being supposed independent, $g_{U,i}$ and $g_{C,i}$ are both positive with probability $\phi_{U,i}\phi_{C,i}$. Similarly, they are both negative with probability $(1 - \phi_{U,i})(1 - \phi_{C,i})$. □
## Proof of Proposition 5

*Proof.* Case 1: For $p = 1/2$, we have

$$ 
\begin{aligned}
& \phi_{U,i}\phi_{C,i} + (1 - \phi_{U,i})(1 - \phi_{C,i}) > \frac{1}{2} \\
\Leftrightarrow & 2\phi_{U,i} \left( \phi_{C,i} - \frac{1}{2} \right) > \phi_{C,i} - \frac{1}{2} \\
\Leftrightarrow & \left\{ \phi_{U,i} > \frac{1}{2} \cap \phi_{C,i} > \frac{1}{2} \right\} \cup \left\{ \phi_{U,i} < \frac{1}{2} \cap \phi_{C,i} < \frac{1}{2} \right\} \\
\Leftrightarrow & \{ \hat{g}_{U,i} > 0 \cap \hat{g}_{C,i} > 0 \} \cup \{ \hat{g}_{U,i} < 0 \cap \hat{g}_{C,i} < 0 \} \\
\Leftrightarrow & \hat{g}_{U,i}\hat{g}_{C,i} > 0.
\end{aligned}
$$

Case 2: The function $\Phi(x/\sigma)$ tends to an Heaviside step function when $\sigma \to 0$:

$$ \Phi(x/\sigma) \underset{\sigma \to 0}{\to} \begin{cases} 1 & \text{if } x > 0 \\ 0 & \text{if } x < 0 \end{cases} $$

Therefore

$$ \phi_{U,i}\phi_{C,i} + (1 - \phi_{U,i})(1 - \phi_{C,i}) \underset{\sigma \to 0}{\to} \begin{cases} 1 \cdot 1 + 0 \cdot 0 = 1, & \text{if } \hat{g}_{U,i} > 0 \text{ and } \hat{g}_{C,i} > 0 \\ 1 \cdot 0 + 0 \cdot 1 = 0, & \text{if } \hat{g}_{U,i} > 0 \text{ and } \hat{g}_{C,i} < 0 \\ 0 \cdot 1 + 1 \cdot 0 = 0, & \text{if } \hat{g}_{U,i} < 0 \text{ and } \hat{g}_{C,i} > 0 \\ 0 \cdot 0 + 1 \cdot 1 = 1, & \text{if } \hat{g}_{U,i} < 0 \text{ and } \hat{g}_{C,i} < 0. \end{cases} $$

□

## Proof of Proposition 6

*Proof.* For any positive $\alpha$ and $\beta$, we write

$$ 
\begin{aligned}
\mathbb{E}[\langle \Delta_F, g_C \rangle] &= -\alpha \mathbb{E}[\langle f \odot \hat{g}_U, g_C \rangle] - \beta \mathbb{E}[\langle f \odot \hat{g}_C, g_C \rangle] \\
&= -\alpha \mathbb{E} \left[ \sum_i f_i \hat{g}_{U,i} g_{C,i} \right] - \beta \mathbb{E} \left[ \sum_i f_i \hat{g}_{C,i} g_{C,i} \right].
\end{aligned}
$$

For the first term with $\alpha$, we condition on the observation $(\hat{g}_C, \hat{g}_U)$. From Prop. 5 Case 1, we know that $f_i > \frac{1}{2} \Leftrightarrow \hat{g}_{U,i}\hat{g}_{C,i} > 0$. We split the sum so that

$$ 
\begin{aligned}
\mathbb{E} \left[ \sum_i f_i \hat{g}_{U,i} g_{C,i} \middle| \hat{g}_{U,i}, \hat{g}_{C,i} \right] &= \sum_{i, f_i < \frac{1}{2}} f_i \hat{g}_{U,i} \hat{g}_{C,i} + \sum_{i, f_i > \frac{1}{2}} f_i \hat{g}_{U,i} \hat{g}_{C,i} \\
&= - \sum_{i, f_i < \frac{1}{2}} f_i |\hat{g}_{U,i} \hat{g}_{C,i}| + \sum_{i, f_i > \frac{1}{2}} f_i \hat{g}_{U,i} \hat{g}_{C,i} \\
&\geq -\frac{1}{2} \sum_{i, \hat{g}_{U,i} \hat{g}_{C,i} < 0} |\hat{g}_{U,i} \hat{g}_{C,i}| + \frac{1}{2} \sum_{i, \hat{g}_{U,i} \hat{g}_{C,i} > 0} \hat{g}_{U,i} \hat{g}_{C,i} \\
&= \frac{1}{2} \sum_i \hat{g}_{U,i} \hat{g}_{C,i} = \frac{1}{2} \langle \hat{g}_U, \hat{g}_C \rangle
\end{aligned}
$$
Then, in expectation over $(\hat{g}_U, \hat{g}_C)$:

$$ \mathbb{E}\left[\sum_i f_i \hat{g}_{U,i} g_{C,i}\right] = \mathbb{E}\left[\mathbb{E}\left[\sum_i f_i \hat{g}_{U,i} g_{C,i} \mid \hat{g}_{U,i}, \hat{g}_{C,i}\right]\right] \\ \geq \frac{1}{2} \mathbb{E}[\langle \hat{g}_U, \hat{g}_C \rangle] = 0. $$

The second term with $\beta$ is trivially positive:

$$ \mathbb{E}\left[\sum_i f_i \hat{g}_{C,i} g_{C,i}\right] = \mathbb{E}\left[\mathbb{E}\left[\sum_i f_i \hat{g}_{C,i} g_{C,i} \mid \hat{g}_{C,i}\right]\right] \\ = \sum_i \mathbb{E}[f_i \hat{g}_{C,i}^2] \geq 0. $$

This concludes that in expectation

$$ \mathbb{E}[\langle \Delta_F, g_C \rangle] \leq 0. $$

By the same token, interchanging the roles of $\alpha$ and $\beta$, we obtain $\mathbb{E}[\langle \Delta_F, g_U \rangle] \leq 0$ and the update $\Delta_F$ is feasible in expectation. $\square$

## Proof of Proposition 7

*Proof.* Case 1 is a corollary of Prop 5. For Case 2, we have $\Phi(x/\sigma) \xrightarrow{\sigma \to \infty}^{1/2}$, therefore

$$ \phi_{U,i}\phi_{C,i} + (1 - \phi_{U,i})(1 - \phi_{C,i}) \xrightarrow{\sigma \to \infty}^{1/2 \cdot 1/2} + (1 - 1/2)(1 - 1/2) = 1/2. $$

<figure><img src="image_11.png" /></figure>