Evaluating Deep Unlearning in Large Language Models

## Abstract

Abstract Machine unlearning is a key requirement of many data protection regulations such as GDPR. Prior work on unlearning has mostly considered superficial unlearning tasks where a single or a few related pieces of information are required to be removed.
However, the task of unlearning a fact is much more challenging in recent large language models (LLMs),
because the facts in LLMs can be deduced from each other.
In this work, we investigate whether current unlearning methods for LLMs succeed beyond superficial unlearning of facts.
Specifically, we formally propose a new setting of unlearning, deep unlearning , which considers the logical deductions between facts.
Moreover, we design the metric, recall , to quantify the extent of deep unlearning.
To systematically evaluate deep unlearning, we construct a synthetic dataset EDU-RELAT, which consists of a synthetic knowledge base of family relationships and biographies, together with a realistic logical rule set that connects them.
We use this dataset to test four unlearning methods in four LLMs at different sizes.
Our findings reveal that in the task of deep unlearning only a single fact, they either fail to properly unlearn with high recall, or end up unlearning many other irrelevant facts.
Our dataset and code are publicly available at
: https://github.com/wrh14/deep_unlearning .

## 1 Introduction

Large foundation models of today are trained on massive amounts of uncurated data obtained from the internet.
Machine unlearning has emerged as a solution for removing specific data, concepts, or facts from these models in a more efficient way than retraining from scratch.
It is practical in situations where regulations, such as the GDPR (Parliament & of the European Union, ), mandate the removal of a user’s data.
Moreover, unlearning is important when models inadvertently capture copyrighted(^1^11https://casetext.com/case/tremblay-v-openai-inc-2) or offensive content.
For large language models (LLM) in particular, the ability to unlearn specific facts becomes especially important to ensure compliance with copyright laws and privacy requirements (Jang et al., ; Liu et al., )

Prior work in unlearning in large language models have looked at this problem, but for the target fact, the focus has been on removing the target itself. However, this can be superficial – LLMs not only know single facts in isolation, but many connected facts – and very often, the fact that has been unlearnt can be deduced from facts that are already known by the model. Thus, successful unlearning in this setting should also remove other facts that imply the fact to be unlearnt. As a concrete example, consider Figure 1. Here, the target fact “Camila Flores’s child is Wyatt Ross” can be deduced from fact A “Wyatt Ross’s father is Xavier Ross” and fact B “Camila Flores’s husband is Xavier Ross”.
If the LLM only unlearns the target fact but retains A and B, this is insufficient as an adversary who extracts A and B from the LLM can deduce the target fact.

In this work, we consider a new setting for unlearning, which we call deep unlearning, and investigate to what extent current unlearning methods succeed at this setting. Deep unlearning is formulated by stating a set of facts and logical rules that connect the facts.
The fact is deeply unlearnt if the target fact cannot be deduced from the retained facts in the LLM through the given logical rules.
We further propose two metrics, recall and accuracy, for evaluating unlearning methods at deep unlearning.
Recall measures to how well an unlearning method unlearns the relevant facts so that the target fact cannot be deduced; while accuracy measures what extent other irrelevant facts are retained by the unlearning process.

In order to systematically evaluate deep unlearning, we construct a synthetic dataset EDU-RELAT as a benchmark.
The dataset consists of two parts: a synthetic knowledge base and a realistic logical rule set. The knowledge base contains biographical information about a group of people (e.g., “The birthyear of Sloane Lee is 1908”), as well as family relationships (e.g., “Wyatt Ross’s father is Xavier Ross”).
The logical rules are realistic rules describing family relationships (e.g. (X, husband, Z)$\wedge$ (Y, father, Z) $\to$ (X, child, Y)).
The unlearning method is tested on a subset of facts on family relationships together with the rules.

Figure: Figure 1: An example that unlearning only the target fact is insufficient. The successful extraction of “Wyatt Ross’s father is Xavier Ross” and “Camila Flores’s husband is Xavier Ross” can imply the target fact.
Refer to caption: x1.png

We then use our dataset to evaluate four common unlearning methods (Gradient Ascent, Negative Preference Optimization, Task Vector, and Who’s Harry Potter) on four popular LLMs (Phi-1.5, GPT2-XL, Llama2-7b, Llama3-8b).
We find that while these methods are good at unlearning the target fact itself without losing accuracy, they either fail to deeply unlearn with high recall or lose more than $20\%$ irrelevant facts while deeply unlearning just single target fact.
Additionally, it is found that the unlearning methods have better performance on larger LLMs; this validates the hypothesis that more inherent understanding of facts in larger LLMs helps with deep unlearning naturally.

This illustrates that the machine unlearning methods of today are largely insufficient for properly unlearning facts from LLMs;
either they do not unlearn what is needed to deduce a target fact, or they over-unlearn and unlearn many extraneous facts.
We hypothesize that this might be because the existing unlearning methods do not sufficiently account for the nature of facts and the reasoning capabilities of LLMs.
We posit that future methods that unlearn facts suitably from LLMs should be aware of these enhanced capabilities.

## 2 Preliminary

Given a set of objects $\mathcal{O}$ and relations $\mathcal{T}$, a fact $k$ is represened by the triplet $(o_{1},r,o_{2})$ of the relation $r$ and two objects $o_{1},o_{2}$.
For example, “Camila Flores’s child is Wyatt Ross” can be represented in $(\textit{Camila Flores},\textit{child},\textit{Wyatt Ross})$.
The knowledge base $\mathcal{K}\subseteq\mathcal{O}\times\mathcal{T}\times\mathcal{O}$ is a set of facts.

Facts can be deduced from each other through reasoning; an example has been shown in Figure 1.
A popular way to formulate the reasoning among facts is through a logical rule $R$, which is widely used to discover new knowledge (Galárraga et al., ; Yang et al., ; Xu et al., ; Cheng et al., ; Luo et al., ).
The rule $R$ has the form of $B_{1}\wedge\cdots\wedge B_{n}\to A$,
where $B_{1}\cdots,B_{n}$ and $A$ are atoms and each atom is a tuple $(X,r,Y)$ of logical variables $X,Y$ and a relation $r$.
One example of rule is (X, husband, Z)$\wedge$ (Y, father, Z) $\to$ (X, child, Y).
By substituting the objects in $\mathcal{O}$ to the logical variables in $B_{1},\cdots,B_{n},A$, facts on the left can together deduce the fact on the right.

With a set of rules $\mathcal{R}$, a knowledge base $\mathcal{K}$ is deductively closed (Cheney et al., ; Cohen, ; Huang et al., ) with respect to $\mathcal{R}$, if there is no new fact can be deduced from $\mathcal{K}$ and $\mathcal{R}$.
For example, suppose
$(\textit{Camila Flores},\textit{husband},\textit{Xavier Ross}),(\textit{Xavier
Ross},\textit{child},\textit{Wyatt Ross})\in\mathcal{K}$ and (X, husband, Z)$\wedge$ (Y, father, Z) $\to$ (X, child, Y) is in the rule set $\mathcal{R}$.
If $\mathcal{K}$ is deductively closed with respect to the rule set including the rule set $\mathcal{R}$,
$(\textit{Camila Flores},\textit{child},\textit{Wyatt Ross})$ must be in the knowledge base $\mathcal{K}$ too.
The deductive closure is defined as follows.

###### Definition 1 (Deductive closure) .

The deductive closure of $\mathcal{K}$ with respective to the rule set $\mathcal{R}$, notated as $\Omega(\mathcal{K},\mathcal{R})$, is the smallest set such that 1. $\mathcal{K}\subseteq\Omega(\mathcal{K},\mathcal{R})$; 2. $\Omega(\mathcal{K},\mathcal{R})$ is deductively closed with respect to a set of rules $\mathcal{R}$.

###### Definition 1 (Deductive closure) .

## 3 deep unlearning

Prior work in fact unlearning from LLMs focuses on simply unlearning the target fact in isolation, and not other facts that logically imply it.
This might cause the LLM to forget only this one specific fact, but retain others that can be combined to deduce the fact in question.
In this section, we introduce the new setting of unlearning, deep unlearning, which considers the logical deductions between facts.
To assess the effectiveness of unlearning methods in this setting, we propose two evaluation metrics: recall and accuracy.

### 3.1 Fact deep unlearning

Figure: Figure 2: An illustration of deep unlearning. (a) an example of superficial unlearning; (b) an example of deep unlearning; (c) two different minimal deep unlearning sets for unlearning the same target fact; (d) the calculation of our proposed evaluation metric recall.
Refer to caption: x2.png

Let $\mathcal{K}$ represent the knowledge base of the LLM prior to unlearning and let $U_{k}^{\mathcal{A}}\subseteq\mathcal{K}$ denote the set of facts that has been removed by any unlearning method $\mathcal{A}$ aimed at unlearning the target fact $k$.
If method $\mathcal{A}$ deeply unlearns the fact $k$, it is expected that the fact $k$ should not be deduced from the retained facts $\mathcal{K}\backslash U_{k}^{\mathcal{A}}$ by the rule set $\mathcal{R}$, i.e. $k$ should not be in the deductive closure $\Omega(\mathcal{K}\backslash U_{k}^{\mathcal{A}},\mathcal{R})$.

###### Definition 2 (Deep unlearning) .

The unlearning method $\mathcal{A}$ deeply unlearns the fact $k$ with respect to the rule set $\mathcal{R}$ if the fact $k$ does not belong in the deductive closure $\Omega(\mathcal{K}\backslash U_{k}^{\mathcal{A}},\mathcal{R})$.

We call the unlearning, which successfully unlearns the target fact but does not satisfy deep unlearning, as superficial unlearning; we show an example in Figure 2(a).
Figure 2(b) shows an example of deep unlearning; from the only retained fact $(\textit{Camila Flores},\textit{husband},\textit{Xavier Ross})$, the target fact cannot be deduced by any rules.
We further notice that in the example of deep unlearning (Figure 2(b)), even if $(\textit{Xavier Ross},\textit{wife},\textit{Camila Flores})$ is not unlearnt, it is still an example of deep unlearning.
In practice, we would prefer the LLM that deeply unlearns  the target fact but retains other facts as much as possible.
Therefore, We define what minimal deep unlearning is.

###### Definition 3 (Minimal deep unlearning) .

Given a fact $k$, the minimal deep unlearning  set $U_{k}^{*}$ to unlearn the fact $k$ with respect to the rule set $\mathcal{R}$ should meet two requirements: 1. $k\notin\Omega(\mathcal{K}\backslash U_{k}^{*},\mathcal{R})$, 2. $\forall U\subset U_{k}^{*}$, $k\in\Omega(\mathcal{K}\backslash U,\mathcal{R})$.
Moreover, the unlearning method minimally deeply unlearns  $k$ with respect to the rule set $\mathcal{R}$ if $U_{k}^{\mathcal{A}}$ is any minimal deep unlearning  set.

Note that minimal deep unlearning  set needs not be unique.
For example, Figure 2(c) shows two possible minimal deep unlearning  sets for unlearning the same target fact.

###### Definition 2 (Deep unlearning) .

###### Definition 3 (Minimal deep unlearning) .

### 3.2 Evaluation Metric

Figure: Algorithm 1 DUS$(k,\mathcal{K},\mathcal{R})$ – Random generation of the Deep Unlearning Set

Figure: Algorithm 2 RP$(k,\mathcal{K},\mathcal{R},U_{k})$ – Random Pruning the deep unlearning set

Figure: Algorithm 3 MDUS$(k,\mathcal{K},\mathcal{R};N_{\rm seed})$ – Generating multiple Minimal Deep Unlearning Sets

We propose two evaluation metrics to evaluate an unlearning method $\mathcal{A}$: recall and accuracy.
Recall is to measure the extent of deep unlearning  of an unlearning method $\mathcal{A}$.
It calculates the percentage of any minimal deep unlearning  set that has been unlearnt by the method $\mathcal{A}$.
Because the minimal deep unlearning  set is not unique, the recall is defined w.r.t. the minimal deep unlearning  set that $U_{k}^{\mathcal{A}}$ covers the most.
Formally, let $\mathcal{M}_{k,\mathcal{R},\mathcal{K}}$ denote the set of all minimal deep unlearning  sets to unlearn $k$ (from the knowledge base $\mathcal{K}$ with respective to the rule set $\mathcal{R}$).
The recall for a given unlearning method $\mathcal{A}$ to unlearn $k$ is defined as

$$ $\text{Recall}(\mathcal{A},k;\mathcal{K},\mathcal{R})=\max_{U_{k}^{*}\in \mathcal{M}_{k,\mathcal{R},\mathcal{K}}}\frac{|U_{k}^{\mathcal{A}}\cap U_{k}^{ *}|}{|U_{k}^{*}|}.$ (1) $$

We also denote $U_{k}^{\mathcal{A},*}:=\arg\max_{U_{k}^{*}\in\mathcal{M}_{k,\mathcal{R},
\mathcal{K}}}\frac{|U_{k}^{\mathcal{A}}\cap U_{k}^{*}|}{|U_{k}^{*}|}$ the minimal deep unlearning  set that $U_{k}^{\mathcal{A}}$ covers the most, which is used for calculating the recall.
Figure 2(d) shows an example of calculating this recall.
There are two minimal deep unlearning  sets for unlearning the target fact and by definition $U_{k}^{\mathcal{A},*}=U_{k}^{*,(1)}$ is picked for the final recall value.

Subsequently, we can define the set of facts to measure the accuracy of the LLM.
$U_{k}^{\mathcal{A},*}$, as previously defined, is the minimal deep unlearning set to calculate the recall.
We calculate the accuracy among the knowledge base excluding this minimal deep unlearning  set, $\mathcal{K}\backslash U_{k}^{\mathcal{A},*}$, for measuring the model utility:

$$ $\text{Accuracy}(\mathcal{A},k;\mathcal{K},\mathcal{R})=\frac{|(\mathcal{K} \backslash U_{k}^{\mathcal{A},*})\backslash U_{k}^{\mathcal{A}})|}{|\mathcal{K }\backslash U_{k}^{\mathcal{A},*}|}.$ (2) $$

At the ideal situation where the unlearning method $\mathcal{A}$ exactly unlearns a minimal deep unlearning set, both recall and accuracy are $1$; otherwise, either the unlearning method does not deeply unlearn the target fact (recall$<1$), or it unlearns extraneous facts (for unlearning the target fact $k$) (accuracy$<1$).

##### Approximation algorithm for calculating recall and accuracy.

The calculations of both recall and accuracy rely on an optimization problem

$$ $U_{k}^{\mathcal{A},*}:=\arg\max_{U_{k}^{*}\in\mathcal{M}_{k,\mathcal{R}, \mathcal{K}}}\frac{|U_{k}^{\mathcal{A}}\cap U_{k}^{*}|}{|U_{k}^{*}|},$ $$

where $\mathcal{M}_{k,\mathcal{R},\mathcal{K}}$ denote the set of all minimal deep unlearning  sets to unlearn $k$ (from the knowledge base $\mathcal{K}$ with respective to the rule set $\mathcal{R}$).
However, finding this exact $U_{k}^{\mathcal{A},*}$ in general can be NP-hard (Skiena, ).
Alternatively, we propose Algorithm 3, which are able to find multiple minimal deep unlearning  sets $\hat{\mathcal{M}}_{k,\mathcal{R},\mathcal{K}}$.
With $\hat{\mathcal{M}}_{k,\mathcal{R},\mathcal{K}}$, it is efficient to find $\hat{U}_{k}^{\mathcal{A},*}:=\arg\max_{U_{k}^{*}\in\hat{\mathcal{M}}_{k,
\mathcal{R},\mathcal{K}}}\frac{|U_{k}^{\mathcal{A}}\cap U_{k}^{*}|}{|U_{k}^{*}|}$ and approximately calculate the recall and accuracy afterwards.

The idea in Algorithm 3 is to generate a single deep unlearning set with some randomness (line 3) and to repeat this generation process to attain multiple different deep unlearning sets (line 4);
the proof that $\hat{M}_{k,\mathcal{R},\mathcal{K}}$ returned by Algorithm 3 is a collection of minimal deep unlearning sets is in Appendix A.
There are two steps to find a single minimal deep unlearning  set;

- 1.
Find the deep unlearning  set (Algorithm 1). We enumerate the rules and find all combinations of facts that can imply $k$ (line 4).
For each combination, if no facts in this combination are in the returning set $U_{k}$, we randomly pick one fact from this combination and add it to the returning set $U_{k}$ (lines 5-7).
Additionally, for the picked fact in any combination, we repeat the above process but for this fact recursively.
This algorithm guarantees that $k\notin\Omega(\mathcal{K}\backslash U_{k},\mathcal{R})$ and randomness from picking fact in each combination and the order for going through the combinations brings the diversity of the results.
- 2.
Prune $U_{k}$ to a minimal deep unlearning  set $U_{k}^{*}$ (Algorithm 2). We go through every fact $k_{\rm cur}$ in $U_{k}$ one by one and check if $U_{k}\backslash\{k_{\rm cur}\}$ from $\mathcal{K}$ is still a deep unlearning set. If yes, we can safely remove $k_{\rm cur}$ from current $U_{k}$ and repeat this process until there is no $k_{\rm cur}\in U_{k}$ that can be removed. The $U_{k}^{*}$ returned by this algorithm is guaranteed to be a minimal deep unlearning set, and the randomness in the order of checking $k_{\rm cur}\in U_{k}$ brings the diversity of the results.

By running Algorithm 3 on the facts in the synthetic dataset introduced in the later section, we find that Algorithm 3 is capable of generating a diverse set of minimal deep unlearning sets.
For more than half of the facts in our synthetic dataset, Algorithm 3 can return 6-17 different minimal deep unlearning sets.
This demonstrates the effectiveness of Algorithm 3 and hence leads to a good approximation for computing the recall in Equation 1.
Please check more details together with the example of minimal deep unlearning sets found by Algorithm 3 in Appendix C.

## 4 EDU-RELAT: Evaluating Deep Unlearning through RELATionships

To systematically evaluate deep unlearning in LLMs, we need a dataset that is already in the LLMs and consists of multiple instances where one or more facts imply other facts by some realistic rules.
One plausible way of constructing such a dataset is to use real-world knowledge bases such as the triplets in Wikipedia dump(^2^22[https://github.com/neelguha/simple-wikidata-db](https://github.com/neelguha/simple-wikidata-db)).
However, we found it challenging to extract a good dataset from the real-world knowledge base because of two reasons.
First, the existing real-world knowledge bases are noisy and incomplete, e.g., we found some cases where (Country A, is neighbor of, Country B) is in the knowledge base but (Country B, is neighbor of, Country A) is not.
Another difficulty is at the engineering level.
It is challenging to find the correct prompt to check whether a fact is in the LLM or not (Zhong et al., ), and this can bring many false negatives.
Because of these two reasons, there are only few and sparse examples where the fact can be deduced from other facts in the real-world knowledge base.

Alternatively, we construct a synthetic dataset named EDU-RELAT for systematically evaluating deep unlearning through relationships in the family.
We locate our synthetic dataset in a family network, which is a common scenario to study rule mining and knowledge discovery in the literature (Galárraga et al., ; Cheng et al., ; Luo et al., ).
This synthetic dataset includes a synthetic knowledge base consisting of $400$ family relationships and $300$ biographical facts among $100$ fictitious people, as well as a set of realistic logical rules, which are deductions among family relationships.
Family relationships include child, father, mother, husband, wife, brother, sister, aunt, uncle, nephew, niece.
Biographies include birthyear, birthplace, and job.
Table 1 shows some examples of facts in family relationships and biographies, together with the question-answer pairs for checking whether this fact is in the LLM or not.
Moreover, the rule set $\mathcal{R}$ has 48 rules, which are used to deduce the facts in family relationships.
Table 2 shows all rules that can imply the fact that has child as the relationship.

**Table 1: Examples of synthetic facts in family relationships and biography.**
| Fact | Question | Answer |
| --- | --- | --- |
| (Reid Perry, father, Richard Perry) | Who is Richard Perry to Reid Perry? | Father |
| (Richard Perry, child, Quentin Perry) | Who is Quentin Perry to Richard Perry? | Child |
| (Quinn Gray, sister, Rachel Gray) | Who is Rachel Gray to Quinn Gray? | Sister |
| (Sloane Lee, birthyear, 1908) | What is the birth year of Sloane Lee? | 1908 |
| (Sloane Lee, birthplace, Washington state) | What is the birthplace of Sloane Lee? | Washington state |
| (Sloane Lee, job, Banker) | What is the job of Sloane Lee? | Banker |

**Table 2: Examples of rules in $\mathcal{R}$ that deduce the fact that has child as relation.**
| (B, mother, A) $\to$ (A, child, B) | (B, father, A) $\to$ (A, child, B) |
| --- | --- |
| (C, mother, A) $\wedge$ (B, brother, C) $\to$ (A, child, B) | (C, mother, A) $\wedge$ (B, sister, C) $\to$ (A, child, B) |
| (C, father, A) $\wedge$ (B, sister, C) $\to$ (A, child, B) | (C, father, A) $\wedge$ (B, brother, C) $\to$ (A, child, B) |
| (A, child, C) $\wedge$ (B, sister, C) $\to$ (A, child, B) | (A, child, C) $\wedge$ (B, brother, C) $\to$ (A, child, B) |
| (A, child, C) $\wedge$ (B, wife, C) $\to$ (A, child, B) | (A, child, C) $\wedge$ (B, husband, C) $\to$ (A, child, B) |

We make several efforts to better mimic a knowledge base of real-world including:

- •
Family network generation. We recursively expand the network. Given a node (person), with a certain probability, we generate the parents, spouse, and children of this person. We control the whole family network in 4 generations. The number of children from any couple is sampled from a truncated geometric distribution between 1 and 4.
- •
Name generation. We collect two lists of first names for males and females separately and assign the first name to each person according to gender. As for the last name, each person’s last name is the same as the father’s if the father exists in the network. There is only one special case where the female’s last name has a small probability of switching to her husband’s.
- •
Biography generation. We have three biographical attributes, birth year, birthplace, and job:
–
The birth years of people are aligned with their relationships. The birth year of any child is from a truncated Gaussian distribution given his/her mother’s birth year. The difference in birth years of a couple is sampled from a reasonable distribution as well.
–
The birthplace is the state in the United States. The child’s birthplace is the same as the birthplace of the parent with a high chance, or sampled from the population distribution in the United States.
–
The job list is collected from GPT4 for every ten years in 1900-2020. The job of a person is picked based on the birth year.

We believe these realistic considerations reduce the gap in evaluations between the unlearning task in our synthetic dataset and the real-world unlearning task.
More statistics of this synthetic dataset are presented in Appendix B.

Within the synthetic dataset EDU-RELAT, we finetune pre-trained LLM on it for the next unlearning evaluation.
The unlearning method is run with the finetuned LLM and tested on a subset of facts on family relationships together with the rules.
The recall defined in Equation 1 is calculated with an approximation Algorithm 3;
in addition, we measure the accuracy of the remaining facts in both family relationships and biographies to reflect the cost of unlearning.

## 5 Experiments

In this section, we investigate to what extent current unlearning methods succeed at deep unlearning.
We release our dataset and code as a benchmark at
: [https://github.com/wrh14/deep_unlearning](https://github.com/wrh14/deep_unlearning).

### 5.1 Set-up

##### Unlearning methods.

For each target LLM, we evaluate four common unlearning methods in the literature: gradient ascent, negative preference optimization, task vector and who’s harry potter.

- •
Gradient ascent (GA; (Jang et al., )) maximizes the loss on the target data, which is a reversed process of learning that does gradient *descent*. More optimization steps $T$ result in better unlearning but worse accuracy on extraneous facts.
- •
Negative preference optimization (NPO; (Zhang et al., )) minimizes the difference between the likelihood of the target data $L(x_{\rm target};f_{\theta})$ and the likelihood $L(x_{\rm target};f_{\rm original})$ from the original model $f_{\rm original}$, while not allowing the unlearnt model to diverge too much from the original model. The objective is defined as
$\mathcal{L}(x_{\rm target},\theta)=-\frac{2}{\beta}\log\sigma\left(\beta\log
\left(\frac{L(x_{\rm target};f_{\theta})}{L(x_{\rm target};f_{\rm original})}
\right)\right).$
As suggested by the literature, the parameter $\beta$ that controls the degree of divergence between unlearnt and original models is set to $0.1$.
Similar to gradient ascent, the step $T$ of optimizing $\mathcal{L}(x_{\rm target},\theta)$ is to control the trade-off between the unlearning performance and the model utility.
- •
Task vector (TV; (Ilharco et al., )) first finetunes the original model $f_{\rm original}$ on the target data $x_{\rm target}$ until the original model overfits to the target data. Let $f_{\rm overfit}$ denote the overfitting model. Then the difference overfitted and original model $f_{\rm overfit}-f_{\rm original}$ can be used as the direction towards learning $x_{\rm target}$, and intuitively its negative direction can be used for unlearning the target data. Therefore, TV defines the unlearning model as $f_{\rm original}-\alpha\cdot\left(f_{\rm overfit}-f_{\rm original}\right)$. A larger value of parameter $\alpha$ gives a higher degree of unlearning but will hurt the model utility.
- •
Who’s harry potter (WHP; (Eldan & Russinovich, )) is based on a similar idea as TV and uses the overfitted model $f_{\rm overfit}$.
Instead of being guided by the difference in weights it uses the probability. Let $P_{f}(x_{t}|x_{1:t-1})$ denote the next token $x_{t}$ distribution given the language model $f$ and prompt $x_{1:t-1}$. WHP samples the next token by the token distribution defined as
$P_{f_{\rm original}}(x_{t}|x_{1:t-1})-\alpha\max(P_{f_{\rm overfit}}(x_{t}|x_{
1:t-1})-P_{f_{\rm original}}(x_{t}|x_{1:t-1}),0).$
(3)
The role of $\alpha$ is similar to $\alpha$ in TV.

##### Target LLMs.

We evaluate with four popular LLMs: GPT2-XL ((Radford et al., ), 1.5B) Phi-1.5 ((Li et al., ), 1.3B), Llama2-7b ((Touvron et al., ), 7B), Llama3-8b ((Dubey et al., ), 8B).
We first finetune these pre-trained LLMs on our synthetic dataset EDU-RELAT; see Appendix D for more finetuning details.
The finetuned LLMs are the target to study the unlearning.

##### Target data and evaluation metric.

We have 11 different family relationships (e.g., child) in the synthetic knowledge base EDU-RELAT.
For each family relationship, we pick 5 facts, which results in 55 facts in total for the unlearning evaluation.
Our task is deep unlearning *single* fact and report the average performance among these 55 facts.

Specifically, for each fact, we run four unlearning methods from four LLMs and measure the recall (Equation1) and accuracy (Equation 2).
Because each unlearning method has its parameter to control the trade-off between unlearning performance and model utility, to unlearn each target fact, we can collect a list of accuracy and recall by varying this parameter. See Appendix D for the details of the hyperparameter settings.
We calculate the accuracy when the recall is larger than $0.8$ (Acc$@$Recall$\geq 0.8$; higher is better) and the recall when the accuracy is larger than $0.8$ (Acc$@$Recall$\geq 0.8$; higher is better), and report their average values among the 55 facts.

Figure: Figure 3: Acc@Recall$\geq 0.8$ and Recall@Acc$\geq 0.8$ of four unlearning methods when evaluated with four LLMs. We observe that there is no unlearning method reaching the region of both Recall$\geq 0.8$ and Accuracy$\geq 0.8$; Moreover, three relatively more promising methods, GA, NPO and TV, perform better on larger LLMs (Llama2-7b and Llama3-8b) than smaller LLMs (GPT2-XL and Phi-1.5)
Refer to caption: x3.png

Figure: Figure 4: Accuracy-recall curve when testing four unlearning methods for deeply unlearning from four LLMs. GA, NPO and TV have better trade-off between accuracy and recall than WHP.
Refer to caption: x5.png

### 5.2 Results

Main observation: no unlearning method succeeds in deep unlearning even for just a single fact.
In Figure 3, we show the average Acc@Recall$\geq 0.8$ and Recall@Acc$\geq 0.8$ across 55 target facts for four unlearning methods on four LLMs, as well as the standard deviation presented as the error bar.
It is observed that no unlearning method reaches the region of both Recall$\geq 0.8$ and Accuracy$\geq 0.8$; this means that all unlearning methods are not capable of attaining a high degree of deep unlearning while keeping an LLM utility after the unlearning.
Notice that accuracy of $0.8$, i.e., dropping $0.2$ from $1$, is actually a high cost, as this is the cost of unlearning only single fact; in practice, there will usually be more target facts and hence a harder setting.
For easier future comparison, we also report the numbers (Acc$@$Recall$\geq 0.8$ and Acc$@$Recall$\geq 0.8$) in Table 3.

Comparison among four unlearning methods.
We visualize the accuracy-recall curve in Figure 4, where each point is the average accuracy and recall of an unlearning method with one hyperparameter across the evaluation of 55 target facts.
We observe that GA, NPO, and TV are better WHP on all four LLMs.
Particularly, the curve of WHP stops at a low recall; this is also reflected in Figure 3 where the Acc@Recall$\geq 0.8$ of WHP is very low.
This is because of the definition as shown in Equation 3: for those negative dimensionalities in $P_{f_{\rm overfit}}(x_{t}|x_{1:t-1})-P_{f_{\rm original}}(x_{t}|x_{1:t-1})$, they are invariant when varying $\alpha$ due to the operator $\max(\cdot,0)$.
Therefore, even if we select $\alpha=10^{8}$ in our experiment, the recall cannot increase because no more facts would be lost due to the invariance.

Superficial unlearning versus deep unlearning.
We measure the accuracy when the unlearning method has unlearnt the target fact but not necessarily any deep unlearning set (Acc@Superficial Unlearning).
As shown in Figure 5, we find that GA is capable of carrying out this superficial unlearning – it can successfully unlearn the target fact without losing significant accuracy.

Figure: Figure 5: Accuracy@Superficial Unlearning, where only the target fact itself is required to be unlearnt. GA successfully unlearns the target fact without losing significant accuracy.
Refer to caption: x9.png

However, as discussed in the previous paragraph, even the best method GA loses more than $0.2$ in accuracy to attain recall$\geq 0.8$ of deep unlearning.
This is because deep unlearning is a more challenging setting than superficial unlearning – deep unlearning single fact in EDU-RELAT can require unlearning more than 10 facts from the LLM by the definition.

Unlearning on small models versus unlearning on larger models.
As shown in Figure 3, while the poor-performing WHP has consistently low performance across four LLMs, three relatively more effective methods GA, NPO and TV perform better on larger LLMs (Llama2-7b and Llama3-8b) than the smaller LLMs (Phi-1.5 and GPT2-XL).
To be specific, the Acc@Recall$\geq 0.8$ of GA, NPO or TV on Llama2-7b or Llama3-8b is around $0.1$ or more higher than it on Phi-1.5 or GPT2-XL;
It is observed similarly for Recall@Acc$\geq 0.8$.
the Recall@Acc$\geq 0.8$ of NPO on Llama2-7b or Llama3-8b is also about $0.2$ higher.
We hypothesize this is because larger LLM has a better inherent understanding of the correlations between facts, which can be important in attaining a good performance of deep unlearning.

Family relationships accuracy versus biographies accuracy.
We take a closer look at the accuracy of two subgroups of the knowledge base, the family relationships and biographies.
The results are plotted in Figure 6.
We observe that the accuracy of biographical facts can be retained better than the accuracy of family relationships.
Specifically, the Acc@Recall$\geq 0.8$ of GA on biographical facts is higher than $0.8$ on Llama2-7b and Llama3-8b, while it is only $0.6$ on facts in family relationships.
This is because, during unlearning, facts in family relationships are closer to the target facts (which are also family relationships) than the biographical facts and are likely to get more easily affected.

Figure: Figure 6: Acc@Recall$\geq 0.8$ on facts in family relationships and biographies. It is observed that the accuracy of biographical facts can be retained better than the accuracy of family relationships.
Refer to caption: x10.png

**Table 3: Recall@Acc$\geq 0.8$ and Acc@Recall$\geq 0.8$ of four unlearning methods on four LLMs.**
| Metric | Recall@Acc$\geq 0.8$ ($\uparrow$) | Acc@Recall$\geq 0.8$ ($\uparrow$) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Unlearning methods | GA | NPO | TV | WHP | GA | NPO | TV | WHP |
| GPT2-XL | 0.65 | 0.18 | 0.61 | 0.02 | 0.66 | 0.53 | 0.56 | 0.13 |
| Phi-1.5 | 0.59 | 0.38 | 0.49 | 0.00 | 0.62 | 0.41 | 0.56 | 0.21 |
| Llama2-7b | 0.75 | 0.70 | 0.58 | 0.01 | 0.77 | 0.69 | 0.64 | 0.07 |
| Llama3-8b | 0.73 | 0.44 | 0.48 | 0.01 | 0.72 | 0.74 | 0.63 | 0.19 |

## 6 Related Work

##### Benchmarks and evaluations in LLM unlearning.

TOFU (Maini et al., ) is a benchmark containing fictitious authors and their related biographic question-answering texts such as “Awards” and “Book”.
They evaluate the unlearning utility by comparing the answer from LLM given the to-be-unlearned question and the ground truth answer.
WMDP (Li et al., ) provides knowledge in biosecurity, cybersecurity, and chemical security, which matches the realistic desire for studying unlearning.
A more recent benchmark MUSE (Shi et al., ) in the domain of news articles and books enriches the evaluation by introducing the metric from both memorization and privacy leakage aspects.
Yao et al. () introduces a benchmark of evaluating the unlearning in pre-trained data and the metric of unlearning utility is to compute the perplexity of the data from the memorization aspect.
Patil et al. () and Łucki et al. () evaluate the unlearning from an adversarial attack aspect of knowledge extraction.
This branch of work focuses on proposing more realistic domains and more robust ways to evaluate the unlearning, and the challenge at their benchmark is to unlearn a large batch of facts or texts while keeping the model utility.
However, none of them consider the interrelation between the target facts and other facts also in the LLM.
Deep unlearning in our paper is introduced to capture the connections among multiple facts in the unlearning, and we found that deep unlearning  is challenging enough for just unlearning a single fact.

##### Unlearning methods in LLM.

In addition to the methods evaluated in Section 5, one popular extension is assuming the existence of a “retain” set independent of the target facts. When doing gradient ascent or other gradient-based variants, Yao et al. () and Chen & Yang () minimize the loss on the “retain” set simultaneously to avoid quickly losing other irrelevant facts and hence help with the model utility.
Another popular category is the model-editing based (Meng et al., ; ; Wang et al., ), which hypothesizes that the knowledge is saved in certain MLPs in the transformer and proposes an explicit-form solution for the weight update to unlearn the target facts.

##### Other settings of machine unlearning.

There are other settings of machine unlearning because of different purposes and model structures.
Ginart et al. () and Guo et al. () define the goal of unlearning as the model trained without a single person’s data to meet the data regulation for data owner’s rights.
Tiwary et al. (); Kong & Chaudhuri () study the concept or feature removal in the generative models, e.g. remove “white breast” in all generated bird photos; this is to avoid generating violent and racial content.

## 7 Conclusion and Discussion

In this paper, we propose a new setting for machine unlearning, referred to as deep unlearning, aimed at identifying reliable fact unlearning.
As a starting point, we construct a synthetic dataset EDU-RELAT of family relationships and biographies as a benchmark for research in this emerging setting.
From empirical evaluation using our metrics, we find that current unlearning methods are not capable of deeply unlearning even a single fact while keeping the model utility.
We hypothesize that this shortcoming arises from these methods not fully considering the nature of facts and the deductions between each other.

This work opens several promising directions for future research.
Firstly, more effective methods can be developed with the deep unlearning setting, with an awareness of the connections between facts.
One can assume the complete set of rules and the entire knowledge base may or may not be fully known, referred to as the white-box or the black-box scenarios.
Additionally, there is potential to construct a more nuanced framework that captures more intricate facts and sophisticated deductive processes beyond the current scope of relations and logical rules.
Such advancements will enhance the modeling of deep unlearning and contribute to the development of methods for deeply unlearning a broader range of facts.

#### Acknowledgments

RW, CY and KC are supported by the grant from National Science Foundation NSF (CIF-2402817, CNS-1804829), SaTC-2241100, CCF-2217058, ARO-MURI (W911NF2110317), and ONR under N00014-24-1-2304.

## Appendix A The guranteee of Algorithm 3

In this section, we are going to prove that $\hat{M}_{k,\mathcal{R},\mathcal{K}}$ returned by Algorithm 3 is a collection of minimal deep unlearning sets.

###### Proof.

We can first prove $k\notin\Omega(\mathcal{K}\backslash U_{k},\mathcal{R})$, where $U_{k}$ at line 3 in Algorithm 3 is returned by Algorithm 1. The proof has two steps:

- 1.
We can have $\Omega(\Omega(\mathcal{K},\mathcal{R})\backslash\hat{U}_{k},\mathcal{R})=
\Omega(\mathcal{K},\mathcal{R})\backslash\hat{U}_{k}$, where $\hat{U}_{k}$ here is the $\hat{U}_{k}$ after line 8 in Algorithm 1. Otherwise, by the definition of deductive closure, there exists $k^{\prime}\notin\Omega(\mathcal{K},\mathcal{R})\backslash\hat{U}_{k}$ and $k^{\prime}$ can be deduced from initiation of the rule where all facts on the left of the rule are in $\Omega(\mathcal{K},\mathcal{R})\backslash\hat{U}_{k}$, i.e. not in $\hat{U}_{k}$. However, this can be a contradiction because if $k^{\prime}\notin\Omega(\mathcal{K},\mathcal{R})\backslash\hat{U}_{k}$, $k^{\prime}$ must be in $\hat{U}_{k}$ and line 5-7 in Algorithm 1 can guarantee that for any initiation of any rule that can imply $k^{\prime}$, there is at least one fact on the left of the rule in $\hat{U}_{k}$.
- 2.
From line 1 in Algorithm 1, we know that $k\in\hat{U}_{k}$. This means that $k\notin\Omega(\mathcal{K},\mathcal{R})\backslash\hat{U}_{k}=\Omega(\Omega(
\mathcal{K},\mathcal{R})\backslash\hat{U}_{k},\mathcal{R})$, where the equality is from step 1. On the other hand, $(\mathcal{K}\backslash U_{k})=(\mathcal{K}\backslash\hat{U}_{k})\subseteq
\Omega(\mathcal{K},\mathcal{R})\backslash\hat{U}_{k})$ where the equality comes from the definition $U_{k}=\mathcal{K}\cap\hat{U}_{k}$ at line 9 in Algorithm 1. $k\notin\Omega(\Omega(\mathcal{K},\mathcal{R})\backslash\hat{U}_{k},\mathcal{R})$ and $(\mathcal{K}\backslash U_{k})\subseteq\Omega(\mathcal{K},\mathcal{R})
\backslash\hat{U}_{k})$ together imply $k\notin\Omega(\mathcal{K}\backslash U_{k},\mathcal{R})$.

We now have $k\notin\Omega(\mathcal{K}\backslash U_{k},\mathcal{R})$, then we are going to prove $U_{k}^{*}$ returned by Algorithm 2 is a minimal deep unlearning  set.
From Algorithm 2, it is obvious that $k\notin\Omega(\mathcal{K}\backslash U_{k}^{*},\mathcal{R})$. If it is not the minimal deep unlearning  set, then there exists $U^{\prime}\subset U_{k}^{*}$ s.t. $k\notin\Omega(\mathcal{K}\backslash U,\mathcal{R})$ and there is an $k^{\prime}$ s.t. $k^{\prime}\notin U_{k}^{*}$ and $k^{\prime}\in U^{\prime}$. However, this is a contradiction, because Algorithm 2 only returns $U_{k}^{*}$ if $\forall k^{\prime}\notin U_{k}^{*}$, $k\in\Omega(\mathcal{K}\backslash U_{k}^{*}\backslash\{k^{\prime}\},\mathcal{R})$.

Now we can conclude $U_{k}^{*}$ at line 4 in Algorithm 3 is a minimal deep unlearning  set, and our proof is done.
∎

###### Proof.

## Appendix B Statistics of EDU-RELAT

For a better understanding of our synthetic dataset EDU-RELAT, we present some statistics here.

- •
The distribution of family relations Figure 9. It is observed that child, father and mother are top-three relationships in our dataset.
- •
The distribution of the birth year is plotted in Figure 10, in a range of 1890 - 2000.
- •
The set of jobs, collected from the job list across years 1900-2020, is {Lawyer, Physician, Sales Manager, Machinist, Systems Administrator, Factory Worker, Police Officer, Plumber, Firefighter, Librarian, Television Repairman, Pilot, Network Administrator, Carpenter, Steelworker, Financial Analyst, Clerk, Bank Teller, Secretary, Banker, Radio Technician, Customer Service Representative, Remote Work Consultant, Postman, Baker, Movie Theater Usher, Stenographer, Software Engineer, Doctor, Maid, Construction Worker, Systems Analyst, Electrician, Auto Mechanic, Account Manager, Journalist, Welder, Mechanic, Real Estate Agent, Radio DJ, Telephone Operator, Chauffeur, Taxi Driver, Telemarketer, Car Salesman, Truck Driver, Accountant, Teacher, Airline Pilot, Draftsman, Software Developer, Nurse, Advertising Executive, Graphic Designer, IT Consultant}
- •
The distribution of birthplace is summarized in Figure 11.

Figure: Figure 7: Histogram of # minimal deep unlearning sets founded by Algorithm 3.
Refer to caption: x12.png

Figure: Figure 8: An example of 4 minimal deep unlearning sets founded by Algorithm 3.
Refer to caption: x13.png

Figure: Figure 9: Distribution of relations in our synthetic dataset.
Refer to caption: x14.png

Figure: Figure 10: Distribution of birth years of fictitious people in our synthetic dataset.
Refer to caption: x15.png

Figure: Figure 11: Distribution of birthplaces of fictitious people in our synthetic dataset.
Refer to caption: x16.png

## Appendix C Empirically Evaluating Algorithm 3 on EDU-RELAT

By running Algorithm 3 on the facts from our synthetic dataset, we find that Algorithm 3 does generate a rich set of minimal deep unlearning sets.
In Figure 7, we show the number of minimal deep unlearning sets founded by Algorithm 9 in a histogram.
It is observed that for more than half of the facts as the target fact, Algorithm 3 can return 6-17 different minimal deep unlearning sets.
This demonstrates the effectiveness of Algorithm 3 and hence a good approximation when computing the recall in Equation 1.

We also show an example of minimal deep unlearning sets founded by Algorithm 3 in Figure 8.

## Appendix D More Details on Experimental Settings

##### Details of finetuning LLMs on EDU-RELAT.

The finetuning is under the question-answering format, where the question is given in the prompt and the loss is computed from the answer.
The batch size of finetuning on all four LLMs is 16.
The learning rate is $2e-5$ for GPT-XL and Phi-1.5 and $1e-5$ for Llama2-7b and Llama3-8b; the learning rate scheduler is the linear scheduler from HuggingFace (Wolf, ).
The number of epochs is $10$ for Phi-1.5, Llama2-7b, and Llama3-8b and $15$ GPT-XL to guarantee a full memorization after finetuning.

##### Details of hyperparameters in unlearning methods

For GA, the learning rate is $2e-5$ for GPT-XL and Phi-1.5 and $1e-5$ for Llama2-7b and Llama3-8b; the learning rate scheduler is the linear scheduler from HuggingFace (Wolf, ).
The hyperparameter of the optimization iteration $T$ is selected from $\{1,2,4,8,16\}$ for Phi-1.5, Llama2-7b and Llama3-8b and $\{1,2,4,8,16,32\}$ for GPT-XL.
For NPO, the learning rate is $4e-5$ for GPT-XL and Phi-1.5 and $2e-5$ for Llama2-7b and Llama3-8b; the learning rate scheduler is the linear scheduler from HuggingFace (Wolf, ).
The hyperparameter of the optimization iteration $T$ is selected from $\{1,2,4,8,16\}$ for Phi-1.5, Llama2-7b and Llama3-8b and $\{1,2,4,8,16,32\}$ for GPT-XL.
For both TV and WHP, the “overfit” model is finetuned with 10 more iterations on the target data point.
In TV, the hyperparameter $\alpha$ is from $\{0.2,1.0,5.0,10.0,30.0,60.0,80.0\}$ for GPT-XL and $\{0.2,0.5,1.0,5.0,10.0\}$ for Phi-1.5, Llama2-7b and Llama3-8b.
In WHP, the hyperparameter $\alpha$ is from $\{10^{1},10^{3},10^{5},10^{7},10^{8}\}$.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Q1: How would the accuracy and recall metrics change w.r.t. to the # of minimal deep learning sets discovered when applied on a KB with incomplete/partial logical dependencies? Would recall increase since the minimal deep unlearning set is smaller and closer in size to the actual set unlearned by the algorithm, thus having higher overlap? What about accuracy?

Q2: Given the small size of the target facts to be unlearnt, how do you satisfy the fine-tuning data requirements of methods such as NPO & WHP?

Q3: What motivated 0.8 as threshold value for comparing recall/accuracy (Figures 3 and 6)?
soundness: 2
strengths: This paper extends the discussion on the "illusion of machine unlearning" vis-à-vis current unlearning methods, to the relatively simple yet seemingly hard task of unlearning a single fact from the LLM's parametric memory, given that logically related facts are also present in the LLM. This work shows, somewhat surprisingly, that when current unlearning methods are tasked with erasing a single fact from the LLMs parametric knowledge, they do not erase related facts which also share entities/objects with the target fact, for e.g. erasing the target fact **F1**: Y is child of X, may fail to erase logically related facts such as: **F2**: Z is husband of X, and **F3**: Z is father of Y, even though Y as an entity appears in F3, and this would allow the erased fact to re-surface via logical deduction. On a synthetic toy dataset, they provide a reasonable set overlap based metric for quantifying the degree of such unlearning. While it is already known that most knowledge that is supposedly deleted by current unlearning methods can be re-surfaced via adversarial/jailbreak prompting and probing, this work highlights limitations of current methods at the atomic level of single-fact deletion, hinting at a tension between an LLM's reasoning capabilities and the effectiveness of unlearning techniques.
summary: This paper presents a critique of current unlearning methods through the lens of fact unlearning - it shows that while current unlearning methods can unlearn the target fact in isolation, they often fail to unlearn other related facts through which the target fact can be logically deduced, thus negating the effect of unlearning. The authors refer to the task of unlearning additional facts which are logically related to the target fact, as "deep unlearning". They introduce 2 metrics based on set overlap: recall and accuracy, which quantify the extent of "deep unlearning". All experiments are conducted on a small, synthetic dataset derived from a synthetic knowledge base and a set of logical rules.
weaknesses: - Firstly, I find it a bit counter-intuitive that even an unlearning method such as WHP (Who's Harry Potter) would not suppress the effect of other facts which share entities with the target fact, such as in the case of facts related to familial connections where a target entity appears verbatim in other facts. This leads me to wonder if the failure to "deeply unlearn" a target fact is simply an artifact of the way in which the unlearning methods are applied e.g. WHP requires finetuning a reinforced model on a corpus that is much larger than just a handful of facts, in order for the logit difference with the baseline to be significant. The paper does not really discuss how each of the unlearning methods are actually applied, and whether any of the tested unlearning methods are even compatible with the setting of single fact unlearning - in my view, methods such as NPO, TV, and WHP are clearly more suited for a setting where the unlearning target is defined around a *concept* or *topic*, rather than single **fact**. A discussion around the compatibility of the evaluated unlearning methods with the single fact unlearning setting, and what role the small size of the synthetic dataset plays in limiting the effectiveness of unlearning methods, would be a useful addition. For example, if the WHP method is properly applied as it is designed i.e. for concept unlearning rather than fact unlearning, I imagine that it would successfully suppress the fact that "Harry is child of Lily" even if it was fine-tuned on other facts that mentioned that "James is father of Harry" and "James is husband of Lily". To help readers better understand the experimental setup and interpret the results, I recommend that the authors discuss the adaptations they made to apply WHP, NPO, TV etc., to the fact unlearning task.

- The synthetic dataset appears to be biased towards highly deducible relationships (e.g., familial connections) which is not really representative of real-world knowledge structures where logical connections are typically more complex and less *deducible*. In my view, the authors should have included an evaluation on a dataset extracted from a real-world knowledge base with partial/noisy/incomplete logical connections between facts - my guess is that since most real facts require more than just simple deductive reasoning e.g. may require multi-hop reasoning, this problem of knowledge deductibility or fact reconstruction would be less of an issue even with current unlearning methods. While the authors claim that it is hard to conduct prompt based evaluations for determining if a fact is in the LM (false negatives), I'd like to point out that alternatives to the simple prompts used by the authors (as shown in Table 1) do exist (see [1]), such as MCQ-based binary choice Q&A or latent supervised/unsupervised probing. It would also useful have been to see how the "approximate" recall and accuracy metrics will scale to real KBs where many of the logical connections between facts are unknown. 

- The authors should discuss the limitations of their current dataset and how these might affect the generalizability of the results.

- Minor typos: typo in appendix C, should be “algorithm 3” instead of “Algorithm 9”.  Caption of Figure 7 should also say “algorithm 3” instead of “algorithm 1”.  Typos in each caption of subfigures in figure 8: should say “Minimal Deep Unlearning” sets. 

[1] [Eight Methods to Evaluate Robust Unlearning in LLMs](https://arxiv.org/abs/2402.16835)

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 1
questions: Refer to peaknesses, I hope that you can solve these concerns.
soundness: 2
strengths: 1. Novel Contribution: The introduction of deep unlearning and a curated dataset highlights a significant gap in current research, drawing attention to an important and underexplored issue. 
2. Innovative Metrics: The proposal of Recall as a new evaluation metric, accompanied by a detailed algorithm for addressing the NP-hard nature of the problem, demonstrates thoughtful consideration of the challenges involved in deep unlearning. 
3. Comprehensive Experiments: The thorough experimental setup across four LLMs and unlearning methods strengthens the paper's credibility and provides valuable insights into the effectiveness of the proposed approach.
summary: This paper introduces "deep unlearning" as a novel approach in the domain of large language models (LLMs), emphasizing its importance in effectively erasing certain facts. As the target fact can be deduced from logical rules, superficial unlearning methods, which solely unlearn the target fact, cannot unlearn it successfully. The authors also present new metrics, including Accuracy and Recall, to evaluate this process, backed by experiments across various LLMs and unlearning methods.
weaknesses: 1. Code Availability: The lack of code and dataset release until after acceptance may hinder reproducibility and limit the community's ability to validate the findings. If possible, we encourage the author to give an anonymous link to the git repo. If it is forbidden in rebuttal process, you can include more raw samples in Appendix with explanation. More importantly, as you proposed a benchmark, the statistics of the datasets are of vital importance, which should be discussed in the main part, rather than appendix.
2. Missing Results: The author did not report the accuracy on the dataset of the models fine-tuned but not subjected to unlearning, which makes it difficult to gauge the impact of the unlearning methods accurately. Please include the baseline accuracy results for the fine-tuned models before unlearning.
3. Poor Presentation: The author should improve the writing and figure for better illustration, especially the figures. For instance, Figure 3 and 6 are not clear to the analyzed conclusions. Maybe a organized table will be better. From my perspective, for this work, the related work is crucial for the understanding of the benchmark, which can be moved to Sec 2.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: Please refer to the weakness part.
soundness: 3
strengths: 1. Presents an important and novel problem - deep unlearning that considers logical reasoning between facts
2. Constructs a structured evaluation dataset with reasonable logical rules and relationships
3. Proposes reasonable evaluation metrics (recall and accuracy) and designs approximate algorithms to compute these metrics
4. Comprehensive experiments that reveal the limitations of current methods in deep unlearning
summary: This paper introduces the concept of "deep unlearning", which studies the unlearning tasks in large language models where logical relationships between facts need to be considered. The authors constructed a synthetic dataset EDU-RELAT, containing family relationships and biographical information, to evaluate four unlearning methods across four different-sized LLMs.
weaknesses: 1. While deep unlearning sounds reasonable overall, the paper's setting may not align with practical scenarios. In real machine unlearning cases, related knowledge is typically forgotten together (e.g., forgetting all content related to Harry Potter) rather than just forgetting a single relationship. Under this setting, is it still important to forget all knowledge that could potentially derive the current relationship?

2. Given that real-world relationships can be far more complex than the R defined in this paper's dataset, could there be situations where forgetting one piece of knowledge requires forgetting an excessive amount of content? In multi-hop reasoning scenarios, a large amount of knowledge might need to be forgotten, which raises another question: is deep unlearning always necessary in unlearning scenarios? If unlearning is applied for copyright protection, can knowledge derived through multi-hop reasoning constitute infringement?

3. The paper only focuses on logical rules in the specific domain of family relationships, which is rather narrow in scope.

4. No new solutions are proposed to improve deep unlearning effectiveness, remaining only at the problem analysis level.

Meta-review: This paper introduces "deep unlearning" as a novel approach in the domain of large language models (LLMs), emphasizing its importance in effectively erasing certain facts. As the target fact can be deduced from logical rules, superficial unlearning methods, which solely unlearn the target fact, cannot unlearn it successfully. This work presents an important and novel problem - deep unlearning that considers logical reasoning between facts. The authors also present new metrics, including Accuracy and Recall, to evaluate this process, backed by experiments across various LLMs and unlearning methods.

There were several non-trivial issues raised in the reviews. First, the experiments did not compare the utility performance of a vanilla model with the unlearned model. Second, there have been concerns regarding the limited studied scope that was merely focusing on logical rules. Third, there were also concerns among the reviews regarding the quality of presentation. Some (not all) of the issues were incorporated into the revision.

Decision: Reject