Why Fine-Tuning Struggles with Forgetting in Machine Unlearning? Theoretical Insights and a Remedial Approach

Authors: Meng Ding, Jinhui Xu, Kaiyi Ji
Primary area: learning theory
Keywords: Machine Unlearning, Fine-Tuning, Learning Theory

Abstract:
Machine Unlearning has emerged as a significant area of research, focusing on 'removing' specific subsets of data from a trained model. Fine-tuning (FT) methods have become one of the fundamental approaches for approximating unlearning, as they effectively retain model performance. However, it is consistently observed that naive FT methods struggle to forget the targeted data. In this paper, we present the first theoretical analysis of FT methods for machine unlearning within a linear regression framework, providing a deeper exploration of this phenomenon. We investigate two scenarios with distinct features and overlapping features. Our findings reveal that FT models can achieve zero remaining loss yet fail to forget the forgetting data, unlike golden models (trained from scratch without the forgetting data). This analysis reveals that naive FT methods struggle with forgetting because the pretrained model retains information about the forgetting data, and the fine-tuning process has no impact on this retained information. To address this issue, we first propose a theoretical approach to mitigate the retention of forgetting data in the pretrained model. Our analysis shows that removing the forgetting data's influence allows FT models to match the performance of the golden model. Building on this insight, we introduce a discriminative regularization term to practically reduce the unlearning loss gap between the fine-tuned model and the golden model. Our experiments on both synthetic and real-world datasets validate these theoretical insights and demonstrate the effectiveness of the proposed regularization method.

Decision: reject
Reviewer scores: 6, 3, 3, 6

Reviews and discussion:
<thread>
creation date:
1730554081302

thread_type:
official_review

<thread_object>
creator: reviewer_j9sb

<content>
summary:
This paper investigates machine unlearning, which aims to protect user privacy by removing specific data from trained models in response to data deletion requests. The authors examine why fine-tuning often fails to fully erase targeted data. They consider over-parameterized linear regression in the case of overlapping and no overlapping features. They propose a regularization term that diminishes the role of forgetting. Experimental results on both synthetic and real-world datasets validate that this regularization approach significantly enhances unlearning performance.

strengths:
Studying unlearning from the perspective of overparametrizied regression is a great concept. This setting (even though questionable in practice) allows to perform theoretical analyses. 

The entire concept of introducing a regularization term to unlearning is very sound and novel. 

The experimental results show improvements if such a term is included.

weaknesses:
The setting for the analyses is simplistic. It would be great to consider more general cases (for example strongly convex). 

I think the vast majority of the practical cases consider 100% overlapping cases which puts in question the bulk of the analyses. 

The distinct features section is a special case of the overlapping section and thus it should be omitted. I think the distinct features results are not stronger and thus they are a 'strict' special case. 

Option B is 'void' if all of the features are overlapping which captures the majority of the use cases.

questions:
1. Why bothering with the case of non overlapping features? While such cases sometimes occur in FL they are a much more seldom occurrence in standard ML. 
2 Can the analyses be done for the strongly convex case? It seems it would require a completely different approach since a closed form expression is not available in such a case. What about a 2 layer linear network?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730556630071

thread_type:
official_review

<thread_object>
creator: reviewer_naau

<content>
summary:
This paper works on why the current fine-tuning unlearning method cannot perform well in many unlearning tasks. This paper provides a theoretical analysis within a linear regression framework to show when fine-tuning retains model performance on remaining data, it cannot fully remove the influence of the forgetting data. Then this paper proposes a discriminative regularization term to close the performance gap between fine-tuned model and retrained model. The experimental results validate the effectiveness of this approach in improving unlearning accuracy.

strengths:
1. The topic of this paper is quite interesting. The fine-tuning approach is one of the mainstream approaches to unlearning. However, such methods are usually unstable across different unlearning tasks and datasets. Thus, the research on why it can fail is meaninful.
2. This paper provides a theoretical analysis of the linear regression model for the analysis.
3. The experimental results can clearly show the performance improvements compared with the other fine-tuning methods.

weaknesses:
The total contribution of the paper is not enough:  

1. Theoretical Analysis: Target on theorem 3.2 and 3.4: this paper claims that the MSE on remaining data and forgetting data keep 0 during fine-tuning for overparameterized models. However, in real-world datasets, the model cannot fit the training data perfectly, and the two theorems are hard to extend to other larger models and datasets. In addition, such analysis is based on a regression model, while the following experiment part is mainly based on classification tasks. Whether the theoretical analysis on regression can be extended to classification still needs to be proved.  

2. Discriminative Regularization: This paper does not explicitly show the loss of Inverse CE-FT. Is it simply to remove the hyperparameter $\alpha$ from the second term to the first term? If so, I cannot find a significant difference between Inverse CE-FT and original CE-FT. In addition, regarding the loss function of KL-FT, many other methods have tried to incorporate KL loss to align the output logits [1] or interlayer embeddings [2, 3]. Therefore, the proposed Discriminative Regularization does not show any improvement compared with previous works.

3. Experiment results: This paper only conducts experiments on single-class unlearning (classes 3,6 and 9). In addition, this paper only compares the proposed method with naive fine-tuning and the loss proposed in [4]. It is not sufficient to prove the effectiveness of the proposed methods. This paper can include more SOTA unlearning methods in the recent two years and compare unlearning results in more complex settings like random sample unlearning or backdoor attack unlearning.

4. The technical part of this paper needs to be improved. Some notations need to be further checked, for example, $1-n_f$ in line 194.

[1]  Chundawat, Vikram S., et al. "Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 6. 2023.

[2]  Chundawat, Vikram S., et al. "Zero-shot machine unlearning." IEEE Transactions on Information Forensics and Security 18 (2023): 2345-2354.

[3]  Shen, Shaofei, et al. "CaMU: Disentangling Causal Effects in Deep Model Unlearning." Proceedings of the 2024 SIAM International Conference on Data Mining (SDM). Society for Industrial and Applied Mathematics, 2024.

[4] Fan, Chongyu, et al. "Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation." arXiv preprint arXiv:2310.12508 (2023).

questions:
1.  What do the distinct and overlap features mean? Could the author give some examples to explain it?

2.  Considering that this paper mainly conducts class-wise unlearning experiments. How do different methods perform under the evaluation of relearn time [1]?

[1]  Chundawat, Vikram S., et al. "Zero-shot machine unlearning." IEEE Transactions on Information Forensics and Security 18 (2023): 2345-2354.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730680841067

thread_type:
official_review

<thread_object>
creator: reviewer_h2yv

<content>
summary:
The paper focuses on the fine-tuning based unlearning scheme, where approximate unlearning is obtained by performing additional learning steps with samples from the retained set to induce "catastrophic forgetting" of the forget set in the model. To understand failure of this technique, the paper considers the linear models and a couple of simplistic data sets where the set of non-zero features of the retain set (i) do not overlap, or (ii) partially overlap with that of the forget set. In both these cases, the theoretical results demonstrate that the fine-tuned based unlearned model (under a specific version of fine-tuning) has very different performance on the forget set compared to the gold standard unlearned model (which is retrained from scratch using only the retain set). Based on these results, the paper discusses a modification to the fine-tuning based unlearning scheme (the paper states them as forms of regularizations) where we are able to reset the parts of the model corresponding to the set of features that are only non-zero on the forget set, and demonstrates how this procedure improves unlearning performance. Based on these insights, the paper motivates the use of a fine-tuning objective for unlearning that combines both unlearning/forget accuracy and accuracy on the retain set, and empirical evaluations highlight how this combined objective improves the unlearning performance of fine-tuning while also maintaining high performance on the retain set.

strengths:
I think one of the main strengths of this paper is the focus on the fine-tuning based unlearning schemes which have (in general) various advantages such as being relatively very efficient, and not requiring the forget set for the unlearning, which has significant practical implications.

weaknesses:
- (W1) To me, one of the main weaknesses of this paper is that there is no clear link between the theory inspired proposed "regularizations" in sections 3 and 4 and the empirical evaluations of section 5. The proposed regularizations (and the related and motivating theoretical analyses) require the knowledge of the distinct and overlapping feature which is usually not available. Thus, it is obvious that these regularization schemes are not practical. However, the connection between the analysis and the use of the combined objective of unlearning/forget accuracy and retain accuracy is not clear at all even in the form of motivation.
- (W2) Even with the considered combined loss function for fine-tuning based unlearning, it is not clear what is novel here. The combined objective has been considered before (as the paper itself mentions), but the paper claims a difference between treating one as an "objective" and one as a "penalty". The difference between what is a penalty term and what is the main objective in equations (6) and (7) seems not compelling enough. Different values of $\alpha$ would lead to different effects, and there is no inherent need to restrict $\alpha \in [0,1]$. Often both can be written as $\lambda (\text{Retain Loss}) + (1 - \lambda) (\text{Forget Loss})$ for some $\lambda \in [0,1]$, and treated as a single hyperparameter which can range from focusing on the loss on the retain set and the performance on forget set. In that respect, to me CE-FT and ICE-FT are the same thing. The main difference between (I)CE-FT and KL-FT is use of KL divergence instead of cross-entropy to penalize the performance on the forget set. In that case, any difference between KL-FT and (I)CE-FT (if there is any) is attributed to the use of KL divergence.
- (W3) This is less of a weakness, but the evaluated method KL-FT and (I)CE-FT require access to the forget set, in contrast to vanilla FT, which does not. This removes the one advantage of FT based fine-tuning. In this case, a more full-scale evaluation across various schemes is warranted including efficient schemes such as Gradient Ascent and influence function based schemes. However, I do not see any novel "method" being presented here (see W2), so there is nothing to evaluate thoroughly here.



Minor comments:
- (C1) In Figure 1, it appears that the quantities plotted are Retain/Unlearning loss (1 - Retain/Unlearning accuracy), which is a bit confusing given that the legend mentions RA/UA, instead of RL/UL as considered in Theorems 1 and 2.
- (C2) In the equation at line 289, there is also a difference of $\mathbf{P} \mathbf{w}\_*^r$ in the definition of $\mathbf{w}\_t$ from the $\mathbf{P}\_r$ in the definition of $\mathbf{w}\_g$ = $\mathbf{P}\_r \mathbf{w}_*^r$. How does that affect the ensuing discussion?
- (C3) In lines 315-319, it is not clear what $d_o$ is as it does not appear to be defined anywhere? Is $d_o$ another name for $d_{\text{lap}}$ or something else?
- (C4) The function $\mathcal{L}_{\text{KL}}(...)$ in equation (6) lacks any clear definition. There are multiple KL divergence based losses discussed in Golatkar et al (2020a), and many of them are related to the Fisher Forgetting scheme proposed therein (unless I have my references wrong). Random label is a baseline there, but I could not find a random label + KL based loss function in there. A more explicit definition here will be very useful.

questions:
- (Q1) Forgetting in linear regression (considered in Sections 3 and 4) would be similar to random data forgetting in classification. What is a "class-wise forgetting" equivalent in the regression setup?
- (Q2) All the evaluations are performed on class-wise forgetting while the theoretical analysis is performed for regression. Is there a reason for why the random forgetting scenario is not considered in the evaluations?
- (Q3) For the results in Table 2, where we have multiple unlearning metrics, how is the hyperparamter $\alpha$ selected for (I)CE-FT and KL-FT?
- (Q4) If hyperparameter optimization is done appropriately (as mentioned above), the main difference between KL-FT and CE-FT from (unmasked) SalUn is the use of KL instead of CE with the forget set. Is there any reason / intuition why we should expect KL divergence based forget set penalty to perform better in terms of all the unlearning metrics compared to the cross-entropy based forget set penalty (that is (I)CE-FT vs KL-FT)?
- (Q5) In the overparameterized regime, the optimal solution to the learning problems (1)-(3) are not necessary singleton sets. Is there any reason we expect the $\arg \min_{\mathbf{w}}$ to be a singleton set and not a set of solutions? If it is in fact not guaranteed to be a singleton set, how does that affect unlearning results in this paper?
- (Q6) In the overlapping feature case, if $d_{\text{lap}} = d$, (that is, full overlap of features) what happens to the bounds? In this case, is there any provable difference between $L(\mathbf{w}_t, D_f)$ and $L(\mathbf{w}_g, D_f)$?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730687412518

thread_type:
official_review

<thread_object>
creator: reviewer_bhdy

<content>
summary:
This work explores a known issue with naïve fine-tuning approaches for the machine unlearning problem: it struggles to forget the targeted data. To address this problem, the authors begin by constructing a synthetic experiment (an overparameterized linear regression model) and show that, in this case, fine-tuned weights decompose into two components—one that targets the remaining data and one that can be considered the residual from the data to forget.

Based on this decomposition, they compare two approaches that aim to reduce the unwanted component in the final solution. They find both empirically and theoretically that the approach focusing on solving for the remaining data, rather than solely forgetting the target data, performs better. This observation leads them to propose a loss function that prioritizes overall accuracy over the forgetting term. The performance of this loss is empirically evaluated on a real dataset.

strengths:
1. The overall paper is well-structured and pleasant to read.

2. The theoretical results inspired insights into the practical effect of the real loss, which were validated on a real data example. The theoretical section is strong and insightful. Supporting empirical experiments presented alongside each step to further ground the understanding are sensible, well presented, and convey each point effectively.

weaknesses:
1. I found the presentation of the two terms in the loss (Eq. 6), with one being the main term and the other the “regularizing term,” to be problematic. Unless I am mistaken, this distinction is artificially created by placing an arbitrary cap on the regularization scaling term $\alpha \in 0-1$, which effectively upper bounds the contribution of each term to the loss. Therefore, the second point, 2) Regularization Focus, does not represent a real difference, in my opinion, and needlessly obfuscates the work (though I appreciate the desire to differentiate from previous work). The crux of the theoretical insight is that the accuracy term should be given more importance than the unlearning term. What is the procedure to tune this $\alpha$ parameter, as it is a crucial value of the experiment? This detail is missing in the main text and should definitely be included.
From my understanding, the ICE-FT is effectively the same as CE-FT with $\alpha \in [1,\infty]$ instead of being restricted to $\alpha \in 0,1$. (If that is not the case, please add an explicit formula for the ICE-FT and disregard the following comment). Related to that point, the optimal $\alpha$  value for CE-FT should then be then as close as 1 as possible. Is it the case? In figure 4b, why is ICE and CE with alpha=1 not equal?

2. The experiment on the real datasets could be improved in a few areas. First, why is only one fine-tuning baseline presented? Second, details about the tuning and training procedure are missing, which are particularly important as the authors highlighted the sensitivity to the $\alpha$ parameter, and there is no single target metric to optimize for. The presentation of the results on the real datasets could also be improved. Looking at Table 2, it is hard to see how each method performs. Showing Figure 3 for all the datasets could help for example, or using UA vs. RA curves would be more convincing than presenting point predictions in a table. Additionally, the legend in Figure 4b is hidden behind the lines and has the wrong colors.

3. Assumption 3.1 and Remark 1. I found the construction of the matrices $F$ and $R$ to be a little bit vague. Shouldn’t it have some constraints on$d_f$ and $d_r$ since they $w_{f*}$ and $w_{r*}$ are to be exact solutions to both problems? The remark really feels more like a part of the assumption (similar comments apply to Remark 2 paired with Assumption 3.3). The concept of feature overlapping should be clarified, as it has a different meaning than how it is usually used.
These points should be clarified to understand the limits of the conclusions we can draw from this synthetic setup.



Minor
- The discussion after Theorem 3.2 could be clarified. The constructed example is a scenario where the weights learned from the different tasks are completely orthogonal to each other, so the fine-tuning step is performed in a totally unrelated space. This is a great illustrative example to showcase how fine-tuning cannot affect the performance on the initial task. However, this is very specific to this particular crafted setting with extreme overparameterization. Therefore, it doesn’t really “suggest that the fine-tuning model is unable to forget the information it previously acquired from…” in general; it applies only to that particular model. Discussing the relations to the setup from Ding et al. (2024) would be interesting.

- The discussion following Theorem 3.2 and Theorem 3.4 feels somewhat repetitive, as the same points are made. You could instead focus more on discussing the differences between the two.
- There is no reference in the main text to the appendix.
- The norm in Eqs. 1, 2, and 3 is undefined.
- The point that "we favor the principle that regularization should prioritize remaining accuracy over unlearning accuracy" should be made before presenting the loss. Without it, the loss feels somewhat disconnected from the previous section.
- Consider introducing UA and RA earlier (perhaps as part of the problem description), as you present various results before their formal introduction.

Typos and small details/suggestions.

- (13), (38) (109), and a few others… typo inverted bracket ‘removing’ -> `removing’. 
- Table 1:  FT (Fine-Tuning) Methods. -> FT (Fine-Tuning) Method ?
- In Section 2, overparameterized linear regression should be defined when it is introduced (n<<d). (also typo overparamterized)
- You could define RL and UL with mathematical notation as they are introduced.
- Theorem 3.2 , missing reference to the proof in the appendix after the  theorem statement.
- Font of Figure 1 are too small.
- Introduce the notation $y’$ as a wrong label to before Eqn. 6.
- You can drop the line ``The evaluation metrics include Unlearning Accuracy (UA), MIA-Efficacy, Retaining Accuracy (RA), Test Accuracy (TA), and RunTime'' in Table 2 caption to save space.

questions:
1. For Figure 2a and the accompanying discussion and conclusions, it is important to note the fraction of overlapping features, as it likely has a significant impact on these aspects. Could you comment on this point?

2. Line 413: “Notably, the regularization parameter is typically constrained to the range (0, 1].” What is notable about that? Could you clarify the point of this comment? I couldn’t find any reference in (Fan et al., 2023) to bounding this parameter to that range.

3. Why is UA–RA more important than RA–TA? UA seems to relate to the training set.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732067489386

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We sincerely thank all reviewers and chairs for their valuable time and efforts in providing detailed feedback on our work. In particular, we appreciate the insightful reviews and constructive suggestions from Reviewers bHDY, h2Yv, NAAU, and J9SB, as well as the positive feedback from Reviewers bHDY and J9SB.

Moreover, we would like to address some common concerns raised by the reviewers here:

**1. Why choose overparameterized linear models and focus on class-wise unlearning experiments?**

Overparameterized linear models are widely adopted as a foundational framework for studying learning problems (e.g., transfer learning [1], continual learning [2], In-context learning [3,4]) and can be extended to more general settings such as neural tangent kernel (NTK) analysis. While the setting of overparameterized linear regression is indeed simplistic, it provides a valuable starting point for analyzing training dynamics, capturing the trajectory of learning rather than just upper or lower bounds. 
In this work, we start with the overparameterized linear regression model and hope to extend the analysis to more general cases such as multi-layer neural network in the future.

As for the focus on regression versus classification, although we use linear regression to analyze the fine-tuning process, our
assumptions about the data structure are based on a classification problem, which is why our experiments focus on class-wise forgetting.

**2. The novelty of the discriminative regularizer.**

We agree that there is limited distinction between our proposed regularization approach and existing objective functions, primarily differing in the choice of the loss function and the parameter $\alpha$. 

However, the main novelty of our paper lies in providing the first theoretical framework to understand the empirical phenomenon of why naive fine-tuning fails to forget. Based on the analysis in Sections 3 and 4, we aim to address this phenomenon by designing suitable objectives. Additionally, Theorem 4.1 provides the rationale for selecting the parameter $\alpha \in(0,1]$ for the unlearning loss, whereas in [1], the constraint $\alpha \in(0,1]$ is applied to the retaining loss. **While our regularizer may appear similar to existing formulations, our novelty/contribution extends beyond the regularizer to the broader theoretical and empirical insights we provide.**

In response to the insightful comments from Reviewers h2Yv and NAAU, we have rewritten Section 5 to emphasize the redesign rationale for the regularizer and its implications, rather than introducing it. Section 6 further validates our design through experimental results.

Thank you again for your valuable feedback!

References:

[1] Wu, Jingfeng, et al. "The power and limitation of pretraining-finetuning for linear regression under covariate shift." Advances in Neural Information Processing Systems 35 (2022): 33041-33053.

[2] Ding, Meng, et al. "Understanding Forgetting in Continual Learning with Linear Regression." arXiv preprint arXiv:2405.17583 (2024).

[3] Wu, Jingfeng, et al. "How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?." arXiv preprint arXiv:2310.08391 (2023).

[4] Chen, Xingwu, Lei Zhao, and Difan Zou. "How transformers utilize multi-head attention in in-context learning? a case study on sparse linear regression." arXiv preprint arXiv:2408.04532 (2024).
</content>
</thread_object>

</thread>


Metareview:
metareview:
Summary

This paper studies specially the fine-tuning problem in machine unlearning. As it is observed in many papers that fine-tuning using the retaining data only will not incur a good enough forgetting performance as the retrain, the paper explores further this phenomenon by using a simple linear regression model, and simulated simplified data, from both theoretical and empirical perspectives. The identified insights from these theoretical and empirical studies are that – when both retaining data and forgetting data are considered, more weight should be given to the retaining data. The paper then demonstrates empirically that such insights are effective.

Strengths

1. Reviewer(s) pointed out that the fine-tuning is efficient and does not require forgetting data, so it is worth exploring direction.
2. The paper provides some theoretical studies and insights.
3. Empirical results demonstrate the effective of the insights. 

Weaknesses

1. The whole analysis and empirical study to draw the most important insights and conclusions regarding fine-tuning is based on an oversimplified model (linear regression) and an oversimplified dataset (on which it is easy to tell different classes can be related to different features). It is not convincing that such a result or insight drawn is practical enough. 
2. The most important insight drawn is that more weight should be given to retaining loss instead of forgetting loss in machine unlearning. Such a conclusion is anti-intuition and is in contradiction with the main motivation of the paper that fine-tuning on retaining data sole is not effective in forgetting. This insight is claimed to be one of the most important contributions of the paper, but not sufficiently due to such contradiction exists – such contradiction may be an important observation (but the paper still needs to show that both losses are necessary but just one should be given fewer weights), or a result from the oversimplified study but not extendable to more practical cases. The analysis and study around this insight is not sufficient in the current paper. 
3. The studying point of the paper is some empirical evidence – when the retaining error is zero, the forgetting is far below the gold model (retraining model). I believe the problem with fine-tuning still exists, but an empirical study forcing the retaining error to be zero could be too strict and potentially could lead to overfitting. A more practical demonstration of the fine-tuning issue is recommended.  
4. While the conclusion of the paper still recommends using forgetting data, then the pure fine-tuning methods may not be worth studying from the beginning. The paper needs to find a better way to argue the meaning of studying the fine-tuning problem while forgetting data is finally used. If forgetting data is not used, then the fine-tuning problem is worth studying; but if the forgetting data is used, the story could be different – which has not been sufficiently discussed in the paper either. 

Recommendation

I recommend rejecting the paper at the current phase, due to the unsolved issues concerning the oversimplified model and problem setting, and the insufficient arguments about the contradiction of the insight provided from the theoretical results (especially given that the insight comes from an oversimplified study). On the other hand, I believe if the arguments and insights are considered in a more comprehensive way, the direction is worth a study even with the oversimplified theoretical studies. The theoretical studies themselves are good to start from something simple.