NegMerge: Consensual Weight Negation for Strong Machine Unlearning

Authors: Hyoseo Kim, Dongyoon Han, Junsuk Choe
Primary area: transfer learning, meta learning, and lifelong learning
Keywords: Machine Unlearning, Image Classification, Model Merging

Abstract:
Machine unlearning aims to selectively remove specific knowledge from a model. Current methods, such as task arithmetic, rely on fine-tuning models on the forget set, generating a task vector, and subtracting it from the original model. However, we argue the effectiveness of this approach is highly sensitive to hyperparameter selection, necessitating careful validation to identify the best model among many fine-tuned candidates. In this paper, we propose a novel method that leverages all given fine-tuned models rather than selecting a single one. By constructing task vectors from models trained with varied hyperparameters and merging only the components of the task vectors with consistent signs, we perform unlearning by negating the merged task vector from the original model. Given that existing methods also utilize multiple fine-tuned models, our approach delivers more effective unlearning without incurring additional computational costs. We demonstrate the effectiveness of our method on both vision-language models and standard image classification models, showing improved unlearning performance with minimal degradation on the retain set, outperforming state-of-the-art techniques.

Decision: reject
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730394361528

thread_type:
official_review

<thread_object>
creator: reviewer_6rn3

<content>
summary:
The article introduces a model aggregation method for unlearning, which relies on task arithmetic. It uses multiple sets of hyperparameters to generate several task vectors for a single forget task, then merges them. During this merge process, parameters with sign conflicts are discarded, as they show low correlation with the forget set. Experiments demonstrate that this method is more effective in unlearning compared to other model merge techniques.

strengths:
*   The paper proposes aggregating multiple models in unlearning, which is more advantageous than using a single model.
*   The method is simple and effective.

weaknesses:
*   The paper lacks an explanation of why the proposed method specifically targets unlearning. From the description, it seems more like an optimization method for task vectors.

*   Naturally, this raises further questions about how this model merge method performs on other tasks, such as multi-task adding and task analogies in task arithmetic.

*   The comparison of computational complexity is puzzling. The paper discusses the complexity of the validation process, but the training process's complexity is of greater concern. Moreover, for vanilla task arithmetic, multiple models are not used, so the (O(mn)) notation is misleading.

questions:
*   The paper uses an averaging method to obtain the final task vector. Has there been any exploration of other methods, such as taking the maximum (similar to MagMax) or minimum values?
*   Why was RandAugment used for the CLIP model instead of varying hyperparameters?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730458530853

thread_type:
official_review

<thread_object>
creator: reviewer_ebwf

<content>
summary:
This paper proposes a method for machine unlearning in deep learning models. Unlike existing methods that rely on a single fine-tuned model's task vector, NegMerge combines task vectors from multiple fine-tuned models trained with different hyperparameters, keeping only elements with consistent signs. This approach enables robust forgetting of the forget set while minimizing the impact on the retain set.

strengths:
* Easy to follow
* Simple yet effective performance : intuitive approach 
* Experiments with different archihtectures (e.g CLIP and ResNet) which are commonly used.

weaknesses:
* Lack of technical contribution 
: They introduce practical approach but still lacks the theoretical depth. The paper may need for more analysis why sign-consistency aligns with forget set or exploring alternative merging method. 
* Reliance on empirical evidence
: As mentioned in Section 4.3, the paper relies on empirical results regarding performance on the forget set and retain set. The experimental results show that while the method achieves effective unlearning on the forget set, it compromises performance on the retain set, showing degradation in preserving non-targeted information compared to baselines. Without further analysis, it remains unclear whether these findings generalize across diverse datasets and model architectures.

questions:
* As mentioned in the paper, only the final layer of CLIP's text encoder remains frozen during fine-tuning. Given that the CLIP model’s image-text alignment might still influence the unlearning process, I wonder if unfreezing the final layer would provide any additional benefits. Was this approach tested, or was it determined that freezing the final layer would not significantly impact the results? If so, what was the reasoning behind this choice?

* The method relies on multiple fine-tuned models, which can increase memory costs due to the need of several sets of weights. Compared to baselines, could you compare how computationally efficient this approach is and whether this efficiency is associated with performance improvements across the models?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730605920850

thread_type:
official_review

<thread_object>
creator: reviewer_s5f2

<content>
summary:
The paper extends the existing work of machine unlearning by generating a task vector based on a forget set for finetuning and subtracting it from the original model. The new proposal gives an approach to merge finetuned candidates by merging only the components of the task vectors with consistent signs. 
One advantage of the approach is computational efficiency in the merging step. 
The model both shows improvements on unlearning tasks and maintains performance on the retain set.

strengths:
The paper has sufficient numerical results to support the claim for advantages of the proposal on several tasks, including better unlearning, less degradation on the retain set, and computation efficiency. These experiments show concrete evidence, especially the appendix. 
For “relationship with TIES-merging,” it is stated that “elements with inconsistent signs across task vectors are more closely related to the retain set than the forget set.” This is an essential claim for this paper, which is indicated by aggregated metrics of two methods and visualization of model activation during inference.

weaknesses:
The evaluation set of forget and retain do not have clear separability and they may have overlap - ImageNet still requires knowledge relevant to the eight tasks of fine-grained datasets and the CIFAR-10 is sampled randomly. Though the higher accuracy for the current retain set can still be a positive evidence, some eval sets completely irrelevant to the forget set may be more strong.

questions:
The paper shows the feasibility of the theory. My personal intuition is that the most promising application of machine unlearning is safety and privacy. However, the experiments focus on some functional recognition and, thus, have a retain eval set with overlap with the forget set. Is there any reason for not having related experiments?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731026026714

thread_type:
official_review

<thread_object>
creator: reviewer_aekn

<content>
summary:
This paper focuses on the machine unlearning problem, which aims to selectively remove knowledge from a pre-trained model without retraining from scratch. Instead of subtracting a single task vector from a pre-trained model, this paper proposed to use multiple task vectors with different hyperparameters. Multiple task vectors are merged by averaging elements with the same signs, while the remaining elements with different signs are replaced with zeros. Experimental results showed the effectiveness of merging multiple models during the unlearning process. More importantly, the elements with the same signs have more impact than the elements with conflict signs.

strengths:
* This paper tackles an important and relatively new topic of machine unlearning. It would be very impactful to remove specific data slices from a pre-trained model, without retraining from scratch due to the training cost. 
* This paper proposed an intuitively simple idea to merge multiple fine-tuned task vectors into a single one. Empirical results showed the effectiveness. 
* Not only the classification results are reported, but also the Membership Inference Attack (MIA) metric is used to assess privacy protection. This could be more useful in practice than just checking classification results.

weaknesses:
* The core idea of using task vector negation for machine unlearning was proposed in (Ilharco et al., 2022). The idea proposed in this paper is a bit incremental, though coming up with this simple idea could still be non-trivial. It would be more convincing to augment this paper with more in depth discussion and analysis of why the proposed method could work.
* Evaluation of a few design choices are missing, please refer to the following questions section for more details.

questions:
* Could you provide stats about how much ratio of the merged task vector becomes ratio? Especially when tuning the number of task vectors (e.g. 5, 10, 20, 30), would that have a significant impact on the zero ratio of the merged vector?
* How do you decide on the hyperparameters? For example, augmentation variants are used for CLIP finetuning but weight decay / epochs / label smoothing are used for standard image classifications. Is it sensitive to choose the hyperparameters to create a set of task vectors? What're the recommendations for a practitioner to create variants?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732527590342

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We thank all the **Reviewers AEkn, s5f2, EBwf, 6Rn3** for their thorough and constructive reviews with valuable and insightful advice. We appreciate the positive comments from all reviews:

**Reviewer AEkn** - 1) Importance of addressing machine unlearning. 2) Simple yet effective approach. 3) Use of MIA metrics to evaluate privacy protection in addition to classification performance.

**Reviewer s5f2** - 1) Computational efficiency of the merging step. 2) Robust numerical results supporting our claims. 3) Concrete evidence provided by our experiments. 

**Reviewer EBwf** - 1) Simple yet effective and intuitive approach. 2) Practicality and versatility of our approach through experiments across various architectures, emphasizing its applicability.

**Reviewer 6Rn3** - 1) Superiority of using multiple models over a single model. 2) Simple yet effective approach.

We have made every effort to address all the comments by carefully revising the paper. Throughout the revised paper, we highlighted the newly added or edited materials in blue. Please refer to the following summarization of the materials as follows:

- Provided theoretical insights are provided in Appendix F (**Q2** by **Reviewer AEkn**, **Q1** by **Reviewer EBwf**).
- Added experiments and analysis on the ratio of zeroed elements in Appendix C (**Q3** by **Reviewer AEkn**).
- Added experiments that show the robustness of the method on a larger model pool in Appendix D.1 (**Q4** by **Reviewer AEkn**, **Q5** by **Reviewer 6Rn3**).
- Provided recommendations for creating variants in Appendix D.2 (**Q5** by **Reviewer AEkn**).
- Clarified that it is appropriate to compare the effectiveness of techniques based solely on their forget set performance for the CLIP unlearning scenario  in Section 4.2. (**Q2** by **Reviewer EBwf**).
- Added experiments on a wider range of architectures and datasets in Appendix A.3 (**Q3** by **Reviewer EBwf**).
- Clarified the minimal impact of freezing the final layer of CLIP on performance in Section 4.2 (**Q4** by **Reviewer EBwf**).
- Clarified the memory and computational efficiency of the method in Appendix E (**Q5** by **Reviewer EBwf**, **Q3** by **Reviewer 6Rn3**).
- Added an ablation study to derive the improved final task vector in Appendix A.2 (**Q4** by **Reviewer 6Rn3**).

Detailed responses to other comments are left in individual comments.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper addresses the machine unlearning problem, aiming to selectively remove knowledge from a pre-trained model without retraining. The paper proposes generating multiple task vectors using different hyperparameters based on a forget set and merging them efficiently. Experimental results highlight the importance of elements with consistent signs in the task vectors.

After the rebuttal, this paper received four scores slightly above the borderline, but there are still some important concerns that remain unaddressed.

The authors state that the motivation for effective unlearning is that "the fine-tuned model should maintain high performance on the forget set while avoiding degradation on the retain set." However, I believe this is not the core objective of unlearning. As discussed in prior work, such as Task Arithmetic, the key goal of unlearning is for the final model (after applying unlearning techniques) to maintain performance on the retain set while effectively removing knowledge related to the forget set. Therefore, avoiding degradation on the retain set is crucial for the final model, not necessarily for the fine-tuned models used during intermediate steps. Section 4.3 does not adequately address this distinction, and other methods have demonstrated good results even when remaining accuracy is relatively low.

While the paper proposes an intriguing approach of aggregating only same-sign parameters for unlearning, several critical questions remain unanswered, leaving some uncertainty about the true impact of this method: Does obtaining fewer parameters with the same sign correlate with poorer unlearning? Does obtaining more same-sign parameters correlate with better unlearning? Won't parameters with opposing signs cancel out to some extent in expectation? If so, what degree of cancellation is significant for effective unlearning?
It appears that this paper essentially proposes averaging k models to obtain the unlearning task vector. However, the theoretical depth and contribution of this approach to unlearning remain open for further discussion.

Finally, regarding inference time complexity, the paper emphasizes that the task vector acquisition phase can reduce complexity to O(m) compared to Task Arithmetic (which involves m hyperparameters with n models). However, this paper requires a much larger n (e.g., 30 models), and the computational cost of fine-tuning 30 models far outweighs the evaluation on the forget set.

Therefore, the ACs believe this paper still requires further theoretical justification to fully address the remaining concerns.