Title: Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classification
Venue: NeurIPS 2025
Accepted: no
Keywords: Machine Unlearning

Forget Vector

University Input Perturbation

Image Classification
Abstract: Machine unlearning (MU), which seeks to erase the influence of specific unwanted data from already-trained models, is becoming increasingly vital in model editing, particularly to comply with evolving data regulations like the "right to be forgotten". Conventional approaches are predominantly model-based, typically requiring retraining or fine-tuning the model's weights to meet unlearning requirements. In this work, we approach the MU problem from an input perturbation-based perspective, where the model weights remain intact throughout the unlearning process. We demonstrate the existence of a proactive input-based unlearning strategy, referred to forget vector, which can be generated as an input-agnostic data perturbation and remains as effective as model-based approximate unlearning approaches. We also explore forget vector arithmetic, whereby multiple class-specific forget vectors can be combined through simple operations (e.g., linear combinations) to generate new forget vectors for unseen unlearning tasks, such as forgetting arbitrary subsets across classes. Extensive experiments validate the effectiveness and adaptability of the forget vector, showcasing its competitive performance relative to state-of-the-art model-based methods while achieving superior parameter efficiency.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- Machine Unlearning (MU) aims to remove the influence of specific irrelevant data from a trained model. The proposed method achieves unlearning by adding specific vectors to input images. More specifically, this paper proposes a proactive input-based unlearning strategy called “forget vectors,” which are data perturbations that can be generated independently of the input Additionally, we propose a method that combines multiple class-specific forget vectors using simple operations (e.g., linear combination). Experiments using public datasets discuss the effectiveness of forget vectors.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n- The paper is well structured and easy to understand.\n- Machine learning is an important application of machine learning.\n\n# Weaknesses\n- Concerns regarding the scalability of the proposed method\n  - The proposed method calculates a forgetting vector for each class. The design of preparing at least one forgetting vector for all classes significantly limits the scalability of the proposed method.In the experiments, the effectiveness of the proposed method has only been evaluated for cases with a small number of classes, such as CIFAR10 and ImageNet-10. It has not been verified whether the proposed method is effective for larger classes. Furthermore, would the proposed method work in fine-grained cases? \n  \n- The connection between Sec. 4 (Generalization of MU to Forget Data Shifts) and other chapters should be made clearer.\n  - At the beginning of Section 4, the paper states, “Before designing the forget vector as formulated in (2), we examine the sensitivity of existing models.” However, the discussion in this chapter is limited to a very simple experiment on catastrophic forgetting. In my opinion, this does not constitute an analysis that demonstrates the validity of the proposed method design discussed in Section 5. \n  - Additionally, the use of a line graph in Figure 2 is logically inappropriate (there is no continuity between GN, ET, and PGD). \n\n- Insufficient experiments\n  - Detailed experiments and discussions on class combinations\n    - Adding a forgetting vector to the input image alone may limit the expressive power. The performance of forgetting may vary significantly with simultaneous forgetting of more classes (or data) in larger datasets. \n  - Furthermore, as already mentioned, comprehensive experiments on the effectiveness of the proposed method in fine-grained cases are lacking.\n\n- Numerical instability\n  - As shown in Figure 3(a), UAgap is unstable with respect to the weight magnitude (i.e., the difference between w1 and w2). This suggests numerical instability in the proposed method.\n\n- More detailed discussion is needed regarding the novelty of the proposed method.\n  - The proposed method claims to be inspired by visual prompting methods.However, a method using such a forgetting vector for MU has already been proposed. Therefore, I believe that additional discussion is necessary to claim novelty. (For example, in the following paper, MU is achieved by introducing a forgetting vector (^delta in this paper) for the input image (x in this paper). It is likely that many subsequent studies have proposed similar approaches.)\n  - The proposed method claims that its strength lies in not changing model parameters, but in the context of machine learning, is there an application advantage in having information about the forgotten classes or data remaining in the model parameters? \n  - T. Shibata et al. “Learning with Selective Forgetting.” IJCAI. Vol. 2. No. 4. 2021. (https://www.ijcai.org/proceedings/2021/0137.pdf)\n\nQUESTIONS:\nAs described in the weaknesses, more detailed discussion is needed on the following points.\n- (1) Scalability of the proposed method\n  - Can the proposed method claim effectiveness even for larger classes? Can the proposed method claim effectiveness even in fine-grained cases?\n- (2) Numerical instability\n - Figure 3(a) suggests numerical instability.A convincing explanation is required for this point. \n- (3) Novelty of the proposed method\n  - Please explain the novelty of the forgetting vector. Additionally, please explain the significance of not modifying existing model parameters in the context of machine learning.\n- (4) More comprehensive experiments\n  - Detailed experiments and discussions regarding class combinations are insufficient.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a data-driven machine unlearning using a forget vector. Unlike traditional unlearning methods that require model retraining or weight modification, this method utilizes input-agnostic perturbations to achieve unlearning. Furthermore, compositional forget vectors can be generated via linear combinations of class-wise forget vectors without the need for re-optimization.\n\nSTRENGTHS AND WEAKNESSES:\nStrenth\n- The motivation is clearly presented, and the paper provides detailed descriptions of the task formulation and evaluation methods.\n- The analysis of unlearning in OOD settings is interesting.\n- The method shows competitive performance without model retraining.\n\nWeakness\n- While the paper shows some robustness under certain data shifts, pixel-level perturbations may still be vulnerable to broader transformations. In contrast, feature-level perturbations like visual prompt tuning could offer stronger robustness. \n- While the paper provides some analysis on combining class-wise forget vectors, it does not examine how the method scales when the number of classes or the size of data to be forgotten grows significantly. The potential performance degradation or limitations in compositional unlearning as the forget set expands remain unclear.  \n- Input perturbation-based methods may not provide formal deletion guarantees compared to retrain-based methods.  \n- Experiments are limited to datasets with relatively small number of classes (e.g., CIFAR-10, ImageNet-10), which raises questions about scalability to larger, more diverse real-world datasets.\n\nQUESTIONS:\n1. How effective is the method in forgetting not just at the level of specific classes or subsets, but truly at the level of individual data points?\n2. How does the approach scale when the number of classes or the amount of data to be forgotten increases significantly? Can a large amount of data still be handled effectively with a fixed-size forget vector?\n\nLIMITATIONS:\nThe potential vulnerability to white-box adversaries is well acknowledged in the paper, with the need for future work on improving robustness clearly noted.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes machine-unlearning methodology which leaves the trained model unchanged but perturbs the model inputs through suitably selected additive noise. Notably, the noise is applied to *all* input data, including from the retain and test set. Such a forget vector is computed for each forget class. For random-sample unlearning, a suitable linear combination of the class-specific forget vectors is then used.\n\nSTRENGTHS AND WEAKNESSES:\n**STRENGTHS:**\n\n1. I believe this work is overall well written and well structured.\n\n2. The proposed methodology is novel to my knowledge\n\n3. The mathematical derivations seem formally correct.\n\n\n\n**WEAKNESSES:**\n\n1. I am not convinced that the proposed strategy is suitable for dealing with \"right to be forgotten\" legislation which the authors cite in the abstract. Of course, most known approximate unlearning techniques are unable to guarantee that the influence of \"to-be-forgotten\" data is fully erased and so may not be compliant either. But at least they *attempt* to remove the influence of these data whereas the approach presented here does not even make such an attempt. Worse, even, the forget set is used once more to find the forget vector(s).\n\n2. As the authors point out perturbing the inputs can lead to larger performance losses on the retain set than alternative methods. So, I'm not sure if the proposed method -- though interesting conceptionally -- is useful in practice.\n\nQUESTIONS:\n1. Can the authors explain the following sentence (from Line 222): \"The above indicates that a reduction in prediction performance on the forget set could translate into enhanced unlearning effectiveness on that set.\"?\n\n2. How does this approach scale with the number of classes overall and the number of classes in the forget set?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Forget Vectors (FV), a novel input perturbation-based approach to machine unlearning (MU), which aims to remove the influence of specific data points from a pre-trained model. The method learns an input-agnostic perturbation vector by jointly minimizing a margin loss on the forget set, a cross-entropy loss on the retain set, and a regularization term on the perturbation norm. At test time, the learned vector is added to the input image to induce forgetting of the unlearning data. The paper also introduces a new setting called compositional unlearning, in which FVs derived from different classes can be linearly combined with learnable weights to enable randomized data forgetting. Experimental results on CIFAR-10 and ImageNet-10 demonstrate that the proposed method effectively induces unlearning while preserving model performance on the retained data.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- [S1] **Novel unlearning paradigms.** The paper explores two underexplored directions in machine unlearning: (1) Data-based unlearning, which performs unlearning without modifying model parameters; and (2) Compositional unlearning, which dynamically adjusts the model's behavior to forget specific data subsets using the optimization results from individual unlearning requests. These are both creative and valuable contributions that expand the applicability of unlearning.\n- [S2] **Clear presentation and thoughtful analysis.** The paper is well-written and easy to follow. The methodology is explained with good clarity, and the experimental section includes insightful analyses, such as ablations and saliency-based visualizations, that help understand how the proposed method behaves in practice.\n\n## Weaknesses\n- [W1] **Insufficient motivation for data-based unlearning.** While data-based unlearning (e.g., via input perturbation) is a novel direction, its motivation remains unconvincing. Appendix H acknowledges a key vulnerability: these methods may be fragile under white-box adversaries who can simply remove or reverse the input transformation. Given this risk, the paper needs to establish strong, practical advantages that justify pursuing this direction. Currently, the primary implied benefit seems to be lower computational cost which, while important, may not be enough on its own unless clearly quantified and shown to hold across meaningful settings. Empirical demonstrations of additional benefits such as deployment efficiency, ease of integration, or robustness in black-box settings would help significantly.\n- [W2] **Missing discussion of computational savings.** Despite suggesting that a major advantage of data-based MU lies in reduced compute cost (e.g., Lines 102–103), the paper does not present any actual measurements or comparisons to support this claim. Given the relatively small scale of the image classifier architectures used, these savings might not be apparent. To meaningfully support this argument, the authors should test on larger models where tuning the model weights is expensive, or at least provide rough estimates of compute savings when using FVs.\n- [W3] **Weak empirical support for compositional unlearning.** While compositional unlearning is conceptually novel, its practical utility is unclear. Results in Table 3 suggest that directly applying the Forget Vector (FV) performs better than linearly combining class-level FVs in most cases, especially in preserving model utility. The paper also does not articulate a clear computational benefit to compositional unlearning (CU-FV) compared to direct FV. Without such an advantage, it is hard to justify using CU-FV at all. As it stands, the compositional setup seems theoretically interesting but practically unnecessary, which weakens its contribution.\n- [W4] **Limited experimental scope.** The paper's experimental design is somewhat narrow in several ways: (1) It focuses exclusively on image classifiers, while recent interest in machine unlearning is shifting toward generative models, which pose greater challenges and risks. (2) It does not evaluate robustness to counter-unlearning attacks, which have become increasingly relevant [A, B]. (3) It only tests a fixed forget ratio of 10%, leaving open questions about how well the method scales with varying degrees of unlearning. While some of these directions may be non-trivial to implement, exploring at least some of them more thoroughly would strengthen the paper’s significance.\n\n[A] Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning. ICLR 2025.\\\n[B] Catastrophic Failure of LLM Unlearning via Quantization. ICLR 2025.\n\nQUESTIONS:\n- [Q1] Are there any interpretable patterns observed in the trained forget vectors for each unlearned class when visualizing them on the pixel space?\n- Typo in Line 195: should be \"tend to\" instead of \"tent to\"\n- Typo in Line 316: did you mean \"random data forgetting\" instead of \"data-wise forgetting\"? The following sentences all seem to refer to that setting as the former.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 3, "DATE": "2025-07-09"}