Paper: UGradSL: Machine Unlearning Using Gradient-based Smoothed Label (ICLR-2024)

Reviewer comment (Writing):
There is a confusing claim made right after Definition 1: “The MU performance of retrain using iterative training is sensitive to different hyper-parameters and the distribution of the prediction on $D_f$ is not random enough as given in Section 5, showing that retrain is effectively an approximation of exact MU.” This contradicts Definition 1 in which retraining is defined as the gold standard of exact unlearning. The statement is likely not what the authors intended but it is very confusing to readers.

Author response:
As mentioned above, one (not all) of the retrain models can be the exact MU model, which is what Definition 1 tells us. For example, we can train CIFAR-10 using ResNet for 1, 10, and 100 epochs. They are all from the distribution of the retraining model. We don’t think they are all exact MU, or maybe none of them are exact MU. As stated above, given a fixed forgetting set and model, the prediction distribution on $D_f$ from the exact MU model should be determined. The exact MU should treat the forgetting dataset as never seen before at the sample level. Please note that we have a sentence below Definition 1 showing that retrain is effectively an approximation of exact MU, meaning that most retrain is effectively an approximation of exact MU, which is not exact MU because exact MU is very hard to reach and determine in the deep learning scenario. Exact MU is a further case of the retrain model in general.

Reviewer comment (Evaluation):
Using higher entropy $H$ on the prediction probability of $D_f$ as a privacy evaluation metric is also unreasonable. Even if a sample is completely removed, the retrained model could still have low $H$ in scenarios such as when datasets contain points identical to class centroids that are well-clustered. In such cases, removing one sample and retraining would leave decision boundaries unchanged and the model would remain confident (low $H$) on the removed sample. Here, an unreasonably high $H$ would actually hurt privacy by indicating that the model was altered in a way other than retraining. This counter-example illustrates the issue with using $H$ as a privacy metric.

Author response:
Thanks for pointing it out. I understand your perspective is from privacy protection, making forgetting datasets more distinguishable. The entropy metrics are from the information theory perspective. The extreme of forgetting is to randomly guess, which is our idea about exact machine unlearning. For exact MU, the model should behave as not seeing the sample completely, which should randomly guess. As for your concerns about privacy protection, we also have the experiment shown in Section D.10 in the Appendix. The prediction on the forgetting dataset and testing dataset are almost the same. They are both unseen datasets, which should be reasonable.

Reviewer comment (Novelty):
Forgetting based on linearization of deep networks was already studied in [2,3] which is not cited in the paper, nor is the method compared against.

Author response:
Thanks for your advice. We will add these references and compare these methods in our next version.

Reviewer comment (Theory):
[3] proposed a method where they linearized a network, such that the optimization problem is convex, such an approach could be extremely useful with the theoretical framework provided in the paper, as it would make the theorems more applicable and rigorous.

Author response:
Thanks for your advice. We will look at them very carefully and seriously consider integrating the framework into ours in our next version.

Reviewer comment (Theory):
The proposed method is an approximate unlearning method, however the paper does not provide a bound on the error as a function of unlearning requests. This is an imperative result for approximate unlearning algorithms.

Author response:
Thanks for your question. Our method is based on the existence of ideal retraining in close form. This expression is hard to get. The current widely accepted definition is given in Definition 1. It is hard for us to provide the error range based on the expression in Definition 1.

Reviewer comment (Experiments):
To make the theoretical results be more compatible with the empirical results the authors can consider fine-tuning (as the model will be more convex) compared to training from scratch where the landscape is highly non-convex.

Author response:
Thanks for your advice. UGradSL+ is a method based on the finetune. Actually, all our methods are based on the pre-trained model. Thus, it can be regarded as locally convex as what we mention in the first answer.

Reviewer comment (Evaluation):
It will be interesting to see how the weights in different layers move after learning, i suspect that the last layer moves the most while the lower layers wont change much in value, this could correspond to unlearning in the activations but not the weights. Having such a plot could be interesting for analysis.

Author response:
Thanks for your advice. We will seriously consider adding this plot in our next version.

Reviewer comment (Theory):
[4] has shown that the algorithm and definition proposed by Bourtoule et al may not be sufficient against adaptive unlearning requests, which may require shades of DP( differential privacy) in the unlearning algorithm. How do you plan to add this to the existing approach?

Author response:
Thanks for your advice. Our settings is not based on the adaptive unlearning framework. But your suggestion can be a direction of our future work.

Reviewer comment (Evaluation):
All the empirical results measure the unlearning in the final activations of the model, which does not consider unlearning from the weights of the model. How do you ensure that the information is removed from the weights? and how do you plan to show it empirically?

Author response:
First, evaluating the unlearning performance based on weight is unreasonable. According to [1], the same output can be from two models with totally different weights. Thus, a model which is not unlearned is also likely to predict similar results as the one unlearned. That's the reason why we use the output rather than the weight. As for the information removal, as we mentioned in Evaluation Metrics and [2], A higher MIA-Efficacy implies less information about $D_f$ in $\theta_u$ where $D_f$ is the forgetting set and $\theta_u$ is the unlearned model weight.

Reviewer comment (Writing):
Some issues exist in the presentations, like notations and terminologies. For example, the term Unlearning Diagram is not explained either on page 7 or the appendix reference is missing on page 9.

Author response:
(c) Thanks for your advice. We will recheck the notation and terminologies in our next version. As for the Unlearning Diagram, it is a typo of the Unlearning Paradigm.

Reviewer comment (Novelty):
This paper claims that the group unlearning is one of the contributions of this paper. However, this type of unlearning task has been discussed in [1].

Author response:
(d) Thanks for your advice. We will modify the contribution list.

Reviewer comment (Evaluation):
The evaluation part of this paper mentions that the higher unlearning errors, remaining accuracies, and test accuracies stand for the better unlearning performances for all unlearning tasks. However, some other papers like [2] which this paper cites, [3], and [4], all mention that the closer to the models that are trained on the remaining data will be better.

Author response:
(e)  Thanks for your question. We have noticed that references [2-4] do use a retrained model as the baseline. However, it should be noted that we often cannot access the retrained model in real usage, especially for industry-level large language models. Some of the methods are sensitive to hyperparameters like the number of epochs, learning rate, etc. It is impossible to find a more suitable parameter that makes the performance of the unlearned model closer to the retrained model because we don’t have the retrained model in real usage. Thus, we believe that we cannot use the performance of the retrained model as a baseline. For the reason why the higher is the better, it is natural to have the unlearned model hurt the testing or remaining performance as little as possible. For the classification task, if the fewer data points the unlearned model can remember, the higher the unlearning errors and the better the unlearning is.

Reviewer comment (Theory):
Why use the approximately equal symbol in eq.(8)?

Author response:
Questions:

(a) Because Eq. 8 is the subtraction of Eq. 6 and 7. Eq. 6 and 7 are the approximations of Taylor Series which are approximately equal. We will revise this statement in our next version.

Reviewer comment (Theory):
How to understand if the model outputs random predictions, ⟨a-b,c-b⟩ ≤ 0 will hold?

Author response:
Questions:

(b) Thanks for your question. First, a, b, and c are vectors here with directions. Random outputs mean that the gradient of the target label and the non-target label are diverse or not in the same direction because every time, the largest gradient will point to different labels (directions), which is what <a-b, c-b> tells us.

Reviewer comment (Reproducibility):
What are the training details of the proposed methods, especially for the alpha, E, and p? Does this paper use any pre-train parameters for the models?

Author response:
Questions:

(c) For different datasets, the alpha, E and p are different which shows the different difficulties to unlearn. For example, class-wise unlearning should be easier than random unlearning. Alpha should be smaller in random unlearning. But it should be noted that our method does not rely on the unseen data because our metrics are higher, the better. Thus, it is suitable for the method always to find a better parameter as given in Section D.9 in the Appendix.

No. Our method only relies on the pre-trained model and does not rely on the pre-trained parameters.

Reviewer comment (Experiments):
This paper has mentioned 5 baselines. What are the standards for selecting baselines and why some results of these baselines are missing in experiments?

Author response:
Questions:

(d) We selected the baseline methods mainly based on [1] and boundary unlearn, which is accepted in CVPR 2023. The reason why we have some missing values is that some of the methods only work on one modality or the method is too slow for a very large dataset like ImageNet, CelebA. Our methods are general for both vision and language. 

[1] Jia et al. Model sparsification can simplify machine unlearning[J]. arXiv preprint arXiv, 2023.