Paper: Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models (ICLR-2024)

Reviewer comment (Theory):
I have a concern about the objective (2). In my opinion, data forgetting is not to maximize the distribution discrepancy to forgotten data. The relationship may be more complex.

Author response:
**W2: Concern about the objective (2). Data forgetting is not to maximize the distribution discrepancy to forgotten data. The relationship may be more complex.**
​
>We agree with the reviewer that "Data forgetting is not to maximize the distribution discrepancy to forgotten data. The relationship may be more complex". In light of this, our approach to unlearning aims to condition the model to behave as though it was never trained on the forgetting data. To achieve this, we have established two constraints within our unlearning objective. 
># 
>The first is to minimize the distribution discrepancy relative to the remaining data, as delineated in Eq.1. This step is pivotal for preserving the maximal amount of knowledge from the remaining data's distribution. The second constraint, outlined in Eq.2, is to maximize the distribution discrepancy concerning the forgetting data. This approach is integral to effectively diminishing the influence of the forgetting data distribution on the model. These two complementary objectives are designed to navigate the complexities of data forgetting, ensuring a more thorough and nuanced unlearning process.

Reviewer comment (Novelty):
The VAE seems redundant. It is utilized to output the features on the given dataset. However, the original model can be directly used instead of training an extra model. In fact, (4) and (6) just mimic the features of the original model. So, the VAE can be viewed as an equivalent version of the original model.

Author response:
**W3: The VAE seems redundant because it is utilized to output the features on the given dataset. Thus, the VAE can be viewed as an equivalent version of the original model.**

>Thank you for your insightful comment. We understand the concern is about the potential redundancy of VAEs; however, the role of VAEs in our framework is significantly different from simply utilizing the features extracted from the original model.
># 
>Firstly, VAEs are probabilistic models that offer a distinct advantage in our context. Unlike the deterministic feature extraction of the original model, VAEs are designed to learn the underlying distribution of the extracted features. By training a VAE on these features, we are not just capturing the data points but also modeling the probability distribution that these points follow. This probabilistic representation is pivotal as it allows us to address the unlearning challenge from a distribution perspective, which is not feasible with the deterministic features alone.
># 
>Secondly, the unlearning process we have designed plays a crucial role in differentiating our approach. This process involves carefully designed objectives to maximize the discrepancy in representations of the forgetting data and minimize it for the remaining data. Such a targeted approach in the unlearning process, leveraging the distributional capabilities of VAEs, is fundamentally different from and more effective than a straightforward application of the original model's extracted features. It enables a more effective modification of the model in response to the unlearning requirement, ensuring that the forgetting data's influence is minimized while preserving the integrity of the remaining data.
># 
>In summary, the use of VAEs in our framework is justified by their ability to model data distributions probabilistically and our specific design of the unlearning process that exploits these probabilistic representations to achieve effective unlearning. This approach goes beyond the capabilities of merely using the original model's extracted features, providing a more sophisticated solution to the unlearning problem.

Reviewer comment (Writing):
Several typos, e.g., jointly using $g_u^e$ and $g_U^e$. P in (1) should be P_r.

Author response:
**W4: Several typos, e.g., jointly using $g_u$ and $g_U$. $P$ in (1) should be $P_r$.**
​
>We appreciate your attention to detail in identifying these typographical errors. We have corrected the instances where $g_u$ was incorrectly used instead of $g_U$, and have also amended $P$ to $P_r$ in Eq.1. We have conducted a comprehensive review of the entire manuscript to ensure that similar errors are rectified and the accuracy of our paper is maintained. Thank you for your valuable feedback.

Reviewer comment (Theory):
No proper explanation in the optimization process. Why some terms are dropped and even if they are considered what could have happened?

Author response:
**Q2 and W1: - Is the formulation as VAE necessary if dropping the terms of KL in the final minimization? - Why some terms are dropped and even if they are considered what could have happened?**
># 
>The VAE plays an integral role in modeling the representation distributions, which is essential for our approach. This necessity is evident even when we opt to drop the KL divergence terms during the unlearning stage. Our method leverages the full potential of VAE, utilizing its complete training loss (i.e., Eq.4 and Eq.6) to accurately capture these distributions. This approach differs fundamentally from using a single encoder, underscoring the indispensability of VAE in our model.
># 
>In the optimization phase of VAE, the KL divergence term is crucial for effectively projecting input data into a latent Gaussian variable. Thus, during VAE training, we retain the KL divergence terms to optimize the VAEs with the complete loss function. This step ensures accurate modeling of the data distributions, which is vital for the subsequent unlearning process.
># 
>For the extractor unlearning phase, we strategically drop the KL terms. This decision is based on our objective to enhance unlearning performance. Preliminary results on the DIGIT and FASHION datasets demonstrate that omitting these KL terms during unlearning leads to better outcomes. The comparison of performances with and without KL terms, i.e., "LAF" and "Add KL" respectively,  provides empirical evidence supporting our approach.
># 
>| **Data** | **Method** | **Train_r**     | **Train_f**    | **Test**        | **ASR**       |
>|----------|------------|-----------------|----------------|-----------------|---------------|
>| DIGIT    | Retrain    | 99.56±0.05      | 98.84±0.10     | 99.04±0.10      | 49.80±0.53    |
>| DIGIT    | Add KL     | 53.36±3.54      | 85.78±5.14     | 58.90±1.40      | 41.19±0.32    |
>| DIGIT    | **LAF**    | **98.03±0.68**  | **97.29±1.43** | **97.30±0.78**  | **47.92±0.84**|
>| FASHION  | Retrain    | 96.43±0.35      | 92.15±0.41     | 90.23±0.22      | 47.32±0.76    |
>| FASHION  | Add KL     | 59.97±0.06      | 11.93±2.94     | 48.93±0.23      | 41.57±0.06    |
>| FASHION  | **LAF**    | **91.54±2.67**  | **90.91±7.00** | **87.53±3.26**  | **46.89±0.88**|
># 
>The bold results are the closest ones to the results of the retrained model. More comprehensive results and discussions are detailed in Tables 2, 3, and 4, Appendix 5.1 of our revised supplementary material.


**W1: No proper explanation in the optimization process.**

>The optimization process utilizes two key loss functions: extractor unlearning loss and representation alignment loss. In each iteration of the optimization, we optimize the two losses alternately. Below we give a brief overview of the optimizations of the two losses on the Fashion dataset. The details of other datasets have been provided in Appendix 3.
># 
> 1. **Extractor Unlearning**:
>    - **Two Objectives**: Introduce objectives to remove knowledge related to $D_f$ by maximizing the discrepancy between the well-trained model's representation and the unlearned representation on $D_f$, and remaining knowledge related to $D_r$ by minimizing the discrepancy between the well-trained model's representation and the unlearned representation on $D_r$.
>    - **Optimization**: We first train two VAEs, $h$ on training data and $h_f$ on forgetting data. We then merge these two objectives into a single one, dropping the KL terms. In each optimization iteration, we sample a batch of remaining data to minimize the  normalized L2 loss corresponding to $h$, and sample another batch of forgetting data to maximize the normalized L2 loss corresponding to $h_f$.
>    - **Optimziers**: Adam optimizier with learning rate as 5e-5.
># 
> 2. **Representation Alignment**:
>    - **Mitigating Approximation Effects**: Adjust the unlearned representation extractor to align with the original classifier on the classification level.
>    - **Preserving Predictive Performance**: Introduce a contrastive loss that aligns post-unlearning representations with pre-unlearning ones.
>    - **Optimization**: In each optimization iteration, we sample a batch of remaining data and another batch of forgetting data to calculate the cosine similarity loss between the representations of the original and updated models. Then the two cosine similarity loss are merged into the representation alignment loss for minimization, where the cosine similarity loss of remaining data will be reduced and the cosine similarity loss of forgetting data will be enlarged.
>    - **Optimziers**: Adam optimizier with learning rate as 1e-4.

Reviewer comment (Writing):
I think there is a typo in equation 1. The distribution should be P_r instead of P. Similarly for D. Please change the typo. If not please provide an explanation. A lot of notational errors are there such as from argmin equation.8 suddenly becomes argmax in equation.9.

Author response:
**W2: Typos in equation 1 and other notational errors in Section 3.1**
​
>We apologize for all the typographical errors and thank the reviewer for pointing out them.
># 
>The distribution should be $P_r$ and the condition should be $g_{U}^{e}(x)\sim Q(D_r)$ because the unlearning objective requires preserving the knowledge of remaining data. The argmax in equation.9 should be argmin.
># 
>We have thoroughly checked all the notation parts of our paper to ensure the notation correctness of the technical part. We want to thank the reviewer again for bringing this up.

Reviewer comment (Experiments):
There is also a zero-shot method of unlearning which does not assume access to the data at all. This raises questions about the experiment settings. Here you need to have access to the retain and forget set to model with VAE. Is not this assumption too much as in many real-world applications the training dataset remains hidden? So in zero-shot settings, as no data is needed unlearning methods don't assume access to the labels.

Author response:
**W3: Comparison with the zero-shot method of unlearning which does not assume access to the data.**
​
>We agree with the reviewer regarding the importance and potential benefits of a fully zero-shot unlearning approach. Such a method , which does not rely on access to either forgetting or remaining data, including labels, represents an ideal paradigm in unlearning scenarios. It stands out for its minimal reliance on data, aligning well with the increasing emphasis on data privacy and security. 
># 
>The field of zero-shot unlearning is in its early stages, primarily focusing on class-unlearning problems. The current methods developed in [1] are limited to unlearning entire classes and cannot extend to more complex scenarios. While these methods demonstrate effective forgetting of data, their performance on retaining the information of remaining data appears constrained. This limitation could be attributed to their restrictive data access. As demonstrated below, on class-unlearning tasks, the results indicate that while forgetting is achieved effectively by the zero-shot method GKT, there is room for improvement in preserving the accuracy of the remaining data.
># 
>| **Data**  | **Method** | **Test_r**       | **Test_f**       | **ASR**          |
>|------------|-----------|------------------|------------------|------------------|
>| DIGITS    | Retrain    | 98.81±0.15       | 0±0              | 26.49±1.41       |
>| DIGITS    | GKT        | 93.21±0.21       | 0±0              | 50.02±0.04       |
>| DIGITS    | **LAF**    | 98.03±0.68       | 0.26±0.11        | 52.25±2.61       |
>| FASHION   | Retrain    | 92.66±0.29       | 0±0              | 38.24±3.13       |
>| FASHION   | GKT        | 43.19±11.67      | 0±0              | 47.63±0.02       |
>| FASHION   | **LAF**    | 91.54±2.67       | 2.46±1.46        | 31.35±0.71       |
># 
>Recent research efforts, as seen in [2], have been directed towards reducing dependency on the remaining data. However, these approaches still necessitate specifying the data and labels to be forgotten, to generate appropriate training loss and gradients. This requirement indicates that while progress has been made in reducing data dependency, the field has not yet reached the stage of complete zero-shot unlearning, where no data specification is needed at all. In addition, these methods still require label information, which contrasts with our current work where label information is not a necessity for the unlearning process.
># 
> **References**
>
>[1] Chundawat, et al. Zero-shot machine unlearning. IEEE Transactions on Information Forensics and Security (2023).
>
>[2] Sekhari, et al. Remember what you want to forget: Algorithms for machine unlearning. NIPS 2021

Reviewer comment (Theory):
In equation 4 the term that is minimized how does it fit a VAE framework. In VAE we maximize the ELBO. Now If we take the KL term in Eq.-4 to be the Corresponding KL term in ELBO how does the first term correspond to the first term in ELBO? A detailed explanation will be better because in VAE we want to maximize the log-likelihood of data and then formulate an ELBO term. In this case, a detailed derivation with proper assumption will help to see how it is a VAE. What do we want to maximize here?

Author response:
**Q1: In VAE we maximize the ELBO. Now If we take the KL term in Eq.-4 to be the Corresponding KL term in ELBO how does the first term correspond to the first term in ELBO? A detailed explanation will be better because in VAE we want to maximize the log-likelihood of data and then formulate an ELBO term. What do you want to maximize here?**
​
> Thank you for your question regarding the formulation of our VAE and its relationship to the ELBO. Your query touches on the core of how we have adapted the VAE framework for our purposes. Let us provide a detailed explanation below. For clarity and ease of understanding, this response employs simplified notations that adhere to the conventional standards of VAE. Please note that these notations may differ from those used in our paper. 
>    
>    In the standard VAE framework, the objective is to maximize the ELBO, which is formulated as:
>    $$
>    ELBO = \mathbf{E}\_{z \sim q\_\phi(z|x)}(\text{log } p\_\theta(x|z)) - KL(q\_\phi(z|x) || p(z)).
>    $$
>    In our approach, the L2 term in our equation (Eq. 4) can be seen as analogous to the first term of the ELBO. This term represents the expected log-likelihood of the data, given the latent variables.
>    
>    The derivation of our approach starts with the assumption that the decoder in a VAE, particularly for real-valued inputs, can be approximated as a Gaussian decoder, where $p_\theta(x|z) \sim \mathcal{N}(\mu_\theta(z), \sigma_\theta(z)^2)$. This assumption is based on [1], which posits that the negative log-likelihood of $p_\theta(x|z)$ in this context can be represented as:
>    $$
>    -\log{p_\theta(x|z)} = \frac{1}{2\sigma^2}||x - \hat{x}||^2 + D\ln{\sigma} + C,
>    $$
>    where $\hat{x}$ is the output of the decoder, and $D$ and $C$ are constants.
>    
>    Following the insights from [2,3], the optimization process typically treats $\sigma$ as fixed, leading to the simplification of the log-likelihood term to an L2 loss:
>    $$
>    -\log{p_\theta(x|z)} = \frac{1}{2}||x - \hat{x}||^2 + C.
>    $$
>    Thus, we choose the L2 loss as the reconstruction loss in the optimization process, aligning with the maximization of $\log{p_\theta(x|z)}$ and, consequently, the ELBO.
>
>    
> In addition, we have reformulated the first term in our approach to resemble the log-likelihood form for ease of understanding and to highlight its parallel to the ELBO maximization in VAEs. Thank you again for pointing out this issue. 
>
> **References**
>
> [1] Kingma, et al. Auto-encoding variational bayes. ArXiv 2013.
>
> [2] Rybkin, et al. Simple and effective VAE training with calibrated decoders. ICML 2021.
>
> [3] Liu, et al. Towards visually explaining variational autoencoders. CVPR 2020.

Reviewer comment (Theory):
If my understanding is correct you drop the terms of KL in the final minimization. So it is simply an encoder without the decoder (as it is generally posed in the VAE framework). Is the formulation as VAE necessary?

Author response:
**Q2 and W1: - Is the formulation as VAE necessary if dropping the terms of KL in the final minimization? - Why some terms are dropped and even if they are considered what could have happened?**
># 
>The VAE plays an integral role in modeling the representation distributions, which is essential for our approach. This necessity is evident even when we opt to drop the KL divergence terms during the unlearning stage. Our method leverages the full potential of VAE, utilizing its complete training loss (i.e., Eq.4 and Eq.6) to accurately capture these distributions. This approach differs fundamentally from using a single encoder, underscoring the indispensability of VAE in our model.
># 
>In the optimization phase of VAE, the KL divergence term is crucial for effectively projecting input data into a latent Gaussian variable. Thus, during VAE training, we retain the KL divergence terms to optimize the VAEs with the complete loss function. This step ensures accurate modeling of the data distributions, which is vital for the subsequent unlearning process.
># 
>For the extractor unlearning phase, we strategically drop the KL terms. This decision is based on our objective to enhance unlearning performance. Preliminary results on the DIGIT and FASHION datasets demonstrate that omitting these KL terms during unlearning leads to better outcomes. The comparison of performances with and without KL terms, i.e., "LAF" and "Add KL" respectively,  provides empirical evidence supporting our approach.
># 
>| **Data** | **Method** | **Train_r**     | **Train_f**    | **Test**        | **ASR**       |
>|----------|------------|-----------------|----------------|-----------------|---------------|
>| DIGIT    | Retrain    | 99.56±0.05      | 98.84±0.10     | 99.04±0.10      | 49.80±0.53    |
>| DIGIT    | Add KL     | 53.36±3.54      | 85.78±5.14     | 58.90±1.40      | 41.19±0.32    |
>| DIGIT    | **LAF**    | **98.03±0.68**  | **97.29±1.43** | **97.30±0.78**  | **47.92±0.84**|
>| FASHION  | Retrain    | 96.43±0.35      | 92.15±0.41     | 90.23±0.22      | 47.32±0.76    |
>| FASHION  | Add KL     | 59.97±0.06      | 11.93±2.94     | 48.93±0.23      | 41.57±0.06    |
>| FASHION  | **LAF**    | **91.54±2.67**  | **90.91±7.00** | **87.53±3.26**  | **46.89±0.88**|
># 
>The bold results are the closest ones to the results of the retrained model. More comprehensive results and discussions are detailed in Tables 2, 3, and 4, Appendix 5.1 of our revised supplementary material.

Reviewer comment (Writing):
The explanation of the contrastive loss component, particularly the utilization of the "sim loss," lacks comprehensive coverage, leaving room for confusion regarding which sim loss should be used and why use it.

Author response:
In our proposed method, we utilize cosine similarity loss as the sim loss. We choose this cosine similarity because
># 
>- Normalization Advantage: Cosine similarity loss is inherently normalized, preventing exponential expansion during representation alignment.
>- Robustness in Deep Learning: Empirical evidence and research (as cited in references [1,2]) have demonstrated that cosine similarity loss is more robust and effective in deep representation learning contexts. 
>- High-Dimensional Representation: According to findings in reference [3], normalized representations can be interpreted as points on a high-dimensional spherical surface. The distance between two such embeddings is aptly captured by the cosine of the angle between them, hence the relevance of cosine similarity in our context.
># 
>We believe that these factors collectively make cosine similarity loss an optimal choice for our method. However, we are open to suggestions from the reviewer regarding alternative similarity measures. 
># 
>**References**
>
>[1] Chen, et al. A simple framework for contrastive learning of visual representations. ICML 2020.
>
>[2] Robinson, et al. Contrastive learning with hard negative samples. arXiv 2020.
>
>[3] Wang, et al. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. ICML 2020.

Reviewer comment (Theory):
The paper assumes that training data, forgetting data, and remaining data are sampled from different distributions. Could the authors provide an illustrative example or rationale for this particular setup to justify its validity?

Author response:
> Consider the MNIST dataset, which comprises images of handwritten digits, categorized into 10 classes (0 to 9), with each class containing 60,000 images. Suppose our objective is to unlearn the data associated with the digit '1'. 
> # 
> In this scenario, the 'forgetting data' is the subset of the MNIST dataset where the label is '1'. This represents a specific distribution: the conditional probability distribution of an image given it belongs to class '1'.
># 
>The 'remaining data', in contrast, would include all images from the other 9 classes (0 and 2-9). This subset has its distinct distribution, characterized by the conditional probability distribution over these nine classes.
># 
>The 'training data distribution', from which our model initially learns, is the aggregate or marginal distribution encompassing all 10 classes. It represents the broader, undifferentiated probability distribution of any image in the dataset, irrespective of its class.
># 
>In this example of class removal from the MNIST dataset, the training, forgetting, and remaining data can indeed originate from different distributions, each with its unique statistical properties.

Reviewer comment (Theory):
The proposed method involves two approximations, one pertaining to Equation 4 where "h()" operates on all training data rather than just D_r, and the other involving Equation 8 which removes the second part. Could the authors elaborate on how these approximations may potentially impact the final output and results of the method?

Author response:
>In our approach, the first approximation strategically replaces the remaining data with the complete training data set. This decision is primarily driven by an efficiency perspective: it allows us to utilize the well-trained existing model directly, avoiding the need for retraining with only the remaining data. This efficiency gain is crucial in practical applications where computational resources and time are significant considerations. The discrepancy, introduced by incorporating the forgetting data into the training set, can be effectively controlled and minimized through our proposed dual-loss optimization strategy. 
># 
>The second approximation in our methodology involves omitting two KL divergence terms from Equation 8, a decision made to enhance the unlearning performance of our model. This approximation is rooted in our focus on the distribution discrepancies during the extractor unlearning phase, specifically targeting the differences between the inputs and outputs of the Variational Autoencoder (VAE). In the context of our model, the KL divergence terms typically act as penalty terms, enforcing a regularization effect. By eliminating these terms, we aim to reduce the constraints on the model, thereby allowing a more flexible adjustment of the distribution during the unlearning process. 
># 
>To evaluate the analysis, we also add comparison experiments between our proposed LAF and its two without-approximation versions: "Add-KL" and "Remaining" in the following table on two datasets. The results relative to the Retrain demonstrate that after adding the KL terms, the performance dropped. Using the remaining data only may improve the performance a little bit, but still comparable to LAF. More results and details will be provided in Appendix 5.1, mainly in Tables 2, 3, and 4. 
># 
>To thoroughly assess our methodology, we conducted comparative experiments between our proposed method, LAF, and its two non-approximated versions: "Add KL" and "Remaining Data." The results on the two datasets are summarized in the table below. When compared to the baseline "Retrain" method, the incorporation of KL terms in the "Add-KL" resulted in a noticeable decrease in performance. On the other hand, using only the remaining data ("Remaining Data") shows a slight improvement in performance; however, it remains closely aligned with the results achieved by LAF. These findings are further elaborated with additional experiments and detailed analysis provided in Appendix 5.1, specifically in Tables 2, 3, and 4. 
># 
>| **Data** | **Method**        | **Train_r**        | **Train_f**        | **Test**          | **ASR**          |
>|-------------------|----------|--------------------|--------------------|--------------------|------------------|
>| FASHION  | Retrain           | 96.43±0.35         | 92.15±0.41         | 90.23±0.22         | 47.32±0.76       |
>| FASHION  | Add KL           | 59.97±0.06         | 11.93±2.94         | 48.93±0.23         | 41.57±0.06       |
>| FASHION  | Remaining Data    | **92.49±0.37**     | 90.17±1.57         | **88.22±0.42**     | 44.57±0.87       |
>| FASHION  | **LAF**           | 91.54±2.67         | **90.91±7.00**     | 87.53±3.26         | **46.89±0.88**   |
>| CIFAR10 | Retrain            | 84.03±0.20         | 78.05±1.34         | 87.20±0.65         | 57.48±0          |
>| CIFAR10 | Add KL            | 44.88±32.38        | 40.81±39.45        | 46.33±36.33        | 57.30±5.20       |
>| CIFAR10 | Remaining Data    | 77.70±0.67         | **75.59±1.81**     | 81.79±0.84         | 55.73±0.73       |
>| CIFAR10 | **LAF**            | **78.03±1.55**     | 73.30±3.96         | **82.22±2.57**     | **57.65±0.70**   |

Reviewer comment (Experiments):
The first weakness of this framework is its efficiency. To capture data distribution, a certain amount of instance $x$ and two distribution modeling are needed. Though the framework doesn't need retraining process, framework efficiency and computational workload are encouraged to study and present in the paper.

Author response:
> Thank you for your suggestion to include studies on the efficiency and computational workload of our framework. 
> # 
> In the revised paper, we have incorporated detailed results in Appendix 4.1, including time cost analysis (Figures 3 and 4) and workload analysis (Figures 5 and 6). In this rebuttal, we summarize our findings below on the SVHN dataset, showing the time and storage efficiency of our LAF and seven other baselines.  These results from the table demonstrate that our LAF method not only achieves a favorable balance in time efficiency for larger datasets and models, as evidenced by its average ranking but also maintains a manageable computational workload. 
> # 
> Despite the introduction of two VAEs into our framework, the total computational demand of LAF is comparable to, or even less than, other methods. This efficiency is attributed to the VAEs' simplistic structures, which contain less than 150K parameters in total. This is significantly lower compared to the parameter counts in more complex models such as CNNs (450K parameters) and ResNet-18 (11.3M parameters). This efficiency in both time and computational resources underscores LAF's practical applicability and versatility in handling various data-intensive scenarios
> # 
> | Method    | Time (s) | Time Rank | Storage (MB) | Storage Rank | Average Rank |
> |-----------|----------|-----------|--------------|--------------|--------------|
> | Retrain   | 2079     | 7         | 2998         | 1            | 4            |
> | NegGrad   | 21       | 1         | 3903         | 5            | 3            |
> | Boundary  | 870      | 3         | 3900         | 4            | 3.5          |
> | SISA      | 1722     | 6         | 6781         | 7            | 6.5          |
> | Unrolling | 88       | 2         | 19032        | 8            | 5            |
> | T-S       | 2660     | 8         | 3937         | 6            | 7            |
> | SCRUB     | 885      | 4         | 3809         | 2            | 3            |
> | **LAF**   | 905      | 5         | 3848         | 3            | 4            |

Reviewer comment (Experiments):
The quality of representation extractor may affect framework performance. More representation extractors are needed to be considered to enhance the soundness of this method.

Author response:
> We agree that evaluating the performance with various-quality representation extractors is essential to demonstrate the robustness of our proposal and have conducted experiments in data removal and class removal scenarios. Specifically, we extended our experiments to include models with less effective (low-quality) extractors, i.e. CNN [1] and ResNet [2] trained on noisy labelled data rather than clean data. The preliminary results on the FASHION dataset, as seen below, indicate that our LAF method, particularly the LAF-R variant, performs robustly, often achieving the best outcomes. More comprehensive results will be detailed in Table 3, Section 4.2 of our revised paper.
> # 
> | **Method**  | **Train_r**       | **Train_f**       | **Test**          | **ASR**          |
> |----------|-------------------|-------------------|-------------------|------------------|
> | Retrain     | 97.04±0.83        | 2.16±0.06         | 88.15±0.45        | 37.65±1.88       |
> | NegGrad     | 91.91±1.04        | 2.82±0.43         | 85.96±0.85        | 32.39±1.61       |
> | Boundary    | 72.82±6.71        | 11.21±1.79        | 54.54±10.37       | 30.58±1.19       |
> | SISA        | 92.22±5.95        | **1.69±0.06**     | **88.90±0.01**    | 25.00±0.06       |
> | Unrolling   | 61.73±1.83        | 3.76±0.83         | 80.02±3.85        | 33.97±1.31       |
> | T-S         | 85.56±3.13        | 5.64±1.32         | 74.19±5.23        | 28.86±0.78       |
> | SCRUB       | 87.29±1.35        | 4.29±0.50         | 79.41±2.28        | 33.32±0.26       |
> | **LAF+R**   | **93.42±0.44**    | 2.06±0.20       | 87.71±0.36        | **34.33±0.32**   |
> | **LAF**     | 92.32±0.66        | 4.80±0.71         | 81.21±1.22        | 22.36±0.72       |
># 
> **References**
>
> [1] Yann, et al. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks 1995
>
> [2] Kaiming, et al. Identity mappings in deep residual networks. In
ECCV 2016

Reviewer comment (Experiments):
The availability to access $x$ in training data under machine unlearning scenarios lacks of description. Is this practical in the real-world scenarios?

Author response:
> We appreciate the reviewer pointing out the need for clarification on the availability of training data in our framework. Our method indeed necessitates access to a subset of training data subsampled uniformly to conveying information of the training data distribution. This is essential for training the Variational Autoencoder (VAE).
> # 
> 
> It is noteworthy that in many academic studies, the assumption of having access to training data, or at least a subset of it, is a common approach in the domain of deep model unlearning, as evidenced by several key publications in this field [1,2,3,4]. While there are efforts to reduce dependency on training data [5], this method limited generally limited to scenarios like class removal and are not feasible for tasks such as data removal or noisy label removal. In addition, [6] is a zero-shot method which do not require both training and forgetting data. But it can only be limited to class removal and its performance drops significantly after unlearning.
> # 
> In practical applications, consider a social media platform that uses a machine learning model to recommend content to users. This model is initially trained on a large dataset comprising user interactions, such as likes, comments, shares, and viewing times. Over time, due to privacy concerns or user requests, the platform may need to unlearn specific data points related to certain users' interactions. For example, if a user requests to remove his records in recent days, his previous records can still be accessed to recommend content to him. In this example, both training data and forgetting data can be accessed, but the platform may not want to retrain the model from scratch using only remaining data. 
># 
> **Reference**
> 
> [1] Chundawat, et al. Can bad teaching induce forgetting? Unlearning in deep networks using an incompetent teacher. AAAI 2023.
> 
> [2] Thudi, et al. Unrolling sgd: Understanding factors influencing machine unlearning. EuroS\&P 2022.
> 
> [3] Bourtoule, et al. Machine unlearning. SP 2021.
> 
> [4] Tarun, et al. Fast yet effective machine unlearning. TNNLS 2023.
> 
> [5] Chen, et al. Boundary Unlearning: Rapid Forgetting of Deep Networks via Shifting the Decision Boundary. CVPR 2023.
> 
> [6] Chundawat, et al. Zero-shot machine unlearning. IEEE Transactions on Information Forensics and Security (2023).

Reviewer comment (Experiments):
In the implementation, the authors optimized the extractor unlearning loss and representation alignment loss ***alternately***. Does alternately update parameter works better than two-stage learning (saying optimize the extractor unlearning loss with the full epochs then optimize representation alignment loss)?

Author response:
> Yes, we found that alternating the optimization is more effective than a two-stage optimization. The alternating approach is better at addressing the key challenge in machine unlearning: balancing the need to forget certain data while maintaining performance on the remaining data. In a two-stage process, optimizing extractor unlearning loss for full epochs can significantly impair the model's capacity to retain essential information, making subsequent alignment optimization inefficient or even impractical.
> # 
> To validate our approach, we conducted experiments comparing our method (LAF) with a two-stage learning process. 
> The results, which we will elaborate on in the revised version of our paper, clearly demonstrate the superiority of the LAF method in approximating the performance of the retrain. In our comparison, the method that more closely aligns with the 'Retrain' baseline is highlighted in bold. Specifically, the LAF method excels in preserving performance on the remaining data, outperforming the Two-stage method in this respect.
> # 
>|   Data   |   Method  |   Train_r      |   Train_f      |   Test         |   ASR          |
>|----------|-----------|----------------|----------------|----------------|----------------|
>| DIGITS   | Retrain   | 99.56±0.05     | 98.84±0.10     | 99.04±0.10     | 49.80±0.53     |
>| DIGITS   | TwoStage  | 88.63±7.06     | 69.22±19.74    | 84.22±9.45     | 44.01±1.29     |
>| DIGITS   | **LAF**   | **98.03±0.68** | **97.29±1.43** | **97.30±0.78** | **47.92±0.84** |
>| FASHION  | Retrain   | 96.43±0.35     | 92.15±0.41     | 90.23±0.22     | 47.32±0.76     |
>| FASHION  | TwoStage  | 81.82±0.16     | 71.26±1.37     | **91.28±0.30** | 56.92±0.96     |
>| FASHION  | **LAF**   | **91.54±2.67** | **90.91±7.00** | 87.53±3.26     | **46.89±0.88** |
># 
>
>The bold results are the closest ones to the results of the retrained model.

Reviewer comment (Presentation):
(minor comment) Dot labels in Figure 1 can be replaced with their exact labels for better understanding.

Author response:
> Thank you for your suggestion. We have revised Figure 1 in Section 4.3, replacing the dot labels with exact digits for enhanced clarity.