Boosting Alignment for Post-Unlearning Text-to-Image Generative Models

Authors: myeongseob  ko
henry  li
zhun  wang
jonathan  patsenker
jiachen t wang
qinbin  li
ming  jin
dawn  song
ruoxi  jia
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 18:39:34

Abstract: Large-scale generative models have shown impressive image-generation capabilities, propelled by massive data. However, this often inadvertently leads to the generation of harmful or inappropriate content and raises copyright concerns. Driven by these concerns, machine unlearning has become crucial to effectively purge undesirable knowledge from models. While existing literature has studied various unlearning techniques, these often suffer from either poor unlearning quality or degradation in text-image alignment after unlearning, due to the competitive nature of these objectives. To address these challenges, we propose a framework that seeks an optimal model update at each unlearning iteration, ensuring monotonic improvement on both objectives. We further derive the characterization of such an update.
  In addition, we design procedures to strategically diversify the unlearning and remaining datasets to boost performance improvement. Our evaluation demonstrates that our method effectively removes target classes from recent diffusion-based generative models and concepts from stable diffusion models while maintaining close alignment with the models' original trained states, thus outperforming state-of-the-art baselines.

Main paper: 
 introduction: 
 Large-scale text-to-image generative models have recently gained considerable attention for their impressive image-generation capabilities. Despite being at the height of their popularity, these models, trained on vast amounts of public data, inevitably face concerns related to harmful content generation [Heng and Soh, 2024] and copyright infringement [Zhang et al., 2023b] . Although Figure 1 : Generated images using SalUn [Fan et al., 2023] , ESD [Gandikota et al., 2023] , and Ours after unlearning given the condition. Each row indicates different unlearning tasks: nudity removal, and Van Gogh style removal. Generated images from our approach and SD [Rombach et al., 2022] are well-aligned with the prompt, whereas SalUn and ESD fail to generate semantically correct images given the condition. On average, across 100 different prompts, SalUn shows the lowest clip alignment scores (0.305 for nudity removal and 0.280 for Van Gogh style removal), followed by ESD (0.329 and 0.330, respectively). Our approach achieves scores of 0.350 and 0.352 for these tasks, closely matching the original SD scores of 0.352 and 0.348. exact machine unlearning-retraining the model by excluding target data-is a direct solution, its computational challenge has driven continued research on approximate machine unlearning.To address this challenge, recent studies [Fan et al., 2023 , Gandikota et al., 2023 , Heng and Soh, 2024] , have introduced approximate unlearning techniques aimed at boosting efficiency while preserving effectiveness. These approaches have successfully demonstrated the ability to remove target concepts while maintaining the model's general image generation capabilities, with generation quality assessed using the Fréchet Inception Distance. However, these studies generally overlook the impact of unlearning on image-text alignment, which pertains to the semantic accuracy between generated images and their text descriptions [Lee et al., 2024] . While pretrained generative models generally demonstrate high alignment scores, our study reveals a critical gap: state-of-the-art unlearning techniques fall short in achieving comparable text-image alignment scores after unlearning, as illustrated in Figure 1 . This could lead to potentially problematic behaviors in real-world deployments, necessitating further investigation.We attribute the failure of existing techniques to maintain text-image alignment to two primary factors. Firstly, the unlearning objective often conflicts with the goal of maintaining low loss on the retained data, illustrating the competitive nature of these two objectives. Traditionally, approaches to optimizing these objectives have simply aggregated the gradients from both; however, this method of updating the model typically advances one objective at the expense of the other. Hence, while these approaches may successfully remove target concepts, they often compromise text-image alignment for retained concepts in the process. Secondly, current methods employ a simplistic approach to constructing a dataset for loss minimization on retained concepts. For example, in Fan et al. [2023] , this dataset is composed of images generated from a single prompt associated with the concept to be removed. This lack of diversity in the dataset might lead to overfitting, which in turn hampers the text-image alignment.To address these issues, we propose a principled framework designed to optimally balance the objectives of unlearning the target data and maintaining performance on the remaining data at each update iteration. Specifically, we introduce the concept of the restricted gradient, which allows for the optimization of both objectives while ensuring monotonic improvement in each objective. Furthermore, we have developed a deliberate procedure to enhance data diversity, preventing the model from overfitting to the limited samples in the remaining dataset. To the best of our knowledge, the strategic design of the forgetting target and remaining sets has not been extensively explored in the existing machine unlearning literature. In our evaluation, we demonstrate the improvement in both forgetting quality and alignment on the remaining data, compared to baselines. For example, our evaluation in nudity removal demonstrates that our method effectively reduces the number of detected body parts to zero, compared to 598 with the baseline stable diffusion (SD) [Rombach et al., 2022] , 48 with erased stable diffusion (ESD-u), and 3 with saliency map-based unlearning (SalUn) [Fan et al., 2023] . Particularly, while achieving this effective erasing performance, our method reduces the alignment gap to SD by 11x compared to ESD-u and by 20x compared to SalUn on the retained test set. 
machine unlearning: 
 Machine unlearning has primarily been propelled by the "Right to be Forgotten" (RTBF), which upholds the right of users to request the deletion of their data. Given that large-scale models are often trained on web-scraped public data, this becomes a critical consideration for model developers to avoid the need for retraining models with each individual request. In addition to RTBF, recent concerns related to copyrights and harmful content generation further underscore the necessity and importance of in-depth research in machine unlearning. The principal challenge in this field lies in effectively erasing the target concept from pre-trained models while maintaining performance on other data. Recent studies have explored various approaches to unlearning, including the exact unlearning method [Bourtoule et al., 2021] and approximate methods such as using negative gradients, fine-tuning without the forget data, editing the entire parameter space of the model [Golatkar et al., 2020] . To encourage the targeted impact in the parameter space, [Golatkar et al., 2020 , Foster et al., 2024] proposed leveraging the Fisher information matrix, and [Fan et al., 2023 ] leveraged a gradient-based weight saliency map to identify crucial neurons, thus minimizing the impact on remaining neurons. Furthermore, data-influence-based debiasing and unlearning have also been proposed [Chen et al., 2024 , Bae et al., 2023] . Another line of work leverages mathematical tools in differential privacy [Guo et al., 2019 , Chien et al., 2024] to ensure that the model's behavior remains indistinguishable between the retrained and unlearned models.Recent advancements in text-conditioned generative models [Ho and Salimans, 2022, Rombach et al., 2022] , trained on extensive web-scraped datasets like LAION-5B [Schuhmann et al., 2022] , have raised significant concerns about the generation of harmful content and copyright violations. A series of studies have addressed the challenge of machine unlearning in diffusion models [Heng and Soh, 2024 , Gandikota et al., 2023 , Zhang et al., 2023a , Fan et al., 2023] . One approach [Heng and Soh, 2024] interprets machine unlearning as a continual learning problem, showing effective removal results in classification tasks by employing Bayesian approaches to continual learning [Kirkpatrick et al., 2017] , which enhance unlearning quality while maintaining model performance using generative reply [Shin et al., 2017] . However, this approach falls short in removing concepts such as nudity compared to other methods [Gandikota et al., 2023] . Another proposed method [Gandikota et al., 2023] guides the pre-trained model toward a prior distribution for the targeted concept but struggles to preserve performance. The most recent work [Fan et al., 2023] proposes selectively damaging neurons based on a saliency map and random labeling techniques, although this method tends to overlook the quality of the remaining set, focusing on improving the forgetting quality, which does not fully address the primary challenges in the machine unlearning community. Although [Bae et al., 2023] presents a similar multi-task learning framework for variational autoencoders, their work does not show the optimality of their solution, and their experiments mainly focus on small-scale models, due to the computational expense associated with influence functions. 
our approach: 
 We study the efficacy of our approach in unlearning by removing target classes from class-conditional diffusion models or eliminating specific concepts from text-to-image models while maintaining their general generation capabilities. We will call the set of data points to be removed as the forgetting dataset. To set up the notations, let D denote the training set and D f ⊂ D be the forgetting dataset. We will use D r = D\D f to denote the remaining dataset. Our approach only assumes access to some representative points for D f and D r . As discussed later, depending on specific applications, these data points can be either directly sampled from D f and D r or generated based on the high-level concept of D f to be removed. With a slight abuse of notation, we will use D r and D f to also denote the actual representative samples used to operationalize our proposed approach. Furthermore, we denote the model parameter by θ. Let l be a proper learning loss function. The loss of remaining data and that of forgettng data are represented by L r (θ) := z∈Dr l(θ, z) and L f (θ) := -λ z∈D f l(θ, z), respectively, where λ is a weight adjusting the importance of forgetting loss relative to the remaining data loss. We term L r and L f remaining loss and forgetting loss, respectively. We note that in the context of diffusion models, loss function l is defined as l = E t,x0,ϵ∼N (0,1) ∥ϵ -e θ (x t , t)∥ 2 , where x t is a noisy version of x 0 generated by adding Gaussian noise to the clean image x 0 ∼ p data (x) at time step t with a noise scheduler, and e θ (x t , t) is the model's estimate of the added noise ϵ at time t [Xu et al., 2023 , Ho et al., 2020] . For text-to-image generative models, the loss function l is specified as l = E t,q0,c,ϵ ∥ϵ -ϵ θ (q t , t, η)∥ 2 , where q 0 is an encoded latent q 0 = E(x 0 ) with encoder E, and q t is a noisy latent at time step t. The noise prediction ϵ θ (q t , t, η) is conditioned on the timestep t and a text η.Optimizing the Update. Similar to existing work Fan et al. [2023] , our objective is to find an unlearned model with parameters θ u , starting from a pre-trained model with weights θ 0 , such that the model forgets the target concepts in D f while maintaining its utility on the remaining dataset D r . Formally, we aim to maximize the forget error on D f , represented by L f (θ), while minimizing the retain error on D r , represented by Lr(θ). This can be formulated as min θ L r (θ) + L f (θ), where our approach applies iterative updates to achieve both goals simultaneously. A simple approach to optimize this objective, often adopted by existing work, is to calculate the gradient ∇L r (θ) + ∇L f (θ) and use it to update the model parameters at each iteration. However, empirically, we observe that the two gradients usually conflict with each other, i.e., the decrease of one objective is at the cost of increasing the other; therefore, in practice, this approach yields a significant tradeoff between forgetting strength and model utility on the remaining data. In this work, we aim to present a principled approach to designing the update direction at each iteration that more effectively handles the tradeoff between forgetting strength and model utility on the remaining data. Our key idea is to identify a direction that achieves a monotonic decrease of both objectives.To describe our algorithm, we briefly review the directional derivative. Definition 1 (Directional Derivative). The directional derivative [Spivak, 2006] of a function f at x in the direction of v is written asEQUATIONThis special form of the derivative has the useful property that its maximizer can be related to the gradient ∇ x f (x), which we formally state below.Theorem 2 (Directional derivative maximizer is the gradient). Let f be a function on x. Then the maximum value of the directional derivative of f at x is |∇f (x)| the ℓ 2 norm of its gradient. Moreover, the direction v is the gradient itself, i.e.,EQUATIONIn unlearning, we are specifically interested in the gradient of two losses, the forgetting loss L f and the remaining loss L r . Moreover, we seek gradient directions that simultaneously improve on both. This motivates the restricted gradient, which we define below. Definition 3 (Restricted gradient). The negative restricted gradient of a pair of objectives L α , L β is the direction v at θ that satisfiesmin v D v (L α + L β )(θ) s.t. L α (θ) ≥ L α (θ + v) and L β (θ) ≥ L β (θ + v).Intuitively, with the restricted gradient we seek to define the ideal direction for unlearning. We would like to optimize the joint loss L = L r + L f subject to the condition that at every parameter update step, L r and L f experience monotonic improvement. This is precisely the step prescribed by the negative restricted gradient. Since the learning rates used to fine-tune the parameters in the unlearning process are typically quite small, we can approximate the updated loss at each iteration via a simple first-order Taylor expansion. In this case, the restricted gradient takes a simple form.Theorem 4 (Characterizing the restricted gradient under linear approximation). Given θ, suppose that L r (θ + δ) -L r (θ) ≈ δ • ∇L r and L f (θ + δ) -L f (θ) ≈ δ • ∇L f . The restricted gradient can be written asEQUATIONwhereEQUATIONwhen we have conflicting unconstrained gradient terms, i.e. ∇L f • ∇L r < 0. The theorem presented demonstrates that the restricted gradient is determined by aggregating the modifications from ∇L f and ∇L r . This modification process involves projecting ∇L f onto the normal vector of ∇L r , yielding δ * f , and similarly projecting ∇L r onto the normal vector of ∇L f , resulting in δ * r . The optimal update, as derived in Theorem 4, is illustrated in Figure 2 . Notably, when ∇L f and ∇L r have equal norms, the restricted gradient matches the direct summation of the two original gradients, namely, ∇L f + ∇L r . However, it is more common for the norm of one gradient to dominate the other, in which case the restricted gradient provides a more balanced update compared to direct aggregation.Remark 1. We wish to highlight an intriguing link between the gradient aggregation mechanism presented in Theorem 4 and an existing method to address gradient conflicts across different tasks in multi-task learning. This restricted gradient coincides exactly with the gradient surgery procedure introduced in Yu et al. [2020] . While their original paper presented the procedure from an intuitive perspective, our work offers an alternative viewpoint and rigorously characterizes the objective function that the gradient surgery procedure optimizes.Diversify D r . Since D \ D f is usually of enormous scale, it is infeasible to incorporate all of them into the remaining dataset D r for running the optimization. In practice, one can only sample a subset of points from D r . In our experiments, we find that the diversity of D r plays an important role in maintaining the model performance on the remaining dataset, as seen in Section 4.2. Therefore, we propose procedures for forming a diverse D r . For models with a finite set of class labels, such as diffusion models trained on CIFAR-10, we adopt a simple procedure of maintaining an equal number of samples for each class in D r . Our ablation studies in Section 4.4 show that this is more effective in maintaining model performance on the remaining dataset than more sophisticated procedures, such as selecting the most similar examples to the forgetting samples. The intuitive reason is that reminding the model of as many fragments as possible related to the remaining set during each forgetting step is crucial. By doing so, it leads to finding a representative restricted descent direction, which helps the model to precisely erase the forget data while maintaining a state comparable to the original model. When the text input is unconstrained, such as in the stable diffusion model setting, to strategically design diverse information, we propose the following procedure to generate D r based on the concept to be forgotten, denoted by c. Using a large language model (LLM), we first generate diverse text prompts related to concept c, yielding prompt set Y c . These prompts are then modified by removing all references to c, creating a parallel set Y. By passing both Y c and Y through the target diffusion model, we obtain corresponding image sets X c and X . This process allows us to construct our final datasets: D f = {(x, y) | x ∈ X c , y ∈ Y c } and D r = {(x, y) | x ∈ X , y ∈ Y}. 
experiment: 
 In this study, we address the crucial challenge of preventing undesirable outputs in text-to-image generative models. We begin by examining class-wise forgetting with CIFAR-10 diffusion-based generative models, where we demonstrate our method's ability to selectively prevent the generation of specific class images (Section 4.2). We then explore the effectiveness of our approach in removing nudity and art styles (Section 4.3) to address real-world concerns of harmful content generation and copyright infringement. We further study the impact of data diversity (Section 4.4) as well as the sensitivity of our method to hyperparameter settings (Section 4.4).For our CIFAR-10 experiments, we leverage the EDM framework [Karras et al., 2022] , which introduces some modeling improvements including a nonlinear sampling schedule, direct x 0 -prediction, and a second-order Heun solver, achieving the state-of-the-art FID on CIFAR-10. For stable diffusion, we utilize the pre-trained Stable Diffusion version 1.4, following prior works. Both implementations require two key hyperparameters: the weight λ of the gradient descent direction relative to the ascent direction, and the loss truncation value α, which prevents unbounded loss maximization during unlearning. Detailed hyperparameter configurations are provided in Appendix C. For dataset construction, we used all samples in each class for the CIFAR-10 forgetting dataset and 800 samples for Stable Diffusion experiments. Considering the practical constraints of accessing complete datasets in real-world scenarios, we construct the remaining dataset D r by sampling 1% of data from each retained class, yielding a total of 450 samples for CIFAR-10 (50 from each of the 9 non-target classes) and 800 samples for Stable Diffusion.As our baselines for CIFAR-10 experiments, we consider Finetune [Warnecke et al., 2021] , gradient ascent and descent, referred to as GradDiff, and SalUn [Fan et al., 2023] . For concept removal, our baselines include the pretrained diffusion model SD [Rombach et al., 2022] , erased stable diffusion ESD [Gandikota et al., 2023] , and SalUn [Fan et al., 2023] . To fairly compare, We further consider the variants of ESD, depending on the unlearning task. We note that we do not consider the baseline by [Heng and Soh, 2024] due to its demonstrated limited performance in nudity removal, compared to ESD. Our approach is referred to as RG when applied only with the restricted gradient, and RGD when data diversity is incorporated. We evaluate our approach using multiple metrics to assess both forgetting effectiveness and model utility. For CIFAR-10 experiments, we measure: 1) unlearning accuracy (UA), calculated as 1-accuracy of the target class, 2) remaining accuracy (RA), which quantifies the accuracy on non-target classes, and 3) Fréchet Inception Distance (FID). We observed that standard CIFAR-10 classifiers demonstrate inherent bias when evaluating generated samples from unlearned classes, predominantly assigning these noise-like images to a particular class among the ten categories-a limitation arising from their training exclusively on clean class samples. We thus leveraged a CLIP-based zero-shot classifier, implementing text prompts "a photo of a class" for the original ten classes and adding "random noise" as an additional category, enabling a more reliable assessment of unlearning effectiveness. We generate 50K images for FID calculation. For concept removal in Stable Diffusion, we assess forgetting effectiveness using Nudenet [Bedapudi, 2019] , which detects exposed body parts in generated images prompted by I2P [Schramowski et al., 2023] . After filtering prompts with non-zero nudity ratios, we obtain 853 evaluation prompts from an initial set of 4,703. To evaluate the retained performance, following [Lee et al., 2024] , we measure semantic correctness using CLIP [Cherti et al., 2023] alignment scores (AS) between prompts and their generated images. We evaluate model performance on both training prompts (D r,train ) used during unlearning and a separate set of held-out test prompts (D r,test ). These two distinct sets are constructed by carefully splitting semantic dimensions (e.g., activities, environments, moods). Detailed construction procedures for both sets are provided in Appendix D.We present the CIFAR-10 experiment results in Table 1 . To fairly compare, we use the same remaining dataset for other baselines. Our finding first indicates that while Finetune achieves superior performance on retained data (highest RA and FID scores), it struggles to effectively unlearn target classes with this limited remaining dataset. Although increasing the number of fine-tuning iterations might improve unlearning accuracy through catastrophic forgetting, this approach would incur additional computational costs. Secondly, we observe that SalUn has low RA, compared to other baselines even with their comparable FID performance. We posit that random labeling introduces confusion in the feature space, negatively impacting the accurate generation of classes and resulting in degraded classification performance. Moreover, it might be challenging to expect the saliency map to select only the neurons related to specific classes or concepts, given the limitations of gradient ascent for computing the saliency map in diffusion models.The Impact of Restricted Gradient and Data Diversity Our observations are as follows. 1) RG outperforms Gradiff and other baselines by decreasing FID and increasing RA, indicating that the restricted gradient leads to an optimally balanced solution for both tasks. 2) RGD shows improvements over RG, suggesting that data diversification, in conjunction with the restricted gradient, further enhances performance in terms of RA and FID. We vary the hyperparameters and provide the results in section 4.4.Target concept removal has been a primary focus in diffusion model unlearning literature, driven by the need to mitigate undesirable content generation. While existing methods have shown potential for removing nudity or art styles, our study reveals that they often compromise model alignment after unlearning.Nudity Removal. We summarize our results in Figure 4 and Table 2 . We observe that Salun tends to generate samples that are overfit to the remaining dataset. Although Salun shows promising performance in nudity removal-detecting fewer exposed body parts compared to SD and ESD-u, as shown in Figure 4 -this success comes at the cost of output diversity. In particular, SalUn often generates semantically similar images (e.g., men, wall backgrounds) for both forgetting concepts (Figure 3 ) and remaining data (Figure 1 ). Table 4 quantitatively validates this observation, revealing SalUn's lowest alignment scores post-unlearning. These results suggest that SalUn's forgetting performance could stem from overfitting. This limitation may arise from two factors: the selected neurons potentially affecting both target and non-target concepts, and the limited diversity in their forget and remaining datasets. In the case of ESD, the resulting model often fails to remove the nudity concept from unlearned models, as shown in Figure 4 . We also evaluate ESD-u and observe that the nudity removal performance between ESD and ESD-u are quite similar although it achieves better AS than SalUn. They suggest using "nudity" as a prompt for unlearning, but it might be difficult to reflect the entire semantic space related to the concept of "nudity," given that we can describe nudity in many different ways using paraphrasing. Figure 4 : The nudity detection results by Nudenet, following prior works [Fan et al., 2023 , Gandikota et al., 2023] . The Y-axis shows the exposed body part in the generated images, given the prompt, and the X-axis denotes the number of images generated by each unlearning method and SD. We exclude bars from the plot if the corresponding value is zero. RGD outperforms state-of-the-art baselines in terms of forget quality (i.e., zero detection of exposed body part given I2P prompts as described in Figure 4 ) and retain quality (i.e., high AS presented in Table 2 ), effectively mitigating the trade-off between the two tasks. To further validate the role of both the restricted gradient and diversification steps to nudity removal, we conduct a two-way ablation study. Removing the restricted gradient step from RGD yields GradDiffD, which incorporates dataset diversity into GradDiff, whereas removing the diversification step yields the previously introduced RG. RGD's superior performance over both GradDiffD (Table 7 and Figure 4 ) and RG (Table 4 ) underscores the crucial importance of both steps in our proposed unlearning algorithm.Art Style Removal. Similar to nudity removal, the task of eliminating specific art styles presents a significant challenge. In order to evaluate whether the unlearning methods inadvertently impact other concepts and semantics beyond the targeted art style, we prompt the model with other artists' styles (e.g., Monet, Picasso) while targeting to remove Vincent van Gogh's style. The results of generation examples are shown in Figure 1 and Figure 5 , and the average alignment scores are shown in Table 2 . It is observed that SalUn cannot follow the prompt to generate other artists' styles and shows a significant drop in alignment scores (AS) compared with the pre-trained SD. We also train ESD-x by modifying the cross-attention weights, which is more suitable for erasing artist styles than full-parameter training (shown as plain ESD without any suffix) as proposed in ESD work. Although ESD-x performs similarly to RG in terms of alignment scores, after manual inspection of the generated images, we find ESD-x sometimes generates images ignoring the style instructions as presented in Figure 1 , while RG generates images with lower quality details like noisy backgrounds but adheres well to the style instructions. Consequently, after incorporating gradient surgery to prevent interference between retain and forgot targets, our RGD achieves better image quality and shows the best alignment score, almost equivalent to the performance of the pre-trained SD. Ablation in Diversity. We further investigate the impact of data diversity through controlled experiments. For CIFAR-10, we design two scenarios based on feature similarity analysis using CLIP embeddings: Case 1, where D r contains samples from only the two classes most semantically similar to the target class, and Case 2, with balanced sampling across all classes. This design stems from our hypothesis that unlearning a target class may particularly affect semantically related classes, making their retention critical. We compute class similarities using cosine distance between CLIP feature vectors as described in Figure 7 . Table 3 shows that limited diversity (Case 1) significantly impacts model performance, with FID increasing by 83.803 for RGD. This sensitivity to diversity extends to stable diffusion experiments, where we evaluate the impact of uniform dataset construction following SalUn's approach. As shown in Table 4 , RG with uniform datasets shows a larger performance gap from SD (∆ = 0.032 in test alignment scores) compared to RGD (∆ = 0.001). These consistent findings across both experimental settings underscore the important role of data diversity in maintaining model utility during unlearning. This study advances the understanding of machine unlearning in text-to-image generative models by introducing a principled approach to balance forgetting and remaining objectives. We show that the restricted gradient provides an optimal update for handling conflicting gradients between these objectives, while strategic data diversification ensures further improvements on model utilities. Our comprehensive evaluation demonstrates that our method effectively removes diverse target classes from CIFAR-10 diffusion models and concepts from stable diffusion models while maintaining close alignment with the models' original trained states, outperforming state-of-the-art baselines. 
limitation and broader impacts: 
 While our solution introduces computationefficient retain set generation using LLMs, the strategic sampling of retain sets for stable diffusion models presents intriguing research directions. Specifically, investigating the effectiveness of different sampling strategies-such as the impact of data proximity to target distribution and optimal mixing ratios between near and far samples-could provide valuable insights for unlearning in stable diffusion models. Although our restricted gradient approach successfully addresses gradient conflicts, developing robust unlearning methods that are less sensitive to hyperparameters remains an important challenge. 
acknowledgement: 
 RJ and the ReDS lab acknowledge support through grants from the Amazon-Virginia To prove this theorem, we establish the following lemma. We notate the ℓ 2 norm as ∥ • ∥ throughout. Lemma 5 (Projected gradients obtain optimal solution to a constrained objective). Let L f (θ), and L r (θ) be K-Lipschitz smooth negative forget and retain losses under the ℓ 2 norm respectively. Then, the updateEQUATION)in terms of δ f , for a value η ≪ 1 K when we have conflicting unconstrained gradient terms, i.e. ∇L f • ∇L r < 0.Proof of Lemma 5. For δ r , δ f , both of norm η, we have good approximation by the Taylor expansion due to the Lipschitz condition on L f , L r . Therefore, we have,L r (θ + δ r ) -L r (θ) ≈ δ r • ∇L r L f (θ + δ f ) -L f (θ) ≈ δ f • ∇L f L f (θ + δ r ) -L f (θ) ≈ δ r • ∇L f L r (θ + δ f ) -L r (θ) ≈ δ f • ∇L rWe can re-express the two objectives as, arg minEQUATIONBy the method of Langrangian multipliers, for each objective we create slack variables λ f , λ r , and obtain the unconstrained objectives, arg min||δ f ||=η δ f • ∇L f + λ f δ f • ∇L r = arg min ||δ f ||=η δ f • (∇L f + λ f ∇L r ) arg min ||δr||=η δ r • ∇L r + λ r δ r • ∇L f = arg min ||δr||=η δ r • (∇L r + λ r ∇L f )We first observe since both are now linear objective, that the minima is trivially observed whenδ * f ∝ -(∇L f + λ f ∇L r ), and δ * r ∝ -(∇L r + λ r ∇L f ).For the rest of this proof, without loss of generality, suppose η is scaled such that we hold the previous proportionality statements as equalities.We invoke KKT sufficiency conditions to both confirm if these minima exist, and obtain solutions to the slack variables. In the case of conflicting gradients, since ∇L f • ∇L r < 0, the minimizers of the unconstrained objectives in Equations 7, 8 are not satisfied within the constraints. Therefore, λ f , and λ r do not vanish, and are maximizers of their respective objectives. Taking the gradients in respect to the slack variables and setting to 0, we have∇ λ f δ * f • (∇L f + λ f ∇L r ) = -∇ λ f δ * f • δ * f = -2∇L r • δ * f = 0 ∇ λr (δ * r • (∇L r + λ r ∇L f )) = -∇ λr (δ * r • δ * r ) = -2∇L f • δ * r = 0.We can solve this in a way that satisfies the objective by requiring δ * r to be orthogonal to ∇L f , and δ * f to be orthogonal to ∇L r . In this case, we haveλ f = - ∇L f •∇Lr ∥∇Lr∥ 2 and λ r = - ∇L f •∇Lr∥∇L f ∥ 2 as the optima. We verify that these are maximizers by computing the second derivatives, which are constants at -2∥∇L r ∥ 2 and -2∥∇L f ∥ 2 respectively. Both are strictly negative, confirming the second order sufficient condition for a maximizer.Therefore it is precisely the restricted gradient steps,δ * f = ∇L f - ∇L f •∇Lr ∥∇Lr∥ 2 ∇L r and δ * r = ∇L r - ∇L f •∇Lr ∥∇L f ∥ 2 ∇L f ,which solve the optimization problems in Equations 5, 6 respectively.Proof of Theorem 4. We take the Taylor expansions in respect to v of L f and L r around θ. We have mutatis mutandis for some h ∈ R,L f (θ + hv) = L f (θ) + h∇L f (θ) • v + O(h 2 ∥v∥ 2 )It follows that, for v, w, such that w • ∇L f (θ) = 0,D v+w L f (θ) = lim h→0 L f (θ + hv + hw) h = lim h→0 L f (θ) + h∇L f (θ) • (v + w) h = lim h→0 L f (θ) + h∇L f (θ) • v h = lim h→0 L f (θ + hv) h = D v L f (θ).We observe that we can bound the optimization,min v * D v (L f + L r )(θ) ≥ min v D v L f (θ) s.t. L r (θ) ≥ L r (θ + v) + min w D w L r (θ) s.t. L f (θ) ≥ L f (θ + w) = lim h→0 min v * 1 h L f (θ + hv) + lim h→0 min w * 1 h L r (θ + hw).We use min * to signify the presence of constraints as previously defined for the respective expression to simplify notation.We invoke Lemma 5 to solve each minimization problem above, yielding,v * ∝ δ * f = ∇L f - ∇L f •∇Lr ∥∇Lr∥ 2 ∇L r , and w * ∝ δ * r = ∇L r - ∇L f •∇Lr ∥∇L f ∥ 2 ∇L f .Note that since we are taking the limits as h → 0, the Taylor expansion in Lemma 5 is exact as the relevant constant in the lemma, ∥η∥ → 0.We also have thatD v * L f (θ) = D v * +w * L f (θ) since w * • ∇L f (θ) = 0 (and similarly we have D w * L r (θ) = D v * +w * L r (θ)).min v * D v (L f + L r )(θ) ≥ min v * D v L f (θ) + min w * D w L r (θ) = D v * L f (θ) + D w * L r (θ) = D v * +w * L f (θ) + D v * +w * L r (θ) = D v * +w * (L f (θ) + L r (θ))If v * + w * satisfies the constraints of the original optimization, and bounds the minimizer from below, this is the optimal solution. Therefore, we require for both losses,L f (θ + v * + w * ) ≥ L f (θ) L r (θ + v * + w * ) ≥ L r (θ)By the constraints of the optimization problem, we know that L f (θ + v * ) ≥ L(θ), and L r (θ + w * ) ≥ L(θ). Again, using the Taylor expansion, mutatis mutandis we have,L f (θ + v * + w * ) = L f (θ + v * ) + ∇L f (θ + v * ) • w * + O(∥w * ∥ 2 ) ≃ L f (θ + v * ) ≥ L f (θ).Therefore, η(δ * f + δ * r ), solves the optimization for a small enough constant η ∈ R + , so δ * f + δ * r solves the optimization up to a constant. This completes the proof.Denoising Diffusion Probabilistic Models Diffusion models consist of a forward diffusion process and a reverse diffusion process. The forward diffusion process progressively deteriorates an initial data point x 0 ∼ q{x 0 } by adding Gaussian noise with a variance schedule β t ∈ (0, 1) to generate a set of noisy latents {x 1 , x 2 , ..., x T } with a Markov transition probability:EQUATIONwhere T indicates the maximum time steps. In the reverse process, we aim to predict the latent representation of the previous time step, which can be written as p θ (x t-1 |x t ) = N (x t-1 ; µ θ (x t , t), Σ θ (t)). The training objective to predict the previous step can then be defined as:EQUATIONwhere q(x t-1 |x t , x 0 ) = N (x t-1 ; µ q (x t , x 0 ), Σ q (t)). Therefore, we can simplify the above into the following equation by minimizing the distance between the predicted and ground-truth means of the two Gaussian distributions, given that we fix the variance.EQUATIONwhere e θ (x t , t) is the model's estimate of the noise ϵ added into the clean image x 0 at time t [Xu et al., 2023 , Ho et al., 2020] .Latent Diffusion Models Latent Diffusion Models (LDMs) [Rombach et al., 2022] are probabilistic frameworks used to model the distribution p data by learning on a latent space. Based on the pre-trained variational autoencoder, LDMs first encode high-dimensional data x 0 into a more tractable, lowdimensional latent representation z 0 = E(x 0 ), where E represents an encoder. Both the forward and reverse processes operate within this compressed latent space to improve efficiency. The objective can be described as L = E t,z0,c,ϵ ∥ϵ -ϵ θ (z t , t, c)∥ 2 , where the noise prediction ϵ θ (z t , t, c) is conditioned on the timestep t and a text c. Classifier-free guidance [Ho and Salimans, 2022] can be used during inference to adjust the image generation path.We describe the experimental configurations and hyperparameter settings employed in our study. All experiments were conducted using an NVIDIA H100 GPU.For experiments on CIFAR-10, we implemented our method using hyperparameters α = 1 × 10 -1 and λ = 5. Our EDM implementation used a batch size of 64, a duration parameter of 0.05, and a learning rate of 1e-5. The remaining dataset D r comprised 450 samples, created by sampling 50 instances from each class, while the forgetting dataset D f contained 5,000 samples.For nudity removal experiments with Stable Diffusion, we set α = 1.6 and λ = 1.5. Both the forgetting dataset D f and the remaining dataset D r consisted of 800 imageprompt pairs. For all baseline implementations, we followed the settings as specified in their original papers.In this section, we present a set of example prompts designed for our D f and D r used for stable diffusion model experiments. To generate these prompts, we leverage the ChatGPT. Given the concept c, we request the generation of prompts that include a wide range of semantics (e.g., environment, time, mood, actions) to describe the concept c for D f . As explained before, we remove the word related to c to generate D r . Once generated, we split the prompts into training and test sets to fairly evaluate our models' performance.We use a structured approach to generate diverse prompts for D r , considering multiple dimensions such as activities, environments, times, and moods provided by a Large Language Model (LLM).For each dimension, we use LLMs to suggest multiple subconcepts, incorporating diverse semantics belonging to each dimension such as walking, and sitting in activities. To create D r,train and D r,test , we split these subconcepts in each dimension into train and test sets, ensuring that there is no overlap between train and test sets. We then combine these subconcepts to generate D r . Table 5 shows a set of example prompts we used for nudity removal. Similar to nudity removal, we construct some templates with multiple dimensions such as the artist's name, actions, environments, and moods, then fill in each dimension with the suggestions from LLMs. Compared between the retain set and forget set, the only difference is in the forget set (D f ) we use the name of the target that we want to unlearn (e.g., Van Gogh), and use other artists' names or some virtual names in the retain set (D r ). We will revise the paper to incorporate these details, providing a clearer and more comprehensive explanation of our method for generating D r,train and D r,test in each application. Table 6 shows a set of example prompts we used for style removal. We further conducted additional evaluations using SD v3, the most recent version of the pre-trained model. SD v3 employs a transformer-based architecture (e.g., Diffusion Transformer models) instead of the UNet-based architecture used in previous versions. This significant change allows us to test our method's performance across different model structures. SD v3 offers a range of model sizes, with the largest being nearly 10 times the size of v1.4. We choose a medium size model with 2B parameters, which is approximately 2 times larger than v1.4. This variability enables us to assess how our method performs across different model capacities. We maintained the same hyperparameter settings as in v1.4 to ensure an easy generalization capability. We evaluated two baselines alongside our method, observing their performance under multiple hyperparameter tunings. We observed high alignment scores for both D r,train and D r,test splits with SD v3, while effectively mitigating harmful output generation. On the other hand, both baselines showed alignment score drops. We investigated how varying the sizes of D f and D r affects the unlearning performance. Our analysis reveals several key findings. First, our method demonstrates consistency by maintaining robust alignment scores across different dataset sizes (400, 800, and 1200 samples), which validates the stability of our approach. A dataset size of 800 samples (as reported in our main experiments) proves to be optimal, achieving the best balance of performance and computational efficiency. Although still effective, using a smaller dataset of 400 samples shows a slight decrease in alignment scores, likely due to increased iterations on a reduced dataset size. When using a larger dataset of 1200 samples, we can achieve alignment scores comparable to the 800-sample configuration by adjusting λ from 1.5 to 1.15, which helps balance the increased gradient ascent steps. Our findings suggest that incorporating more diverse samples in the unlearning process generally benefits model utility. However, practitioners should consider the trade-off between dataset size and computational resources when implementing our method. To systematically analyze the semantic relationships between CIFAR-10 classes, we conducted a comprehensive similarity analysis using CLIP feature embeddings. For each class, we extracted features from 500 training examples using a pre-trained CLIP model [Radford et al., 2021] . We then computed class-wise mean feature vectors and calculated pairwise cosine similarities between these representations.Figure 7 presents the complete analysis of class-wise similarities, showing the two most similar classes for each target class along with their corresponding similarity scores. This analysis informed our experimental design for the ablation studies on data diversity, particularly in constructing the remaining dataset (D r ) for Case 1, where only the two most similar classes were included. In our ablation study 4.4, we specifically focused on three target classes (plane, bird, and dog) and their respective most similar classes when constructing the limited diversity scenario (Case 1).We provide qualitative results comparing generations from the unlearned models (Salun, ESD-u, RGD and the pretrained model SD using the retain prompts D r .  
 
 Appendix:  


Decision: The reviewers highlighted several key strengths of the paper, including the significance of the addressed problem, the soundness of the proposed method, easy to follow writing, thoroughness of the empirical validation, and the balanced approach between experimental results and theoretical justifications.

Reviewers also noted some issues, such as the need for additional empirical validation, improvements in presentation, further discussion of the problem definition, and more detailed justifications for certain design decisions in the experimental protocol. These concerns were adequately addressed by the authors during the rebuttal and subsequent discussions. 

Furthermore, the authors  positioned well their work within the relevant literature, notably citing arXiv:2001.06782, which employed a similar gradient surgery technique in multitask learning. It might be beneficial for readers of the paper to mention also that a similar gradient orthogonalization method has recently been applied in disentanglement learning, as discussed in arXiv:2308.12696.

In light of the above, acceptance is recommended. The authors are strongly encouraged to incorporate the proposed improvements into the final version of the paper.

Reviews: review: paper_summary: paper_summary: This paper tackles the approximate machine unlearning task of the target class and concept removal from diffusion models. This work endeavors to improve the existing literature’s output quality and text alignment after unlearning. Firstly, A concept of the restricted gradient is proposed, allowing monotonic decreases of the two losses from the objectives of unlearning quality and the remaining data alignment. Secondly, data is also deliberately processed to improve its diversity, which is beneficial to text alignment. According to the two aforementioned objectives, The two components have proved effective from the ablation studies and other comparative results.
strength_weakness: strength_weakness: 1. The presentation in Table 1 is misleading, where the baseline method Finetune has the best RA and FID results; however, the results of the proposed RGD method are highlighted in bold.
2. There are two closely relevant baselines [1, 2] that need to be compared as they tackle the same task of concept removal and safe generation and have been popular as the baseline for this line of work. Missing such a comparison will lead to doubt about the proposed method’s practical significance. 

[1] Ablating Concepts in Text-to-Image Diffusion Models. ICCV2023.

[2] Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models. CVPR2023.
questions: questions: Please see the Weaknesses section, the missing baseline comparison is my main concern.
limitations: limitations: The authors have adequately addressed the limitations and broader impacts.
paper_summary: paper_summary: This paper addresses the problem of unlearning in generative text-to-image models. They formulate a training objective that improves upon the commonly-used one that simultaneously minimizes the loss on the retain set while maximizing it on the forget set (referred to as GradDiff here); i.e. gradient descent on the retain set and ascent on the forget set, simultaneously. Specifically, as is well-established in the literature, there are trade-offs inherent in this optimization problem, e.g. maximizing the loss on the forget set may lead to “accidentally” destroying information that is permissible, with unwanted consequences like overly compromising on utility. The authors in particular emphasize a different unwanted consequence of not handling this trade-off well, that is specific to the context of text-to-image diffusion models: the generated images are no longer as well aligned with the text prompt after unlearning, compared to before. They measure this via an alignment score based on CLIP. They show that SalUn, for instance (a state-of-the-art method) actually has the lowest alignment score after unlearning, compared to other baselines.

Motivated by these issues, the authors propose a modified objective that builds on GradDiff but leads to a solution that is more balanced between the two objectives. Instead of summing the retain set gradient and the (negated) forget set gradient directly, they sum two quantities which are the projection of the former gradient on top of the normalized gradient of the latter, and the other way around. This approach (which they refer to as the “restricted gradient”) is closely related, as they discuss, to tricks used in the multi-task learning literature to jointly optimize competing objectives.

In addition, they discuss issues relating to the selection of the subset of the retain set that is used when operationalizing the training objective. They find that explicitly encouraging diversity there is important for avoiding issues relating to “overfitting” and maintaining the ability to generate diverse images that align well with textual prompts.

They show results on CIFAR-100 and using stable diffusion, comparing against different unlearning baselines, for removing a class in CIFAR-10 or a concept (e.g. nudity) from SD models. They report qualitative results (showing generations for different prompts) as well as some quantitative results, like the accuracy of the unlearned model on different sets (based on using pretrained classification models, when given as input images generated by the unlearned model), a perceptual metric for the quality of generated images as well as the alignment score. They show that their method outperforms previous baselines in terms of unlearning effectiveness (according to the particular metric) with the smallest sacrifice to alignment score and utility compared to those baselines.
strength_weakness: strength_weakness: - The paper is missing a clear problem formulation. Section 3 defines notation, but the objective isn’t stated. What is the goal of unlearning? What does it mean to unlearn something? The authors say this can be used for privacy or harmful generations or copyright, but it’s not clear how these all connect under a unified formulation and how the proposed method (and experimental design) actually addresses these problems. There is no formal definition and one must rely on the empirical metrics presented to infer how success is measured. Other types of unlearning (e.g. for privacy) are defined formally in the literature, e.g. [1,2]. Why is this definition not relevant here, since privacy is listed as one of the applications? See the below point too.

- The authors don’t present any comparisons with retrain-from-scratch which is assumed to be the gold standard for unlearning, at least according to some definitions, as discussed above (and thus serves as a reference point for the desired level of performance, e.g. how high the accuracy on the forget set should be). Several metrics have been developed, like membership inference attacks and other statistical tests according to this definition [3,4]. The authors also state that retrain-from-scratch is the gold standard solution (and that privacy is one of the applications of interest) but don’t use these metrics. It would be great to either expand the experiments accordingly or adjust the scope of claims to include only the unlearning formulation (and applications) that best corresponds to the experiments conducted here. 

- Related work and baselines: [5] seems closely related to this work. It would be great to discuss the differences and compare empirically. It would also be great to compare against Safe Latent Diffusion [6]. In the experiments, how were the baselines chosen? Why not include also (Zhang et al, 2023), that was mentioned in section 2.2?

Some claims are not well substantiated:
- “these models, trained on vast amounts of public data, inevitably face concerns related to privacy” – if the training data is public, what are the privacy considerations?

- “In this study, we aim to address the crucial concerns of harmful content generation and copyright infringement [...] by focusing on the removal of target classes or concepts” – it is not clear to me how copyright infringement can be addressed by removing classes or concepts. Can you please elaborate on the setup that you have in mind?

Clarity issues relating to the experimental setup:
- For UA, why is it that higher is better? I would have thought the opposite: unlearning a concept results in poorer accuracy on classifying images of that concept. This concern is also tied to the fact that the paper is missing a problem formulation, making it hard to understand how success is defined.

- What is the rationale of evaluating model utility on the retain set only, and not the test set (e.g. in Table 1)?

- the authors discuss the size of the retain set and how it’s sampled but not the size / sampling approach of the forget set. In general, several experimental details are lacking about the benchmarks used (if they are in the appendix, please refer to the relevant sections from the main paper). In CIFAR-10, for instance, is the forget set comprised of all and only the examples of a particular class? If so, which one? In the SD experiments, how is the forget set generated? 

- Table 1 caption “The metrics are averaged across all 10 classes” - the setup here is unclear; is there a different class that is unlearned each time, so UA is computed according to that one class, and then this process is repeated for unlearning different classes, and all UA’s are finally averaged?

- In Table 2, how are D_{r,train} / D_{r, test} generated for the different applications? 

- “prompted by I2P” – what is I2P? Please add a citation if possible. 

Minor
- “competitive nature” → “competing nature”?

- “generative reply” → “generative replay”?

- The sentence at the end of line 227 is incomplete “To fairly compare,”.


References

[1] Making AI Forget You: Data Deletion in Machine Learning. Ginart et al. 2019.

[2] Descent-to-delete: Gradient-based Methods for Machine Unlearning. Neel et al. 2020.

[3] Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy. Hayes et al. 2024.

[4] Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition. Triantafillou et al. 2024.

[5] One-dimensional adapter to rule them all: concepts, diffusion models and erasing applications. Lyu et al. 2023.

[6] Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models. Schramowski et al. 2023.
questions: questions: - I don’t understand how the proposed approach for diversifying the retain set in the case where the textual input is unconstrained actually achieves diversification (or to what extent). If I understand correctly, some prompts are first generated that relate to the target concept to erase, and then, to generate the retain set, these text prompts are modified for the purpose of removing words related to the target concept, and then retain images are generated from these modified prompts. But given that the prompts were initially generated to relate to the target concept, even with erasing some words, perhaps these prompts maintain some relationship to the target concept; or at least don’t uniformly cover the space of possible prompts. It seems that generating an initial set of prompts that aren’t necessarily related to the target concept (instead, they can relate to a wide range of diverse concepts) would yield more diversity. Did the authors consider this? In what ways do the authors view this approach as diversification? How does it relate to the approach usually taken in the literature for constructing the retain set?

- How different do the authors think the accuracy results would be if using a different pretrained model to assess this? Would the overall conclusions or relative rankings between methods be the same? 

- The authors claim that their solution offers a monotonic improvement of each task. Is there any empirical evidence to support this claim?

Regarding my overall score / assessment, I am leaning towards weak reject due to issues with the problem description, the motivation and associated applications, and describing / justifying design decisions in the experimental protocol, tying in experiments a little better with related work (see the weaknesses section). I look forward to the author's rebuttal especially on those points.
limitations: limitations: I encourage the authors to think about potential societal impact of their work.
paper_summary: paper_summary: This work addresses the issue in generative models where powerful models may be generating harmful or undesired contents that should be unlearned. This work proposes to balance the unlearning objective and the text-image alignment on the remaining data, by identifying a gradient direction that achieves a monotonic decrease of both objectives (with theoretical analysis), along with a LLM-powered augmentation to encourage the diversity in the data to be unlearned. The experiments show capacity of the proposed method’s capacity in unlearning, emphasizing its retaining text-image alignment.
strength_weakness: strength_weakness: While this work claims to be able to do unlearning for both forgetting harmful concepts / copy-righted styles and forgetting individual objects (at least as the introduction suggests), the overall seems to be limited to the former one. For example, the diversity object is through LLM produced examples, thus  limiting to concepts that are expressible in the text, rather than obese supported by image examples. Also, the experiments do not show removing identities (e.g. celebrities), which are demonstrated in previous works.

The improvements over Salum are shown mostly in quantitative results. Qualitatively, one may argue the difference is small. This should be better explained.
questions: questions: N/A
limitations: limitations: Yes. This work addresses the negative impact of generative models, which the work should be awarded for.
paper_summary: paper_summary: This paper presents a novel perspective on model unlearning for text-to-image generative models, considering it a constraint optimization problem. It introduces a new loss function for the unlearning process, which is a combination of the remaining loss and the forgetting loss. The paper's key technical contribution is the concept of "restricted gradient." Additionally, the proposed method uses an "LLM in the loop" approach.
strength_weakness: strength_weakness: - The use of the proposed loss function isn't sufficiently justified. The unlearning process employs a combination of the remaining loss and the forgetting loss, with the latter being the negative of the loss used to train the diffusion model. Making a random prediction without considering image fidelity would minimize the forgetting loss. In this work, the proposed method seems to work due to the balance between the two losses and the restricted gradient. Nonetheless, justifying the use of the forgetting loss remains challenging.
- The diversification process largely depends on the performance of LLM. For the class conditional tasks, the stratified sampling technique can work since the number of classes is quite limited. For the target concept removal tasks, it is unclear how to obtain diverse prompts that can retain the performance of the diffusion models (since there will be more than a million concepts in reality).
- This paper's primary technical contribution revolves around the concept of "restricted gradient." However, no experimental results support this technical contribution.
questions: questions: - When training is complete, the remaining loss is zero or close to zero. Therefore, the constraints defined in definition 3 might be violated. What action does the optimizer take under such a condition?
- What is the performance of the baseline diffusion mode without any unlearning method in Table 1? Due to the classifier performance, the baseline may not achieve 100% accuracy for both UA and RA.
- How robust the clip alignment score is? Is clip alignment score well aligned with human judgement, especially in the context of artist removal?
- Do you have any detailed ablation studies on the "LLM in the loop" process? Does the number of prompts used to generate D_f and D_r affect the overall performance? If so, what would be your practical suggestions for this process?
limitations: limitations: Limitations are well addressed.

Comments: title: Official Comment by Authors
response to authors
Additional Questions
Additional Question
Additional Questions.
Clarity issues relating to the experimental setup:
Some claims are not well substantiated:
Comparison to other baselines
comment: I thank the authors for their rebuttal, which has well-addressed my concerns. I keep my positive recommendation for acceptance.
Thanks for the additional experiments and further clarification. After reading the rebuttal and other reviews, I decided to maintain my original score.
The authors rebuttals addressed some of my concerns. Based on that and the conversation between the authors and other reviewers, I updated my rating.
Dear Reviewer RWKV,

We are happy to hear that our responses have addressed your concerns and questions. We appreciate you taking the time to read our rebuttal and adjust your evaluation accordingly. We will incorporate all the clarifications, additional experimental results, and suggested modifications discussed during the rebuttal into our revised version.
Thank you once again for your valuable, constructive feedback and for your consideration.

Best regards, 

Authors
Dear authors,

Thank you for the thorough responses.

I am satisfied with the discussion of the problem definition and with the promised modifications on clarifying the problem definition and the scope of the paper (in particular, discussing the application-specific definition of "Unlearning" as defined in the rebuttal, the differences from privacy definitions and reducing the scope of the claims to exclude privacy applications).

Thank you as well for all of the the clarifications, including regarding the copyright application, the UA metric (please do emphasize more that it's 1 - the accuracy, as I found this not to be obvious), and the experimental setup. I also appreciated the additional experiments with the different pretrained model, to test generalizability, and the explanation and results re: the "monotonic improvement" question.

I will increase my score in light of the above. Please do incorporate these clarifications in the paper (or in a section in the appendix).

thanks!
We thank the reviewer for those questions. These are great comments and feedback.
 
**Performance of the baseline diffusion model in Table 1.**

Thank you for this important point. We've added the baseline SD performance on UA/RA/FID as follows:

- **UA (↑):** 0.052
- **RA (↑):** 0.955
- **FID (↓):** 3.815

We will incorporate this result in the revised version.

**Robustness of the CLIP alignment score. Is it aligned with human judgment?**

This is a very insightful question. We thank the reviewer for this comment. We acknowledge that there might be a gap between the clip alignment score and human judgment. Therefore, we additionally conducted the human evaluation and attached the results here.

|          | Human Judgment on $D_r$ (↑) | AS on $D_r$ (↑)  |
|----------|----------------|-----------|
| SD       | 3.4            | 0.348     |
| ESD      | 3.0            | 0.330     |
| Salun    | 1.1            | 0.280     |
| RGD (Ours) | 3.2            | 0.352     |

For human judgment, we collected responses from 9 subjects who were asked to score a test set. The scoring range was from 1 (least aligned with a given prompt) to 5 (most aligned with a given prompt).

Our results demonstrate that our method achieves judgment scores close to those of SD, while Salun performs poorly. The relative ranking on the retain set aligns well with the CLIP alignment score. We will incorporate these results into our revised version.
Thank you again for your feedback.

**Detailed ablation studies on the LLM in the loop process. Does the number of prompts affect performance?**

We thank the reviewer for this great question. We acknowledge that it is important to study the impact of dataset size from “LLM in the loop” and therefore, we have investigated how varying the size of $D_f$ and $D_r$ affects performance. Here are our key findings:

Consistency: Across different sizes (400, 800, 1200), our method maintains higher alignment scores. 

Optimal Size: The size of 800 (reported in our paper) shows the best balance of performance. It matches the setting used in [1], allowing for fair comparison.

Smaller Set (400): While still effective, this size shows a slight decrease in alignment scores. This is likely due to simply increased iterations on a smaller dataset.  

Larger Set (1200): This size can achieve high alignment scores comparable to 800 if we reduce $\alpha$ from 1.5 to 1.15 to balance the increased gradient ascent steps.

Therefore, the practical suggestion will be in general, it would be beneficial to include more diverse samples for unlearning to maintain the model utility. 

**Ablation on the size of $D_f$ and $D_r$ for Nudity Removal.**

|  AS (↑)       |  $D_{r, train}$  | $D_{r, test}$  |
|------------------------|------------------------------|------------------------------|
| SD  | 0.357                        | 0.352                        |
| RGD (Ours) \|$D_r$\| = \|$D_f$\| = 400 | 0.336 (0.021)               | 0.339 (0.013)               |
| RGD (Ours) \|$D_r$\| = \|$D_f$\| = 800 | 0.354 (0.003)               | 0.350 (0.002)               |
| RGD (Ours) \|$D_r$\| = \|$D_f$\| = 1200 | 0.352 (0.005)               | 0.346 (0.006)               |


We believe that we have initiated the importance of proper usage of 'LLM in the loop' through this paper, and a more comprehensive study of the design will be valuable for future work.

[1] SALUN: EMPOWERING MACHINE UNLEARNING VIA GRADIENT-BASED WEIGHT SALIENCY IN BOTH IMAGE CLASSIFICATION AND GENERATION
**The authors claim that their solution offers a monotonic improvement of each task. Is there any empirical evidence to support this claim?**

We appreciate your question regarding empirical evidence for our claim of monotonic improvement. Our claim refers to consistent improvement on both objectives—forgetting the target concept (the 'forget' task) and maintaining performance on retained concepts (the 'retain' task)—without sacrificing one for the other. Theoretically, our claim for monotonic improvement is guaranteed given access to the true loss gradients in Eq. 4, and as the step size tends towards zero. In practice, there are two considerations:

1. The step size cannot be infinitely small and must be balanced against the computational budget of the training algorithm, as smaller step sizes also result in longer training times.

2. The true expected loss (i.e., the expectation of the loss over the entire data distribution) is not tractable, and we must approximate it via the empirical batch-wise loss. This results in minor deviations at each step and a stochastic approximation of the gradients in Eq. 4.

To verify our claim in the empirical setting, we perform unlearning with our approach and baseline for the class-wise forgetting experiment, please see Figure 2 in the PDF:

- **Forget Losses Comparison:** The left figure shows the loss for the 'forget' task. Our method maintains a higher loss compared to the baseline. This higher loss indicates better 'forgetting' of the target concept, as we want the model to perform poorly on this task.

- **Retain Losses Comparison:** The right figure shows the loss for the 'retain' task. Our method maintains a relatively stable and low loss throughout the iterations, especially when compared to the baseline, which shows a sharp increase after 300 iterations. This stability is crucial as it indicates our method consistently preserves model utility on retained concepts.

We note that the retain loss doesn't show a decreasing trend because our primary goal for the retain set is to maintain utility, not necessarily to improve it. The unlearning process naturally tends to increase the retain loss as we do gradient ascent. However, our "$\min D_r$" objective counteracts this increase (i.e., decrease the retain loss), resulting in the observed stability.
**Explain the proposed approach for diversifying the retain set. In particular, why not generate an initial set of prompts (that aren’t necessarily related to the target concept)?**

Thank you for your insightful question about our approach to diversifying the retain set. 
We fully agree with your suggestion that generating an initial set of prompts unrelated to the target concept would yield more diversity. In fact, this is precisely the approach we take. 

To clarify: our initial set of retain prompts ($D_r$) is generated to cover a wide range of diverse dimensions unrelated to the target concept we aim to erase. These dimensions include activities, environments, times, moods, and others. At the same time, our retain set aims to preserve the model's understanding of a broader category (e.g., "a person") that could be potentially affected by erasing “nudity.”

To create $D_f$, we include target-related concept words (e.g., "nude", "naked") in the diverse prompts generated. For example, a prompt in $D_r$ might be "A person walking in a forest at sunset", while the corresponding $D_f$ prompt would be "A nude person walking in a forest at sunset". This approach ensures no direct relationship between the target concept and the retain concepts, as the initial prompts are generated independently of the target concept.

We will incorporate the details of generating a diversified retain set in our revised paper. 

**How does it relate to the approach usually taken in the literature for constructing the retain set?**

We appreciate your question about how our approach relates to existing methods in the literature. The design of the retain set has been largely overlooked in previous studies, despite its crucial role. For example,

**ESD** doesn't utilize retain set information, yet claims alignment scores comparable to the original SD based on COCO dataset evaluation. However, our evaluation using "person"-related benign prompts reveals a drop in their alignment scores (as shown in Table 2), likely due to the interconnection between erasing "nudity" and general person image generation.

**Salun** uses single repeated prompts for the forget (e.g., "a photo of a nude person") and retain (e.g., "a photo of a person wearing clothes") sets, which can potentially lead to overfitting.

Therefore, we believe we have initiated the discussion on the importance of carefully designing the retain set, and our more systematic approach to retain set design opens up interesting avenues for further research.

**How different would accuracy results be if using a different pre-trained model? Would the overall conclusions or relative rankings between methods be the same?**

 Thank you for this important question about the generalizability of our results. To address this, we've conducted additional evaluations using SD v3, the most recent version of the pre-trained model. Our approach and findings are as follows (please refer to the tables in the general response):

*Model Architecture:* SD v3 employs a transformer-based architecture (e.g., Diffusion Transformer models) instead of the UNet-based architecture used in previous versions. This significant change allows us to test our method's performance across different model structures.

*Model Size:* SD v3 offers a range of model sizes, with the largest being nearly 10 times the size of v1.4. We choose a medium size model with 2B parameters, which is approximately 2 times larger than v1.4. This variability enables us to assess how our method performs across different model capacities.

*Evaluation Approach:* We maintained the same hyperparameter settings as in v1.4 to ensure an easy generalization capability. We evaluated two baselines alongside our method, observing their performance under multiple hyperparameter tunings.

*Results:* a) Alignment Scores: We observed high alignment scores for both $D_{r,train}$ and $D_{r,test}$ splits with SD v3, effectively mitigating harmful output generation. b) Baseline Comparison: Both baselines showed significant alignment score drops with multiple hyperparameter tunings, and our method continued to outperform them.
**UA – how come higher is better on this metric?**

We thank the reviewer for pointing out the potential confusion regarding the UA metric. We define UA as 1 - accuracy of the unlearned model on $D_f$ (as noted in line 232 of our paper). This definition aims to measure the model's inability to generate or correctly classify forgotten concepts. Higher UA values indicate better unlearning performance, as they represent a greater error rate on the forget set. In other words, a higher UA suggests that the model has effectively 'forgotten' how to generate or recognize the target concepts. We adopted this metric definition to maintain consistency with previous research [1].

**Rationale of evaluating model utility on the retain set only, not the test set.**

Thank you for your question about our evaluation. Our evaluation pipeline for Table 1 focuses on two main aspects: the effectiveness of forgetting the target class and the preservation of model utility for the remaining classes. We don't use a separate test set in the traditional sense but rather evaluate on generated images. 

*Remaining Accuracy (RA)* is used to evaluate whether the unlearned model can still generate the remaining classes correctly after erasing the target class. We ask the unlearned model to generate images belonging to the remaining classes, which effectively serve as a 'test set'. We calculate the classification performance of these generated images to ensure each class of remaining images is well preserved in the unlearned model. 

*Unlearning Accuracy (UA)* evaluates how effectively the model has forgotten the target class. We prompt the unlearned model to generate images of the target class and then use a pre-trained classifier to determine if these generated images still belong to the target class. We will make it clear in our revised version by including more details. Thank you again for your question.

**Missing experimental details for forget set construction in CIFAR-10 and SD experiments.**

Thank you for your great point. We acknowledge the lack of detail regarding the forget set in our initial presentation and will incorporate these details in our revised version. For CIFAR-10 experiments: The forget set $D_f$ comprises all images of a particular target class, which is 5000 images. For Stable Diffusion (SD) experiments: Our forget set has 800 images with the corresponding prompts. The forget prompts are generated by adding the target concept (e.g., "nudity" or specific artist styles) into the retain prompts ($D_r$).

**Table 2: How are $D_\text{r, train}$, $D_\text{r, test}$ generated?**

We appreciate the reviewer's questions regarding the generation of $D_{r,train}$ and $D_{r,test}$ sets. 

*For the nudity removal application:* a) We use a structured approach to generate diverse prompts for $D_r$, considering multiple dimensions such as activities, environments, times, and moods provided by a Large Language Model (LLM). b) For each dimension, we use LLMs to suggest multiple subconcepts, incorporating diverse semantics belonging to each dimension such as walking, and sitting in activities. c) To create $D_{r,train}$ and $D_{r,test}$, we split these subconcepts in each dimension into train and test sets, ensuring that there is no overlap between train and test sets. d) We then combine these subconcepts to generate $D_r$. 

*For the style removal application:* similar to nudity removal, we construct some templates with multiple dimensions such as the artist’s name, actions, environments, and moods, then fill in each dimension with the suggestions from LLMs. Compared between retain set and forget set, the only difference is in the forget set ($D_f$) we use the name of the target that we want to unlearn (e.g., Van Gogh), and use other artists’ names or some virtual names in the retain set ($D_r$). We will revise the paper to incorporate these details, providing a clearer and more comprehensive explanation of our method for generating $D_{r,train}$ and $D_{r,test}$ in each application.

**Table 1: How are the metrics computed across the 10 classes?**

Thank you for bringing attention to this potentially confusing point. The interpretation provided is correct and reflects our intended meaning. To clarify: the caption "The metrics are averaged across all 10 classes" indeed means that we repeat the unlearning process for each class separately, and the results are averaged to produce the final metrics presented in the table. We will revise the caption to make this clearer in the revised version.

**What is I2P?**

We thank you for your question for further clarification. I2P stands for "Inappropriate Image Prompts", a collection of prompts designed to evaluate the effectiveness of content moderation in text-to-image models. We have cited this work in line 237 of our paper.

[1] SALUN: EMPOWERING MACHINE UNLEARNING VIA GRADIENT-BASED WEIGHT SALIENCY IN BOTH IMAGE CLASSIFICATION AND GENERATION
**What are the privacy considerations of training on public data?**

We thank the reviewer for highlighting this point. Upon reflection, we agree that our statement about privacy concerns in the context of models trained on public data might be too broad for the scope of our paper, given that our work focuses on concept removal instead of privacy. We will revise the paper to calibrate the framing.

**Elaboration on how copyright infringements can be addressed by removing concepts from a model.**

Thank you for your insightful question regarding the motivation and associated application of our work. Our focus on copyright infringement is motivated by recent developments in the field of generative AI. As described in [1], companies like Stability AI and MidJourney are facing civil lawsuits for training their models on artists' work without consent, enabling their models to replicate the style and work of these artists.

In response to this issue, we study "style removal" as a practical application of our unlearning method on the copyright infringement application. For example, we aim to demonstrate that our method can effectively remove a particular artist's style from a model's capabilities. This removal would prevent the model from generating images in the style of the artist whose work was used without permission, thus addressing the copyright infringement issue.

[1] On Copyright Risks of Text-to-Image Diffusion Models
We appreciate the reviewer's thoughtful suggestions regarding baseline comparisons. Our baseline selection was guided by two main principles:

**Complementarity:** 
In inexact unlearning, we can broadly categorize approaches into different categories:
1. **Modifying the parameters:** These methods directly alter the model's weights to remove target knowledge.
2. **Modifying the inference:** These approaches change the inference process without altering the original model.

Our work falls into the first category, focusing on modifying the model's parameters. Safe Latent Diffusion (SLD) [1] modifies the inference process to prevent certain concepts from being generated, which falls into the second category. We consider works from the second category as complementary to our approach. Moreover, the method proposed in [2] trains only an adapter and can be applied as a plug-and-play solution to other pre-trained models, which differs from our parameter-level modifications. We believe this also falls into the second category. Therefore, we did not include these methods in our comparisons, as they could potentially be used in conjunction with our method rather than as direct alternatives. However, since [6] falls into the first category, we have added additional comparison results in the general response. 

**State-of-the-art performance:**
We prioritized comparisons with methods that represent the current state-of-the-art in the field. For example, both [1] and [3] have been outperformed by more recent methods such as [4] and [5], which we included in our comparisons.


[1] Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models

[2] One-dimensional adapter to rule them all: concepts, diffusion models, and erasing applications.

[3] Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models

[4] Erasing Concepts from Diffusion Models

[5] SALUN: EMPOWERING MACHINE UNLEARNING VIA GRADIENT-BASED WEIGHT SALIENCY IN BOTH IMAGE CLASSIFICATION AND GENERATION

[6] Ablating Concepts in Text-to-Image Diffusion Models