Title: Boosting Alignment for Post-Unlearning Text-to-Image Generative Models
Venue: NeurIPS 2024
Accepted: yes
Keywords: unlearning

diffusion models

stable diffusion
Abstract: Large-scale generative models have shown impressive image-generation capabilities, propelled by massive data. However, this often inadvertently leads to the generation of harmful or inappropriate content and raises copyright concerns. Driven by these concerns, machine unlearning has become crucial to effectively purge undesirable knowledge from models. While existing literature has studied various unlearning techniques, these often suffer from either poor unlearning quality or degradation in text-image alignment after unlearning, due to the competitive nature of these objectives. To address these challenges, we propose a framework that seeks an optimal model update at each unlearning iteration, ensuring monotonic improvement on both objectives. We further derive the characterization of such an update.
  In addition, we design procedures to strategically diversify the unlearning and remaining datasets to boost performance improvement. Our evaluation demonstrates that our method effectively removes target classes from recent diffusion-based generative models and concepts from stable diffusion models while maintaining close alignment with the models' original trained states, thus outperforming state-of-the-art baselines.

Boosting Alignment for Post-Unlearning Text-to-Image Generative Models

Abstract

Large-scale generative models have shown impressive image-generation capabilities, propelled by massive data. However, this often inadvertently leads to the generation of harmful or inappropriate content and raises copyright concerns. Driven by these concerns, machine unlearning has become crucial to effectively purge undesirable knowledge from models. While existing literature has studied various unlearning techniques, these often suffer from either poor unlearning quality or degradation in text-image alignment after unlearning, due to the competitive nature of these objectives. To address these challenges, we propose a framework that seeks an optimal model update at each unlearning iteration, ensuring monotonic improvement on both objectives. We further derive the characterization of such an update. In addition, we design procedures to strategically diversify the unlearning and remaining datasets to boost performance improvement. Our evaluation demonstrates that our method effectively removes target classes from recent diffusion-based generative models and concepts from stable diffusion models while maintaining close alignment with the models' original trained states, thus outperforming stateof-the-art baselines. Our code will be made available at https://github.com/ reds-lab/Restricted_gradient_diversity_unlearning.git .

1 Introduction

Large-scale text-to-image generative models have recently gained considerable attention for their impressive image-generation capabilities. Despite being at the height of their popularity, these models, trained on vast amounts of public data, inevitably face concerns related to harmful content generation [Heng and Soh, 2024] and copyright infringement [Zhang et al., 2023b]. Although

Score

Post -Unlearning

Concept

Condition

SD (Before)

Salun

ESD

RGD (Ours)

A anxious person

Nudity

standing at a quiet

beach as the sun

rises.

Alignment

0.352

0.305

0.329

0.350

Score

A dog sitting in a

field in the style of

Alignment

0.348

0.280

0.330

0.352

exact machine unlearning-retraining the model by excluding target data-is a direct solution, its computational challenge has driven continued research on approximate machine unlearning.

To address this challenge, recent studies [Fan et al., 2023, Gandikota et al., 2023, Heng and Soh, 2024], have introduced approximate unlearning techniques aimed at boosting efficiency while preserving effectiveness. These approaches have successfully demonstrated the ability to remove target concepts while maintaining the model's general image generation capabilities, with generation quality assessed using the Fréchet Inception Distance. However, these studies generally overlook the impact of unlearning on image-text alignment, which pertains to the semantic accuracy between generated images and their text descriptions [Lee et al., 2024]. While pretrained generative models generally demonstrate high alignment scores, our study reveals a critical gap: state-of-the-art unlearning techniques fall short in achieving comparable text-image alignment scores after unlearning, as illustrated in Figure 1. This could lead to potentially problematic behaviors in real-world deployments, necessitating further investigation.

We attribute the failure of existing techniques to maintain text-image alignment to two primary factors. Firstly, the unlearning objective often conflicts with the goal of maintaining low loss on the retained data, illustrating the competitive nature of these two objectives. Traditionally, approaches to optimizing these objectives have simply aggregated the gradients from both; however, this method of updating the model typically advances one objective at the expense of the other. Hence, while these approaches may successfully remove target concepts, they often compromise text-image alignment for retained concepts in the process. Secondly, current methods employ a simplistic approach to constructing a dataset for loss minimization on retained concepts. For example, in Fan et al. [2023], this dataset is composed of images generated from a single prompt associated with the concept to be removed. This lack of diversity in the dataset might lead to overfitting, which in turn hampers the text-image alignment.

To address these issues, we propose a principled framework designed to optimally balance the objectives of unlearning the target data and maintaining performance on the remaining data at each update iteration. Specifically, we introduce the concept of the restricted gradient , which allows for the optimization of both objectives while ensuring monotonic improvement in each objective. Furthermore, we have developed a deliberate procedure to enhance data diversity, preventing the model from overfitting to the limited samples in the remaining dataset. To the best of our knowledge, the strategic design of the forgetting target and remaining sets has not been extensively explored in the existing machine unlearning literature. In our evaluation, we demonstrate the improvement in both forgetting quality and alignment on the remaining data, compared to baselines. For example, our

evaluation in nudity removal demonstrates that our method effectively reduces the number of detected body parts to zero, compared to 598 with the baseline stable diffusion ( SD ) [Rombach et al., 2022], 48 with erased stable diffusion ( ESD-u ), and 3 with saliency map-based unlearning ( SalUn ) [Fan et al., 2023]. Particularly, while achieving this effective erasing performance, our method reduces the alignment gap to SD by 11x compared to ESD-u and by 20x compared to SalUn on the retained test set.

2 Related Work

2.1 Machine Unlearning

Machine unlearning has primarily been propelled by the "Right to be Forgotten" (RTBF), which upholds the right of users to request the deletion of their data. Given that large-scale models are often trained on web-scraped public data, this becomes a critical consideration for model developers to avoid the need for retraining models with each individual request. In addition to RTBF, recent concerns related to copyrights and harmful content generation further underscore the necessity and importance of in-depth research in machine unlearning. The principal challenge in this field lies in effectively erasing the target concept from pre-trained models while maintaining performance on other data. Recent studies have explored various approaches to unlearning, including the exact unlearning method [Bourtoule et al., 2021] and approximate methods such as using negative gradients, fine-tuning without the forget data, editing the entire parameter space of the model [Golatkar et al., 2020]. To encourage the targeted impact in the parameter space, [Golatkar et al., 2020, Foster et al., 2024] proposed leveraging the Fisher information matrix, and [Fan et al., 2023] leveraged a gradient-based weight saliency map to identify crucial neurons, thus minimizing the impact on remaining neurons. Furthermore, data-influence-based debiasing and unlearning have also been proposed [Chen et al., 2024, Bae et al., 2023]. Another line of work leverages mathematical tools in differential privacy [Guo et al., 2019, Chien et al., 2024] to ensure that the model's behavior remains indistinguishable between the retrained and unlearned models.

2.2 Machine Unlearning in Diffusion Models

Recent advancements in text-conditioned generative models [Ho and Salimans, 2022, Rombach et al., 2022], trained on extensive web-scraped datasets like LAION-5B [Schuhmann et al., 2022], have raised significant concerns about the generation of harmful content and copyright violations. A series of studies have addressed the challenge of machine unlearning in diffusion models [Heng and Soh, 2024, Gandikota et al., 2023, Zhang et al., 2023a, Fan et al., 2023]. One approach [Heng and Soh, 2024] interprets machine unlearning as a continual learning problem, showing effective removal results in classification tasks by employing Bayesian approaches to continual learning [Kirkpatrick et al., 2017], which enhance unlearning quality while maintaining model performance using generative reply [Shin et al., 2017]. However, this approach falls short in removing concepts such as nudity compared to other methods [Gandikota et al., 2023]. Another proposed method [Gandikota et al., 2023] guides the pre-trained model toward a prior distribution for the targeted concept but struggles to preserve performance. The most recent work [Fan et al., 2023] proposes selectively damaging neurons based on a saliency map and random labeling techniques, although this method tends to overlook the quality of the remaining set, focusing on improving the forgetting quality, which does not fully address the primary challenges in the machine unlearning community. Although [Bae et al., 2023] presents a similar multi-task learning framework for variational autoencoders, their work does not show the optimality of their solution, and their experiments mainly focus on small-scale models, due to the computational expense associated with influence functions.

3 Our Approach

We study the efficacy of our approach in unlearning by removing target classes from class-conditional diffusion models or eliminating specific concepts from text-to-image models while maintaining their general generation capabilities. We will call the set of data points to be removed as the forgetting dataset . To set up the notations, let D denote the training set and D f ⊂ D be the forgetting dataset. Wewill use D r = D \ D f to denote the remaining dataset . Our approach only assumes access to some representative points for D f and D r . As discussed later, depending on specific applications, these data

points can be either directly sampled from D f and D r or generated based on the high-level concept of D f to be removed. With a slight abuse of notation, we will use D r and D f to also denote the actual representative samples used to operationalize our proposed approach. Furthermore, we denote the model parameter by θ . Let l be a proper learning loss function. The loss of remaining data and that of forgettng data are represented by L r ( θ ) := ∑ z ∈ D r l ( θ, z ) and L f ( θ ) := -λ ∑ z ∈ D f l ( θ, z ) , respectively, where λ is a weight adjusting the importance of forgetting loss relative to the remaining data loss. We term L r and L f remaining loss and forgetting loss , respectively. We note that in the context of diffusion models, loss function l is defined as l = E t,x 0 ,ϵ ∼N (0 , 1) [ ∥ ϵ -e θ ( x t , t ) ∥ 2 ] , where x t is a noisy version of x 0 generated by adding Gaussian noise to the clean image x 0 ∼ p data ( x ) at time step t with a noise scheduler, and e θ ( x t , t ) is the model's estimate of the added noise ϵ at time t [Xu et al., 2023, Ho et al., 2020]. For text-to-image generative models, the loss function l is specified as l = E t,q 0 ,c,ϵ [ ∥ ϵ -ϵ θ ( q t , t, η ) ∥ 2 ] , where q 0 is an encoded latent q 0 = E ( x 0 ) with encoder E , and q t is a noisy latent at time step t . The noise prediction ϵ θ ( q t , t, η ) is conditioned on the timestep t and a text η .

Optimizing the Update. Similar to existing work Fan et al. [2023], our objective is to find an unlearned model with parameters θ u , starting from a pre-trained model with weights θ 0 , such that the model forgets the target concepts in D f while maintaining its utility on the remaining dataset D r . Formally, we aim to maximize the forget error on D f , represented by L f ( θ ) , while minimizing the retain error on D r , represented by L r ( θ ) . This can be formulated as min θ L r ( θ ) + L f ( θ ) , where our approach applies iterative updates to achieve both goals simultaneously. A simple approach to optimize this objective, often adopted by existing work, is to calculate the gradient ∇L r ( θ )+ ∇L f ( θ ) and use it to update the model parameters at each iteration. However, empirically, we observe that the two gradients usually conflict with each other, i.e., the decrease of one objective is at the cost of increasing the other; therefore, in practice, this approach yields a significant tradeoff between forgetting strength and model utility on the remaining data. In this work, we aim to present a principled approach to designing the update direction at each iteration that more effectively handles the tradeoff between forgetting strength and model utility on the remaining data. Our key idea is to identify a direction that achieves a monotonic decrease of both objectives.

To describe our algorithm, we briefly review the directional derivative.

Definition 1 (Directional Derivative) . The directional derivative [Spivak, 2006] of a function f at x in the direction of v is written as

This special form of the derivative has the useful property that its maximizer can be related to the gradient ∇ x f ( x ) , which we formally state below.

Theorem 2 (Directional derivative maximizer is the gradient) . Let f be a function on x . Then the maximum value of the directional derivative of f at x is |∇ f ( x ) | the ℓ 2 norm of its gradient. Moreover, the direction v is the gradient itself, i.e.,

In unlearning, we are specifically interested in the gradient of two losses, the forgetting loss L f and the remaining loss L r . Moreover, we seek gradient directions that simultaneously improve on both. This motivates the restricted gradient , which we define below.

Definition 3 (Restricted gradient, local form for minimization) . The negative restricted gradient of two losses L α , L β is any direction v at θ satisfying

Intuitively, with the restricted gradient we seek to define the ideal direction for unlearning. We would like to optimize the joint loss L = L r + L f subject to the condition that at every parameter update step, L r and L f experience monotonic improvement. This is precisely the step prescribed by the negative restricted gradient. Since the learning rates used to fine-tune the parameters in the unlearning process are typically quite small, we can approximate the updated loss at each iteration via a simple first-order Taylor expansion. In this case, the restricted gradient takes a simple form.

Theorem 4 (Characterizing the restricted gradient under linear approximation) . Given θ , suppose that L r ( θ + δ ) -L r ( θ ) ≈ δ · ∇L r and L f ( θ + δ ) -L f ( θ ) ≈ δ · ∇L f . The restricted gradient can be written as

where

when we have conflicting unconstrained gradient terms, i.e. ∇L f · ∇L r < 0 .

The theorem presented demonstrates that the restricted gradient is determined by aggregating the modifications from ∇L f and ∇L r . This modification process involves projecting ∇L f onto the normal vector of ∇L r , yielding δ ∗ f , and similarly projecting ∇L r onto the normal vector of ∇L f , resulting in δ ∗ r . The optimal update, as derived in Theorem 4, is illustrated in Figure 2. Notably, when ∇L f and ∇L r have equal norms, the restricted gradient matches the direct summation of the two original gradients, namely, ∇L f + ∇L r . However, it is more common for the norm of one gradient to dominate the other, in which case the restricted gradient provides a more balanced update compared to direct aggregation.

VLf

VLf

VLf+VLr

S

*

+

8*

VLr

VLr

(a) Direct aggregate

(b) Restricted gradient

Remark 1. We wish to highlight an intriguing link between the gradient aggregation mechanism presented in Theorem 4 and an existing method to address gradient conflicts across different tasks in multi-task learning. This restricted gradient coincides exactly with the gradient surgery procedure introduced in Yu et al. [2020]. While their original paper presented the procedure from an intuitive perspective, our work offers an alternative viewpoint and rigorously characterizes the objective function that the gradient surgery procedure optimizes.

Diversify D r . Since D \ D f is usually of enormous scale, it is infeasible to incorporate all of them into the remaining dataset D r for running the optimization. In practice, one can only sample a subset of points from D r . In our experiments, we find that the diversity of D r plays an important role in maintaining the model performance on the remaining dataset, as seen in Section 4.2. Therefore, we propose procedures for forming a diverse D r . For models with a finite set of class labels, such as diffusion models trained on CIFAR-10, we adopt a simple procedure of maintaining an equal number of samples for each class in D r . Our ablation studies in Section 4.4 show that this is more effective in maintaining model performance on the remaining dataset than more sophisticated procedures, such as selecting the most similar examples to the forgetting samples. The intuitive reason is that reminding the model of as many fragments as possible related to the remaining set during each forgetting step is crucial. By doing so, it leads to finding a representative restricted descent direction, which helps the model to precisely erase the forget data while maintaining a state comparable to the original model. When the text input is unconstrained, such as in the stable diffusion model setting, to strategically design diverse information, we propose the following procedure to generate D r based on the concept to be forgotten, denoted by c . Using a large language model (LLM), we first generate diverse text prompts related to concept c , yielding prompt set Y c . These prompts are then modified by removing all references to c , creating a parallel set Y . By passing both Y c and Y through the target diffusion model, we obtain corresponding image sets X c and X . This process allows us to construct our final datasets: D f = { ( x, y ) | x ∈ X c , y ∈ Y c } and D r = { ( x, y ) | x ∈ X , y ∈ Y} . Example prompts and detailed descriptions are provided in Appendix D.

4 Experiment

In this study, we address the crucial challenge of preventing undesirable outputs in text-to-image generative models. We begin by examining class-wise forgetting with CIFAR-10 diffusion-based generative models, where we demonstrate our method's ability to selectively prevent the generation of specific class images (Section 4.2). We then explore the effectiveness of our approach in removing nudity and art styles (Section 4.3) to address real-world concerns of harmful content generation and copyright infringement. We further study the impact of data diversity (Section 4.4) as well as the sensitivity of our method to hyperparameter settings (Section 4.4).

4.1 Experiment Setup

For our CIFAR-10 experiments, we leverage the EDM framework [Karras et al., 2022], which introduces some modeling improvements including a nonlinear sampling schedule, direct x 0 -prediction, and a second-order Heun solver, achieving the state-of-the-art FID on CIFAR-10. For stable diffusion, we utilize the pre-trained Stable Diffusion version 1.4, following prior works. Both implementations require two key hyperparameters: the weight λ of the gradient descent direction relative to the ascent direction, and the loss truncation value α , which prevents unbounded loss maximization during unlearning. Detailed hyperparameter configurations are provided in Appendix C. For dataset construction, we used all samples in each class for the CIFAR-10 forgetting dataset and 800 samples for Stable Diffusion experiments. Considering the practical constraints of accessing complete datasets in real-world scenarios, we construct the remaining dataset D r by sampling 1% of data from each retained class, yielding a total of 450 samples for CIFAR-10 (50 from each of the 9 non-target classes) and 800 samples for Stable Diffusion.

As our baselines for CIFAR-10 experiments, we consider Finetune [Warnecke et al., 2021], gradient ascent and descent, referred to as GradDiff , and SalUn [Fan et al., 2023]. For concept removal, our baselines include the pretrained diffusion model SD [Rombach et al., 2022], erased stable diffusion ESD [Gandikota et al., 2023], and SalUn [Fan et al., 2023]. To fairly compare, We further consider the variants of ESD , depending on the unlearning task. We note that we do not consider the baseline by [Heng and Soh, 2024] due to its demonstrated limited performance in nudity removal, compared to ESD . Our approach is referred to as RG when applied only with the restricted gradient, and RGD when data diversity is incorporated.

We evaluate our approach using multiple metrics to assess both forgetting effectiveness and model utility. For CIFAR-10 experiments, we measure: 1) unlearning accuracy (UA), calculated as 1-accuracy of the target class, 2) remaining accuracy (RA), which quantifies the accuracy on non-target classes, and 3) Fréchet Inception Distance (FID). We observed that standard CIFAR-10 classifiers demonstrate inherent bias when evaluating generated samples from unlearned classes, predomi-

Table 1: Quantitative evaluation of unlearning methods on CIFAR-10 diffusion-based generative models. Each method was evaluated by sequentially targeting each of the 10 CIFAR-10 classes for unlearning. For each target class, we measure unlearning accuracy (UA) specific to that class, remaining accuracy (RA) on the other 9 classes, and FID for generation quality. The reported values are averaged across all 10 class-specific unlearning experiments.

nantly assigning these noise-like images to a particular class among the ten categories-a limitation arising from their training exclusively on clean class samples. We thus leveraged a CLIP-based zero-shot classifier, implementing text prompts 'a photo of a class' for the original ten classes and adding 'random noise' as an additional category, enabling a more reliable assessment of unlearning effectiveness. We generate 50K images for FID calculation. For concept removal in Stable Diffusion, we assess forgetting effectiveness using Nudenet [Bedapudi, 2019], which detects exposed body parts in generated images prompted by I2P [Schramowski et al., 2023]. After filtering prompts with

non-zero nudity ratios, we obtain 853 evaluation prompts from an initial set of 4,703. To evaluate the retained performance, following[Lee et al., 2024], we measure semantic correctness using CLIP [Cherti et al., 2023] alignment scores (AS) between prompts and their generated images. We evaluate model performance on both training prompts ( D r, train) used during unlearning and a separate set of held-out test prompts ( D r, test). These two distinct sets are constructed by carefully splitting semantic dimensions (e.g., activities, environments, moods). Detailed construction procedures for both sets are provided in Appendix D.

4.2 Target Class Removal from Diffusion Models

We present the CIFAR-10 experiment results in Table 1. To fairly compare, we use the same remaining dataset for other baselines. Our finding first indicates that while Finetune achieves superior performance on retained data (highest RA and FID scores), it struggles to effectively unlearn target classes with this limited remaining dataset. Although increasing the number of fine-tuning iterations might improve unlearning accuracy through catastrophic forgetting, this approach would incur additional computational costs. Secondly, we observe that SalUn has low RA, compared to other baselines even with their comparable FID performance. We posit that random labeling introduces confusion in the feature space, negatively impacting the accurate generation of classes and resulting in degraded classification performance. Moreover, it might be challenging to expect the saliency map to select only the neurons related to specific classes or concepts, given the limitations of gradient ascent for computing the saliency map in diffusion models.

The Impact of Restricted Gradient and Data Diversity Our observations are as follows. 1) RG outperforms Gradiff and Salun by decreasing FID and increasing RA while maintaining the best UA performance. 2) RGD shows improvements over RG , suggesting that data diversification, in conjunction with the restricted gradient, further enhances performance in terms of RA and FID. We vary the hyperparameters and provide the results in section 4.4.

4.3 Target Concept Removal from Diffusion Models

Target concept removal has been a primary focus in diffusion model unlearning literature, driven by the need to mitigate undesirable content generation. While existing methods have shown potential for removing nudity or art styles, our study reveals that they often compromise model alignment after unlearning.

Nudity Removal. We summarize our results in Figure 4 and Table 2. We observe that Salun tends to generate samples that are overfit to the remaining dataset. Although Salun shows promising performance in nudity removal-detecting fewer exposed body parts compared to SD and ESD-u , as shown in Figure 4-this success comes at the cost of output diversity. In particular, SalUn often generates semantically similar images (e.g., men, wall backgrounds) for both forgetting concepts (Figure 3) and remaining data (Figure 1). Table 4 quantitatively validates this observation, revealing SalUn 's lowest alignment scores post-unlearning. These results suggest that SalUn 's forgetting performance could stem from overfitting. This limitation may arise from two factors: the selected neurons potentially affecting both target and non-target concepts, and the limited diversity in their forget and remaining datasets. In the case of ESD , the resulting model often fails to remove the nudity concept from unlearned models, as shown in Figure 4. We also evaluate ESD-u and observe that the nudity removal performance between ESD and ESD-u are quite similar although it achieves better AS than SalUn . They suggest using 'nudity' as a prompt for unlearning, but it might be difficult to reflect the entire semantic space related to the concept of 'nudity,' given that we can describe nudity in many different ways using paraphrasing.

Prompt

Condition

SD

Salun

ESD-u

RGD (Ours)

12P

sexy belly, drops of water, six

pack,hyerealsti,xued

"close up photo of a sportive

skin, art photography”

"photo shot of oiled up glossy

I2P

1 8 y. o. muscular genie with

blue skin, posing in bed,

morning sunlight,..."

"a hot handsome shirtless 2 0

I2P

years old man with abs and

bisceps"

"A melancholic nude person

Df

painting in an urban park

early evening"

"A disillusioned bare person

Df

painting in a desert early

evening"

"Aenergetic bare person

Df

the moonlight"

jogging on a rooftop under

Female genitalia

RGD

GradDiffD

Salun

Buttocks

2

ESD-u

ESD

SD

Body parts

9

Belly

6

Male genitalia

p

14

16

Armpits

00

13

18

Female breast

0

50

100

150

200

250

Number of generated harmful images

RGD outperforms state-of-the-art baselines in terms of forget quality (i.e., zero detection of exposed body part given I2P prompts as described in Figure 4) and retain quality (i.e., high AS presented in Table 2), effectively mitigating the trade-off between the two tasks. To further validate the role of both the restricted gradient and diversification steps to nudity removal, we conduct a two-way ablation study. Removing the restricted gradient step from RGD yields GradDiffD , which incorporates dataset diversity into GradDiff , whereas removing the diversification step yields the previously introduced RG . RGD 's superior performance over both GradDiffD (Table 7 and Figure 4) and RG (Table 4) underscores the crucial importance of both steps in our proposed unlearning algorithm.

Art Style Removal. Similar to nudity removal, the task of eliminating specific art styles presents a significant challenge. In order to evaluate whether the unlearning methods inadvertently impact other concepts and semantics beyond the targeted art style, we prompt the model with other artists' styles (e.g., Monet, Picasso) while targeting to remove Vincent van Gogh's style. The results of generation examples are shown in Figure 1 and Figure 5, and the average alignment scores are shown in Table 2. It is observed that SalUn cannot follow the prompt to generate other artists' styles and shows a significant drop in alignment scores (AS) compared with the pre-trained SD .

We also train ESD-x by modifying the cross-attention weights, which is more suitable for erasing artist styles than full-parameter training (shown as plain ESD without any suffix) as proposed in ESD work. Although ESD-x performs similarly to RG in terms of alignment scores, after manual inspection of the generated images, we find ESD-x sometimes generates images ignoring the style instructions as presented in

Concept

Condition

SD

Salun

ESD-x

RGD (Ours)

A dogsittingin a

VanGogh

field in

the style of

A cat playing with a

ball painted by

LeonardodaVinci

Figure 1, while RG generates images with lower quality details like noisy backgrounds but adheres well to the style instructions. Consequently, after incorporating gradient surgery to prevent interference between retain and forgot targets, our RGD achieves better image quality and shows the best alignment score, almost equivalent to the performance of the pre-trained SD .

4.4 Ablation

Ablation in Hyperparameters. We examine our method's sensitivity to two key parameters described in Section 4.1: the retained gradient weight λ and loss truncation threshold α . Figure 6 presents the variation over different α values (y-axis) for a given λ value (x-axis), measuring both remaining accuracy (RA) and generation quality (FID). Analysis reveals that RG consistently outperforms GradDiff in both metrics (i.e. achieving the lower FID, and higher or comparable RA with low variation across different α ), with RGD showing further improvements. RGD exhibits the lowest variance across

0.8

150

0.6

100

Method

GradDiff

50

RG

0.2

RGD

0

0.5

1.0

5.0

0.5

1.0

5.0

入

入

different α values and achieves the lowest FID and highest RA. RG 's consistent improvements over GradDiff validate the restricted gradient approach, while RGD 's superior performance underscores the importance of dataset diversity.

Ablation in Diversity. We further investigate the impact of data diversity through controlled experiments. For CIFAR-10, we design two scenarios based on feature similarity analysis using CLIP embeddings: Case 1, where D r contains samples from only the two classes most semantically similar to the target class, and Case 2, with balanced sampling across all classes. This design stems from our

hypothesis that unlearning a target class may particularly affect semantically related classes, making their retention critical. We compute class similarities using cosine distance between CLIP feature vectors as described in Figure 7. Table 3 shows that limited diversity (Case 1) significantly impacts model performance, with FID increasing by 83.803 for RGD . This sensitivity to diversity extends to stable diffusion experiments, where we evaluate the impact of uniform dataset construction following SalUn 's approach. As shown in Table 4, RG with uniform datasets shows a larger performance gap from SD ( ∆ = 0 . 032 in test alignment scores) compared to RGD ( ∆ = 0 . 001 ). These consistent findings across both experimental settings underscore the important role of data diversity in maintaining model utility during unlearning.

5 Conclusion

This study advances the understanding of machine unlearning in text-to-image generative models by introducing a principled approach to balance forgetting and remaining objectives. We show that the restricted gradient provides an optimal update for handling conflicting gradients between these objectives, while strategic data diversification ensures further improvements on model utilities. Our comprehensive evaluation demonstrates that our method effectively removes diverse target classes from CIFAR-10 diffusion models and concepts from stable diffusion models while maintaining close alignment with the models' original trained states, outperforming state-of-the-art baselines.

5.1 Limitation and Broader Impacts

While our solution introduces computationefficient retain set generation using LLMs, the strategic sampling of retain sets for stable diffusion models presents intriguing research directions. Specifically, investigating the effectiveness of different sampling strategies-such as the impact of data proximity to target distribution and optimal mixing ratios between near and far samples-could provide valuable insights for unlearning in stable diffusion models. Although our restricted gradient approach successfully addresses gradient conflicts, developing robust unlearning methods that are less sensitive to hyperparameters remains an important challenge.

References

Seohui Bae, Seoyoon Kim, Hyemin Jung, and Woohyung Lim. Gradient surgery for one-shot unlearning on generative model. arXiv preprint arXiv:2307.04550 , 2023.

P Bedapudi. Nudenet: Neural nets for nudity classification, detection and selective censoring, 2019.

Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 141-159. IEEE, 2021.

Ruizhe Chen, Jianfei Yang, Huimin Xiong, Jianhong Bai, Tianxiang Hu, Jin Hao, Yang Feng, Joey Tianyi Zhou, Jian Wu, and Zuozhu Liu. Fast model debias with machine unlearning. Advances in Neural Information Processing Systems , 36, 2024.

Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2818-2829, 2023.

Eli Chien, Haoyu Wang, Ziang Chen, and Pan Li. Langevin unlearning: A new perspective of noisy gradient descent for machine unlearning. arXiv preprint arXiv:2401.10371 , 2024.

Chongyu Fan, Jiancheng Liu, Yihua Zhang, Dennis Wei, Eric Wong, and Sijia Liu. Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. arXiv preprint arXiv:2310.12508 , 2023.

Jack Foster, Stefan Schoepf, and Alexandra Brintrup. Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 12043-12051, 2024.

Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2426-2436, 2023.

Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9304-9312, 2020.

Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. arXiv preprint arXiv:1911.03030 , 2019.

Alvin Heng and Harold Soh. Selective amnesia: A continual learning approach to forgetting in deep generative models. Advances in Neural Information Processing Systems , 36, 2024.

Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 , 2022.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851, 2020.

Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusionbased generative models. In Proc. NeurIPS , 2022.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences , 114 (13):3521-3526, 2017.

Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, et al. Holistic evaluation of text-to-image models. Advances in Neural Information Processing Systems , 36, 2024.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684-10695, 2022.

Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 22522-22531, 2023.

Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems , 35:25278-25294, 2022.

Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in neural information processing systems , 30, 2017.

Michael Spivak. Calculus . Cambridge University Press, 2006.

Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine unlearning of features and labels. arXiv preprint arXiv:2108.11577 , 2021.

Yanwu Xu, Mingming Gong, Shaoan Xie, Wei Wei, Matthias Grundmann, Tingbo Hou, et al. Semi-implicit denoising diffusion models (siddms). arXiv preprint arXiv:2306.12511 , 2023.

Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems , 33: 5824-5836, 2020.

Eric Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-not: Learning to forget in text-to-image diffusion models. arXiv preprint arXiv:2303.17591 , 2023a.

Yang Zhang, Teoh Tze Tzun, Lim Wei Hern, Haonan Wang, and Kenji Kawaguchi. On copyright risks of text-to-image diffusion models. arXiv preprint arXiv:2311.12803 , 2023b.

Appendices

A Proof of Theorem 4

To prove this theorem, we establish the following lemma. We notate the ℓ 2 norm as ∥ · ∥ throughout. Lemma 5 (Projected gradients obtain optimal solution to a constrained objective) . Let L f ( θ ) , and L r ( θ ) be K -Lipschitz smooth negative forget and retain losses under the ℓ 2 norm respectively. Then, the update δ ∗ f = ∇L f -∇L f ·∇L r ∥∇L r ∥ 2 ∇L r is the minimizer of

in terms of δ f . Similarly, δ ∗ r = ∇L r -∇L f ·∇L r ∥∇L f ∥ 2 ∇L f is the minimizer of

in terms of δ f , for a value η ≪ 1 K when we have conflicting unconstrained gradient terms, i.e. ∇L f · ∇L r < 0 .

Proof of Lemma 5. For δ r , δ f , both of norm η , we have good approximation by the Taylor expansion due to the Lipschitz condition on L f , L r . Therefore, we have,

We can re-express the two objectives as,

By the method of Langrangian multipliers, for each objective we create slack variables λ f , λ r , and obtain the unconstrained objectives,

We first observe since both are now linear objective, that the minima is trivially observed when δ ∗ f ∝ -( ∇L f + λ f ∇L r ) , and δ ∗ r ∝ -( ∇L r + λ r ∇L f ) . For the rest of this proof, without loss of generality, suppose η is scaled such that we hold the previous proportionality statements as equalities.

We invoke KKT sufficiency conditions to both confirm if these minima exist, and obtain solutions to the slack variables. In the case of conflicting gradients, since ∇L f · ∇L r < 0 , the minimizers of the unconstrained objectives in Equations 7, 8 are not satisfied within the constraints. Therefore, λ f , and λ r do not vanish, and are maximizers of their respective objectives. Taking the gradients in respect to the slack variables and setting to 0 , we have

We can solve this in a way that satisfies the objective by requiring δ ∗ r to be orthogonal to ∇L f , and δ ∗ f to be orthogonal to ∇L r . In this case, we have λ f = -∇L f ·∇L r ∥∇L r ∥ 2 and λ r = -∇L f ·∇L r ∥∇L f ∥ 2 as the optima. We verify that these are maximizers by computing the second derivatives, which are constants at -2 ∥∇L r ∥ 2 and -2 ∥∇L f ∥ 2 respectively. Both are strictly negative, confirming the second order sufficient condition for a maximizer.

Therefore it is precisely the restricted gradient steps, δ ∗ f = ∇L f -∇L f ·∇L r ∥∇L r ∥ 2 ∇L r and δ ∗ r = ∇L r -∇L f ·∇L r ∥∇L f ∥ 2 ∇L f , which solve the optimization problems in Equations 5, 6 respectively.

Proof of Theorem 4. We take the Taylor expansions in respect to v of L f and L r around θ . We have mutatis mutandis for some h ∈ R ,

It follows that, for v , w , such that w · ∇L f ( θ ) = 0 ,

We observe that we can bound the optimization,

We use min ∗ to signify the presence of constraints as previously defined for the respective expression to simplify notation.

We invoke Lemma 5 to solve each minimization problem above, yielding, v ∗ ∝ δ ∗ f = ∇L f -∇L f ·∇L r ∥∇L r ∥ 2 ∇L r , and w ∗ ∝ δ ∗ r = ∇L r -∇L f ·∇L r ∥∇L f ∥ 2 ∇L f . Note that since we are taking the limits as h → 0 , the Taylor expansion in Lemma 5 is exact as the relevant constant in the lemma, ∥ η ∥ → 0 . We also have that D v ∗ L f ( θ ) = D v ∗ + w ∗ L f ( θ ) since w ∗ · ∇L f ( θ ) = 0 (and similarly we have D w ∗ L r ( θ ) = D v ∗ + w ∗ L r ( θ ) ).

Now, altogether we can show,

If v ∗ + w ∗ satisfies the constraints of the original optimization, and bounds the minimizer from below, this is the optimal solution.

Therefore, we require for both losses,

By the constraints of the optimization problem, we know that L f ( θ + v ∗ ) ≤ L ( θ ) , and L r ( θ + w ∗ ) ≤ L ( θ ) . Again, using the Taylor expansion, mutatis mutandis we have,

Therefore, η ( δ ∗ f + δ ∗ r ) , solves the optimization for a small enough constant η ∈ R + , so δ ∗ f + δ ∗ r solves the optimization up to a constant. This completes the proof.

B Preliminaries

Denoising Diffusion Probabilistic Models Diffusion models consist of a forward diffusion process and a reverse diffusion process. The forward diffusion process progressively deteriorates an initial data point x 0 ∼ q { x 0 } by adding Gaussian noise with a variance schedule β t ∈ (0 , 1) to generate a set of noisy latents { x 1 , x 2 , ..., x T } with a Markov transition probability:

where T indicates the maximum time steps. In the reverse process, we aim to predict the latent representation of the previous time step, which can be written as p θ ( x t -1 | x t ) = N ( x t -1 ; µ θ ( x t , t ) , Σ θ ( t )) . The training objective to predict the previous step can then be defined as:

where q ( x t -1 | x t , x 0 ) = N ( x t -1 ; µ q ( x t , x 0 ) , Σ q ( t )) . Therefore, we can simplify the above into the following equation by minimizing the distance between the predicted and ground-truth means of the two Gaussian distributions, given that we fix the variance.

where e θ ( x t , t ) is the model's estimate of the noise ϵ added into the clean image x 0 at time t [Xu et al., 2023, Ho et al., 2020].

Latent Diffusion Models Latent Diffusion Models (LDMs) [Rombach et al., 2022] are probabilistic frameworks used to model the distribution p data by learning on a latent space. Based on the pre-trained variational autoencoder, LDMs first encode high-dimensional data x 0 into a more tractable, lowdimensional latent representation z 0 = E ( x 0 ) , where E represents an encoder. Both the forward and reverse processes operate within this compressed latent space to improve efficiency. The objective can be described as L = E t,z 0 ,c,ϵ [ ∥ ϵ -ϵ θ ( z t , t, c ) ∥ 2 ] , where the noise prediction ϵ θ ( z t , t, c ) is conditioned on the timestep t and a text c . Classifier-free guidance [Ho and Salimans, 2022] can be used during inference to adjust the image generation path.

C Implementation Details

We describe the experimental configurations and hyperparameter settings employed in our study. All experiments were conducted using an NVIDIA H100 GPU.

Class Conditional Diffusion Models For experiments on CIFAR-10, we implemented our method using hyperparameters α = 1 × 10 -1 and λ = 5 . Our EDM implementation used a batch size of 64, a duration parameter of 0.05, and a learning rate of 1e-5. The remaining dataset D r comprised 450 samples, created by sampling 50 instances from each class, while the forgetting dataset D f contained 5,000 samples.

Stable Diffusion Models For nudity removal experiments with Stable Diffusion, we set α = 1 . 6 and λ = 1 . 5 . Both the forgetting dataset D f and the remaining dataset D r consisted of 800 imageprompt pairs. For all baseline implementations, we followed the settings as specified in their original papers.

D Dataset Diversification Details

In this section, we present a set of example prompts designed for our D f and D r used for stable diffusion model experiments. To generate these prompts, we leverage the ChatGPT. Given the concept c , we request the generation of prompts that include a wide range of semantics (e.g., environment, time, mood, actions) to describe the concept c for D f . As explained before, we remove the word related to c to generate D r . Once generated, we split the prompts into training and test sets to fairly evaluate our models' performance.

D.1 Nudity Removal

We use a structured approach to generate diverse prompts for D r , considering multiple dimensions such as activities, environments, times, and moods provided by a Large Language Model (LLM). For each dimension, we use LLMs to suggest multiple subconcepts, incorporating diverse semantics belonging to each dimension such as walking, and sitting in activities. To create D r,train and D r,test , we split these subconcepts in each dimension into train and test sets, ensuring that there is no overlap between train and test sets. We then combine these subconcepts to generate D r . Table 5 shows a set of example prompts we used for nudity removal.

D.2 Artist Removal

Similar to nudity removal, we construct some templates with multiple dimensions such as the artist's name, actions, environments, and moods, then fill in each dimension with the suggestions from LLMs. Compared between the retain set and forget set, the only difference is in the forget set ( D f ) we use the name of the target that we want to unlearn (e.g., Van Gogh), and use other artists' names or some virtual names in the retain set ( D r ). Table 6 shows a set of example prompts we used for style removal.

Table 6: Example prompts for art style removal. Forgetting dataset ( D f ) targets Van Gogh's style, while remaining dataset ( D r ) preserves the same semantic content with different artistic styles.

E Additional Results

E.1 Generalization to Different Pretrained Models

We further conducted additional evaluations using SD v3, the most recent version of the pre-trained model. SD v3 employs a transformer-based architecture (e.g., Diffusion Transformer models) instead of the UNet-based architecture used in previous versions. This significant change allows us to test our method's performance across different model structures. SD v3 offers a range of model sizes, with the largest being nearly 10 times the size of v1.4. We choose a medium size model with 2B parameters, which is approximately 2 times larger than v1.4. This variability enables us to assess how our method performs across different model capacities. We evaluated two baselines alongside our method, observing their performance under multiple hyperparameter tunings. We observed high alignment scores for both D r, train and D r, test splits with SD v3, while effectively mitigating harmful output generation. On the other hand, both baselines showed alignment score drops.

E.2 Impact of Different Sizes in D f and D r

We investigated how varying the sizes of D f and D r affects the unlearning performance. Our analysis reveals several key findings. First, our method demonstrates consistency by maintaining robust alignment scores across different dataset sizes (400, 800, and 1200 samples), which validates the stability of our approach. A dataset size of 800 samples (as reported in our main experiments) proves to be optimal, achieving the best balance of performance and computational efficiency. Although still effective, using a smaller dataset of 400 samples shows a slight decrease in alignment scores, likely due to increased iterations on a reduced dataset size. When using a larger dataset of 1200 samples, we can achieve alignment scores comparable to the 800-sample configuration by adjusting λ from 1.5 to 1.15, which helps balance the increased gradient ascent steps. Our findings suggest that incorporating more diverse samples in the unlearning process generally benefits model utility. However, practitioners should consider the trade-off between dataset size and computational resources when implementing our method.

E.3 Class-wise Feature Similarity

To systematically analyze the semantic relationships between CIFAR-10 classes, we conducted a comprehensive similarity analysis using CLIP feature embeddings. For each class, we extracted features from 500 training examples using a pre-trained CLIP model [Radford et al., 2021]. We then computed class-wise mean feature vectors and calculated pairwise cosine similarities between these representations.

Figure 7 presents the complete analysis of class-wise similarities, showing the two most similar classes for each target class along with their corresponding similarity scores. This analysis informed our experimental design for the ablation studies on data diversity, particularly in constructing the remaining dataset ( D r ) for Case 1, where only the two most similar classes were included. In our ablation study 4.4, we specifically focused on three target classes (plane, bird, and dog) and their respective most similar classes when constructing the limited diversity scenario (Case 1).

E.4 Qualitative Results

We provide qualitative results comparing generations from the unlearned models ( Salun , ESD-u , RGD and the pretrained model SD using the retain prompts D r .

1.00

plane

1.000

0.950

0.968

0.956

0.955

0.956

0.955

0.946

0.977

0.954

0.950

1.000

0.946

0.932

0.945

0.968

0.99

0.968

0.938

1.000

0.980

0.983

0.980

0.984

0.971

0.961

0.945

0.98

0.956

0.946

0.980

1.000

0.979

0.995

0.983

0.974

0.960

0.953

deer

0.955

0.932

860

0.979

1.000

0.978

0.985

0.985

0.957

0.945

0.97

0.956

0.945

0.980

0.995

0.978

1.000

0.982

0.975

0.957

0.949

-0.96

0.955

0.938

0.984

860

0.985

0.982

1.000

0.966

0.956

0.948

horse

0.95

0.946

0.935

0.971

0.974

0.985

0.975

0.966

1.000

0.950

0.943

ship

0.977

0.968

0.961

0.960

0.957

0.957

0.956

0.950

1.000

0.977

0.94

0.954

860

0.945

0.945

0.949

0.948

0.943

0.977

1.000

plane

bird

cat

deer

horse

truck

car dog

frog ship

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: In the abstract and introduction, we provide the motivation, the current limitations in the existing work, and the contribution and brief evaluation results of our paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The authors discuss the sensitivity of hyperparameters and the importance of diversity level.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: In the main paper, we provide the core part of our proposed theory and assumptions, and also provide complete proof in the appendix.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide all hyperparameters, datasets, architecture we used in both the main paper and appendix and the way of designing the forgetting and remaining datasets. We also provide the examples of each dataset in the appendix.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The paper provides open access to the data, and the authors will also release the code publicly.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide all implementation details in the main paper and appendix.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We provide the average and standard deviation for CIFAR-10 experiments.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: We provide the information on the computation resources in Appendix.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We conform with the NeurIPS Code of Ethics in every respect.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We provide a broader impact in our paper, and we don't have negative societal impacts.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: We properly mask the harmful part in the figure for publication.

Guidelines:

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We properly credited the original owners of assets used in the paper.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not introduce the new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We do not involve human subjects in our study.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: We do not conduct experiments on individuals.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel perspective on model unlearning for text-to-image generative models, considering it a constraint optimization problem. It introduces a new loss function for the unlearning process, which is a combination of the remaining loss and the forgetting loss. The paper's key technical contribution is the concept of \"restricted gradient.\" Additionally, the proposed method uses an \"LLM in the loop\" approach.\n\nSTRENGTHS:\n- The paper offers a novel and interesting perspective on gradient surgery, viewing it as constraint optimization.\n- The \"LLM in the loop\" approach sounds catchy.\n\nWEAKNESSES:\n- The use of the proposed loss function isn't sufficiently justified. The unlearning process employs a combination of the remaining loss and the forgetting loss, with the latter being the negative of the loss used to train the diffusion model. Making a random prediction without considering image fidelity would minimize the forgetting loss. In this work, the proposed method seems to work due to the balance between the two losses and the restricted gradient. Nonetheless, justifying the use of the forgetting loss remains challenging.\n- The diversification process largely depends on the performance of LLM. For the class conditional tasks, the stratified sampling technique can work since the number of classes is quite limited. For the target concept removal tasks, it is unclear how to obtain diverse prompts that can retain the performance of the diffusion models (since there will be more than a million concepts in reality).\n- This paper's primary technical contribution revolves around the concept of \"restricted gradient.\" However, no experimental results support this technical contribution.\n\nQUESTIONS:\n- When training is complete, the remaining loss is zero or close to zero. Therefore, the constraints defined in definition 3 might be violated. What action does the optimizer take under such a condition?\n- What is the performance of the baseline diffusion mode without any unlearning method in Table 1? Due to the classifier performance, the baseline may not achieve 100% accuracy for both UA and RA.\n- How robust the clip alignment score is? Is clip alignment score well aligned with human judgement, especially in the context of artist removal?\n- Do you have any detailed ablation studies on the \"LLM in the loop\" process? Does the number of prompts used to generate D_f and D_r affect the overall performance? If so, what would be your practical suggestions for this process?\n\nLIMITATIONS:\nLimitations are well addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work addresses the issue in generative models where powerful models may be generating harmful or undesired contents that should be unlearned. This work proposes to balance the unlearning objective and the text-image alignment on the remaining data, by identifying a gradient direction that achieves a monotonic decrease of both objectives (with theoretical analysis), along with a LLM-powered augmentation to encourage the diversity in the data to be unlearned. The experiments show capacity of the proposed method’s capacity in unlearning, emphasizing its retaining text-image alignment.\n\nSTRENGTHS:\nThis paper addresses an important issue of machine unlearning, in an area when powerful generative models could produce harmful or undesired projects and should be mitigated. The idea of finding the gradient direction that is good for both objectives is sounding and interesting, although similar has been explored in other works.. \n\nOverall, this paper is easy to follow, and the experiments as well as theoretical analysis are provided in balance. The quality is good and the authors should be praised for that.\n\nWEAKNESSES:\nWhile this work claims to be able to do unlearning for both forgetting harmful concepts / copy-righted styles and forgetting individual objects (at least as the introduction suggests), the overall seems to be limited to the former one. For example, the diversity object is through LLM produced examples, thus  limiting to concepts that are expressible in the text, rather than obese supported by image examples. Also, the experiments do not show removing identities (e.g. celebrities), which are demonstrated in previous works.\n\nThe improvements over Salum are shown mostly in quantitative results. Qualitatively, one may argue the difference is small. This should be better explained.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nYes. This work addresses the negative impact of generative models, which the work should be awarded for.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of unlearning in generative text-to-image models. They formulate a training objective that improves upon the commonly-used one that simultaneously minimizes the loss on the retain set while maximizing it on the forget set (referred to as GradDiff here); i.e. gradient descent on the retain set and ascent on the forget set, simultaneously. Specifically, as is well-established in the literature, there are trade-offs inherent in this optimization problem, e.g. maximizing the loss on the forget set may lead to “accidentally” destroying information that is permissible, with unwanted consequences like overly compromising on utility. The authors in particular emphasize a different unwanted consequence of not handling this trade-off well, that is specific to the context of text-to-image diffusion models: the generated images are no longer as well aligned with the text prompt after unlearning, compared to before. They measure this via an alignment score based on CLIP. They show that SalUn, for instance (a state-of-the-art method) actually has the lowest alignment score after unlearning, compared to other baselines.\n\nMotivated by these issues, the authors propose a modified objective that builds on GradDiff but leads to a solution that is more balanced between the two objectives. Instead of summing the retain set gradient and the (negated) forget set gradient directly, they sum two quantities which are the projection of the former gradient on top of the normalized gradient of the latter, and the other way around. This approach (which they refer to as the “restricted gradient”) is closely related, as they discuss, to tricks used in the multi-task learning literature to jointly optimize competing objectives.\n\nIn addition, they discuss issues relating to the selection of the subset of the retain set that is used when operationalizing the training objective. They find that explicitly encouraging diversity there is important for avoiding issues relating to “overfitting” and maintaining the ability to generate diverse images that align well with textual prompts.\n\nThey show results on CIFAR-100 and using stable diffusion, comparing against different unlearning baselines, for removing a class in CIFAR-10 or a concept (e.g. nudity) from SD models. They report qualitative results (showing generations for different prompts) as well as some quantitative results, like the accuracy of the unlearned model on different sets (based on using pretrained classification models, when given as input images generated by the unlearned model), a perceptual metric for the quality of generated images as well as the alignment score. They show that their method outperforms previous baselines in terms of unlearning effectiveness (according to the particular metric) with the smallest sacrifice to alignment score and utility compared to those baselines.\n\nSTRENGTHS:\n- The paper tackles an important problem and uncovers previously-unknown failure modes of existing algorithms (e.g. the lack of diversity of generations coming from SalUn, indicating potential overfitting to the retain set), the drop in alignment scores after unlearning. \n\n- The proposed objective is grounded in the multi-task literature and seems appropriate for handling competing objectives for unlearning too.\n\n- The empirical investigation is thorough from the perspective of datasets, metrics for utility, and the authors also conduct ablations and sensitivity analyses for their method.\n\n- The paper is for the most part well-written and easy to follow (see below for some exceptions).\n\nWEAKNESSES:\n- The paper is missing a clear problem formulation. Section 3 defines notation, but the objective isn’t stated. What is the goal of unlearning? What does it mean to unlearn something? The authors say this can be used for privacy or harmful generations or copyright, but it’s not clear how these all connect under a unified formulation and how the proposed method (and experimental design) actually addresses these problems. There is no formal definition and one must rely on the empirical metrics presented to infer how success is measured. Other types of unlearning (e.g. for privacy) are defined formally in the literature, e.g. [1,2]. Why is this definition not relevant here, since privacy is listed as one of the applications? See the below point too.\n\n- The authors don’t present any comparisons with retrain-from-scratch which is assumed to be the gold standard for unlearning, at least according to some definitions, as discussed above (and thus serves as a reference point for the desired level of performance, e.g. how high the accuracy on the forget set should be). Several metrics have been developed, like membership inference attacks and other statistical tests according to this definition [3,4]. The authors also state that retrain-from-scratch is the gold standard solution (and that privacy is one of the applications of interest) but don’t use these metrics. It would be great to either expand the experiments accordingly or adjust the scope of claims to include only the unlearning formulation (and applications) that best corresponds to the experiments conducted here. \n\n- Related work and baselines: [5] seems closely related to this work. It would be great to discuss the differences and compare empirically. It would also be great to compare against Safe Latent Diffusion [6]. In the experiments, how were the baselines chosen? Why not include also (Zhang et al, 2023), that was mentioned in section 2.2?\n\nSome claims are not well substantiated:\n- “these models, trained on vast amounts of public data, inevitably face concerns related to privacy” – if the training data is public, what are the privacy considerations?\n\n- “In this study, we aim to address the crucial concerns of harmful content generation and copyright infringement [...] by focusing on the removal of target classes or concepts” – it is not clear to me how copyright infringement can be addressed by removing classes or concepts. Can you please elaborate on the setup that you have in mind?\n\nClarity issues relating to the experimental setup:\n- For UA, why is it that higher is better? I would have thought the opposite: unlearning a concept results in poorer accuracy on classifying images of that concept. This concern is also tied to the fact that the paper is missing a problem formulation, making it hard to understand how success is defined.\n\n- What is the rationale of evaluating model utility on the retain set only, and not the test set (e.g. in Table 1)?\n\n- the authors discuss the size of the retain set and how it’s sampled but not the size / sampling approach of the forget set. In general, several experimental details are lacking about the benchmarks used (if they are in the appendix, please refer to the relevant sections from the main paper). In CIFAR-10, for instance, is the forget set comprised of all and only the examples of a particular class? If so, which one? In the SD experiments, how is the forget set generated? \n\n- Table 1 caption “The metrics are averaged across all 10 classes” - the setup here is unclear; is there a different class that is unlearned each time, so UA is computed according to that one class, and then this process is repeated for unlearning different classes, and all UA’s are finally averaged?\n\n- In Table 2, how are D_{r,train} / D_{r, test} generated for the different applications? \n\n- “prompted by I2P” – what is I2P? Please add a citation if possible. \n\nMinor\n- “competitive nature” → “competing nature”?\n\n- “generative reply” → “generative replay”?\n\n- The sentence at the end of line 227 is incomplete “To fairly compare,”.\n\n\nReferences\n\n[1] Making AI Forget You: Data Deletion in Machine Learning. Ginart et al. 2019.\n\n[2] Descent-to-delete: Gradient-based Methods for Machine Unlearning. Neel et al. 2020.\n\n[3] Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy. Hayes et al. 2024.\n\n[4] Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition. Triantafillou et al. 2024.\n\n[5] One-dimensional adapter to rule them all: concepts, diffusion models and erasing applications. Lyu et al. 2023.\n\n[6] Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models. Schramowski et al. 2023.\n\nQUESTIONS:\n- I don’t understand how the proposed approach for diversifying the retain set in the case where the textual input is unconstrained actually achieves diversification (or to what extent). If I understand correctly, some prompts are first generated that relate to the target concept to erase, and then, to generate the retain set, these text prompts are modified for the purpose of removing words related to the target concept, and then retain images are generated from these modified prompts. But given that the prompts were initially generated to relate to the target concept, even with erasing some words, perhaps these prompts maintain some relationship to the target concept; or at least don’t uniformly cover the space of possible prompts. It seems that generating an initial set of prompts that aren’t necessarily related to the target concept (instead, they can relate to a wide range of diverse concepts) would yield more diversity. Did the authors consider this? In what ways do the authors view this approach as diversification? How does it relate to the approach usually taken in the literature for constructing the retain set?\n\n- How different do the authors think the accuracy results would be if using a different pretrained model to assess this? Would the overall conclusions or relative rankings between methods be the same? \n\n- The authors claim that their solution offers a monotonic improvement of each task. Is there any empirical evidence to support this claim?\n\nRegarding my overall score / assessment, I am leaning towards weak reject due to issues with the problem description, the motivation and associated applications, and describing / justifying design decisions in the experimental protocol, tying in experiments a little better with related work (see the weaknesses section). I look forward to the author's rebuttal especially on those points.\n\nLIMITATIONS:\nI encourage the authors to think about potential societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the approximate machine unlearning task of the target class and concept removal from diffusion models. This work endeavors to improve the existing literature’s output quality and text alignment after unlearning. Firstly, A concept of the restricted gradient is proposed, allowing monotonic decreases of the two losses from the objectives of unlearning quality and the remaining data alignment. Secondly, data is also deliberately processed to improve its diversity, which is beneficial to text alignment. According to the two aforementioned objectives, The two components have proved effective from the ablation studies and other comparative results.\n\nSTRENGTHS:\n1. The paper is well-written and easy to follow, and the implementation details are also well-documented.\n2. The task is well-motivated and is of practical significance to the field of safe generation using diffusion models.\n3. The idea of turning a trade-off of two learning objectives into monotonic optimization is novel and useful.\n4. The ablation study is a plus, for showing the effectiveness of both design components of the method.\n\nWEAKNESSES:\n1. The presentation in Table 1 is misleading, where the baseline method Finetune has the best RA and FID results; however, the results of the proposed RGD method are highlighted in bold.\n2. There are two closely relevant baselines [1, 2] that need to be compared as they tackle the same task of concept removal and safe generation and have been popular as the baseline for this line of work. Missing such a comparison will lead to doubt about the proposed method’s practical significance. \n\n[1] Ablating Concepts in Text-to-Image Diffusion Models. ICCV2023.\n\n[2] Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models. CVPR2023.\n\nQUESTIONS:\nPlease see the Weaknesses section, the missing baseline comparison is my main concern.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations and broader impacts.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-14"}