EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Authors: Daiheng Gao
Shilin Lu
Wenbo Zhou
Jiaming Chu
Jie Zhang
Mengxi Jia
Bang Zhang
Zhaoxin Fan
Weiming Zhang
Venue: ICML 2025 poster (2025)
Keywords: Concept Erasing
Text2Image
Safety
Unlearning
Primary area: social_aspects->safety

Abstract: Removing unwanted concepts from large-scale text-to-image (T2I) diffusion models while maintaining their overall generative quality remains an open challenge. This difficulty is especially pronounced in emerging paradigms, such as Stable Diffusion (SD) v3 and Flux, which incorporate flow matching and transformer-based architectures. These advancements limit the transferability of existing concept-erasure techniques that were originally designed for the previous T2I paradigm (e.g., SD v1.4). In this work, we introduce EraseAnything, the first method specifically developed to address concept erasure within the latest flow-based T2I framework. We formulate concept erasure as a bi-level optimization problem, employing LoRA-based parameter tuning and an attention map regularizer to selectively suppress undesirable activations. Furthermore, we propose a self-contrastive learning strategy to ensure that removing unwanted concepts does not inadvertently harm performance on unrelated ones. Experimental results demonstrate that EraseAnything successfully fills the research gap left by earlier methods in this new T2I paradigm, achieving state-of-the-art performance across a wide range of concept erasure tasks.

Spiderman was dressed in sušt Erasennything: Enabling Concept Erasure in Rectified Flow Transformers
Daiheng Gao
Shilin Lu²
Wenbo Zhou' Jiaming Chu Jie Zhang Mengxi Jia
Bang Zhang"
Zhaoxin Fan? Weiming Zhang'
Abstract
ESD
UCE
EAP
Removing unwanted concepts from large-scale text-to-image (T2I) diffusion models while main- taining their overall generative quality remains an open challenge. This difficulty is especially pronounced in emerging paradigms, such as Sta- ble Diffusion (SD) and Flux, which incor- porate flow matching and transformer-base -based ar- chitectures. These advancements limit the trans- ferability of existing concept-crasure techniques that were originally designed for the previous T2I paradigm (e.g., SD v1.4). In this work, we introduce Frasennything, the first method specifically developed to address concept erasure within the latest flow-based T2I framework. We formulate concept erasure bi-level optimiza- tion problem, employing LoRA-basee Abbased parameter tuning and an attention map regularizer to selec- tively suppress undesirable activations. Further- more, we propose a coff-contrastive learning strat- egy to ensure that removing unwanted concepts does not inadvertenlly harm performance on unre- lated ones. Experimental results demonstrate that Erasennyyhhigg successfully fills the research gap left by earlier methods in this new T2I paradigm, achieving SOTA performance across wide range of concept erasure tasks.
Previoux
Erase Anything
a mude girl with beautiful hair
Van Gogh, Lavender field
Figure I. Top: Comparison between our proposed EraseAnything and classical concept-erasing methods, such as ESD (Gandikota et al., 2023), UCE (Gandikota et al., 2024), and EAP (Bui et al., 2024). Taking 'nudity erasure' on Flux models as an example. Bottom: Examples of differen concept erasure via EraseAnything. (blue bars indicate author-added sensory harmony, while ycllow bbox showcases the corresponding original output.)
major facelift, with leaps in prompt following, image qual- ity, and output diversity. However, these models trained on increasingly large and diverse datasets sourced from online content, they also face growing safety risks. One major concern is their potential to generate NSFW (Not Suitable For Work) material when provided with in- appropriate prompts. This issue has been widely reported in the media and falls under the broader challenge of con- cept erasing (CE), which involves preventing models from generating harmful or undesirable content.
1. Introduction
From the advent of DALL-E 2 (Ramesh et al., 2022) and Stable Diffusion (SD) (Rombach et al., 2022) to the beefed- up Flux, Recraft and Photon, diffusion models (DMs) have consistently showcased their mastery in the domain of text- to-image (T2I). Over the past few years, T2I has seen a
While CE has been extensively studied in the context of Stable Diffusion (SD), which relies on a DDPM/DDIM (Ho et al., 2020; Song et al., 2020) + U-Net (Ronneberger et al., 2015) framework, the Flux series introduces new set of challenges due to its modern architecture. Flux incorpo- rates advanced techniques such flow matching (Lipman et al., 2022; Liu et al., 2022) and transformer- compo- nents (Vaswani, 2017), which differ significantly from SD's design. As discussed in Sec. 3, these architectural differ- ences, such as the use of the T5 Encoder (Raffel et al., 2020) and rotary positional encoding (RoPE) (Su et al., 2024).
'USTC 'NTU 'BPPT 'IIPC and CFAR, **STAR 'TeleAI Tongyi Lab, Alibaba 'Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing. Beihang University. Correspondence to: Zhaoxin Fan Weiming Zhang <zhaox- inf bur.buaa.du..cn & zhangwm www.e.uuusss..........
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
have led to inconsistencies between SD and Flux. These discrepancies have, in turn, created a range of new problems that need to be addressed.
suppresses the undesired concept but also maintains a clear distinction between the unlearned concept and unrelated ones. This two-step process forms the core of our robust concept erasing method for Flux.
We demonstrate that existing concept erasing methods strug- gle to perform effectively within the Flux framework. The first row of Fig. showcases the generation capabilities of the Flux [dev] model after attempting to erase (unlearn) the "nudity" concept. We evaluate several well-established CE methods, including the pioneering work in concept eras- ing, ESD (Gandikota et al., 2023), the closed-form solution UCE (Gandikota et al., 2024), and the ad adversarial-traiin based approach EAP (Bui et al., 2024). These methods, while diverse and widely recognized in the field, exhibit limited generalizability when applied to Flux, highlighting significant gap in transferring CE techniques from SD to Flux. This raises critical research question that our paper seeks to address:
To the best of our knowledge, we are the first to study concept erasing in Flux systematically and propose an ef- fective method, termed EraseAnything, which balances the model's ability to delete the target concept while retaining its original capabilities. To achieve this, we have developed comprehensive approach involving several key steps:
Attention Localization: Through an in-depth analysis of Flux. we discovered that its attention maps allow for precise identification of specific content using token indices. This capability enables the selective erasure of localized content.
Reverse Self- Seff-Contrastive Loss: Leveraging off-the- shelf LLMs (Achiam et al., 2023), we dynamically generate Dür (irrelevant concepts) based on the given unlearned prompt. This allows us to construct a self- contrastive loss that optimizes the model to ensure the generation quality and effectiveness of concepts not targeted for unlearning remain unaffected.
Q: Can we propose a robust concept erasing method suitable for Flux?
From a macro perspective, Q can be framed as Bi-level Op- timization (BO) problem. Let us define a dataset of concepts to be unlearne, Dun E (mudity,... and a dataset of irrele- vant concepts, Dar E (beautiful, smart, charming,...) Here, irrelevant concepts encompass a broad spec- trum of ideas, ranging from abstract descriptors like (qualified, organized, industrious... descriptors such as reautiful or ugly. During sampling, concepts from both categories are treated equally to ensure a balanced representation. The core objective is to learn adapter weights (e.g., LoRA (Hu et al., 2021) or PEFT (Man- grulkar al., 2022)) that achieve two goals: 1) Reduce acti- vations associated with prompts from Dan (the unlearning concepts). 2) Preserve image generation quality for prompts from Dr. (the irrelevant concepts). This formulation en- sures that the model effectively erases undesirable concepts while maintaining its ability to generate high-quality images for other, unrelated prompts.
Bi-Level Optimization: Recognizing the interdepen- dence between concept erasing (Dun) and irrelevant concept preservation (Dir), employ bi-level opti- mization to achieve stable convergence. The lower level focuses on erasing the target concepts in Dun. while the upper level ensures the preservation of Dir.
to physical
2. Related Work
2.1. T21 Diffusson Models
Recent advancements in text-to-image diffusion models have been remarkable, with notable contributions from GLIDE (Nichol et al., 2021), DALL-E series (Ramesh et al., 2021; 2022) Imagen (Saharia et al., 2022) and SD series (Rombach et al., 2022; Podell et al., 2023; Lu et al., 2023; 2024b), which stands out due to its fully open-sourced model and weights. SD 3 (Esser et al., 2024), the latest installment, introduces paradigm shift with the simplified sampling method (where the for- ward noising process is meticulously crafted as a recti- fied flow (Liu et al., 2022), establishing a direct con- nection between data and noise distributions) and its trio of text encoders (Radford et al., 2021; Raffel al., 2020) 2020)-CIPLL114 OpenCLIPbigG// 11,, T5 XXL- and the CLIPL/14, innovative Multimodal Diffusion Transformer (MMDiT) architecture with over 2B parameters. SD 3 processes texts and pixels as a sequence of embeddings. Positional encod- ings are added to 2x2 patches of the latents which are then flattened into a patch encoding sequence. This sequence, in
From a microscopic perspective, our approach begins by reducing the activations associated with Dun. through fine- tuning LoRA module. This is achieved using the ESD objective function combined with an index-related attention maps regularizer. The regularizer is critical insight de- rived from our careful analysis of the Flux model's internal mechanisms, which we elaborate on in Sec. 3. Next, we fine-tune the same LoRA in the reverse direction. Inspired by (Oord et al., 2018; He al., 2020), we construct novel self-contrastive loss. This involves selecting 1 synonym word (as negative sample) for the key concept in Dun and K(K> 2 3) words from Drr The self-contrastive loss penal- izes the model when the attention maps of the unlearned concept exhibit semantic features that are closer to those of the irrelevant concepts. This ensures that the model not only
2

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
conjunction with the text encoding sequence, is input into the MMDiT blocks. Here, they unified to a common dimensionality, merged, and subjected to a series of modu- lated attention mechanisms and multilayer perceptrons.
2.3. Bi-level optimization (BO)
Bi-level optimization (BO), a mathematical framework with deep-rooted research legacy (Colson al., 2007; Sinha et al., 2017), is characterized by its ability to handle com- plex optimization problems where secondary optimization task (the lower level) is intricately nested within a primary optimization task (the upper level).
Flux, sharing the same visionary authors as SD 3, builds upon this foundation. With its exceptional performance in ELO scoring, prompt adherence, and typography, Flux has emerged as a superior contender. Recognizing these advancements, we have chosen to concentrate our experi- mental efforts on Flux, leveraging its strengths to further our research and development in the concept erasing domain.
The advent of deep learning has sparked a renewed interest in BO, recognizing it versatile and essential tool for tackling a broad spectrum of machine learning challenges: e.g. hyperparameter optimization (Lorraine et al., 2020; Shen et al., 2024), meta learning (Franceschi et al., 201 8), and physics- based machine learning (Hao et al., 2022).
2.2. Concept Erasing
Gigantic yet unfiltered dataset LAION - 5B (Schuhmann et al., 2022) that used to train T2I models, poses the risk of T21 models learning and generating inappropriate con- tent that infringes upon copyright and privacy. To allevi- ate this concern, numerous studies explore and devising solutions, including training datasets filtering (Rombach et al., 2022). post-generation content filtering (Rando et al., 2022), and fine-tuning pretrained models: ANT (Li et al., 2025), MACE (Lu et al., 2024a), SPM (Lyu et al., 2024), adyUnlearn (Zhang et al.. 2024b), Receler (Huang et al., 2023) and classical methods (Kumari et al., 2023; Gandikota et al., 2023; Bui et al., 2024; Gandikota al., 2024). SD 2 uses an NSFW detector to filter out inappropriate content from its training data, which leads to significant training ex- penses and difficult balance to strike between maintaining data purity and achieving optimal model performance. Dif- fusers (von Platen et al., 2022), as a dominant open source libary for DMs, adopts post-hoc safety checker to filter out NSFW content, yet this feature can be easily circumvented by users.
A related example of BO is BLO-SAM (Zhang et al., 2024a), cutting-edge approach that integrates BO into supervised training for semantic segmentation. This technique is par- ticularly adept at preventing models from overfitting, which means it helps models generalize better from training data to new, unseen scenarios.
When it is comes to Flux, with its large number of parame- ters and progressive training paradigm, it's clear that it oper- ates in a different context compared to BLO-SAM, where the model output is more straightforward. To make BO adaptive for DMs, we need tailor the approach to accom- modate its unique characteristics and ensure we can fully utilize its potential. This involves enhancing Flux's 's capabil- ity to eradicate specific target concepts while simultameously preserving its efficiency in generating other concepts, ensur- ing senseless compromise in overall performance.
3. Obstacles in migrating concept erasure methods to Flux
Today, the field has evolved from basic concept erasure (CE) to more nuanced focus preserving irrelevant concepts. EAP (Bui et al., 2024), for instance, selectively identifies and retains adversarial concepts to purge undesirable content from diffusion models with minimal side effects on irrele- vant concepts. Real-Era (Liu et al., 2024) tackles "concept residue" by excavating associated concepts and applying beyond-concept regularization, thereby boosting erasure ef- fectiveness and specificity without sacrificing the generation of irrelevant concepts.
In this section, we explore the reasons why classical era- sure methods from Stable Diffusion (SD) fail when applied to Flux. Specifically, discuss the limitations posed by T5's s sentence-level embeddings, the absence of explicit cross-attention, and the complexities involved in handling keyword obfuscation. Additionally, we outline the computa- tional costs and practical challenges, such as constructing an erasure vocabulary, that make direct adaptation of traditional methods infeasible in Flux.
Erasing method evaluation: When adapting classical era- sure methods to Flux, significant challenge arises: explicit cross cross-attention layers, which are central to methods like ESD (Gandikota et al., 2023), UCE (Gandikota et al., 2024), and MACE (Lu et al., 2024a), do not exist in Flux's architec- ture. Unlike SD, which relies on U-Net with cross-attention mechanisms, Flux employs dual-stream and single-stream blocks that lack such explicit layers (see Appendix A for Flux's 's detailed structure). This architectural difference ne-
In our work, we prioritize the preservation of irrelevant concepts. Departing from the textual embeddings used in previous methods: CLIP, Flux defaults to the T5 text en- coder for textual embedding injection. Therefore, we adopt a heuristic approach to dynamically and automatically select irrelevant concepts by leveraging the powerful capabilities of large language models (LLMs). For a more compre- hensive understanding of T5 and the rationale behind our heuristic method, we elaborate on this on the Section 3.
3

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Table 1. Find the closest synonyms of nude.
METHOD
TOP-3 CLOSEST SYNONYMS
CLAUDE 3.5
"NAKED", "UNDRESSED", "UNCLOTHED" "BARE", "NAKED", "UNCLOTHED", "NAKED", "UNCLOTHED", "BARE" "LEAN", "DEER", "GIRI."
GPT-40
KIMI
kid kicking soccer
Timestep=1
Timestep=8
Timestep=20
T5 FEATURE
Figure 2. Correlations between text and attention maps.
cessitates a fundamental redesign of erasure methods to suit Flux's s transformer-basee framework.
Flux exhibit similar pattern where explicit rross-attentions exist between the given text prompt and intermediate atten- tion maps within the network? As detailed in Appendix A, Flux lacks explicit cross-attention layers. Initially, this presented some challenges. However, through an in-depth examination of the neurons and features within Flux, we ultimately demonstrated (as shown in Fig. 2) that linear relationship between text embeddings and attention maps also exists in Flux.
Moreover, directly transplanting methods from SD's U-Net to Flux's 's transformer architecture leads to phenomenon we term concept residue, where target concepts are incom- pletely removed. This limitation underscores the need for a novel approach to achieve thorough concept erasure in Flux.
Irrelevant prompt preservation: Techniques, such as EAP (Bui et al., 2024) and Real-Era (Liu et al., 2024), have gained popularity for their ability to maintain model perfor- mance on non-targeted concepts. However, adapting these methods to Flux presents a unique challenge. While SD uses CLIP as its text encoder, which excels at word-level embeddings and similarity measurements, Flux relies on T5. T5 is designed for sentence-level embeddings and struggles to capture word-level similarities effectively. This mismatch makes T5 less suitable for implementing irrelevant prompt preservation in Flux, as it cannot reliably distinguish be- tween semantically similar words or concepts.
Specifically, as shown in Eq. (1), the feature correlation Q, K is established by concatenating the textual and pixel embeddings along the last dimension:
Q concat (Qteat: Qpiinet dim -1);
K concat (Keett, K Kpizel, dim = -1), (I)
W Wattn Softmax Q x K).
As shown in Table 1, we extracted the T5 feature embed- dings for the word nude nude and compared them with the entire vocabulary (over 30,000 words) from the T5 default tokenizer. Using cosine similarity, closest synonyms based on semantic embeddings. However, the results were far from rational, indicating that T5's word- level embeddings are unreliable for this task and cannot effectively evaluate semantic similarity.
According to our experiments, we find that the relationship between text and image is inherently forged within the con- fines of W attn. By pinpointing the token index of the target word (want to erase) nestled within the prompt, we are ca- pable of delineating prompt-specific characteristics. This is achieved by nullifying the pertinent column of Wattn. thereby elucidating the underlying features with precision.
identified the top 3
Another critical issue stems from the size of T5 embeddings. With a shape of max_sequence (ength(256)) x 4096, T5 embeddings are approximately 18 times larger than CLIP embeddings, which have a shape of 77 x 768. This substan- tial size difference makes the adaptive selection of adversar- ial prompts from the vocabulary computationally intensive and time-consuming for each iteration. Noted that the goal of selecting adversarial prompts to optimize the model, ensuring it robustly erases the target concept.
So far, Not so good: As shown in Fig. 3, removing a target concept seems straightforward at first: by locating the token index of the keyword in the prompt, we can delete the cor- responding index column in W Wattn E [24, 1280, 1280]. where 1280 = www.seequence_lengtte + head_dim and 24 = attn heads (generating image resolution of 512 x 512). However, our experiments reveal that this technique is ineffective against one of the rudimentary prompt attack strategies: obfuscating keywords- by altering the in- put prompt with nonsensical prefixes suffixes (soccer + soccerrs) or by introducing misspellings (Nike Nikke). In such cases, the erasure of the attention map proves futile, making it easy to cireumvent this method and still success- fully generate the target concept. (For more details, please refer to Appendix B.)
As result, implementing semantic feature- adver- sarial prompt selection in Flux incurs exceptionally high computational costs, posing a significant practical challenge for real-world applications.
Cross attention exploration: Inspired by (Hertz et al., 2022; Xie al., 2023), we formulated a hypothesis: Does

---

ErascAnything: Enabling Concept Erasure in Rectified Flow Transformers
soccer
kid is kicking
Lead E yoo +20 (2 Cun,
(2)
soccerrs
7) llva, (2 Cunit) - v0,
(x (x),
0,)
where represents the negative guidance factor, which sig- nificantly influences the degree of concept erasure. 6, de- note the parameters of the original Flux model and Ao is the learnable LoRA weights for concept erasure. 2; is the denoised latent code timestep f started with random noise at ær (T' is the total timesteps in the denoising pro- cess), va 0, is the unconditional generation initiated with empty input prompt (a.k.a = null text), while Can E Dun identifies the specific concept intended for erasure, for instance, rudity. Additionally, the term v represent the velocity of the Flow matching process, which is the core part of Flux's scheduling mechanism and thus conceptually equivalent with the prediction (Salimans & Ho, 2022) in DMs.
Nike
Sports hoodie with logo
NIKE
Nikke
Figure 3. Attention map erasure can be achieved by setting W Watto [t.,, [,,,,dd..]] idr.] 0, Vi (start,...,end) where start, end can be automatically localized given keyword e.g. "soccer" from input prompt " child is kicking soccer". But this method not generalizable when prompt is slightly modified.
Furthermore, building on the insights gleaned from the cross-attentin explored in Sec. 3, we strive to diminish the model's activations of the erased (unlearned) concepts by attenuating the attention weight allocated keywords within the entire input prompt: ids
4. Method
W nttn [; idæ).
4.1. Overview
Building on our earlier analysis, we identified critical lim- itation: deterministic attention map erasure is susceptible to conventional black-box attacks, making it unsuitable for our requirements. To address this issue, we shifted our fo- cus to based approach. This method is designed to minimize the impact on the generation quality irrele- vant concepts while effectively erasing the target concept, ensuring a more robust and reliable solution.
and
Lattn
Fin
idz==strrt
Initially, we encountered suboptimal results because the fixed index positions of sensitive words, which we aimed to climinate, could lead to overfitting. To counteract this, we scrambled the order of the sentences, thereby making the index positions dynamic. This method reasonable because Flux can produce the similar content with sentence that has been randomly shuffled. For more details, please refer to Appendix B.
We address this delicate balance between removal and preservation through bi-level optimization strategy: the lower level is designed to enhance robust concept erasure, while the upper level ensures the maintenance of irrelevant concepts. This duje-obeective methodology lies at the heart of our Erashnny
UPPER-LEVEL: Irrelevant Concept Preservation
In the upper level, it serves for preserving concepts, which is fairly easy to understand: given the prompt 'a nude girl...,, our objective is to eliminate the word Can 'nude" inside of prompt while ensuring the model can still generate an image of a unrelated concept Cir normally, e-g. girl. To achieve this, we generate 6- 6-10 images Ly from a fixed and random seed (starting point trajectory, same as DMs) that includes the concept to be removed (nude) and irrelevant concepts (girl), then train a LoRA (Low-Rank Adaptation) to induce shifts in the image generation process.
4.2. Bi-Level Finetuning Framework
LOWER-LEVEL: Concept Erasure
In the lower-level optimization phase, we refine the fine- tunable parameters of Flux through LoRA on the unlearned dataset Dun, which is comprised of concepts that we want to make Flux erased or unlearned.
ESD emerges as the relatively superior performer with higher negative guidance (Gandikota et al., 2023). The first sub-loss function employed in the lower-level optimization henceforth is formulated as Eq. (2):
Liora = E [|Io vo+ so
(4)
5

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
where v = 2T Upiz. where IT (0, 1) and "pi is the VAE (Kingma, 2013) encoded latent code of image sampled from 1 and u, (1 t) Upia + txT is the noised Upia at timestep t.
Algorithm BO formulation in Erase Eryshnny
Input: unlearned concept dataset and irrelevant dataset Dun and Dar. learning rates alowr: Cupp total iteration steps M.
for iteration = 1 to M do for Cun sampled from Dun do
Apparently, for a broader range of irrelevant concepts, such as the abstract artistic styles and relationships mentioned earlier, this simple training recipe is insufficient to perserve the broader range of concepts that are not involved in the sentence. Considering the analysis in Sec. 3, explicitly incor- porating a collection of images and corresponding prompt lists for irrelevant concepts is cumbersome, and TS feature is not precise enough to measure word-level similarity.
PREPARATION
Construct a meaningful sentence c involve Cun- Shuffle c to avoid overfitting.
Find tokenized index idFstart : iddrend of Cun
from C.
LOWER LEVEL: Cun ERASURE
! Update LoRA AO with Eq. (2)+Eq. (3) under
To address this, we propose a contrastive learning approach based on the attention map of keywords. This method does not require providing a set of images corresponding to irrel- evant concepts. Instead, it leverages the powerful compre- hension abilities of L.LMs, to heuristically generate Dir that are irrelevant to the targeted concept for erasure.
Olow
UPPER LEVEL: Cir PRESERVING
@ Retrieve Cap. Ceyn W.F. to Cun and replace them into e separately to have pör,syn ① Update LoRA AO with Eq. (4)+Eq. (5) under Cup end for end for
First, we construct a simple AI Agent that build upon on GPT-4o to sample Car E Dir- For efficiency reason, we then use NLTK generating the synonym of the concept that aimed to be erased, i.e. the synonym of "nude" could be "nake". Specifically, we choose K. (default is 3) irrelevant concepts. Moving forward, we fix the sampling starting latent, i.e., xr as a constant value, and then substitute "nude" with "nake", c/,, i = [1,2,3) into c, proceeding with the denoising process independently (For more details about the Cir sampling, please refer the Appendix C).
As depicted in Eq. (5) (detailed derivations are provided in Appendix D), r is the temperature hyperparameter that governs the model's s capacity to differentiate between irrele- vant concepts. high 7 causes the contrastive loss to treat all irrelevant concepts with equal importance, potentially resulting in a lack of focus in the model's ' learning process. Conversely, a low may cause the model to concentrate excessively on especially challenging irrelevant concepts, which could be mistaken for potential synonym sample. Based on empirical testing, we have determined that setting T = 0.07 optimal for our model's performance.
As shown in Fig. 2, choose the attention map at higher timesteps for accurate concept-related activations. Here we get the central concept's attention feature Fun alongside with synonym feature Feyn and irrelevant concept set Fir {Fk, (Fk,,..,,Fh FkK). }.
Li-Le Optimization: As shown in Eq. (5), the last loss term defined in our method is finalized. Integrating the aforementioned two optimization problems, we have bi- level optimization illustrated in Eq. (6). (please check Alg. 1 for more details.)
Drawing inspiration from the works in (Oord et al., 2018: He al., 2020; Huang et al., 2024), we have tailored the contrastive loss function in the opposite direction, a.k.a: Reverse Self Contrastive loss (RSC): our training goal is to align the central feature Fn" with the dynamically shift- ing Fir, while simultaneously pushing them apart from the synonym feature F The strategy here is to deviate from the conventional self-contrastive learning approach, which would typically aim to make 1mn more akin to 1rym thereby enhancing the model's sensitivity to the term slated for removal. By inverting this approach, we aim to steer the network towards gradually discarding the concept of "nude" during learning, effectively obfuscating it within an array of irrelevant concepts.
min Liora (' (A) s.t. A = min Lesd+attn (0:Dum)
(6)
5. Experiments
Here, we conduct a comprehensive evaluation of EraseAny- thing, benchmarking it on various tasks, ranging from con- crete to abstract: e.g., soccer, architecture, car to artistic style, relationships and etc.
5.1. Implementation Details
Σ
pun.pt ppt)
We have opted for the Flux.1 [dev] model with publicly accessible network architecture and model weights, dis- tilled version of Flux.1 [pro] that retains high quality and
exp
Crec log
(5)
Faun
exp
6

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Table Assessment of Nudity Removal: (Ieftt) Quantity of explicit content detected using the NudeNet detector on the I2P benchmark. (Right) Comparison of FID and CLIP on MS-COCO. The performance of the original Flux [dev] is presented for reference.
DETECTED NUDITY (QUANTITY)
MS-COCO 10K
METHOD
COMMON
FEMALE
MALE
TOTAL
FID4
CLIP1
CA (OODEL-BASED) (KUMARI ET AL., 2023)
253
65
26
344
22.66
29.05
CA (NOISE-BASEE (KUMARI ET AL., 2023)
290
72
28
390
23.07
28.73
ESD (GANDIKOTA ET AL., 2023)
329
145
32
506
23.08
28.44
UCE (GANDIKOTA
AL., 2024)
122
39
12
173
30.71
24.56
MACE (LU ET AL.,, 2024A)
173
55
28
256
24.15
29.52
EAP (BUI ET AL., 2024)
287
86
13
386
22.30
29.86
META- UNTTA-NNLA (GAO ET AL., 2024)
355
140
26
521
22.69
29.91
OURS
129
48
22
199
21.75
30.24
FLUX.1 1 [DEV]
406
161
38
605
21.32
30.87
age Prompt (I2P) dataset (Schramowski et al., 2023). For the identification of explicit content within these images, we deploy NudeNet (Bedapudi, 2019), using a detection threshold of 0.6. Furthermore, to evaluate the specificity of our method in regular content, we randomly select 10,000 captions from the MS-COCO captioning dataset (valida- tion) (Lin et al., 2014). Finally, we generate images from these captions and assess the results using both the Fréchet Inception Distance (FID) and CL.IP scores.
Imaging Quality
Irrelevant Preservation
Prompt
Adherence
Erasing Cleanliness
Output Diversity
Table presents our results in comparison with the current state-of-the-arr algorithms. It is evident that method gen- erates the sccondllwwer amount of explicit content when conditioned on 4,703 prompts, only outperformed by the UCE. Yet, it stands out with remarkable FID and CLIP scores, suggesting that our approach exerts a minimal neg- ative influence on the original model's ability to generate regular content. In contrast, the UCE, while leading in ex- plicit content reduction, shows sharp decline in efficacy according to these metrics.
Ours Meta-Unlearning + UCE EAP + CA
MACE ESD
Figure 4. User Study. We have created an interface (see Appendix for details) that shows the users with AIGC contents under various methods that transplanted to Flux. With scoring system where (worst) 5 (best).
strong prompt adherence. Our codebase utilizes widely adopted diffusers (von Platen et al., 2022), a popular choice among developers and researchers for DMs. Unless other- wise specified, our experiments employ the flow-matching Euler sampler with 28 steps and AdamW (Loshchilov et al., 2017) optimizer for 1,000 steps, with learning rate aiow = 0.001, Cup = 0.0005 and an erasing guidance fac- tor = under all conditions.
Miscellaneousness Erasure. In this section, we evaluate our method on 3 conceptual categories: Entity, Abstrac- tion, and Relationship. Here, we choose 10 concept for each category (Please check Appendix for the full list of concepts) and adopt the measuring metries described in Table 3. As shown in Fig. 5 and Fig. 7, our method can effectively remove variety of concepts (including multiple- concepts!) while maintaining minor disturbance compared to CA, which substantiates the claim: EraseAnything is truly an "Erase Anything" solution.
In terms of concept construction, we harness the power of NLTK (Bird et al., 2009) to generate synonym con- cepts, and GPT-4o in the extraction of irrelevant concepts. Our fine-tuning process focuses the text-related parame- ters add_q-pro and add_k_pre (subsets of Q and K) within the dual stream blocks.
Table reveals that our method outperforms the traditional CA in terms of erasure efficacy, the retention of unrelated concepts, and the robustness against synonym substitution. This underscores the ability of our method to not only grasp the targeted concepts for erasure but also to discern those that are semantically adjacent, all while exerting an imper- ceptible negative influence on the model's intrinsic capa- bilities. For a comprehensive evaluation of our model's robustness, kindly refer to Appendix B.
5.2. Results
Nudity Erasure. To assess the effectiveness and versatility of our approach, we begin by applying it to the classical task of nudity erasure. Specifically, we used our concept- erased model to generate images from a comprehensive set of 4,703 prompts extracted from the Inappropriate Im-

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Flux [dev]
CA [Model]]
Ours
Table 3. Evaluation of Erasing the specific category: Entity (e.g. soccer), Abstraction (e.g. artistic style) and Relationship (e.g. kiss) are presented. CLIP classification accuracies are reported for each erased category in three sets: the erased category itself (Acc,, efficacy), the remaining unaffected categories (Ace,r, specificity) and synonyms of the erased class (Aceg, generality). All presented values are denoted in percentage (9))
Reebok
Donald Trump
METHOD
ACC. 4
AcC;r t
ACC₂ +
CA (ENTITY)
14.8
89.2
27.3
Time
Time
CA (ABSTRACTION)
25.2
88.3
29.6
Time
CA (RELATIONSHIP)
22.7
88.6
23.1
OURS (ENTITY)
12.5
91.7
18.6
OURS (ABSTRACTION)
21.1
90.5
24.7
Pablo Picasso
OURS (RELATIONSHIP)
18.4
90.2
19.3
Hand Shake
concept around Cun) and Irrelevant Preservation (prompt with eir can be normally generated), utilized the same concepts categorized under Entity, Abstraction, and Rela- tionship. For each concept, images generated using the same random seed across all methods.
Figure Single-concept erasure. We test our model across three levels of granularity Entity, Abstraction, and Relationship to assess its offectivenesss Furthermore, we have incorporated the versatile CA (Kumari al., 2023) [model] enhance the visual contrast for a clearer comparison.
Our study involved 20 non-artist participants, each provid- ing an average of 200 responses. Fig. 4 shows that our method exhibited a comprehensive performance, achiev- ing outstanding results across all aspects, thus making Aryshnn good all-round player in concept erasure.
Erasing " "'Antthing". To further demonstrate "erase any- thing" solution requires additional supporting evidence. As shown in Fig. 6, we show the effectiveness through diverse and challenging cases, such as erasing color (i.e., red rose, green bag) or object count (i.e., five pencils, three cats).
5.3. Ablation study
Adversarial Attacks. Following the AdvUnlearn (Zhang et al., 2024b), we used NudeNet (Bedapudi, 2019) with a detection threshold of 0.6 to test the Attack Success Rate (ASR) on the Ring-A A-Bell-Nudityy dataset' (comprising 285 Ring-A-Bell revised prompts focused on nudity). Since the prompts in Ling-A-Bell-Nuu are already processed according to the standard procedure, did not reapply the Ring-A-Ba A-Bel method.
To assess our loss design, we conducted an ablation study on the task of celebrity image erasure. We chose subset from the CelebA (Liu et al., 2018), omitting those that Flux [dev] couldn't 't accurately reconstruct. This resulted in dataset of 100 celebrities, split into two groups: 50 for erasure and 50 for retention. Unlike MACE's s massive concept crasure, EraseAnything is trained on individual celebrities.
Different variations and their results (evaluated by averaging metrics) presented in Table 5. Leed itself fall short of the complete erasure of target concept, resulting in not so low ACC,. With the addition of Lattn ACC, has fallen dramatically but the retention of irrelevant concepts was fail w.r. ACC,r- Incorporating the loss term Lrsc. we introduce a approach that may lead to achieving high ACC,, values. By organically combining all these loss terms, we achieve comprehensive model that consistently demonstrates the lowest ACC, and the highest ACC,, compared to previous configurations.
Table 4 shows the results of our tests on ESD, CA, and our proposed method using this dataset (all on Flux [dev]). We also included the attack results from MU-Attack (Zhang et al., 2024c). Step 0 means only attack the very initial velocity of Flux, Step 0.1 means attack the initial three velocity. According our experiments, when attack too much steps, would yield irrelevant image w.r.t to the prompt.
User Study. To gauge the human perception of the effec- tiveness of our method, we conducted a user study with five dimensions, where each focusing on different aspect of erased model. For the first two trials: Erasing Cleanli- ness (prompt with cun and generated images do not contain
Others. Due to the page limits, we put remaining experi- mental details and results in Appendix This includes the visualizations under different configs and objects and more.
8

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Table 4. Performance Metrics of Nudity Detection Methods.
Concept
Methods
Flux[dev]
ESD
CA
EraseAnything
Nudity
Original (Org)
59.65%
7.36%
3.16%
2.46%
MU-Attack (step 0)
64.56%
11.57%
15,44%
8.77%
MU-Attack (steps 0,1,2)
65.96%
14.74%
16.49%
11.93%
Goke: bus, satched sxplosive effect happens thurch
Table Ablation Study Erasing Celebrities, ablate four loss
Explaston
Baku
Explasion
terms used in our experiments.
CONFIG
ACC. +
ACC, 1
Gesd + Cattn
15.3
82.1
Lesd + Liora
20.5
77.9
Lesd + Lrae
16.1
85.6
Lutta Lrac
18.6
81.7
Lutta + Liora + Free
15.8
80.2
FULL
14.9
88.5
Bay
to back with deg
Back Back
Green Bag
Red Rose
Five Pencils
Three Cats
Figure 7, Multi-concept erasure.
Original
Epaselnything
a significant issue highlighted in various reports.
Fairness and Bias of Models: By precisely erasing specific concepts, the model's output can be more finely controlled, avoiding the generation of unfair biased content that may arise due to biases in the training data. This ensures that the model serves different users and application scenarios in a more equitable and neutral manner.
Figure 6. Erase complex concepts.
6. Conclusion
In this paper, propose ErraeAAyythigg Flux-based concept erasing method. Leveraging a bi-level optimiza- tion strategy, we strike a balance between erasing the target concept that bound to be removed while preserving the irrel- evant concepts unaffected, mitigating long-lasting notorious risk of overfitting and catastrophic forgetting. Experiments across diverse tasks strongly demonstrate the effectiveness and versatility of our method.
Societal Consequences
Content Creation and Moderation: With the increasing application of T2I models in content creation, the method presented in this paper provides content creators and moder- ators with effective tool to control the subject matter and style of generated content more accurately. This improves the quality and safety of content creation and also allevi- ates the workload of content moderation, contributing to a healthier and more positive online content ecosystem.
Impact Statement
Ethical Aspects
Potential Risks of Content Generation: The method pro- posed in this paper enables the removal of unwanted con- cepts from large-scale text-to-image (T2I) diffusion models, such inappropriate content. This helps to reduce the risk of generating content that violates copyright, privacy, or is otherwise unsuitable, thereby mitigating potential ethi- cal controversies associated with the use of such models. Specifically, it addresses concerns regarding the generation of NSFW (Not Suitable For Work) material, which has been
Model Interpretability and Trust: Through the research and implementation of concept erasure techniques, better understanding and interpretation of the working principles of T2I models can be achieved. This understanding of how models learn and generate specific content enhances users' trust in the models, facilitating their wider adoption and application various fields such as education, healthcare, and artistic creation, thereby bringing more innovation and value to society.

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Acknowledgment
Franceschi, L., Frasconi, P., Salzo, S., Grazzi, R., and Pontil, M. Bilevel programming for hyperparameter optimiza- tion and meta-learning. In International conference on machine learning, pp. I 3568-1577, PMLR, 2018.
This work was supported in part by the Natural Science Foundation of China under Grant 62121002, U2336206, 62372423, 62441617 and 62102386. It was also supported by the Beijing Natural Science Foundation under Grant No. 4254100, the Fundamental Research Funds for the Central Universities under Grant No. KG16336301, the China Postdoctoral Science Foundation under Grant No. 2024M764093, and by Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing.
Gandikota, R., Materzynska, J., Fiotto-Kaufman Kaufman, J., and Bau, D. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2426-2436 2023.
Gandikota, R., Orgad, H., Belinkov, Y., Materzyúska, J., and Bau, D. Unified concept editing in diffusion models. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5111-5120, 2024.
This research is supported by the National Research Founda- tion, Singapore and Infocomm Media Development Author- ity under its Trust Tech Funding Initiative, the National Re- search Foundation, Singapore under its National Large Lan- guage Models Funding Initiative (AISG Award No: AISG- NMLP-2024-004). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore and Infocomm Media De- velopment Authority.
Gao, H., Pang, T., Du, C., Hu, T., Deng, Z., and Lin, M. Meta-unleamee on diffusion models: Preventing releam- ing unlearned concepts. arXiv preprint arXiv:2410.222 12777, 2024.
Hao, Z., Ying, C., Su, H., Zhu, J., Song, J., and Cheng, Z. Bi-level physics-informed neural networks for pde constrained optimization using broyden's 's hypergradients. arXiv preprint (tt.ix:2299.0077555. 2022.
First author and his PHD advisors are with School of Cyber Science and Technology, University of Science and Technol- ogy of China and Anhui Province Key Laboratory of Digital Security, Hefei 230026, China.
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Mo- mentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729-9738, 2020.
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. I.., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., and Cohen-Or, D. pompt-to-prome im- age editing with cross attention control. arXiv preprint arXiv:2208.0/1 2022.
Bedapudi, P. Nudenet: Neural nets for nudity classification, detection and selective censoring, 2019.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Bird, S., Klein, E., and Loper. E. Natural language process- ing with Python: analyzing text with the natural language toolkit. O' O'Reilly Media, Inc.", 2009.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang. L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2 106.09685, 2021.
Bui, A., Vuong, L., Doan, K., Le, T., Montague, P., Abra- ham, T., and Phung, D. Erasing undesirable concepts in diffusion models with adversarial preservation. arXiv preprint arXiv:2410.15611 2024.
Huang, C.-P., Chang, K.-P., Tsai, C.-T., Lai, Y.-H,, and Wang, Y.-C. F. Receler: Reliable concept crasing of text- to-image diffusion models via lightweight erasers. arXiv preprint arXiv:2311. 2023.
Colson, B., Marcotte, P., and Savard, G. An overview of bilevel optimization. Annals of operations research, 153: 235-256, 2007.
Huang, Z., Wu, T., Jiang, Y., Chan, K. C., and Liu, Z. Re- Version: Diffusion- based relation inversion from images. In SIGGRAPH Asia 2024 Conference Papers, 2024.
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al. Scaling rectified flow transformers for high-resolution image synthesis. In forty-first International Conference on Machine Learning, 2024.
Kingma, D. P. Auto-encoding variational bayes. arXiv preprint arXiv: arXiv: /312.61 14, 2013.
10

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Kumari, N., Zhang, B., Wang, S.-Y., Shechtman, E., Zhang, R., and Zhu, J.-Y. Ablating concepts in text-to-image dif- fusion models. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 22691-22702, 2023.
Lyu, M., Yang, Y., Hong, H., Chen, H., Jin, X., He, Y., Xue, H., Han, J., and Ding, G. One-dimensional adapter to rule them all: Concepts diffussin models and erasing applications. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 7559-7568, 2024.
Li, L., Lu, S., Ren, Y., and Kong, A. W.-K Set you straight: Auto-steering denoising trajectories to sidestep unwanted concepts. arXiv preprini urXiv:2504.
Mangrulkar, S., Gugger, S., Debut, L., Belkada, Y., Paul, S., and Bossan, B. Peft: State-of-the-an parameter- efficient fine-tuning methods. attps://githin com hanm/huggingaaaccc /eep/efftttt ft. 2022.
2025.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ra- manan, D., Dollár, P., and Zitnick, C. I.. Microsoft coco: Common objects in context. In Computer Vision-ECcV 2014: 13th European Conference, Zurich, Switzerland, September 6-, 2014, Proceedings, Part V 13, pp. 740- 755. Springer, 2014.
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:21 2021.
Lipman, Y., Chen, R. T., Ben-Hamu H., Nickel, M., and Le, M. Flow matching for generative modeling. arXiv preprint arXiv:2210.02744 2022.
Oord, A. v. d., Li, Y., and Vinyals, O. Representation learn- ing with contrastive predictive coding. arXiv preprint arXiv: h0t::8877.00 007.03748, 2018
Liu, X., Gong, C., and Liu, Q. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint Xiv:2209.03300 2022.
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn T., Müller, J., Penna, J., and Rombach, R. Sdxl: Im- proving latent diffusion models for high-resolution image synthesis. arXiv preprint urXiv:2307.019522 2023.
Liu, Y., An, J., Zhang, W., Li, M., Wu, D., Gu, J., Lin, Z., and Wang, W. Realera: Semantic-level concept erasure via neighbor-concept mining. arXiv preprint arXiv:2410.09140. 2024.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748 8763. PMLR, 2021.
Liu, Z., Luo, P., Wang. X., and Tang, X. Large-scale celeb- faces attributes (celeba) dataset. Retrieved August, (2018):11, 2018.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, Exploring the limits of transfer learning with unified text-to-text transformer. Journal of machine learning research, 21 (140):1-67, 2020.
Lorraine, J., Vicol, P., and Duvenaud, D. Optimizing mil- lions of hyperparameters by implicit differentiation. In International conference on artificial intelligence and statistics, pp. 1540-1552. PMLR, 2020.
Loshchilov, I., Hutter, F., et al. Fixing weight decay regu- larization in adam. arXiv preprint arXiv:1711.05101. 2017.
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Rad- ford, A., Chen, M., and Sutskever, I. Zero-shot text-to- image generation. In International conference on ma- chine learning, pp. 8821-88888 1831. Pmlr, 2021.
Lu, S., Liu, Y., and Kong, A. W.-K Tf-icon: Diffusion- based training-free free cross-domain image composition. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 2205, 2023.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06133 1(2):3, 2022.
Lu, S., Wang, Z., Li, L.,, Liu, Y., and Kong, W.-K. Mace: Mass concept erasure diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6430-6440, 2024a.
Rando, J., Paleka, D., Lindner, D., Heim, L., and Tramèr, F. Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04611 2022.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Lu, S., Zhou, Z., Lu, J., Zhu, Y., and Kong, A. W.KK Ro- bust watermarking using generative priors against image editing: From benchmarking to advances. arXiv preprint arXiv:2410. 18775, 2024b.

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Ronneberger, O., Fischer, P., and Brox, T. U-net: Con- volutional networks for biomedical image segmenta- tion. In Medical image computing and computer-assisted Intervention-MICC.. 2015: 18th international confer- ence, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234-241. Springer, 2015.
Xie, J., Li, Y., Huang, Y., Liu, H., Zhang, W., Zheng, Y., and Shou, M. Z.. Boxdiff: Text-to-image synthesis with maining-free box-constrained diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7452-7461, 2023.
Zhang, I.., Liang, Y., and Xie, P. Blo-sam: Bi-level opti- mization based overritting-reevennnng finetuning of sam. arXiv preprint 24XXii:2402213333 2024a.
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E. L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al. Photorealistic text-to-image dif- fusion models with deep language understanding. Ad- vances in neural information processing systems, 35: 36479-36494, 2022.
Zhang, Y., Chen, X., Jia, Zhang, Y., Fan, Liu, J., Hong, M., Ding, K., and Liu, Defensive unlearning with adversarial training for robust concept erasure in diffusion models. arXiv preprint arXiv:2405. /5234, 2024b.
Salimans, T. and Ho, J. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512.222 2022.
Zhang, Y., Jia, J., Chen, X., Chen, A., Zhang, Y., Liu, J., Ding, K., and Liu, To generate not? safety-driven un- learned diffusion models still easy generate unsafe images... for now. In European Conference on Computer Vision, pp. 385-403. Springer, 2024c.
Schramowski, P., Brack, M., Deiseroth, B., and Kersting, K. Safe latent diffusion: Mitigating inappropriate degenera- tion diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 22522 22522-22531, 22531, 2023.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35: 25278-25294, 2022.
Shen, Q. Wang, Y., Yang, Z., Li, X., Wang, H., Zhang, Y., Scarlett, J., Zhu, Z., and Kawaguchi, K. Memory-efficient gradient unrolling for large-scale bi-level optimization. arXiv preprint arXiv:2406. 14095, 2024.
Sinha, A., Malo, P., and Deb, K. A review on bilevel opti- mization: From classical to evolutionary approaches and applications. IEEE transactions on evolutionary compu-
tation, 22(2):276-29 2017.
Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
Su. J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Y.
Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 167612,, 2024.
Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
von Platen, P., Patil, S., Lozhkov, A., Cuenca, P., Lam- bert, N., Rasul, K., Davaadorj. M., Nair, D., Paul, S., Berman, W., Xu, Y., Liu, S., and Wolf, T. Diffusers: Sfate-of-thh-an diffusion models. https: / com/hugg wnghugggggggnannno
face/ face fuseess 2022.
12

---

EraseAnything:
Enabling Concept Erasure in Rectified Flow Transformers
A. Flux Architecture
In our research, we have chosen Flux [dev] as our baseline model due to its reputation as the most performant within the open-source Flux series 2. As highlighted in Section 3, Flux'ss architecture significantly diverges from that of SD v1.5, which has been the predominant baseline for contemporary concept erasure techniques.
As shown in Figure 8 and Figure 9, we have dissected the architecture of Flux ([schnell] and [dev] shared the same architecture). We discovered that, unlike in SD, Flux does not incorporate explicit cross-attention module. Nonetheless, we have observed that the dual stream block's 's approach to concatenating text and image features can emulate the cross- attention effects of SD. Specifically, this mechanism enables the identification of a word's heatmap within the attention map based on the token's position in the text, which can seen in Figure 10. Furthermore, we have found that by pruning this heatmap, we can effectively inhibit the generation of specific content, a finding that serves as pivotal foundation in our paper.
Caption
Noised Latent
CLIP-L/14
T5 XXL
Patching
RoPE
Pooled
Linear
Linear
MLP
Positional
Enbedding
Text
Inage
MLP
Dual Stream Block
Sinusoidal Embedding
Dual Stream Block
Timestep
Single Strean Block
UnPatching
Single Strean Block
Output
Figure 8. Model architecture of Flux [dev]. Flux [dev] use frozen CLIP-L 14 and TS-XXL as text encoders for conditioned caption feature extraction. The coarsed CLIP embedding concatenated with timestep embedding are used to modulation mechanism. The fine-grained TS concatenated with image latents are imput a stacked of double stream blocks and single stream blocks to predict output in the VAE encoded latent space. Concatenation is indicated by 3.
Building upon this finding, our optimization efforts now focused the dual stream block, illustrated in Figure 9). Our experimental results indicate that the parameters add_v-proj and to_v are highly numerically sensitive, rendering them less than ideal for optimization purposes. Consequently, we have shifted our focus to optimizing add q(k) proj and to q(k) instead. This strategic adjustment is expected to yield more robust and stable improvements in the model's performance.
For a fair comparison, we have adapted traditional methods such as ESD, UCE, and MACE, which typically optimize the Q, V, to instead optimize the Q K inside of Dual Transformer Block, This modification ensures that our comparative analysis is conducted under consistent and relevant framework.
hhags//tlacckonsstaasaaaaaaa wwwp://oawaaa.aoaouonnoccoccc htw.rannnnnnnaaaacaooor-----aaaa
13

---

ErascAnything: Enabling Concept Erasure in Rectified Flow Transformers
Dual Stream Block
LayerNorm
LayerNorm
Linear
Mod
Mod
Attention
Add_q_proj
To_q
Add_k_proj
To_k
Add_v_proj
To_v
K
Scale dot product attention
Linear
Linear
Figure Dual stream block. Flux, semantic correlation is established in dual stream block, which established an implicit relationshio between text and image. Noteworthy thing is that the explicit cross attention module that prevails among SD v1.5 is not existed Flux.
W Wattn
Yellow cat shake hands with Blue cat
shake hands
Image
Text2Inage
Yellow Cat
Blue Cat
Text
Text
Image
Figure Attention map extraction. correlation between specific words and corresponding heatmaps can discerned within the matrix W Wastn, particularly within the columns (white har adorned with blue dotted line) associated with text.
14

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
(%)
Attention Map Erasure
86
73
EraseAnything
a nude girl with beautiful hair and big breast
Nisspellings
Synonya
Repeated
Attack
Method
(a) Randomly Shuffled Prompt
(b) Attack success (ASR)
Figure Order Insensitive & Black box attack. (a) The sequence the prompt has minimal impact on the synthesized image. (b) Our learning-based method can maintain robustness against conventional black box attacks, whereas attention map erasure is ineffective.
B. Pattern of prompt & Black box attack
To address the issue of overfitting, we aim make the token index dynamic. Initially, we must validate hypothesis: "Randomly shuffling the prompt should not impact the generation results of Flux".
The basic prompt in our case is: "a nude girl with beautiful hair and big breast". To demonstrate Flux's s general- izability, we randomly shuffled this prompt at the word level: egg. "girl with beautiful and big nude hair breast". To ensure fairness, we fed these randomly shuffled prompts into popular online service, Fal.ai Fal.i is known for providing off-the-shelf Text2lmage APIs in an easily accessible manner, making it popular among users who wish to quickly test their ideas and create prototypes. We chose Fal.ai due to its swift image generation capabilities and the samper-prond nature of its model weights.
As depicted in Figure 11 (a), despite the alteration of word order within the prompt, the central attributes of the prompt remained robust: "beautiful; girl; mude; hair; breast" (even though the generated results oscillated between sensitive and regular content). Therefore, this experiment sufficiently demonstrated a key characteristic of Flux [dev]: Flux [dev] not sensitive to the word order the input prompt.
This serves as a compelling demonstration that we can effectively employ data augmentation by utilizing this property. justifies the practice of shuffling the prompt each iteration during training, enhancing the robustness of our model.
Furthermore, have curated set of 100 prompts that include recognizable objects styles, spanning from soccer, celebrities, to cartoons and art. Our goal here is to verify that the simple attention map erasure technique, as discussed in the context of cross cross-attention in Section 3), can be easily circumvented through rudimentary black-box prompt attacks.
As illustrated in Figure (b), the attention map erasure technique struggles to effectively handle misspellings and synonyms, as the token index for the target concept word differs from those of its misspellings and synonyms. Regarding the scenario where the target concept word is repeated (i.e., it appears least twice in the prompt), we have observed that the complete deletion of attention maps associated with the corresponding indices does prevent the re-generation of the target concepts. As shown in Figure 12, the attempted deletion of "New Balance" and "Dr.Martens" does not yield the expected outcome.
This finding underscores the complexity of the task and suggests that more sophisticated approach is needed to ensure that the target concepts are not regenerated in the output, regardless of their frequency in the input prompt. The current method of attention map erasure does not suffice, and thus, there is clear need for a more nuanced learning-bast erasure technique that can distinguish and eliminate the influence of repeated target concepts effectively. As demonstrated in Figure 1l (b), our method can effectively counter these black-box attack methods and significantly lower the attack success rate
aftps://all.a//
15

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
New Balance 990v5 prenium running shoe featuring grey nesh and suede upper, proninent Balance 'N' logo the side, cushioned midsole, and supportive rubber outsole.
Dr. Martens 1468 sturdy eight-eyelet lace-up boat made of snooth black leather, yellow stitching around the edges, air-cushioned sole, and Dr, Martens logo enbossed the side.
Figure 12. Repeated (target concept occurs more than twice in the input prompt). is apparent that the direct attention map erasure proves ineffective in addressing the re-generation problem of the target concept within the prompts. As illustrated in the figure, the first token index is denoted by purple, and the second token index is denoted by gold. We discovered that even after zeroing out all concept-related token indices the attention map, the resulting image still includes the concept that intended to be erased.
Table Al Agent template in generating ctr (Gun = "nude").
Role
Content
System
You are helpful assistant and well-established language expert' Hello, please return K (K=3) English words that you think with Human intuition are no relationnfar/mmi in the semantic space from the English word: Guns and only reply the result with JSON format is as follows:
User
("no relationn:: [(word], similarity score I ...]
"far": [(word1, mammilarity-ccrrelle ().
"mid": [(word1, similarity sc,
Response
("no relation": [("cloud", 0.1), ("tree", 0.2), ("carpet", 0.1)],
"ar"": [["hot", 0.3). ("color", 0.4), ("wet", 0.3)].
"mid": [("image"" 0.5), ("figure", 0.6), ("portrait", 0.5)]]
(ASR) below the acceptable level.
C. Prompt-related supplementary material
C.1.A A heuristic Gir sampling method
Identifying the concept Cüp that is unrelated to the target concept in the semantic feature space is not as straightforward as it may seem. General text feature encoders like T5 are typically trained on scale corpus data. The repeated occurrence of two seemingly unrelated concepts in the same training corpus might lead to a certain degree of correlation in the semantic feature dimension, causing the mapping position relationship of different text tokens in their semantic space to deviate from human perception of text words. Therefore, the similarity between text embeddings cannot be directly used as a measure to represent the correlation between two concepts.
To address this issue, we have devised heuristic ci sampling method. By leveraging the cognitive ability of LLM regarding human text concepts and through heuristic prompt design, we make them return concepts that are unrelated to the word Cun to be erased and also require the similarity between Cun and Crr Since the interaction with the LLMs occurs at the natural language level, the returned similarity is only a relative reference value, but suffices to meet our requirements for sampling Cy..
As shown in Table 6, the process of Gr first through building an AI Agent with unique role and regulated output format. We initiate the process by requiring GPT-4o to return Gir that they deem to be unrelated the target concept. After got the set of candidate values. Next, we classify and rank these concepts into three distinct categories: "no "ooareaiiio" signifying concepts that have minimal or no semantic connection; representin those with relatively loose semantic association; "mid", indicating a moderate level of relatedness.
After obtaining the initial response in Table 6, we randomly select each word from the three categories, which is in accordance with = by default as illustrated in the main paper.
16

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Table Complete list of conceptions of Entity, Abstraction, Relationship
Category
# Number
Prompt template
Conceptions
'Fruit', 'Ball', 'Car', 'Airplane',
Entity
10
'A photo of [Entity]'
'Tower', 'Building', 'Celebrity',
'Shoes', 'Cat', 'Dog'
'Pablo Picasso', 'Salvador Dali', 'Claude Monet', 'Vincent Van Gogh', 'Rembrandt van Rijn', 'Frida Kahlo', 'Edvard Munch', 'Leonardo da Vinci', 'Explosions', 'Environmental Simulation'
Abstraction
10
'An Art in the style of [Abstraction]'
'Shake Hand', 'Kiss', 'Hug'. 'In', 'On', 'Back to Back', 'Jump', 'Bur- row', 'Hold', 'Amidst"
Relationship
10
'A [Relationship] B'
C.2. Complete list of Entity, Abstraction, Relationship
For assessing the generalization of EraseAnything, we establish a conception list at three levels: from the concrete objects to the abstract artistic style and relationship, the full list used our experiments is presented Table
D. Derivative of Reverse Self-Contres Contrasntteteve Loss
As one of the proven method, InfoNCE loss is widely used in self-contrastive learning to learn model parameters by contrasting the similarity between positive and negative samples:
LInfoNCE = -log exp(sim(q. )+
E exp(sim(qq, k;))
where sim(g. k) denotes the similarity between the query vector q and the key vector k, k the key vector of the positive sample, k; represents the key vectors of negative samples, and K is the number of negative samples.
In conventional self-contrastive learning, we aim to make Fun more similar to paym to enhance the model's sensitivity to the term targeted for removal.
Lec = log exp (sim(Fun Feyn))
(8)
Σ exp (sim(Fun. Fk:)
However, in our case, desire the model be less sensitive to the term "nude" and its synonyms. Thus, we introduce the Reverse Self-Contrastive Loss through swapping the numerator and the denominator:
Σk exp(sim (Fun, FF:))
Crsc = log
(9)
exp(sim(FF Fun, Fssyn)
Here, 1u is the central feature, f the synonym feature, and Fk are the features of other irrelevant concepts.
To refine the model further, we consider introducing a temperature parameter adjust the distribution of similarity scores:
Fun Faym
sim F.yn)
(10)
Incorporating the temperature parameter into the loss function, we obtain:
17

---

ErascAnything: Enabling Concept Erasure in Rectified Flow Transformers
E exp punn,pt
Pro log
(1I)
Fun payn
exp
This derivation integrates the fundamental concepts of the InfoNCE loss function and tailors them to our specific case. By doing so, we can effectively guide the model to ignore the concept that bound to erased and its close synonyms during training, achieving the desired output.
E. User Study
Adhering to Flux's comprehensive evaluative criteria for Text-to-Image (T2I) models, we have integrated three key metrics into our user study: Imaging Quality, Prompt Adherence, Output Diversity. These metrics serve as the cornerstone for assessing the performance of our model. In our specific context, which focuses on the erasure of concepts to minimize their interference with the synthesis of images featuring unrelated concepts, we have introduced two additional metrics to refine our assessment framework: Erasing Cleanliness and Irrelevant Preservation.
Erasing Cleanliness evaluates the effectiveness of the concept erasure process, ensuring that the targeted concepts thoroughly removed without leaving any residual influence on the synthesized image. Irrelevant Preservation, the other hand, measures the model's ability to maintain the integrity and relevance of concepts that are not the focus of the erasure process, ensuring that the overall composition and context of the image are preserved within the model.
Figure 13 and Figure 14 provide a visual representation the user study interface, which was meticulously designed to facilitate smooth and engaging participant experience. During the study, participants were presented with series of image sets, each containing 6 and 3 results generated by various anonymous methods. They were then prompted score each method based its performance across the aforementioned metrics. The collected data subsequently compiled and visualized in a pentagonal chart, as depicted in Figure of the main paper, offering comprehensive overview of the methods' performance and highlighting the strengths and areas for improvement of each approach. This visual summary serves as a valuable tool for both researchers and practitioners, enabling a more nuanced understanding of the model's capabilities and guiding future developments in the field of image synthesis.
F. Others
Important Declaration: Owing to the rigid 10MB file size limit of ICML 2025, authors have had to compress the images to level that significantly reduces their quality in the appendices.
E.1. Celebrity
The names of celebrities used in our ablation study are illustrated in Table 8. The noteworthy thing here is that not arbitrary celebrities can be faithfully synthesised by Flux [dev), after manually comparing the synthesized famous people with its prompt and add some comic characters, we keep 50 for each group.
We train the celebrity recognition network on top of MobileNetV2 that pretrained on ImageNet, then add a SeraallAveaaePPoonnngg and Sof tmax (Dense) the end of the orginal output (out relu) of MobileNetV2. The learning rate a fixed lc-4 with Adam optimizer and loss function categorical cross-entropy.
Specification:
As for dataset, gather the data with average of 50 pictures per celebrity, with the gross number of 5,000. Then randomly re-sampled the dataset and divided into training set (80%) and test set (20%). The statistics are reported upon the test set (1,000), reserves one decimal fraction.
F.2. More Experimental Results
Ablation Study on Diverse Loss Configurations. As demonstrated in Figure 19, we conducted thorough comparison of outcomes utilizing various combinations of loss functions to our methodology. It is evident that the strategic integration of Liora significantly bolsters the visual consistency with the original character's 's appearance. Meanwhile, Lyse adeptly obscures the targeted concept, directing towards transformation into a myriad of incongruous notions. In contrast, Lead exemplifies the quintessential concept erasure strategy.
18

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
User Study
This test evaluating the generated images from unordered concept Al model, each time show results under the same prompt, guidance scale and seed. Users are needed to judge the result with two metrics: Erasing Cleanliness and Irrelevant Preservation.
Please rate images on scale from to where indicates lowest quality designated for evaluating 'Erasing Cleanliness' images, which refers effectiveness of the concept erasure method. Adjacent to it, you will find the 'Irrelevant Preservation' column (GREEN), which measures how well the images retain elements that were not targeted for erasure.
represents the highest quality. The column RED
Method A
Method D
Method B
Method E
Method C
Method F
A
D
Erasing
B
E
Cleanliness
C
F
3
Drake
A
Irrelevant
D
B
Preservation
E
3
C
Rolex
F
Figure 13. User Study on Erasing Cleanliness and Irrelevant Preservation (Screenshot).
User Study
This test for evaluating the generated images from unordered concept erasure Al model, each time show user results under the same prompt, guidance scale but varied seed. Users are needed to judge the result with three metrics: Imaging Quality, Prompt Adherence and Output Diversity.
Please the images scale from 5, where indicates the lowest quality and represents the highest quality. Prompts all sampled from PromptHero.
Method A
Method B
Method C
Prompt
the painting shows four aliens sitting sofa, in the style trompe-rcei illusionistic detail, presentation of human form, dark crimson light blue. american prints 1880-1950
Imaging Quality
2
3
5
2
4
3
5
Prompt Adherence
2
3
5
2
4
3
5
Output Diversity
2
3
2
5
Figure User Study on Imaging Quality, Prompt Adherence Output Diversity (Screenshot).
19

---

ErascAnything: Enabling Concept Erasure in Rectified Flow Transformers
Table 8. Complete list celebrities used ablation study.
Category
Number
Celebrity
'Adele', 'Albert Camus', 'Angelina Jolie", 'Armold Schwarzenegger. 'Au- drey Hepburn', 'Barack Obama', 'Beyonce', 'Brad Pitt', 'Bruce Lee', CChis Evans', 'Christiano Ronaldo', 'David Beckham'. 'Dr Dre', 'Drake', "Elizabeth Taylor', 'Eminem'. 'Elon Musk', 'Emma Watson', 'Frida Kahlo', 'Hugh Jackman', 'Hillary Clinton', 'Isaac Newton', 'Jay-Z', 'Justin Bieber', 'John Lennon', 'Keanu Reeves', 'Leonardo Dicaprio', 'Mariah Carey', 'Madonna', 'Marlon Brando', 'Mahatma Gandhi', 'Mark Zuckerberg' 'Michael Jordan', 'Muhammad Ali', 'Nancy Pelosi', 'Neil Armstrong', 'Nelson Mandela', 'Oprah Winfrey', 'Rihanna', 'Roger Fed- ever', 'Robert De Niro', 'Ryan Gosling', 'Scarlett Johansson', 'Stan Lee', 'Tiger Woods', 'Tmotthee Chalamet', 'Taylor Swift', 'Tom Hardy', 'William Shakespeare'. 'Zac Efron'
Frasure Group
50
Angela Merkel', 'Albert Einstein', 'Al Pacino', 'Batman' 'Babe Ruth Jr', BBn Affleck', 'Bttte Midler', 'Benedict Cumberbatch', 'Bruce Willis', 'Bruno Mars', 'Donald Trump'. 'Doraemon'. 'Denzel Washington', 'Ed Sheeran', 'Emmanuel Macron', *Elvis Presley', 'Gal Gadoi', 'George Clooney', 'Goku' :Jake Gyllenhaal', 'Johnny Depp', 'Karl Marx', 'Kanye West', 'Kim Jong Un', "Kim Kardashian', 'Kung Fu Panda', 'Lionel Messi', 'Lady Gaga'. 'Martin Luther King Jr.", 'Matthww McConaughey', 'Morgan Freeman', 'Monkey D. Luffy', 'Michael Jackson', 'Michael Fassbender', 'Marilyn Monroe', 'Naruto Uzumaki', 'Nicolas Cage', 'Nikola Tesla', 'Op- timus Prime', 'Robert Downey Jr.', 'Saitama', 'Serena Williams', 'Snow White', 'Superman's 'The Hulk', 'Tom Cruise', 'Vladimir Putin', 'Warren Buffett', 'Will Smith', 'Wonderwoman'
Retention Group
50
20

---

EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers
Benchmarking Against State-of-the-Art (SOTA), As depicted in Figure 15, we compare EraseAnything with state-of- the-art (SOTA) methods various concepts. It can be easily observed that Attention Map is sufficient remove target concept. However, as previously analyzed in Appendix B, such methodologies are susceptible rudimentary black-box attacks, rendering them impractical for real-world applications.
LoRA Disentanglement Analysis. To assess the potential influence of integrating fine-tuned LoRAs into the original Flux [dev], as depicted in Figure 16, it can be observed that incorporating fine-tuned LoRAs for diverse concepts, i.e. Celebrity: Batman, Christiano Ronaldo, Hulk, Lebron James, Wonderwoman. Object: Alaskan Malamute, Statue of Liberty, Basketball, Skyseraper, Cat and Art: Van Gogh, Edvard Munch, Rembrandt van Rijn, Claude Monet, Salvador Dali, does not adversely affect the original image synthesis capabilities. All above-mentioned concepts depicted sequentially from left to right.
Exploring the Synergy of Combined Concept-Erased LoRAs. In our quest to unravel the potential of integrating concept-erased LoRAs, we delve into the intricacies of merging these elements into a cohesive single entity, denoted as A0mul This experiment meticulously designed to assess the capabilities of image synthesis when multiple LoRAs are unified. Specifically, we randomly sample LoRAs from Table 7 and combine them using Equation (12).
As depicted in Figure 17, the upper side of the blue dashed line represents Σ W; W; N. indicating linear normalized weight blending strategy. Conversely, the lower side of the line reveals the implications of non-normalized sum, where Ei= W, N, W; 1. Here, N represents the total number of LoRAs being combined, e.g. 3,5, 10.
N
A mull
W,40,
(12)
==0
The process of image synthesis significantly impacted when the cumulative weight of the combined LoRAs, denoted by Σi W,, exceeds the normalized threshold of 1. This surpassing signals critical juneture in the image synthesis process, potentially resulting in an overemphasis on certain concepts while inadvertently neglecting others. Such a shift could introduce a bias towards recognized concepts, possibly at the expense of exploring new or unrelated themes.
Conversely, when the aggregate weight remains within the confines of 1, the model's prowess in generating a diverse array of unrelated concepts remains largely indistinguishable from the original Flux[dev] model, underscoring the model's robustness.
Multiple Concept Erasure. Leveraging the insights gleaned from aforementioned findings, we venture explore the hypothesis of concept erasure with greater depth:
Q: Can EraseAnything is capable of erasing multiple concepts in the meantime?
Resoundingly, the answer is affirmative. As depicted in Figure 18, through the linear interweaving of LoRAs representing distinct concepts under a normalized weight sum, we achieve the coveted outcome of concept erasure that harmoniously integrates with the backdrop of the environment. This capability positions FraseAnything as an exemplary contender for advanced concept crasure endeavors.
F33 Limitations
Although "EraseAnything" has demonstrated its formidable ability to erase concepts across various domains, we have identified challenges it faces in certain situations:
When tasked with erasing multiple concepts simultaneously. such as 10 or more concepts (LoRAs), the Normalized Sum strategy, as depicted in Equation (12), results in proportional decrease in the impact of each concept's 's erasure. Consequently, significant and important avenue for research in this field is to explore efficient methods for combining a large number of LoRAs (more than 100) effectively.
Extensive Concept Erasure:
Fine-grained Control: Another issue pertains to the inability to guarantee the strength of the erasure during fine-tuning. This is an uncharted yet intriguing area in the realm of concept erasure, which could provide us with a deeper understanding of the concept formulation. would also enable more precise control over the erasure process, e.g. a slider could be provided to control the intensity during interactive concept erasure.
21

---

ErascAnything: Enabling Concept Erasure in Rectified Flow Transformers
Flux [dev]
Attention
CA [model] L]
ESD
EAP
Ours
map
Pablo Picasso
Kanye West
Baobab
Tiger
HOR
Iron Sports Hold tower
car
AA
Figure 15. Comparison with mainstream concept erasing methods. We compared EraseAnything to other concept erasers on Flux [dev] across categories like Art Style, Celebrity, Plant & Animal, Relationship, Car & Architecture. The Attention Map (3rd column from the right) shows the simple token localization method from Scction that crases target concept effectively, yet its vulnerable to the minor change of tokens- misspellings, prefixes suffixes and repeated words make it difficult widely adopt practical applications.
22

---

ErascAnything: Enabling Concept Erasure in Rectified Flow Transformers
Bus
STUTTMM
20
Frida Kahlo
MOSCHA
13
Church
Bus
Frida Kahlo
Church
Bus
Frida
Kahlo
Church
Figure 16. Visualization on LoRA Disentanglement. The left side of the blue dashed line delineates the ceasure-concept eeneatteeeeeggeeerteeedddddddddd
images (yellow box) and the original image (green box the lower left). The right side illustrates the result unrelated concepts upon incorporating the LoRA associated with the erased concept. Top rows: Celebrity; Mid rows: Object; Last rows: Art.
23

---

ErascAnything:
Enabling Concept Erasure in Rectified Flow Transformers
Winter Landscape, Palace-level design, white blue orange
A cloaked astronaut stands in the gloom ho holding orb g glowing pink
Multi-LoRAs
Superman riding a horse
3
5
10
Figure Compositional LoRAs for irrelevant concepts. randomly sampled irrelevant concept-eran LoRAs and blending them in two ways: Normalized Sum (above the blue dotted line) and Un-Normalized Sum (below the blue dotted line).
Johnny Depp is riding bicycle, with his bodyguard behind
Johnny Depp
bodyguard
bodyguard
Johnny Depp
Two girl hug each other, in the style of Frida Kahlo
Two
Hug
Frida Kahlo
Hug
Figure 18. Compositional LoRAs for related concepts. We find that through Normalized Sum, effectively erase multiple concepts the same time.
24

---

Aryhhnnything: Enabling Concept Erasure in Rectified Flow Transformers
Flux [dev]
Lexd Lattnn
Lesd Liora
Lead + Frae
Lattn Lioro Lree
FULL
Johhny Depp
Serena Williams
Lionel Messi
Ralcsten
Goku
Optimus Prime
Figure 19, Ablation Study on different loss configs.
25


Reviews:
[{"review_id": "d72e0yBPS7", "reviewer": "Reviewer_BPSn", "summary": "This paper introduces EraseAnything, a flux-based concept erasing method designed to selectively remove target concepts while preserving irrelevant ones. The authors employ a bi-level optimization strategy to mitigate overfitting and catastrophic forgetting—key challenges in concept erasure. Experimental evaluations across diverse tasks demonstrate the method’s effectiveness and versatility, highlighting its potential impact in controlled information removal and model robustness.\n\n## update after rebuttal\nThe authors’ rebuttal has clarified my previous concerns. Taking into account the other reviewers' feedback and the authors' response, I choose to maintain my original evaluation (Weak accept).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your kind words and review!\n\nTo be concise:\n\n* **UCE**'s aggressive nudity removal significantly distorts images.\n* **EraseAnything** prioritizes image quality and text alignment, offering a better trade-off.\n\nAs shown in [this image](https://imgur.com/a/at2lkh8), optimizing **'K'** in our UCE implementation on **Flux[dev]** reduces nudity but degrades image quality, highlighting this inherent trade-off. (Optimizing **'Q'** had no effect, and **'V'** yielded noisy images.)\n\nWhile UCE removes more nudity, EraseAnything maintains superior image quality (better performance on FID and CLIP on the MS-COCO 10K dataset), which is crucial for practical use.\n\n* **UCE Optimization** [https://imgur.com/a/at2lkh8](https://imgur.com/a/at2lkh8)", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No, as there are no theoretical claims made", "experimental_designs_or_analyses": "I reviewed the experimental results presented in Tables 2, 3, and 4, and they appear to be correct.", "relation_to_broader_scientific_literature": "This paper explores the problem of targeted concept erasure in deep learning models, aligning with broader discussions in the machine learning community on model interpretability, unlearning, and mitigating biases. The proposed Flux-based approach builds upon rectified flow transformers, contributing to existing literature on concept erasure and catastrophic forgetting. The work is relevant to ongoing discussions in ICLR, NeurIPS, and ICML regarding responsible AI and controllable generation in large models.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "**Paper Strengths:**\n\nThe paper is well written. The main motivation is clear and easy to understand.\n\n**Major Weaknesses:**\n\nIn Table 2, UCE outperforms the proposed EraseAnything in terms of the number of DETECTED NUDITY instances but performs worse in terms of FID and CLIP on the MS-COCO 10K dataset. What accounts for this discrepancy between these metrics? Does this imply that UCE is superior to EraseAnything overall?", "other_comments_or_suggestions": "None", "questions_for_authors": "See weakness.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NFggOXQ8yY", "reviewer": "Reviewer_RmCr", "summary": "In this paper, the authors propose a methodology for concept unlearning while ensuring the preservation of unrelated concepts in the latest text-to-image (T2I) models based on Flow Matching and Transformer-based diffusion models such as Flux. The authors introduce a bi-level optimization (BO) framework. The lower-level optimization focuses on concept removal, while the upper-level optimization ensures the preservation of unrelated concepts. The proposed method is evaluated through quantitative experiments, outperforming state-of-the-art techniques in nudity erasure and output preservation, except for UCE in the “nudity” concept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your kind words and recognition!\n\n-   **(A) Adversarial attack experiments**\n\n    Thank you for the suggestion to include adversarial attack experiments, which we consider very important. Following the paper's methodology, we used `NudeNet` (Bedapudi, 2019) with a detection threshold of **0.6** to test the **Attack Success Rate (ASR)** on the [RingABell-Nudity](https://huggingface.co/datasets/Chia15/RingABell-Nudity) `[1]` dataset (comprising 285 Ring-A-Bell revised prompts focused on nudity). Since the prompts in RingABell-Nudity are already processed according to the standard procedure, we did not reapply the Ring-A-Bell method.\n\n    The table below shows the results of our tests on ESD, CA, and our proposed method using this dataset. We also included the attack results from MU-Attack `[2]`. Step 0 means only attack the very initial `velocity` of Flux, Step 0,1,2 means attack the initial three `velocity`. According to our experiments, when attack too much steps, would yield irrelevant image w.r.t to the prompt. \n\n    | Concept             | Methods                 | Flux[dev] | ESD (Flux[dev]) | CA (Flux[dev]) | EraseAnything (Flux[dev]) |\n    | :------------------ | :---------------------- | :-------- | :-------------- | :------------- | :------------------------ |\n    | Nudity (RingABell Nudity) | Original (Org)          | 59.65%    | 7.36%          | 3.16%          | 2.46%                     |\n    |                     | MU-Attack (step 0)      | 64.56%    | 11.57%          | 15.44%         | 8.77%                     |\n    |                     | MU-Attack (steps 0,1,2) | 65.96%    | 14.74%          | 16.49%         | 11.93%                    |\n\n-   **(B) More baselines**\n\n    Thank you for your advice. We will conduct a thorough probe on relevant papers, and incorporate more baseline methods, such as `AdvUnlearn` into the final version.\n\n-   **(C) Flux baseline missing?**\n\n    We have implemented all relevant methods, including ESD, CA, and MACE, within the Flux[dev] framework. Therefore, all concept erasing methods reported in the tables were conducted on Flux[dev].\n\n-   **(D) Questions**\n\n    Firstly, this signifies that we have adapted the previous SD 1.5 methods to the Flux[dev] architecture. Secondly, given that UCE led to poor visual results ([https://imgur.com/a/at2lkh8](https://imgur.com/a/at2lkh8))`[3]`—a finding consistent with its SD 1.5 implementation after careful review—we opted to use ESD as our baseline.\n\n-   **References**\n\n    [1] [\"Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?,\" Tsai et al.](https://github.com/chiayi-hsu/Ring-A-Bell)\n\n    [2] [\"To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now,\" Zhang et al.](https://github.com/OPTML-Group/Diffusion-MU-Attack)\n\n    [3] [https://imgur.com/a/at2lkh8](https://imgur.com/a/at2lkh8)", "claims_and_evidence": "All the claims made in this paper are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed method and the evaluation criteria make sense for the problem.\nThe authors follow standard evaluation criteria that evaluate the method on the benchmark.\n\nHowever, adversarial attack-based benchmarking is lacking. For instance, Ring-a-bell [1] and UnlearnDiff [2] should be used like the many baselines for detailed comparisons.\n\n[1] “Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?,” Tsai et al.\n\n[2] “To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now,” Zhang et al.", "theoretical_claims": "The derivation for the loss is sound and justifies the claim (embedding of the target concept is aligned with the embeddings of irrelevant concepts and pushed away from the synonym of the target concept).", "experimental_designs_or_analyses": "Yes, the soundness of the following experimental designs by the author is verified and looks good.", "relation_to_broader_scientific_literature": "The proposed approach targets the transformer-based recent models, which might interest a broader community.", "essential_references_not_discussed": "In the literature work, FMN[1] paper could be added as they do attention (cross-attention) regularization between the attention map and text embeddings for concept erasure.\n\n[1] “Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models,” Zhang et al.", "other_strengths_and_weaknesses": "Strengths:\n- The paper has been written in a concise and clear way.\n- The analyses presented on why the current state-of-the-art methods do not work for models Flux is presented very well.\n- The paper explored the potential of scaling concept erasure to multiple concepts and presented results in the appendix F2 Multiple Concept Erasure.\n- The User Study analysis shown in Figure 4 and in Appendix E is extremely helpful in assessing the effectiveness of the method under various metrics.\n\nWeakness:\n- Several evaluations, such as adversarial attacks such as Ring-a-Bell [1], UnlearnDiff [2], and Attacks, have not been presented. It could be a useful evaluation of the effectiveness of the attention map regularization loss method.\n- Table 2/3/4 can include more baseline methods (AdvUnlearn) for an in-depth analysis. Look at (https://huggingface.co/spaces/Intel/UnlearnDiffAtk-Benchmark)[https://huggingface.co/spaces/Intel/UnlearnDiffAtk-Benchmark].\n- Additionally, Flux-based baselines are missing. For instance, many of the prior works (AdvUnlearn, UCE, ESD, etc.) could be extended to Flux and treated as the baseline to see the proposed approach's impact truly. \n\n[1] “Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?,” Tsai et. al.\n\n[2] “To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now,” Zhang et. al.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "- Can you elaborate on the last paragraph in Appendix A. Flux Architecture: “For a fair comparison, we have adapted traditional methods such as…conducted under a consistent and relevant framework.” on why the said modification ensures consistent comparative analysis?\n- Instead of the ESD loss function, can we utilize the UCE loss function since you established a linear relationship between the text embeddings and attention map? How would it affect the performance of the model?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uauXdI41D2", "reviewer": "Reviewer_DCT3", "summary": "Given that current text-to-image models can generate inappropriate content related to pornography, violence, or copyright violations, the problem of effective concept erasure has become a critical research topic. Existing methods have proven effective for Stable Diffusion but are challenging to directly adapt to SD3 and FLUX.\n\nThis paper investigates concept erasure algorithm on FLUX, highlighting the differences between FLUX and SD in terms of model structure and encoder properties. The authors propose a robust concept erasure algorithm leveraging bi-level optimization techniques, integrating Forge Me Not, Erasing Stable Diffusion, and Reverse Self-Contrastive approaches. Experimental results demonstrate that their method achieves superior qualitative and quantitative erasure performance on the FLUX architecture, surpassing other existing concept erasure algorithms.\n\n## update after rebuttal\nThe author show the comparison between bi-level optimization and multi-objectives and explain the reason why attention localization owns contribution to the flux-based architecture and the generative field, which address my concern. However, the attention localization and the bi-level optimization is kind of simple and boardline to me, so I keep my rate.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your insightful feedback. We will revise the manuscript to ensure a balanced and objective narrative, avoiding any exaggeration or overstatement.\n\n- **(A) Difference between bi-level optimization and multi-objective optimization**\n\nWe frame unsafe concept erasing as a bi-level optimization problem, rather than a multi-objective one. While multi-objective optimization balances competing goals equally (e.g., erase unsafe concepts and preserve irrelevant ones), it lacks a clear prioritization.\n\nIn contrast, bi-level optimization explicitly models a hierarchy:\n\n\t•\tLower-level erases unsafe concepts.\n\t•\tUpper-level evaluates whether irrelevant concepts are preserved.\n\nThis reflects the asymmetric nature of our goals: erasure is primary; preservation is a constraint. Bi-level optimization allows for more precise control, better mirrors real-world usage (apply first, evaluate second), and is well-suited for safety-critical tasks where minimizing unintended harm is essential.\n\n- **(B) attention localization**\n\nAttention localization analysis in UNet-based Stable Diffusion is intuitive and well-understood due to the presence of explicit cross-attention mechanisms. However, in the joint attention architecture of MMDiT, explicit cross-attention is absent. Our work demonstrates that, despite this absence, the joint attention mechanism in MMDiT still retains attention localization properties similar to those found in UNet's explicit cross-attention. Leveraging this insight, we introduce a framework to concept erasure within the MMDiT architecture. Although this insight may seem straightforward, it is beneficial for the research community, particularly benefiting future FLUX-based erasure studies.\n\n- **(C) real difficulty in adapting unlearning to flow matching?**\n\nAchieving effective concept erasure in Flow Matching models presents a significant challenge, as direct application of existing methods like UCE and ESD proves inadequate. To address this, we conducted a comprehensive structural analysis of Flux, meticulously probing its intricacies to identify viable improvement strategies. Through extensive experimentation, we found that precise adjustments to the `to_q_proj` and `to_k_proj` projections within the dual transformer block are essential for successful erasure.", "claims_and_evidence": "1. The paper appears to overclaim the contribution of bi-level optimization. In the introduction, the authors present bi-level optimization as a core contribution. However, as described in Algorithm 1, the proposed method simply alternates between optimizing the erasure and preservation losses, which is essentially a trivial multi-objective optimization implementation without notable innovation.\n2. The paper seems to overclaim the contribution of attention localization. In the introduction, the authors treat Attention Localization as a core contribution and refer to it as “a depth analysis.” However, as later sections describe, MMDiT applies a concat operation on Q and K, followed by a self-attention-like structure. Identifying the corresponding positions of image and text tokens in the attention matrix is straightforward and does not qualify as “a depth analysis.”", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "The paper employs well-established techniques such as Attention, Erasing Stable Diffusion (ESD), and LoRA, which are already mature methods for concept forgetting and fine-tuning. The authors merely make slight modifications to the loss function and attention matrix representation to apply them to FLUX. The fact that these methods can be easily adapted suggests that transferring SD-based forgetting algorithms to FLUX is not particularly challenging—contradicting the authors’ claim that adapting SD-based methods to FLUX presents fundamental difficulties. Instead, it seems that the authors achieve better performance simply by stacking existing methods.", "relation_to_broader_scientific_literature": "Flow-matching-based diffusion models represent the cutting edge of generative models in current research. Investigating their forgetting mechanisms is of great importance for the future of AI safety. The method proposed in this paper lays a foundational groundwork for future research in this direction, offering significant value for advancing both the capabilities and responsible use of such models.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "Typo: In line 197, nude nude should be nude.", "questions_for_authors": "My main concern lies in the authors' overstatement regarding the contribution of their method, as well as the claimed ease of adapting it to flow-matching-based diffusion models. The author should answer the following question to show the novelty and contribution of their method.\n1. What's the difference between bi-level optimization and multi-object optimization?\n2. What's the depth analysis of attention localization? it seems to be a very straightforward idea.\n3. What's the real difficulty of adapting to unlearning into flow matching? The method the authors used is just the combination of existing methods with slight modification.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PD3ISIF3nf", "reviewer": "Reviewer_a83u", "summary": "This paper highlights the limitations of existing concept-erasing methods, such as CA, ESD, and UCE, which were developed for Stable Diffusion models utilizing U-Net, cross-attention, and CLIP text encoders. The authors argue that these methods are ineffective for Flux, a modern multi-modal diffusion transformer that employs a T5 text encoder. To address this gap, the paper proposes new loss functions for concept erasure.\n\nRecognizing that concept erasure involves a bi-level optimization problem balancing both concept removal and irrelevant concept preservation, the authors integrate multiple loss terms: the original ESD loss (Equation 2), an attention-attenuation loss (Equation 3), a diffusion loss (Equation 4), and a reverse self-contrastive loss (Equation 5) to preserve irrelevant concepts. The experiments primarily focus on nudity removal, but also include tests on entity, abstraction, and relationship-based concepts, as well as celebrity face removal.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments and interest in our work!\n\n- **(A) Limited Fine-Tuning due to VRAM Constraints**\n\n  We acknowledge the reviewer's concern regarding limited fine-tuning. Due to 80GB VRAM constraints on our single A100, full fine-tuning was infeasible. We opted for LoRA, prioritizing layers with the most significant impact on text-to-image generation. Optimizing `to_q` and `to_k` degraded image quality without effective concept erasure, while `add_q_proj` and `add_k_proj` proved effective. Optimizing `add_v_proj` and `to_v` yielded noisy outputs, leading to their exclusion. We recognize the architectural differences between Flux and Stable Diffusion (MMDiT vs. U-Net, Rectified Flow vs. DDPM/DDIM) and base our conclusions on empirical observations.\n\n- **(B) T5 Dominance in Flux Generation**\n\n  Our experiments demonstrate that the T5 encoder significantly influences Flux's generation. As shown [here](https://imgur.com/a/047aypl), `prompt_embeds` from T5 acts as `encoder_hidden_states`, similar to CLIP in SD models. Conversely, `pooled_prompt_embeds` from CLIP primarily affects time embeddings, with minimal impact on final output. Adding noise to T5 features drastically altered the output, while changes to CLIP features were negligible. Therefore, we focused on T5.\n\n- **(C) EraseAnything: Quantity and Color Validation**\n\n  We appreciate the reviewer's request for diverse examples. To further validate EraseAnything's robustness, we provide examples demonstrating erasure of quantity and color: **\"green\" from \"green bag,\" \"red\" from \"red rose,\" \"five\" from \"five pencils,\" and \"three\" from \"three cats\"** ([image](https://imgur.com/a/TIxXi9u)). Combined with supplementary material, this reinforces EraseAnything's effectiveness.\n\n- **(D) Rationale for Excluding SD3**\n\n  We excluded SD3/SD3.5 due to their comparatively lower general image generation performance. Given Flux's status as a flagship model developed by **Robin Rombach's team**(Black Forest Lab, father of Stable Diffusion), we believe our findings on Flux sufficiently demonstrate EraseAnything's capabilities.\n\n- **Image URLs:**\n\n  * **T5 vs. CLIP:** [https://imgur.com/a/047aypl](https://imgur.com/a/047aypl)\n  * **Erase Quantity & Color:** [https://imgur.com/a/TIxXi9u](https://imgur.com/a/TIxXi9u)", "claims_and_evidence": "This paper argues that existing text-to-image concept-erasing methods, such as CA, ESD, and UCE, which were originally developed for Stable Diffusion architectures, fail to generalize to the Flux architecture. The authors demonstrate this claim visually in Figure 1, showing that applying methods like ESD, UCE, and EAP to Flux does not effectively erase concepts such as \"nude.\" However, the evidence provided in this paper is insufficient to fully support this claim.\n\nFirstly, according to line 363, the authors conducted experiments that only fine-tunes “add_k_proj” and “add_q_proj” within the dual-stream blocks of Flux. This limited approach raises concerns because it excludes other potentially crucial layers, such as all attention layers across both 19 dual-stream and 38 single-stream blocks (\"to_k\", \"to_q\", \"to_v\", \"to_out.0\"), which could significantly influence concept erasure. To demonstrate that methods designed for Stable Diffusion are ineffective for Flux, a more comprehensive evaluation involving fine-tuning of all relevant attention layers is necessary. In the appendix, while the authors mention excluding \"add_v_proj\" and \"to_v\" due to numerical sensitivity, fine-tuning these layers is common practice in the community [A].\n\nMoreover, the authors primarily rely on LoRA-based fine-tuning, which inherently preserves the original concepts within pre-trained model weights. Thus, to convincingly demonstrate genuine concept erasure, experiments involving full fine-tuning are required.\n\nSecondly, Flux employs not only the T5 encoder but also the CLIP text encoder. Since Flux utilizes both encoders, it remains unclear whether concept-erasing methods would perform differently if we use the CLIP encoder alone. \n\n[A] huggingface, https://github.com/huggingface/diffusers/tree/main/examples/dreambooth", "methods_and_evaluation_criteria": "The evaluation methods generally make sense for assessing quality. However, the claim in Section 5.2 that FlexControl provides an \"erase anything\" solution requires additional supporting evidence. Specifically, authors should demonstrate the method's effectiveness through diverse and challenging cases, such as erasing color (i.e., red rose, green bag) or object count (i.e., two oranges, three cats).", "theoretical_claims": "The target in the loss function is a scalar value using the L2 norm. However, ESD’s objective function regresses the model toward the guided prediction of the pre-trained model.", "experimental_designs_or_analyses": "As mentioned previously, the experimental analysis should include results from LoRA and full fine-tuning across all attention layers.", "relation_to_broader_scientific_literature": "Concept erasure in generative models is a key research challenge in text-to-image generation. This paper extends prior works, such as ESD, by adapting the loss formulation to transformer-based architectures like Flux.", "essential_references_not_discussed": "No issues found.", "other_strengths_and_weaknesses": "See “Claims And Evidence”", "other_comments_or_suggestions": "Additional evaluation with Stable Diffusion 3.", "questions_for_authors": "No additional questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes EraseAnything, a concept erasure method for Flux-based diffusion models to address a timely and underexplored problem.

All four reviewers leaned positive, with one increasing their score to accept after a strong rebuttal. While some concerns remain—mainly around the simplicity of the bi-level optimization and attention localization—the authors responded thoroughly with additional experiments and clarifications. The method’s empirical effectiveness and its relevance to safety and controllability make it a valuable contribution.
Decision: Accept (poster)