Title: TraSCE: Trajectory Steering for Concept Erasure
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Concept Erasure, Diffusion Trajectory, Negative Prompting, Safe Generation

Abstract: Recent advancements in text-to-image diffusion models have brought them to the public spotlight, becoming widely accessible and embraced by everyday users. However, these models have been shown to generate harmful content, such as not-safe-for-work (NSFW) images. While approaches have been proposed to erase such abstract concepts from the models, jail-breaking techniques have succeeded in bypassing such safety measures. In this paper, we propose TraSCE, an approach to guide the diffusion trajectory away from generating harmful content. Our approach is based on negative prompting, but as we show, the widely used negative prompting strategy is not a complete solution for concept erasure and can easily be bypassed in a simple corner case. To address this issue, we introduce two techniques. We first borrow the idea of concept negation from compositional generation and propose to use it instead of the conventional negative prompting. Second, we introduce a localized loss-based guidance that enhances the modified negative prompting technique by steering the diffusion trajectory. We demonstrate that our proposed method achieves state-of-the-art results on various benchmarks in erasing harmful content, artistic styles, and objects, including on benchmarks introduced by red teams without impacting unrelated concepts. Our proposed approach does not require any training, weight modifications, or training data (either image or prompt), making it easier for model owners to erase new concepts.