Finding Dori: Memorization in Text-to-Image Diffusion Models Is Less
  Local Than Assumed
Authors: name: Antoni Kowalczuk, name: Dominik Hintersdorf, name: Lukas Struppek, name: Kristian Kersting, name: Adam Dziedzic, name: Franziska Boenisch

Text-to-image diffusion models (DMs) have achieved remarkable success in
image generation. However, concerns about data privacy and intellectual
property remain due to their potential to inadvertently memorize and replicate
training data. Recent mitigation efforts have focused on identifying and
pruning weights responsible for triggering replication, based on the assumption
that memorization can be localized. Our research assesses the robustness of
these pruning-based approaches. We demonstrate that even after pruning, minor
adjustments to text embeddings of input prompts are sufficient to re-trigger
data replication, highlighting the fragility of these defenses. Furthermore, we
challenge the fundamental assumption of memorization locality, by showing that
replication can be triggered from diverse locations within the text embedding
space, and follows different paths in the model. Our findings indicate that
existing mitigation strategies are insufficient and underscore the need for
methods that truly remove memorized content, rather than attempting to suppress
its retrieval. As a first step in this direction, we introduce a novel
adversarial fine-tuning method that iteratively searches for replication
triggers and updates the model to increase robustness. Through our research, we
provide fresh insights into the nature of memorization in text-to-image DMs and
a foundation for building more trustworthy and compliant generative AI.