ICML 2025

Understanding and Mitigating Memorization in Generative Models via
Sharpness of Probability Landscapes
Dongjae Jeon * 1 Dueun Kim * 2 Albert No 2
Abstract
In this paper, we introduce a geometric framework
to analyze memorization in diffusion models
through the sharpness of the log probability
density. We mathematically justify a previously
proposed score-difference-based memorization
metric by demonstrating its effectiveness in
quantifying sharpness. Additionally, we propose
a novel memorization metric that captures
sharpness at the initial stage of image generation
in latent diffusion models, offering early insights
into potential memorization.
Leveraging this
metric, we develop a mitigation strategy that
optimizes the initial noise of the generation
process using a sharpness-aware regulariza-
tion term.
The code is publicly available at
https://github.com/Dongjae0324/
sharpness_memorization_diffusion.
1. Introduction
Recent advancements in generative models have signifi-
cantly improved data generation across various domains,
including image synthesis (Rombach et al., 2022), natural
language processing (Achiam et al., 2023; Touvron et al.,
2023), and molecular design (Alakhdar et al., 2024). Among
these, diffusion models (Ho et al., 2020; Song et al., 2021c)
have emerged as powerful frameworks, achieving state-of-
the-art results by iteratively refining noisy samples to ap-
proximate complex data distributions (Song et al., 2021b;
Saharia et al., 2022; Rombach et al., 2022).
Despite their successes, diffusion models suffer from mem-
orization, where they replicate training samples instead of
generating novel outputs (Carlini et al., 2023; Somepalli
et al., 2023b; Webster, 2023). This issue is especially con-
*Equal contribution 1Department of Computer Science, Yonsei
University, Seoul, Korea 2Department of Artificial Intelligence,
Yonsei University, Seoul, Korea. Correspondence to: Albert No
<albertno@yonsei.ac.kr>.
Proceedings of the 42 nd International Conferenc...