A Geometric Framework for Understanding Memorization in Generative Models
Authors: Brendan Leigh Ross
Hamidreza Kamkari
Tongzi Wu
Rasa Hosseinzadeh
Zhaoyan Liu
George Stein
Jesse C. Cresswell
Gabriel Loaiza-Ganem
Venue: ICLR 2025 Spotlight (2025)
Keywords: deep generative modelling
generative models
memorization
data copying
privacy
diffusion
diffusion models
GANs
manifold hypothesis
local intrinsic dimension
lid
lid estimation
geometry
Primary area: generative models
TL;DR: We show that the manifold hypothesis is a useful way to understand memorization in generative models

Abstract: As deep generative models have progressed, recent work has shown them to be capable of memorizing and reproducing training datapoints when deployed. These findings call into question the usability of generative models, especially in light of the legal and privacy risks brought about by memorization. To better understand this phenomenon, we propose the *manifold memorization hypothesis* (MMH), a geometric framework which leverages the manifold hypothesis into a clear language in which to reason about memorization. We propose to analyze memorization in terms of the relationship between the dimensionalities of $(i)$ the ground truth data manifold and $(ii)$ the manifold learned by the model. This framework provides a formal standard for "how memorized" a datapoint is and systematically categorizes memorized data into two types: memorization driven by overfitting and memorization driven by the underlying data distribution. By analyzing prior work in the context of the MMH, we explain and unify assorted observations in the literature. We empirically validate the MMH using synthetic data and image datasets up to the scale of Stable Diffusion, developing new tools for detecting and preventing generation of memorized samples in the process.

Published as a conference paper at ICLR 2025
A GEOMETRIC FRAMEWORK FOR UNDERSTANDING MEMORIZATION IN GENERATIVE MODELS
Brendan Leigh Ross, Hamidreza Kamkari, Tongzi Wu, Rasa Hosseinzadeh, Zhaoyan Liu, George Stein, Jesse C. Cresswell, Gabriel Loaiza-Ganem
Layer 6 Al
(brendan, hamid, tongzi, rasa, zhaoyan, aaaaoyan.george.... george,jesae, gabriee 0layer6.
ABSTRACT
As deep generative models have progressed, recent work has shown them be capable of memorizing and reproducing training datapoints when deployed. These findings call into question the usability of generative models, especially in light of the legal and privacy risks brought about by memorization. To better understand this phenomenon, we propose the manifold memorization hypothesis (MMH), geometric framework which leverages the manifold hypothesis into a clear language which to reason about memorization. We propose to analyze memorization in terms of the relationship between the dimensionalities of (i) the ground truth data manifold and (ii) the manifold learned by the model. This framework provides formal standard for "how memorized" datapoint is and systematically categorizes memorized data into two types: memorization driven by overfitting and memorization driven by the underlying data distribution. By analyzing prior work in the context of the MMH, we explain and unify assorted observations in the literature. We empirically validate the MMH using synthetic data and image datasets up to the scale of Stable Diffusion, developing new tools for detecting and preventing generation of memorized samples the process.
INTRODUCTION
Suppose {=r:)==1 is dataset in TR. drawn independently from a ground truth probability distribution (x.).. A deep generative model (DGM) a probability distribution (a))) designed to capture p. (a) only from knowledge of {a, DGMs, and most famously, diffusion models
cct al. 2015 Ho et al., 2020). have led the "generative Al" boom with their ability to generate realistic images from text prompts (Karras al. 2019 Rombach et al. 2022). DMs are thus likely to be deployed in an increasing number of public-facing or safety-critical applications. However, with sufficient model capacity, DGMs known to memorize some of their training data. Memorization occurs at various degrees of specificity, including identities of brands, layouts of specific scenes, or
exact copies of images (Webster et al. 2021 Somepalli et al. 2023a Carlini et al. 2023).
Memorization is undesirable for myriad reasons. Simply put, the more a model reproduces its training data, the less useful it becomes. Memorization is a modelling failure under the DGM definition provided above; if the underlying ground truth p. (z) does not place positive probability mass on individual datapoints, then a (1( that memorizes any datapoint must be failing to generalize (Yoon) et al. 2023). But memorization 's risks go beyond mere utility. Training datasets may contain private information which, if memorized, might be exposed in downstream applications. Copyright law includes "substantial similarity" between generated and training data as criterion in its definition of infringement, meaning that reproduced training samples can open up model builders or users to legal liability. For instance, the recent legal decision by Orrick (2023) hinged on this criterion.
The increasing dependence of society on generative models and resulting risks call for work to better understand memorization. Recent empirical work has identified mechanistic causes of memorization including but not limited to data complexity. duplication of training points, and highly specific labels (Somepalli et al.) 2023b; Gu et al. 2023). We group these insights under the umbrella of "memorization phenomena" a catch-all term for the various interesting neeoriiattoonrellan observations we would like to understand better. Though useful in practice, these memorization

---

Published as a conference paper at ICLR 2025
(a) OD-Mem with LID,
OD-Mem with LID, (x)
memorization with LID, (ar)
DD-Menn with LIDa (x)
(e) Mem with LIDa
Poorly lit model. LIDs irrelvaal
Figure 1: An illustrative example of LID values for models with different quality of fit and degrees of memoriza- tion. In these plots, ground truth manifold M. depicted in light blue, training samples {2. M. are depicted as crosses, and the model manifolds Ms are depicted in red. In (a) and (d), the model assigns O-dimensional point masses around the three leftmost datapoints, indicating that it will reproduce them directly test time; however in the former case this is caused by overfitting (IIID((x) < LIDD, (x)),, while in the latter case it is caused by the ground truth data having small L.I.. The models in (b) and (e) analoguous to (a) a 1-dimensional
(b), respectively. and still memorize, but with an extra degree of freedom the submanifold containing the three points. Only the model in (c). which has learned a 2-dimensional manifold through its full support. has generalized well enough and has learned manifold of high enough dimension to avoid both types of memorization. Finally, (f) shows poorly fit model where LID and memorization are not mcaningfully related.
Figure images along a relatively low-dimensional manifold learned by Stable Diffusion v1.5. The a real image from LAION (flagged as memorized by Webster 2023)), and the remainder were generated by the model.
phenomena have yet to be unified and interpreted under single theoretical framework. Mcanwhile, formal treatments of memorization have led to isolated usecases such as detection (Mechan et al.) model level (Vyas et al.} (023), but have provided little explanatory power for memorization phenomena. In addition to providing theoretical insights, a unifying framework could yield more capabilities such identifying whether training image has been memorized, altering the sampling process to reduce memorization, and detecting memorized generations post hoc.
2020. Bhattacharjee al.) 2023) and prevention
In this work, we introduce the manifold memorization hypothesis (MMH), a geometric framework to explain memorization. In short, we propose that memorization occurs at a point z E R2 when the manifold learned by the generative model contains z but has too small dimensionality at æ. As we will see, this understudied perspective is natural take on memorization that leads to practical insights and effectively explains memorization phenomena like those mentioned above. Although we mainly focus on DMs, the most notorious memorizers, our geometric framework applies to any DGM continuous data space R4; indeed, we empirically validate it on generative adversarial networks (GANs: Goodfellow et al. 20144 Karras et al. 2019) as well. Pidstrigach (2022) was the first show that DMs capable of learning low-dimensional structure in R and that this manifold learning capability is a driver of memorization; this sense, our work extends this connection into a general framework, grounds it in empirical findings, and connects it recent work memorization.
2

---

Published as a conference paper at ICLR 2025
This paper is organized according to the following contributions.
1. We advance the MMH in /Section 2 After defining the key notions of the data manifold and local intrinsic dimension (LID), we describe how LIDs correspond to memorization.
2. We demonstrate the explanatory power of the MMH in [Section by grounding it in prior observations about the behaviour of models that memorize. As this section will show, memorization phenomena observed in past work can be predicted and explained by the MMH.
3. In Subsection 4.1 we empirically test the MMH, showing that it both accurately describes reality and is useful in practice. As predicted by the MMH, estimates of LID are strongly predictive of memorization scales ranging from 2-dimensional synthetic data to Stable Diffusion (Rombach et al., (022).
4. Finally, inspired by the MMH, in Subsection 4.2 we devise scalable approaches to avert memorization during sampling from Stable Diffusion and to identify tokens in the text conditioning that contribute to memorization.
UNDERSTANDING MEMORIZATION THROUGH LID
Preliminaries Here we presume the manifold hypothesis: that data of interest lies on a manifold M Rd (Bengio et al. 2013). We take a generalized definition of manifold in which M is allowed to have different dimensionalities in different regions which is appropriate for realistic, heterogeneous data with varying degrees of structure and complexity. In particular, we assume that both our ground truth distribution (r) and our model (²) produce samples on manifolds, which we refer to as Mr and Ma respectively. We direct readers to Ssaiza-Ganen et al. (2024) for a justification and formal mathematical treatment of both of these assumptions, which are especially valid when the data is high-dimensional and the models are high-performing ones such as DMs and GANs.
Our framework for understanding memorization revolves around the notion of a point's local intrinsic dimension (LID). Given a manifold M and a point x E M,, we define the LID of x. LID(z), with respect to M as the dimensionality of M at æ. In this work, we will mainly consider the LIDs of points x E Idd with respect to two specific manifolds: M. and Mg. We will refer to these quantities as LID (z) and LZD, respectively.
Intuition and the Manifold Hypothesis Before discussing our framework, we review some intuition relating the manifold hypothesis to practical datasets. Manifold structure M C Rd arises from sets of constraints. These can range from very simple, like set of linear constraints (M = {i Az = b)), to highly complex (M = {a x is an image of face)). Locally at a point x E M, each constraint determines a direction one cannot move without leaving the manifold and violating the structure of the dataset ? Hence, a region governed by independent and active constraints will have dimensionality LID((r) (ar) d d f. The value of LID(z) (x: can be intuited as the number of degrees of freedom - valid independent directions of movement in which the characteristics of the dataset are preserved. Another connection is to complexity. For example, estimates of LID from algorithms like FLIPD (ammaari et al. 2024b) or the normal bundle (NB) method of Stanczuk et al. (2024) (which use in experiments; see Appendix B for details) have been shown to correspond closely with the complexity of an image; it is reasonable to expect that images with more complex features can endure more changes (such as morphing, moving, or changing the colours of different parts of the image) without losing coherence. The notions of constraints, degrees of freedom, and complexity along with their relationship to LID will help us understand its connection to memorization later sections.
Geometric Framework for Understanding Memorization In this section we formulate a framework for understanding memorization based on comparisons between LIDx(z) and (I). As a motivating example, consider Figure 1. which depicts six possible models po (o(x) trained on datasets {x, in that each lie on ground truth manifold M.. In the first scenario, Figure la the
Moos authors detine manifold to have constant dimension over the entire set. Under this common definition, our assumption is referred to as the union of manifolds hypothesis (Brown et al. 2023). We more general definition manifold for brevity.
"This statement is captured formally by the regular level set theorem of differential geometry, and manifolds be modelled such (Lee) 20T2 Ross et al. 2023
3

---

Published as a conference paper at ICLR 2025
model (2) has precisely memorized some of the training data. This is well-understood mode of memorization; training datapoints are exactly reproduced. To achieve this, the model has learned a O-dimensional manifold around these datapoints. To our knowledge, [Pidstrigach| (2022) to point out that a model capable of learning O-dimensional manifolds can memorize the training data. From this example, infer that can be perfectly reproduced when LIIx (x) 0. This indicates suboptimality in the model at the datapoints shown, for which LID.(r) = 2.
the first
However, memorization can be more complex than simply reproducing a datapoint. For example, Somepalli et al. (2023a) identify instances where layouts, styles, or foreground or background objects in training images are copied without copying the entire image, a phenomenon they refer to as reconstructive memory. WWbbster (2023) surfaces more instances of the same phenomenon and refers to them as template verbatims. See Figure 2 for an example. In the region of these points æ E Me. the model is able to generate images with degrees of freedom in some attributes (e.g., colour or texture), but is too constrained in other attributes (e.g.. layout, style, or content). Geometrically, Ma is too constrained compared to the idealized ground truth manifold M.; i.e., LID&(x) < LI). We depict this situation in Figure lb, wherein the model has erroneously assigned LID (æ) = 1 for some of the training datapoints.
Two Types of Memorization We expect two types of memorization to be of interest. An academic interested in designing DGMs that learn the ground truth distribution correctly will chiefly be interested in avoiding the memorization scenario (I) < LID. We refer to this first scenario as overfitting-dri driven memorization (OD-Mem). This situation represents modelling failure in that (e(r) is not generalizing correctly to p+ (z)) and is illustrated in Figure Ia and Figure 1b
However, an industry practitioner deploying consumer-facing model might be more interested in hypothetical values of LIDa per se, irrespective of the values of LID.. For any points & E .M. containing trademarked or private information, low values of (ID will be of concern even LID,(x)= = LID, , (z), as this information is likely to be revealed in samples generated from this region. practitioner would rightly refer to this situation as memorization despite the model generalizing correctly. We refer to this second scenario as data-driven memorization (DD-Mem) and illustrate it in Figurre and and FFigure Ie] This certainly happens in practice; for example, conditioning on the title of specific artwork (e.g. "The Great Wave off Kanagawa" by Katsushika Hokusai (Somepalli et al., 2023ab) is a very strong constraint, leaving few degrees of freedom in the ground truth manifold M., but reproducing specific artworks may be undesirable in production model. Unlike OD-Mem, DD-Mem is not overfitting in the classical sense, and a notable consequence is that it cannot be detected by comparing training and test likelihoods. We refer the conceptualization of how LIDs relate to memorization through OD-Mem and DD-Mem as the manifold memorization hypothesis.
No memorization is present in Figure 1c. in which the model manifold Mo matches the ground truth manifold M, and both have sufficiently high LID. We highlight that the MMH assumes high- performing models whose manifold Ma is roughly aligned with the data manifold M.; when this is not the case, as in Figure If
LID, and its relationship to LID, become irrelevant to memorization.
Why is the MMH Useful? The MMH is a hypothesis about how memorization occurs in practice for high-dimensional data. Its utility is best framed in contrast to past treatments of memorization. First, while past theoretical frameworks for memorization have focused on probability mass, our geometric perspective leads more practical tools. For example, Bhattacharjee et al. (2023) propose a purely probabilistic definition of memorization that can be detected only with access to the training dataset and the ability to generate large numbers of samples, which are intractable requirements the scale of LAION-2B (Schuhmann et al. 2022) and Stable Diffusion. In contrast, the MMH suggests that memorization can be detected through LID,(x), for which tractable estimators exist at scale. We explore these estimators in /ection 4
Second, the MMH explains and quantifies the phenomenon depicted in Figure 2 reconstructive memorization. While it has been studied in the past (Somepalli et al.) 2023a Webster 2023) Wen ct al., 2023), it has been resistant to theoretical explanation in part because past work has defined memorization based on distance to the memorized training point (see Apeendppx for more discussion on definitions). It is clear from Figure 2 that distance cannot capture reconstructive memorization; the training datapoint on the left is far in pixel space from the Stable Diffusion Diffusion-generatte generated samples to its right. Our framework overcomes this challenge by interpreting memorization in relation to the model and data manifolds without reference to distances or any specific training datapoint.
4

---

Published as a conference paper at ICLR 2025
Third, the MMH distinguishes between OD-Mem and DD-Mem, while past analyses have not. Bhattacharjee et al. (2023) would allow for OD-Mem but not DD-Mem under their definition of memorization, while empirical work tends to ignore the effect of p. (a) on Po(z), thus subsuming both OD-Mem and DID-Mem in spirit if not formally (Carlini et al.) 2023 Yoon et al.) 2023 Gu et at... 2023). For further details, please see [Appendix A] where formally develop the relationship
between the MMH and definitions of memorization in related work.
Defining and distinguishing between OD-Mem and DD-Mem suggests immediate solutions to each. OD-Mem is overfitting and can be addressed accordingly, such as by collecting more data or improving model's inductive biases. On the other hand, DD-Mem indicates that the training distribution p. (x) does not actually match the desired distribution at inference time, and hence is a misalignment of data and objectives. can be addressed by changing p. (x) itself, such as by altering the data collection, cleaning, and augmentation procedures. We explore this point further in Section
3| Unlike OD-Mem, DD-Mem cannot be addressed by improving the model to generalize better. Both OD-Mem and DD-Mem can also in principle be addressed by augmenting pe(x) to generate higher-LID samples. In Section 4] we propose solutions to alter the data-generating process with precisely this goal.
EXPLAINING MEMORIZATION PHENOMENA
In this section, we demonstrate the explanatory power of the MMH by showing how it explains memorization phenomena in related work. In the process, this section demonstrates two advantages of our geometric framework. First, it provides unifying perspective on seemingly disparate observations throughout the literature (nevertheless, this not meant as a related work section - for that see Section 5). Second, the MMH links memorization to the rich theoretical toolboxes of measure theory and geometry, which we use in this section to establish formal connections to past work. Propositions, theorems, and proof's in this section are presented informally for clarity. For full theorem statements and proofs, please see Appendix E
Duplicated Data and LID It has been broadly observed that memorization occurs when train- ing points are duplicated (Nichol et al.] 22022; Carlini et al., [202]] Somepalli et al.] (023a). In Proposition 3.1] we show that duplicated datapoints lead to DD-Mem; duplicated points 20 indicate LID. = 0, so even a correctly fitted model will have LID,, (zr) = 0 (as in Figure Id).
Proposition 3.1 (Informal). Let {- be a training dataset drawn independently from (x().. Under some regularity conditions, the following hold:
1. If duplicates occur in {x.] with positive probability, then they occur point zo such that LID, (xo) = O.
2. If (xo) = 0 and n is sufficiently large, then duplication will occur in {x with near-certainty.
Proof. See Appendix E for the formal statement of the theorem and proof. To understand both conditions, it suffices note first that duplicate samples are intuitively equivalent to p- (x) assigning positive probability to point. Under mild regularity conditions on the nature of the (x) and M.. positive probability a point is equivalent to a O-dimensional manifold that point.
From this result, we gather that improving model generalization is not the solution to duplication. Instead, one may need to add inductive biases that prevent pa ps(x) from learning O-dimensional points. Of course, the more straightforward path is to change the data distribution (x) by de-duplicating the training dataset. We carry the same intuition forward to "near- -near-duplicated content", where similar but non-identical points occur together in the dataset, in which case LID, would be low but nonzero in the region of the near-duplicated content (as in Figure Tep.
Conditioning and LID Somepalli et al. 2023bb and Yoon et al. (023) observe that conditioning on highly specific prompts encourages the generation of memorized samples. Here, we point out that conditioning decreases LID, making models more likely to generate memorized samples.
Proposition 3.2 (Informal). Let By E M., and let us denote by LIDx I c) the LID of an with respect to the support of the conditional distribution px(x | c). We then have
(No | c) S (vo)
(1)
5

---

Published as a conference paper at ICLR 2025
Proof. See Appenddx Eooor the formal statement of the theorem and proof. Intuitively, conditioning can be interpreted as adding additional constraints to M,. which cannot increase its dimension.
Conditioning on highly specific can be linked to both DD-Mem and OD-Mem. Introducing strong constraints greatly decreases LID,, leading to DD-Mem. However, a relatively low number of training examples satisfy c, the model could overfit, leading to OD-Mem as well.
Complexity and LID For images, Somepalli et al. (2023b) also highlight low complexity as a factor causing memorization. Using the understanding that LID corresponds to complexity discussed in Section 2] we infer that low-complexity datapoints x E M. have low (I). This fact suggests that, like with duplication, memorization of low-complexity datapoints an example of DD-Mem.
The Classifier-Free Guidance Norm and LID Classifier-free guidance (CFG) is way to improve the quality of conditional generation in DMs. Whereas standard conditional generation employs the score function dy (z; t, c), which refers to a neural estimate at time of the conditional score, CFG increases the strength of conditioning by using the following modified score:
90
Memorized
(x cll (x10-1)
80-
Non-Memorized
70
50
50
40
(ectto) c) so(x;t, 0) M(solx;t, c) sa 0)),
30
CPG-adjnsted soore
CRG vector
20
8
10 12
where A is hyperparameter for "guidance strength" and
CPG vector norm (x10-7)
so s0 t, 0) refers to conditioning on the empty string (here we formulate DMs using stochastic differential equations (Song et al. 2021)).
Figure CFGG-ajjusted scores CRG vectors for Stuble Diffusion with = 7.5 and = 0.02 on 20 memorized and non-memorized images from LAION.
Wen et al. (2023) identify that specific conditioning inputs lead to memorized samples when the CFG vector has large magnitude. We explain this observation using the MMH as follows. First, we observe that a large CFG magnitude will generally result in a large magnitude of the CFG-adjusted score 80 (1;; 1, c). We demonstrate this empirically in Figure 3 Furthermore, it is understood in the
literature that large | (x; t, c)ll, and its explosion as + 0, is common for high-dimensional 2021) and is necessary to generate samples from low-dimensional manifolds (Pidstrigach) 2022 Lu et al. 2023). It has been empirically observed that this explosion occurs faster as the dimensionality gap increases between the data manifold and the ambient data space Loaiza-Ganen et al.) 2024), which is one reason that generative modelling lower- diver-dimensional (z;; latent space tends to improves performance (Loaiza-Ganen et al. 2022). The largest (1; c) values should thus generate points with the largest dimensionality difference from TR::; i.e., points with the smallest LID c). P Hence we infer that reducing the CFG-adjusted score norm or cquivalently the CFG vector norm should increase (x c) and lessen memorization, a fact confirmed empirically by Wenn et al. 2023). Since this phenomenon corresponds to any æ with small LID,(x c). it can indicate both OD-Mem and DD-Mem under the MMH.
data (Vahdat et al.
EXPERIMENTS
4.1 VERIFYING THE MANIFOLD MEMORIZATION HYPOTHESIS
In this section, we empirically verify the geometric framework which underpins the MMH. We analyze both LID, and LID, to study DD-Mem and OD-Mem. Several algorithms exist to estimate LIDx(x:) for diffusion models, including the normal bundle (NB) method (Stanczuk et al. 2024), and more recently FLIPD (Kamkari et al. 2024b). For GANs, we approximate LID,(x) of generated data by computing the rank of the Jacobian of the generator. Additionally, we use LPCA (Fukunaga & Olsen| 197I) to estimate LID, where applicable; see Appendix B for details on these methods and Appendix C for their hyperparameter configurations. In general LID, and LID. are unknown quantities that are approximated with the aforementioned estimators, throughout this section we write their respective estimates as LID, and LID..
'Here we understand LIDa(1 c) as the LID with respect to the support of the conditional distribution being sampled from by $ (a;; c).

---

Published as a conference paper at ICLR 2025
Diffusion Model von Mises Mixture In an illustrative exper- iment, we study a mixture of von Mises distribution, which sits on 1-dimensional circle, and O-dimensional point mass the origin in 2-dimensional amhient space, as depicted in Figure 4] every point M. has either LID. (x) = 0 or L.ID. (ar) = =1 From this distri- bution we sample 100 training points, and by chance a single point Fg sits isolated in low-density region of the circle. Next, we train DM on this data. In Figure 4 we depict 100 generated samples, colour-coded by their LID estimates, as estimated by FLIPD. Here, we see OD-Mem and DD-Mem in action: the model overfits at 2o. producing near-exact copies, with ( z (ro) < LID.(zo) = (OD-Mem). The model also faithfully produces copies of the circle's center, but this is caused by low ground truth LIDs (DD-Mem), not modelling error.
1.0
30
0.4
1.0
0.8
0.4
CIFARIO Memorization We analyze the higher-dimensional CI- FARI0 dataset (Krizhevsky Hinton 2009) and use two pre- trained generative models: iDDPM (Nichol & Dhariwal 2021) and StyleGAN2-ADD (Karras et al. 2020). We generate 50,000 images from each model, and for each, identify the most similar training image according to two distance metrics, (t) SSCD distance (Pizzi
0.0
Figure 4: Training diffusion model von Mises mixture. (Top) Ground truth manifold and the associated distribation. (Bottom) Medel-generated samples with their extimates.
al. 2022) and (ii) calibrated le distance (Carlini al. 2023). By
thresholding on these metrics, we arrive at a small subset of potentially memorized examples, which we manually label as either exactly memorized, reconstructively memorized (Somepalli al. 2023a), or not memorized. All other images are not labelled and have a low chance of being memorized. Further details and images deemed memorized are reported Appendix F] The first two panels in Figure 5a show our labels distinguish different types of memorization as we display the generated images vs. the closest SSCD) match in the training dataset.
Next, we estimate LID, for each iDDPM and StyleGAN2- ADA sample. For iDDPM, we use the NB estimator, Figure 5b Figur show that LID, is generally smaller for memorized images compared monooreed ones. As shown in Figure 5d LID, is considerably lower for exact memorization cases within the training dataset, suggesting that exact memorization for both models corresponds to DD-Mem. We also observe that in Figure 5b) reconstructively memorized samples exhibit lower values of LIDg, as compared to not memorized samples, despite the corresponding training data having comparable LID., (Figure 5dy: the LID, estimates enable us to still classify these samples as memorized, showing clear example of detecting OD-Mem.
We have shown that LID estimates are effective at detecting both OD-Mem and DD-Mem, supporting the MMH hypothesis. However, while simpler images tend be memorized more frequently, they are not always memorized (see Figure 5a] right panel), leading to some overlap in estimated LID, This overlap occurs
between memorized and not memorized samples in Figure 5b and Figure 5c
DSPLEC
(Teff) Fract and (Middle) recnstractively manovized samples (Top) with their matched CIFAR detapaints (Romoon). (Right) Non-menorized samples with low L.D, showing LID, be pantillly confounded by complexity.
51
Memorization
0.0
Not Memmriired
Reconsta uctive Man.
3.0
0.0
Detaiity
Not labelled
Denslty
0.0-
0.
0.0
3.0
30
60
100
3.-
200
220
300
250
400
-10
10
30
50
(b) Ting for StyleGAN samples.
T.m0g for IDDPM samples.
(4) T.i5_ for CIFARIO datapaints.
Figure Visualizing OD Mem and DD Mem on StyleGANZ-Al and iDDPM Irained CIFAR10.
7

---

Published as a conference paper at ICLR 2025
0.3
3.0
0.2
LAION-Aestbetics
1.0
0.2
0.2
LATON- Memoriadd
22.0
1.3-
0.8
0.1
1.0-
0.3
0.1
-15009- 10000-5000
5000 10000
5000 10000
0.0
20
40
60
(o) L.Dg
(b) LIDg e).
(e) CRG vector nona.
Figure 6: Density histograms for each memorization metric across different datnsets.
because image complexity serves as a confounding factor: images with simple backgrounds and textures may be assigned low LID, values, not due to memorization, but simply because of their inherent simplicity. We discuss this issue further, along with partial solution, in Appendix C.2
Stable Diffusion on Large-Scale Image Datasets Here, we set (3) to Stable Diffusion v1.5 (Rombach et al. 2022). Taking inspiration from the benchmark of Wen et al. (023). we retrieve memorized LAION (Schuhmann et al. 2022) training images identified by Webster (202)) We focus on the 86 memorized images categorized as "matching verbatim", noting that the other categories of Webster 2023) consist of large numbers of captions that generate samples matching small set of training images. For non-memoriae mom-meedd images, we use a mix of 2000 images sampled from LAION Aesthetics 6.5+, 2000 sampled from COCO (Lin al. 2014), and all 251 images from the Tuxemon dataset (uuxemon Project 2024 Hugging Face 2024
To knowledge, estimator of LIDD, scales images the size of Stable Diffusion; we thus omit these from our analysis. FLIPD is the only LIDa estimator that remains tractable at this scale, so we use for this analysis. Note that Stable Diffusion provides two model distributions: the unconditional distribution (1r and the conditional distribution (o(x c). where is the image's caption. Hence, compute both LIDs, and (- c) for each of the aforementioned images. Additionally, compute the norm of the CFG vector, which was proposed as a memorization detection method by Wen et al. (2023) and which we argued varies inversely LID, in Section 3 Our experiments thus cover three proxies for local intrinsic dimension: LID3, LIDx( c). and the CFG vector norm (see Appendix for details). The density histograms of all these values are depicted in Figure 6
We see that all proxies for LIDs, assign relatively small L.D values to memorized images, further validating the MMH. Due to the unavailability of LID, estimates, it is hard to distinguish between DD-Mem and OD-Mem here. In Figure 6, low conditional or unconditional LID as well as high CFG vector norms are all signals of memorization, strengthening vector norm seemingly provides the strongest signal, the unconditional LID detects memorization well despite lacking caption information. Detecting memorized training images without the corresponding captions is novel capability, and notably cannot be done with the CFG vector norm technique.
argument in Section 3 While the CFG
4.2 MITIGATING MEMORIZATION BY CONTROLLING LID,
In this section we study the problem of sample-time mitigation through the lens of the MMH. Somepalli et al. (2023b) establish text-conditioning as a crucial driver of memorization in Stable Diffusion, where specific tokens in the prompt often cause the model generate replicas of training images. Wen et al. (2023) introduce differentiable metric, which we denote as (CF (formally defined in Apppedixx Dj. which is based on the accumulated CFG vector norm while sampling an image. Wen et al. (2023) observe that this metric shows a sharp increase when the prompt e leads to the generation memorized images. Since ACRG (c) is differentiable with respect c. Wen al. (2023) backpropagate through this metric and find the tokens with the largest gradient magnitude, essentially providing token attributions for memorization.
Here we make two contributions. First, we propose two additional metrics, . CRG (c) and FLIPD (c). which modifications of ACFG (c) use || CPG (x; FLIPD, respectively, instead of the
'The LID estimates provided by FLIPD are sometimes negative in value; Kamkari et al. 2024b) justify this as an artifact of estimating the LID using UNet. Despite underestimating LID in absolute terms, Kamkari et al. (2024b) confirm that FL ranks L.D, estimates correctly, which is sufficient for the purpose of distinguishing memorized from momorzerd examples.
8

---

Published as a conference paper at ICLR 2025
AN
9.2
5571 unittetty soore
0.45
Randam
0.0
0.1
paliciig
kalsu 11
0.40
0.33
0.30
0.54
CLIP somee
0.68
0.72
Traiing Irage
Imrage
O Mijjationn
+ 0 Minguaion
60 Amilysis of the mitization appreach.
(b) Comparing (nommalized) e token altributions for memorized using three methods.
Figure 7: Using token attributions to detect drivers of memorization and to mirigate it at sample time.
norm of the CFG vector. We define these metries fully in Appendix D due space limitations. Since both of these new metrics are also differentiable with respect to c, A (c) can be trivially replaced by either of them in the method of Wen al.] (203). Second, we propose an automated way to use the token attributions from this method into sample- time mitigation scheme. We start by normalizing the attributions across the tokens, and sample & tokens hased on a categorical distribution parameterized by these normalized attributions. We then use GPT-4 (OpenAl] (023) to rephrase the caption, keeping it semantically similar but perturbing the selected tokens that are highly contributing to the memorization metric (see (sce/Appendix D.4 for details).
The bottom panel in Figure 7b shows four images: a training image corresponding to the prompt "The Great Wave off Kanagawa by Katsushika Hokusai", generated image using the same prompt showing clear memorization, a generated image obtained with our mitigation scheme with A A and another generated image using AFLIPD (c) instead. Qualitatively, using FLIPD the norm of the CFG vector perform on par with each other. The top panel of Figure 7h|shows the token attributions obtained from FLIPD are sensible. See Appendix D.5 for additional results.
We present quantitative comparisons in Figuure 7a by analyzing the average CLIP score (higher is better) (Radford al., 2021) and SSCD similarity (lower is better) over matching prompts, varying k E (1,2,3, 3,,,,,,,66,8 8, with 5 repetitions for each prompt. As k increases, both SSCD and CLIP scores decrease across methods. We also include an ablation where the modified tokens are selected uniformly at random, ignoring attributions. All attribution- basod methods achieve lower similarity while maintaining a relatively higher CL.IP score than uniform token selection.
Overall, the results in Figure 7 provide further evidence supporting the MMH, both by showing that encouraging samples have higher LID can help prevent memorization, and by further confirming the relationship between the CFG vector norm, the CFG-adjusted score norm, and LID established in [Section 3 We hypothesize that our results can likely be improved by more efficiently guiding generated samples towards regions of high LID, but highlight that doing is not trivial. For example, in Appendix D.3 we find that optimizing c for large values of (LLIPP(()) (inspired by the inference- time mitigation method of (Wen et al. 2023) during sampling can fail by producing samples with chaotic textures that have artificially high L.Dg
5 RELATED WORK
Detecting and Preventing Memorization for Image Models The task of surfacing memorized samples is well-studied. Consensus in the literature is that Ea distance to the nearest training sample in pixel space is a poor detector of memorized samples (Carlini et al., 2023), but that recalibrating the & distance according to the local concentration of the dataset works better for smaller datasets (Yoon et al. 2023 Stein et al. (002),, and that using retrieval techniques such as distance in SSCD feature space Pizzi et al. 2022) works better still, especially for more complex, higher-resolution images (Somepalli et al. 2023a). However, all of these retrieval techniques are too expensive to be used withhold samples from a live model. To more efficiently prevent memorized samples from being generated, past and concurrent works have altered the sampling procedure, training procedure, or the model itself (Wen al. 2023 Daras et al. 2024 Chen al. 2024 Hintersdorf et al.) 2024).

---

Published as a conference paper at ICLR 2025
Explaining Memorization There is an active community effort attempting explain why and how memorization occurs in DGMs. Early studies focused on GANs, and have taken both theoretical
NNgarajan et al.] 2018) and empirical (Bai et al. [02T perspectives. However, GANs are thought to
be less prone to memorization than DMs (Akbar et al. 2025), except on small datasets (Feng et al.
202T). Several works DMs (Pidstrigach) 2022 Yi et al. 2023 Gu et al. [023] Li al.
(024) have pointed out that, given sufficient capacity, DMs at optimality are capable of learning the empirical training distribution, which complete memorization. Others have focused on generalization,
showing that DMs are capable of generalizing well in theory (Li et al. 2023), have inductive biases
towards generating photorealistic images (Kadkhodaie et al.) 2024), and will generalize when their capacity is insufficient to memorize (Yoon et 2003).
DGM-Based LID Estimation As opposed to statistical LID estimators (e.g., Levina & Bickel (2004), ) which are constructed estimate the dimension of M.. DGM-based ones estimate the dimensionality of Ma. the manifold learned by DGM. These types of estimators are available for many types of DGMs, and in addition being useful for memorization, have found utility in
out-of-distribution detection (Kamkari et al. 2024a). In the literature, LID estimators for normalizing flows (Dinh et al. 2014) have been proposed using the singular values of their Jacobians (Horvat
Pfister 2022 Kamkari et al.) 2024a) or their density estimates (Tempezyk et al. 2022). In Section 4 we applied the singular value method to obtain LID estimates for GANs. Dai Wipt (2019) and
Zheng et al. (2022) proposed estimators for VAEs (Kingma & Welling 201 Rezende et al. 2014)
using the structure of their posterior distribution. Several authors have proposed estimators for DMs well (Stanczuk et al. [024|] Kamkari et al.] 2024b Horvat & Pfister 2024);; we focus those of
Stanczuk et al. 2024) and Kamkari et al. (2024b) because they work with off-the-shelf DMs and do not require modifying the training procedure.
6 CONCLUSIONS, LIMITATIONS, AND FUTURE WORK
Throughout this work, we have drawn connections between the geometry of a DGM and its propensity memorize through the MMH. First, we showed that the notion of LID provides a systematic way of understanding different types of memorization. Second, we explained how memorization phenomena described by prior work can be understood from the perspective of LID. Third, we verified the MMH empirically across scales of data and classes of models. Fourth, we showed that controlling LID, is a promising way to mitigate memorization. We offered several connections, including the insight that some instances of memorization in DMs are due to the DM's inability to generalize (OD-Mem), whereas others are due to low-LID ground truth (DD-Mem).
Despite having shown that the MMH is principled avenue to detect and alleviate memorization, our current approaches can be improved: estimates of LID, have some overlap between memorized and non-memorized samples and our sample- time scheme for mitigating memorization using FFIPP (c) performs on par with, but does not outperform, its more ad-hoc variant using (c). We expect future work to find even better ways of leveraging the MMH and LID towards these goals; e.g. by improving LID) estimation or more efficiently controlling LID during sampling. Finally, although the manifold hypothesis does not apply directly to discrete data such language, some intuitions described in this work carry over, and generalizations or parallels to the concepts here may offer insights for the modellgg space.
Reproducibility Statement To ensure the reproducibility of our experiments, provide two codebase links. The first codebase, accessible at Igggtttb www.ghu.cco/laaeeeeeeeeeeaeeeeee hotm/layerrai-labaaaaa gerlggmggemeerry/.o contains small-scale synthetic experiments and our CIFARIO experiments. The second, ac- cessible at biggttuub com/ httpaaaaaaaaaabsiiffffeeeee /ttp://iiffiissooommmeeeaaaaaaaa extends the work of Wen al. (2023) use the MMH detect and mitigate memorization. Comprehensive details of our experimental setup are provided across /Section 4 Appendix C and (pppendixx D All datasets used in our experiments are freely available from the referenced sources and are utilized in compliance with their respective licenses.
Ethics Statement We do not foresee any ethical concerns with this research. The overarching topic, memorization in generative models, is widely studied to better understand safety concerns associated with using and deploying such models. Our goal is to theoretically explain and to empirically detect and alleviate this phenomenon; we do not promote the use of these models for harmful practices.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We thank Kin Kwan Leung for his valuable help in proving Lemma E.4
REFERENCES
Muhammad Usman Akbar, Wuhao Wang, and Anders Eklund. Beware of diffusion models for synthesizing medical images-a comparison with gans in terms of memorizing brain mri and chest X-ray images. Machine Learning: Science and Technology, 2025.
Ching- Ching-Yuan Bai. Hsuan-Tiee Tien Lin, Colin Raffel, and Wendy Chi-wen Kan. On training sample memorization: Lessons from benchmarking generative modeling with a large-scale competition. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. pp. 2534-2542, 2021.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798-1828, 2013.
Robi Bhattacharrje, Sanjoy Dasgupta, and Kamalika Chaudhuri. Data-copying in generative models: a formal framework. In International Conference on Machine Learning. 2023.
G. Bradski. The OpenCV Library. Dr. Dobb's Journal of Software Tools, 2000.
Bradley CA Brown, Anthony L. Caterini, Brendan Leigh Ross, Jesse Cresswell, and Gabriel Loaiza-Ganem. Verifying the union of manifolds hypothesis for image data. In International Conference on Learning Representations, 2023.
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. The Eleventh International Conference on Learning Representations, 2022.
Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium, pp. 5253-5270, 2023.
Chen Chen, Daochang Liu, and Chang Xu. Towards memorization-free diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Bin Dai and David Wipf. Diagnosing and enhancing VAE models. In International Conference on Learning Representations, 2019.
Giannis Daras, Alex Dimakis, and Constantinos Costis Daskalakis. Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data. International Conference on Machine Learning, 2024.
Laurent Dinh, David Krueger, and Yoshua Bengio. NICE: Non-linear independent components estimation. arXiv:1410.8511 2014.
Qianli Feng, Chengi Guo, Fabian Benitez- Quiroz, and Aleix M Martinez. When do GANs replicate? on the choice of dataset size. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6701-6710, 2021.
G.B. Folland. Real Analysis: Modern Techniques and Their Applications. Wiley, 2013.
Keinosuke Fukunaga and David R Olsen. An algorithm for finding intrinsic dimensionality of data. IEEE Transactions on Computers, 100(2):176-183, 1971.
Ian Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville. and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, volume 27, 2014.
Xiangming Gu, Chao Du, Tianyu Pang, Chongxuan Li, Min Lin, and Ye Wang. On memorization in diffusion models. arXiv:2310.00 12.02664, 2023.
11

---

Published as a conference paper at ICLR 2025
Dominik Hintersdorf, Lukas Struppek, Kristian Kersting, Adam Dziedzic, and Franziska Boenisch. Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models. arXiv:2406.02366, 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilisstic models. In Advances in Neural Information Processing Systems, volume 33, 2020.
Christian Horvat and Jean-Pascal Pfister. Intrinsic dimensionality estimation using normalizing flows. In Advances in Neural Information Processing Systems, 2022.
Christian Horvat and Jean-Paschs Pfister. On gauge freedom. conservativity and intrinsic dimen- sionality estimation in diffusion models. The Twelfth International Conference on Learning Representations, 2024.
Hugging Face. Tuxemon. http:///wuggnnggee...ooaaatett//
2024.
Ahmed Imtiaz Humayun, Ibtihel Amara, Candice Schumann, Golnoosh Farnadi, Negar Rostamzadeh, and Mohammad Havaci. Understanding the local geometry of generative model manifolds. arXiv e-prints, pp. arXiv -408, 2024.
Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian and Computation, 1803):1099-106,
smoothing splines. Communications 1989.
Zahra Kadkhodaie, Florentin Guth, Eero P Simoncelli, and Stéphane Mallat. Generalization in diffu- sion models arises from addaetry- harmonic representation. In International Conference on Learning Representations, 2024.
Hamidreza Kamkari, Brendan Leigh Ross, Jesse C Cresswell, Anthony L Caterini, Rahul G Krishnan, and Gabriel Dpaiza-Ganent A geometric explanation of the likelihood OOD detection paradox. In International Conference Machine Learning, 2024a.
Hamidreza Kamkari, Brendan Leigh Ross, Rasa Hosseinzadeh, Jesse C Cresswell, and Gabriel Loaiza- Ganem. A geometric view of data complexity: Efficient local intrinsic dimension estimation with diffusion models. In Advances in Neural Information Processing Systems, volume 37, 2024b.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 4401-4410, 2019.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Advances Neural Information Processing Systems, volume 33, 2020.
Diederik P Kingma and Max Welling, Auto-encoding variational Bayes. In International Conference on Learning Representations, 2014.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report, 2009.
John M Lee. Introduction Smooth Manifolds. Springer New York, 2012.
Elizaveta Levina and Peter Bickel. Maximum likelihood estimation of intrinsic dimension. In Advances in Neural Information Processing Systems, 2004.
Puheng Li, Zhong Li, Huishuai Zhang, and Jiang Bian. On the generalization properties of diffusion models. Advances in Neural Information Processing Systems, 36, 2023.
Sixu Li. Shi Chen, and Qin Li. A good score does not lead to good generative model. arXiv:2401.04856 2024.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision- ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-1 2014, Proceedings. Part V 13, pp. 740-755. Springer, 2014.
12

---

Published as a conference paper at ICLR 2025
Gabriel Loaiza- Ganem, Brendan Leigh Ross, Jesse C Cresswell, and Anthony L. Caterini. Diagnosing and fixing manifold overfitting in deep generative models. Transactions on Machine Learning Research, 2022.
Gabriel Ganiia Brendan Leigh Ross, Rasa Hosseinzadeh, Authony Caterini, and Jesse C Cresswell. Deep generative models through the lens of the manifold hypothesis: A survey and new connections. Transactions on Machine Learning Research, 2024.
Yubin Lu, Zhongjian Wang, and Guillaume Bal. Mathematical analysis of singularities in the diffusion model under the submanifold assumption. wrXiv:2301.07892 2023.
Casey Meehan, Kamalika Chaudhuri, and Sanjoy Dasgupta. A non-parametric test to detect data- copying in generative models. In International Conference on Artificial Intelligence and Statistics, 2020.
Vaishnavh Nagarajan, Colin Raffel, and Ian J Goodfellow. Theoretical insights into memorization in gans. In Neural Information Processing Systems Workshop, volume pp 3, 2018.
Alex Nichol, Aditya Ramesh, Pamela Mishkin, Prafulla Dariwal, Joanne Jang, and Mark
Chen. DALL-E 2 Pre-Training Mitigations, June 2022. URL https://openai.com combblgg/
dall-e-22 dall---2-pre- www.iiinnniiiiaatiiiooo mitigations/
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162-8171, PMLR, 2021.
OpenAL. Gpt-4. https:// 2023. GPT-4o.
William H. Orrick. Andersen v. Stability Al Ltd., 2023. URL httpp::/caseeext.ccmm/ case/
anderse----dtabiity-------l
Jakiw Pidstrigach. Score-based generative models detect manifolds. In Advances in Neural Informa- tion Processing Systems, 2022.
Ed Pizzi, Sreya Dutta Roy, Sugosh Nagavara Ravindra, Priya Goyal, and Matthijs Douze. A self- supervised descriptor for image copy detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14542, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine Learning, pp. 1278-1286 2014.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusson models. In Proceedings of the IEEE/CVF Confer- on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Brendan Leigh Ross, Gabriel Loaiza Gaaza, Anthony L Caterini, and Jesse Cresswell. Neural implicit manifold learning for topology-awer generative modelling. Transactions Machine Learning Rexearch, 2023.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski. Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION-SB: An open large-scale dataset for training next generation image-text models. In Advances in Neural Information Processing Systems, volume 35, 2022.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International Conference on Machine Learning, pp. 2256-2265, 2015.
13

---

Published as a conference paper at ICLR 2025
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication diffusion models. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023a.
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Understand- ing and mitigating copying in diffusion models. In Advances in Neural Information Processing Systems, volume 36, 2023b.
Jiaming Song. Chenlin Meng. and Stefano Ermon. Denoising diffusion implicit models. arXiv:2010.02502, October 2020.
btttpp://arxxw.orr//bbb.e
02502
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference Learning Representations, 2021.
Jan Pawel Stanezuk, Georgios Batzolis. Teo Deveney, and Carola-Bibiane Schönlieb. Diffusion models encode the intrinsic dimension of data manifolds. In Proceedings of the 41st International Conference on Machine Learning. 2024.
George Stein, Jesse C Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L Caterini, J Eric T Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. In Advances in Neural Information Processing Systems, volume 36, 2023.
Piotr Tempezyk, Rafal Michaluk, Lukasz Garnearek, Przemyslaw Spurek, Jacek Tabor, and Adam Golinski. LIDL: Local intrinsic dimension estimation using approximate likelihood. In Interna- tional Conference on Machine Learning, pp. 21205-21231, 2022.
Tuxemon Project. Tuxemon, 2024. URL. htt.ttttt///tueoonn. org/
Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. In Advances in Neural Information Processing Systems, volume 34, 2021.
Nikhil Vyas, Sham M Kakade, and Boaz Barak. On provable copyright protection for generative models. In International Conference on Machine Learning, pp. 35277-35299, PMLR, 2023.
Ryan Webster. A reproducible extraction of training images from diffusion models. arXhv:2305.08664, 2023.
Ryan Webster, Julien Rabin, Loic Simon, and Frederic Jurie. This person (probably) exists. Identity membership attacks against GAN generated faces. 1xXiv:2107.06018, 2021.
Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan Lyu. Detecting, explaining, and mitigating memo- rization in diffusion models. In The Twelfth International Conference on Learning Representations, 2023.
Mingyang Yi, Jiacheng Sun, and Zhenguo Li. On the generalization of diffusion model. arXiv:2305.14712. 2023.
TаeHo Yoon, Joo Young Choi, Sehyun Kwon, and Emest K Ryu. Diffusion probabilistic models generalize when they fail to memorize. In ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling, 2023.
Yijia Zheng, Tong He, Yixuan Qiu, and David P Wipf. Learning manifold dimensions with conditional variational autoencoders. In Advances in Neural Information Processing Systems, volume 35, 2022.
14

---

Published as a conference paper at ICLR 2025
A CONTEXTUALIZING THE MMH WITHIN DEFINITIONS OF MEMORIZATION
An Overview of Definitions The MMH describes the mechanism through which memorization occurs. How does this mechanism fit into prior definitions of memorization from the literature?
Formal definitions of memorization generally follow the same template: a point 2 memorized
when the model's ' probability measure Po places too much mass within some distance of zg. Some of
these definitions define memorization globally on the level of an entire model (Meehan ct al. [020]
Yoon et al. 2023 Gu et al. 2023), while others define memorization locally for individual datapoints (Carlini et al. [2023] Bhattacharjee et al. 2023). The identical definitions of Yoon et al. (0223) and
Gu et al. (2023) consider point to be memorized based purely distance threshold; in practice, however, distances alone have been unsuccessful at consistently surfacing what would be perceived
by humans as memorized (Somepalli et al. 2023a Stein al. (023). We postulate this is also due to manifold structure; semantically memorized images such as Figure 2 will sit on the same manifold, but may not necessarily be close each other as measured by distance, even when taken in the latent space of an encoder. Meanwhile, Carlini et al. (2023) take a privacy perspective; their definition considers images memorized if they can be extracted from model by any means, not just generated by the model. In this work we take the perspective that memorized samples are most likely to be problematic when they are generated by production model, which are often treated as a black-box, so we focus on generation rather than extraction.
Links Between Formal Memorization and the MMH
For the reasons above, we use here the definition of memorization by Bhattacharjee al. (023). who define a point xp memorized by comparing Pe to the ground truth P, in a neighbourhood of zo- We present their definition here:
Definition A.1. Let P. and Pe be the ground truth and model probability measures, respectively. Let A > 1 and 0 < y < 1. A point x E Id is a (A, 7)-copy of a training datapoint To if there exists radius
0 such that the d-dimensional ball (Tn) of radius r centred Zo satisfies (i) x E (x (ii) Po (B 2 AP. (B and (iit) P. (. 5-
The first and third conditions imply that x is sufficiently close to To relative to the amount of probability mass in the region (P.(B, (x)))) while the second condition implies that the model Po places much more mass in the region compared to the ground truth P.. A natural question about the MMH is whether points satisfying it are also formally memorized by the above definition. The answer is in the negative for DD-Mem (Proposition A.2) and the affirmative for OD-Mem Theorem A.3).
Proposition A.2. There exist models (o(x) that exhibit DD-Mem at Ty E Rdd, but do not generate (X, copppisss of zo-
Proof. Choose any ground truth distribution P, manifold M. with a low-LID point m0 E M. (for example, set LID. (zo) = 0). A perfect model will exhibit DD-Mem at To. but for any ball Bd (xo) containing zo. Pe (B) (2o)) P. (zo)). violating the second condition of Definition A.I
Since DD-Mem is a consequence of the data distribution having points an with inherently low (ID,, these memorized points are likely to be generated even when there is excess probability mass assigned near xo by Po. as required in the definition of (A, 70-coiiss.
Theorem A.3 (Informal). Suppose To E RRd is such that (x) exhibits OD-Mem at To- Then, for every X > 1 and 0 < 7 < 1, (0() will generate (X, ccoopies of zo with near-certainty.
Proof. See Appendix E for the formal statement of the theorem and proof.
The MMH thus provides two important pieces of context for the definition of (A, -))coiiss. The first is that Definition A. is in some sense incomplete; it does not cover DD-Mem. The second is that OD-Mem can be considered a useful refinement of Definition A.I While the algorithm given by Bhattacharjee al. (2023) is intractable at scale, we show in Section 4 that the added strength (in the mathematical sense) of the MMH allows us to flag memorized data more efficiently using only estimators of LID.
15

---

Published as a conference paper at ICLR 2025
B LID ESTIMATION
B.1 LID ESTIMATION WITH DIFFUSION MODELS
As mentioned in the main manuscript, we follow the SDE framework of Song et al. (2021) for DMs, where the so-called forward SDE is given by
dær = f( 1)dt + g())dW,, 20 =o))
where f: Rdd x [0, 1] - Rd and g : [0, R pre-specified functions and Wr is Brownian
motion on Rdd This process progressively adds noise to data from p. (x), and we denote the distribution of za as (xt)). This process can be reversed in time in the sense that if y x1-t, then Yr obeys the so-called backward SDE,
dy: = [g log - (ve, 1 t)] dt + g(1 t)dW, yo Pl
(4)
where W. is another Brownian motion. DMs aim to learn the (Stein) score function, logge
by approximating it with neural network TRdd x (0, 1] - TR. Once the network is trained, (z( V log Po (xx) is plugged in Enuuaation 4] and solving the resulting SDE transforms noise into model samples. Below we brielly summarize two existing methods, FLIPD and NB, for
approximating (z) for a DM.
FLIPD Kamkari al. (2024b) proposed FLIPD, an estimator of LIDe(x) for DMs. Commonly is linear in 24, in which case the transition kernel corresponding to the forward SDE is given by
Pejo z0)= (1as (t) (t)
(5)
where uh [0, 1] IR are known functions which depend on the choices of f and g. and which can be easily evaluated. For DM with such a transition kernel, FLIPD is defined as
FLIPD( fo) = d + o² (1)) (rr (V 80 (v (1))
4))) + ||00 (1o((boo))) (ttt)),; 40)
(6)
where to E [0, 1] is a hyperparameter. Kamkari et al. FLIPD to) is valid approximation of LID, x+). The reason for this is that the rate of change of the log density of the convolution between (z)) and a Gaussian evaluated at 2o with respect to the amount of added Gaussian noise approximates LID₂ and Kamkari al. (024b) showed that FLIPD computes this rate of change. In practice, computing the trace of the Jacobian of 8g is the only expensive operation needed compute FLIPD, and this is easily approximated by using the Hutchinson stochastic trace estimator (Hutchinson) 1989).
2024b) proved that, when tu F 0 and x E MM.
Stanczuk et (0244) proposed another estimator of LID, (x) for DMs. Following Kamkari et al. 2024b), we refer to this estimator as the normal bundle (NB) estimator, Stanczuk et al. (2024) proved that when (x,, t) = 0, so (2+,t) points orthogonally towards Mg as - They leverage
NB
this observation follows: for a given x. [Equatio 3/is started at and run forward until time to:
this is done is times, resulting in (!!)) ....,,0o The matrix So to) E Igd* is then constructed as
(K)
(k) Tto
So tu)
Ii tu
80
to
and thanks to the previous observation, the columns of So space of Mg when to x 0, meaning that rank (r to) ss d LID,(x). The NB estimator is given by
So xz, to) approximately span the normal
NB fn) d rank (o 1o)-
In practice the rank is numerically computed by setting a threshold, carrying out singular value decomposition of 5o (ar, 4n), and counting the number of singular values above the threshold. Stanczuk et al. (2024) recommend setting &t = 4d, and we follow this recommendation. Computing the NB estimator is much more expensive than FLIPD, since 4d forward calls have to be made to construct So (z. fo), and then the singular value decomposition has cost which is cubic in d. Finally, we point out that when ff is not identically equal to 0. the NB method can be easily adapted to still provide a valid approximation of LID, (x) (Kamkari et al. 2024a).
We highlight that both FLIPD and NB were originally developed as estimators of LID.(r) under the view that if the learned score function is good approximation of the true score function, then
16

---

Published as a conference paper at ICLR 2025
(x) = (z). In work, these methods as approximating LID,(x). Note that these views are not contradictory: when the DM properly approximates the true score function, will indeed be the case that (2) LID. LID.(x): importantly though, when this approximation fails, interpret (1, (x, (o) and (2(2, fo) as still providing a valid approximation of (IDx(x) rather than poor estimate of (r).
B.2 LOCAL PRINCIPAL COMPONENT ANALYSIS
Local PCA (Fukunaga & Olsen 1971 offers a straightforward method for estimating the LID, of the data manifold. Given æ, local PCA first
a datapoint by using linear local approximations identifies a set of nearby points in the dataset, representing neighbourhood; this is typically done through k-nearest neighbours algorithm. Next, the algorithm performs a principal component analysis (PCA) this neighbourhood to get (i) principal components and (ti) explained variances for each component; the resulting principal components capture the directions of data variation, with the explained variance showing the amount of variation along each direction. Off-manifold directions are expected to have negligible explained variance. Hence, local PCA determines the number of components with non-zero (or negliiglgg explained variance estimate for (x).
B.3 LID ESTIMATION WITH GANS
We assume the GAN is given by generator Go Rdd R R2 which transforms latent variables from distribution in H the ambient space Rd. For a generated sample z Gz(z), estimate LID, (x)) as the rank of the Jacobian the generator, i.c. rank VGg(z). As for the NB estimator with DMs, the rank numerically computed by thresholding singular values. We highlight that this is standard approach to estimate LID, in decoder-based DGMs (Horvat & Pfister 2022 Kamkari et al. 2024a
HHumayun et al. 2024).
17

---

Published as a conference paper at ICLR 2025
East Neceriaaa
1.2
DOCO
4000
200.0
1.-
Donaity
Densiit
0.8
2000
0.5
0.2
00
0.035
0.023
8.005
0.000
0,35
0.10
0016
0,20
0,985
0,30
8,05
0.600
(a) LiDg adjusted PNG for IDDPM.
(b) LiDg adjusted by PNG for StyleGAN-ADAZ.
(c) PNG compression length for Stable Diffusion in-
Figure 8: Removing and analyzing image complexity as confounding factor memorization detection for
CIFARIO (a-b) and Stable Diffusion (c).
C EXPERIMENTAL DETAILS
C.1 HYPER-PARAMETER SETUP FOR LID ESTIMATION METHODS
LID. with Local PCA As established in Appendix B local PCA estimates the intrinsic di- mensionality of datapoint by counting the number of significant explained variances from PCA performed on the datapoint's local neighbourhood, determined by its k nearest neighbours (k 100 our experiments). Finding the sigmificant explained variances is done through a threshold hyper- parameter, т, where explained variances above T are considered significant. For our approach in Figure 5d. we introduce two key modifications to better adapt the original Local PCA algorithm for detecting DD-Mem: (i) instead of selecting individually for each datapoint, we define it globally as the 10th percentile all explained variances across the entire dataset;
(ti) if a datapoint has neighbours within the 10th percentile of all pairwise distances, we restrict the neighbourhood to those points. The second modification allows us to avoid including distant points in the neighbourhood closer ones already exist and especially helps us detect ccro-dimensional point masses.
As detailed Appendix B.3 the rank of the Jacobian VG can be used to estimate LID,. However, in practice, the rank or, equivalently, the number of non non-zero singular values tends equal the latent dimension; this is because singular values are typically close to zero but rarely exactly zero. To account for this, we apply thresholding approach: singular value considered significant (non-zero) if it exceeds hyperparameter т.
LIDg for GANs
We define т as the 10th percentile of all singular values computed from the generated images in Figure 5b and Figure 8b
LIDg with NB We used to 0.1 and thresholded the singular values of (1. to) by 10th percentile; the results are presented in Figure 5c and Figure Sal The choice to is empirically determined by observing how the NB score correlates with the memorization behavior with fixed subset of 1000 randomly generated samples.
LIDa with FLIPD Unless stated otherwise, we set to = 0.05 for FLIPD and use the Hutchin- son estimator approximate the trace of the score gradient in Equation 6 In line with Kamkari et al. (2024b). we apply this in the latent space of Stable diffusion and use a single Hutchin- son sample to estimate Equation 6 for all of our large-scale experiments.
CFG Norm for Detecting Memorized Samples in the Training Set Note that while Wen et al. 20033 use the generation process to measure whether a synthesized image has been memorized, we were interested in detecting whether real, training-set set images have been memorized in Figure 6 which requires some methodological changes.
compute a memorization score, we take Fuler steps forward using the conditional score (a I, c) with the probability flow ODE (Song et al. until time to to get a point at To e Rd. We then compute the CFG norm [ to. c) Sp x 0) We use timestep to 0.01 and 3 Euler steps.
2021
C.2 THE CONFOUNDING EFFECT OF COMPLEXITY FOR DETECTING MEMORIZATION
LID, is correlated with image complexity (Section 2, Kamkari et al. (2024b)). which raises valid concern: the correlation, combined with the fact that simpler images more likely to be memorized,
18

---

Published as a conference paper at ICLR 2025
suggests that image complexity may confound our analysis. This is evident in Figure 5a (right panel), where -AN-geeneateed images with the lowest LID, values are the simplest ones, not necessarily the memorized ones. To address this confounding factor, we draw inspiration from Kamkari et al. (2024b) and normalize by PNG compression length, using it as a proxy for image complexity. We use the maximum compression level of 9 with the cv2 package (Bradski 2000). According to this adjusted metric, the smallest values now correspond memorized images that are not necessarily simple, such as the cars in CIFAR 10. Figure 8a and Figure 8b show these adjusted LID estimated values, which achieve a slightly improved separation between memorized and not memorized images (as well as between exactly memorized and reconstructively memorized images) than the Non-PNG-normalizee results the main text. is worth noting that complexity did not appear to be a confounding factor in the Stable Diffusion analysis shown in Figure 6 In fact, depicted in Figure 8c) the Tuxemon images are relatively simpler than the LAION memorized images, as measured by their PNG compression length. However, despite their simplicity, Tuxemon images have consistently higher LID, values compared to the memorized images Figure 6b) and Figure 6a
19

---

Published as a conference paper at ICLR 2025
D TEXT CONDITIONING AND MEMORIZATION IN STABLE DIFFUSION
D.1 ADAPTING DENOISING DIFFUSION PROBABILISTIC AND IMPLICIT MODELS
Following Wen et al. 2023), we use denoising diffusion probabilistic models (DDPMs) (Ho et al. 2020 This model can be seen as a discretization of the forward SDE process of a score-based DM (Sogg et al. 2021 Here, instead of continuous timesteps t, timestep instead belongs to sequence ...... ) with being the largest timescale; we use = 50. We use the colour to denote the discretized notation used by Ho et al. (2020).
With that in mind, DDPMs can be seen as a Markov noising process with the following transition kernel, parameterized by ae, mirroring the notation Ho al. (0200):
(1 (r)
(are Vor Tos (1 a Tg))
DDPMs do not directly parameterize the score function, but rather use a neural network e (x(,,)). which relates the score function as:
5 t/T) = (2 tttviv/vVV/// ai, or equivalently, - -0(t//))o((x co
(10)
Note that in this context, we have (1(7)) = 1 - oe and (1/77) = Vor. Equation 9 (the transition kernel) and Equation T0 (the score function) provide with the recipe for estimating LID,, using FLIPD with DDPMs (recall Equation 6).
When sampling from DMs we use the DDIM sampler (Song et al. 2020 mirroring the setup Wen et al. 2023). In our notation, this sampler defines ā, := zz/ (f), where Ty is given as in Equation 3 In turn, 2y obeys the forward SDE:
dē, = g())dW,, To ~ p. (x),
(11)
where ğ(t) = g(t)//(()). This SDE has corresponding score function
50(z, t) = (v)))) t).
(12)
and DDIM uses this score function to sample from the model. The transition kernel corresponding to Equation 11 has v²()) = and a (t) which can be computed in closed form. Analogously to
Equation 10] define
(13)
We highlight that FLIPD (Equation 6) can be applied using the forward SDE in Eqqation II along with its corresponding score function in Equation 12 resulting in the estimate
FLIPD(z, x. t) = d + 52 (V) + |1 I1?)
(14)
Note that this estimate can be computed when having access to éa thanks to Equation 13 We also note that in our coxtiiondiioning analysis, we are interested in the probabilities conditioned by the text prompt, thus, these score functions are extended by the conditioning variable c, resulting in the modified forms 6o (x;; I, c), 6o (,),, (0(2; I/T, c). and (o;
D.2 UNIFYING DIFFERENTIABLE METRICS FOR TEXT-CONDITIONEL MEMORIZATION
We begin by revisiting the differentiable memorization metric used by Wen et al. (0233 for detecting and mitigating memorization, reformulating it within the continuous, score- based framework of diffusion models. Building on this, we perform an analysis, making minimal modifications to the original formulation to derive alternative metrics that remain effective and are prreticalll-groundeer
As result, here we will formally derive three differentiable metrics: AAFGG/ (c). A (c). and finally (FLIPD (). We show that the value Wen al. (2023) compute in their paper f'ct an estimator of C (c), rescaled by constant. We then make minor modifications to introduce the two new
metrics A (c) and FFIPP and interpret them through the lens of the MMH.
20

---

Published as a conference paper at ICLR 2025
The Differentiable Metric of Wen et al. (2023) For any text condition Wen et al. (023)
generate multiple samples (āo (")) m== with the nuth sample following the (DDIM) trajectory
{āy. (n) - (n), ..... Ty (4)) from noise to data through the denoising process. They then intro-
duce the following metric, which we have slightly reformulated to match our notation:
ACRO (c
1 TN
ΣΣ Eo (e,())),()) Eo n=1
( (z,
it, ;t,00,,
(15)
We colour-code the metric in red to distinguish between it and the analogous metric that we will shortly derive at the end of this section.
Let ff represent the marginal probability at time induced by the DDIM sampler conditioned on e with the addition of the CFG term. Recall from Equation 2 that the score used for sampling from DCKG Pt
(- c) with CFG is
CPG 38
(xtt, t, c) 50(x;; + (15 50 (x; 0))
(16)
Using Equation 13 and Equation 2] we can rewrite A (c: N, T) as follows:
N
a(/TT)
C (c: N. 1)
ΣΣ
CFG 58
(m) ; c) - 50 (2 0)
(17)
A
We now assume 7 + oo, which will reformulate Equation with an integral that we will replace with expectation:
CHC Nc: = lim
CFG N,
(18)
T-000
N
2
8² (5) (0) (t.)) 60(;, (m); 0)
(19)
N
R=1
N
x 22
E 10.1
(1)]|
CFG (7. (m); (,) - 50(2, (,,,
(20)
N
1
- (
Σ
(1)
130
CFC
(7. 10 ()
50 (a. (,,
(21)
N
n=1
Here, (0, 1) denotes the uniform distribution. Next, we observe that the inner term of the expectation on the hight-hand-sid of Equation 2 in fact Monte-Carlo estimator. By the law of large numbers, we have the following:
CFG (c)
lim
CFG
Nc
(22)
N
1.. (f [b²(t) -- El
||50 t, (ā,;t
(āx; t, 0)]
(23)
We now see that with the new formulation, all the red terms in Equation 23 have gone away, making it fully amenable to the score-based formulation of diffusion models. The À factor merely scales the metric, and for the purposes of detection and mitigation, this scaling is inconsequential: if a metric effectively predicts memorization, rescaling it will not diminish its effectiveness thus disregard the scaling factor to make the derivation cleaner and replace the uniform distribution U(00 1) with a general "scheduling" distribution T(0, 1) of timesteps in (0, 1]; this would allow our metric to be a generalization of the one proposed by Wenn et al. (2023):
predictor. We
CFG (c) EE
[o²(t)
||5 (ānt,c) (0 []
(24)
Simplifying Further We have shown that the CFG vector norm and the CFG adjusted score norm behave similarly in Figure 3 If, instead of considering the CFG vector norm Equation 24 we
consider the CFG-adjusted score sp SCFG (:tt,c) c), we arrive at the following metric:
A (c) = 10 B
(Zr
(25)
We have shown this to be viable memorization metric, able to detect tokens driving memorization
in Figure 11] and behaving comparable to AACFG (c), the original metric proposed by Wen et al. (2023).
However, nice property of A (c) is that it can now be linked to MMH: for memorized prompt where LID, c) is small, the score function of GCFG c), especially for small t, tends to become large, causing the metric in Equation 25 to increase significantly.
21

---

Published as a conference paper at ICLR 2025
We now propose more direct proxy for LID based on the FLIPD estimate of LIDg. Recalling Equation 6 we can define the class-conditional LIDa(- c) estimate based on FLIPD follows, analoguously to Equation 14
Linking to FLIPD
FLIPD
CRG
(x) =do)) (w (ir(V8g PCG (15 (,), + ||
(x; (w)4)))1))
(26)
has a similar term to Equation 25 we add d and the trace term from Equation 26 into Equation 25 and propose the following MMH-based metrie:
Noting that FLIPD
d + (c) + ET-TT p (1lc)
(t) tr (v
CPG
(Zri c))]
(27)
CFG
FLIPD
(T (trit,
FLIPD
(c).
(28)
CRG (c), the former indicates memoriza- Despite the fact that (FLIPD can be expressed terms of A tion when it is small, while the latter indicates memorization when it is large.
Note that while Equation 28 averages FLIPD values over (potentially) all the timesteps E (0, 1]. the theory linking FLIPD and LID₃ is only rigorously justified when < (Kamkari et al. 2024b). Hence, we set the scheduling distribution such that it primarily samples / close to zero As such, FIIIP will average FLIPD estimate terms that are closely linked to LIDg(- c). Notably, our experiments also revealed that although setting as small as possible makes sense from a mathematical perspective, the score function, and result, FLIPD estimates, become unstable as t ++ 0 (Pidstrigach) 2022 Kamkari et al.] 2024b). Therefore, in practice, we choose T as uniform supported on (0.0, 0.2]; therefore, putting more emphasis on these small f values but at the same time avoiding instabilities in (CLIPD
The scheduling is small, but important distinction between A FLIPP one hand, and A and A on the other hand; while AFLIPD sets as a uniform on (0.0, 0.2). A t and ACCR set to uniform distribution on (0, 1]. to mirror the setup in Wen et al. (2023).
D.3 INCREASING IMAGE COMPLEXITY BY OPTIMIZING FLIPD
Wen et al. (023) have an experiment where they optimize the prompt (embedding) e directly to minimize (CTC(c), (c), and as result decrease (c)) with the purpose of obviating memorization.
Here, we take a similar approach but instead optimize e to maximize AFLIPD (c).
In Figure 9 we optimize with Adam using multiple steps, and as we increase (FLIPD(C), (c). we sample images using the prompt embedding which is being optimized. We see that images sampled from these prompts indeed increase in complexity. This is fully consistent with our expectations and understanding of LID. We see, however, that while at a certain range the images are relatively less memorized, the method tends introduce excessively chaotic textures artificially increase LIDØ(- c). often at the expense of the image's 's semantic coherence. Despite this, we still find this to be an interesting result and invite future work on using different scheduling approaches for AFLIPD (c) that can stabilize the optimization process of c.
D.4 TEXT PERTURBATION APPROACHES
GPT-based Perturbations As outlined the main text, we sample tokens without replacement from categorical distribution obtained by normalizing the token attributions, then use GPT-4 to replace these tokens; this ensures that tokens with the highest attributions are replaced more frequently. After selecting these A tokens, we ask GPT to follow the instructions provided in Box [ and use the output of the conversation as the new prompt. This process is repeated five times in our analysis to account for any randomness in the output of GPT. It is important note that these perturbations designed to preserve the semantic structure of the prompt. To ensure this, the instruction specifically asks GPT not to replace names of places or characters, and to keep the new prompt as semantically close to the original as possible.
22

---

Published as a conference paper at ICLR 2025
(a) Alongoide Adapcatica Eggere" Circie</1>
(h) Awro S1-9842108IX Sariex 13x8 Wheal, 9pun 4-1/4 BP Tnch RS
ec) Air Conditiionra Parta
WEPZA
KRFELLX
STTTION
MITLON
50
IK HEE
GEZ TEL
NIFLX Y
(d) Metrlix Hits 50 Killion Subecrsbere
(e) Stripe
äjörk Explaina Deciaion Pull <i>Wuleicurwe/1> Spotify
(g) Herw's 'a. Vist. You Keed to Know About St., Vincent's Appla Maxin Radio Show
Figure 9: Samples and their original memorized captions from directly optimizing text conditioning to increase A frIPD and reduce memorization. The images progress through different stages of the optimization process from left to right. While increasing LID, helps reduce memorization, uncontrolled increases often introduce chaotic textures, resulting in unrealistic images.
23

---

Published as a conference paper at ICLR 2025
Box I: Instruction for perturbing caption prompt using GPT-4
I have the following caption as sequence of tokens: (original tokens
I want to create new caption based this one, but I want to perturb the following tokens: (selected tokens
These are the rules follow for perturbing tokens:
1. If the token is special character punctuation without significant semantics, you can remove it or change it to any special character
2. If the token is number, you can replace it with another number that is close to
If the token a special name, such as the name of someone or some place or some culture,
it should not be replaced
4. If the token is any other word, you can replace and rephrase it with any synonym that makes sense in the context
Given these requirements, please provide with new caption, as a natural language sentence that semantically matches closely with the original caption except for the perturbed tokens. Do not say anything else in response, only provide the new caption.
sequence of tokens, but
Qualitative Comparison Figure 10 presents qualitative comparison of our GPT-based pertur- bations applied to three memorized prompts. We have selected these specific examples to illustrate how the prompt perturbations function practice. In this case, we set and randomly perturb the tokens based attributions derived from AFLIDD, Additionally, Figure 10 includes column demonstrating the random token addition (RTA) approach proposed by /Somepalli et al. (2023a), where & random tokens from the CLIP library inserted into the prompt. We see that the GaT-b perturbations better preserve the semantic integrity of the text caption, resulting in images that are not memorized and significantly more coherent.
24

---

Published as a conference paper at ICLR 2025
Memorized Prompt
GPT AFLIDD Mitigation
RTA (Somepalli et al. 2023b
Talks the Precepts and Buddhist Ethics
discussions about the precepts and Buddhist principles
Talks mellon dragonball the vil- lar Precepts and reformed Buddhist Ethics
CRINZAMN
Ti>The Long Dark</i> Gets First Trailer, Steam Early Access
<h>The Long Darke/i> Gets First Trailer: Steam Early Access
barbershop relying <i>The idal Long Dark</i> Gets First Trailer, Steam Early ghorn Access
JOHN W DOYLE BoundAduie
SOUDGUINE
GUINIVONE
Podcas!
UOHI WHH WHs WIHNETE, COIT. JOOH.ML
Sound Advice with John W Dayle
sound guidance with john dayle
Sound Advice with John payments
grill hsfh acadi W Doyle
Figure 10: Comparison of mitigation approaches. The tokens highlighted in red indicate the changes and perturbations made by each approach.
25

---

Published as a conference paper at ICLR 2025
D.5
MORE EXAMPLES OF TOKEN ATTRIBUTIONS
A CPG
0.50
FUPD
0.25
0.00
netflix
hits
5
million
subscribers
METELIX
ONITUSE
Training Image
Generated Image
GPT CFG Mitigation
GPT + FLIPD Mitigation
(a) the methods detect "Nettli" private trademark driving memorization.
0.4
CFG
FLIPD
0.2
0.0
sony
won
release
the
interview
on
vod
SYINNF
TTE II STHOW!
SEEN BOGIN JAMES FRANCO
SITH HOSEN JAMES FRANCO 0f N7
Of WPPE BE EXI 1M
INTERVIEW
INTERVIEW
Training Image
Generated Image
GPT + CFG Mitigation
GPT + L4 FLIPD Mitigation
All the methods dctect "interview" (the movie title) the driver memorization, A CFG detects "Sony" significant token.
A
0.4
CFG
0.2
FLIPD
0.0
the
no
limits
business
woman
podcast
No Linits
No Limits Business Woman Podcast
The
Business Woman Podcast
THE LIITTS BSSSINSSS OLDS DSSSS
NO LIITTS'
Training Image
Generated Image
GPT + A Miligation
GPT FLIDD Mitigation
(c) All the methods detect "podcast" as the token driving memorization.
Figure 11: Memorized Stable Diffusion samples with comparison of taken attributions hased on three different memorization metrics: the CFG vector norm proposed by Wen et al. (2023), the CFG-adjusted score CFC (x;; norm. and FLIPD estimate for LIDs
26

---

Published as a conference paper at ICLR 2025
E PROOFS
We restate each theorem in full formality below along with their proofs.
Throughout this section, we let P. and Pa be the probability measures of the ground truth data and model, respectively. We assume that the respective supports of P, and Py are My C IR smooth Riemannian submanifolds of the Euclidean space E' with metrics 9, and ge respectively. We denote the Riemannian measures on M, and Ma as and po. respectively, so that p.(x) dPP/du, (x) and po (a: dP /yu (a=). As mentioned Section 2 we take a lax definition manifold which allows them to vary in dimensionality in different components. A single manifold under our definition equivalent to disjoint union of manifolds under the more standard definition.
E.1 PROPOSITION 3.1
Lemma E.1. Assume that pa (x) > for every x M., and let zo M.. Then, the following are equivalent:
1. P. ((rO)) > 0, and
2. LID, (Io) 0.
Proof.
(1) (2) Assume Ps (fxo)) >
0 P. (120))
p. (rrrdd (2)),
(29)
(mo)
which necessitates (f=n}) > 0. If we had LID (xp) this would incur a contradiction: letting ()) be chart around xp. then by the definition of pea
ff Sscenp) Vdet(g.)d),
< (.xul))
(30)
where À the Lehesgue measure on LED... or the counting measure L.ID, (.r) Due to the singleton domain of integration, positivity of the integral in Equation 30 would impossible unless LID.
(2) - (1) Suppose LID (aro) = 0. This implies that {mo} an open set in the subspace topology of M. Since zo E supp jr,, any open set containing sp must have positive measure under so that n. (1xo)) > 0. Then, since P. (((0)) = p. (zo)), and p. (x0) > follows that P. ((xo)) 0.
Proposition E.2 (Formal Restatement of Proposition 3.1 Assume that px for every x M.. Let { be a training dataset drawn independently from (x). Then:
If duplicates occur { with positive probability, then they will occur at a point To such that LID,, (xo) = O.
2. If (xo) 0 for some un Ma, then the probability of duplication in {xi)i will converge to as + co.
Proof.
(1) Due to Lemma E. it suffices show that any duplicates in {x.)7_ must a point 2o such that P. (1xx)) > 0. thus enough to show that P. (f(x)))) = for every so S., then P. (m x2) 0. Assume that (.(o))) 0 for every
27

---

Published as a conference paper at ICLR 2025
E M.. Since and 22 are independent, P. x2)
P. x (D()), where
== = ((I,z) E Mr x Ma x E M.}. We then have:
P. x P.(D) J. dP. x 22)1 JS. J (12)
(P.(3))dPP(2a)
(31)
= =Jh.
(32)
where the second equality follows from Fubini's 's theorem (see e.g. Theorem 7.26 in Folland (203)), and the last equality follows by assumption. This finishes this part of the proof.
(2) Suppose LID. (zo) = 0 for some xo .... By Lemma E. we have P. (f2o)) >
In this case, P. (I, zo) 0 for all i {l...., ..). meaning that
P. (x; zj for some 1 S < j S n)
2 P. (zt 2
for some 13i<j S n)
(33)
2 P. (r: * zo for all 2 2)
(34)
1 - Pæ + mo) ... (2 + xn)
(35)
(1 P. (120 "
(36)
1,
(37)
where the last line depicts the limiting behaviour as n + 00.
E.2 PROPOSITION 3.2
Here we presume the joint distribution of model samples and k-dimensional conditioning inputs (z. c) E Bd+ has support S Rd+h such that {x : (x, S for some E R$} .Mg. We define the conditional support of x given e to be S(c) = {x (2r, c) E S}.
Proposition E.3 (Formal Restatement of Proposition 3.2 Let ag E MM and c E R Suppose that S(c) is also submanifold of Hd and denote its LID at zo by LIDe(ro c). We then have
LIDo(aa | c) s LIDa(zo).
(38)
Proof. If S(c) is submanifold of Rd, then it is also submanifold of Mg. The inequality follows directly.
E.3
HEOREM A.3
Here we show that mem implies data-copying under the definition of Bhattacharjee et al. (202))
Lemma E.4. Suppose (M, g) is a da-dimensional smooth Riemannian submanifold of Euclidean space Rd. Let u be the Riemannian measure of M. If Bd denotes the d-dimensional ball of radius r centred at da in Id,, then there exist constants C.M > and C.M. > 0 not depending on r such that for all small enough r:
MM, S (BA( n s M,
(39)
Proof. Without loss of generality, by rotating and translating, assume In = 0 E Rd and that the tangent plane of M in at Jo is RRd x {0)d-da.
As M is smooth, in neighbourhood U of zo 0, M be written of graph of function u : U C R + Rd-do, such that u(0) = and all first derivatives vanish at 0. Then, for small enough O, we have
(x00)) MM= (())) E nd | e do .| - [|x(2)]12 < rr
(40)
Let
G(r) = ((2, E Rd | IZERS R2 |||||| < r)
(41)
28

---

Published as a conference paper at ICLR 2025
be the graph of in the open do-ball (0)( (0), and
G(r) {(2(,(110 u(=)) E Rd 1 R ||||| - rr
(42)
be the graph of u in the closed do-ball (0). Note that both are defined when u is defined, i.e. small enough r, and are subset of M. Then it is clear that have
(20) NM S G(r).
(43)
Now, consider the function v(2)
Iu(=]ll 1=ll Note that v=2) is continuous everywhere in 1 (0) \ {0).
Since and derivatives vanish at z O. from the definition, we have limæ-0 e(2) = 0 as well.
Thus (z) can be extended to continuous function in 3 (0). Fix R >> 0, and let K be the maximum
of v over Bd Then, if ||=|| < ar where R
VI+K. we have
|| + [=](2 [1=1 + v(2) & (1 + K3) <?
(44)
This shows that G(ar) _ (zo) MM for 0 < r < R. Thus we have
m(G(a))) & p n MM S H(G(r))
(45)
Let Ki and K2 be the minimum and maximum of det y over 1 (0), respectively. Then we have
m(G(r)) = ( /det & /
K2dz = K2Vo, n
(46)
B.0(0)
and
((G(r) /
det 2
Kjd= = K2 VV,,
(47)
8 (0)
(0)
where Vá, is the Euclidean volume of the unit do-ball. Combining the above results, have
Ki a I <((((r)) S (x)) MM S u(G(r)) 12V which finishes the proof with CiM K2 Va, d and C.M K2V
(48)
Theorem E.5 (Formal Restatement of Theorem A.3). Assume that (x) and po(x) are continuous and that pa (x) is strictly positive. Let To E ManM. and let po(x) be model undergoing OD-mem at Fop iee. O S (1)) LLD,, (Iz)). Then for any X and 0 < < there exists a radius To such that any x E B (zo) is a T)-opy of zo according to Definition A.l and if { is generated independently from (x), then the probability of (X, >--cpyyngg converges to 1 as m DO.
Proof. For an arbitrary value of r > O, we have that
(x( 2 ( Ma) inf =E po(x)
(49)
and similarly,
P.(1)) S (., (
sup
(x))
(50)
aERM(rO))
Using Lemma E.4
(B(((a)) Po (Bc( (o( n Ma) inf (z)
He (B n M.). SUPaeB/(UO)M Px(x)
<
(51)
M.M. LiD. LTD. (LD)) C
(f P. (x)
(52)
inf i (x)
Note that by continuity and positivity of (o()), inf, inf Po(x) is bounded away from 0 as T - 0, and by continuity of (x). B is bounded. turn, since by assumption (xo) > (I)) Equation 52 converges to 0 as r + 0. As result, there exists some ro sufficiently small enough for both
(. (2o)
<
(53)
Po Bdd (xo))
29

---

Published as a conference paper at ICLR 2025
and
(B (zo)) 5
(54)
to be true (the latter arising from the fact that P. (z²)) 0 as - 0, which follows from P. being absolutely continuous with respect to μ. and μ., not assigning positive measure to singletons because LID. (x.) > 0).
Thus, any I E Bi, (zo) is coopy of zu. Since B (ro) n Ma contains an open set in the subspace topology of Me. it follows that μe B (2o) n Ma) > 0. Then, since pe( is strictly positive, Pa (B (ro)) > 0, so that
Polzy is (A, -opy of zo for some - m)
(55)
Po Pay is not (Aooyy of un for every S m)
(56)
(21 is not coopy of zu) ... Po(Im is not (), ccopy of zo)
(57)
(B (Io)))"
(58)
converges 1
30

---

Published as a conference paper at ICLR 2025
F MEMORIZED IMAGES FROM CIFAR-10
We generate 50,000 images from each model. Since StyleGAN2-ADA class-conditioned, we generate equal number of samples per class. We then retrieve nearest neighbors from the training dataset using both (i) SSCD distance (Pizzi et al. 2022) and (ii) calibrated l2 distance (Carlini et al. 2023). We find that each metric produces very different results, so we combine results from both to maximize our probability of identifying as many memorized images as possible. Furthermore, both metrics produce many false negatives, so we follow a manual process to produce accurate labels. For MtyleGAN2-ADA we take the closest 250 neighbours according to each distance, and for iDDPM, we take the top 300, producing set of just under 500 or 600 images visually examine for each model We then label all of these instances either not memorized, exactly memorized, or reconstructively memorized (Somepalli et al. 2023a). All other images are not labelled, and have a low chance of being memorized.
Here we show each generated image we identified (odd rows) along with its matched training image (even rows below) for iDDPM and StyleGAN2-ADA on CIFARIO. For StyleGAN2-ADA, labelled no pairs as reconstructive under the calibrated &z distance.
Figure 12: iDDPM: reconstructive pairs in the top 300 according to calibrated Eg.
Figure 13: iDDPM: human labelled exact pairs in the top 300 according to calibrated E
1A
Figure 14: iDDPM: reconstructive pairs in the top 300 according to SSCD.
'The cutoffs of 250 and 300 were chosen by inspection; from these ranks onwards, images ceased to appear memorized.
31

---

Published as a conference paper at ICLR 2025
Figure 15: iDDPM: human-labelled exact pairs in the top 300 according SSCD.
Figure 16: StyleGAN2-ADA: human-labelled exact pairs in the top 250 according to calibrated &g.
6B
EB
Figure 17: StyleGAN2-ADA: human-labelled reconstructive pairs in the top 250 according to SSCD.
32

---

Published as a conference paper at ICLR 2025
Figure 18: StyleGAN2-ADA: human-labelled exact pairs in the top 250 according to SSCD.
33


Reviews:
[{"review_id": "3mXRrnqWV2", "reviewer": "Reviewer_6rGM", "summary": "Memorization in Generative model occurs when model reproduce its training data instead generalising from it. This phenomenon is undesirable because it represent to modelling failure. The paper introduces the \"manifold memorization hypothesis\" (MMH) to explain memorization geometrically. It suggests that memorization happens when the manifold learned by the model at a point has a lower dimensionality than the true data manifold at that point. This framework defines two types of memorization:\nOverfitting-Driven Memorization (OD-Mem): This occurs when the model assigns low dimensionality to a region of the manifold, overfitting specific training data points.\nData-Driven Memorization (DD-Mem): This happens when the true data distribution inherently has low dimensionality in certain regions, leading the model to reproduce specific details in these regions even if it generalizes well.\n\nThe paper introduces the Manifold Memorization Hypothesis (MMH) as a geometric framework to analyze and understand memorization in deep generative models (DGMs). It explores memorization through the lens of local intrinsic dimension (LID), a metric used to assess whether a model's manifold dimension aligns with that of the underlying data. The authors identify two types of memorization: overfitting-driven memorization (OD-Mem) and data-driven memorization (DD-Mem), categorizing instances of memorization based on LID discrepancies. This hypothesis is validated empirically with synthetic and real datasets, demonstrating how MMH can predict memorization tendencies and proposing methods to mitigate memorization in diffusion models, particularly Stable Diffusion.", "questions": "How does the MMH framework extend to discrete generative models, such as those in NLP (transformer)?\nCould alternative LID estimation methods reduce the overlap issue observed in the experiments, or is this an inherent limitation of the current approach?\nAre there specific types of datasets or model architectures where MMH may not apply effectively?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The introduction of MMH offers a fresh perspective on memorization, connecting it to geometric properties and adding depth to existing theoretical understandings.\n The paper proposes actionable techniques to reduce memorization in generative models, which has direct implications for improving model safety and compliance.", "weaknesses": "The reliance on geometric and measure theory could make it challenging for a broader audience to fully understand and appreciate the framework. \nThere should be more experiments\nWhile LID proves to be an effective measure, there is some overlap between memorized and non-memorized samples in LID estimates, which may affect its reliability in distinguishing memorized samples in real-world applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S8mIM1v3dJ", "reviewer": "Reviewer_QHDy", "summary": "Memorization is a big issue in deep generative models. Recent works have highlighted that such models are prone to duplicating their training points, given certain prompts, or generating samples that appear very close to training points with slight modifications (e.g., background and color). To combat and understand this phenomenon, this paper proposes a geometric framework which leverages the manifold hypothesis. Particularly, the work analyzes memorization in terms of the relationship between dimensionalities of the ground truth data manifold and the manifold that is learned by the model. This is done by analyzing and calculating the Local Instrinsic Dimension (LID) of the data and generated points, and building a theoretical understanding around the model's memorization behaviors. Overall, the paper validates the applicability of the manifold analysis in the detection of memorized samples through the development of new tools and prevention methods in generating such samples.", "questions": "For definition 3.1, could you elaborate on condition (2) or why $p_\\theta$ cannot be equal to $p_*$?\n\nCould you elaborate the differences between $s_\\theta$ and $s_\\theta^\\text{CFG}$ in equation 41 of the Appendix D.2? \n\nWhat is the conclusion of figure 6? The importance of this figure/result is not clear to me. \n\nCould you provide more figures like figure (a), of memorized samples, in Figure 5? What was the threshold used in the detection metric from [Carlini et al. (2023)](https://arxiv.org/pdf/2301.13188), which was used to identify memorized samples?\n\nBoth $\\hat{LID_\\theta}$ and $\\hat{LID_*}$ are estimators of LID. Is it fair to assume that $\\hat{\\text{LID}}_*$ is more accurate? Or, is that the wrong interpretation?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "The paper provides a very valuable and interesting perspective about memorization, which comes in two forms: Overfitting-driven Memorization (OD-Mem) and Data-driven Memorization (DD-Mem). These two forms of memorization highlight that memorization is a convoluted aspect of trained deep networks, and it should not always be naively thought of as just overfitting. The paper does a decent job with making a case for this topic, by using LID as a metric of identifying memorized samples. \n\nFurthermore, the work introduces a differential metric, $\\mathcal{A}^\\text{FLIPD}$, which allows for the optimization of a conditional variable $c$ to prevent the reproduction of training points during sampling. Specifically, the authors extended and combined the work of [Wen at al. (2024)](https://arxiv.org/pdf/2407.21720) and [Kamkari et al. (2024)](https://arxiv.org/pdf/2406.03537) to produce the aforementioned metrics, which show promising results in the prevention of memorization for stable diffusion models.", "weaknesses": "The comparison of various LID metrics to other memorization detection metrics is not quite clear, especially in figure 6. It is difficult to tell how much more effective LID metrics are since the authors did not contrast them well in their written text, and it is difficult to interpret the results in the density histograms of fig. 6 without showcasing what is classified as memorization, visually.\n\nFurthermore, it is still unclear on how to quantitatively differentiate between OD-Mem and DD-Mem samples. The description, visualization (fig. 1), and toy example (fig. 4) certainly highlight the differences, but for natural datasets, it is difficult to tell still.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oSQs7udChM", "reviewer": "Reviewer_bhfc", "summary": "The paper proposes MMH, a geometric framework for understanding memorization in generative models by focusing on the dimensions of learned manifolds. It aims to analyze memorization as a function of the ground truth data manifold and the learned model manifold. The framework distinguishes between two types of memorization: overfitting-driven and data-driven, providing a theoretical base and empirical tools for detecting memorization across a range of models, from synthetic data to image generation using Stable Diffusion.", "questions": "1. I am confused about the definition of LID. From the whole paper, I can tell the smaller the LID is the more memorization. In line 134, it is defined as ''the number of degrees of freedom'' instead of the number of variables needed in a minimal representation of the data. However, based on lines 125-126, it seems to be the same as the intrinsic dimension. Could you clarify?\n2. In line 182: ''a notable consequence is that it cannot be detected by comparing training and test likelihoods''. Can you elaborate more about this?\n3. As mentioned in the weaknesses, could you explicitly state your novel contributions more, or provide specific examples of how the framework goes beyond existing work? Currently, there is no comparison with the existing frameworks, such as [2]. I will be happy to increase my score if this is resolved.\n\n[2] Robi Bhattacharjee, Sanjoy Dasgupta, and Kamalika Chaudhuri. Data-copying in generative models: a formal framework. In International Conference on Machine Learning, 2023.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The authors combine the manifold memorization hypothesis and a geometric perspective to memorization, potentially unifying existing observations into a cohesive framework.\n2. The paper provides clear motivation and background on memorization in DGMs, connecting memorization to legal and ethical implications.\n3. The empirical experiments well support the propositions and theorems.", "weaknesses": "1. Some minor typos, for example ''subtantial'' should be ''substantial'' in line 44.\n2. The MMH concept does not introduce substantial novel contributions beyond covering DD-Mem rather than only considering OD-Mem. This work is more like summarizing the existing work with their framework. For example, Proposition 3.5 is a (in)-formal summary of the findings in [1]'s experiments with MMH.\n\n[1] Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan Lyu. Detecting, explaining, and mitigating memorization in diffusion models. In Proc. ICLR, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)