Exploring Local Memorization in Diffusion Models via Bright Ending Attention
Authors: Chen Chen
Daochang Liu
Mubarak Shah
Chang Xu
Venue: ICLR 2025 Spotlight (2025)
Keywords: Diffusion Models
Local Memorization
Bright Ending Attention
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Text-to-image diffusion models have achieved unprecedented proficiency in generating realistic images. However, their inherent tendency to memorize and replicate training data during inference raises significant concerns, including potential copyright infringement. In response, various methods have been proposed to evaluate, detect, and mitigate memorization. Our analysis reveals that existing approaches significantly underperform in handling local memorization, where only specific image regions are memorized, compared to global memorization, where the entire image is replicated. Also, they cannot locate the local memorization regions, making it hard to investigate locally. To address these, we identify a novel "bright ending" (BE) anomaly in diffusion models prone to memorizing training images. BE refers to a distinct cross-attention pattern observed in text-to-image diffusion models, where memorized image patches exhibit significantly greater attention to the final text token during the last inference step than non-memorized patches. This pattern highlights regions where the generated image replicates training data and enables efficient localization of memorized regions. Equipped with this, we propose a simple yet effective method to integrate BE into existing frameworks, significantly improving their performance by narrowing the performance gap caused by local memorization. Our results not only validate the successful execution of the new localization task but also establish new state-of-the-art performance across all existing tasks, underscoring the significance of the BE phenomenon.

Published as a conference paper at ICLR 2025
EXPLORING LOCAL MEMORIZATION IN DIFFUSION MODELS VIA BRIGHT ENDING ATTENTION
Chen Chen' Daochang Liu? Mubarak Shah' Chang Xu'
School of Computer Science, Faculty of Engineering, The University of Sydney, Australia
School of Physics, Mathematics and Computing. The University of Western Australia, Australia
CCnter for Research Computer Vision, University of Central Florida, USA c. sydney. edu. daochang. www.uwa...ea........ eewiaaedduaaa au shahfcraw.ucf.edu edd
Tiche0711@ue
ABSTRACT
Text-to-image diffusion models have achieved unprecedented proficiency in gener- ating realistic images. However, their inherent tendency to memorize and replicate training data during inference raises significant concerns, including potential copy- right infringement. In response, various methods have been proposed to evaluate, detect, and mitigate memorization. Our analysis reveals that existing approaches significantly underperform in handling local memorization, where only specific image regions are memorized, compared to global memorization, where the en- tire image is replicated. Also, they cannot locate the local memorization regions, making hard to investigate locally. To address these, we identify a novel "bright ending" (BE) anomaly in diffusion models prone to memorizing training images. BE refers to a distinct cross- cooss-tttention pattern observed in text-to-image diffusion models, where memorized image patches exhibit significantly greater attention to the final text token during the last inference step than non-memorized patches. This pattern highlights regions where the generated image replicates training data and enables efficient localization of memorized regions. Equipped with this, we propose simple yet effective method integrate BE into existing frameworks, sig- nificantly improving their performance by narrowing the performance gap caused by local memorization. Our results not only validate the successful execution of the new localization task but also establish new state-of-the-ar performance across all existing tasks, underscoring the significance of the BE phenomenon.
INTRODUCTION
Text-to-image diffusion models like Stable Diffusion (Rombach et al. 2022) have achieved unparal- leled proficiency in creating images that not only showease exceptional fidelity and diversity but also closely correspond with the user's input textual prompts. This advancement has garnered attention from broad spectrum of users, leading to the extensive dissemination and commercial utilization of models trained on comprehensive web-scale datasets, such as LAION (Schuhmann et al. 2022), alongside their produced images. However, this widespread usage introduces legal complexities for both the proprietors and users of these models, particularly when the training datasets encompass copyrighted content. The inherent ability of these models to memorize and replicate training data during inference raises significant concerns, potentially infringing on copyright laws without notifying either the model's owners or users or the copyright holders of the replicated content. The challenge further exacerbated by the training datasets' vast size, which makes thorough human scrutiny unfeasible. Illustratiell, several high-profile lawsuits (Saveri & Matthew (0023) have been initiated against entities like Stability AI, DeviantArt, Midjourney, and Runway Al by distinguished artists. These lawsuits argue that Stable Diffusion acts as "21st-century collage tool', remixing copyrighted works of countless artists used in its training data.
In response to these legal challenges, Carlini et al. (203)) and Somepalli et al. (202a)) proposed similarity metrics to evaluate memorization, and recent efforts (Somepalli et al. 2023b Wen et al.. 2024; Chen et al. 2024) have focused on developing strategies to detect and mitigate memorization, achieving notable success. However, these metrics and strategies adopt a global perspective, com- paring entire generated images to training images. We identify significant gap in these approaches when dealing with cases where only parts of the training image are memorized, which we refer to as local memorization, as opposed to cases where the entire training image is memorized, termed global memorization. Specifically, current methods can be easily tricked in local memorization scenarios by introducing diversity in mon-meeriieed regions, resulting in many false negatives during evaluation

---

Published as a conference paper at ICLR 2025
INSIGHTS
Training Image
LAJRA
Ixtreaated Generoted
INSIGIPE
Image
WITH POmape
Mask
Figure I: Memorization in diffusion models can occur both globally (right) and locally (left). The first row displays the memorized training images, the second row shows the corresponding generated images, and the third row shows the local memorization masks extracted using "bright ending', where brightness values quantify the memorization effect. Most pixel values in masks the right brighter due to global memorization compared to masks on the left due to local memorization.
and detection. Consequently, this failure correctly identify local memorization may prevent the activation of mitigation strategies. Motivated by this research gap, we propose a novel view of the memorization issues, where we argue that improved metrics and strategies should concentrate exclusively on the locally memorized regions, as these areas pose risks. In contrast, unmemorized parts of the image, which do not present risks, should be completely ignored.
Building on this localization insight, we introduce new task: extracting localized memorized regions. This task is crucial for better understanding local memorization and developing targeted strategies to reduce the associated performance gap. To accomplish this task effectively and efficiently, we introduce the concept of the 'bright ending' (BE). a phenomenon observed in the cross-ate -tttsnttt maps of text-to-image diffusion models. The bright ending occurs when the end text token in the final inference step exhibits abnormally high attention scores on specific image patches in memorized generations compared to memorzedd ones. This distinctive pattern effectively highlights the regions where the model has memorized the training data, requiring only a single inference pass without relying on access the training data. To our knowledge, this is the first method achieve such precise and efficient localization, emphasizing the significance of BE. Finally, we propose simple yet effective method to integrate BE and its extracted local memorization masks into existing state-of tfiie--of-hh-aa art memorization evaluation, detection, and mitigation strategies. Extensive experiments demonstrate that this integration significantly enhances the performance of these existing tasks by narrowing the gap caused by local memorization, further underscoring the contribution of BE.
In summary, our contributions are three-fold: 1) We analyze the performance gap between local and global memorization in existing tasks and introduce novel localized view of memorization in diffusion models. 2) We propose new task of detecting localized memorization regions and propose the novel 'bright ending' (BE) phenomenon, which is the first method to successfully solve this task using only a single inference process without requiring access to training data and conducting nearest neighbor search over all ground truth images. 3) By integrating the local memorization mask extracted using BE into existing evaluation, detection, and mitigation strategies, we refine these approaches focus on local perspectives, in contrast to the global focus of all existing strategies. This refinement narrows the gap caused by local memorization and sets a new state-of-the-art in these existing tasks. This simple and effective incorporation further demonstrates the power of BE.
PRELIMINARIES
2.1 DIFFUSION MODELS
Diffusion models (Rombach al. 2022 Ho et al. 2020 Nichol & Dhariwal 2021 represent
state-of-the-art class of generative models, emerging as the predominant choice for various generation tasks. They entail two-stage mechanism, beginning with forward process that incrementally introduces Gaussian noise to an image originating from a real-data distribution q(x) across T timesteps, culminating .y AV(0,)); I):
( =N(xi; v 1,I)),
(1)
2

---

Published as a conference paper at ICLR 2025
Global Namorization
Training Imago
1.75
Non Memorization
LEMUNTE
1.50
"Unmemorized" "Memorized"
Generation
1.25
Densiyy
1.00
0.75
0,68
8hrr:: 0.87
Stmm: 0.55
Generatin
Sim: 0.89
8im: 0.87
Sir:: 0.38
Sim: 0.21
Sim: 0.44
Magnitude
Figure 2: For local memorization cases, variations in non-memorized regions significantly impact existing global similarity and detection measures. Left: Using the current similarity metric SSCD, the second row shows memorized images with similarity scores above 0.5, and the third row shows non-memorized images with scores below 0.5. However, the difference between the second and third rows is minimal, differing only in local portion, and are, in fact, both local memorization cases, which demonstrates the failure of the global SSCD metric. Right: Currently, magnitude is used to detect memorization, where higher magnitude corresponds to greater memorization risk. The distribution reveals noticeable performance gap: local memorization cases are more challenging to detect than global ones, as their magnitude values overlap more with those of non-memorized cases.
T II (())((1)))
where xe represents the version of zy with added noise at timestep f, and {B:)T= 3 is the noise schedule controls the amount of noise injected into the data each step. By using the reparameterization trick (Kingma & Welling 2014), we can derive the Diffusion Kernel, allowing for the sampling of æt any given timestep closed form:
q( (1o) = Na Varao. (1 a.).,,
where au = 1 3, and āi =
II'' Os. Subsequently, in the reverse process, generative model- ing is accomplished by training a denoiser network to closely estimate the true distribution of q(x2-1/<1, zo):
Po( -i(0 of
po(x):)) p(rr) II po(
(5)
where (z(z)) and oz are the approximated mean and variance of ((x;-10 |2,, zo), whose true values can be derived using Bayes rule and the Diffusion Kernel as in Eq. The learning objective simplifies having the denoiser network co predict the noise c, instead of the image 2t-1 at any arbitrary step t:
C = E.e ()
12
(6)
where the denoiser network can be easily reformulated as a conditional generative model eg (x,,, V, I) by incorporating additional class or text conditioning y.
3 A LOCALIZED VIEW OF MEMORIZATION IN DIFFUSION MODELS
Fig. [ iillustrates that diffusion models can exhibit both global and local memorization. Global memorization refers to the model remembering the entire image. whereas local memorization pertains to the model retaining only portion of the image. Notably, global memorization can be seen as a special case of local memorization when the memorized region encompasses the whole image.
3

---

Published as a conference paper at ICLR 2025
Global Memorization
Local Memorization
No Memorization
20.0
11.0
175-
12.5
12:51
75
2:5
50-
23
1033
232
60
1000
000
200
ARL
Generation
Training Data
Generation
Training Data
Generation
Training Data
20.0
30.
15.0
15.0
5
10.0
35
25-
1000
Time
Time stop
Time sics
Generation
Training Data
Generation
Training Data
Generation
Training Data
Figure 3: Local memorization tends to have smaller magnitudes than global memorization, making
harder to distinguish from non- memorization. Incorporating BE into the existing detection method effectively increases the magnitude of local cases while keeping the other cases mostly unchanged.
Existing global metrics fall short for local memorization evaluation. We observe that all existing studies memorization in diffusion models are from global perspective, where the effectiveness of any proposed detection methods and mitigation strategies are evaluated using global metrics, such the L2 distance SSCD scores (i.c., cosine similarity of embeddings extracted using the Self- Supervised Copy Detection (SSCD) method (Pizzi et al. 2022)). These metrics measure similarity between entire generated and training images rather than focusing However, these metrics are not robust to variations outside the locally memorized areas and can be easily tricked. Specifically, variations in non-memorized regions can significantly impact these similarity scores despite the similar local memorization regions. A generated image may memorize only local area of the training image while making the remaining areas novel and diverse, leading to false negatives. Fig. (lett) illustrates failures of the global metric approach, where variations in non memorized regions significantly reduce the similarity scores measured by SSCD. The current standard considers SSCD scores greater than 0.5 as memorization. This results in some images being falsely identified as non-memorized despite all being locally memorized, with differences only the non-memorized regions.
locally memorized areas.
Existing global strategies struggle with local memorization detection and mitigation. Current detection and mitigation strategies are evaluated using global metrics and are thus developed from a global perspective. For instance, /Wen al. (2024) distinguishes memorized cases from unmemorized ones by utilizing the magnitude over the global latent space (Fq. 7
D 1 T
Σ (Eo(I), (x co))
where ep represents the embedding of the user-provided prompt, es denotes the embedding of an empty string, E (z -) is the conditional latent-space noise prediction at time t, and D is the computed magnitude. The detection method identifies memorized cases by their larger magnitudes
4

---

Published as a conference paper at ICLR 2025
D compared to memommerreed cases. This approach surpasses previous methods, establishing the current of-te-of-the-art However, this approach is less practical for local memorization cases than for global ones, variations in monmmmeee regions impact magnitude computations, increasing variance and diminishing the reliability of the detection signal, which leads to more false negatives. To verify this theory, visualize the density plot of magnitudes for all three scenarios (global, local, and no memorization) in Fig. (right) and Fig. and observed that local cases indeed have smaller magnitudes than global ones, making them closer in magnitude unmemorized cases and harder to detect. Regarding mitigation, local memorization also underperforms. Wen et al. (2024) employs prompt engineering upon a positive detection, i.e., when D exceeds a threshold X, the generation is identified as memorized. Subsequently, ep is optimized via gradient descent until D S X, using the global magnitude D as the loss function. Consequently, this optimization process is also sensitive to variations in non-memorized regions. Additionally, the compromised global detection method, which produces more false negatives, often leaves the mitigation strategy untriggered, increasing the risk of unaddressed memorization and potentially leading to legal challenges.
The localization insight: local memorization is a more generalized and practical notion of memorization. Recognizing the gap in research on measuring, detecting, and mitigating local memorization, we propose that further investigations should adopt a local perspective. Local memo- rization is a more meaningful and generalized concept for the following reasons: (1) Unmemorized regions pose no litigation risk and can be disregarded, while even small locally memorized area can present significant legal concerns. (2) Local memorization is more encompassing definition, with global memorization being a specific instance. Focusing on local memorization does not diminish the value of global investigations; instead, it complements and extends the scope of existing work.
4 BRIGHT ENDING
Introducing a new task: extracting localized memorized regions. The observed performance gap caused by local memorization has prompted us explore new task: extracting localized memorized regions as a mask. This task is essential for investigating memorization from a local perspective, and the resulting mask can be integrated into existing evaluation, detection, and mitigation strategies. This integration refines these strategies into localized approaches, effectively reducing the performance gap associated with local memorization. One naive method to create such a local memorization mask is to compare generated images with the closest training image. However, this reliance on training data has following limitations: (1) compromises privacy for task aimed at preventing the memorization of training data and assumes that the user of the pre-trained model has access to the training data; (2) requires significantly more computation due to the need for nearest neighbor searches on the large LAION training dataset. Consequently, this naive approach diminishes its practical significance when the mask is used in detection and mitigation algorithms. Therefore, we aim to extract such masks without using training data, relying solely on the pre-trained model's s memory to identify distinguishing memorization patterns.
Recent work has highlighted the important role of prompts in causing memorization. For example, Somepalli et al. (2023b) identified that duplicated prompts during the training of text-to-image Stable Diffusion models make the prompts become the "keys" to the models' memory. In other words, when a model encounters a frequently seen prompt, it retrieves and generates images that closely resemble memorized training examples, much like a key unlocking stored content. Similarly, /Wen al. (2024) observed that prompts prone to memorization tend to result in larger text-conditional noise predictions during inference than non-memorization cases. Additionally, Chen et al.) (2024) successfully utilized classifier-free guidance to steer generations away from memorized prompts, improving mitigation. Inspired by these findings, we focus on using prompts to extract our desired local mask. This naturally led us to explore the cross- cotss-attettinn mechanism in the U-Net architecture within diffusion models, which links prompts to specific attention- fociusing areas in the generated images. One of our key intuitions is that the pre- tra-trained diffusion model's memories resulting from overfitting should differ from those learned through generalization. Memorized generations are inflexible and follow a fixed denoising trajectory that is observably different from non-memrri scenarios. This is evidenced by Wen et al. (2024). who observed that in memorized cases, the magnitude of rext-conditional noise prediction during the denoising trajectory is abnormally higher than in non non-memorization cases. Automatic local memorization mask extraction via bright ending (BE). Inspired by our intuition, we visualize each token's 's cross- attention maps in memorized and non-memorized prompts to identify distinguishing patterss (Fig. 4). We discover interesting phenomenon in pre-traine, text-to-image Stable Diffusion models: during the final denoising step, the end token typically shows a dark cross-attention map. Specifically, these dark maps often contain only extremely low-attention finer
5

---

Published as a conference paper at ICLR 2025
Global Mem
<jtartofeell
Bing
11
grohom
-jendoteat
Gerarason
Training
Local Mem
Quaatstedd
<lnddhoosip
No Mom
Generotion
Training Data
Figure 4: Visualization of cross-attention maps during the final denoising step in pre-trained Stable Diffusion models. Typically, the end token shows dark coss-aatenttoon map. shifting the denoiser's attention from semantic meanings to fine details. However, in memorized models, 'bright ending' anomaly occurs, where the end token displays abnormally high cross-attention scores, focusing on coarser structures, specifically, the local memorized regions, effectively serving an efficient automatic extraction of the local memorization mask without needing access to the training data.
Closs Coos- ettentin weights on <end>
0.0175
Method
0.0150
10
Base line
Ours
0.0125
Magnitude
0.0100
0.0050
2
0.0025
0.0000
Global
Non
Local
Global
Non
Local
Memorization Type
Memorization Type
Figure 5: Left: Box plot showing the distribution of end token cross-attention scores during the final inference step for different memorization types. The plot is based on 9,600 generated images, with 16 images generated for 300 memorized and 300 non-memorized prompts each. clearly distinguishes the three types: memomrerd cases have attention scores close to zero, while memorized cases exhibit abnormally high scores. Global memorization shows higher scores than local memorization due the larger memorized regions. This further validates the "bright ending' observation. Right: Violin plots displaying the impact on magnitudes (detection signal) for different memorization types.
edges or random scatter points. Since the end token summarizes all semantic meanings in the prompt, this indicates that when the noised image very close clean image, the denoiser's attention on text-conditioning shifts from focusing on its semantic meanings to concentrating only on fine details.
However, when the model exhibits memorization, we observe an anomaly where the end token displays abnormally high cross-attention scores at the end of denoising steps phenomenon we refer to as the 'bright ending' (BE). Specifically, it focuses on the coarser structure of image patches rather than finer details random scatter points. This indicates that the denoiser remains focused on text-conditioning even during the final inference step, demonstrating overfitting to the text-conditioning. This aligns with the finding in Somepalll et al.) (2023b) that memorized generations result from overfitting to repeated captions used as conditioning signals for text-to-image generation during training, thereby following a fixed noise trajectory during inference. Thus, BE is robust indicator for differentiating memorized generations from unmemorized ones, with bright regions identifying the memorized areas. This effectively functions the BE attention map as an automatic memorization mask, acting as a global mask during global memorization and local mask during local memorization, as can be visually observed in Fig. and To further validate this observation, we used 300 memorized prompts from Webster (203) and 300 non-memorized prompts to generate 16 images for each prompt, resulting in a total of 9600 generations. We then employ a box plot (Fig. 5) to visualize the distribution of the attention scores of the end token during the final inference step for these generated images. We observe a clear distinction among the three memorization types: non-memorized cases consistently had attention scores close to zero, while memorized cases exhibited abnormally high scores. Furthermore, global memorization showed higher scores than local memorization due to the larger regions being memorized in global cases compared to local ones.

---

Published as a conference paper at ICLR 2025
Beyond BE's effectiveness in performing the new task of extracting localized memorized regions, it is also remarkably efficient, requiring only a single inference pass without needing access to the training data. To our knowledge, BE is the first method to achieve such precise and efficient localization.
5
LOCALIZED DETECTION, MITIGATING AND EVALUATION METHODS
5.1
INCORPORATING THE BRIGHT ENDING (BE)
BE's success in extracting localized memorized regions can also benefit existing tasks of evaluating. detecting, and mitigating memorization in diffusion models. By integrating the extracted local mask into current strategies, these approaches can adopt a local perspective, thereby reducing the performance gap associated with local memorization. Specifically, as discussed in Sec. F the bright ending (BE) attention map inherently serves as automatically extracted memorization mask. This mask directly utilizes the BE attention scores as its values, eliminating the need for thresholding to convert it into binary mask of only Os and 1s. By doing so, it avoids the added variability introduced by thresholding hyperparameters, which may be less effective under certain threshold settings, thereby improving robustness. The approach remains effective regardless of whether the memorized regions are subtle or abstract, as it solely relies on the pre-trained model's overfitted memory during memorization extract the mask. This eliminates the dependence on factors such as the degree of subtlety or abstraction of memorized regions or thresholding based on size or intensity of memorization. For a qualitative evaluation of the mask, please refer Fig. and Fig. 4 where brighter patches indicate higher attention scores and darker patches indicate lower ones.
Detection strategy. As discussed in Sec. 3 computing magnitudes in the global space, shown in Eq. 7 can be misleading during local memorization. To ensure that the magnitude computation corresponds solely to the locally memorized area and remains invariant other areas, propose element-wise multiplication of the magnitude by our memorization mask extracted via BE:
N
LD
co(2,, eb)) =e(lt,
m
m
(8)
T
N
Here, N is the number of elements in the mask m. Unlike methods requiring hyperparameter tuning, such as setting thresholds to convert attention scores into a binary mask, our approach uses the attention scores directly as weights. This reweights the relative importance of each pixel in the magnitude computations. To ensure comparability, the result is normalized by the mean of the attention weights m.
Mitigation strategy. Also, as discussed Sec. 3 the baseline mitigation strategy employs prompt engineering when a positive detection outcome identified, using global magnitude as the loss function for such prompt optimization. Our proposed BE introduces two key enhancements to this process: (1) Accurate trigger: The mitigation strategy is triggered more accurately due to our improved detection method, which utilizes masked local magnitude by incorporating BE. (2) Improved loss function: By using masked local magnitude as the loss function, we provide more effective gradient information during backpropagation for prompt optimization, further enhancing the efficacy of the mitigation strategy, again thanks to the incorporation of BE.
Evaluation strategy. To verify the effectiveness of the bright ending when used for evaluating metrics, we design localized similarity metric, denoted as LS, to measure the similarity between generated image î and training image æ. This metric is a masked version of the L2 distance and is combined with the standard SSCD metric, defined in Eq. 9 We expect this approach to outperform the sole use of SSCD and the combination of SSCD with the original global 1.2 distance (denoted as S), as in Eq. 10
LS(i,x) 11---lscoon ISCD-<o.. [|@ x:) ml|2
5(ē,x) = LsscD<os [|( - 2)112
(10)
5.2 EXPERIMENTS
Setup. Following previous works, we conducted experiments on Stable Diffusion v1-4. We adhered to the baseline prompt dataset (Wen et al. 2024), using 500 prompts each from Lexica, LAION, COCO, and random captions as non-memorized prompts. We used the dataset organized by Webster (2023) for memorized prompts. However, since not all 500 prompts in Webster (2023)'s 's dataset are prone to memorization, we selected 300 memorized prompts for our experiments. For each memorized and mon-memorieed prompt, we generated 16 images.
7

---

Published as a conference paper at ICLR 2025
We follow the baseline methodology from Wen et al. (2024) and evaluate performance using the area under the curve (AUC) of the receiver operating characteristic (ROC) curve, the True Positive Rate at 1% False Positive Rate @T@ 1%F), and the Fl score, and report the detection performance during the 1, 10, and 50 inference steps. However, the baseline detection performance measures whether a prompt is prone to memorization rather than if specific generation memorized. They experimented with how well the average magnitude of 1, 4, and 32 generations per prompt differentiates between memorized and non-memorized cases. This approach is flawed because using different random seeds, prompt prone to memorization can sometimes generate non memoized images. Consequently, a prompt that rarely yields memorization might still be classified as memorized. For example, generating four non-memorized images from prompt classified as memorized due to its occasional memorized output can falsely inflate performance metrics. Conversely, failing to classify these as non-memorized when they are all non-memorize should not be penalized. To address this, we assess detection performance based on accurately predicting each generation as memorized or not rather than each prompt. We re-implement the bascline method under the same conditions for fair comparison. For mitigation, the output utility is evaluated based on the CLIP score, which measures the text-alignment of generated outputs. We evaluate similarity scores using both traditional SSCD and our proposed localized metric to provide comprehensive performance view, experimenting with multiple privacy levels. Due to current metrics' failure accurately identify memorized cases, we manually label the ground truth for each generated image from the memorized prompts. Websstrr (2023) organizes the nearest training images for these generations, making easy to observe local memorization cases with minimal subjectivity in manual labeling. Please check out Sec. 9.3 for detailed illustration of the labeling process.
We separately evaluate performance for local and global memorization, contrasting our localized strategies with the baseline's globalized strategies. Table [ presents a comparison between our localized detection strategy, which uses LD as the detection signal (Eq. [8). and the baseline globalized detection strategy that relies on D (Eq. (7). Figures and 7 highlight comparisons of localized mitigation, utilizing LD (Eq. 8 as both the triggering signal and loss function, with globalized mitigation strategies that use D (Eq. 77.. Finally, Table 2 compares the localized similarity metric LS (Eq. 9 against S (Eq. 10 and SSCD in terms of Fi-score, using memorization thresholds. For SSCD, we follow the standard threshold of 0.50, while for LS and S, we adopt a threshold of -50.
Implementationallly, we experimented with the cross attsntattnnionn maps different layers of the U-Net and found that averaging the first two pixe downsampling layers most effectively extracts the local memorization mask. The inference process takes about 2 seconds per generation using RTX4090.
Table 1: Detection performance for local and global memorization at different inference steps T'.
=
T T' 10
=
50
AUC
F1
T@I
AUC
F1
T 1SF
ALC
F1
TØ 1%F
Baseline (D) Local
0.918
0.864
0.629
0.989
0.982
0.953
0.990
0.983
0.560
Ours (I.D) Local
0.893
0.731
0.995
0.987
0.985
0.996
0.988
Baseline Global
0.979
0.944
0.934
1,000
0.987
1,0000
0.999
0.976
1,000
Ours (I.D) Global
0.981
0.948
0.940
1.000
0.987
1.000
0.999
0.977
1.000
0.75
Method
Method
Method
Stast D grosi
Methed Swee
Stade Ditoitr
Similarit Seare (SSCD)
0.85
0.75
Basoling
Ours 3UD0
Simlaarity (1.5)
-13
Busoling (2)
Ours 3.D0
Stmitortty Score (S5CD)
2.7
2.8
Basni (D)
(LD)
Du's (LD)
0.62
2.6
0.35
355
-20
0.53
0.45
2.3
0.40
CLIP Score
CLIP Score
CLIP Score
CLIP Score
Figure 6: Local memorization's 's mitigation.
Figure 7: Global memorization's mitigation.
Results. As the baseline detection strategy in Eq. 7 incorporates hyperparameter T, where the detection signal is defined as the average magnitude over T inference steps, Tab. [ evaluates performance across different values of T' This evaluation compares the baseline's 's magnitude with our proposed masked/localizes magnitude, incorporating the BE mask as outlined in Eq. [ under such different choices of baseline's 's hyperparameter values for demonstrating BE's ablation effects.
8

---

Published as a conference paper at ICLR 2025
The results in Tab. | consistently demonstrate that integrating the BE mask improves detection performance under varying baseline hyperparameter settings across all metrics for local memorization while maintaining the performance for global memorization. These findings validate (1) the BE mask's effectiveness in accurately identifying memorized regions and (2) the importance of adopting a localized perspective for studying memorization, which remains under-explored. Additionally, Fig. and (Right) further demonstrate this targeted improvement in local memorization. The magnitudes for non-memorization and global memorization remain almost unchanged, but the magnitudes for local memorization are successfully amplified, making them more distinguishable from non- memorization cases. This effectively reduces the performance gap caused by local memorization by converting previous False Negative cases into True Positives (examples shown in Fig. 8)
For mitigation, we also include the original Stable Diffusion (SD) without any mitigation strategies applied in Fig. aand for comparison, illustrating the extent to which we have reduced the local memorization performance gap. The horizontal axis (CLIP score) represents utility, while the vertical axis (similarity score) reflects privacy risks. More favorable privacy-utility trade-off's are indicated by points further toward the bottom right. This placement signifies lower privacy risks for the same utility level (same x-axis value) and higher utility for the same level of privacy risks (same y-axis value). Specifically, for local memorization, under the high-utility scenario where we aim to minimally reduce the CLIP score while mitigating memorization, our approach achieves significantly better trade-off than the baseline. We reduce SD's s similarity score by twice as much as the baseline while simultaneously observing a much smaller reduction in the CLIP score less than half compared the baseline. We also tested different privacy levels and consistently achieved a better tradc-off for local memorization. Specifically, we observe lower similarity score for the same CLIP score, as measured by both SSCD and our localized metric. Additionally, we consistently observe a higher CLIP score for the same similarity score. As expected, the results for global memorization remain almost identical to the baseline. This further confirms BE's effectiveness in addressing the performance gap introduced by local memorization the task of mitigating memorization.
For evaluation metric, Tab. 2 demonstrates the superiority of our localized metric, LS (Eq. 9. which integrates SSCD, L2, and our BE mask, over the globalized metric, S (Eq. (O). which uses only SSCD and L2. This highlights the contribution of incorporating the BE mask. Furthermore, the results underscore the limitations of solely relying on SSCD, which performs well for global memorization but struggles with local cases. Notably, our design of LS, which incorporates SSCD, retains strong performance for global memorization while addressing the performance gap in local memorization, further validating the benefit of such a design.
It is worth noting that the effectiveness of these strategies not only demonstrates our improvement over the current state-of-the-art in existing tasks after incorporating BE's local memorization mask, but also serves as an extrinsic validation of BE's effectiveness in extracting local memorization masks. To our knowledge, BE is the first method to precisely extract local memorization masks in highly efficient manner, requiring only one inference pass without the need to query the training data.
Table 2: Fl-score comparisons for dif- ferent evaluation metrics.
Local
Global
SSCD
0.940
1.000
S
0.991
1.000
I.S (Ours)
0.995
1.000
Figure 8: Examples of turning FN to TP via mitigation strategy. Generated image followed by training image.
6 RELATED WORK
Memorization in Diffuusin Models Numerous studies have previously explored memorization
phenomena in language models (Carlini et al. 2021 Kandpal ct al. 2022a by Lee et al., 2022) and GANs (Tinsley et al. 2021 Arora al. [018]] Heusel et al. 20. Over the past year, memorization within diffusion models has increasingly been scrutinized. Somepalli et al. (2023a) and Carlini ctt al. (202)) concurrently pioneered the examination of this issue, discovering that pretrained text- conditional models, such as Stable Diffuusion, frequently replicate their training data. Similar findings were observed for unconditional generations in pretrained DDPMs on CIFAR-10 (Krizhevsky 2009), well as DDPMs trained on smaller datasets like CelebA (Liu al. 2015) and Oxford Flowers (Nilsback & Zisserman 2008). Additionally, Chen et al.) (2024) explored class consiconniiio DDPMs CIFAR-10, further highlighting the significance of this issue.
9

---

Published as a conference paper at ICLR 2025
categorizes memorized captions into three types: (I) Matching Verbatim (MV), where the generated image using a memorized caption is an exact replica of the corresponding image; (2) Retrieval Verbatim (RV), where the generated image using memorized caption replicates other training images rather than the corresponding one; and (3) Template Verbatim (TV), where the generated image using memorized caption replicates only the template shared by a subset of training images, with variations in fixed locations outside the template, corresponding to different captions. This categorization focuses on the causes of memorization: MV results from overfitting the one-to-one mapping between image and caption in the training set, while many-to- many mappings lead to RV and TV. However, the concepts of Local Memorization (LM) and Global Memorization (GM) differ, focusing on whether the generated image exactly replicates an entire training image or just a portion of it, regardless of the causes of memorization. RV and TV can be seen subsets of LM, as there are additional causes of LM beyond the "many-to-many mapping overfitting" " that remain unidentified. For instance, in Fig. 8 the image in the third column is classified as MV by Webster (2023) due to the one-to-one mapping of the training image-caption rather than many-to- tto-many. However, it only memorizes local portion of the image in the last column. This suggests that a pre-trained model may occasionally exhibit creativity when recalling overfitted memories, which can also lead to local memorization.
Types of Memorization Webster
(2023)
Detection and Mitigation Strategies Carlini et al. (2023) detects memorization by calculating the Euclidean distance between generated images and training images, recommending re-training
on de-duplicated data mitigation strategy Daras et al. (2023) suggests mitigating memorization
by training models on corrupted data. (Someaali et al. (2023b) proposes to mitigate memorization by injecting randomness into text prompts, such as adding random tokens or noise, to break the link between memorized prompts and their corresponding images. [Chen et al. (2024) employs a nearest neighbor search, similar to Carlini al. (2023), identify potential memorized outputs and then selectively applies guidance methods to instruct these generations away from the memorized training images, demonstrating outstanding mitigation performance. Recently, Wen et al. (2024) proposed an efficient method for detecting memorization during the inference phase of diffusion models. This approach uses the magnitude of text-conditional predictions for prompt detection, enabling high accuracy from the initial inference step, and triggers a mitigation strategy to optimize the prompt embedding when memorization is detected. Both the proposed detection and mitigation strategies have achieved state-of-the-arr results. A most recent work (Ren et al. 2024) utilized cross-attention patterns to design detection and mitigation strategies, achieving performance comparable to Wen et al. (2024). The key insight is that the cross atteetiin scores for memorized prompts are less concentrated across all tokens in a prompt during multiple inference steps. This lack of concentration is quantified using entropy, which captures the distribution of cross-attention the prompt-level. Whhiellllllleee both leverage cross-attention, it is important to note that our bright ending cross-attention (BE) differs from the cross-altention entropy approach regarding objectives, insights, inputs, and the final computed statistic. Specifically, different from Ren et al. (2024)'s 's objective of leveraging cross-attention for designing detection and mitigation strategies, BE also aims to achieve a new task of precisely and efficiently locating localized memorization regions. Also, BE does not rely on the concentration patterns of all tokens during multiple steps as the insight and inputs but instead uses the raw final step attention score of the end token. Also, BE employs the raw image-patch level distribution rather than the prompt-leva distribution as the computed statistic, serving as a local memorization mask.
7 CONCLUSION
In this paper, we identify performance gap in existing tasks of measuring, detecting, and mitigating memorization in diffusion models, particularly in cases where the generated images memorize only parts of the training images. To address this, we introduce new localized view of memorization in diffusion models, distinguishing between global and local memorization, with the latter being a more meaningful and generalized concept. To facilitate investigation from a local perspective, we propose a new task: extracting localized memorized regions. We propose leveraging the newly observed 'bright ending' (BE) phenomenon to precisely and efficiently accomplish this task, resulting in a local memorization mask. Through extensive experiments, we demonstrate that the mask generated by this new task can also be used to enhance performance in existing tasks. Specifically, we propose a simple yet effective strategy to incorporate the BE local memorization mask into existing frameworks, refining them to adopt local perspective. This approach reduces the performance gap associated with local memorization and achieves new state-of-the -het---rr-e art results. The improved results not only highlight our improvements in existing tasks but also serve as extrinsic validation of BE's effectiveness in the new task of local memorization mask extraction, underscoring the significant contribution of BE.
10

---

Published as a conference paper at ICLR 2025
8 ACKNOWLEDGEMENT
This work was supported in part by the Australian Research Council under Projects DP210101859 and FT230100549.
REFERENCES
Sanjeev Arora, Andrej Risteski, and Yi Zhang. Do gans learn the distribution? some theory and empirics. In International Conference on Learning Representations, 2018.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagiclski, Ariel Herbert- Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In USENIX Security Symposium, 2021.
Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Schwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In USENIX Security Symposium, pp. 5253-5270, 2023.
Chen Chen, Daochang Liu, and Chang Xu. Towards memorization-fre diffusion models. In CVPR, 2024.
Giannis Daras, Kulin Shah, Yuval Dagan, Aravind Gollakota, Alexandros G Dimakis, and Adam Klivans. Ambient diffusion: Learning clean distributions from corrupted data. In Advances in Neural Information Processing Systems, 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Berharr Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge a local nash equilibrium. In Advances in Neural Information Processing Systems, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, volume 33, pp. 6840-6851, 2020.
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. Large language models struggle to learn long-tail knowledge. arXiv preprint arXiv:2211.08.. 2022a.
Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, 2022b.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. 2014.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Katherine Lee, Daphne Ippolito. Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In Proceedings of the Association for Computational Linguistics, 2022.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015.
Alex Nichol and Prafulla Dhariwal. Improved denoising diffussin probabilistic models. In ICML, 2021.
Maria-Flena Nilsback and Andrew Zisserman. Automated flower classification a large number of classes. In Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing, pp. 722-729, 2008
Fd Pizzi, Sreya Dutta Roy, Sugosh Nagavara Ravindra, Priya Goyal, and Matthijs Douze. A self- supervised descriptor for image copy detection. In CVPR, pp. 14532-14542, 2022.
Jie Ren, Yaxin Li. Shenglai Zeng, Han Xu, Lingjuan Lyu, Yue Xing, and Jiliang Tang. Unveiling and mitigating memorization in text-to-image diffusion models through cross attention. arXiv preprint arXiv:2403. 1052, 2024.
11

---

Published as a conference paper at ICLR 2025
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pp. 10684-10695, 2022.
Joseph Saveri and Butterick Matthew. Stable diffusion litigation, 2023. 2023. wtw.blleddffu.....iiii.....a.a.e
URL
https:
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models. 2022.
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication diffusion models. In CVPR, pp. 6048-60885 2023a.
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Under- standing and mitigating copying in diffusion models. In NeurlPS, 2023b.
Patrick Tinsley, Adam Czajka, and Patrick Flynn. This face does not exist... but it might be yours! identity leakage generative models. In IEEE/CVF Winter Conference on Applications of Computer Vision, 2021.
Ryan Webster. A reproducible extraction of training images from diffusion models. arXiv preprint arXiv:2305.08669, 2023.
Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan Lyu. Detecting, explaining, and mitigating memorization diffusion models. ICLR, 2024.
12

---

Published as a conference paper at ICLR 2025
9 APPENDIX
9.1 LIMITATIONS AND FUTURE WORK
One limitation of using the memorization mask extracted through the bright ending phenomenon in tasks like detection and mitigation is that it can only be observed at the end of the inference process. This requirement results in a longer processing time than existing strategies when applied existing tasks such as detection and mitigation. However, we believe the additional time is justified by its significant positive contributions to investigating local memorization in diffusion models. Moreover, each inference takes only few seconds on a consumer- GPU, making the process practical for real-world applications (elaborated in Sec. 9.2}. While existing tasks are well-studied with established evaluation criteria, future work could explore how to evaluate the performance of the newly proposed local memorization region extraction task. Our current approach uses an extrinsic evaluation method that demonstrates improvements by incorporating the mask into existing strategies. Future research could also consider an intrinsic evaluation by directly comparing the extracted mask with ground truth masks. Another future direction could involve developing more advanced evaluation metrics for local memorization. In our work, we employed a straightforward approach by combining a masked version of the L2 distance with the widely adopted SSCD to compute similarity. Future research could focus on creating methods similar to SSCD, trained in a self-supervised manner but specifically tailored for more accurate localized copy detection.
9.2 EFFICIENCY OF THE BRIGHT ENDING (BE) APPROACII
The Bright Ending (BE) method demonstrates significant efficiency in its dual capabilities: localiza- tion of memorization regions and integration into existing memorization strategies.
Localization Efficiency. BE is the first method to accurately localize memorization regions in generated images without requiring access to training data or conducting computationally expensive nearest-neighbor searches over large datasets. Instead, uses single inference pass, which takes only a few seconds on consumer-grade GPUs. This streamlined approach ensures practicality for real-world applications and scalability for various scenarios.
Integration into Existing Strategies. In addition its standalone capabilities, BE integrates seamlessly into existing memorization detection, mitigation, and evaluation frameworks, cnabling ofte--of-theaar performance improvements. While this integration does require additional inference pass (e.g., 50 denoising steps, approximately 2 seconds on an RTX-4090) to extract the BE mask, the trade-off is well justified by the method's significant contributions to addressing local memorization challenges.
Insight into BE's Efficiency. The BE phenomenon is rooted in the distinct behavior of the end token's cross-attention map during the denoising process. Non-memorized generations typically exhibit low-attention patterns at the end token in later steps, reflecting a shift from semantic focus to fine-detail refinement. In contrast, memorized generations follow an overfitted denoising trajectory, resulting in a distinct anomaly: the "bright ending', which can be easily identified.
While the final-step BE is currently used for its robustness and clear anomaly distinction, leveraging BE at earlier inference steps is also feasible. Earlier-step BE could potentially reduce computational costs further, as the anomaly is observable throughout the denoising process, albeit less pronounced. Future work could explore the use of earlier-step BE to optimize the efficiency of the approach further.
Justification for Final-Step BE. The decision to focus on the final-step BE is supported by its demonstrated practical contributions and significant improvements the detection, mitigation, and evaluation of local memorization. Despite the marginal increase in computational cost, the benefits in accuracy and utility outweigh the overhead. The final-step BE ensures that the method remains robust and effective while providing reliable foundation for further advancements.
9.3 ILLUSTRATION OF THE LABELING PROCESS
As discussed in Sec. 5.2 current metrics often fail to accurately identify memorized cases, as shown in Fig. Therefore, thresholding their reported similarity scores is not reliable for categorizing
13

---

Published as a conference paper at ICLR 2025
generations into memorized and non-memorized cases to establish the ground truth for evaluating detection strategies. To address this limitation, we manually label the ground truth for each gener- ated image. This section provides additional details about the manual labeling process ensure reproducibility.
First, download all ground truth images from Wen et al. (2024) 's official GitHub repository, where the data is systematically organized according to the memorization prompt dataset introduced by Webstter (2023). Specifically, for each memorized prompt, the repository includes sub-folders containing training images prone to memorization. This may include a single image when the prompt consistently generates a unique replicate, or multiple images when the prompt leads variations of replicated outputs. We then compare each generated image conditioned on a memorized prompt with the corresponding training images in the sub-folder. Each generation labeled as one of three categories: global memorization, local memorization, or non-memorization. Examples of such sraniinggenneation image pairs used for labeling are shown in Fig. g The straightforward nature of this process enables quick and accurate labeling.
14

---

Published as a conference paper at ICLR 2025
Thaining
Generszet
Mem Type
Global
Global
Global
Globel
Global
Global
Global
Global
Prompt: TTTThe Long Dark</>> Geta First Trailer, Bteem Early Access"
Trstning
imaate
LAMANT
Genersled Imauge
Mom Type
Locsl
Local
Local
Local
Losal
Local
Local
Local
Prompt: "Meditation Ploor Pillow"
Trarring Irunge
Generalent Irage
dad
OK
Mem Type
Global
Non
Global
Non
Global
Non
Global
Global
Prompt: "Preddy Adu Signs For Yet Another Club You Probabiy Don't Know
Thatning Imaade
Gerersied Imugçe
Mema Pype
Locel
Non
Non
Local
Local
Local
Local
Local
Tark Gray
Bug by Mills"
Treining
Inage
Oenoraled Imaage
Mim Type
Local
Iaual
Local
Lowal
Loxal
Loual
Lowal
Loxal
Prompt: "lymouth Panel feakuring Madelyn
Botanical Floral by bestherdutzon"
Figure 9: Examples of sonning-enneration image pairs used for labeling memorization types.
15


Reviews:
[{"review_id": "OO50y5ufu6", "reviewer": "Reviewer_ve5E", "summary": "This paper is a followup work of Wen et al. (2024), which discovered interesting patterns that \"abnormally high predicted noise magnitude\" indicate \"global memorization\". This work finds similar patterns between \"abnormally high cross-attention values\" of the EOS token in the prompt with the image tokens indicate \"local memorization\". The experiments validate such patterns can indeed be used to find local memorization.", "questions": "1. AFAIK there's no dedicated EOS token used by stable diffusion. Instead, there are only 77-N padding tokens which are almost identical to each other (except for being added with different positional encodings). Suppose the prompt contains 15 tokens, then there will be 77-15=62 padding tokens which have identical token embeddings. I believe the cross-attention maps of these tokens are highly similar to each other. Do you compute the cross-attention map by averaging these 77-N attention maps? Or do you choose the last padding token to extract the cross-attention map?\n\nEDIT after author response:\nApparently I got it wrong. There's indeed an EOS token inserted before padding tokens.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The proposed idea is simple and is a natural extension of Wen et al. (2024).\n2. The occurrence of \"local memorization\" has important practical values but is under-explored.", "weaknesses": "1. In multiple places, the authors state that the \"bright ending\" is obvious at the final denoising step. However, in the experimental results presented in Table 1, the 3 designs were using \"First Step\", \"First 10 Steps\", and \"All Steps\", respectively. Only \"All steps\" include \"the final denoising step\", but apparently \"the final denoising step\" plays only a minor role in \"All steps\". Why this discrepancy?\n2. In section 5.1, \"we propose element-wise multiplication of the magnitude by our memorization mask extracted via BE\". However, the authors didn't give details about how to extract a memorization mask from BE in Section 4. Is this done using a fixed threshold? a dynamic threshold? Or a threshold predicted by a dedicated NN?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dhY0zqx5PP", "reviewer": "Reviewer_PxZo", "summary": "This paper presents a problem with existing techniques to detect memorization in diffusion models in their lack to detect instances of memorization localized to specific subsections of generated images. To resolve this problem, they propose a new technique that creates a memorization mask for a generation.  The technique is based on a correlation between high attention values and memorization that the authors observe at the end of the diffusion process.", "questions": "1. What is the motivation for incorporating SSCD similarity into the metric? Why not just use some function of the mask itself to generate a score?\n2. Could you please share more details about the manual labeling process and the dataset itself?\n3. Will you share the code implementing your method?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The problem presented by the authors is compelling, and they demonstrate failure cases of existing techniques in the literature\n2. The authors propose a novel and effective method to address the problem\n3. The proposed method seems to solve this problem while also maintaining the success cases of earlier techniques", "weaknesses": "1. Some of the figures and text are somewhat difficult to understand, especially relating to the method itself. For example, Table 1 and Figs 6/7 are not explained thoroughly enough, even though they seem to be central results for the paper. Section 5.1 could also benefit from further elaboration to ensure that the method and its formulation are clear.\n2. The evaluation is not well explained in my opinion. For example, more details about the manual labelling process will make the results more reproducible.\n3. Sharing the code and the dataset will help making the method more reproducible", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QVFpJyLdT7", "reviewer": "Reviewer_8TzW", "summary": "This paper introduces a novel perspective on memorization in text-to-image diffusion models by focusing on local memorization, where specific regions of a training image are memorized rather than the entire image. It identifies a unique \"bright ending\" (BE) anomaly in cross-attention patterns, where memorized regions show higher attention to the end token during the final denoising step. This insight enables the extraction of localized memorization regions without access to training data. The paper proposes a new task—locating these localized memorization regions—and integrates this approach with existing evaluation, detection, and mitigation strategies. The paper demonstrate that incorporating BE significantly improves performance in existing tasks, particularly in narrowing the gap caused by local memorization, achieving state-of-the-art results.", "questions": "How does the proposed BE method perform in scenarios where memorization is more subtle or occurs in highly abstract image regions? Is there a threshold of memorization size or intensity where BE becomes less effective?\n\nIs it possible for the method to be generalized to other types of generative models? Can the BE-based method be generalized to generative models that do not rely on text conditioning, such as unconditional image generation models? Can it be applied to video generation?\n\nGiven the increased inference time due to analysis at the final denoising step, do the authors have suggestions for optimizing the process to make it more efficient for real-time applications?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper introduces the concept of \"bright ending\" as a novel phenomenon in diffusion models, which has not been explored in prior research. This unique observation enables a new task of localized memorization detection, which broadens the scope of understanding memorization in generative models.\n\nThe paper is good written. The paper clearly differentiates between global and local memorization, offering detailed explanations of how BE is used for local region detection. Figures and visualizations of attention maps provide clarity on how the BE mechanism operates in practice, enhancing understanding of the proposed method.\n\nThe motivation of the paper is sound. It has implications for legal and ethical considerations, especially in cases where models might unintentionally replicate copyrighted content. The findings are particularly relevant for improving privacy-preserving techniques in model training and inference.", "weaknesses": "The efficiency of the proposed approach needs to be further demonstrated. The BE-based approach requires analysis at the end of the inference process, making it slower compared to some existing global methods that might detect memorization earlier in the process. \n\nWhile the BE phenomenon is well-documented for text-to-image diffusion models, it is not clear how well this method would generalize to other types of diffusion models, like recently the flow-matching based models.\n\nMore evaluation metrics for local memorization could be explored for a comprehensive study.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)