Image-level Memorization Detection via Inversion-based Inference Perturbation

Authors: Yue Jiang, Haokun Lin, Yang Bai, Bo Peng, Zhili Liu, Yueming Lyu, Yong Yang, Xingzheng, Jing Dong
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Text-to-image diffusion model, data memorization detection, DDIM Inversion

Abstract:
Recent studies have discovered that widely used text-to-image diffusion models can replicate training samples during image generation, a phenomenon known as memorization. Existing detection methods primarily focus on identifying memorized prompts. However, in real-world scenarios, image owners may need to verify whether their proprietary or personal images have been memorized by the model, even in the absence of paired prompts or related metadata. We refer to this challenge as image-level memorization detection, where current methods relying on original prompts fall short. In this work, we uncover two characteristics of memorized images after perturbing the inference procedure: lower similarity of the original images and larger magnitudes of TCNP.
Building on these insights, we propose Inversion-based Inference Perturbation (IIP), a new framework for image-level memorization detection. Our approach uses unconditional DDIM inversion to derive latent codes that contain core semantic information of original images and optimizes random prompt embeddings to introduce effective perturbation. Memorized images exhibit distinct characteristics within the proposed pipeline, providing a robust basis for detection. To support this task, we construct a comprehensive setup for the image-level memorization detection, carefully curating datasets to simulate realistic memorization scenarios. Using this setup, we evaluate our IIP framework across three different memorization settings, demonstrating its state-of-the-art performance in identifying memorized images in various settings, even in the presence of data augmentation attacks.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730418333536

thread_type:
official_review

<thread_object>
creator: reviewer_8dpk

<content>
summary:
The paper introduces a method for image-level memorization detection. It is motivated by an analysis showing two key characteristics of memorized images under perturbed inference: similarity discrepancy and a large magnitude of text-conditional noise prediction. The key idea of the method seems to be using an unconditional DDIM inversion to derive latent codes and optimizing non-memorized prompt embeddings for effective perturbation. The method is evaluated on a number of datasets, showing strong ability to detect memorized images.

strengths:
* The results are strong and the method seems reliable in detecting memorized images
* There is an analysis section to motivate the approach and explain why it may work
* The paper is generally well-written and easy to read

weaknesses:
* The prompts appear to be an input to the optimization procedure that generates the optimized prompt embedding, so even if the prompts are not directly paired with the images, we may not be able to say the method does not rely on prompts (as may seem from some parts of the paper). This can be discussed more to explain to what extent we do not need access to the original prompts.
* All settings are based on Stable Diffusion v1.4: while this is a common model for image generation, one would like to see if the same method works across different models. There are currently various image generation models available, so it would be good to try the method on more - e.g. two more - to see if this method generalizes more broadly.

Minor:
* Ori prompt would be easier to understand if it was written as original prompt or orig. prompt

questions:
* To what extent are the perturbed prompts related to the original prompts? Can we really say we do not use the original prompts in any way? E.g. if we use something derived from them
* How expensive is it to run the method for an image to test if it is memorized?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730622159858

thread_type:
official_review

<thread_object>
creator: reviewer_dgyc

<content>
summary:
The authors propose an algorithm named IIP to detect memorized images generated by LDMs. The algorithm features two observations during DDIM generation with perturbed prompts (1) a notable similarity discrepancy (2) a large magnitude of text-conditional noise prediction.

strengths:
(1) The observed phenomenon seems to be universal and statistically notable.

(2) Within the experiment setup of this paper, the algorithm shows promising performance.

(3) The presented figures are illustrative and depictive.

weaknesses:
**Major**

(1) The biggest concern I have about this paper is its high overlap with existing work [1], making its technical and empirical contribution weak. I shall support my claim with the following facts: (a) The metric TCNP/MTCNP is proposed by [1], which also applies the metric to memorization detection. (b) Perturbing the textual embedding during DDIM generation is already used by [1] to mitigate memorization. (c) minimizing the magnitude of text-conditional prediction is already used by [1] to achieve the perturbed prompt embedding. It wasn't until I carefully re-read [1] that I noticed so much overlap between the proposed algorithm and an existing work. I believe the authors fail to give enough credit to the existing work, as all this necessary information is not shown in the submitted manuscript.

(2) The authors tend to use big words like "SOTA" and "pioneer". However, I am not fully convinced by the importance of the "image-level detection" setting proposed by the authors. If we can detect memorization as early as in the generation phase like [1], why would we bother doing the "image-level detection"?

[1] IMAGE-LEVEL MEMORIZATION DETECTION VIA INVERSION-BASED INFERENCE PERTURBATION; https://openreview.net/pdf?id=84n3UwkH7b

**Minor**

(1) Only SD v1.4 is considered, which is not enough because the performance of the proposed algorithm might be highly affected by the architecture, the training data, and the sampling configuration of the LDMs.

(2) Missing period in line 181 "during the subsequent generation process [period here] Interestingly..."

questions:
Please see the weakness part.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730662881665

thread_type:
official_review

<thread_object>
creator: reviewer_jfcj

<content>
summary:
The paper presents a new method for identifying whether an image was part of the diffusion model training set, without access to the prompt used for generation. As far as the authors know, they are the first to identify this task. To this end - inference under prompt perturbations is examined for memorized / not memorized samples, with the aid of DDIM's inversion capabilities (specifically unconditional inversion to avoid dependency on the prompt). Under this process, they find two key properties that differentiate between memorized and non-memorized samples. Building on their insights, they propose a Inversion-based Inference Perturbation (IIP) -  a novel method for the task at hand and out-perform the competitors on an extensive test suite.

strengths:
1. The results present SOTA performance with respect to the defined task and the evaluated existing methods.

2. The experiments are extensive: Evaluate both success, sensitivity, and soundness.

3. The writing is good and articulate.

4. Novelty of the method: The authors propose for the first time to experiment with inference with perturbed prompts for the task at hand.

weaknesses:
1. Novelty of the task: Please have a look at [1], [2] - these work engage in membership inference without using prompts, as far as I know. Could the authors clarify how their proposed task of image-level memorization detection differs from membership inference, particularly in light of the cited works that perform membership inference without prompts?

[1] Matsumoto, T., Miura, T., & Yanai, N. (2023, May). Membership inference attacks against diffusion models. In 2023 IEEE Security and Privacy Workshops (SPW) (pp. 77-83). IEEE.
‏

[2] Hu, H., & Pang, J. (2023, November). Loss and Likelihood Based Membership Inference of Diffusion Models. In International Conference on Information Security (pp. 121-141). Cham: Springer Nature Switzerland.

2. Comparison to other methods: Could the authors add comparisons to relevant methods that do not require prompt access (e.g. the methods mentioned above)?

3. Additional experiments with a different stable diffusion model would substantiate the proposed method's advantage.

4. Requiring inversion of the model is quite restrictive, considering how quickly generative technology changes, and how rare it is for generative models to enable such inversions. Can the authors think of ways adapt their findings to non-invertible scenarios?

5. Minor Weaknesses: 
a. In all related figures - change "ori" to the full word "original".
b. The expression " images exhibit greater similarity discrepancy " in the introduction is rather confusing

questions:
Please refer to Weaknesses above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730722247034

thread_type:
official_review

<thread_object>
creator: reviewer_8z99

<content>
summary:
This paper investigates the memorization limitation of the Stable Diffusion model to help protect the training data’s privacy. The paper proposes a new task: image-level memorization detection, which is different from existing works that detect memorization during inference. Then, based on two insights that memorized images under perturbed inference have a notable similarity discrepancy and a large magnitude of text-conditional noise prediction, the paper proposes IIP framework that uses unconditional DDIM inversion to derive latent noises for the images and conducting perturbations. The paper also construct a setup for this new task and demonstrated better performance than baselines.

strengths:
1. This work tackles an important and practical problem: diffusion models can memorize their training data, potentially leading to privacy concerns. These issues are thoroughly discussed and effectively motivated in the paper.
2. The paper is well-presented, clearly structured, and easy to follow. 
3. This paper further proposes a new task of image-level memorization detection and a correspondingly designed method for this new task.

weaknesses:
1. The motivation for the proposed new image-level memorization detection task is based on a misunderstanding of the related work. Specifically, in lines 46-47, the paper claims that “However, these approaches rely heavily on access to the original prompts, which is often impractical.” Since all baselines and this paper experiment on the text-to-image Stable Diffusion model, using text prompts is practical. Also, the baseline methods do not need an organized prompt list that contains triggering prompts since they can obtain detection signals (such as TCNP in [1]) during the inference process. They can then detect the potential memorized images being generated within only one inference step and with good accuracy. Such lines of method are actually more practical than the proposed image-level memorization detection task, as the memorization is detected and halted from the source (way before the image is even generated). Thus, the proposed task is of limited practical significance. I would suggest the authors consider investigating unconditional diffusion models (where there is no text prompt) and see if the proposed method works.
2. The finding that large TCNP correlates with memorization is not novel. I understand that the paper differs in performing DDIM inversion to the clean image, however, it is actually the identical finding of [1] that a noise can present large TCNP even after the first step of DDIM denoising.

[1] Detecting, Explaining, and Mitigating Memorization in Diffusion Models.

questions:
Please see the weaknesses regarding motivation and novelty.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732801014451

thread_type:
public_comment

<thread_object>
creator: ~zhe_ma3

<content>
comment:
The proposed image-level memorization detection task and the inversion-based detection idea, which are the main motivation of this work and considered novel by the reviewers, are also proposed in [1]. However, I didn't see adequate discussion on the connection or difference between the two works, but only a short and point-missing description of [1] in Sec 5.2.

[1] Zhe Ma, et al. Could It Be Generated? Towards Practical Analysis of Memorization in Text-To-Image Diffusion Models. arXiv preprint arXiv:2405.05846, 2024.
</content>
</thread_object>

</thread>
<thread>
creation date:
1742349606041

thread_type:
public_comment

<thread_object>
creator: ~gunjan_dhanuka1

<content>
title:
Missing Code Repository for the paper

comment:
Dear Authors,

Congratulations on your paper's acceptance to ICLR 2025! I am very interested in reproducing your results and exploring the ideas further. However, I noticed that the GitHub repository mentioned in the paper is not working anymore, and I haven’t received a response to my email which I sent to the corresponding authors about the same.

Could you kindly share the code or provide an updated link to the repository? It would greatly help in advancing research in this direction.

Thank you!
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper presents a new setting of image-level memorization detection given a user-provided reference image. It is generally well-received and has promising experimental results. 

The authors are encouraged to improve the paper in the final version regarding the following aspects.
- clearer distinctions from related work
- broader experimental validation across different models (experiments from the rebuttal should be included)
- a more compelling discussion of the practical significance of the task
- a (nice to have) discussion about how possibly removing the detected image-level memorization 

Although some margins need to be improved, this paper still stands with its merits. So it is recommended that it be accepted.