Real-World Benchmarks Make Membership Inference Attacks Fail on Diffusion Models

Authors: Chumeng Liang, Jiaxuan You
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Membership inference attack, Diffusion models, Benchmark

Abstract:
Membership inference attacks (MIAs) on diffusion models have emerged as potential evidence of unauthorized data usage in training pre-trained diffusion models. These attacks aim to detect the presence of specific images in training datasets of diffusion models. Our study delves into the evaluation of state-of-the-art MIAs on diffusion models and reveals critical flaws and overly optimistic performance estimates in existing MIA evaluation. We introduce CopyMark, a more realistic MIA benchmark that distinguishes itself through the support for pre-trained diffusion models, unbiased datasets, and fair evaluation pipelines. Through extensive experiments, we demonstrate that the effectiveness of current MIA methods significantly degrades under these more practical conditions. Based on our results, we alert that MIA, in its current state, is not a reliable approach for identifying unauthorized data usage in pre-trained diffusion models. To the best of our knowledge, we are the first to discover the performance overestimation of MIAs on diffusion models and present a unified benchmark for more realistic evaluation.

Decision: reject
Reviewer scores: 6, 5, 6, 5

Reviews and discussion:
<thread>
creation date:
1730236118768

thread_type:
official_review

<thread_object>
creator: reviewer_y1k9

<content>
summary:
This paper investigates the evaluation of state-of-the-art membership inference attacks (MIAs) on diffusion models in real-world scenarios. Specifically, it highlights flaws in current MIA evaluations, where over-training and dataset shifts lead to overestimated performance of the membership detection. To address this, the paper introduces a unified benchmark for MIAs on diffusion models, named CopyMark, which is built without over-training, using non-shifted datasets and blind testing. The experiments cover the recent loss-based MIA methods and classifier-based MIA methods, conducted on both defective setups and real-world setups. The results reveal that existing MIAs perform poorly on diffusion models in realistic scenarios.

strengths:
1. The paper is well-written and easy to follow. It explains the flaws in existing MIA evaluations, i.e., over-training and dataset shifts, and is structured to understand these two problems through quantitative and qualitative analyses. 

2. This paper makes valuable thoughts about the limitations of current MIA evaluations on diffusion models. The significance of the proposed realistic evaluation for MIA is substantial, particularly in the context of AI copyright lawsuits and data privacy.

weaknesses:
1. The originality of these two flaws, i.e., over-training and dataset shifts, remains a concern. Similar concepts like over-fitting and distribution shifts have been discussed in previous works (Carlini et al., 2022; Maini et al., 2024) on traditional deep learning models and large language models. This paper may potentially adapt the MIA setting to diffusion models while providing more assessments.

2. Although the paper assesses existing MIA methods on diffusion models, it does not explore possible adjustments to improve MIA performance on CopyMark. For example, how to address the challenges identified on existing loss-based and classifier-based MIA methods and how to achieve better results under realistic scenarios.

3. The evaluation may lack comprehensiveness as a benchmark, as the experiments are limited to loss-based and classifier-based MIA methods on diffusion models. Other types of MIAs, such as likelihood-based MIAs (Hu & Pang, 2023) and MIAs using Quantile Regression (Tang et al., 2024), are not included.

Carlini et al. Membership inference attacks from first principles. 2022 IEEE Symposium on Security and Privacy. 2022.

Maini et al. LLM Dataset Inference: Did you train on my dataset? arXiv preprint arXiv:2406.06443. 2024.

Hu & Pang. Loss and likelihood based membership inference of diffusion models. In International Conference on Information Security. 2023.

Tang et al. Membership inference attacks on diffusion models via quantile regression. International Conference on Machine Learning. 2024.

questions:
1. How do the issues of over-training and dataset shifts differ between diffusion models and traditional deep learning models or large language models? Will the proposed realistic scenarios similarly reduce MIA effectiveness on these other model types?

2. How do MIA methods based on likelihood and quantile regression perform on diffusion models in the proposed realistic scenarios? Will their performance also see a significant reduction?

3. Minor point: a typo “randomlyy” in the third paragraph of section 4.3.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730633852717

thread_type:
official_review

<thread_object>
creator: reviewer_ixff

<content>
summary:
The paper presents a novel approach to assessing MIA on diffusion models by introducing a new benchmark called CopyMark. This benchmark aims to provide a realistic and unbiased environment for testing the effectiveness of MIAs against these models. The study underscores the potential overestimation of MIA effectiveness due to biased experimental setups in previous research and argues for a more nuanced understanding and evaluation of MIAs in practical applications. The paper pinpoints that current MIAs on diffusion models are not trustworthy tool to provide evidence for unauthorized data usage in diffusion models.

strengths:
1. Good and significant topic. The paper identifies a critical gap in the evaluation of MIAs, offering a novel approach to benchmarking that could reshape how these attacks are studied, and providing valuable insights that can influence the future research.
2. Comprehensive experiment. The experiments conducted are extensive, providing evidence that challenges the overestimation of MIA effectiveness on diffusion models.

weaknesses:
1. Lack of discussion. The discussion on the practical implications of the findings is somewhat superficial and lacks depth in Section 6, particularly in how these results could influence real-world security strategies.

questions:
The paper aims to construct a real-world benchmark, pinpointing the current limitation of MIA setups, specifically the unknown distribution of members and non-members in real-world MIAs. It is reasonable that a newly proposed benchmark can cause current methods to yield poor performance. However, I find the discussion lacking in adequately demonstrating how this benchmark accurately reflects real-world settings from my perspective. In my opinion, additional evidence and a more thorough discussion would strengthen this aspect.

In the evaluation setup part, the paper mentions that (d) has a slight data shift but is more minor than other settings. Can you provide further insight into how minor dataset shifts were quantified and their potential impact on the validity of MIA results? It would be beneficial to have a more detailed analysis of how significant these shifts need to be in impacting the effectiveness of MIAs. What thresholds for dataset similarity were considered, and how were they determined?

The paper demonstrates that current MIAs are less effective under realistic conditions on diffusion models. How do you envision these findings being applicable to other types of generative models? Are there specific characteristics of diffusion models that may limit the generalizability of the results? A discussion on this could clarify potential broader applications of your findings.

This paper conducts a comprehensive experiment and concludes that the current MIAs on diffusion models do not perform well in real-world scenarios. However, I think the discussion part is relatively superficial and requires a deeper analysis based on the experimental results. Can you provide more implications and extend the discussion to promote future research?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730695170910

thread_type:
official_review

<thread_object>
creator: reviewer_eney

<content>
summary:
The paper proposed a simple but effective benchmark for evaluating the existing MIA’s performance on the pre-trained diffusion models for the data authorization problem. The authors first found that “overtraining” and “dataset shifts” are two major defects of the existing MIA methods. Then, to overcome the two challenges, the authors proposed a benchmark that incorporates five different experimental setups, where the last three avoids the dataset shifting problem by using members and non-members from the same distributions, and over-training problem by only considering pre-trained models training for 1 epoch.

strengths:
- Presentation is good and easy to follow.
- The addressed problem is meaningful.

weaknesses:
- I am confused about the upper part of Table 1. What do the “✅” and “❌” symbols represent in each entry? Additionally, are “Over-training” and “Shifted Datasets” considered issues in each experimental setup (e.g., is over-training a problem in the DDPM + CIFAR10 setup)? If so, why is over-training necessarily a problem for DDPM + CIFAR10? I believe this only holds when certain factors, like training epochs, are fixed as you reported in the common setting; otherwise, this claim seems overstated.

- Could the benchmark allow for more varied experimental setups—for instance, having no dataset shift but including over-training? A simple example could involve training a DDPM on the CIFAR10 training set and using the CIFAR10 test set as non-members, which would meet the no-shift criterion.

- Furthermore, the concept of “dataset shift” is somewhat unclear to me. The benchmark assumes there’s no distribution shift when two datasets come from the same source. I suggest the authors delve deeper into this by considering metrics to quantify dataset distance (distribution distance), such as the Wasserstein distance.

questions:
See weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730734048571

thread_type:
official_review

<thread_object>
creator: reviewer_o9bc

<content>
summary:
The paper introduces a straightforward yet powerful benchmark to assess the performance of existing Membership Inference Attacks (MIA) on pre-trained diffusion models within the context of data authorization. The authors identified "overtraining" and "dataset shifts" as two significant limitations of current MIA methods. To address these issues, they developed a benchmark featuring five experimental setups.

strengths:
- The writing is clear
- The structure is easy to follow
- The paper considered comprehensive comparison with the relate works

weaknesses:
- I am unsure about the input for the membership inference attacks. In Lines 113-116, does x refer solely to the image, or is it a combination of the image and its prompt? I recommend that the authors clarify this in the problem setup.

- In Table 1, why does "LDM + CelebA" have $\times$ for both “Over-training” and “Shifted Datasets,” while in the bottom table, "LDM + CelebA" (i.e., the third row) has $\checkmark$ for both? Is this a typo, or have I misunderstood the notation?

- While I appreciate the authors’ efforts in benchmarking MIA methods in practical scenarios, I believe the paper’s analysis of the two challenges, “Over-training” and “Shifted Datasets,” could be more in-depth. For example, I recommend adding an analysis of how shifted datasets impact MIA performance based on the distance of non-members from the target data (e.g., considering extremely close, moderately distant, and far distant non-members).

questions:
See Weakness part.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732216528328

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Updating a new draft of the paper

comment:
We thank all reviewers for the insightful reviews and have accordingly updated a new draft of our paper. We list the updated points, noted as blue text in our new draft, as follows:

[1] Quantifying the dataset shift of benchmarks in CopyMark and showing the threshold between qualified / unqualified member and non-member datasets (Section 5.1).

[2] Relation between different levels of dataset shifts and MIA performances (Section 5.3).

[3] Fixing typos in Table 1.

[4] Brief guidelines on how to address the challenges identified on existing MIA methods under realistic scenarios (Appendix 3.3.1).

[5] Updating extra references.

[6] Explaining the difference between over-training in diffusion models and traditional over-fitting (Section 3).
</content>
</thread_object>

</thread>
<thread>
creation date:
1732949243001

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Kindly request for further discussions

comment:
Dear Reviewers,

We highly appreciate your constructive review and appreciate your constructive advice. To address these concerns, we have updated our paper draft and conducted in-depth analysis on the impact of dataset shifts with extensive experiments, which further validates our assertion. As the rebuttal period draws to a close, we sincerely look forward to your further ideas on these concerns and kindly request your engagement in the discussion. We also want to hear your advice on the updated content. We would sincerely appreciate it if you could consider engaging in further discussions.

Best regards,

Authors of Submission7638
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper aims to evaluate the state-of-the-art MIAs on diffusion models and reveal critical flaws and overly optimistic performance estimates in existing MIA evaluation. The presentation is clear and the experiments are also good. Reviewers like this paper, but find many issues that can not be easily fixed in a short time, such as the lack of discussion and the lack of the evaluation and so on.