Towards Black-Box Membership Inference Attack for Diffusion Models
Authors: Jingwei Li
Jing Dong
Tianxing He
Jingzhao Zhang
Venue: ICML 2025 poster (2025)
Keywords: diffusion model
membership inference attack
Primary area: social_aspects->safety
TL;DR: A Black-Box Membership Inference Attack Algorithm for Diffusion Models

Abstract: Given the rising popularity of AI-generated art and the associated copyright concerns, identifying whether an artwork was used to train a diffusion model is an important research topic. The work approaches this problem from the membership inference attack (MIA) perspective. We first identify the limitation of applying existing MIA methods for proprietary diffusion models: the required access of internal U-nets.
To address the above problem, we introduce a novel membership inference attack method that uses only the image-to-image variation API and operates without access to the model's internal U-net. Our method is based on the intuition that the model can more easily obtain an unbiased noise prediction estimate for images from the training set. By applying the API multiple times to the target image, averaging the outputs, and comparing the result to the original image, our approach can classify whether a sample was part of the training set. We validate our method using DDIM and Stable Diffusion setups and further extend both our approach and existing algorithms to the Diffusion Transformer architecture. Our experimental results consistently outperform previous methods.

Towards Black-Bor Membership Inference Attack for Diffusion Models
Jingwei Li
Jing Dong
Tianxing He 12 Jingzhao Zhang 12
Abstract
generation (Rombach al., 2022; Yu al., 2022; Nichol et al., 2021) and image-to-image generation (Saharia et al., 2022a). This surge has given rise to powerful AI art models such as DALL-E 2 (Ramesh et al., 2022), Stable Diffu- sion (Rombach et al., 2022), and Imagen (Saharia et al., 2022b). Al-generated art has a promising future and is expected to have widespread impact.
Given the rising popularity of Al-generated art and the associated copyright concerns, identify- ing whether an artwork was used to train diffu- sion model is important research topic. The work approaches this problem from the member- ship inference attack (MIA) perspective. We first identify the limitation of applying existing MIA methods for proprictary diffusion models: the required access of internal U-nets. To address the above problem, we introduce novel mem- hership inference attack method that uses only the image-to-image variation API and operates without access to the model's internal U-net. Our method is based on the intuition that the model can more easily obtain an unbiased noise predic- tion estimate for images from the training set. By applying the API multiple times the target im- age, averaging the outputs, and comparing the result to the original image, approach classify whether a sample was part of the training set. We validate our method using DDIM and Stable Diffusion setups and further extend both our approach and existing algorithms to the Diffu- sion Transformer architecture. Our experimental results consistently outperform previous methods.
Effective training of diffusion models requires high-quality data. It is thus crucial to design algorithm that can iden- tify whether a specific artwork has been used during the training of a model, thereby providing protection for these artworks and detecting misuse data. This is especially important due to the rapid growth of generative models, which has raised concerns over intellectual property (IP) rights, data privacy, and the ethical implications of training copyrighted proprietary content without consent. As these models are increasingly deployed across industries, detecting whether a specific piece of content was used in training can help prevent unauthorized use of artistic works, protecting creators' copyrights and ownership rights. This is a classic problem in the field of machine learning, first introduced by Shokri al. (2017) and named "membership inference attack".
A series of studies have been conducted on membership inference attacks against diffusion models. Hu & Pang (2023) was the first to examine this issue, utilizing the loss function values of diffusion models to determine whether an image is in the training set. Duan et al. (2023) and Kong et al. (2023) extended this work by relaxing assumptions about the access requirements of the model.
1. Introduction
Recently, there has been surge in the popularity of gener- ative models, with diffusion models in particular, gaining huge attention within the AI community (Sohl-Dickstein et al., 2015; Song & Ermon, 2019; Song et al., 2020b). These models have demonstrated remarkable capabilities across various tasks, including unconditional image gener- ation (Ho et al., 2020; Song et al., 2020a), text-to-image
Despite great progress, previous methods are not yet ready for MIA in proprietary diffusion models. Most existing approaches heavily rely on checking whether the U-net of the model predicts noise accurately, which is not practical since most commercial diffusion models available today offer only API access, while the U-net remains hidden.
Instittte for Interdisciplinary Information Sciences, Ts- inghua University "Shanghai Qizhi Institute 'The Chinese University of Hong Kong. Correspondence to: Jing- wei Li <1jw22 mails. tsinghua.cdu.cn..... Jingzhao Zhang <jingzhaoz mail. www.gnua.ccuu.c.n........
To address the above issue, we propose a membership infer- ence attack method that relies on the variation API and does not require access to the denoise model (e.g.. U-net). We observe that we alter an image using the target diffusion model's variation API, the sampling process will be cap- tured by "a region of attraction" if the model has seen this image during training (illustrated in Figure 2). Based on the
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Towards Black-Box Membership Inferenee Attack for Diffusion Models
above observation, we propose the REDIFFUSE algorithm for MIA with image-to-image variation API, and can detect member images without accessing the denoise model. Our main contributions are listed as follows:
laws. Membership inference attacks, initially introduced by Shokri et al. (2017), serve an effective means to detect potential misuse of data without proper authorization. Its objective is to ascertain whether particular data sample participated in the training phase of a target model. This approach is instrumental in probing privacy breaches and identifying illicit data utilization. Researchers primarily fo- cus on membership inference attacks for classification mod- els (Salem et al., 2018; Yeom et al., 2018; Long et al., 2018; Li & Zhang, 2021), embedding models (Song & Raghu- nathan, 2020; Duddu et al., 2020; Mahloujifar et al., 2021), and generative models (Hayes et al., 2017; Hilprecht et al., 2019; Chen et al., 2020).
1.
We propose a membership inference attack method that does not require access to the model's internal structure. Our method only involves using the model's variation API to alter an image and compare it with the original one. We name our method REDIFFUSE.
2. We evaluate our method using DDIM (Song et al., 2020a) and Stable Diffusion (Rombach et al., 2022) models on classical datasets, including CI- FAR10/100 (Krizhevsky
In the domain of membership inference attacks against dif- fusion models, Wu al. (2022); Hu & Pang (2023) use white-box approach, which assumes access to the entire diffusion model and utilizes loss and likelihood to determine whether a sample is in the training set. Duan et al. (2023); Kong et al. (2023); Tang et al. (2023) have relaxed these re- quirements, climinating the need for the entire model. They leverage the insight that samples within the training set yield more accurate noise predictions, thereby achieving high ac- curacy in membership inference attack tasks. However, they also require the outputs of the U-net, as it is necessary to obtain the noise predictions of intermediate steps. Recently, Pang & Wang (2023) proposed a black-box membership inference attack method against diffusion models. Their method identifies whether a specific image is in a finetuning dataset of 100 images by calculating the difference between the generated image and the target image, using the corre- sponding prompt as input. Their approach leverages the model's tendency to memorize finetuning images and gen- erate similar outputs. In contrast, we focus on detecting whether an image is in the pretraining dataset, where a pre- trained model often produces diverse outputs for the same prompt, making detection more challenging.
al., 2009), STL10- Unlabeled (Coates al., 2011), LAION-5B (Schuh- mann et al., 2022), etc. Our method outperforms the previous methods.
3. We extend both existing algorithms and our own al- gorithm to the Diffusion Transformer (Peebles & Xie, 2023) architecture, implementing the membership in- ference attack within this model framework for the first time. Experimental results demonstrate that our algorithm is consistently effective.
2. Related Works
Diffusion Model The diffusion model, initially proposed by Sohl-Dickstein al. (2015), has achieved remarkable re- sults in producing high-quality samples across variety of domains. This ranges from image generation (Song & Ermon, 2019; Song et al., 2020b; Dhariwal & Nichol, 2021), audio synthesis (Popov et al., 2021; Kong et al., 2020; Huang et al., 2022). and video generation (Ho et al., 2022a;b; Wu et al., 2023), etc. Among existing diffusion models, the Denoising Diffusion Probabilistic Model (DDPM) (Ho et al., 2020) is one of the most frequently adopted. This approach introduces a dual-phase process for image gen- eration: initially, forward process gradually transforms training data into pure noise, followed by a reverse process that meticulously reconstructs the original data from this noise. Building on this model, there have been numerous follow-up studies, such as Stable Diffusion (Rombach et al., 2022), which compresses images into latent space and generates images based on text, and the Denoising Diffu- sion Implicit Models (DDIM) (Song et al., 2020a), which removes Gaussian randomness to accelerate the sampling generation process. These advancements demonstrate the versatility and potential of diffusion models.
3. Preliminary
In this section, we begin by introducing the notations used for several popular diffusion models. We first introduce the Denoising Diffusion Probabilistic Model (DDPM) (Ho et al., 2020). Then, we extend to the Denoising Diffusion Implicit Model (DDIM) (Song et al., 2020a) and Stable Diffusion (Rombach et al., 2022), which are variants of DDPM used to accelerate image generation or generate images grounded in text descriptions. Lastly, we discuss Diffusion Transformer (Peebles & Xie, 2023), model that replaces the U-net architecture with transformer and achieves higher-quality image generation.
Data Safety and Membership Inference Attack In the era of big data, preserving data privacy is paramount. The training of diffusion models may involve sensitive datasets like artists' artworks, which are protected by copyright
Denoising Diffusion Probabilistic Model (DDPM) A diffusion model provides a stochastic path between an image and noise. The forward process (denoted as q) iteratively incorporates Gaussian noise into an image, while the reverse
2

---

Towards Black-Box Membership Inferenee Attack for Diffusion Models
process (denoted as pa) gradually reconstructs the image from noise.
of previous membership inference attack methods. In our formulations, we assume DDIM our target model. The formulations for DDPM are highly similar and we omit it for brevity. We will discuss the generalization to the latent diffusion model in Section 4.3.
q(xe x)-1)= =N N (Zei V1- -,0,,,1,1100 Bel1 ( 21) = N (12-1; t), ΣT (ze: (Z(::)) t))
4.1. The variation API for Diffusion Models
where (+)(+) and (-) are the mean and covariance of the denoised image parameterized by the model parameters A,
Most previous works on membership inference attacks against diffusion models aim to prevent data leakage and hence rely on thresholding the model's s training loss. For instance, Hu & Pang (2023) involves a direct comparison of image losses, while Duan al. (2023); Kong et al. (2023) evaluates the accuracy of the model's noise prediction at initial or intermediate steps. However, the required access to the model's internal U-net structure prevents applications from copyright protection because most servers typically provide only black-box API access.
and Be is noise schedule that controls the amount of noise added at each step.
Denoising Diffusion Implicit Model (DDIM) DDIM modifies the sampling process to improve efficiency while maintaining high-quality image generation. Unlike DDPM, which requires large number of denoising steps, DDIM uses a Mar-Markovi process to accelerate sampling.
Ft
VI - (xCo(Fs,
Tt-1 = (o( = v
Vor
In contrast, method represents a step towards black- box MIA, we do not directly access the model's internal structure. Instead, we rely solely on the variation API, which takes an input image and returns the corresponding output image. Below, we formalize the definition of the variation API used in our algorithm.
+ V1 11-
where ā, = II= O4e, ar + B, = 1 and cp (2,, t) is the noise predicted by the model at step t. This formulation requires fewer sampling steps without compromising the quality of the generated images.
Definition 4.1 (The variation API). We define the variation API Vo t) of a model follows. Suppose we have an in- put image and the diffusion step of the API is 1. The vari- ation API randomly adds t-step Gaussian noise ~~ (0, I) to the image and denoises it using the DDIM sampling pro- cess de (Iz, t), returning the reconstructed image Vo t). The details are as follows:
Stable Diffusion Stable Diffusion leverages a variational autoencoder (VAE) (Kingma & Welling, 2013) to encode images into a latent space and perform diffusion in this compressed space. The model uses text encoder to guide the diffusion process, enabling text-to-image generation:
Zt-1 p=(=11 z2, To()))
Decoder(2o),
St = Vant + v1 - Ōte, Vo = Do(2 0) = (o(o 1)) 1),
where x represents the output image. 2, represents the latent variable at step f, and the text conditioning ry(y) is incor- porated into the denoising process generate the image. This approach significantly reduces computational costs and allows for high-quality image synthesis from textual descriptions.
This definition aligns with the image-to-image generation method of the diffusion model, making access to this API practical in many setups (Lugmayr et al., 2022; Saharia et al., 2022a: Wu & De la Torre, 2023). Some APIs provide the user with choice of t, while others do not and use the default parameter. We will discuss the influence of different diffusion steps in Section 5.5, showing that the attack performances are relatively stable and not sensitive to the selection of t. We also note that for the target model, we can substitute (u( 0) with other sampling methods, such as the Euler- Maller-Mruyaama Method (Mao, 2015) or Variational Diffusion Models (Kingma et al., 2021).
Diffusion Transformer Diffusion Transformer leverages the Vision Transformer (Dosovitskiy, 2020) structure to re- place the U-net architecture traditionally used in diffusion models for noise prediction. Its training and sampling meth- ods remain consistent with DDIM, with the only difference being the replacement of noise prediction network eo with Eg. where 0 represents a Vision Transformer-baseee architec- ture. This approach further enhances the generation quality and ensured that the model possesses good scalability prop- erties.
4.2. Algorithm
In this section, we present the intuition of our algorithm. We denote || . as the L2 operator norm of a vector and
4. A Algorithm Design
In this section, we introduce our algorithm. We begin by discussing the definition of variation API and the limitations
T = {1, 2, ......IT T) as the set of diffusion steps. The key insight derived from the training loss function of fixed
3

---

Towards Black-Box Membership Inference Attack for Diffusion Models
Member
sample
Add t-step
Denoising t-step by
Member
Gaussian Naise
Taking average output images
Compare with the original image
Threshold
Forward
Reverse
process
process
Non-member
Non-member sample
Black-box variation API
The overview of REDIFFUSE. We independently input the image to the variation API times with diffusion step t. We take the average of the output images and compare them with the original ones. If the difference is below a certain threshold, we determine that the image the training set.
Figure I:
sample Zo and time step T::
L(0) Ee-NN(O.) |||e -cc cB (Vāizo ca
+ v1 t)e,t)
R1
Denote Zt Va,to + v1 0ce, we assume that the de- noise model is expressive enough (the neural network's dimensionality significantly exceeds that of the image data) such that for the input xy Idd and time step T, the Jacobian matrix Veco (3,, is full rank (2 d). This sug- gests that the model can adjust the predicted noise eg (z,, locally in any direction. Then for a well trained model, we would have (0) = O. This implies
82
33
3
Member
Non-member
The intuition of our algorithm design. We de- note x as the target image, å; as the i-th image generated by the variation API, and é as the average image of them. For member image x, the difference v = x - f will be smaller after averaging due to I, being an unbiased estimator.
Figure 2:
Voca (Iee 1) Be-Ai [e - (xe, O)] 0, E.-.OO (c (Ir =0
This is intuitive because if the neural network noise predic- tion exhibits high bias, the network can adjust to fit the bias term, further reducing the training loss.
Our algorithm classifies a sample as being in the training set D(x, .) is smaller than a threshold т, where D(x, .) represents the difference between the two images. It can be calculated using traditional functions, such as the SSIM metric (Wang et al., 2004). Alternatively, neural network proxy. In Section 5, will introduce the details of (zz, 2) used in experiment.
Therefore, for images in the training set, we expect the net- work to provide an unbiased noise prediction. Since the noise prediction is typically inaccessible in practical appli- cations, we use the reconstructed sample as proxy. By leveraging the unbiasedness noise prediction, we show that averaging over multiple independent reconstructed sam- ples à; can significantly reduce estimation error (see The- orem 4.2). However, for images not in the training set, the neural network may not provide unbiased prediction at these points. The intuition is illustrated in Figure 2.
can train
Our algorithm outlined in Algorithm 1, and we name it REDIFFUSE, The key ideas our algorithm are illustrated in Figure 1, and we also provide some theoretical analysis in Theorem 4.2 support
We give a short analysis to justify why averaging over samples in REDIFFUSE can reduce the prediction error for training data. We have the following theorem showing that if we use the variation API to input member x ~ Diraining then the error || x|l| from our method will be small with high probability.
Analysis
With the above intuition, we introduce the details our algorithm. We independently apply the variation API n times with our target image x as input, average the output images, and then compare the average result & with the original image. We will discuss the impact of the averaging number in Section 5.5. We then evaluate the difference between the images using an indicator function:
Theorem 4.2. Suppose the DDIM model can learn a pa- rameter such that, for any x. Dirainng with dimension d. the prediction error co Vorr v1 04C, 1) is a ran-
f(x) (2( 8) <

---

Towards Black-Box Membership Inference Attack for Diffusion Models
Table I: Comparison of different methods on four datasets for the DDIM model. Previous methods require access to the U-Net, whereas our methods do not. We use AUC, ASR, and TP as the metrics, TP refers to the True Positive Rate when the False Positive Rate is 1%
Method
CIFARIO
CIFARI00
STLI0
Algorithm
U-Net
AUC
ASR
TP
AUC
ASR
TP
AUC
ASR
TP
Loss (Matsumoto et al., 2023)
0.88
0.82
14.2
0.92
0.84
20.9
0.89
0.82
15.6
SecMI (Duan al., 2023)
0.95
0.90
40.7
0.96
0.90
44.9
0.94
0.88
26.9
PIA (Kong al., 2023)
0.95
0.89
48.7
0.96
0.90
47.0
0.94
0.87
29.8
PIAN (Kong et al., 2023)
0.95
0.89
50.4
0.91
0.85
39.2
0.92
0.86
28.5
REDIFFUSE
0.96
0.91
40.7
0.98
0.93
48.2
0.96
0.90
31.9
D: Require the access of U-Net.
Do not require the access of U-Net.
Algorithm MIA through REDIFFUSE
5 = Vo (), and the details are as follows:
Input:
Target image z, diffusion step t. average number n, threshold r, the variation API of the target model Vo. distance function D.
Encoder(1:), 2t Vāiz + v1 OrgE;
2 = Φz 0), 2 Decoder(3).
for As 1, ...... do
This definition aligns with the image generation process of the stable diffusion model.
Use the variation API Vo to generate the variation im- age 2 = Vo t) according to Definition 4.1.
end for
For the Diffusion Transformer, we define the variation API (z t), where @ corresponds to the Vision Transformer architecture instead of the U-net. We repeatedly call the vari- ation API and calculate the difference between the original image and the reconstructed image, as done in DDIM.
Average the reconstructed images from each iteration = 1 n (â1 + #2 +... ..).
return "YES" if the distance between the two images ((x, is less than т, otherwise "NO".
5. Experiments
dom variable X = X2..,. with zero expectation and finite cumulant-generaatnng function for each coordi- nate (Durret, 2010). Suppose the sampling interval & is equal to the variation API diffusion step Let & be the average of output images of using the variation API, Then we have
In this section, we evaluate the performance of our methods across various datasets and settings. We follow the same ex- periment setup in previous papers (Duan et al., 2023; Kong et al., 2023). The detailed experimental settings, including datasets, models, and hyper-pann pappe-prrrmeter settings can be found in Appendix A!.
5.1. Evaluation Metries
Bo
P(I æ|| 2 23) dexp (-nmin V'
We follow the metrics used in previous papers (Duan et al., 2023; Kong et al., 2023), including Area Under Receiver Op- erating Characteristic (AUC), Attack Success Rate (ASR), the True Positive Rate (TP) when the False Positive Rate is 1.. We also plot the ROC curves.
Vd(1 a.)
where 3 > 0 and xxx is the Fenchel-Legendre dual of the sumulant-generatt function V Xx.
5.2. MIA with DDIM Models
The theorem suggests that averaging the random variational data from training set will result in lower relative error with high probability when we use a large n. We defer the proof of this theorem to Appendix C.
We follow the experimental setup of (Duan et al., 2023; Kong et al., 2023). We train DDIM model on the CIFAR- 10/100 (Krizhevsky datasets (Coates et al., 2011), using the image generation step T 1000 and sampling interval A = 100. For all the datasets, we randomly select 50% of the training samples to train the model and denote them as members. The remain-
al., 2009) and STLll-Unlabeled
4.3. MIA on Other Diffusion Models
In this section, we discuss how we can generalize our al- gorithm to other diffusion models. We note that the vari- ation API for stable diffusion is different from DDIM, as includes the encoder-decoder process. Again we denote
Code available
https: https://github.com
1i wijiingwei002////0 mnaaa..
5

---

Towards Black-Box Membership Inference Attack for Diffusion Models
Table 2: Comparison of different methods for Diffusion Transformer using the same set of metrics as Table 1. Previous methods require access to the Vision Transformer, whereas our methods do not. We use AUC, ASR, and TP as the metrics, TP refers to the True Positive Rate when the False Positive Rate 1%.
Method
ImageNet 128 x 128
ImageNet 256 x 256
Algorithm
Transformer
AUC
ASR
AUC
ASR
TP
Loss (Matsumoto et al., 2023)
0.83
0.76
10.7
0.78
0.70
7.3
SecMI (Duan et al., 2023)
0.80
0.73
8.3
0.88
0.80
16.3
PIA (Kong et al., 2023)
0.97
0.92
32.1
0.91
0.85
6.8
PIAN (Kong al., 2023)
0.66
0.64
6.2
0.67
0.66
12.8
REDIFFUSE
0.98
0.95
44.1
0.97
0.94
47.3
Require the access of Transformer.
Do not require the access of Transformer.
Table 3: Comparison of different methods for Stable Diffusion using the same set of metrics as Table 1. Again, previous methods require access to U-Net, whereas our methods do not. We use AUC, ASR and TP as the metrics, TP refers to the True Positive Rate when the False Positive Rate is 1%.
Method
Laion5
LaionS with BLIP
Algorithm
U-Net
AUC
ASR
TP
AUC
ASR
TP
Loss (Matsumoto al., 2023)
D
0.62
0.61
13.2
0.62
0.62
13.3
SecMI (Duan et al., 2023)
0.70
0.65
19.2
0.66
19.8
PIA (Kong et al., 2023)
0.70
0.66
19.7
0.73
0.68
20.2
PIAN (Kong et al., 2023)
0.56
0.53
4.8
0.55
0.51
4.4
REDIFFUSE
0.81
0.75
20.6
0.82
0.75
21.7
Require the access of U-Net.
Do not require the access of U-Net.
ing 50% are utilized as nonmembers. We use (Matsumoto et al., 2023), (Duan et al., 2023), (Kong et al., 2023) our baseline methods. We fix the diffusion step at = 200, and independently call the variation API 10 times to take the av- erage the output images as 4. We will discuss the impact of the diffusion step and the average number in Section 5.5.
effective and robust.
5.3. MIA with Diffusion Transformers
We train a diffusion transformer model on the Ima- geNet (Deng et al., 2009) dataset following the setup (Pee- bles & Xie, 2023). We randomly select 100,000 images from the ImageNet training set train the model with resolutions of either 1 128 x 128 or 256 x 256. For the membership infer- ence attack setup, 1000 images are randomly chosen from our training set as the member set, and another 1000 images are randomly selected from the ImageNet validation set as the non-member set. We fix the diffusion step at 150 and the DDIM step the variation API 10 times take the average of the output images as â.
For the difference function D(x, i.), following the setup in (Duan et al., 2023), we take the pixel-wise absolute value -- to obtain a difference vector u for each image. Using the ResNet-18 18 network (He et al., 2016) and denoting it as fR. we perform binary classification on these difference vectors. We use 20% of the data as the training set and obtained the label of each difference vector being classified member nonmember. The difference function is obtained by the negated value of the probability outputed by the neural network predicting member: (xx,tt) -fR(v).
50, and independently call
We use (Matsumoto et al., 2023), (Duan et al., 2023). (Kong et al., 2023) as our baseline methods. Since these work did not study the case of Diffusion Transformers, we in- tegrate their algorithms into the DIT framework for evalu- ation. For the difference function D(r, ④), following the setup in (Duan et al., 2023; Kong et al., 2023), take the L2 norm of o to measure the differences between two
The result shown in Table Our method achieves high performance, surpassing several baseline algorithms in most setups, and does not require access the internal structure of the model. This demonstrates that our algorithm is highly
6

---

Towards Black-Box Membership Inference Attack for Diffusion Models
image. The results, presented in Table 2, demonstrate that method outperforms baseline algorithms and does not require access to the Vision Transformer.
0.98
0.96
We also plot ROC curves for the DDIM train on CIFAR-10 and the Diffusion Transformer train on ImageNet in Ap- pendix B. The curves further demonstrate the effectiveness of our method.
0.94
CIFARIO
CIFAR100
STL10
5.4. MIA with the Stable Diffusion Model
0.90
TINY-I
We conduct experiments the original Stable Diffusion model, i.c., Siio--iiffiioo-v-----1 provided by Huggingface, without further fine-tuning or modifications. We follow the experiment setup of (Duan et al., 2023; Kong et al., 2023), use the LAION-5B dataset (Schuhmann et al., 2022) as member and COC02017-val (Lin et al., 2014) as non- member. We randomly select 2500 images in each dataset. We test two scenarios: Knowing the ground truth text, which we denote as Laion5; Not knowing the ground truth text and generating text through BLIP (Li et al., 2022), which we denote as Laion5 with BLIP.
3
5
10
15
20
Average numbers
Figure 3: The impact of average numbers. We train DDIM model on CIFAR-IO. Averaging multiple independent sam- ples proves to be effective in further improving the overall performance of our algorithm, which validates the intuition of our algorithm design.
for other diffusion models and datasets in the Appendix B.
ALC
For the difference function D(a, .t), since the images in these datasets better correlate with human visual percep-
0.9
NiC and Accurecy
0.8
tion, we directly use the SSIM metric (Wang et al., 2004) to measure the differences between two images. The results, presented in Table 3, demonstrate that our methods achieve high accuracy in this setup, outperforming baseline algo- rithms by approximately 10%. Notably, our methods do not require access U-Net.
0.7
0
100
200 300
400 500
Diffusion steps
Figure 4: The impact of diffusion steps on DDIM, We train a DDIM model on the CIFAR-IO dataset and use differ- ent diffusion steps for inference. We find that high accuracy can be achieved as long as a moderate step number is cho- sen. This opens up possibilities for practical applications in real-world scenarios.
5.5. Ablation Studies
In this section, we alter some experimental parameters to test the robustness of our algorithm. We primarily focus on the ablation study of DDIM and Diffusion Transformer, while the ablation study related to Stable Diffusion is provided in Appendix B.
The Impact of Sampling Intervals In DDIM and Diffu- sion Transformer, the model uses set of steps denoted by Ti, T2...., -y- It samples each of these steps to create the image. The spacing between these steps is referred to the sampling interval. We change the sampling interval k and check the influence on the results. As shown in Figure 5, we adjust this parameter for attacks on both DDIM and Diffusion Transformer. We find that our method achieves high AUC values across different sampling intervals, demon- strating that our detection capabilities are not significantly limited by this parameter. Additionally, in Appendix B, we plot the effect of different sampling intervals on ASR and find that the impact is minimal.
The Impact of Average Numbers We test the effect of using different averaging numbers n on the results, as shown in Figure 3. It can be observed that averaging the images from multiple independent samples to generate the output & further improves accuracy. This observation validates the algorithm design intuition discussed in Section 4.2. Ad- ditional figures showing the ASR results are presented in Appendix B.
The Impact of Diffusion Steps We adjust the diffusion step examine its impact on the results. The experi- ments are conducted using the DDIM model on CIFAR-10 with diffusion steps for inference. The outcomes pre- sented in Figure 4. Our findings indicate that as long as moderate step is chosen, the attack performance remains excellent, demonstrating that our algorithm is not sensitive to the choice of t. This further underscores the robustness of our algorithm. We also plot the change of diffusion step
6. An Application to DALL-E's API
In this section, we conduct a small experiment with online API services to test the effectiveness of our algorithm. We test with the DALL-E (Ramesh et al., 2022) model since
7

---

Towards Black-Box Membership Inference Attack for Diffusion Models
0.98
DALLEE's Variation API
0.96
AUC
0.94
CIFAR10
0.92
CIFAR10O
0.90
TINY-IN
Figure 6: Main observation of our attack. DALL-E 2's variation API makes minimal changes to famous artworks, while nonmember images with similar content undergo sig- nificant alterations. More examples of changes to the mem- ber and non-member inputs after applying the variation API can be found Appendix D.
10
50
100
DDIM steps
Figure 5: The impact of sampling intervals. We train DDIM model CIFAR-10. We find that adjusting the sampling interval has a relatively small influence in the first case and does not affect our method in the latter case. This makes our algorithm applicable to more setups.
non-member inputs after applying the variation API can be found in Appendix D.
DALL-E 2 provides variation API service. We select different thresholds and classify image with a variation error below the threshold as members and those above the threshold as non-members. We then calculate metrics such as AUC and ASR. Since the baseline algorithms (Matsumoto al., 2023; Duan et al., 2023; Kong et al., 2023) require intermediate results, are unable test these algorithms under the online API setup.
Table 4: The results of applying our algorithm to DALL- E 2's variation API. We assume some famous paintings as members and use Stable Diffusion along with the titles of these artworks to generate corresponding non-members. Our algorithm also achieves high accuracy under this setup.
Metries
L1 distance
L2 distance
AUC
76.2
88.3
One challenge with MIA for DALL-E 2 is that it does not disclose its training set. However, since it is adept at generat- ing various famous artworks, we select 30 famous paintings from five famous artists: Van Gogh, Monet, Da Vinci, Dali, and Rembrandt, to form our member set. We believe that it reasonable hypothesize that these artworks are used in DALL-E 2's training set. For constructing the non-member set, we used Stable Diffusion 3 (Esser et al., 2024) to gener- ate images based on the titles of each painting in the member set. The benefit of constructing non-members in this way is that it allows for control over the content of the artwork de- scriptions, reducing bias caused by content shift. Moreover, these generated images are certainly not in the DALL-E 2's training set.
ASR
74.5
81.4
The results indicate that we can apply our algorithm with online API services. We acknowledge that this part of the experimental design has certain limitations. Not every fa- mous painting we selected may be present in DALL-E 2's training set, and our construction of non-members may ex- hibit some distribution shift relative to the member dataset. Here, we aim to provide real-world application for black- box evaluation, leaving a more comprehensive experimental design future work.
7. Conclusion, Limitations and Future Directions
The results, presented in Table 4, demonstrate that our al- gorithm achieves a relatively high accuracy under this eval- uation method. Our observation is illustrated in Figure 6. As seen in the figure, for Monet's iconic painting "Water Lily Pond", the original artwork shows minimal changes when using DALL-E 2's variation API, retaining most of its main features. In contrast, the artwork generated by Stable Diffusion 3 undergoes significant changes, with variations in both the number of flowers and lily pads. Therefore, we hypothesize that artworks with smaller changes after API usage are more likely to have appeared in the model's 's training set. Other results of changes to the member and
In this work, we introduce a novel membership inference attack method specifically designed for diffusion models. Our approach only requires access to the variation API of the model, bypassing the need for internal network compo- nents such as the U-net. This represents advancement for commercial diffusion models, which typically restrict internal access. We demonstrate the effectiveness of our approach across various datasets, showing that achieves high accuracy in identifying whether an image was included in the training dataset. Our algorithm can detect data misuse
8

---

Towards Black-Box Membership Inference Attack for Diffusion Models
by the model, representing step forward in protecting the copyright of artworks.
Dosovitskiy, A. An image is worth 16x16 words: Trans- formers for image recognition scale. arXiv preprint arXiv:2010. 1929, 2020.
However, our method has certain limitations, particularly the requirement for a moderate diffusion step in the variation API. The algorithm's 's accuracy declines when the diffusion step is excessively high. As such, we propose our method as an initial step towards black-box MIA, with a more com- prehensive solution left for subsequent exploration.
Duan, J., Kong, F., Wang, S., Shi, X., and Xu, K. Are diffu- sion models vulnerable membership inference attacks? In International Conference on Machine Learning, pp. 8717-8730. PMLR, 2023.
Duddu, V., Boutet, A., and Shejwalkar, V. Quantifying privacy leakage in graph embedding. In MobiQuitous 2020-17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Ser- vices, pp. 76-85, 2020.
Future work could focus on developing more robust algo- rithms capable of handling broader range of diffusion steps. Improving the interpretability of our method and extending it to other generative models are also valuable directions for further research.
Durret, R. Probability, theory and examples, cambride ser. Stat. and Prob. Math., Cambridge University Press, New York, pp. 65, 2010.
Impact Statement
This paper addresses the critical issue of protecting intellec- tual property and data privacy in the era of rapidly advancing generative models. By proposing a novel membership in- ference attack (MIA) method, our work aims to identify whether specific artworks have been used during the train- ing of diffusion models. This has significant implications for safeguarding creators' copyrights, detecting unautho- rized use of artistic works, and promoting ethical practices in the development and deployment of generative models. While our method is designed with privacy protection in mind, we acknowledge the potential for misuse. We hope this research serves as a foundation for further discussions on data privacy and ethical considerations, encouraging re- sponsible use and development of diffusion models in the community.
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F,, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning. 2024.
Hayes, J., Melis, L., Danezis, G., and De Cristofaro, E. Logan: Membership inference attacks against generative models. arXiv preprint arXiv: /05.00660,.0663,, 2017.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Hilprecht, B.. Härterich, M., and Bernau, D. Monte carlo and reconstruction membership inference attacks against generative models. Proceedings on Privacy Enhancing Technologies, 2019.
References
Chen, D. Yu, N., Zhang, Y., and Fritz, M. Gan-leaks: A taxonomy of membership inference attacks against generative models. In Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, pp. 343-362, 2020.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6855 6851, 2020.
Coates, A., Ng, A., and Lee, H. An analysis of single- layer networks in unsupervised feature learning. Pro- ceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02330, 2022a.
Deng, J., Dong. W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., and Fleet, D. J. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646, 2022b.
Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794 2021.
Hu, H. and Pang, Membership inference of diffusion models. arXiv preprint arXiv:2301.0940 009956, 2023.
9

---

Towards Black-Box Membership Inference Attack for Diffusion Models
Mao, X. The truncated euler-maruyama aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa method for stochas- tic differential equations. Journal of Computational and Applied Mathematics, 290:370-384, 2015.
Huang, R., Lam, M. W. Wang, J., Su, D., Yu, D., Ren, Y., and Zhao, Z. Fastdiff: A fast conditional diffusion model for high-quality speech synthesis. arXiv preprint 0rXiv:2204.09934 2022.
Matsumoto, T., Miura, T., and Yanai, N. Membership in- ference attacks against diffusion models. In 2023 IEEE Security and Privacy Workshops (SPW), pp. 77-83. IEEE, 2023.
Kingma, D., Salimans, T., Poole, B., and Ho, J. Varia- tional diffusion models. Advances in neural information processing systems, 34:2166-211777, 2021.
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:21 12. 2021.
Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 2013.
Kong, F., Duan, J., Ma, R., Shen, H., Zhu, X., Shi, X., and Xu, K. An efficient membership inference attack for the diffusion model by proximal initialization. arXiv preprint arXiv:2305. 18355, 2023.
Pang. Y. and Wang, T. Black-box membership inference at- tacks against fine-tuned diffusion models. arXiv preprint 2at::2322.020.. 2023.
Kong, Z., Ping, W., Huang, J., Zhao, K., and Catanzaro, B. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.0977
Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEF/CVF Interna- tional Conference on Computer Vision, pp. 4195-4205, 2023.
2020.
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Popov, V., Vovk, I., Gogoryan, V., Sadekova, T., and Kudi- nov, M. Grad-tts: A diffusion probabilistic model for text-to-speech. International Conference on Machine Learning, pp. 8599-8608. PMLR, 2021.
Li, J., Li, D., Xiong, C., and Hoi, S. Blip: Bootstrapping language-iman image pre-training for unified vision- languaee understanding and generation. In International confer- ence machine learning, pp. 12900. PMLR, 2022.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C.,, and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06112 06125, 1(2):3, 2022.
Li, Z. and Zhang, Y. Membership leakage in label-only exposures. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 880-895, 2021.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ra- manan, D., Dollár, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740- 755. Springer, 2014.
Saharia, C., Chan, W., Chang, H., Lee, C., Ho, J., Salimans, T., Fleet, D., and Norouzi, M. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022a.
Long. Y., Bindschaedler, V., Wang. L., Bu, D., Wang, X., Tang, H., Gunter, C. A., and Chen, K. Understanding membership inferences on well-generalized learning mod- els. arXiv preprini arXiv: 1002v 1802.0488, 2018.
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E. L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al. Photorealistic text-to-image dif- fusion models with deep language understanding. Ad- vances in neural information processing systems, 35: 36479-36494, 2022b.
Lugmayr, A.. Danelljan, M., Romero, A., Yu, F., Timofte, R., and Van Gool, L. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 1461111477, 2022.
Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz,, M., and Backes, M. MI-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.012. 2018.
Mahloujifar, S., Inan, H. A., Chase, M., Ghosh, E., and Hasegawa, M. Membership inference on word embedding and beyond. arXiv preprint arXiv:2 1384, 2021.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis,
10

---

Towards Black-Box Membership Inference Attack for Diffusion Models
C., Wortsman, M., et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35: 19278-252999 2022.
Yeom, S., Giacomelli, Fredrikssn, M., and Jha, S. Privacy risk in machine learning: Analyzing the connection overfitting. In 2018 IEEE s1 computer security founda- tions symposium (CSF), pp. 268-282, IEEE, 2018.
Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Mem- bership inference attacks against machine learning mod- els. In 2017 IEEE symposium on security and privacy (SP), pp. 3-18. IEEE, 2017.
Yu, J., Xu, Y., Koh, J. Y., Luong, Baid, G., Wang, Z., Va- sudevan, V., Ku, A., Yang, Y., Ayan, B. K., al. Scaling autoregressive models for content-rich text-to-image gen- eration. arXiv preprint arXiv:2206.10788 2(3):, 2022.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequi- librium thermodynamics. In International conference on machine learning, pp. 2256-2265 PMLR, 2015.
Song, C. and Raghunathan, A. Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAC conference computer and communications security, pp. 377-390, 2020.
Song, J., Meng, C., and Ermon, S. Denoising diffusion im- plicit models. arXiv preprint arXiv:20/0.02500 2020a.
Song. Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13455 2020b.
Tang, S., Wu, 7. S., Aydore, S., Kearns, M., and Roth, A. Membership inference attacks on diffusion models via quantile regression. arXiv preprint arXiv:23/2.05140, 2023.
Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. Image quality assessment: from error visibility to struc- tural similarity. IEEE transactions on image processing, 13(4):600- 612,,11222220222 2004.
Wu, C. H. and De la Torre, F. A latent space of stochastic diffusion models for zero-shot image editing and guid- ance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7378-7387, 2023.
Wu, J. Z., Ge, Y., Wang, X., Lei, S. W., Gu, Y., Shi, Y., Hsu, W., Shan, Y., Qie, X., and Shou, M. Z. Tune-a- video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 7623- 7633, 2023.
Wu, Y., Yu, N., Li, Z., Backes, M., and Zhang, Y. Member- ship inference attacks against toxt-to-image generation models. arXiv preprint arXiv:2210. 10.00688, 2022.

---

Towards Black-Box Membership Inferenee Attack for Diffusion Models
A. Datasets, Models and Hyperparameters
We use NVIDIA RTX 6000 graphics cards for all our experiments.
For DDIM, we follow the training hyperparameters of (Duan et al., 2023) to train a DDIM model on the CIFAR- 10/100 (Krizhevsky et al., 2009) and STL.1O-Unlablled (Coates et al., 2011) datasets, using 1000 image generation steps (T' 1000) and sampling interval of k 100. The training iterations are set to 800,000. For all the datasets, we randomly select 50% of the training samples to train the model and designate them as members. The remaining 50% of the training samples are used as nonmembers.
We use (Matsumoto et al., 2023; Duan et al., 2023; Kong et al., 2023) as our baseline methods. We use their official code repositories and apply the optimal hyperparameters from their papers. For our algorithm, we fix the diffusion step f 200 and independently call the variation API 10 times average the output images as
For the Diffusion Transformers, we train the model on the ImageNet (Deng al., 2009) dataset, using 1000 image generation steps (7 = 1000), while for other training hyperparameters, we follow the setup of (Peebles & Xie, 2023). We randomly select 100,000 images from the ImageNet training set to train the model at resolutions of either 128 x 128 or 256 x 256. For the 128 x 128 image size, we use 200,000 training iterations. For the 256 x 256 image size, we use 300,000 training iterations. These numbers of training iterations are chosen to ensure the generation of high-quality images.
For the membership inference attack setup, 1000 images are randomly selected from our training set as the member set, and another 1000 images randomly chosen from the ImageNet validation set the non-member set. We fix the diffusion step at 100 and the DDIM step at le 50, and independently call the variation API 10 times to average the output images as 2.
We also use (Matsumoto et al., 2023; Duan et al., 2023; Kong et al., 2023) as our baseline methods. Since these papers do not study Diffusion Transformers, we adapt their algorithms to the DiT framework for evaluation. We fix the DDIM step at = 50 and choose diffusion steps E [50, 100, 150, 200, 250, 300), recording their optimal solutions under these different hyperparameter settings.
For the Stable Diffusion experiments, use the original Stable Diffusion model, i.e., stable-diifusion----4 provided by Huggingface, without further fine-tuning modifications. We follow the experimental setup of (Duan et al., 2023; Kong et al.. 2023), using an image generation step of T = 1000 and a sampling interval of k = 10. We use the LAION-5B dataset (Schuhmann et al., 2022) as the member set and COCO2017-val (Lin et al., 2014) as the non-member set. We randomly select 2500 images from each dataset. We test two scenarios: knowing the ground truth text, denoted as Laion; and generating text through BLIP (Li al., 2022), denoted Laion with BL.P.
We use the hyperparameters from the papers (Duan et al., 2023; Kong al., 2023) to the baseline methods. For algorithms REDIFFUSE, we fix the diffusion step t 10 to call the variation API and directly use the SSIM metric (Wang et al., 2004) to measure the differences between two images. Other hyperparameters remain the same as in the baseline methods.
12

---

Towards Black-Box Membership Inference Attack for Diffusion Models
B. More Experiment Results
In this section, we show other experiment results which is not in our main paper. We conduct more ablation studies.
The Impact of Average Numbers We use different average numbers n and test the influence on the results. Besides the figures AUC in the main paper, the figures of ASR of DDIM and Diffusion Transformer are also plotted in Figure In addition, we plot the figures of AUC and ASR for Stable Diffusion in Figure We observe that in the DDIM and Diffusion Transformer setup, averaging the images from multiple independent samples the output further improves accuracy. In the stable diffusion setup, since the image size in the dataset is larger (512x512), the reconstructed images more stable and not influenced by perturbations specific coordinates. Therefore, averaging multiple images is not necessary.
0.96
0.975
0.94
0.950
ASR
0.90
0.900
0.86
0.875
CIFAR100
STL10
0.850
ImageNet 128-128
TINY-IN
ImageNet 256
0.82
10 Average numbers
15
Average numbers
Figure 7: The impact of average numbers. Left: DDIM model on CIFAR-10. Right: Diffusion Transformer model on Imagenet. Averaging can further improve the performance of our algorithm.
0.95
LaionS
0.85
Laion5
0.90
LaionS with BLIP
Laion5 with BLIP
0.80
0.85
AUG
0.80
ASR
0.75
0.75
0.70
0.70
0.65
0.60
10
20
Average numbers
Average numbers
Figure 8: The impact of average numbers on Stable Diffusion. We plot the AUC and ASR metrics, and averaging does not improve performance.
The Impact of Diffusion Steps We adjust the diffusion step / to examine its impact on the results. We train the DDIM model on 100 (Figure 9), STLI0 (Figure 10) dataset, Diffusson Transformer on the Imagenet 256 x 256 (Figure 11)aatte and Stable Diffusion on Laion5 dataset (Figure 12). From the results, we see that our algorithm can achieve high performance over a wide range diffusion steps. This opens up possibilities for practical applications in real-world scenarios.
13

---

Towards Black-Box Membership Inference Attack for Diffusion Models
1.0
AUC
ASR
AUC ancandd Accurscy
0.8
0.8
steps
Figure 9: The impact of diffusion steps. We train a DDIM model on the CIFAR- 100 dataset and use different diffusion step for inference.
AUIC
AUC and Accuracy
0.8
0.6
0.5
100
200
300
400
Diffusion steps
Figure 10: The impact of diffusion steps. We train DDIM model on the STL-10 dataset and use different diffusion step for inference.
AUC
AUC and Accuracy
0.7
0.6
0.5
200
400
600
Diffusion steps
Figure 11: The impact of diffusion steps. We train a Diffusion Transformer model on the Imagenet 256 x 256 dataset and use different diffiusion step for inference. The robust results imply that our algorithm is also not very sensitive to the choice of in this setup.
0.80
AUC
ASR
AUC and Accuracy
0.75
0.70
0.65
0.60
0.55
Diffusion stegs
Figure 12: The impact of diffusion steps. We use the Stable Diffusion model with the Laion5 dataset for evaluation and test different diffusion steps for inference. Specifically, using relatively small diffusion steps results in better performance.
14

---

Towards Black-Box Membership Inference Attack for Diffusion Models
The Impact of Sampling Intervals We change the sampling intervals to if there is any influence the results. In addition the AUC figures in the main paper, the ASR figures of DDIM and Diffusion Transformer also plotted in Figure 13. We also plot the AUC and ASR for Stable Diffusion in Figure 14. From the results, we observe that our algorithm consistently performs well across different sampling intervals.
0.94
ImageNttt 120"129
ImageNet 256*256
0.92
0.90
0.88
CIRAR10
0.84
STL10
TIMHN
0.86
0.82
10
20
25
50
100
steps
DDIM steps
Figure 13: The impact of sampling intervals. Left: DDIM model on CIFAR-10. Right: Diffusion Transformer on Imagenet. We find that adjusting the sampling interval does not significantly affect of the accuracy.
Laion
0.90
Laion5 with BLIP
0.85
LeionS with BUP
0.85
0.80
0.75
0.75
0.70
0.65
0.65
10
25
50
100
0.60
10
25
50
DDIM
DDIM steps
Figure 14: The impact of sampling intervals on Stable Diffusion. We plot the AUC and ASR metrics, observe that different sampling intervals have minimal impact on it.
The ROC curves We plot ROC curves for the DDIM train CIFAR- 10 and Diffusion Transformer train ImageNet 256 x 256 in Figure 15. The curves further demonstrate the effectiveness of our method.
ROC
curve
1.0
0.5
Postive
0.6
ReGiMfuse
0.4
Loss
SecVi
0.2
PM
PAN
PIA
0.0
Random Cuess
0.0
Random Duess
0.2
0.4
0.8
1.0
False Rositive Rate
False Positive Rate
Figure 15: The ROC curves of various setups. Left: DDIM model on CIFAR-IO. Right: Diffusion Transformer on ImageNet 256 x 256. The curves show that our algorithm outperforms the baseline algorithms.
15

---

Towards Black-Box Membership Inference Attack for Diffusion Models
C. Proof
In this section, we present the proof of the theorem.
Theorem 4.2. Suppose the DDIM model can learn parameter such that, for any x ~ Droining with dimension d, the with zero expectation and finite
prediction error e E- ca( (elvāir + v1 - e,e, t) a random variable X = (2,..x,...... X4) cumulant cumulant-generatingg generating function for each coordinate (Durret, 2010). Suppose the sampling interval is equal to the variation API diffusion step I. Let & be the average of ouiput images of using the variation API. Then we have
Byas Vd(11 āe.)
P(I - dexp
-n V X,
where B > O and V x, is the Fenchel-Legendre dual of the comullnt-generaainn function V X.-
Proof. We denote x' the i-th output image of æ using the variation API. We denote the i-th Gaussian noise e and the forward process yields z V + vI ayc'.
As the sampling interval is equal to the variation API diffusion step 1, we have
2 vI aeol + vI āgE', I,
x² -
Vana + vI -aze" v1 areal + v1 ane',()
Vāa
V1
- colVan + -I-
Val
If we denote the random variable X [X).X,,.. .2.., 8t E1 Xj. From the assumption know that all the X are i.i.d. random variables(with the same distribution of X; in Theorem 4.2) with zero expectation and finite cumulant- -geeerang function xx, (s) = log E[e*x.] < +oo. Using the theorem of Cramér-Chernoff method for sums of i.i.d. random variables (Durret, 2010), we get the following probability inequality for any 8 >
X.) represents e¹ 155-oeVVa calVār vI- age", (gce,,) t). then we consider
P(S) 2 B) < exp(-nVV (
where 3 > 0 and V (v) sup x (sy- xx (s)) is the Fenchel-Legende dual of the cumulant-generating function W Mx.
Therefore, denote S, (S), S ... 5. taking the definition of || xII = =1 Ei (x+
x)]1
1/-
we get ||
Ry Ure
the following bound of the reconstruction error:
nBvor ) dexp(-nminn -- V d(1 ac)
B )), d(1 a.)
P(I - 2 B)
P(IS
So averaging the randomly reconstructed data from a training will result in smaller reconstruction error with high probability of O(exp(-n)) when large
16

---

Towards Black-Box Membership Inference Attack for Diffusion Models
D. More Results of Variation Images
In this section, we provide more results the variation of member and nonmember image when applying to DALL-E variation API. The experimental results are recorded in Figure 16, from which we can see that the variation in the member inputs after applying the DALL-E 2's variation API is relatively small than non-member inputs.
Members
After Variation
Non-members
After Variation
Figure 16: More results the variation images. From the figures we can see that DALL-EE 2's variation API makes less changes to images in member than non-member
17


Reviews:
[{"review_id": "ae2gXKpqxn", "reviewer": "Reviewer_1UDx", "summary": "The paper introduces a novel black-box membership inference attack method for diffusion models. The authors show their method can reliably detect whether an image was part of the training set or not. They do this by repeatedly applying the variation API and averaging the outputs. They have extensive experiments across multiple diffusion architectures (DDIM, Stable Diffusion, and Diffusion Transformer) and datasets (e.g., CIFAR-10/100, STL10, ImageNet, LAION-5B) to show that REDIFFUSE outperforms existing white-box based methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's positive feedback. We address the questions in detail below:\n\n>Q1: When comparing to other existing methods, this paper do not show a comparison in runtime/cost. I'd suggest to add these details for their audience to have a better understanding of the trade offs between these methods.\n\n**A1:** We thank the reviewer for raising this important point regarding computational cost comparison. From a computational complexity perspective, the runtime primarily depends on the average number $n$, where each detection requires $n$ times the computation of the baseline method. To better illustrate this trade-off, we conduct additional experiments measuring runtime across different values of $n$, evaluating DDIM on CIFAR-100 and DiT on ImageNet 256×256. The results are as follows:\n\n|Method|DDIM|DiT|\n|:---:|:----:|:----:|\n|Loss[1]|0.92|0.78|\n|SecMI[2]|0.96|0.88|\n|PIA[3]|0.96|0.91|\n|PIAN[3]|0.91|0.67|\n|**ReDiffuse with $n=1$ (Ours)**|0.94|0.94|\n|**ReDiffuse with $n=5$ (Ours)**|0.97|0.95|\n|**ReDiffuse with $n=10$ (Ours)**|**0.98**|**0.97**|\n\nThe results show our method achieves accuracy comparable to baselines even at $n=1$, with matching runtime and no UNet access required. When increasing $n$, the extra computation time further improves performance. In contrast, baseline methods rely on deterministic UNet outputs (no randomness), so they can’t benefit from averaging. We believe this cost is reasonable because at $n=10$, our algorithm infers $100,000$ CIFAR-100 images in ~5 minutes on an NVIDIA L40 GPU. We’ll add these discussions to the paper.\n\n\n>Q2: Did you try this method for detecting copyrighted or proprietary content in foundation models?\n\n**A2:** We thank the reviewer for this question. In **Section 6**, we discuss an application scenario where we perform membership inference attacks using the variation API provided by OpenAI DALL-E [4], a popular API-only model. Specifically, we constructed a dataset consisting of famous artworks and AI-generated artworks with the same titles. We then conduct detection tests to determine whether certain famous artworks are included in DALL-E's training dataset. The results show our approach can effectively work with real-world diffusion-model APIs. We plan to extend this to other copyright-protected content as future work.\n\nOnce again, we sincerely thank the reviewer for the constructive comments, and we are eager to engage in further discussions to clarify any concerns.\n\n\n**References**\n\n[1] Matsumoto, Tomoya, Takayuki Miura, and Naoto Yanai. \"Membership inference attacks against diffusion models.\" 2023 IEEE Security and Privacy Workshops (SPW). IEEE, 2023.\n\n[2] Duan, Jinhao, et al. \"Are diffusion models vulnerable to membership inference attacks?.\" International Conference on Machine Learning. PMLR, 2023.\n\n[3] Kong, Fei, et al. \"An efficient membership inference attack for the diffusion model by proximal initialization.\" arXiv preprint arXiv:2305.18355 (2023).\n\n[4] The variation API of DALL-E. https://platform.openai.com/docs/guides/images/variations-dall-e-2-only", "claims_and_evidence": "They have majorly supported their claim through comprehensive empirical results (and based on the theoretical result). The authors provide detailed quantitative comparisons (via AUC, ASR, and true positive rates) across several benchmark datasets.", "methods_and_evaluation_criteria": "Their empirical results are very extensive and includes both DDIM and Stable Diffusion models tested on most of the commonly used classical datasets. Also the choice of evaluation metrics like AUC, ASR, and true positive rate at a fixed false positive rate is natural and common for MIA methods.", "theoretical_claims": "The theoretical contribution in Theorem 4.2 gives an error bound for the averaged output of the variation model (API) under certain assumptions:\n1. unbiased noise prediction, 2. full rank of the Jacobian\nThe proof is mathematically coherent. The assumptions may not necessarily be realistic, but the theorem provides an intuition for the test statistic and is shown to be accurate in practice.", "experimental_designs_or_analyses": "As mentioned before, their experimental design, and choice of models and datasets is valid. They have extensive ablations for various factors such as average numbers, diffusion steps, and sampling intervals.", "relation_to_broader_scientific_literature": "Since major Diffusion model developers in the industry are not transparent about their training datasets, advancements in MIA methods are very impactful as they help us gain a bit more clarity. These methods can also help detect private and copy righted data used for training.", "essential_references_not_discussed": "Not that I am aware of.", "other_strengths_and_weaknesses": "The main strength of the paper is that it's practical and easy to use in the real-world on proprietary models without internal access. Their extensive experiments and their theoretical analysis provide together provide a solid foundation for the claims. \nHowever, when comparing to other existing methods, they do not show a comparison in runtime/cost. I'd suggest to add these details for their audience to have a better understanding of the trade offs between these methods.", "other_comments_or_suggestions": "Please add run-time and cost estimations (both for your method and other existing methods)", "questions_for_authors": "Did you try this method for detecting copyrighted or proprietary content in foundation models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "14HYpFYaZ0", "reviewer": "Reviewer_nFid", "summary": "This paper investigates black-box membership inference attacks against diffusion models where attacker has no access to the internal model. The target of attacker is to determine whether or not an artwork was used to train a diffusion model. In this paper, authors firstly identify the limitation of applying existing MIAs for proprietary diffusion models and then propose a novel black-box membership inference attack to determine the membership privacy of an image. Users validate the proposed method using DDIM and Stable Diffusion models on benchmark datasets and further extend both the proposed approach and existing algorithms to the Diffusion Transformer architecture. Experimental results show the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We express our gratitude to the reviewer for the insightful comments and suggestions. Please find the details below. \n\n>Q1: Is the intuition of the proposed method general to other datasets? Or just observable to specific datasets utilized in these datasets??\n\n**A1:** Our method has been evaluated across diverse datasets (CIFAR-10/100, STL-10, ImageNet, LAION-5) in the main paper, demonstrating its generalizability. To further validate this, we conduct additional experiments: we train a DDIM model on Tiny-ImageNet [1] and SVHN [2]. For each dataset, we randomly select 50,000 member and 50,000 non-member images, training the DDIM model for 800K iterations using Appendix A's hyperparameters. The result are as follows:\n|Dataset|Tiny-ImageNet|SVHN|\n|:---:|:----:|:----:|\n|AUC|0.98|0.95|\n|ASR|0.95|0.88|\n\nThe experimental results demonstrate that our method remains effective on these new datasets. We will include these findings in the revised manuscript and plan to extend evaluation to other datasets in future work.\n\n\n>Q2: There are other diffusion models in existing works. Is the proposed attack methodology suitable to other diffusion models?\n\n\n**A2:** We appreciate the reviewer’s question. Our main paper already demonstrates the method's effectiveness on DDIM, Diffusion Transformer, and Stable Diffusion. To further validate generalizability, we conduct additional experiments with DDPM[3] across four datasets. For each dataset, we randomly select 50,000 member and 50,000 non-member images, training the DDPM model for 800K iterations using Appendix A's hyperparameters. The results are as follows:\n\n|Dataset|CIFAR-10|CIFAR-100|STL-10|Tiny-ImageNet|\n|:---:|:----:|:----:|:----:|:----:|\n|AUC|0.87|0.85|0.81|0.89|\n|ASR|0.80|0.78|0.75|0.82|\n\nThe experimental results confirm our method's effectiveness on DDPM models. We will include these findings in the revised manuscript. Due to the need for dataset-specific retraining, we cannot evaluate additional models within the rebuttal period. We plan to extend this evaluation to other diffusion architectures in future work.\n\nFinally, we thank the reviewer once again for the efforts in providing us with valuable and helpful suggestions. We will continue to provide clarifications if the reviewer has any further questions.\n\n**Reference**\n\n[1] Tiny ImageNet Dataset, Stanford University.  Available: http://cs231n.stanford.edu/tiny-imagenet-200.zip. \n\n[2] Netzer, Yuval, et al. \"Reading digits in natural images with unsupervised feature learning.\" NIPS workshop on deep learning and unsupervised feature learning. Vol. 2011. No. 2. 2011.\n\n[3] Ho, Jonathan, Ajay Jain, and Pieter Abbeel. \"Denoising diffusion probabilistic models.\" Advances in neural information processing systems 33 (2020): 6840-6851.", "claims_and_evidence": "Claims made in the submission supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The evaluation criteria are commonly used in the existing literature.", "theoretical_claims": "The proofs for theoretical claims are correct.", "experimental_designs_or_analyses": "Authors adopted three commonly used metric, i.e., AUC, ASR, and TP to evaluate the performance of the proposed method compared with other baselines. And the experimental results can show the improved performance of the proposed method. In addition, parameter analysis, such as the diffusion steps, average numbers, has also been conducted to evaluate the proposed method.", "relation_to_broader_scientific_literature": "Even though there are existing works focus on MIA on diffusion models, this paper provides a more practical scenario, i.e., with only API query access.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "n/a", "other_comments_or_suggestions": "n/a", "questions_for_authors": "1. Is the intuition of the proposed method general to other datasets? Or just observable to specific datasets utilized in these datasets? \n2. There are other diffusion models in existing works. Is the proposed attack methodology suitable to other diffusion models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JZPuj9QLTe", "reviewer": "Reviewer_ZaW2", "summary": "The paper proposes using the average of img2img outputs and comparing it with the original input image, with the difference serving as a metric for black-box MIA. This approach eliminates the need for predicted noise at intermediate time steps, making it applicable to a broader range of scenarios while achieving stronger results than previous methods.\n\n## Update after Rebuttal\n\nThe rebuttal addresses my concerns, and I recommend acceptance of the paper.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments and constructive suggestions. In the following, we address the main concern raised. \n\n>Q1: The author assumes for a well-trained diffusion model with full rank Jacobian. I am curious if there is a deeper motivation behind this assumption. One possible intuition is that it might be too strong to directly assume the loss function of the DM is sufficiently low, so the focus shifts to hypothesizing a property that is easier to satisfy than the loss function itself. Nonetheless, I would appreciate a more detailed explanation or clarification regarding this choice.\n\n**A1:** We appreciate the reviewer’s feedback. We hypothesize the full-rank condition of Jacobian because the neural network's dimensionality $p$ significantly exceeds that of the image data $d$. Hence a full rank matrix would require the $p \\times d$ matrix to have rank $d$ and hence is most likely true.\n\nConversely, if the Jacobian are rank-deficient, there would exist initial points from which denoising could not recover plausible images, contradicting the empirically demonstrated generation capabilities of modern diffusion models. Still, we agree with the reviewer that we cannot verify this for all data points as computing the rank of a huge matrix is expensive. We will add the discussion of this assumption in the next version of our paper.\n\n\n>Q2: I’m curious about one aspect of the ablation study. In Figure 5, the AUC decreases when the DDIM steps exceed 25. This seems counterintuitive, as a larger number of DDIM steps should theoretically reduce error and improve accuracy. Could you clarify why this occurs?\n\n\n**A2:** We appreciate the reviewer’s question. Our results in Figure 5 demonstrate that changing the DDIM step has minimal impact on detection accuracy. For DDIM steps of 20, 25, 50, and 100, the AUC remains virtually unchanged (difference < 0.003). We present relevant experiments of DDIM on CIFAR-100 here, with different random seeds and DDIM steps: \n|Random Seed|1|2|3|\n|:---:|:----:|:----:|:----:|\n|DDIM with step 20|0.968|0.969|0.971|\n|DDIM with step 25|0.971|0.969|0.970|\n|DDIM with step 50|0.970|0.970|0.971|\n|DDIM with step 100|0.967|0.970|0.968|\n\nFrom the results, we know that as we vary random seeds, the AUC remains similar across these DDIM steps, with no consistent decreasing trend observed as step size increases.\n\n\nWe thank the reviewer once again for the valuable and helpful suggestions. We would be happy to provide further clarifications if the reviewer has any additional questions.", "claims_and_evidence": "The author assumes that for a well-trained dm:\n> the Jacobian matrix $\\nabla_{\\theta}\\epsilon(x_t,t)$ is full rank. \n\nI am curious if there is a deeper motivation behind this assumption. One possible intuition is that it might be too strong to directly assume the loss function of the DM is sufficiently low, so the focus shifts to hypothesizing a property that is easier to satisfy than the loss function itself. Nonetheless, I would appreciate a more detailed explanation or clarification regarding this choice.", "methods_and_evaluation_criteria": "The method is clear and the evaluation criteria makes sense.", "theoretical_claims": "I check the proof and it should be correct.", "experimental_designs_or_analyses": "The experiments are generally solid, but I’m curious about one aspect of the ablation study. In Figure 5, the AUC decreases when the DDIM steps exceed 25. This seems counterintuitive, as a larger number of DDIM steps should theoretically reduce error and improve accuracy. Could you clarify why this occurs?", "relation_to_broader_scientific_literature": "The paper proposes a new method for MIA that is more practical for most closed-source diffusion models. The approach is simple yet effective, significantly strengthening MIA and yielding more promising results for potential applications in copyright authentication.", "essential_references_not_discussed": "Not found.", "other_strengths_and_weaknesses": "No.", "other_comments_or_suggestions": "See comments above.", "questions_for_authors": "Seem comments above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m0OfIWzMX6", "reviewer": "Reviewer_mEXb", "summary": "This paper introduces a black-box membership inference attack method targeting diffusion models. Unlike previous MIA approaches that require access to the U-net or other internal components of diffusion models, their method only utilizes the variation API to determine whether a given image was part of the model's training data.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the comments and suggestions. Below, we address the primary concern that has been raised.\n\n>Q1: The proposed REDIFFUSE algorithm relies on the existence of a variation API. While the paper claims robustness across different diffusion steps, the experimental results suggest that selecting a poor diffusion step could impact detection performance.\n\n**A1:** We thank the reviewer for the comments. As shown in Figure 4 of our paper, our method achieves over 80% accuracy when the diffusion step is between 50 and 350. On the other hand, the variation API in diffusion models[1] typically uses a moderate step value to edit the image: **too few steps produce negligible edits, while too many steps cause excessive distortion, making it hard to preserve consistency with the original image**. Thus, our results suggest that the variation API could be viable for detection tasks. We leave more robust algorithm design across diffusion steps for future work.\n\n>Q2: The real-world validation with DALL-E 2 is based on only 30 famous artworks.\n\n**A2:** We appreciate the reviewer’s question. The purpose of Section 6 is to demonstrate the practical applicability of our algorithm to commercial models. Specifically, we use DALL-E 2’s variation API to show how our method can detect membership by leveraging the model’s outputs. Since DALL-E 2 **does not provide a publicly accessible training set**, the experiment in Section 6 primarily serves as **demonstrative application scenario rather than an extensive evaluation with large-scale datasets**. We agree that further evaluation on more diverse datasets and models would enhance the robustness of the findings, and we add additional experiments of WikiArt dataset[2]. We randomly sample 1000 images of the dataset as members and generate corresponding nonmembers with the method in Section 6 in our paper. The results are as follows:\n\n|Metrics|$L_1$ distance|$L_2$ distance|\n|:---:|:----:|:----:|\n|AUC|0.70|0.74|\n|ASR|0.67|0.70|\n\nThe experimental results indicate that our method remains effective on this new dataset. However, since many artworks in WikiArt are less prominent than the 30 canonical paintings we initially selected, they likely had lower probability of being included in DALL-E 2's training data. This introduces potential dataset bias when expanding the evaluation scope. We leave more comprehensive large-scale experiments for future work.\n\nWe thank the reviewer once again for the valuable and helpful suggestions. We will continue to provide clarifications if the reviewer has any further questions.\n\n\n**References**\n\n[1] Meng, Chenlin, et al. \"Sdedit: Guided image synthesis and editing with stochastic differential equations.\" arXiv preprint arXiv:2108.01073 (2021).\n\n[2] \"WikiArt Visual Art Encyclopedia.\" *WikiArt*, n.d., https://www.wikiart.org.", "claims_and_evidence": "Several key claims are generally supported by empirical evidence.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "The paper includes a mathematical justification for why training images produce more stable reconstructions than non-members.", "experimental_designs_or_analyses": "The experimental design is generally well-structured.", "relation_to_broader_scientific_literature": "The paper extends prior work on membership inference attacks by introducing a novel black-box attack tailored for diffusion models.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Strengths:\nThe paper presents a black-box MIA attack on diffusion models that does not require U-net access.\nEvaluations are thorough, covering multiple datasets, architectures.\nThe authors provide mathematical support for their attack’s effectiveness.\n\nWeaknesses:\nThe proposed REDIFFUSE algorithm relies on the existence of a variation API. \nWhile the paper claims robustness across different diffusion steps, the experimental results suggest that selecting a poor diffusion step could impact detection performance.\nThe real-world validation with DALL-E 2 is based on only 30 famous artworks.", "other_comments_or_suggestions": "None.", "questions_for_authors": "see Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper studies the black box membership inference attacks for Diffusion Models.

All the reviewers agree that the proposed method is effective and the experiments are comprehensive and solid. Therefore, I recommend the acceptance the of paper.
Decision: Accept (poster)