Dual-Model Defense: Safeguarding Diffusion Models from Membership Inference Attacks through Disjoint Data Splitting

## Abstract

Abstract Diffusion models have demonstrated remarkable capabilities in image synthesis, but their recently proven vulnerability to Membership Inference Attacks (MIAs) poses a critical privacy concern. This paper introduces two novel and efficient approaches (DualMD and DistillMD) to protect diffusion models against MIAs while maintaining high utility. Both methods are based on training two separate diffusion models on disjoint subsets of the original dataset. DualMD then employs a private inference pipeline that utilizes both models. This strategy significantly reduces the risk of black-box MIAs by limiting the information any single model contains about individual training samples. The dual models can also generate “ soft targets ” to train a private student model in DistillMD, enhancing privacy guarantees against all types of MIAs. Extensive evaluations of DualMD and DistillMD against state-of-the-art MIAs across various datasets in white-box and black-box settings demonstrate their effectiveness in substantially reducing MIA success rates while preserving competitive image generation performance. Notably, our experiments reveal that DistillMD not only defends against MIAs but also mitigates model memorization, indicating that both vulnerabilities stem from overfitting and can be addressed simultaneously with our unified approach.

## 1 Introduction

In recent years, diffusion models (Sohl-Dickstein et al., ; Ho et al., ; Rombach et al., ) have rapidly emerged as a powerful tool for image generation, outperforming traditional methods such as Generative Adversarial Networks (GANs) (Goodfellow et al., ) and Variational Autoencoders (VAEs) (Kingma, ). These models, including well-known examples like Stable Diffusion models (Rombach et al., ; Podell et al., ), DALL-E 2 (Ramesh et al., ) and Imagen (Saharia et al., ), utilize a progressive denoising process that results in higher-quality and more stable image generation compared to previous architectures. By gradually transforming random noise into clean images, diffusion models excel at producing detailed and realistic visuals across various applications, from graphic design to medical imaging.

However, the superior performance of diffusion models relies heavily on large and diverse datasets, which often include sensitive information such as copyrighted images, personal photos, medical data, and even stylistic elements from contemporary artists. The nature of these datasets poses significant privacy risks, as diffusion models can inadvertently memorize and reproduce parts of their training data during the generation (Carlini et al., ). This replication of training data during inference makes diffusion models vulnerable to Membership Inference Attacks (MIAs) (Shokri et al., ; Matsumoto et al., ; Wu et al., ), which aim to determine whether specific samples are present in their training data. If a model has been trained on sensitive datasets, an attacker might extract or infer specific details about the data used in training, leading to unintended exposure of private or proprietary information.

Therefore, implementing robust defense mechanisms to protect against MIAs and other privacy-related attacks is crucial. Existing defense methods for MIAs, such as those based on model distillation (Tang et al., ; Shejwalkar & Houmansadr, ; Mazzone et al., ), have proven effective in image classification models by reducing overfitting and limiting memorization of training data. However, these approaches cannot be directly applied to diffusion models due to their unique structure and the resource-intensive nature of distillation processes, especially in large diffusion models.

To address these challenges, we propose a tailored distillation method optimized for diffusion models, namely DistillMD (see Fig. 1), which is computationally efficient and effective in preventing MIAs. Compared to other distillation defenses, one key advantage of our method is that it does not require additional test data for the teacher to produce non-member labels. This limitation of other approaches hinders their applications in cases where we do not have much data to train and evaluate the models. To evaluate this benefit, we perform our defense in the model fine-tuning paradigm with a small dataset in Section 4.2.

Figure: Figure 1: Our proposed defense method DistillMD with model distillation. Two teacher models are trained on two disjoint subsets with standard diffusion models training (Left). A student model is then distilled from the two teachers using the distillation loss. Only one teacher model is used to produce the target noise given one data sample, and the teacher is chosen so that the input noisy image did not appear at its training subset in the previous phase (Right).
Refer to caption: extracted/5931066/figures/Training.png

While effectively alleviating any attack, the distillation method often requires a high computational cost to train a student model, hindering the method’s application to resource-constraint settings. For resource-constrained environments where the overhead of model distillation is impractical, we propose another dual-model defense (DualMD) method that does not require additional training other than the two teacher models but can still efficiently mitigate MIAs in black-box settings. The method is illustrated in Fig. 2.

Figure: Figure 2: The efficient defense method DualMD with modified inference pipeline. The two models, which are trained on disjointed subsets, are used to denoise images alternately.
Refer to caption: extracted/5931066/figures/Inference.png

Although the mentioned techniques can be effective for unconditional diffusion models, they can fail to protect conditional diffusion models due to the strong overfitting to the conditions. For example, Pang & Wang () designed their attack to exploit this property using text prompts to guide diffusion models to produce images in a distribution close to the target images.

Prompt overfitting has been extensively studied in diffusion model memorization. For example, Somepalli et al. () observed that prompt overfitting plays a crucial role in model memorization and proposed several techniques to reduce the effect. Wen et al. () further argued that some tokens can be more important than others to guide the generation. In Section 4.2, we show that DualMD and DistillMD alone cannot effectively defend against attacks utilizing the text guidance and propose a technique following Somepalli et al. () to diversify the training prompts. Although this simple approach is inspired by model memorization, it appears to be crucial to defend against MIAs, as presented in Section 4.2. This implies a strong connection between the two areas, and we provide a more in-depth discussion in Section 3.6.

We summarize our contributions as follows:

- •
We propose two mitigation strategies to defend against Membership Inference Attacks (MIAs): DualMD, targeting black-box attacks via an inference-only approach, and DistillMD, which defends against both white-box and black-box attacks through a distillation-based method. Our evaluation reveals that the distillation approach is more suitable to maintain high generation quality of unconditional diffusion models, while dual-model inference better preserves the quality of text-to-image diffusion models.
- •
We evaluate the effectiveness of our methods in training large text-to-image diffusion models and propose a technique to prevent the models from overfitting to the prompts. Our experiments demonstrate that we can significantly reduce the risk of personal data leakage in both white-box and black-box settings.
- •
We show that memorization mitigation techniques can be applied to defend MIAs and that defending against MIAs can mitigate model memorization. To the best of our knowledge, we are the first to establish this connection between these two areas.

## 2 Background and Related Work

##### Diffusion Models

Recent breakthroughs in diffusion models have demonstrated remarkable success across various generative tasks. As powerful generative models, diffusion models (Sohl-Dickstein et al., ; Ho et al., ) produce fascinating images by progressively denoising inputs. They first incorporate noise into data distributions through a forward process, then reverse this procedure to recover the original data. In particular, starting with an initial data original image $\mathbf{x}_{0}$ sampled from a (unknown) distribution $q(\mathbf{x}_{0})$, the forward process gradually diffuses $\mathbf{x}_{0}$ into a standard Gaussian noise $\mathbf{x}_{T}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$ through $T$ consecutive timesteps, where $\mathbf{I}$ is the identity matrix. Specifically, at timestep $t\in\{1,\dots,T\}$, the diffusion process $q(\mathbf{x}_{t}|\mathbf{x}_{t-1})$ and the denoising process $p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})$ are defined as follows:

$$ $\begin{split}q(\mathbf{x}_{t}|\mathbf{x}_{t-1})&=\mathcal{N}(\mathbf{x}_{t}; \sqrt{1-\beta_{t}}\mathbf{x}_{t-1},\beta_{t}\mathbf{I}),\\ p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})&=\mathcal{N}(\mathbf{x}_{t-1};\bm{ \mu}_{\theta}(\mathbf{x}_{t},t),\Sigma_{\theta}(\mathbf{x}_{t},t)),\end{split}$ (1) $$

where $\beta_{t}\in(0,1]$ is an increasing noise scheduling sequence. By denoting $\alpha_{t}=1-\beta_{t}$ and $\bar{\alpha}_{t}=\prod_{s=1}^{t}\alpha_{s}$, the diffused image $\mathbf{x}_{t}$ at timestep $t$ has a closed form as follows:

$$ $\mathbf{x}_{t}=\sqrt{\bar{\alpha}_{t}}\mathbf{x}_{0}+\sqrt{1-\bar{\alpha}_{t}} \bm{\epsilon}_{t},\;\text{where }\epsilon_{t}\sim\mathcal{N}(0,\mathbf{I}).$ (2) $$

During the training process, a noise-predictor $\bm{\epsilon}_{\theta}$ learns to estimate the noise $\bm{\epsilon}$ that was previously added to $\mathbf{x}_{0}$ by minimizing the denoising loss:

$$ $L(\theta)=\mathbb{E}_{\mathbf{x}_{0},\bm{\epsilon},t}\left[\|\bm{\epsilon}-\bm {\epsilon}_{\theta}(\mathbf{x}_{t},t)\|^{2}\right].$ (3) $$

After that, in the reverse diffusion process, a random Gaussian noise $\mathbf{x}_{T}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$ is iteratively denoised to reconstruct the original image $\mathbf{x}_{0}\in q(\mathbf{x}_{0})$. At each denoising step, using the output of the trained noise-predictor $\bm{\epsilon}_{\theta}$, the mean of the less noisy image $\mathbf{x}_{t-1}$ is computed as follows:

$$ $\bm{\mu}_{t}=\frac{1}{\sqrt{\alpha_{t}}}\left(\mathbf{x}_{t}-\frac{1-\alpha_{t }}{\sqrt{1-\bar{\alpha}_{t}}}\bm{\epsilon}_{\theta}(\mathbf{x}_{t},t)\right).$ (4) $$

##### Membership Inference Attacks

The membership inference attacks (MIAs), introduced by Shokri et al. (), aim to identify whether a specific data point was part of the model’s training set. Based on the threat models or the level of access attackers have, MIAs can be classified into white-box and black-box attacks. White-box MIAs usually utilize the internal parameters and gradients of the diffusion model to perform threshold-based attacks (Hu & Pang, ; Dubiński et al., ), gradient-based attacks (Pang et al., ) and proximal initialization (Kong et al., ). In contrast, black-box MIAs such as Pang & Wang () target the output generated by the diffusion models without direct access to internal parameters. Studies have demonstrated that these attacks can effectively differentiate between training and non-training samples by analyzing the generated image quality (Wu et al., ; Matsumoto et al., ; Carlini et al., ), and the estimation errors (Duan et al., ). Moreover, Li et al. () recently find that fine-tuning models on small datasets can augment their vulnerability to MIAs.

##### Membership Inference Defenses

Existing studies have demonstrated that overfitting in the threat models is a primary factor contributing to their vulnerability to MIAs. Consequently, various defenses have been proposed to counter MIAs, for example, by addressing overfitting, including techniques such as adversarial regularization (Hu et al., ), dropout (Salem et al., ), overconfidence reduction (Chen & Pattabiraman, ), and early stopping (Song & Mittal, ). Furthermore, differential privacy (DP) (Yeom et al., ; Abadi et al., ; Wu et al., ) has been widely used to mitigate MIAs by limiting the influence of any training data point on the model. However, DP methods often face trade-offs between privacy and utility. Additionally, knowledge distillation-based defenses such as distillation for membership privacy (Shejwalkar & Houmansadr, ) and complementary knowledge distillation (Zheng et al., ) aim to protect against MIAs by transferring knowledge from unprotected models. More recently, multiple techniques (Tang et al., ; Mazzone et al., ; Li et al., ) have been proposed to combine knowledge distillation with ensemble learning to preserve data privacy. Nevertheless, none of the methods are designed specifically for diffusion models which are usually large and constrained by resource limitation.

##### Diffusion Memorization and Mitigation

It is widely recognized that generative language models pose a risk of replicating content from their training data (Carlini et al., ; ). Similarly, Webster () observe the same behavior of large diffusion models, while Somepalli et al. () argue that diffusion models trained on smaller datasets tend to produce images that closely resemble those in the training set. As the size of the training dataset increases, the likelihood of such replication decreases. Several mitigation strategies have been explored to address these issues of diffusion models by either modifying the text conditioning (Somepalli et al., ; Wen et al., ; Ren et al., ), manipulating the guidance scale (Chen et al., ), or model pruning (; Chavhan et al., ).

## 3 Methodology

### 3.1 Membership Inference Attacks (MIAs) and Defenses

Given an image $\mathbf{x}$ and a pre-trained diffusion model $\bm{\epsilon}_{\theta}$ on the training dataset $D_{\text{train}}$. Denoting the test dataset by $D_{\text{test}}$, the goal of MIAs (Shokri et al., ) is to detect if this image belongs to $D_{\text{train}}$. By viewing this as a binary classification problem, we have the dataset $B=\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{m}$, where

$$ $y_{i}=\begin{cases}1,&\text{ if }\mathbf{x}_{i}\in D_{\text{train}}\\ 0,&\text{ if }\mathbf{x}_{i}\notin D_{\text{train}}.\end{cases}$ $$

The task of MIAs then becomes to learn an attack function $\bm{f}_{\bm{\epsilon}_{\theta}}$ for the model $\bm{\epsilon}_{\theta}$ to maximize the probability of $\bm{f}_{\bm{\epsilon}_{\theta}}(\mathbf{x}_{i})=y_{i}$, i.e.,

$$ $\max_{\bm{f}_{\bm{\epsilon}_{\theta}}}\mathds{P}\left(\bm{f}_{\bm{\epsilon}_{ \theta}}\left(\mathbf{x}_{i}\right)=y_{i}\right).$ $$

The design of $\bm{f}_{\bm{\epsilon}_{\theta}}$ depends on the specific choice of attacks and the attack settings. For example, in white-box attacks, the attacker can access all or parts of the training configuration and the model $\bm{\epsilon}_{\theta}$. In black-box attacks, the attacker can only access the images generated by the model. Regardless of the settings, MIAs are usually based on the assumption that the models overfit the training data. For example, consider diffusion models, in which the model $\bm{\epsilon}_{\theta}$ takes the input image $\mathbf{x}$, condition $\mathbf{c}$ ($\mathbf{c}=\emptyset$ in unconditional case), timestep $t\in\{1,\dots,T\}$ and a random noise $\bm{\epsilon}\sim N(\mathbf{0},\mathbf{I})$ to compute the denoising loss in Eq. 3, we have the following assumption:

$$ $\displaystyle\underset{\begin{array}[]{c}(\mathbf{x}_{\text{train}},\mathbf{c} _{\text{train}})\in D_{\text{train}}\end{array}}{\mathbb{E}}\Big{[}\left\|\bm{ \epsilon}_{\theta}\left(\mathbf{x}_{\text{train}},\mathbf{c}_{\text{train}},t \right)-\bm{\epsilon}\right\|\Big{]}\leq\underset{\begin{array}[]{c}(\mathbf{x }_{\text{test}},\mathbf{c}_{\text{test}})\in D_{\text{test}}\end{array}}{ \mathbb{E}}\Big{[}\left\|\bm{\epsilon}_{\theta}\left(\mathbf{x}_{\text{test}}, \mathbf{c}_{\text{test}},t\right)-\bm{\epsilon}\right\|\Big{]}.$ $$

The larger the gap between the two terms, the more accessible the attacker can extract the training data. Therefore, our defense aims to make this assumption less intense so that the attacker cannot separate member data from non-member data using this property. To this aim, we design a new training paradigm to minimize the gap between train and test data, which is equivalent to the following optimization problem:

$$ $\min_{\bm{\epsilon}_{\theta}}\underset{\begin{array}[]{c}(\mathbf{x}_{\text{ train}},\mathbf{c}_{\text{train}})\in D_{\text{train}}\\ (\mathbf{x}_{\text{test}},\mathbf{c}_{\text{test}})\in D_{\text{test}}\end{ array}}{\mathbb{E}}\Big{[}\left\|\bm{\epsilon}_{\theta}\left(\mathbf{x}_{\text {train}},\mathbf{c}_{\text{train}},t\right)-\bm{\epsilon}\right\|-\left\|\bm{ \epsilon}_{\theta}\left(\mathbf{x}_{\text{test}},\mathbf{c}_{\text{test}},t \right)-\bm{\epsilon}\right\|\Big{]}.$ (5) $$

The key idea is to modify the training loss so that our models do not fit directly into the training set. For this aim, we train two teacher models on two disjoint datasets and then let them produce “soft targets” from the other dataset to train a student model. Since these targets are the outputs of teacher models to their “non-member” images, the outputs of the student model to these data will be close to their outputs to the test data. More details are presented in the following Sections 3.2 and 3.3.

### 3.2 Disjoint Training

Although ensemble learning has been used to defend against MIAs in image classification models (Tang et al., ; Li et al., ), they are not applicable to diffusion models, and the growing number of models poses a significant challenge when applying to large architectures. Therefore, we propose an efficient method with only two models on two disjoint subsets of the training data.

Formally, given a training dataset $D_{\text{train}}$, a test dataset $D_{\text{test}}$, and an original learning model parameterized by $\theta$. Our first step is to subdivide the training dataset into two disjoint subsets, and each is used to train a separate model, i.e., $D_{\text{train}}=D_{1}\cup D_{2}$, where $D_{1}\cap D_{2}=\emptyset$. The two trained models parameterized by $\theta_{1}$ and $\theta_{2}$, respectively, can be used to generate images directly while keeping the privacy of both training subsets thanks to our customized inference pipeline proposed in Section 3.4. Alternatively, they can be distilled into a new private student model.
Our basic assumption is that the two models “see” the training data of the other as test data, i.e.,

$$ $\begin{split}\underset{\begin{array}[]{c}(\mathbf{x}_{2},\mathbf{c}_{2})\in D_ {2}\\ t\in\{1,\dots,T\}\end{array}}{\mathbb{E}}\Big{[}\left\|\bm{\epsilon}_{\theta_{ 1}}\left(\mathbf{x}_{2},\mathbf{c}_{2},t\right)-\bm{\epsilon}\right\|\Big{]}&= \underset{\begin{array}[]{c}(\mathbf{x}_{\text{test}},\mathbf{c}_{\text{test}} )\in D_{\text{test}}\\ t\in\{1,\dots,T\}\end{array}}{\mathbb{E}}\Big{[}\left\|\bm{\epsilon}_{\theta_{ 1}}\left(\mathbf{x}_{\text{test}},\mathbf{c}_{\text{test}},t\right)-\bm{ \epsilon}\right\|\Big{]}.\\ \underset{\begin{array}[]{c}(\mathbf{x}_{1},\mathbf{c}_{1})\in D_{1}\\ t\in\{1,\dots,T\}\end{array}}{\mathbb{E}}\Big{[}\left\|\bm{\epsilon}_{\theta_{ 2}}\left(\mathbf{x}_{1},\mathbf{c}_{1},t\right)-\bm{\epsilon}\right\|\Big{]}&= \underset{\begin{array}[]{c}(\mathbf{x}_{\text{test}},\mathbf{c}_{\text{test}} )\in D_{\text{test}}\\ t\in\{1,\dots,T\}\end{array}}{\mathbb{E}}\Big{[}\left\|\bm{\epsilon}_{\theta_{ 2}}\left(\mathbf{x}_{\text{test}},\mathbf{c}_{\text{test}},t\right)-\bm{ \epsilon}\right\|\Big{]}.\end{split}$ (6) $$

The two models are trained with the typical denoising loss as in Eq. 3. The details of that disjoint training mechanism is presented in Algorithm 1.

Figure: Algorithm 1 Disjoint Training with DDPM

### 3.3 Alternating Distillation (DistillMD)

##### Choosing teacher models

Based on the assumption given in Eq. 6, we alternately use the two teacher models to generate targets for the student model to learn from. Specifically, the first model $\theta_{1}$, which is trained on the first subset $D_{1}$, will infer on the second subset $D_{2}$, while the second model $\theta_{2}$ trained on $D_{2}$ will infer on the first subset $D_{1}$. Fig. 1 illustrates the training pipeline, and the algorithm is described in Algorithm 2.

##### Distillation loss

To prevent the student model from overfitting to training data, the real noise term in Equation 3 is replaced by outputs of the teacher models as in Equation 7.

$$ $L(\theta)=\mathbb{E}_{\mathbf{x}_{0},t}\left[\|\text{stopgrad}(\bm{\epsilon}_{ \text{teacher}}(\mathbf{x}_{t},t))-\bm{\epsilon}_{\theta}(\mathbf{x}_{t},t)\|^ {2}\right].$ (7) $$

By minimizing the loss in Eq. 7 with suitable choices of the teacher models, we can make the outputs of the student model on train data closer to its outputs on test data. This closes the gap in Eq. 5 thanks to the assumption provided in Eq. 6.

In practice, our defense method can effectively mitigate both white-box and black-box attacks while maximally preserving the generation capability of the model, as shown in Section 4.

Figure: Algorithm 2 Alternating Distillation (DistillMD)

### 3.4 Self-correcting Inference Pipeline (DualMD)

##### Motivation

Black-box MIAs typically rely on training shadow models or assessing the distance between the target image and generated samples, exploiting the model’s tendency to generate images close to its training data due to overfitting. However, our training paradigm ensures that for any given sample, there always exists a model that treats it as a test sample, enabling uniformly diverse sample generation.

Diffusion models uniquely require an iterative inference process that run the model multiple times. We leverage this characteristic by using our two teacher models to “correct” each other during inference. For instance, if the noisy image at time step $t$ causes model 1 to produce output close to the target image, model 2 will generate a more uniformly distributed image at time step $t-1$. This “self-correcting” inference process ensures diverse generation instead of concentration near training samples. Our experimental results in Section 4 demonstrate that this method efficiently mitigates black-box MIAs on text-to-image diffusion models.

### 3.5 Enhancing Privacy for Conditional Diffusion Models

Although disjoint training divides the data into disjoint subsets, text prompts in different subsets can still have overlapping words or textual styles that can be overfitted by both models. Therefore, we propose to enhance prompt diversity during training by using an image conditioning model to generate multiple prompts for each image of the training dataset. Then, a prompt is randomly sampled for each image in each epoch during training. More details about the limitations of DualMD and DistillMD on text-to-image diffusion models and the significance of prompt diversification are presented in Section 4.2.

### 3.6 Membership Inference Defenses Help Mitigating Data Memorization

Recently, an increasing body of research (Somepalli et al., ; ) has highlighted the issue of data memorization in modern diffusion models, where some generated images are near-identical reproductions of images from the training datasets. Previous studies (Shokri et al., ; Yeom et al., ) have shown that overfitting renders models vulnerable to MIAs. Given that data memorization is often considered a more extreme form of overfitting, this raises an important question: Is there a connection between MIAs and data memorization?

Our findings suggest that loss-based MIA techniques can effectively detect memorization in diffusion models. Specifically, we use the t-error (Eq. 8) introduced by Duan et al. () to detect whether the model memorizes a prompt. This detection is applied to a set of 500 memorized prompts and 500 non-memorized prompts (Wen et al., ). The resulting detection performance is reported in Fig. 3, where it is evident that the loss function effectively distinguishes between memorized and non-memorized prompts. Given this observed link between data memorization and MIAs, we are led to explore a further question: Can membership inference defenses help mitigate data memorization?

To investigate this, we conduct experiments on Stable Diffusion v1.5 (Rombach et al., ) as detailed in Section 4.3. More information about the t-error and the memorization experiments are presented in Appendix A.1.

Figure: Figure 3: Distribution of the values of the loss in Eq. 8 between 500 memorized prompts and 500 non-memorized prompts on Stable Diffusion v1.5 (Rombach et al., ).
Refer to caption: extracted/5931066/figures/mem_loss.png

## 4 Experiments

We present the effectiveness of our defenses against white-box MIAs in Section 4.1 and against black-box MIAs in Section 4.2. We also analyze the importance of prompt diversification in both cases and find that this technique slightly improves defense in white-box case and significantly enhances defense in black-box case.

##### Datasets

We utilize various datasets to verify the effectiveness of the methods. The unconditional experiments use CIFAR10 (Krizhevsky et al., ), CIFAR100 (Krizhevsky et al., ), Tiny-ImageNet (), and STL10-Unlabeled (Coates et al., ) datasets. For the text-to-image experiments, we employ the popular Pokemon dataset (^1^11[https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions)). Each dataset is divided equally, with one half used for training the models (member set) and the other half serving as the non-member set.

In Sections 4.2 and 4.3, we fine-tune the Stable Diffusion v1.5 (SDv1.5) (^2^22[https://huggingface.co/stable-diffusion-v1-5](https://huggingface.co/stable-diffusion-v1-5)) (Rombach et al., ) on Pokemon dataset so that it overfits to the dataset. We train the default DDPM (Ho et al., ) from scratch for other datasets. More training details are given in Section A.2.

##### Metrics

Following Kong et al. (), we utilize Area Under the ROC Curve (AUC) and True Positive Rate when the False Positive Rate is $1\%$ (TPR@1%FPR) as the key metrics to measure the vulnerability of the models to MIAs. Since we are defending MIAs, an AUC closer to 0.5 indicates better performance. For quality measurements, the popular Frenchet Inception Distance (FID) and Inception Score (IS) are measured. For unconditional models, FID and IS are computed on 25,000 generated images. In contrast, for text-to-image models, these metrics are calculated on images generated from training prompts.

Table 1 presents the quantitative performance of our methods in terms of quality preservation compared to the baseline model. It can be seen that DistillMD shows superior quality preservation in unconditional models, whereas DualMD performs better for conditional models. Additionally, we later observe a similar trend in defending against MIAs, which indicates that DualMD is more effective for text-to-image diffusion models, while DistillMD is better suited for unconditional diffusion models.

**Table 1: Quantitative evaluation of the quality of the defended models compared to the original model. Unconditional diffusion model is evaluated on CIFAR10 with DDPM, and text-to-image diffusion model is evaluated on Pokemon dataset with SDv1.5. Bold and underlined numbers are the best and the second best, respectively.**
|  | CIFAR10 | Pokemon |  |  |
| --- | --- | --- | --- | --- |
| Method | FID ($\downarrow$) | IS ($\uparrow$) | FID ($\downarrow$) | IS ($\uparrow$) |
| Original model | 14.127 | 8.586 | 0.22 | 3.02 |
| DualMD | 21.389 | 8.011 | 0.26 | 3.34 |
| DistillMD | 14.192 | 8.391 | 0.44 | 3.52 |

### 4.1 White-box Attacks

For white-box MIAs, we perform two attacks SecMIA (Duan et al., ) and PIA (Kong et al., ) and defend against them with DistillMD. Since the attackers are assumed to have white-box access to the model, it is not realistic to perform DualMD defense. The results for unconditional diffusion models are given in Table 2, and for text-to-image diffusion models in Table 3. Although PIA cannot attack fine-tuned Stable Diffusion model, it is still clear that DistillMD significantly increases the privacy of both unconditional diffusion models and text-to-image diffusion models.

In the case of prompt diversification training, the BLIP model (Li et al., ) is used to generate five prompts for each image. During training, one prompt is randomly drawn from the six (including the original) to serve as the text condition for the image. Our results show that this technique slightly improves the defense against white-box attacks.

**Table 2: Effectiveness of our DistillMD against white-box MIAs on DDPM. The closer AUC to 0.5, the better. Bold numbers are the best.**
|  |  | CIFAR10 | CIFAR100 | Tiny-ImageNet | STL10-Unlabeled |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Attack | Method | AUC | TPR@1% FPR ($\downarrow$) | AUC | TPR@1% FPR ($\downarrow$) | AUC | TPR@1% FPR ($\downarrow$) | AUC | TPR@1% FPR ($\downarrow$) |
| SecMIA | No defense | 0.93 | 0.35 | 0.96 | 0.45 | 0.96 | 0.53 | 0.94 | 0.30 |
|  | DistillMD | 0.59 | 0.03 | 0.61 | 0.02 | 0.57 | 0.02 | 0.58 | 0.02 |
| PIA | No defense | 0.89 | 0.13 | 0.88 | 0.14 | 0.84 | 0.08 | 0.83 | 0.09 |
|  | DistillMD | 0.59 | 0.02 | 0.59 | 0.03 | 0.56 | 0.02 | 0.58 | 0.02 |

**Table 3: Effectiveness of our DistillMD against white-box MIAs on SDv1.5. The closer AUC to 0.5, the better. Bold numbers are the best.**
|  |  | w/o prompt diversification | w/ prompt diversification |  |  |
| --- | --- | --- | --- | --- | --- |
| Attack | Method | AUC | TPR@1%FPR ($\downarrow$) | AUC | TPR@1%FPR ($\downarrow$) |
| SecMIA | No defense | 0.99 | 0.79 | 0.99 | 1.00 |
|  | DistillMD | 0.48 | 0.02 | 0.44 | 0.01 |
| PIA | No defense | 0.46 | 0.02 | 0.61 | 0.03 |
|  | DistillMD | 0.49 | 0.01 | 0.50 | 0.02 |

### 4.2 Black-box Attacks

For black-box MIAs, we employ the recently proposed attack in Pang & Wang (), which utilizes text guidance to augment the attack. The SDv1.5 model is fine-tuned on the Pokemon dataset with and without our methods. The results in Table 4 show that training defenses alone cannot completely defend against MIAs. Unlike white-box attacks, both DistillMD and DualMD can only mitigate MIAs with the help of prompt diversification, which implies the importance of prompt overfitting. Moreover, DualMD can not only better preserve the generation quality but also better defend in the case of text-to-image diffusion models.

**Table 4: Effectiveness of our defenses against black-box MIA on SDv1.5. The closer AUC to 0.5, the better. Bold and underlined numbers are the best and the second best, respectively.**
|  | w/o prompt diversification | w/ prompt diversification |  |  |
| --- | --- | --- | --- | --- |
| Method | AUC | TPR@1%FPR ($\downarrow$) | AUC | TPR@1%FPR ($\downarrow$) |
| No defense | 0.90 | 0.57 | 0.45 | 0.009 |
| DualMD | 0.82 | 0.35 | 0.52 | 0.014 |
| DistillMD | 0.66 | 0.09 | 0.46 | 0.005 |

### 4.3 Membership inference defenses mitigate data memorization

**Table 5: Mitigation results of our methods. Bold and underlined numbers are the best and the second best, respectively.**
| Method | SSCD ($\downarrow$) | CLIP Scores ($\uparrow$) |
| --- | --- | --- |
| No mitigation | 0.60 | 0.26 |
| Wen et al. () | 0.28 | 0.25 |
| DualMD | 0.52 | 0.27 |
| DistillMD | 0.27 | 0.28 |

We use the fine-tuned SDv1.5 model using our methods to evaluate its capability of data memorization. For comparison, we employ the inference-time memorization mitigation method proposed by Wen et al. (), which reduces memorization by adjusting the prompt embedding to minimize the difference between unconditional and text-conditional noise predictions.

To measure the level of memorization, we calculate the SSCD similarity score (Pizzi et al., ; Somepalli et al., ) between the generated images and the images in the training dataset, given the same set of prompts. In addition, the CLIP score (Radford et al., ) is used to assess the alignment between the generated images and their corresponding prompts. A lower SSCD similarity score indicates reduced memorization, while a higher CLIP score reflects better alignment between the generated image and the prompt.

##### Results and Discussion

Table 5 shows the effectiveness of our proposed method in mitigating data memorization. A thoroughly fine-tuned model without any mitigation produces highly similar images with an SSCD similarity score of 0.60 for given prompts, indicating significant memorization. In contrast, our DualMD and DistillMD approaches significantly reduce the SSCD score to 0.52 and 0.27, respectively, suggesting that membership inference defenses can help mitigate data memorization. Notably, both methods also show a slight improvement in CLIP scores. Furthermore, the method proposed by Wen et al. (), which directly targets mitigating memorization, achieves an SSCD similarity score of 0.28. Our DistillMD approach, despite being designed to defend against MIAs, not only reduces data memorization more effectively but also improves image-text alignment compared to the most recently proposed method in Wen et al. ().

## 5 Conclusion

This paper presents comprehensive and novel approaches to protect diffusion models against training data leakage while mitigating model memorization. Our methodology focuses on training two models using disjoint subsets of the training data. This results in two significant contributions, including DualMD for private inference and DistillMD for developing a privacy-enhanced student model. Both techniques effectively reduce model overfitting to training samples. We further enhance privacy protection for text-conditioned diffusion models by diversifying training prompts, preventing models from overfitting specific textual patterns. Notably, our experiments reveal that model memorization represents a more severe form of overfitting than membership inference attacks (MIAs), and our unified approach successfully addresses both vulnerabilities simultaneously, eliminating the need for separate mitigation strategies. In short, our paper presents inference-time and training-time strategies to defend diffusion models against MIAs. It provides new insights into the intersection between MIAs and model memorization, advancing our understanding of privacy preservation in generative models.

##### Limitations and Future Directions

Our methods rely on dividing the training dataset into two halves, which may limit the generative capabilities of the teacher models in data-scarce scenarios. This limitation can affect the quality of the distilled model, as evidenced by the slight performance degradation shown in Table 1. Future research could focus on developing methods that allow models to leverage the entire dataset during training while maintaining strong privacy guarantees, potentially enhancing the performance of all models.

Furthermore, our inference-time defense method (DualMD) requires storing and alternating between two models, which may limit its applicability in resource-constrained environments. Future work could explore inference-time solutions that, like our DistillMD method, do not necessitate additional model storage while maintaining robust privacy protection.

## Appendix A Appendix

### A.1 SecMI Loss

Diffusion models optimize the variational bound $p_{\theta}(\mathbf{x}_{0})$ by matching the forward process posteriors at each step $t$. The local estimation error for a data point $\mathbf{x}_{0}$ at time $t$ is then expressed as:

$$ $\displaystyle\ell_{t,\mathbf{x}_{0}}=||\hat{\mathbf{x}}_{t-1}-\mathbf{x}_{t-1} ||^{2},$ $$

where $\mathbf{x}_{t-1}\sim q(\mathbf{x}_{t-1}|\mathbf{x}_{t},x_{0})$ and $\hat{\mathbf{x}}_{t-1}\sim p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})$. Due to the non-deterministic nature of the diffusion and denoising processes, calculating this directly is intractable. Instead, deterministic processes are used to approximate these errors:

$$ $\displaystyle\mathbf{x}_{t+1}=\phi_{\theta}(\mathbf{x}_{t},t)=\sqrt{\bar{ \alpha}_{t+1}}f_{\theta}\left(\mathbf{x}_{t},t\right)+\sqrt{1-\bar{\alpha}_{t+ 1}}\epsilon_{\theta}\left(\mathbf{x}_{t},t\right),$ $\displaystyle\mathbf{x}_{t-1}=\psi_{\theta}(\mathbf{x}_{t},t)=\sqrt{\bar{ \alpha}_{t-1}}f_{\theta}\left(\mathbf{x}_{t},t\right)+\sqrt{1-\bar{\alpha}_{t- 1}}\epsilon_{\theta}\left(\mathbf{x}_{t},t\right),$ $$

where $f_{\theta}\left(\mathbf{x}_{t},t\right)=\frac{\mathbf{x}_{t}-\sqrt{1-\bar{
\alpha}_{t}}\epsilon_{\theta}\left(\mathbf{x}_{t},t\right)}{\sqrt{\bar{\alpha}
_{t}}}$. Define $\Phi_{\theta}\left(\mathbf{x}_{s},t\right)$ as the deterministic reverse and $\Psi_{\theta}\left(\mathbf{x}_{t},s\right)$ as the deterministic denoise process:

$$ $\displaystyle\mathbf{x}_{t}=\Phi_{\theta}\left(\mathbf{x}_{s},t\right)=\phi_{ \theta}\left(\cdots\phi_{\theta}\left(\phi_{\theta}\left(\mathbf{x}_{s},s \right),s+1\right),t-1\right)$ $\displaystyle\mathbf{x}_{s}=\Psi_{\theta}\left(\mathbf{x}_{t},s\right)=\psi_{ \theta}\left(\cdots\psi_{\theta}\left(\psi_{\theta}\left(\mathbf{x}_{t},t \right),t-1\right),s+1\right)$ $$

Duan et al. () define SecMI loss or t-error as the approximated posterior estimation error at step $t$:

$$ $\displaystyle\tilde{\ell}_{t,\mathbf{x}_{0}}=||\psi_{\theta}(\phi_{\theta}( \tilde{\mathbf{x}}_{t},t),t)-\tilde{\mathbf{x}}_{t}||^{2},$ (8) $$

given sample $\mathbf{x}_{0}\in D$ and the deterministic reverse result $\tilde{\mathbf{x}}_{t}=\Phi_{\theta}\left(\mathbf{x}_{0},t\right)$ at timestep $t$.

This SecMI loss helps identify memberships as member samples tend to have lower t-errors compared to hold-out samples. We leverage this t-error to separate memorized and non-memorized prompts. The experiment is performed similar to Wen et al. () in which we plot the distribution of the loss values of the member set and the hold-out set. We utilize 500 memorized prompts of Stable Diffusion v1 extracted by Webster () for the member set, and 500 non-memorized prompts that are randomly sampled from the Lexica.art prompt set (^3^33[https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts](https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts)) for the hold-out set. The result is illustrated in Fig. 3.

### A.2 Training Details

#### A.2.1 Dataset

Table 6 provides a summary of the diffusion models used, the datasets, and the details of the data splits.

**Table 6: Adopted diffusion models and datasets.**
| Model | Dataset | Resolution | # Train | # Test | Condition |
| --- | --- | --- | --- | --- | --- |
| DDPM | CIFAR10 | 32 | 25,000 | 25,000 | - |
| CIFAR100 | 32 | 25,000 | 25,000 | - |  |
| STL10-Unlabeled | 32 | 50,000 | 50,000 | - |  |
|  | Tiny-ImageNet | 32 | 50,000 | 50,000 | - |
| SDv1.5 | Pokemon | 512 | 416 | 417 | text |

#### A.2.2 Training and Attack Hyperparameters

According to Matsumoto et al. (), the vulnerability of the models to MIAs increases with the number of training steps because overfitting makes the models more susceptible to attacks. Therefore, in order to ensure a fair comparison, we train both the baseline model, the two models trained on two disjoint subsets, and the distilled model with a same number of training steps.

For unconditional diffusion models, we train all the models for 780,000 iterations with a batch size of 128, a learning rate of 2e-4.

For SDv1.5, we use the Huggingface Diffusers codebase (^4^44[https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README.md](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README.md)) to fine-tune the model in 20,000 iterations, with batch size of 16 and learning rate of 1e-5.

For white-box attacks on all models, we use the codebase and default settings of SecMIA (^5^55[https://github.com/jinhaoduan/SecMI](https://github.com/jinhaoduan/SecMI)) (Duan et al., ) and PIA (^6^66[https://github.com/kong13661/PIA](https://github.com/kong13661/PIA)) (Kong et al., )

For black-box attacks on SDv1.5, we generate 3 images for each prompt, each is generated using DDIM (Song et al., ) with 50 inference steps.

For evaluating data memorization in Section 4.3, we use the codebase from (Wen et al., ) (^7^77[https://github.com/YuxinWenRick/diffusion_memorization](https://github.com/YuxinWenRick/diffusion_memorization))

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: - The paper in [2] uses a very similar idea for realizing differentially private diffusion models. 


- The utility of your models need to be evaluated more extensively. First, you need to provide some real generation so that reviewers can inspect the visual effect of your method. You also need more quantitive evaluation of your technique. It seems you are improving the metrics over non-private models? Does it mean you can claim SOTA for generating with diffusion models? 

- I think there is a need for exploring adaptive attacks. For example, is there something the attacker can do by calling the decoding with different number of steps? If you believe the existing attacks are optimal against your defense, you need to bring a convincing argument.
soundness: 3
strengths: - Membership inference and memorization is a key challenge with diffusion models. Mitigating such vulnerabilities is hence an important topic.

- The idea of using an ensemble of model is clever.
summary: This paper presents a comprehensive study on a defense mechanism against membership inference attacks specifically targeting diffusion models. The authors employ an ensemble of two diffusion models, each trained on two disjoint datasets, to develop their defense strategy. Subsequently, they propose two distinct defenses utilizing these two models.
The first defense, referred to as DualMD, involves alternating between the two models during the diffusion process. By doing so, half of the steps are executed without any influence from half of the training data. This approach is expected to significantly reduce the overall impact of the training data, thereby making membership inference more challenging, particularly for black-box attacks.
The second defense, denoted as DistilMD, involves distilling the resulting model from DualMD into a new model. This process aims to eliminate even white-box attacks by further reducing the vulnerability of the model to membership inference attacks.
The authors provide empirical evidence demonstrating the effectiveness of their proposed defense mechanism in reducing the performance of existing white-box and black box attacks.
weaknesses: - Although the idea is clever the same idea is explored in previous work as a way to defend against MIA both in classification [1] and generative models (diffusion models in particular!) [2]

- The paper is not evaluated against any adaptive attacks. Is there an attack that can be optimized for DualMD and DistillMD?

- The evaluation of utility is limited.

[1] https://www.usenix.org/system/files/sec22-tang.pdf

[2] https://openreview.net/forum?id=vuVGcl0ed1#all

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: See above.
soundness: 2
strengths: 1. The idea is intuitive and easy to follow, training two diffusion models on disjoint dataset and designing models on how to design defense against MIAs based on these two models.
2. The evaluation considers both white-box attacks and black-box attacks on both unconditional diffusion models and text-to-image diffusion models.
summary: This paper studies the membership inference defense for diffusion models by training two diffusion models on disjoint data. This work introduce two methods, DistillMD and DualMD. After training model A on dataset A and model B on dataset B, DistillMD uses model A to generate image for dataset B and model B to generate image for dataset A. DistillMD then use generated images to train a student model. DualMD use the model A and B to perform denoising together in denoising the images. At each step, DualMD use one model to perform denoising at that step, then at next step, DualMD use the other model to perform the denoising.
Experimental evaluation include unconditional diffusion model and text-to-image diffusion model, including black-box attacks and white-box attacks.
weaknesses: I have concerns regarding the privacy and utility evaluation in this work.

1. For the privacy evaluation, it seems to me that similar to Tang et al. 2022, the disjoint dataset training in this work still face some issues for the correlated pairs, Tang et al. 2022 provides a case study how severe the correlated pairs issue is in CIFAR10 dataset. It would be helpful to investigate and discuss the correlated pairs and potential mitigations. It is also unclear to me whether DualMD is better or DistillMD is better for text-to-image diffusion models, as the AUC and TPR-FPR show different trends and Carlini et al. 2022[1] emphasize the importance of worst-case analysis for privacy leakage.

2. For the utility evaluation, it would be helpful to provide the FID and IS metric for images generated from model A and model B with the original data to understand utility trend among DualMD and DistillMD. I wonder if there is a explanation for why DualMD is better than DistillMD for text-to-image diffusion models, or utility gap is small, or the conclusion is dawn on a single dataset Pokemon that may not be true in general. Also it is unclear to me if results for Pokeman in Table 1 is with prompt diversification or without. 

3. The writings could be improved. In introductions, the contribution 2 states the "proposing a technique to prevent the models from overfitting to prompts", it seems to me this method is prompt diversification and is from Somepalli et al. 2023b, maybe the author could clarify what is different from prompt diversification and Sompalli et al. 2023b.  The connection between MIA and memorization are also studied in prior works. For example, in the extraction attack for diffusion models by Carlini et al. 2023, MIA is applied to select those images that are likely to be memorized. Minor question, what is "stopgrad" in Equation (7)?


[1] Carlini et al., Membership Inference Attacks From First Principles. IEEE S&P 2022

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 1
questions: * How is the proposed method compared to distillation-based baselines?
soundness: 2
strengths: * The proposed method is simple yet effective in defending MIA attacks.
* In the tested datasets and models, the proposed method is effective in defending MIA attacks.
*
summary: The paper proposed an ensemble-like method to defend membership inference attacks (MIA) in diffusion models (DM). Instead of training a single DM on the whole dataset, the proposed method train two models on disjoint subsets of the training set. At inference time, the denoising process will be alternately done by two models. The method is tested on the SoTA MIA attacks.
weaknesses: 1. The experiments lack essential baselines and ablation studies. Though most defense methods mentioned in the Related Work were not designed for diffusion models, the distillation-based method should be generally applicable. Note that the proposed method is also a distillation-based method. Thus, the distillation of a single diffusion model (following prior arts noted in the Related Work) should be included as a baseline. More detailed ablation studies should be conducted. For example, if the two models are necessary. 
2. The claim in the introduction, "We evaluate the effectiveness of our methods in training large text-to-image diffusion models and propose a technique to prevent the models from overfitting to the prompts.", is misleading. In the experiment (Table 4), without using their method, the no-defense baseline can effectively reduce the risk of MIA. I don't think the proposed method contribute to the prompt overfitting.
3. The logic of the paper is confusing. 
* It is not clear why the prompt overfitting is considered. In Line 112-113, the prompt overfitting is raised for model memorization, whose connection to the MIA is not clarified in the context.
* Throughout the paper, there are many forward references, making the logic very hard to follow.
* Lacks intuitive explanation on why this method works.
4. In MIA evaluation, the ROC curve should be presented to show the rigor of evaluation in MIA.
5. Only one diffusion model (SDv1.5) is used, which makes the conclusion less generalizable.

---

confidence: 2
contribution: 3
initial_rating: 5
presentation: 2
questions: Could you clarify why you randomly sample t instead of doing it sequentially? in the Algorithm 1 and 2?
soundness: 3
strengths: DualMD and DistillMD effectively adapt ensemble learning and distillation to protect diffusion models from MIA attacks
The paper includes evaluations across various datasets, models, and attack types (both white-box and black-box). This thorough testing underscores the robustness of the proposed methods.
summary: The paper introduces two  defense strategies—DualMD and DistillMD—to protect diffusion models against Membership Inference Attacks (MIAs) without significantly compromising model utility. The two defense strategies introduced operate differently: DualMD mitigates MIA risks in black-box settings by dividing data between two models and alternating inference to reduce data exposure. DistillMD combines dual models into a student model via distillation, offering robust defenses against both white-box and black-box attacks.
weaknesses: Lack of Comparative Analysis with Existing Methods: The study does not explicitly compare its defense methods with existing privacy defense methods beyond describing MIA vulnerabilities. A side-by-side comparison with traditional distillation or differential privacy approaches in image synthesis could underscore the novelty and effectiveness of the proposed methods.

Limited Discussion of Practical Constraints: The methods require dual models (DualMD) or distillation (DistillMD), both of which can be computationally intensive. Discussing potential resource constraints, such as memory limitations or model scalability in real-world applications, would make the methods' practicality more transparent.

Lack of Explanation on Evaluation Metrics: While metrics like FID, IS, and AUC are widely used, a brief explanation of their relevance to privacy defense and utility preservation would clarify the evaluation approach for readers.

Improve Clarity on Methods and Results: I find the caption for Figure 1 a bit confusing. I find the presentation a bit confusing. For example, I dont remember the DDPM being introduced but somehow where is an algorithm 1 for the DDPM while there is no algorithm for the DualMD.

Meta-review: This paper studies the membership inference defense for diffusion models by training two diffusion models on disjoint data. This work introduce two methods, DistillMD and DualMD. After training model A on dataset A and model B on dataset B, DistillMD uses model A to generate image for dataset B and model B to generate image for dataset A. DistillMD then use generated images to train a student model. DualMD use the model A and B to perform denoising together in denoising the images. At each step, DualMD use one model to perform denoising at that step, then at next step, DualMD use the other model to perform the denoising. Experimental evaluation include unconditional diffusion model and text-to-image diffusion model, including black-box attacks and white-box attacks.

This paper is on the borderline. Although their experiments got strengthened during the rebuttal period, I agree with the two reviewers in that the paper lacks methodological novelty.

Decision: Reject