Data Exfiltration in Diffusion Models: A Backdoor Attack Approach

Authors: Cheng Yu Yeo, Yi-Lun Wu, Hong-Han Shuai
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Backdoor Attack, Data Exfiltration, Diffusion Model

Abstract:
As diffusion models (DMs) become increasingly susceptible to adversarial attacks, this paper investigates a novel method of data exfiltration through strategically implanted backdoors. Unlike conventional techniques that directly alter data, we pioneer the use of unique trigger embeddings for each image to enable covert data retrieval. Furthermore, we extend our exploration to text-to-image diffusion models such as Stable Diffusion by introducing the Caption Backdoor Subnet (CBS), which exploits these models for both image and caption extraction. This innovative approach not only reveals an unexplored facet of diffusion model security but also contributes valuable insights toward enhancing the resilience of generative models against sophisticated threats.

Decision: reject
Reviewer scores: 8, 3, 6, 6

Reviews and discussion:
<thread>
creation date:
1730263857208

thread_type:
official_review

<thread_object>
creator: reviewer_srex

<content>
summary:
This paper delves into a novel approach for data exfiltration by strategically implanting backdoors into diffusion models. Data exfiltration through backdoor attacks is a formidable task as it demands that the model memorize the entire dataset without compromising the normal diversity of images. However, memorization and diversification often conflict with each other. To address this issue, the paper presents unique trigger embeddings for image exfiltration. Additionally, to overcome the challenge of extracting corresponding captions, a Caption Backdoor Subnet (CBS) module is trained. This paper resolves an important security-related question through a reasonable method.

strengths:
1. The problem addressed is extremely crucial and the application scenario is rather novel.
2. The design of the method and the model structure are relatively reasonable. 
3. The writing and organization of this paper is very clean and well-followed.

weaknesses:
1. **Too strong attacker’s capabilities assumption**. The attacker is depicted as being extremely capable of attacking, having access to all training data, mastering the training and inference processes of the model, and obtaining the trained backdoored model. This raises doubts about whether such an attack scenario is practical in the real world.

2. **Lack of theoretic analysis**. How does $\mathbb{L}^{Trig}(\theta)$ and $\\mathcal{L}^C (\theta)$ impact the convergence of training? Why can we select the parameters from the U-Net layers as the parameters of CBS?

3. **More experiments should be conducted.** For example, 1) How do different TGFs impact the ASR of the backdoored model and the process of the training. 2) What’s the effectiveness of the model when applied to more complex data scenarios, such as the COCO datasets, rather than CIFAR10 and AFHQv2 datasets which are relatively simple with relatively less image details. 3) Since models containing sensitive data are generally subject to rigorous censorship, it is suggested that the authors add experiments to verify the effect of the model under existed trigger detection methods.

questions:
see the weekness
</content>
</thread_object>

</thread>
<thread>
creation date:
1730656794322

thread_type:
official_review

<thread_object>
creator: reviewer_fwzp

<content>
summary:
The paper proposed a novel idea to exploit the backdoor attack to review the secret of the training data. For the DDPM model, by exploiting the special time embedding as the trigger, the attacker who injected the backdoor can recover the training data that he wants to leak. For the T2I models, the author uses special text embeddings instead of time embeddings as the trigger to help retrieve the training data. A Caption Backdoor Subnet, whose weights are retrieved from the U-Net of the victim T2I model was proposed to help recover the caption of the training samples.

strengths:
1. Good writing, easy to follow;

2. Novel idea and method, I have learned from, and been greatly inspired by the paper;

3. Extensive experiments and evaluations, demonstrating the proposed methods are really effective.

weaknesses:
I love the idea of exploiting the backdoor attack to cause secret leakage, yet mainly concerned with the threat model of the paper before I could convince myself to vote for the acceptance of the paper.

Firstly, the proposed method requires the attacker to have full access to the victim model in both the backdoor injection phase and the triggering phase, for the special embeddings need to be added to the model. This could greatly impact the applicability of this method-- seems that only open-sourced model are vulnerable to the proposed attack.

Secondly, the motivation for the attack is unclear to me. I doubt whether an insider, as mentioned in Sec.3, would carry out such an attack when he seems capable of releasing the training data directly afterward.  The insider can also just copy the data secretly before they leave the organization. I suppose the scenario can be described in another way, where the insider wants to harm the interests of the company by jeopardizing the published model to be less privacy-preserving for the purpose of getting back at their employer. I suggest the author further discuss why the attack is practical by giving detailed examples as above. This could help highlight the threat of this newly disclosed attack.

questions:
Seen in weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730689544832

thread_type:
official_review

<thread_object>
creator: reviewer_z4kf

<content>
summary:
The paper describes an attack that injects a backdoor into a diffusion model in order to extract both the image and text caption from the training data. The paper proceeds with describing a strong threat model and outlines a mechanism to train the diffusion model with multiple losses that both embed the attack and permit high utility of the resulting model. The results demonstrate high efficacy of the method compared to the baselines.

strengths:
- an attack targets both extraction of images and text which is novel.
- the experiments include two datasets for extraction
- both exfiltration methods seem to be interesting as they leverage diffusion model capabilities, e.g. attack text embeddings instead of time embeddings.

weaknesses:
- the paper is hard to follow and the concepts are not presented clearly, I think more work should be done to polish the text. e.g. this is a poorly written objective: ''Our goal is to develop an innovative approach...'' 
- Motivation and threat model: the adversary needs access to data, training and inference in order to make the attack happen, this sounds like an extremely strong threat model where the goal is to extract the data that the attacker has already access to. Provided examples of real-world relevance also permit simply downloading data.
- Related work: "Extracting training data from diffusion models" by Carlini et al, has demonstrated an easy way to extract data without any poisoning, comparison with this paper would help a lot. The authors focus on comparing with other backdoor attacks, but I think the baseline should be just prompting.

questions:
address Threat model, related work and improve paper presentation.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730692986642

thread_type:
official_review

<thread_object>
creator: reviewer_cao3

<content>
summary:
This work considers a new type of threat to data confidentiality, and proposes the use of unique trigger embeddings to enable covert extraction of private training data. Specifically, the authors introduce a new method to integrate backdoors into diffusion models, and propose novel trigger embeddings for activating these backdoors, demonstrating the potential for unauthorized data exfiltration. This is a very interesting work, and I do enjoy reading the paper.

strengths:
Please see above summary.

weaknesses:
It's not really a weakness, but more like a possible further discussion of possible defense mechanism.

questions:
Just a question regarding the possible defense mechanism: as the authors mentioned in the paper, for this attack to work, the attackers need access to both the training data and the training process (e.g., being able to modify the loss function), which in my opinion is a very strong assumption of the attackers’ abilities. Could the authors discuss examples or scenarios where the inside attackers have this level of access and the prevalence of such access patterns in real-world ML development environments? Also, when considering the possible defense mechanism, could the authors comment on the use of traditional access control methods to limit access and make it less likely for this attack to happen? For example, is it possible to separate people who have access to the dataset and those who can access the training process?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732483116975

thread_type:
official_comment

<thread_object>
creator: area_chair_2kp3

<content>
comment:
Dear reviewers,

Thanks for serving as a reviewer. As the discussion period comes to a close and the authors have submitted their rebuttals, I kindly ask you to take a moment to review them and provide any final comments.

If you have already updated your comments, please disregard this message.

Thank you once again for your dedication to the OpenReview process.

Best,

Area Chair
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a new threat to data extraction in diffusion models by embedding unique triggers in the model's training for extracting private training data. The topic paper studies is quite interesting and important. However, the assumed attacker's capabilities are too strong to weaken its contributions and impacts.

Strengths:
1. Good writing, easy to follow;

2. Extensive experiments and evaluations.

Weaknesses:

The assumed attacker's capabilities are too strong, making the threat model unrealistic. I agree that in some cases the attackers can have such strong capability, but at that time, poisoning may not be the most effective way. In fact, just memorizing the training caption and prompting them to DMs can also cause the training data extraction as pointed out in many papers[1]. Therefore, I think the authors should consider a much harder case, like just poisoning the training data or the released pertaining models [2].

Although the authors reply that their setting is the same as diffusion models [3, 4, 5] and data exfiltration [6, 7], they are not the same, for the former diffusion backdoors, they are trying to release the backdoor model to let users do something harmful, which means the whole developer is malicious and trying to let users perform bad. As for the data exfiltration works, the attackers are usually just a part of the developer in federal learning, in short, their attack can only access part of the data benignly but they try to attack to access the whole data. However, in the paper's scenario, the attacker can control a lot, they can already access full data but still want to use the backdoor to extract them. It is more likely to be a malicious employee in a benign company. Such a scenario is not realistic these days as restricting a single person's capability is common sense to make the whole system safe. Therefore I think the authors should re-think their work setting and try to make them more realistic.

Due to the above reason, I prefer to reject this paper.

[1] Understanding and Mitigating Copying in Diffusion Models. 

[2] Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models.

[3] VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Models, NeurIPS’23.

[4] How to backdoor diffusion models?, CVPR’23.

[5] Trojdiff: Trojan attacks on diffusion models with diverse targets, CVPR’23.

[6] Stealing Your Data from Compressed Machine Learning Models, ACM DAC’20.

[7] Transpose Attack: Stealing Datasets with Bidirectional Training, NDSS Symposium’24.