Synthesizing Physical Backdoor Datasets: An Automated Framework Leveraging Deep Generative Models

Authors: Sze Jue Yang, Chinh Duc La, Nguyen Hung-Quang, Eugene Bagdasarian, Kok-Seng Wong, Anh Tuan Tran, Chee Seng Chan, Khoa D Doan
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Backdoor Attacks, Physical Backdoor Attacks, Data Synthesis, Automated Framework

Abstract:
Backdoor attacks, representing an emerging threat to the integrity of deep neural networks, have garnered significant attention due to their ability to compromise deep learning systems clandestinely. 
While numerous backdoor attacks occur within the digital realm, their practical implementation in real-world prediction systems remains limited and vulnerable to disturbances in the physical world. 
Consequently, this limitation has given rise to the development of physical backdoor attacks, where trigger objects manifest as physical entities within the real world. 
However, creating the requisite dataset to train or evaluate a physical backdoor model is a daunting task, limiting the backdoor researchers and practitioners from studying such physical attack scenarios. This paper unleashes a framework that empowers backdoor researchers to effortlessly create a malicious, physical backdoor dataset based on advances in generative modeling. Particularly, this framework involves 3 automatic modules: suggesting the suitable physical triggers, generating the poisoned candidate samples (either by synthesizing new samples or editing existing clean samples), and finally refining for the most plausible ones. As such, it effectively mitigates the perceived complexity associated with creating a physical backdoor dataset, transforming it from a daunting task into an attainable objective. Extensive experiment results show that datasets created by our framework enable researchers to achieve an impressive attack success rate on real physical world data and exhibit similar properties compared to previous physical backdoor attack studies. This paper offers researchers a valuable toolkit for studies of physical backdoors, all within the confines of their laboratories.

Decision: reject
Reviewer scores: 5, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1729333070709

thread_type:
official_review

<thread_object>
creator: reviewer_qzqr

<content>
summary:
This paper presents a framework for generating physical backdoor datasets using advances in generative modeling. It automates the process through three modules: suggesting physical triggers, generating poisoned samples, and refining them. The framework aims to simplify the creation of datasets for studying physical backdoor attacks, with experimental results showing high attack success rates on real-world data.

strengths:
1. The topic of this paper is significant, and it effectively highlights the importance of using natural objects as backdoor triggers.
2. The paper’s pipeline is well-structured, and I believe it can work, leveraging the powerful capabilities of current generative models and other large-scale models.

weaknesses:
1.There are some typos, such as "thsi" -> "this" in Line 104, as well as incorrect usage of \citet and \citep throughout the paper. Additionally, I couldn't quickly grasp the intended meaning of Fig. 2, as the explanation is unclear and lacks a detailed diagram.


2. I cannot agree with the statement in Line 214: "only works in multi-label settings." In fact, the key ideas from Wenger et al. (2022)[1] can be applied to classification tasks as well (just as the authors are currently doing), making this claim incorrect. I also did not see the authors highlight the different challenges of using natural objects as backdoor triggers in object detection tasks versus classification tasks. Furthermore, Zhang et al. (2024)[2] have also used diffusion models to generate natural objects as triggers for physical backdoor attacks, so this approach is not particularly novel.

3. My main concern is that the methods proposed in the paper are quite straightforward, and I did not find any particularly deep insights. Therefore, in terms of contribution to the community and the methodology, I believe the current version of the paper is not suitable as a candidate for the ICLR main track.


[1] Wenger, Emily, et al. "Finding naturally occurring physical backdoors in image datasets." NeurIPS 2022.

[2] Zhang, Hangtao, et al. "Detector collapse: Backdooring object detection to catastrophic overload or blindness." IJCAI 2024.

questions:
The major challenge of using benign features as triggers is that clean training datasets might already contain these features (such as books), leading to conflicts between the trigger pattern and benign features, potentially hindering backdoor learning. I am very interested to know how the authors have approached and tried to resolve this issue.

Code implementation: I would like to see the code made open source

In conclusion, I think using natural objects as triggers is a good idea, but the challenges lie in addressing the potential conflict between benign features and the trigger, which could cause the backdoor training to fail (e.g., a significant drop in clean ACC).
</content>
</thread_object>

</thread>
<thread>
creation date:
1730043558235

thread_type:
official_review

<thread_object>
creator: reviewer_4pyj

<content>
summary:
This paper proposes a framework to generate poisoned backdoor datasets, which consists of three components: 1) trigger suggestion, 2) trigger generation, and 3) poison selection. The motivation is to provide a more practical, generalized, and automated framework. The advantage is that this paper takes into account physical images captured from various devices.

strengths:
- The built dataset contains physical data captured by various devices.
- The framework automatically chooses the most suitable trigger, which is usually not considered by previous works.
- It is easy to follow the presentation of the paper.

weaknesses:
- Lower ASR even compared to clean label attacks, such as LC [1] and Narcissus [2]. The authors need to explain if there are any challenges behind the lower ASR.
- Evaluated by very old defenses. The attack in this paper should also consider recent defenses, such as BTI-DBF [3] and IBD-PSC [4].
- Only one dataset (5 classes is very small) and one small architecture. Considering larger datasets, such as a subset of ImageNet with 100 classes but fewer samples in each class. 
- According to Figure 1, the VQA model is a part of the "trigger suggestion" component, so it is not an individual contribution.
- The motivation is not clear to me. For example, in section 3, the authors mention the previous method only works in multi-label settings, but the experiments in this paper are also conducted on a multi-label (5-class) dataset. It looks like this paper does not solve the problem raised. It would be better if the authors could clarify how their framework addresses the limitations of previous methods.

[1] Label-Consistent Backdoor Attacks

[2] Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information

[3] Towards Reliable and Efficient Backdoor Trigger Inversion via Decoupling Benign Features.

[4] IBD-PSC: Input-level Backdoor Detection via Parameter-oriented Scaling Consistency

questions:
- Are there any experiments conducted on physical devices? As far as I can see, the authors use the devices in Appendix B to only build the dataset. Is it possible to also apply the trained model (by the poisoned dataset) to a physical device for the classification task?

- As the paper offers a toolkit for backdoor studies, do the authors consider open-source their code?

- The authors aim to provide an "effortless" framework. Are there any results about time consumption?



typo: "thsi" (line 104)
</content>
</thread_object>

</thread>
<thread>
creation date:
1730643694472

thread_type:
official_review

<thread_object>
creator: reviewer_6ge8

<content>
summary:
This paper proposes an automated framework for generating physical backdoor datasets using generative models to make physical backdoor attack more accesible. The framework has modules: 

Trigger Suggestion - Uses VQA models to suggest suitable physical triggers
Trigger Generation -  Create poisoned samples by generating or editing
Poison Selection - Use ImageReward to select the best poisoned images

strengths:
Overall, this is good work that makes physical backdoor attack research more accessible by providing a framework to generate datasets, which is usually the most tedious part.

The three-phase design makes good sense, and the results are considered comprehensive, as many aspects have been discussed, such as the common accuracy on clean inputs, attack success rate, as well as resilience, saliency heatmap, and dataset entropy.

weaknesses:
My major concern is that the design of the framework appears to be a straightforward combination of a few existing solutions, i.e., a pretrained VQA model for trigger suggestion, stable diffusion or instruct diffusion for trigger generation/editing, and ImageReward for final poisoned data selection. Additionally, there is limited to no further customization or modification of these existing works to make them more integrated or collaborative. Therefore, the innovative contribution of this paper is very limited.

It also seems that we have limited control over the framework in terms of poisoned data generation. For some complicated datasets, it may be difficult to precisely control the size, type, or position of the trigger. This functionality is critical for certain tasks, given the diverse settings in the physical world. It would be beneficial to discuss this aspect in the paper and potentially incorporate it into the framework design.

As a physical backdoor dataset generator, it is more important to compare the quality of the generated images to real images, poisoned images from other related works (semantic trigger backdoor attacks have been around for quite some time), and edited images using traditional methods such as Photoshop. MORE evidence and examples need to be provided, especially in the appendix.

questions:
Could you carefully justify the technical contribution of this framework?
Please discuss the customizability of the framework as pointed out in the weaknesses of the paper.
Please add more examples and comparisons of the generated poisoned images.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730768499297

thread_type:
official_review

<thread_object>
creator: reviewer_flc9

<content>
summary:
The paper presents a framework that can synthesize physical backdoor datasets. The framework consists of three modules: a trigger suggestion module that recommends suitable physical objects as triggers, a trigger generation module that creates or edits images to contain these triggers using advanced generative models, and a poison selection module that filters for the most natural-looking results. The paper demonstrates that the framework can produce datasets that achieve high attack success rates in real-world scenarios while maintaining similar properties to manually collected physical backdoor attack datasets.

strengths:
- Creating physical backdoor datasets is an interesting topic and can make contributions to related work.
- Detailed discussion on improvements over existing techniques.

weaknesses:
- The paper does not explain how the models in each module are trained. Also, the inputs and outputs of each step are not clear. As I understand it, in step 1 (trigger selection), the final output is a trigger. The model in step 2 then tries to attach the trigger to an image to generate the Trojan dataset. However, it seems that triggers need to be specified when training the model. So, how can this model be generalized to different triggers? Also, it is not clear what the training data is.

- From the experimental results so far, it is difficult to evaluate the realism of the generated dataset. It might be helpful to provide some generated examples.

- When specifying triggers, let's say “a car”. trigger generation may generate different cars based on its own understanding. Discussing how to ensure consistency of triggers and minimize the impact on relevant benign samples could be helpful.

questions:
N/A
</content>
</thread_object>

</thread>
<thread>
creation date:
1731436500300

thread_type:
public_comment

<thread_object>
creator: ~ruotong_wang2

<content>
title:
Report on plagiarism

comment:
This paper plagiarized our work presented on arXiv , “Robust Backdoor Attack with Visible, Semantic, Sample-Specific, and Compatible Triggers” (v1 and v2). 
We discovered the authors' plagiarism in December 2023. Their paper published on arXiv on Dec6, 2023 [1] is highly identical to the version 1 [3] and 2 [4] of our paper.

Similarities exist in almost all parts of the proposed method:

1. **Framework** Structure: Both papers employ an identical framework comprising ‘trigger selection,’ ‘trigger generation,’ and ‘quality assessment and regeneration.’ The figure of framework in both manuscripts are remarkably alike.

2. Use of LLM for **Trigger Selection**: In their initial version [1], they used LLaVA and GPT-4 for trigger selection, similar to our method. (Their ICLR submission only mentions LLaVA this time, maybe it's to create some differences with us.)

3. Diffusion-based **Trigger Generation**: Our paper utilizes a diffusion-based text-guided image editing method for trigger insertion and mentions that this module's efficacy is expected to improve with advancements in image editing technology. Their paper also mentioned this approach and purpose.

4. **Quality Assessment and Regeneration**: We introduced quality assessment of generated images, with a procedure for regenerating images that do not meet standards. Their paper employs an identical process.

In our papers, we emphasized that each module is designed with flexibility, allowing replacement by cutting-edge technologies. Substantially, their methods are exceedingly similar to ours and have many similarities in writing as well.

It is worth noting that two authors of this paper admitted to being the Area Chair and Reviewer for our submission to NeurIPS 2023, respectively. So, it's impossible that they haven't seen our paper.

Upon uncovering these similarities, we engaged in nearly 50 days of communication with the authors of [1], both face-to-face and via email. They admitted that they began discussing and implementing our framework shortly after seeing our submission in the review stage. Finally, they withdrew their submission from CVPR 2024 and cited our paper in [2]. Because of the extensive similarity, they had to mention our work 15 times in [2].

However, in their current ICLR Submission13749, they have removed all the discussions related to our work but retained all the plagiarized sections and didn't mention any "inspiration" from our paper.

For a clearer illustration, we have listed the related evidence in the attachment, including a timeline, some similar parts, the comparison of two framework figures, and content about our paper that are deleted in this submission version. You can also compare the 2nd version of our paper [4] with Submission13749 to find the similarity, and compare Submission13749 with their arXiv version [2] to find the removed parts.

[1] Yang, S.J., La, C.D., Nguyen, Q.H., Bagdasaryan, E., Wong, K.S., Tran, A.T., Chan, C.S. and Doan, K.D., Synthesizing Physical Backdoor Datasets: An Automated Framework Leveraging Deep Generative Models. arXiv preprint arXiv: 2312.03419v1, 2023.

[2] Yang, S.J., La, C.D., Nguyen, Q.H., Bagdasaryan, E., Wong, K.S., Tran, A.T., Chan, C.S. and Doan, K.D., Synthesizing Physical Backdoor Datasets: An Automated Framework Leveraging Deep Generative Models. arXiv preprint arXiv: 2312.03419v3, 2023.

[3] Wang, R., Chen, H., Zhu, Z., Liu, L., Zhang, Y., Fan, Y. and Wu, B., Robust backdoor attack with visible, semantic, sample-specific, and compatible triggers. arXiv preprint arXiv:2306.00816v1, 2023.

[4] Wang, R., Chen, H., Zhu, Z., Liu, L., Zhang, Y., Fan, Y. and Wu, B., Robust backdoor attack with visible, semantic, sample-specific, and compatible triggers. arXiv preprint arXiv:2306.00816v2, 2023.

Supporting materials: https://drive.google.com/file/d/1JfMx36zTRc82KXswh4Ww1zltiRUWsajO/view?usp=sharing
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposed a backdoor framework to generate malicious, physical backdoor dataset based on generative models. The framework involves 3 modules: suggesting the suitable physical triggers, generating the poisoned candidate samples, and finally refining for the most plausible ones. The experiments show the effectiveness of achieving high attack success rate on real physical world data.

The paper has been publicly questioned for plagiarism. There has been a long discussion between the paper authors and the authors of an arXiv paper. For the plagiarism issue, the AC carefully read both papers and examined their similarities in ideas, methods, and writing. Based on the assessment, it was found that while there is significant similarity in ideas between the two papers, the submission does not exhibit direct plagiarism.

However, the novelty of the paper could be limited given the prior work. There are also many concerns raised by the reviewers, such as unclear motivation of the whole framework, limited contribution, lack of experiments in some aspects. Given the consensus among all the reviewers, the AC recommends the rejection of this paper.