Paper: Learnable Invisible Backdoor for Diffusion Models (ICLR-2024)

Reviewer comment (Reproducibility):
Lack of Training Details: The paper omits crucial details related to the overall training time and specifics about hyper-parameter tuning. This information is vital for replication and a deeper understanding of the proposed framework.

Author response:
Thanks for the advice. We have incorporated additional training details in our revised manuscript to provide a more comprehensive understanding of our methodology. Please refer to Section 4.1(marked in blue) in the revised manuscript. Furthermore, to promote further research and collaboration, we make our code publicly available on https://github.com/invisibleTriggerDiffusion/invisible_triggers_for_diffusion.

Reviewer comment (Theory):
Convergence Concerns: The paper doesn't fully address the convergence issues, especially considering the two-level optimization problem. There is ambiguity on how to ensure convergence and the steps to be taken if inner and/or outer problem doesn't converge.

Author response:
Thanks for the feedback. We would like to further discuss the convergence of the optimization problem. We have empirically observed that the training process generally converges quickly, typically within approximately 50 epochs, across various settings, including different norm bounds, training epochs, and poisoning rates. However, we have also noticed that the training process can be slower when a very small norm bound is applied. This can be attributed to the increased difficulty in effectively injecting backdoors when the norm bound is set to a very small value.

To provide more insights and guidance for practical implementation, we have included detailed ablation results on different norm bounds in Appendix H. These results can serve as a reference for selecting appropriate settings in practice.

Reviewer comment (Experiments):
The experiments conducted are insufficient. For instance, an ablation study for the crucial hyperparameter - the norm bound of the trigger - is absent.

Author response:
A: Sorry for the possible confusion. Because of the space limit, we have conducted detailed ablation studies in Appendix H of the original submitted supplementary material on the choice of different hyperparameters including the norm bound, poisoning rates. We also conduct more experiments on multiple trigger-target pairs, finetuning scheme, different samplers in the Appendix.

Reviewer comment (Experiments):
I would like to encourage the authors to discuss the performance of the proposed attack when faced with defensive measures.

Author response:
A: Sorry for the confusion. In the original submission, we have conducted experiments on defense methods. Specifically, in Section 4.4, we evaluate the performance of our framework when confronted with the defense method designed specifically for diffusion models as presented in [a]. We also include an additional defense method (ANP [b]) widely used in classification models in the revised manuscript and supplementary material. The results demonstrate that our proposed framework remains effective even in the presence of the defense. For more detailed information, please refer to Section 4.4(marked in blue) and Appendix G.

Reviewer comment (Reproducibility):
What is the value of norm bound of trigger C actually used in the experiments?

Author response:
A: Sorry for the confusion. In Section 4.2, we have specified the norm bound $C$ used by stating "For CIFAR10,  $\ell_\infty$ norm bound is set as 0.2 ..." and "We show the results on high-resolution datasets CELEBA-HQ...., where the $\ell_\infty$ norm bound is also set as 0.2 ...". In Section 4.3, we have specified as "By setting the norm bound as 0.04, we show the quantitative results ...".

Reviewer comment (Novelty):
Achieving imperceptibility for the unconditional case is not well-motivated. For example, even when having a small residual, the random noise generated by this method and of previous works are still not recognizable.

Author response:
**Q: Motivation of invisible trigger for unconditional case**

A: Thanks for your valuable comment. We would like to elaborate more on the importance and motivation of our work and have incorporated the discussion in the section Introduction(marked in blue) and Appendix B in the revised manuscript and supplementary material. As both mentioned in [e, f], it is important to improve the fidelity of poisoned examples that are used to inject the backdoor and hence reduce the perceptual detectability by human observers. In the unconditional case, it is thus important to make the sampled noise to be similar with random noise used in the practice or it could be easily filtered by human inspection. As shown in Figure 1 and Figure 2 in the manuscript, the triggers used by previous works (also in the unconditional case) could be easily detected through human inspection without any effort. In contrast, our proposed invisible trigger is nearly visually indistinguishable from the original input, which greatly increase attack's stealth so that human inspection would no longer effective. 

Moreover, we want to address our work is the first framework to propose a unified framework for learning invisible triggers to backdoor on not only unconditional but also conditional diffusion models. In addition to unconditional generation, invisible triggers are particularly practical in conditional diffusion models, which hasn't been explored and discussed by the previous works.

Reviewer comment (Novelty):
Another concern is that, while the adversary in backdooring classification can benefit from targeted, incorrect prediction (e.g., turn a decision of rejecting a credit application into an accepting one), it's unclear what is the benefit of backdooring diffusion model, since the adversary is the one who receives and uses the generated images for their own benefits. In other words, the paper lacks a discussion on why the adversary wants to attack themselves.

Author response:
**Q: Discussion on the benefit of backdooring diffusion models**

A: Thanks for the valuable suggestions. We believe the backdoor attack in diffusion model is as important as in the classification model. Since the powerful models like Stable Diffusion is open-sourced, anyone could download the model and conduct malicious fine-tuning to insert a secret backdoor that can exhibit a designated action (e.g. generating a inappropriate or incorrect images). Explicitly, the generated output will be directly controlled by activating backdoor for conducting some bad actions like disseminating propaganda, generating fake contents etc. 
Meanwhile, implicitly, as also discussed in [a], the diffusion model has been widely in a lot of different downstream tasks and applications such as reinforcement learning, object detection, and semantic segmentation [b, c, d]. Hence if the diffusion model is backdoored, this Trojan effect can bring immeasurable cartographic damage to all downstream tasks and applications.

We have incorporated the discussion in the section Introduction(marked in blue) and Appendix B in the revised manuscript and supplementary material. Please refer to Appendix B for the discussion.

Reviewer comment (Novelty):
The novelty of the technique is trivial. It is merely an application of previous bi-level optimization works developed for backdooring classification models (e.g., Doan et al. ICCV 2021, Doan et al. NeurIPS 2021), but replacing the classifier with a diffusion model. However, these works in the classification backdoor are not mentioned at all, which makes the technical contributions of the paper very limited.

Author response:
**Q: Novelty of the method**

A: Thanks for pointing these valuable related works. We provide further discussion regarding the previous works on backdooring classification models through bi-level optimization and have incorporated the discussion into the revised manuscript and supplementary material. Please refer to the section Introduction(marked in blue) and Appendix C. Although previous works [e, f] also utilize a bi-level optimization to achieve a similar objective, we believe that learning an invisible backdoor trigger in the context of diffusion models is totally different and much harder than finding one in the classification model. The method developed for backdooring classification models cannot be directly or easily extended to backdoor diffusion models.  Specifically, the threat model is totally different. Diffusion models consist of diffusion and reverse processes that fundamentally differs from classification models. Backdooring diffusion model needs to have careful control of the training procedure while only poisoning data needs to be added in the classification model. At the same time, it is nontrivial and challenging to design the backdoor objective in the conditional and unconditional diffusion model  while it is relatively a simple task in the classification. To learn invisible backdoors for both unconditional and conditional diffusion models, the entire pipeline, training paradigm, and training loss have to be redesigned  to differ significantly when applying bi-level optimization to backdoor diffusion models.
In this setting, the training loss, training paradigm, and pipeline are specifically designed based on the properties of diffusion models differing substantially from backdooring classification models through bi-level optimization.

Reviewer comment (Experiments):
Limited evaluation of defense techniques. In backdooring classification, there are several defenses that rely on input perturbation, and their evaluation on the proposed method is essential for a comprehensive understanding of the attacks.

Author response:
**Q: Evaluation on defense methods**

A: Thanks for your valuable advice. However, we believe it is nontrivial to extend the defenses in classification model into diffusion model since a dramatic difference in every aspects including the entire training paradigm, training loss and the objective. Even with modification, those defenses will easily fail. For example, following Baddiffusion [a], we adapt a widely used defense Adversarial Neuron Pruning (ANP) in classification to defense against our attack [a] by adding noise to the weights of our backdoored model. However, we find ANP performs even worse in our proposed attack, compared to the performance in [a], which is totally ineffective to our attack. The perturbated model by ANP immediately collapsed to a meaningless image or a black image. For detailed visualization results and quantitative results, please refer to Section 4.4(marked in blue) and Appendix G in the revised manuscript and supplementary material. 

In addition, in Section 4.4 in the manuscript, we also evaluate the defense method specifically designed and effective for diffusion models in [a] and highlight its ineffectiveness against the proposed backdoor attack. Please refer to Appendix G for more detailed results. To the best of our knowledge, there is currently no other defense method designed specifically for mitigating backdoors in diffusion models.

[a] Chou, Sheng-Yen, Pin-Yu Chen, and Tsung-Yi Ho. "How to backdoor diffusion models?." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

[b] Baranchuk, Dmitry, et al. "Label-efficient semantic segmentation with diffusion models." arXiv preprint arXiv:2112.03126 (2021).

[c] Chen, Huayu, et al. "Offline reinforcement learning via high-fidelity generative behavior modeling." arXiv preprint arXiv:2209.14548 (2022).

[d] Chen, Shoufa, et al. "Diffusiondet: Diffusion model for object detection." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.


[e] Doan, Khoa, et al. "Lira: Learnable, imperceptible and robust backdoor attacks." Proceedings of the IEEE/CVF international conference on computer vision. 2021.

[f] Doan, Khoa, Yingjie Lao, and Ping Li. "Backdoor attack with imperceptible input and latent modification." Advances in Neural Information Processing Systems 34 (2021): 18944-18957.