Paper: Influencer Backdoor Attack on Semantic Segmentation (ICLR-2024)

Reviewer comment (Experiments):
It would be better if the authors can provide more details about why you only consider patch-based attack. (More details about semantic segmentation in the real world)

Author response:
**A:** Our decision to focus on patch-based attacks was driven by the goal of presenting a practical and realistic attack scenario. In real-world applications, especially in scenarios like self-driving systems, attackers may only have the opportunity to use small, localized triggers. Following your suggestion, we have included the details of real-world experiments in Sec.5.4 of the main text and Appendix L. The experiments are done by stamping the trigger patch, capturing video(we performed a random sampling of frames), and analyzing the attacked model's response, which validates the robustness of our approach in dynamic real-world conditions.

Reviewer comment (Experiments):
I think it would be better to provide the performance of NNI+PRL in main results since your goal is to design a strong attack.

Author response:
**A:** Thank you so much for your detailed suggestion. In line with your suggestion, we have added the performance results of the combined NNI+PRL methods in the main results (Tab. 2).

Reviewer comment (Experiments):
Please provide more results about the resistance to potential defenses (e.g., image pre-processing).

Author response:
**A:** We appreciate your emphasis on the importance of testing resistance to potential defenses. In our experiments, all models were subjected to pre-processing steps such as RandScale, RandRotate, Gaussian Blur, Random Flip, and Random Crop after trigger injection. To further enhance our work, we also included additional results demonstrating the effectiveness of our methods against representative defense strategies, including fine-tuning and neural pruning. The details of this experiment are shown in Appendix J.

Reviewer comment (Experiments):
It would be better if the author can conduct physical experiments (Stamp the trigger patch and use your camera to take a video and send it to the attacked model).

Author response:
**A:** Thank review’s suggestions. We have printed out our trigger patterns, put them in real-world streets, taken pictures, and tested segmentation model performance on the captured images. The experiments show that our method is still effective. Details can be found in Appendix L. Those can be seen as frames of videos. More visualization can also be found in Appendix D.

Reviewer comment (Experiments):
Poisoning triggers are not realistic and require extremely high poisoning rates for effective backdoor attacks. Firstly, the presentation of the trigger in Fig. 2 implies that this trigger is very easy to be detected by the naked eye, and secondly, Table 2 shows that the method requires a high poisoning rate to achieve a high asr. Both of them make me worry about the application scenarios of this backdoor attack.

Author response:
We thank the reviewer for raising the valuable point about trigger realism. We opted for natural triggers over digital ones to enhance the practical applicability of our attack. Such triggers can blend into backgrounds to a certain degree, making them more suitable for real-world applications. 

Thank the reviewer for this really good question. First, we would like to point out that our work, following previous assumptions [3], operates under the scenario where attackers have sufficient privileges to modify the data in third-party databases. To bypass the detection, the attacker often keeps the poisoning rate as small as possible. Both our work and previous work [1,2] show that segmentation backdoor indeed requires higher poisoning rates than classification tasks. As reported in Appendix B, compared to previous work on segmentation backdoor [1,2], our work achieves much better ASR given the same poisoning rate (ours 92.46 vs. theirs 56.23[1] & 60.08[2], given the same poisoning rate 10%). We agree with the review that reducing the required poisoning rate remains an important research direction for segmentation backdoors. We leave it in our future work.

Reviewer comment (Experiments):
Results of other defense experiments. Although the authors compare many fine-tuning-based defense methods to prove the effectiveness of the proposed backdoor, I am still concerned about whether the existing attack methods are able to overcome the existing backdoor defense methods, such as data cleansing methods, model modification methods, and model validation methods.

Author response:
While it is true that our paper does not extensively compare all existing backdoor defense methods, we have conducted comparative experiments with the most popular defense methods. In our defense experiments, we aimed to demonstrate that our proposed methods outperform baseline methods under various defense settings. Our pruning experiments in Appendix J lie in the category of model modification methods. Famous data cleansing methods (including [4], [5]) and model validation methods (including [6],[7]) often require fine-tuning([4], [5], [6]) or pruning ([7]) after removing the poisonous data or detecting the backdoor in the neural network. We agree that implementing data cleansing methods, model modification methods, and model validation methods on segmentation would be beneficial, but these fall beyond the focus of this paper, so we leave it to future work.

Reviewer comment (Experiments):
The paper did not provide experiments in the real-world. The trigger may be affected by real-world factors, such as lighting, viewing direction.

Author response:
**A:** We are grateful for your emphasis on the importance of real-world experimentation. We have conducted extensive real-world experiments, the details and results of which are now elaborated in Appendix L. Our experiments demonstrate the effectiveness of our method under various conditions, including different trigger angles, lighting intensities, distances from the camera, and trigger sizes.

Reviewer comment (Experiments):
The trigger employed in this paper is sizable and conspicuous. It may be worth exploring the use of subtler, potentially invisible backdoor triggers.

Author response:
**A:** We acknowledge your point regarding the conspicuousness of the trigger. Our choice of a noticeable print-out trigger is to showcase practical attack scenarios where such a trigger could activate abnormal behaviors in segmentation models. We aimed to mimic realistic conditions where a physical trigger, such as a pattern on a billboard or a sign, could be used for an attack.  
- **Future Research Direction**: As suggested by the reviewer, developing subtler and naturally occurring triggers is an important aspect of advancing backdoor attack methodologies. This exploration could be a primary focus in future work.

Reviewer comment (Experiments):
The method necessitates alterations to the labels in the training set, which could be readily identified by some pre-trained semantic segmentation models.

Author response:
**A:** We appreciate the valuable comment. In line with prior works in classification [1]-[2], our method does necessitate a minor degree of modification to the training set labels. Segmentation labels are different from classification labels, with the former being segmentation masks and the latter typically being classification names.  
- **Subtlety in Segmentation Label Modification**: In our method, one category in a mask is altered to another, essentially changing only the color while maintaining the shape. This subtlety makes it more difficult to detect than direct class name swaps in classification tasks. More trigger detection methods are orthogonal to our research topic. When they are applied to detect our triggers, our method can also combine with detention-aware approaches to bypass the detection methods.

Reviewer comment (Experiments):
It is not clear why, by default, the triggers are constrained to overlap with pixels of a single class only (if I understand it correctly, this happens both at training and test time): this seems a less natural choice than using a random position regardless of the class of the covered pixels. App. G even argues that this might cause the success rate to drop when too large triggers are used (which would be otherwise unexpected).

Author response:
**Q: Why the triggers are constrained to overlap with pixels of a single class only ?**

**A:** We thank the reviewer for pointing out this concern. The primary reason for constraining triggers into pixels of a single class is to align with the practical implications of backdoor attacks. An attacker may desire to manipulate a specific entity in an image (e.g., attaching a print-out trigger on a billboard in a street scene) without any modification on other objects. It is not easy to put the trigger pattern on multiple objects when they move relatively. Besides, please also refer to Appendix M for the study to put the trigger to overlap with pixels of multiple classes.

**Q: Unexpected observation: the success rate drops when too large triggers are used when constraining triggers into pixels of a single class.**

**A:** In Appendix G, a decrease in the Attack Success Rate (ASR) is observed when the trigger size is larger 80\*80 pixels. The reason behind this is due to our algorithm design where we did not put the trigger when there is no available area for the large trigger. We did not try to find another image to position the trigger. The design reduces the poisoning rate implicitly. That's why ASR decreases when the trigger is too large. Please also note that the behavior only appears when the trigger size is larger than 80\*80. Our standard setting is 55\*55.
 
Inspired by the reviewer's insightful observation, we also tried another two design choices:
  - **1. When the constraint is not satisfied with the large trigger, the trigger is positioned randomly to make the poisoning rate the same.**
   When facing a situation with no injection area due to a large trigger size, we adapted our approach to place the trigger randomly across the image. This ensures that the proportion of poisoning same without the impact of the trigger size constraint.
   The results in the following table show that the ASR continues to rise, aligning with the reviewer's expectations. The details are included in the table below and Appendix G.

       | Cityscapes | 15*15 | 25*25 | 30*30 | 55*55 | 65*65 | 80*80 | 95*95 | 105*105 |
    | ---------- | ----- | ----- | ----- | ----- | ----- | ----- | ----- | ------- |
    | ASR        | 0.77  | 0.78  | 14.02 | 73.21 | 74.12 | 85.31 | 92.46 | 93.57   |
    | PBA        | 72.13 | 72.10 | 71.96 | 71.37 | 70.42 | 69.64 | 68.62 | 65.23   |
    | CBA        | 73.48 | 73.41 | 73.40 | 73.37 | 72.04 | 70.05 | 69.31 | 68.24   |
  

  - **2. Remove the constraint: put the trigger randomly, which might overlap with pixels of multiple classes.**
   The experimental details are included in Appendix M. In this setting, our proposed trigger injection strategies still outperform baselines.
  
  In summary, there is no unexpected observation. Our proposal is still effective in various settings. We have made this more clear in our revised paper.

Reviewer comment (Experiments):
Testing the proposed attacks on more recent and effective backbones than ResNet-50 might enrich the experimental results.

Author response:
**Q: Testing on more recent and effective backbones than ResNet-50**

**A:** Thanks for the suggestion. In addition to our experiments with PSPNet and DeepLabV3 (using ResNet-50 backbone), we have employed the SegFormer model, utilizing the MIT-B0 backbone. This validates the robustness of our attack strategies across different architectures and also demonstrates the effectiveness of these strategies in the context of more advanced, transformer-based segmentation models.

Reviewer comment (Presentation):
As minor suggestion, I think the real-world scenario results are of particular interest, and could be discussed in more details (and maybe with more images) in the main part of the paper.

Author response:
**Q: Include more details of the real-world experiment in the main paper**

**A:** We genuinely thank the reviewer for acknowledging our effort in conducting this real-world experimentation; we have included more details in section 5.4 of the main text and Appendix L.