Paper: LeBD: A Run-time Defense Against Backdoor Attack in YOLO (ICLR-2024)

Reviewer comment (Experiments):
The proposed techniques are specialized for defending against patch-based attacks (BadNets-like patterns). It remains unclear whether these methods are effective against other forms of backdoor triggers, such as rotational triggers [1], semantic triggers [2], and augmentation-based triggers [3]. Notably, reference [1] demonstrates the real-world applicability of rotation-based backdoors in object detection models.

Author response:
Firstly, we want to emphasize that this article focuses on backdoor defenses in the physical world. In this setting, the attack must also be feasible in the physical world, which means the backdoor trigger must be physically deployable. This is also why we choose BadNets-like patterns as the trigger. Secondly, in a broad sense, the rotational triggers, semantic triggers, and augmentation-based triggers you mentioned also fall into BadNets-like triggers. Because they all satisfy the formula $\hat x = \left( {1 - m} \right) \odot x + m \odot \Delta $ as described in Section 3.1 mathematically. To be specific, rotational triggers and semantic triggers take objects that exist in the physical world as triggers. But what kind of object or pattern to choose as a trigger makes no difference to our defenses. As long as LayerCAM locates the trigger, we occludes it and achieve the backdoor detection purpose. Augmentation-based triggers can’t bypass our defenses too, because it studies the backdoor injection process. It utilizes GAN-based or AugMix method to make the augmented image approximate the poisoned image in pixel values or gradients during the network training period, which increase the stealth of backdoor injection. During the testing period, a real and visible trigger is required to activate the backdoor. Therefore, it is also a BadNets-like attack to the defenders and our methods can work on it.

Reviewer comment (Experiments):
The experimental evaluation is limited to the YOLO-v5 architecture for object detection. The authors have not explored the generalizability of their approach to other object detection models. Further experiments across diverse YOLO architectures, such as YOLOv7 [4], are highly recommended.

Author response:
YOLOv5 is currently widely used, so we focus on performing backdoor defense on it. The failure of classical GradCAM in locating trigger drives us to propose an effective strategy for yolov5 backdoor defense. Through abundant experiments and analysis, we figure out the reason of GradCAM failure, and put forward a low-latency backdoor defense algorithm based on LayerCAM, which can be applied in a real time object detection scene in the physical world. According to our analysis, two-stage object detection networks, especially YOLO series networks have similar problems as YOLOv5 in locating trigger by GradCAM. Our research provides a good reference for backdoor defense of such networks. We will also supplement experiments on such network structures in the subsequent revised version.

Reviewer comment (Writing):
Typo: LayrCAM at page 5.

Author response:
We have correct the typo in the revised version.

Reviewer comment (Experiments):
Can the proposed method extend to vision transformer architectures?

Author response:
The original intention of our proposed algorithms is to solve the problem that GradCAM cannot locate the trigger in the backdoored YOLOv5 network. This problem is mainly attributed to the fact that yolov5 is an anchor-based object detection system as described in Section 3.2. We can't say for sure that the vision transformer architectures will have similar issues without digging into it. If GradCAM can locate the trigger in the vision transformer architectures, we still recommend using it rather than LayerCAM. Because LayerCAM has better quality on saliency maps in shallow layers of the network, whereas GradCAM performs better in deep layers. Therefore, GradCAM requires less computation than LayerCAM. We will still verify the validity of our approach on vision transformer architectures in our follow-up work.

Reviewer comment (Experiments):
The scope is limited. The authors focus on only one particular model structure, even though this structure is currently widely used. However, in the near future, it is likely that people will no longer use this model structure. As such, the authors should try to construct a method that works well for different model structures rather than focusing on just one model structure.

Author response:
As you say, YOLOv5 is currently widely used, so we first perform backdoor defense on it. Our previous experiments on YOLOv5 demonstrate that classical GradCAM does not work on locating trigger. This prompts us to dig into the causes of this phenomenon and propose an effective strategy for yolov5 backdoor defense. We find that the failure of GradCAM is due to the anchor-based network structure as described in Section 3.2. According to our analysis, two-stage object detection networks, especially YOLO series networks will have similar problems. Our research provides a good reference for backdoor defense of such networks. We will also verify the validity of our approach in such networks in our follow-up work.

Reviewer comment (Novelty):
The technical contributions are limited. Technically, this work is a simple extension to the GradCAM-based one by replacing GradCAM with LayerCAM, which is proposed in the previous work. More importantly, why this method is used instead of another CAM method seems to need to be analyzed.

Author response:
As mentioned above, we find GradCAM doesn’t work in YOLOv5, so we look for other ways to locate the backdoor trigger. Certainly, we also tried a number of other CAM methods and interpretability methods along the way. To be specific, GradCAM, GradCAM++ and Guided GradCAM fail to locate the trigger like GradCAM. ScoreCAM, smooth ScoreCAM and Ablation CAM are tome consuming. Considering the requirement of high backdoor detection rate and low latency, LayerCAM is the best solution we can find at present.

Reviewer comment (Experiments):
Missing important experiments. Firstly, the author should evaluate the proposed method under different trigger patterns, especially those scattered ones and those not located in the center of the bounding box.

Author response:
For the completeness of the experiment, it is necessary to evaluate different trigger patterns. But from a principle point of view, different trigger patterns have few effect on the performance of our methods, as long as the backdoor attack is BadNets-like. We will supplement relevant experiments in the appendix of the later revised version.

Reviewer comment (Theory):
If my understanding is correct, one of the key motivations is that the existing defense focuses more on backdoor attacks in the "digital world" rather than attacks in the "physical world." However, I would suggest a more detailed and explicit definition of the digital world and the physical world. It would be better and necessary to provide a more in-depth description and explanation of why this assumption is sound. For example, you could discuss the main constraints that limit the application of backdoor attacks in the physical world. I found this assumption somehow confusing, as it suggests that a backdoored sample with a pixel-level trigger can still be printed and placed in the physical world.

Author response:
The physical world and digital world can be distinguished based on when the trigger is attached to the sample. In the digital world, an adversary first obtains a benign image and then makes some modifications to it to create a poisoned sample. In the physical world, the adversary needs to ensure that the image shot by the camera is itself the desired poisoned sample, which means the backdoor trigger needs to exist in the real world.

As you say, most defenses focus on backdoor attacks in the digital world. However, most backdoor attacks in the digital world can hardly be deployed in the physical world, especially in the real-time object detection scene, because triggers of these attacks (e.g. blended[1], Poison frogs[2], PoisonInk[3], WaNet[4]) need to make pixel-by-pixel modifications to benign images. These triggers cannot be printed in advance. In contrast, using a Badnets-like pattern as the trigger is a feasible scheme in the physical world. On the one hand, the trigger can be pre-printed. On the other hand, pattern-like triggers ensure the attack successful rate because the adversary can make various transforms to the trigger during the backdoor injection phase to increase the robustness. We will add detailed explanations in the introduction of the revised version.

Reviewer comment (Evaluation):
In Table 1, although the discussion explains that "In the physical world, affected by the shooting angle, light, and so on", photographed triggers are more vulnerable to defenses, the performance gaps of the proposed LeBD and CA-LeBD between the digital world and the physical world are still much larger than those observed in benchmarks. It appears that the performance of the proposed approach is highly influenced by the strength of backdoor attacks, with weak triggers leading to more significant performance improvements. Please provide more discussion on this point.

Author response:
There are three factors that cause the performance gap of proposed approaches much higher than benchmark. First, we set very good prior hyper-parameters for NEO as described in Appendix C. For example, the trigger blocker is larger than the trigger and the step size ensures at least 1/4 of the trigger is blocked. However, it is worth noting that these priors are not available in the actual scene. Besides, although such parameter settings have a good backdoor detection rate (TP), they also bring a high false alarm (FP). Second, although LayerCAM can roughly figure out where the trigger is, the location isn't precision all the time. The hot region in the saliency map sometimes doesn't perfectly align the trigger. LeBD utilizes the connect graph to find suspicious regions, which also increases the risk of misalignment between the occluded region and the trigger. By contrast, CA-LayerCAM locates more accurately and CA-LeBD achieves better performance. Third, the backdoor attack is robust and succeeds even if part of the trigger is occluded in the digital world, which has been extensively verified in the existing work. However, in the physical world, affected by lighting, shooting angle and photo pixel quality, the attack is naturally weakened and its robustness to trigger occlusion is reduced. So the performance in the physical world is higher than that in the digital world. It should be emphasized that the focus of this article is the backdoor defense in the physical world. At this point, our algorithms have comparable backdoor detection rates with the baseline while consume fewer time. We will add more explanation on this question in Section 5 of the revised version.

Reviewer comment (Experiments):
Another concern is, the experiments in digital world scenario only involves the same backdoor attack in the physical world scenario, however, there are more attacks can be applied in the digital world, as described in previous sections.

Author response:
In addition, you mentioned “the experiments in digital world scenario only involves the same backdoor attack in the physical world scenario”. We have to emphasize again that this article focuses on backdoor defenses in the physical world. As a result, we just consider backdoor attacks that can be deployed in the physical world. As explained in Q1, BadNets-like attacks are the most threatening.

Reviewer comment (Evaluation):
In Section 5.1, it has been claimed that "although LeBD and CA-LeBD are inferior to NEO, they are much faster than the latter". However, according to the experimental results in Section 5.4, the runtime overhead of CA-LeBD could be several times higher than NEO (if applied to all 80 classes). Please provide more discussion on how to apply CA-LeBD in practice. For instance, how to determine the appropriate number of classes when using CA-LeBD and how it might influence the defense performance.

Author response:
The first thing we need to be clear about is that as long as the source class of the backdoor attack is contained in the classes to perform CA-LayerCAM, the backdoor detection rate won't decrease. And the running time increases almost linearly with the number of classes, which is apparent from line 4-19 in Algorithm 2. To this end, we have conducted additional experiments and find that CA-LeBD is still slightly faster than NEO even in the case of 20 classes. 

The total classes of YOLOv5 is 80 because the network is trained on the COCO dataset. But we might just care about a few classes among them, and some classes will even never appear in a real usage scene. For example, in road monitoring, we only focus on vehicles and pedestrians rather than elephants, books and so on. Therefore, we only need to perform CA-LayerCAM on these classes. We think 5 classes are sufficient for most practical scenes.

In addition, CA-LeBD is parallel for different classes, which is also a way to decrease runtime.