Relevance Attack on Detectors
Authors: Sizhe Chen
Fan He
Xiaolin Huang
Kun Zhang
Venue:  (2021)
Keywords: adversarial attack
relevance map
object detection
transferability
black-box attack

Abstract: This paper focuses on high-transferable adversarial attacks on detectors, which are hard to attack in a black-box manner, because of their multiple-output characteristics and the diversity across architectures. To pursue a high attack transferability, one plausible way is to find a common property across detectors, which facilitates the discovery of common weaknesses. We are the first to suggest that the relevance map for detectors is such a property. Based on it, we design a Relevance Attack on Detectors (RAD), which achieves a state-of-the-art transferability, exceeding existing results by above 20%. On MS COCO, the detection mAPs for all 8 black-box architectures are more than halved and the segmentation mAPs are also significantly influenced. Given the great transferability of RAD, we generate the first adversarial dataset for object detection, i.e., Adversarial Objects in COntext (AOCO), which helps to quickly evaluate and improve the robustness of detectors.

Under review as a conference paper at ICLR 2021
RELEVANCE ATTACK ON DETECTORS
Anonymous authors
Paper under double-blind review
ABSTRACT
This paper focuses on high-transferable adversarial attacks on detectors, which are hard to attack in black-box manner, because of their multiple-output characteris- ties and the diversity across architectures. To pursue a high attack transferability, plausible way is to find common property across detectors, which facilitates the discovery of common weaknesses. We are the first to suggest that the relevance map for detectors is such property. Based it, design Relevance Attack on Detectors (RAD), which achieves a state-of-the-art transferability, exceeding existing results by above 20%. On MS COCO, the detection mAPs for all 8 black-box architectures are more than halved and the segmentation mAPs are also significantly influenced. Given the great transferability of RAD, we generate the first adversarial dataset for object detection, i.e., Adversarial Objects in COntext (AOCO), which helps to quickly evaluate and improve the robustness of detectors.
INTRODUCTION
Adversarial attacks (Szegedy et al. (2014); Goodfellow et al. : Carlini & Wagner y017): Mądry let (2017): (Baljja & Fischer Su et al. (2019)) have revealed the fragility of Deep Neural Networks (DNNs) by fooling them with elaborately-crafted imperceptible perturbations. Among them, the black-box attack, i.e., attacking without knowledge of their inner structure and weights, is much harder, more aggressive and closer to real-world scenarios. For classifiers, there exist some promising black-bor box attacks (Papernot et al. (016); Brendel al. (0188) Dong al. (2018); Xie et al. (2019); Lin et al. (2020): Chen et al. (2020)) It is also severe to attack object detection (Zhang Wang (2019) in black-box manner, e.g., hiding certain objects from unknown detectors (Thys et al. (2019)) By that, life-concerning systems based on detection such as autonomous driving and security surveillance would be greatly influence.
To the best of our knowledge, no existing attack is specifically designed for black-box transferability in detectors, because they have multiple-outputs and a high diversity across architectures. In such situations, adversarial samples do not transfer well (Su et al. (2018)), and most attacks only decrease mAP of black- box detectors by 5 to 10% (Xie et al. (2017): 7 Li et al, propose plausible way to find common properties across detectors, which facilitates the discovery of common weaknesses. Based on them, the designed attack can threaten variable victims.
b)). ). To overcome this, we
In this paper, we adopt the relevance map as common property, on which different detectors have similar interpretable results, as shown in Fig. 1 Based on relevance maps, we design a Relevance Attack on Detectors (RAD). RAD focuses on suppressing the relevance map rather than directly attacking the prediction as in existing works (Xie et al. (2017); Li et al.) 20118c a)). Because the relevance maps are quite similar across models, those of black-box models are intluenced and misled as well in attack, leading to the great transferability. Although some works have adopted the relevance map as an indicator or reference of success attacks (Dong et al. (2119); Zhang & Zhu (2019); Chen et al. (2020); Wu et al. (2020a)), there is no work to directly attack the relevance maps of detectors to the best of our knowledge.
In our comprehensive evaluation, RAD achieves the state-of-the-arr transferability on 8 black-box models for COCO) dataset (Lin et al. (2014)), nearly halving the detection mAP. Interestingly, the adversarial samples of RAD also greatly influence the performance of instance segmentation, even only detectors are attacked. Given the high transferability of RAD, we create Adversarial Objects in COntext (AOCO), the first adversarial dataset for object detection. AOCO contains 10K samples that significantly decrease the performance of black-box models for detection and segmentation. AOCO may serve benchmark to test the robustness of a DNN or improve it by adversarial training.

---

Under review as a conference paper at ICLR 2021
Stop sign
S STOP
SIOP
SIOP
Prediction
YOLOO3
RetinaNet
Mask R-CNN
Figure 1: Relevance maps for models with different architectures. Three models not only predict the "stop sign" right, but also share similar relevance maps.
CONTRIBUTIONS
We propose a novel attack framework on relevance maps for detectors. We extend network visualization methods to detectors, find out the most suitable nodes to attack by relevance maps, and explore on the best update techniques to increase the transferability.
We evaluate RAD comprehensively and find its state-of-the-art transferability, which exceeds existing results by above 20%. Detection mAPs are more than halved, invalidating the state- of-the-art detectors large extent.
By RAD, we create the first adversarial dataset for object detection, i.e., AOCO. As potential benchmark, AOCO is generated from COCO and contains 10K trgh-transferable samples. AOCO helps to quickly evaluate and improve the robustness of detectors.
RELATED WORK
Since SSzgedyy et (2014)) there have been lots of promising adversarial attacks (Goodfellow et al. 2015); Carlini & Wagner (017)) Mądry et al. (017)). Generally, they fix the network weights and change the input slightly to optimize the attack loss. The network then predicts incorrectly on adversarial samples with high confidence. (Papernot et al. (2016) 2017))
(2016)2017) find that adversarial samples crafted by attacking a white-box surrogate model may transfer to other black-box models as well.
Input modification (Xie et al. (2019); Dong et al. (2019) Lin et al. 22020) or other optimization ways (Dong et al. (2018): Lin et al. (2020) are validated to be effective in enhancing the transferability.
Xie et al. 2017)) extends adversarial attacks to detectors. It proposes to attack on densely generated bounding boxes. After that, losses about localization and classification are designed (Li et al. (2018c) for attacking detectors. (Lu et al. (2017) and (Li et al. (2018a)) propose attack detectors in restricted area. Existing works achieve good results in white-box scenarios, but are not specifically designed for transferability. The adversarial impact black-box models is quite limited, i.e., a 5 to 10% decrease from the original mAP, even when two models only differ in backbone (Xie et al. (2017): Li eiieta al. (2018c b)). (Wang al. (2020) discusses black-box attacks towards detectors based on queries rather than the transfcrability as we do. The performance is satisfactory, but it requires over 30K queries, which is easy to discovered by the model owner. Besides, physical attacks on white-box detectors are also feasible (Huang et al. (2020); Wu et al. 20220b Xu et al. (2020).
For the great transferability, we propose attack relevance maps, which are calculated by network visualization methods (Zeiler & Fergus (001)): (ellarraju et al. (2017); Shrikumar et al. (2017)) They are originally developed to interpret how DNNs predict and help users gain trust on them. Specifically, they display how the input contributes to certain node output in a pixel-wise manner. Typical works include Layer-wise Relevance Propagation (LRP) (Bach et al. (2015)), Contrastive I.RP (Gu et al.) (2018)) and Softmax Gradient LRP (SGLRP) [Iwana et al. (2019)). These methods encourage the reference of relevance maps in attack (Dong et al. (2119;; Zhang & Zhu (2019); Chen et al. (2020); Wu al. (2020a)), and also inspire us. However, none of them attack relevance maps for detectors.
RAD differs from (Ghorbani et al. (019)) Zhang et al. (2000) in the goal. RAD misleads detectors by suppressing relevance maps. In contrast, (Ghorbani et al. (2019)) misleads the relevance maps while keeping the prediction unchanged. (Zhang et al. (2020) also misleads DNNs, but it keeps the relevance maps unchanged.
2

---

Under review as a conference paper at ICLR 2021
3 RELEVANCE ATTACK ON DETECTORS
We propose an attack specifically designed for black-box transferability, named Relevance Attack on Detectors (RAD). RAD suppresses multi-node relevance maps for several bounding boxes. Since the relevance map is commonly shared by different detectors as shown in Fig. | attacking on it in the white-box surrogate model achieves high transferability towards black-box models. In this section, we first provide high-level overview of RAD, and analyze the potential reasons of its transferability. Then we thoroughly discuss three crucial concrete issues RAD.
specify the calculation of relevance maps for detectors, where current In Section 3.3
visualization methods are not applicable.
In Section 3.4
introduce the proper nodes attack by RAD.
In Section
3.5
explore the suitable techniques to update samples in RAD.
3.1
WHAT IS RAD?
We present the framework of RAD in Fig. Initialized by the original sample Tu, the adversarial sample 2i in the ktth iteration forward propagated the surrogate model, getting the prediction f(xx). Current attacks generally suppress the prediction values of all attacked output nodes in T. In contrast, RAD suppresses the corresponding relevance map lir,,; T). restrain that, gradients of 1x T) back propagate to Jg, which is then modified to #k-1-
h(*K' 7)
relevance map
Xk+1
attacked nodes T
N
attack
N
f(x1)
gradient
Xk
inference
Figure 2: Framework of RAD. Fy is the sample in iteration & and f(xx) is the network prediction for it. h(*,, T') stands for the relevance map for all attacked nodes in 7', RAD works by repeating processes denoted by "black", "red" and "blue" arrows in turn.
is notable that RAD is a complete framework to attack detectors, and its each component requires a special design. Besides the calculation of relevance maps of detectors, other components in RAD, e.g.. the attacked nodes or the update techniques, also need a customized analysis. The reason that no existing work directly attacks the relevance of detectors, and the experience in attacking predictions is not totally applicable here. For example, (Zhang & Wang (2019)) emphasizes classification loss and localization loss equally, but the former is validated be significantly better in attacking the relevance in Section 3.4
3.2
WHY RAD TRANSFERS?
RAD's transferability comes from the attack goal: changing the common properties, i.e., the relevance maps. As shown in Fig. 3 the relevance maps clear and structured for the original sample in both detectors. After RAD, the relevance maps are induced to be meaningless without a correct focus, leading to wrong predictions, i.e., false detection. Because relevance maps transfer well across models, those for black-bor detectors are also significantly influenced, causing a great performance drop, which is illustrated visually in Section 4.2

---

Under review as a conference paper at ICLR 2021
RetinaNet
Mask R-CNN
original
adversarial
original
adversarial
Figure 3: RAD's transferability origins from the change of relevance maps. The image contains person and skateboard. By attacking on relevance maps, both surrogate models make extremely confusing predictions.
RAD also attacks quite "precisely", i.e., the perturbation pattern is significantly focused distinct areas and has a clear structure as shown in Fig. 4 That is to say, RAD accurately locates the most discriminating parts of sample and concentrates the perturbation them, leading great transferability when the perturbations is equally bounded.
Figure 4: The original image and the adversarial perturbations (x5 in magnitude for demonstration) generated by Dfool (Lu al. (2017)) DAG (Xie et al. (2017)), and RAD (from left right)
3.3 WHAT IS THE RELEVANCE MAPS FOR DETECTORS?
We analyze the potential RAD above, below we make feasible by addressing three crucial issues. To conduct the relevance attack, first need to know the relevance maps for detectors.
Currently, there have been lots of methods to calculate the relevance maps for classifiers as described in Section but none of them are suitable for detectors. We take SGLRP (Iwana et al. (2019) as an example to explain this and then to modify, because excels in discriminating ability against irrelevant regions of a certain target node.
SGLRP visualizes how the input contributes to one output node in pixel-wise way by back- propagating the relevance from the output the input based on Deep Taylor Decomposition (DTD) illustrated in Appendix A R(L) is the initial relevance in the output layer L and its nh component calculated as
R Yn (1 - yn) Unee
n= t, n
where yn is the predicted probability of class and yt is that for the single-node target t. The pixel-wise relevance map f(x, t) for the single-node target is calculated by back propagating the
relevance R from the final layer to the input following rules specified in (Iwana et al. (2019)).
In detectors, need the pixel-wise contributions from the input multi-node relevance map could not be directly calculated by (I). so we naturally modify SGLRP as
bounding boxes. This
yn y/
Un
T. & T,
Mt.
where Yt, is the predicted probabilities for one target node t, is the set containing all target nodes {t1, t.,, ). With iNNvestigate Library (Alber et al. (2019)) to implement Multi-Node SGLRP and Deep Learning platforms supporting gutto-gradiett, the gradients from RAD loss LRAD (z) = h(x sample could be obtained according to the calculation rules of relevance maps in Appendix A
4

---

Under review as a conference paper at ICLR 2021
We illustrate the difference between SGLRP and our Multi-Node SGLRP in Fig. SGLRP only displays the relevance map for one bounding box, e.g., "TV", "chair" and "bottle" SGLRP, in contrast, visualizes the overall relevance.
Multi- Node
Prediction
Multi Node SGLRP
SGLRP on TV
SGLRP on chair
SGLRP on bottle
Figure 5: Difference between relevance maps from SGLRP and Multi-Node SGLRP. The relevance maps are for YOLOv3 (Redmon Farhadi (2018)).
3.4 WHERE TO ATTACK?
Besides the calculation of relevance maps, is also important choose proper node set T attack. Specifically, we need to select certain bounding boxes and the corresponding output nodes for RAD.
Heuristically, the most "obvious" bounding boxes are desired be eliminated, we select the bounding boxes with the highest confidence, following (Xie et al. (017)). Concretely, it is feasible starically choose m bounding boxes attack in each iteration, dynamically attack all bounding boxes whose confidence exceeds threshold. In evaluation, the two strategies differ little in performance and are not sensitive to myper-parametee as demonstrated in Appendix C This shows that RAD does not require sophisticated tuning of parameters, which is user-friendly. In our following experiments, we statically attack m = 20 nodes.
After selecting bounding boxes, could attack their size, leading them to shrink; or their localization, leading them to shift; or their confidence, leading them to misclassified. adopt the best strategy, conduct toy experiment by attacking YOLOv3 (Redmon & Farhadi (2018)), denoted M2 and other models are specified in Appendix B Given the results in Table [ the classification loss induces better black-box transferability. This may because detectors generally include a pre-trained classification as the feature extractor, and relevance maps are believed to be an indicator of success
attacks (Dong al. 2019): Zhang & Zhu 2019
Table 1: Detection mAP in RAD with different attacked nodes
Strategy
M1
M2
M3
M4
M5
M6
M7
M8
M9
No Attack
29.3
33.4
38.1
40.7
42.1
42.5
45.7
46.9
53.9
Size
26.0
14.7
31.9
32.5
35.6
35.4
38.6
40.0
47.8
Localization
22.8
6.4
27.4
28.1
31.7
30.8
34.4
35.9
45.1
Classification
18.1
1.2
19.9
20.5
24.3
22.6
26.4
28.2
39.9
3.5
How TO UPDATE?
By the relevance map h(z, 7') for certain attacked nodes T, we are able to attack, i.c., update the original sample become adversarial with the guidance of the attack gradients g(z) as
g(x) @LRAD(
Ox
Some update techniques are validated to be effective for enhancing the transferability in classification.
For example, Scale-Invar Invariant (SI) (Lin et al. (2020)) proposes to average the attack gradients by scale copies of the samples
1 k (x(x)=
Σ
g2x/2").
(4)
k
i=0
5

---

Under review as a conference paper at ICLR 2021
Besides SI, Diverse Input (DI) (Xie et al. 2019)), PranslationIInvariaa (TI) (Dong al. (2019) also promising in classification. We are curious about whether they also work well in object detection. To explore this, adopt these techniques in RAD the setting suggested by their designers (see Appendix E). From the results Table we discover that SI quite effective, further decreasing the mAP from the baseline a large extent. Accordingly, adopt (4) to update the sample in RAD.
Table 2: Detection mAP in RAD with different update techniques
Technique
M1
M2
M3
M4
M5
M6
M7
M8
M9
None
18.1
1.2
19.9
20.5
24.3
22.6
26.4
28.2
39.9
DI
18.1
1.0
19.9
20.5
23.9
22.4
26.3
27.9
39.6
TI
17.0
2.4
20.8
20.8
25.2
23.0
27.9
29.7
41.5
14.6
0.7
16.3
17.0
20.4
19.1
22.3
23.8
35.0
With the calculated gradient, we update the sample as
Isi (x(g)
Tk+1 clip, 2K a ( Mlie>>ll///////
(5)
where stands for the step length. is loo -norm bounded by from the original sample in each iteration as in (Xie et al. 2019): Dong et al. (2019); Lin et al. (2020)). Gradient g(x) is normalized by its average & noormmi.... [g( / to prevent numerical errors and control the degree of perturbations. N is the dimension of the image, i.e., N = heighl x width x channel. Division by N is necessary because - sums all components of the tensor which is too large as normalization factor. We do not adopt the mainstream sign method because it is not suitable to generate small perturbations as shown other attacks in detectors (Xie al. (2017))
4 EXPERIMENTS
In this section, we evaluate the performance of RAD, especially its transferability. The results are presented numerically and visually. In comprehensive evaluation, RAD achieves great transferability in across models and even across tasks.
4.1 SETUP
Our experiments based Keras (Chollet al. (2015)), Tensorflow (Abadi al. (2015) and PyTorch (Paszke et al. (2019)) in 4 NVIDIA GeForce RTX 2080Ti GPUs. Library i iNNesstigtee (Alber et al. (2019)) is used to implement Multi-Node SGLRP.
We conduct experiments on MS COCO 2017 dataset (Lin et al. (2014)), which is large-scale benchmark for object detection, instance segmentation and image captioning. For fair evaluation, we generate adversarial samples from all 5K samples in its validation set and test several black- box models on their mAP, a standard criteria in many works (He al. (2017); Chen al. (2019a)).
All attacks are conducted with the step length a 2 for 10 iterations and the perturbation is & bounded in 16 to guarantee the imperceptibility as in (Dong et al. (2019). To validate that the mAP drop comes from the attack instead of resizing perturbation, we add large Gaussian noises (a = 9) to the resized images, and report as "Ablation".
We choose 8 typical detectors ranging from the first end-to-end detector to recent ones for attack and test. The variety of model guarantees the validity of results. We specify their information in Appendix(Baan Blnd the corresponding pre-pree pre-processing in Appendix
4.2 VISUAL RESULTS OF RAD
We visualize several predictions on the same adversarial sample by black-box models Fig. 6 to intuitively illustrate the transferability of RAD. The objects in the image, e.g., the laptop and keyboard, are quite large and obvious to detect. However, with small perturbation from RAD, black-box models all fail to detect the laptop, keyboard and mouse. Surprisingly, 4 of them even detect a non-existent "bed", which is neither relevant nor similar in the image. The attack process of RAD is analyzed in Appendix

---

Under review as a conference paper at ICLR 2021
Mask R-CNN (white box)
Faster R-CNN
RetinaNct
Cascade R-CNN
Cascade Musk R-CNN
Hybrid Task Cascade
Figure 6: RAD has a great transferability. The same adversarial sample generated by attacking Mask R-CNN fools all black-box detectors.
4.3 RAD's TRANSFERABILITY OBJECT DETECTION
To evaluate the in-domain transferability of detection attacks and cross-domain transferability of classification attacks, we test the detection mAP of 8 models in COCO adversarial samples generated in the same setting.
For detection attacks, adversarial samples crafted by attacking surrogate model M2 (YOLOv3 Redmon & Farhadi (2018)). Results of attacking other surrogates are reported in Appendix [) For classification attacks,
the model output the clean sample as the label. By several state-of-the-art attacks on surrogate classifiers (Inception V3 Szegedy et al. (2016) here as in Xic al. (2019): Dong al. (2019)) the adversarial samples are generated and tested the mAP as the transferability towards detectors. Details of implementation are described in Appendix
We present the results in Table 3 Among the classification attacks and detection ones, cross-domain attack (Naseer et al. (2019) is effective, but RAD is more aggressive. RAD enjoys state-of-the- art transferability towards most black-box models, outperforming other methods for above 20%. The detection mAPs are more than halved, making state-of-the-art detectors worse than the early single-shot detector (SSD Liu et al. (2016). MI).
Table 3: Detection mAP in different attacks
Method
M1
M2
M3
M4
M5
M6
M7
M8
M9
No Attack
29.3
33.4
38.1
40.7
42.1
42.5
45.7
46.9
53,9
Reference
Ablation
24.9
31.4
31.2
31.6
35.0
34.3
37.5
38.8
48.6
PGD
26.4
30.4
34.4
35.4
38.4
38.3
41.7
43.1
51.1
SI-PGD
27.5
31.6
36.1
37.1
40.0
40.1
43.5
44.8
52.4
Classification Attack
MI-DI-PGD
22.9
26.2
29.3
30.0
33.2
32.1
36.0
37.5
48.0
MI-TI-PGD
20.1
23.7
24.9
25.4
30.1
27.4
32.8
34.5
47.1
CD-painting
16.4
20.8
21.3
22.8
26.6
24.5
28.9
29.5
42.3
CD-comics
16.6
21.6
21.7
22.7
26.8
24.3
29.1
42.3
43.7
Dfool
23.3
2.5
29.2
29.8
33.3
32.9
36.5
38.0
47.5
Detection
Loc
21.9
0.2
25.8
26.6
29.8
29.4
33.2
33.2
45.2
Attack
DAG
20.8
0.6
22.8
23.4
26.8
25.6
28.9
31.0
40.6
Ours
RAD
14.6
0.7
16.3
17.0
20.4
19.1
22.3
23.8
35.0

---

Under review as a conference paper at ICLR 2021
The influence of e detection mAP in RAD displayed in Fig. 7 With the foe bound increases, the resulting mAP greatly decreases for all black-box models especially for E from to 12.
M3
M4
M7
detection
MB
30
25
20
15-
4
12
16
20
epsilon
Figure 7: The influence & on detection mAp in RAD
4.4 RAD'S ARILITY TO INSTANCE SEGMENTATION
Detection and segmentation are similar some aspects, so they could be implemented in one network (He et al. )2017): : Cai & Vasconcelos (2018); Chen et al. (2019a) ) Also, adversarial samples for object detection tend transfer to instance segmentation (Xie et al. (2017)) Accordingly, we evaluate this cross-task transferability by RAD on surrogate detectors YOLOv3 (Redmon & Farhadi) (2018), M2), RetinaNet (Lin al. 20177) M3) and Mask R-CNN (He et al. (0117), M5). From the results in Table 4 we find that RAD also greatly hurts the performance of instance segmentation, leading to drop on mAP of over 70%. This inspire the segmentation attackers to indirectly attack detectors.
Table 4: Segmentation mAp of RAD
mAP
mAP50
mAP75
Surrogate
M5
M7
M8
M5
M7
M8
M5
M7
M8
None
38.0
39.4
40.8
60.6
61.3
63.3
40,9
42.9
44.1
Ablation
31.0
31.9
33.5
51.2
51.0
53.7
32.4
34.3
35.4
M2
17.9
18.6
20.3
31.6
31.7
34.5
18.0
18.9
20.7
M3
11.6
11.9
12.9
19.2
19.1
20.7
12.1
12.6
13.7
M5
1.2
11.1
11.8
2.4
17.9
18.9
1.0
11.9
12.6
5 ADVERSARIAL OBJECTS IN CONTEXT
Given the great transferability of RAD, we create Adversarial Objects in COntext (AOCO). the first adversarial dataset for object detection. AOCO dataset serves as potential benchmark evaluate the robustness of detectors, which will be beneficial to network designers. It will also be useful for adversarial training, as the most effective practice to improve the robustness of DNNs Zhang et al.) (2019); Tramer al. (2018). Notice that there is no other adversarial dataset for detection at all. This is not because the dataset is useless, but due to the low transfcrability of attack methods such that the examples are deeeeder-deeemdeett. Now have achieved high transferability and can then make such an adversarial dataset publicly available.
AOCO is generated from the full COCO 2017 validation set (Lin et al. (2014)) with 5k samples. It contains 5K adversarial samples for evaluating object detection (AOCO detection) and 5K for instance segmentation (AOCO segmentation). All 10K samples in AOCO are crafted by RAD. The surrogate
8

---

Under review as a conference paper at ICLR 2021
model attack is YOLOv3 for AOCO detection and Mask R-CNN for AOCO segmentation given the results in Table 3 and Table 4
We measure the perturbation Ar in AOCO by Root Mean Squared Error (RMSE) in (Xie et al. (2117) Liu et al. 20.7)). It is calculated V., (Ax,) / in pixel-wise way, and N is the size of the image. Performance of AOCO is reported T The RMSE AOCO is lower than that in Wu et al. (2019)), and the perturbation is quite imperceptible. Details and samples of AOCO presented in Appendix
Table 5: Detection mAP and segmentation mAP COCO and AOCO
RMSE
M1
M2
M3
M4
M5
M6
M7
M8
M9
COCO detection
0.000
29.3
33.4
38.1
40.7
42.1
42.5
45.7
46.9
53,9
AOCO detection
6.469
14.6
0.7
16.3
17
20.4
19.1
22.3
23.8
35.0
COCO segmentation
0.000
38.0
39.4
40.8
AOCO segmentation
6.606
1.2
11.1
11.8
6
CONCLUSION
To pursue a high transferability, this paper proposes Relevance Attack on Detectors (RAD), which works by suppressing the multi-node relevance, common property across detectors calculated by our Multi-Node SGLRP. We also thoroughly discuss where to attack and the how to update in attacking relevance maps. RAD achieves a state-of-the-ar transferability towards diverse black-box models, exceeding existing results by above 20%, and also significantly hurts the instance segmentation. Given the great transferability of RAD. we generate the first adversarial dataset for object detection, i.c., Adversarial Objects in COntext (AOCO), which helps quickly evaluate and improve the robustness of detectors. Also, attacking other common properties is promising for a good transfcrability.
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthicu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. URL httts:: //ww. tensorf low. org/ Software available from tensorflow.org.
Maximilian Alber, Sebastian Lapuschkin, Philipp Seegerer, Miriam Hägele, Kristof T Schütt, Gré- goire Montavon, Wojciech Samek, Klaus-Robert Müller, Sven Dähne, and Pieter-Jan Kindermans. innvestigate neural networks. Journal of Machine Learning Research, 20- 2019.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7), 2015.
Shumeet Baluja and Ian Fischer. Adversarial transformation networks: Learning to generate adver- sarial examples. arXiv preprint arXiv:1773: 09387, 2017.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision- adversarial attacks: Reliable attacks against black-box machine learning models. 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018.
Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6154-6162 2018. 8.

---

Under review as a conference paper at ICLR 2021
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks, In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57. IEEE, 2017.
Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, et al. Hybrid task cascade for instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4974-4983, 2019a.
Kai Chen, Jiaqi Wang, Jiangmiao Pang. Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng. Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv: / 0006007155, 2019b.
Sizhe Chen, Zhengbao He, Chengjin Sun, and Xiaolin Huang. Universal adversarial attack on attention and the resulting dataset damagenet. arXiv preprint arXiv:2001.06325, 2020.
François Chollet et al. Keras. htttps://kerase
2015.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9185-9193, 2018.
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses transferable adversarial examples by munslation-invarian attacks. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 4312-4321, 2019.
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pp. 3681-3688, 2019.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. STAT, 1050:20, 2015.
Jindong Gu, Yinchong Yang, and Volker Tresp. Understanding individual decisions of cnns via contrastive backpropagation. In Asian Conference on Computer Vision, pp. 119-134. Springer, 2018.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969, 2017.
Lifeng Huang. Chengying Gao, Yuyin Zhou, Cihang Xie, Alan Yuille, Changqing Zou, and Ning Liu. Universal physical camouflage attacks object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 720-729, 2020.
Brian Kenji Iwana, Ryohei Kuroki, and Seiichi Uchida. Explaining convolutional neural networks using softmax gradient layer-wise relevance propagation. In 2019 JEEE/CVF International Conference on Computer Vision Workshops, ICCV Workshops 2019, Seoul, Korea (South), October 27-28, 2019, 2019.
Yuezun Li, Xian Bian, and Siwei Lyu. Attacking object detectors via imperceptible patches on background. CoRR, abs//880.05966, 2018a. 8a.
Yuezun Li, Xiao Bian, Ming-Ching Chang, and Siwei Lyu. Exploring the vulnerability of single shot module in object detectors via imperceptible background patches. In 30th British Machine Vision Conference 2019, BMVC 2019, Cardiff, UK, September 9-12, 2019, 2018b.
Yuezun Li, Daniel Tian. Xiao Bian, Siwei Lyu, et al. Robust adversarial perturbation on deep proposal-based models. 2018c.
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hoperoft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In 8th International Conference on Learning Representations, ICLR, 2020.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference Computer Vision, pp. 740-755. Springer, 2014.
10

---

Under review as a conference paper at ICLR 2021
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017.
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European Conference Computer Vision, pp. 21-37. Springer, 2016.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In Proceedings of Sth International Conference on Learning Representations, 2017.
Jiajun Lu, Hussein Sibai, and Evan Fabry. Adversarial examples that fool detectors. arXiv preprint atXh::111..22494, 2017.
2017.
Aleksander Mądry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. STAT, 1050:9, 2017.
Muhammad Muzammal Naseer, Salman H Khan, Muhammad Haris Khan, Fahad Shahbaz Khan, and Fatih Porikli. Cross-domain transferability of adversarial perturbations. In Advances in Neural Information Processing Systems, pp. 12905-12915, 2019.
Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint orXiv:1605.07277 2016.
Nicolas Papernot, Patrick McDanicl, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506-519, ACM, 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, pp. 8024-8035, 2019.
Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arX- iv: 18:4000.07767, 2018.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, pp. 99-99, 2015.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 618 -2-62, 2017.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 3145-3153. JMLR. org, 2017.
Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy? ?-a comprehensive study on the robustness 18 deep image classification models. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
11

---

Under review as a conference paper at ICLR 2021
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-2826, 2016.
Mingxing Tan, Ruoming Pang, and Quoc V Le. Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 10781-10790, 2020.
Simen Thys, Wiebe Van Ranst, and Toon Goedemé. Fooling automated surveillance cameras: adversarial patches attack person detection. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition Workshops, 2019.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, lan Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 2018.
Yajie Wang, Yu-an Tan, Wenjiao Zhang, Yuhang Zhao, and Xiaohui Kuang. An adversarial attack on dnn-based black-box object detectors. Journal of Network and Computer Applications, pp. 102634, 2020.
Dongxian Wu, Yisen Wang. Shutao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In Proceedings of the International Conference on Learning Representations, 2019.
Weibin Wu, Yuxin Su, Xixian Chen, Shenglin Zhao, Irwin King, Michael R Lyu, and Yu- Yu-Wing Tai. Boosting the transferability of adversarial samples via attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1161-1170, 2020a.
Zuxuan Wu, Ser-Nam Lim, Larry S Davis, and Tom Goldstein. Making an invisibility cloak: Real world adversarial attacks on object detectors. In European Conference on Computer Vision, pp. 1-17. Springer, 2020b.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for semantic segmentation and object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1369-1378, 2017.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pp. 2730-2739, 2019.
Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin- Yu Chen, Yanzhi Wang, and Xue Lin. Adversarial t-shirt! evading person detectors in physical world. In European Conference on Computer Vision, pp. 665-681. Springer, 2020.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision, pp. 818-88 833. Springer, 2014.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Painless adversarial training using maximal principle. arXiv preprint arXiv: 190500000000077 2019.
Haichao Zhang and Jianyu Wang. Towards adversarially robust object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 421-430, 2019.
Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA. pp. 7502-7511, 2019.
Xinyang Zhang, Ningfei Wang, Hua Shen, Shouling Ji, Xiapu Luo, and Ting Wang. Interpretable deep learning under fire. In 29th {USENIX) Security Symposium (!USENIX) Security 20), 2020.
12

---

Under review as a conference paper at ICLR 2021
A RELEVANCE BACK-PROPAGATION RULES (DTD)
DTD-based network visualization methods, such as LRP, CLRP and SGLRP, back-propagate the relevance from the output layer to the input layer according to the rules specified in this section. Their only difference is the relevance the initial output layer R(!
For each layer / in a DNN with L layers in total, suppose layer has N nodes and layer / + 1 has M nodes, the relevance R())) node in layer is defined recursively by
(1) +(1)
R
On Wn.m
(((+1)
T En nn W. +()) n'm
for nodes with definite positive values (such as after ReLU), and
(1)
(1) +1))
(1) (1)
R Σ
Zn Wm.m
Wn,m hn Wn.m R(++1)
(1) En 2xd W b Wn'm
+(1)
(1) Ryd na.m
for nodes that may have negative values. In the formulas above, an is the post-activation output of node n layer and 2 is the pre-activation one. The range [b0]
1 stands for the minimum and
(!) maximum of 2n Finally, Wn,m max
+(1)
-(t) Winmm 0) and Wn.m min (1)
(1)
Wn.m
According to the propagation rules above as mentioned in (Iwana et al. (2019)), we could naturally obtain the attack gradients
all (0
-1
(
(
En (9 t0
for nodes with definite positive values
aR!!!
(2) (i)
6 w+(0
(19 w-
otherwise
En 600 w +0)
B MODEL INFORMATION
Table
6 presents the models' information from evaluation and MMdetection (Chen et al. (019b))
Table 6: Model backbone and mAPs
ID
Model
Backbone
mAP
MI
SSD512 (Liu et al. (2016))
VGGT6
29.3
M2
YOLOv3 Redmon & Farhadı (2018)
Darknet
33.4
M3
RetinaNct (Lin et al. (017))
ResNet-101
38.1
M4
Faster R-CNN (Ren et al. (2015))
ResNeXt-101-64444
40.7
M5
Mask R-CNN (He al. ((017)
ResNexX-100164444
42.1
M6
Cascade RCNN (Cai & Vasconcelos (0018)
ResNet-101
42.5
M7
Cascade Mask R-CNN (Car & Vasconcelos (2018
ResNext-101-64*44
45.7
M8
Hyrbrid Task Cascade (Chen et al. (2019))
ResNextt1001-64%4
46,9
M9
EfficientDet (Tan al. (2020))
EfficientNet BiFPN BiFPN
53.9
C INFLUENCE OF HYPER-PARAMIN PARAMETERS IN NODE SELECTION
Performance of RAD is not sensitive hyper-parre parameter, matter the strategy to select bounding boxes dynamic or static as Table Attackers are not bothered to tune them carefully. The parameter for dynamic strategy refers to the pre-softmax confidence threshold to select bounding box. The parameter for static strategy refers to the fixed number of selected bounding boxes in each iteration.
D RAD ON MORE SURROGATES
The results attacking surrogates by RAD reported in Table
13

---

Under review as a conference paper at ICLR 2021
Table 7: Detection mAP in different hyper-parameters in RAD
Strategy
Parameter
M1
M2
M3
M4
M5
M6
M7
M8
M9
-1
18.3
1.2
20.0
20.6
24.3
22.8
26.7
28.3
40.0
Dynamic
-2
18.2
1.2
20.1
20.7
24.2
22.8
26.6
28.0
39.8
-3
18.4
1.3
20.3
20.8
24.3
22.9
26.7
28.5
40.2
10
18.2
1.1
19.9
20.5
24.2
22.8
26.2
28.0
39.8
Static
20
18.1
1.2
19.9
20.5
24.3
22.6
26.4
28.2
39.9
30
18.2
1.3
20.1
20.7
24.3
22.8
26.0
27.9
40.1
Table 8: Detection mAP of RAD on different surrogates
Surrogate
M1
M2
M3
M4
M5
M6
M7
M8
M9
YOLOv3
14.6
0.7
16.3
17.0
20.4
19.1
22.3
23.8
35.0
Mask R-CNN
20.4
24.2
25.7
26.5
1.1
28.9
33.1
34.9
45,4
RetinaNet
20.7
6.1
2.3
25.7
29.3
28.2
31.7
33.8
44.1
E IMPLEMENTATION DETAILS
E. PRE-PROCSSIING
To pre-process, resize the image with its long side as 416 for YOLOv3 or RetinaNet and 448 for Mask R-CNN, and then zero-pad it to a square. The resolution kept relatively the same for fair evaluation. Images normalized [0,1] in YOLOv3 or subtracted by the mean of COCO training set in RetinaNet and Mask R-CNN. Accordingly, samples in AOCO detection have the long side 416 and that for AOCO segmentation is 448.
E.2 TRANSFFER-ENHNCCEE UPDATE TECHNIQUES
DI (Xie et al. (2019)) transforms the image for 4 times with probability (p for better trans- ferability as suggested) and averaging the gradients. The transformation is to resize the image to 0.9x its size and randomly padding the outer areas with white pixels. SI (Lin et al. (2020)) divides the sample numerically by the power for times and averages the 4 obtained gradients. TI (Dong al. (2019) translates the image to calculate the augmented gradients. To implement it efficiently, adopts kernel to simulate the averaging of gradients. We choose the kernel size 15 as suggested. MI (Dong et al. (018)) uses momentum optimization (parameter μ 1 as suggested) for better transferability and a faster attack. Cross-domain attack (Naseer et al. (2019)) uses extra datasets (paintings, denoted CD-paintings, and comics, denoted CD-comics) train perturbation generator with the relative loss. The adopted surrogate model is also Inception for consistency. All perturbations are resized to fit the sample size.
E.3 DETECTION ATTACKS
For DAG (Xie et al. (207)). ) we follow the setting of generating dense proposals. The classifica- tion probabilities of 3000 bounding boxes with highest confidence are attacked. But we alter its optimization to (5) because its original update produces quite small perturbation, leading to a poor transferability, which unfair for comparison. Dfool (Lu et al. (2017) suppresses the classification experiment. Localization loss
confidence for the original bounding boxes, which is the same shown to useful in (Zhang & Wang (2019)) and here we suppress the width and height of the original bounding boxes.
F VISUAL RESULTS OF RAD PROCESS
By RAD, the relevance map is attacked be meaningless and loss its focus. In Fig. 8 the initial prediction correct and the relevance map is clear. RAD constantly misleads the relevance map to be unstructured without outline of objects. Finally, all bounding boxes vanish.
14

---

Under review as a conference paper at ICLR 2021
Ventura
Ventera
3.75 SINEE
Ventura
201
Figure 8: Transition of prediction and relevance map in RAD (from top to bottom and left to right).
G MORE ABOUT AOCO
We report the mAP50 and mAP75 of AOCO in Table 9 and Table TO comparative results.
We show in Fig. 9 the visual
Table Detection mAP50 and segmentation mAP50 on COCO and AOCO
M1
M2
M3
M4
M5
M6
M7
M8
M9
COCO detection
49.2
56.4
58.1
62.0
63,8
60.7
64.1
66.0
74.3
AOCO detection
26.7
1.6
27.6
29.1
34.5
29.9
34.3
37.4
51.7
COCO segmentation
60.6
61.3
63.3
AOCO segmentation
2.4
17.9
18.9
Table 10: Detection mAP75 and segmentation mAP75 COCO and AOCO
M1
M2
M3
M4
M5
M6
M7
M8
M9
COCO detection
30.8
35.8
40.6
44.6
46.3
46.3
50.0
51.2
59.9
AOCO detection
14.2
0.6
16.5
17.1
20.8
19.9
23.3
24.6
37.4
COCO segmentation
40.9
42.9
44.1
AOCO segmentation
1.0
11.9
12.6
15

---

Under review as a conference paper at ICLR 2021
COCO detection
AOCO detection
COCO segmentation
AOCO segmentation
Figure 9: Detection and segmentation results in COCO and AOCO by YOLOv3 and Mask R-CNN. For COCO, both networks predict correctly. For AOCO segmentation results, the top image contains two big masks for "chair" and "potted plan"; the second image contains one false mask for "sports ball"; the bottom image contains "dog" in green, "car" in purple and "elephant" in red.
16


Reviews:
[{"review_id": "TJmcGcoKjnY", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a method for adversarial attacks on object detectors by exploiting relevance maps that are originally intended for model interpretation. Unlike most of the existing methods that attack detection scores directly, the proposed approach focuses on suppressing the relevance map associated with target objects by image perturbation. The idea is interesting and demonstrates good transferability on the tasks of object detection and segmentation. \n\nThe paper is mostly well written and easy to follow. The adversarial object dataset can also be helpful to the research community.\n\nThe main downside of the paper is that some of the comparisons are not apple-to-apple in the experiments. For example, the proposed approach applies update techniques (i.e. Translation-Invariant) to improve transferability. However, it is not my impression that this was done on the baseline methods, which leads to unfair comparison.\n\nSome technical details need to be better articulated in the paper. For example, there is no mentioning of the adversarial loss function and how it is optimized. As pointed out in [2], attacking CNN interpretations is not trivial. Without the details of how to update the gradients of the relevance map, It would make reproducibility difficult.\n\nWhile focusing on a different problem,  the proposed approach shares some similarities with methods designed to attack model interpretations such as [1] and [2], which should be discussed as related work.\n\n[1] Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 3681–3688, 2019\n[2] Xinyang Zhang, Ningfei Wang  Hua Shen, Shouling Ji, Xiapu Luo,Ting Wang, Interpretable Deep Learning under Fire, USENIX Security '20\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "M1non0gR8rB", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe paper found a new way, called Relevance Attack on Detectors (RAD),  to generate high transferability adversarial examples against detectors by suppressing the multi-node relevance.\nMoreover, a dataset generated by this attacking method is introduced.\n\nPros:\n1. The designed RAD is new and technically sound.\n2. The experimental results on multiple detectors, like YOLOv3,  RetinaNet, Mask R-CNN, etc. are promising.\n\nCons:\n1. It's not surprising that the object detectors can be attacked together.  One naive way is that using the ensemble attack by average the negative CE loss like the paper \"Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors\" proposed. So I think some baselines need to be added in experiments. \n2.  By imposing diversity transformation on input image can also improve transferability (\"Boosting Adversarial Attacks with Momentum\"). Similar rules also can be applied in detectors.\n3. I cannot see a very significant value of the generated dataset. Like in the image classification dataset, one can generate adversarial examples easily, but there is not an individual dataset that only contains generated adversarial examples generated by one specific method. Some famous dataset like imagenet-C (\"Benchmarking Neural Network Robustness to Common Corruptions and Perturbations\") is designed by natural perturbations.", "extra_scores": {}, "extra_text": {}}, {"review_id": "F-pJXaTIQb2", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**:\nThis work proposes to attack object detectors by targeting their relevance maps of the different detected objects. The proposed RAD attack shows better black box transferability across different detectors on MSCOCO dataset. The relevance maps are calculated based on SGLRP act as an attention mechanism to the attack to focus on relevant regions in the more meaningful image and hence produce more transferable attacks. \n\n**Strengths** :\n- Good attack performance and transferability between detectors, which poses a security threat for SDV applications that use object detectors\n- Eight different detectors and three segmentation models are used in the RAD attack, which shows good generalization.\n\n**Weaknesses**:\n- Missing important references [a,b,c]. All of these works attack object detectors and target transferability.\n- The paper is poorly written and ambiguous. Variables are introduced without proper definitions. It is not clear how to obtain the gradients in eq(3) with respect to the relevance maps.\n- No use of the proposed dataset. The authors propose a new dataset of adversarial objects but never mention or showcase the dataset's usefulness. A straightforward way to show the dataset's usefulness is by performing adversarial training and making robust detectors against the proposed attacks. \n- No enough ablation is performed. The only ablation to the proposed method is in table 7 regarding the way to pick the detection target. The relevance maps based on LRP are expensive and worse than recent saliency maps like CAM and grad-CAM. The attack budget $\\epsilon =16$ picked in the experiments is not justified ( it might be big or small for attack success ), and a plot of mAP vs. $\\epsilon$ for different detectors would give more information about the effect of the attack.\n- All the attacks in the paper are performed on YOLOv3 and transferred to other models. It would be more informative to show transferability matrices of attacks performed on all models and transferred to all others.\n- The novelty of the proposed methodology is limited. While the use of relevance maps to improve the transferability of attacks on object detectors is novel, no proper explanation is provided. The attacks are based on PGD, and the relevance map is adapted from SGLRP. The paper offers no theoretical results or exciting insights. \n\n\n\n[a] Huang et al. \"Universal Physical Camouflage Attacks on Object Detectors\", ( CVPR 2020)\n[b] Wu et al. \"Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors\", (ECCV 2020 )\n[c] Xu et al. \"Adversarial T-shirt! Evading Person Detectors in A Physical World\" (ECCV 2020).\n\n\nMinor issues :\n- Many grammar mistakes:\" because they possess multiple-output.\", \"Among the classification attacks and detection ones, cross-domain attack (Naseer et al. (2019)) is the most effective, but RAD is more aggressive\" ..etc.\n- No question marks in titles 3.1-3.5.\n- Table 2-5 could have been visualized better by using a bar chart, for example, to observe the relative performance of attacks and defenses. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Adlnb8Kbl2o", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "A transferable adversarial attack method is proposed for object detection. A relevance map is used to discover the common weakness of existing detectors. An adversarial dataset for object detection is created for experimental validation.\n\nThe relevance map proposed indeed derives from SGLRP as mentioned in Sec. 3.3. The only modification is to change a single node target t to the average of a set of nodes.  As the major contribution claimed is on the relevance map, the minor modification makes the contribution limited.\n\nThe relevance map is computed by back-propagating the relevance R from the final layer to the input following rules as illustrated in Sec. 3.3. However, the proposed method is claimed to perform a black-box transferable attack. Also, the gradients are utilized to update the relevance map. It is thus not clear how this relevance map correlates to the black-box transferable attack.  \n\nIn the experiments, the proposed attack method shall be compared to sota detection or segmentation-based methods (e.g., EfficientDet, Cascade-RCNN, Libra-RCNN). \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a transferable adversarial attack method for object detection by using the relevance map. Four reviewers provided detailed reviews: 2 of them rated “Ok but not good enough - rejection”, 1 rated “Marginally below” and 1 rated “Marginally above”. While reviewers consider the paper well written and using relevance map novel, a number of concerns are raised, including limited novelty, the lack of theoretical results, no use of the proposed dataset, insufficient ablation, etc. During the rebuttal, the authors made efforts to response to all reviewers’ comments. However, the major concerns remain, and the rating were not changed. The ACs concur these major concerns and agree that the paper can not be accepted at its current state.
Decision: Reject