What's Wrong with the Robustness of Object Detectors?
Authors: ZiYi Dong
Pengxu Wei
Liang Lin
Venue: Submitted to ICLR 2023 (2023)
Keywords: Object Detection
Adversarial Robustness

Abstract: Despite tremendous successes achieved, object detection models confront the vulnerability to adversarial attacks. Even with imperceptible adversarial perturbations in images, they probably yield erroneous detection predictions, posing a threat to various realistic applications, e.g., medical diagnosis and automatic driving. Although some existing methods can improve the adversarial robustness of detectors, they still suffer from the detection robustness bottleneck: the significant performance degradation on clean images and the limited robustness on adversarial images. In this paper, we conduct empirically a comprehensive investigation on what's wrong with the robustness of object detectors in four different seminal architectures, i.e., two-stage, one-stage, anchor-free, and Transformer-based detectors, inspiring more research interest on this task. We also devise a Detection Confusion Matrix (DCM) and Classification-Ablative Validation (ClsAVal) for further detection robustness analyses. We explore underlying factors that account for robustness bottleneck. It is empirically demonstrated that robust detectors have reliable localization robustness and poor classification robustness. The classification module easily mis-classifies the foreground objects into the background. Furthermore, Robust Derformable-DETR suffers from a poor classification and localization robustness. Our source codes, trained models, and detailed experiment results will be publicly available.

Under review as a conference paper at ICLR 2023
WHAT'S WRONG WITH THE ROBUSTNESS OF OBJECT DETECTORS?
Anonymous authors Paper under double-blind review
ABSTRACT
Despite tremendous successes achieved, object detection models confront the vul- nerability to adversarial attacks. Even with imperceptible adversarial perturbations in images, they probably yield erroneous detection predictions, posing threat to various realistic applications, c.g., medical diagnosis and automatic driving. Although some existing methods can improve the adversarial robustness of de- tectors, they still suffer from the detection robustness bottleneck: the significant performance degradation on clean images and the limited robustness adversarial images. In this paper, we conduct empirically a comprehensive investigation on what's wrong with the robustness of object detectors four different seminal architectures, i.e., two-stage, one-stage, anchor- fteee--ree, and Transformer-base de- tectors, inspiring more research interest on this task. We also devise Detection Confusion Matrix (DCM) and Plassiiiationn-Alaaii further detection robustness analyses. We explore underlying factors that account for robustness bottleneck. It is empirically demonstrated that robust detectors have reliable localization robustness and poor classification robustness. The classifi- cation module easily mis-classifies the foreground objects into the background. Furthermore, Robust Derformaale-DErT localization robustness. Our source codes, trained models, and detailed experiment results will be publicly available.
Validation (CIsAVal) for
suffers from poor classification and
INTRODUCTION
With numerous breakthroughs in recent years, deep neural networks (DNNs) have built series of milestones the computer vision community HHe al. (2016); Chen et (201 (2018a): Gi al.) (2021). Nevertheless, with millions of model parameters, they are verified to be easily fooled to generate completely wrong predictions under slight and imperceptible image perturbations Szegedy cet al. (2014). Many recent works are devoted to exploring the model robustness with adversarial perturbations crafted by attack models including FGSM Goodfellow et al. (2015), PGD Madry et al. (2018), AdvGAN Xiao al. (2018), Carlini and Wagner Attack (C&W) Carlini & Waanner (2017). However, those mainly focus on the image classification task. The robustness of object detectors is quite under-explored.
Object detectors not only identify which categories objects belong to (classification), but also recognize where objects exactly are (localization). Inevitably, detection models suffer from more complex and challenging adversarial robustness from classification and localization, exposing more possibilities of being attacked. A few recent approaches Xie al. (2017); (eei:::Wiiee al. (019): Sarkar
et al. (2017) Chen et al. 2018b) have been proposed to attack the -fate-of-the-art object detectors. Though those methods achieve successful attacks on Faster RCNN and SSD object detectors, is not trivial to defense attacks to ensure the adversarial robustness for object detectors. Generally, few object detectors. Three early attempts are MTD Zhang Wang (2019). CAWT Chen al. (2021) and RobustDet Dong et al. (2022), which are derived from the one-stage SSD detector.
works are devoted to adversarially-roohae
Nevertheless, existing robust detection methods present typical detection robustness bottleneck on both clean images and adversarial images for object detection: a significant performance decline adversarial images. As demonstrated in Fig. [[]
on clean images and the limited performance on one hand, robust object detectors (MTD and CWAT) only obtain about 22 22%~376 % mAP on adversarial images! On the other hand, robust object detectors (MTD and CWAT) suffer from the

---

Under review as a conference paper at ICLR 2023
100
(ccuracy(5)
mAP(4)
96.34
9223
#TT
STD
90
8492
TRADES
90
MTD
80
IRGD
80
77.49
70
66.58
70
60
53.08
60
51.30
50
48,00
40
40
36.70
29.10
31.90
30
30
22.40
20
20
10
Q12
10
1.82
Clean images Adversarial images Adversarial images
Clean images
Adversarial images
(Aw)
(Awe)
(a) Classification Robustness
(b) Object Detection Robustness
Figure 1: Robustness for image classification models and object detection models. "STD" is the standard
model (non-robust), i.e., ResNet in classification and SSD in detection. Ast. and Alee denote the attacks for classification and localization in detection.
performance decrease by nearly 30 mAP on clean images (77.49% mAP for standard SSD vs. 48% mAP for MTD, 51.3% mAP for CWAT)! Instead, this is not an intuitive phenomenon, because the robust classification models (e.g., TRADES Zhang et al. (2019), IRGD Gowal et al. (202I)) have small amount of the performance decline on clean images while gaining the robustness, as shown in Fig. [1] A same thing for those classification and detection methods is adversarial training to ensure the robustness, but this problem occurs notably in object detection. Therefore, it is worthy of more attention to exploring what's wrong with the robustness of object detectors.
In this paper, rather than proposing specific solution for robust object detectors usual, we specifi- cally aim to make an initial attempt to investigate the robustness of object detectors systematically and comprehensively as possible we can, paving way for future works that are inspired to essentially mitigate the detection robustness bottleneck. The involved object detectors cover four seminal object detectors with different structures, i.e., the two-stage, one-stage and anchor-free and Transformer-bant based detectors. Specifically, we devise novel Detection Confusion Matrix (DCM) in object detection analyze the classification and localization detail and Classifcction-AAlaas Validation (CIsAVal) for detection robustness analysis. Four robust object detectors with adversarial training present the similar robustness bottleneck aforementioned, which is mainly attributed to the inferior classification robustness, i.e., the mnisclassification of the foreground as the background in the classification module, with less confusion among foreground categories. Besides, in Deformable- DETR, both the robustness of classification and localization is poor, unlike SSD, Faster RCNN, and YOLOX with reliable localization robustness and poor classification robustness.
ADVERSARIAL ROBUSTNESS OF OBJECT DETECTORS
With the remarkable learning capability of deep neural networks, three representative series of deep learning based object detectors are prevalent, leading the recent research on object detection. They are two-stage, one-stage, anchor-free and Transformer-bast
object detectors. In our work, we will empirically investigate the adversarial robustness of those three types of detectors. Concretely, Faster RCNN Ren et al.] (2015) (Iwo-slage), SSD Liu et al. (2016) (one-stuge), YOLOX (.e et al.) (2021) (anchor-free) and Deformable-DEri Zhu et al. (2021) (Transformer-bused) are selected.
this section, adversarial training for robust object detection will be firstly described and then empirical analyses on detection robustness will be claborated. For SSD, Faster RCNN and YOLOX. we use PASCAL VOC Everingham al. (2055) dataset for analysis. For Deformable-DETR. MS- AS-COCOLLi et al. (014)) is adopted, since is unable to successfully train Deformable-DEr DEER on PASCAL VOC using the official code. (However, we still provide successful training results on VOC dataset using MMDetection codes in the supplementary material.)
2. PRELIMINARIES
Object detection can be regarded as a multi-task learning for classification and localization. Formally, detection model f is parameterized by 0, which consists of backbone fo and two headers of classification H.I. and localization Hine- Given an input image x E D, two headers yield probabilistic confidence and the predicted localization for each bounding box, respectively. After Non-Maximum
2

---

Under review as a conference paper at ICLR 2023
90
90
83.56
IVOCI
80
Faster R-CNK (VOC)
80
7744
Faster &-CWN (VOC)
YoLox (MC))
YOLOX [VOC)
Defomabie-DETR (cOCO)
Deformable DETA (C0CO)
64.17
60
50.25
60
59.70
53,65
50
50
40 00
Non-Robust
Robust
30
30
Robust
34.00
28.10
31.90
20
13.70
20
1408
20.20
16.13
10
clean
Ase
Abe
clean
clean
Adr
Abc
(a) Performance of non-robust detectors
(b) Performance of robust detectors
Figure 2: Performance of non-robust and robust detectors with different architectures on clean images and adversarial images. (Best viewed zoomed color.)
Suppression (NMS) and filtering the object detector ff outputs detected bounding boxes bg (k 1, N) with the confidence 8% E RC+1 over C' object categories and the background. In the training phase, the objective function integrates the classification loss Lats and localization loss Ltoe to be optimizedly minimized:
min (v..)))
Hato (f)))))))) (v). Lioe = (foe(f()))), b,)). (1)
where ys is the category label of Bhound-Truuth (GT) bounding box b;-
Existing attack methods for object detectors essentially cast attacking by using variants of
individual classification or localization task losses or their combinations Xie et al. (2017); Wei et al.)
:2019;(Liu et al. (2019); Chen al. (2018b). Following those detection attack works, denote the attack against the classification as Adt and the attack against the localization as Atoes respectively.
Acts(z) arg max Lets (H (z)), (ss, b,)), BES=
Atoo arg max Ltoe H (z)), (y: b:)), #ES,
where is the adversarial image of x and Sg = f n [0, 255] image space centered on clean images x with perturbation budget c = 8. Accordingly, the objective
| x x1/00 S c) } is the adversarial
function for robust object detection is formulated as
min Exz, (ur.d, D-aas (fo((())))))) (v: b;}) (Ho
(vt, b.)).
where represents the attack in the adversarial training. Ideally, a robust object detector should not only have promising localization robustness and classification robustness, but also present remarkable performance on clean images.
2.2 EMPIRICAL ANALYSES ON DETECTION ROBUSTNESS
In this section, empirically explore the adversarial robustness for four series of otfte-of-theeart detection models. Our attack in the adversarial training involves simultaneously classification and localization attacks this paper, e.g., MTD Wang (209)). We adopt the similar experimental setup of attacks and criteria as MTD. We evaluate detection models under the attack Late and Lioc using the PGD-20 (e 8), and use the mean Average Precision (mAP) as the robustness evaluation metric. The performance evaluation on the non-robust and robust models with different detection structures is shown in Fig.
I) Non-robust models clean and adversarial images: The standard non-robust models have remarkable detection performance over 60% mAP on clean images. But, all of them have extremely poor results on adversarial attacks which is lower than 10% mAP.
2) Robust models on cleun and adversarial images: Robust detectors have limited robustness to adversarial attacks achieving lower than 40% mAP in the best case and only 10% in the worst case. Meanwhile, they have a performance decrease on clean images compared with non-robust models.
3) Robust models vs. non-robust models clean images: Compared with the non-robust model, robust detectors suffer from a significant degradation by 15% mAP drops on average, and even by up to 30% most. Deformable-Der DETR also has about 25% mAP drop COCO.
It specifically represents the process of filtering bounding boxes whose scores large than the threshold for final detection results in detectors.
3

---

Those detected bounding boxes after Non-maximum Suppression (NMS) are considered for calculat- ing DCM. Those whose IOU with all the GT' is lower than Tio are regarded as its true category being assigned to background. Otherwise, the true category of a detected bounding box b, can be assigned to a certain object category. Two is the IOU acceptable threshold (it is 0.5 in this paper). calculate Under review as a conference paper at ICLR 2023
Delection (ofter
Dow2 Aut by e Bl
M DAG
Densily
YOU
hist (g:Wouum))
(1007)4
DCM
by ((),4)) d2 8(a.sa)
Figure 3: Illustraioon of Detection Confusion Matrix.
4) Robust models vs. non-robust models on adversarial images: Robust models can maintain a certain detection ability on adversarial images which has about 20% mAP improvement, while non-robust models hardly detect correctly objects on adversarial images (lower than 10% mAP).
5) Robust models resistant classification attack and localization attack:
The performance Aets adversarial image is generally lower than those of Atoc. indicating the robustness of the classification module is commonly weaker than that of the localization module. Compared with Faster R-CNN, SSD has a smaller robustness gap between the classification and localization modules. The performance of YOLOX on clean images is better than SSD, but its robustness on the classification module weaker than that of SSD. The robustness gap of YOLOX between the classification and localization module is similar to the two-stage detector.
WHAT'S WRONG WITH THE ROBUSTNESS OF OBJECT DETECTORS?
In this section, we first devise Detection Confusion Matrix and Clasiffcation-Ablatie Validation in object detection for detailed analyses, and then mainly analyze the robustness in classification and localization, taking one-stage SSD for example.
3.1 PRELIMINARIES
Detection Confusion Matrix (DCM). Considering that adversarial images would be easily mis- identified as any other category. it matters that we figure out what the detection confusion is among different object categories due adversarial attacks. Accordingly, we resort to the confusion matrix which intuitively indicates the detection performance for each category and the confusion between different categories. However, what is responsible for the detection performance evaluation involves classification categories, classification confidences, and the regression localization of bounding boxes. Thus, the conventional confusion matrix for classification improperly suits object detection. In our work, we devise Detection Confusion Matrix to diagnose object detection, which indicates the confusion of detection results among different categories and their localization.
Once the matching between GT and the predicted bounding box is determined, DCM via this information, shown in Fig.
Specifically, detection confusion matrix M has the size of (1 + +C) x (1 + C),, whose items named cells. In each cell, simultaneously contains the category confusion and the IOU distribution. 1) For the category confusion, in cell (m,J),, M represents the sum of confidences of the bounding boxes whose true category is Gn but predicted to be category Ca- As per routine of mAP, we assign the true bounding box and the true category to the predicted bounding box based its IOU value with GT boxes. All the GT bounding boxes in an image are denoted as B and all the predicted bounding boxes as B. b, E B is the GT bounding box with the largest IOU with predicted bounding box b B. If TOU(b.. b.) >= Timu. then by matches with b, and the true category of be is also the category of b;; otherwise, by does not match any GT bounding box and its true category is Cby- Thus the set of all predicted bounding boxes with the true category en can be indicated as: B(Cm). then the category confusion is defined as: M = E bb, 8k, where B(cm, Cn) C B(cm)
is the subset of B(cm) whose predicted category Cn. but the true category is Cmn-
2) For the IOU distribution, in cell (m,n), Miny, represents the distribution of IOU values for the bounding boxes whose true category is c. but predicted to c,,, this distribution, r-axis is the IOU value, and y-axis is the percentage of the number of bounding boxes in the corresponding interval. It
4

---

Under review as a conference paper at ICLR 2023
Sangle 8.8)
te (.7))
anchore
NIM
anchhrrs
M
NMS
Fp
93
9p
NMS
predicted boxes
idee
conhhers
botg
anchors
Ride
X
ancherr
Ag
fn
bg
X
hn.
M
NMS
amchory
(()
ide
X
id
ClsAVal
ancchorg
dozg
anchors
Rall
X Kens
Labehorr
M
NMS
anchorg
box)
X
(*) (2)
Figure 4: PlassifcaatinnAbllae Validation. Based on the conventional detection process, the output of the current model classification module is replaced with the output of reliable model classification module, to eliminate the impact of the classification module and evaluate the localization module performance individually.
(IOU bi)), where b, is the GT bounding box matching is defined as M = hist b. (i.
by and hist(-) is the histogram operation.
Classification-AA blative Validation (CIsAVal). To independently evaluate the robustness of the localization module, we propose the ClsAVal method. To eliminate the impact of the classification module on the results, it assumes that each proposal has reliable classification score (this can be provided by reliable detector).
Given a detection model M, its detected object set is Ozz). After applying NMS and filtering, the reserved boxes with their indexes idæ will be recorded: (xx) = {(bb, 8k), k E idx). To diagnose the localization robustness of M, it resorts to another model M whose performance on & is better than M x and its detected object set is (*(x)
{(ff. *f), & E idr#), where x and à from the same image but with or without certain manipulation (e.g. adversarial perturbation). Two validation modes are defined, shown in Fig. 1) ClsAVal Rude: directly use the indexes idz of & to select the bounding boxes on x: RR (x) [(b,. sk) E idz At this time, the selected boxes are assigned with correct indexes for positions, but their scores are still from M. This indicates what scores M predicts for the boxes in those correct positions. 2) ClsAVal Rant If the selected boxes are assigned with correct indexes for positions and meanwhile their scores are also from M on é, then the selected bounding boxes on are OR R.., (2) {(bk, { ( id eliminate the interference of the classification module. ClsAVal Rull reflects the localization predicted by M for those that are correct positions. Suppose å and M are the clean image and the standard model, respectively; and M are their adversarial and robust counterparts, respectively. If M produces similar boxes to M R, means that M is robust in localization; otherwise, it non-robust localization.
3.2 ROBUSTNESS IN CLASSIFICATION AND LOCALIZATION
As demonstrated in Sec. 2.2 robust detection models have limited robustness on adversarial images, and degenerate significantly on clean images. That is, existing robust object detectors often have an unsatisfactory performance on both adversarial and clean images. One reason likely that robust models have difficulties to collaboratively learning clean images and adversarial images together. That indicates that those models have serious conflicts between learning clean images and adversarial images in the adversarial training. Thus, they have to compromise trade-off between the accuracy (on clean images) and adversarial robustness (on adversarial images), leading to the performance sacrifice, especially clean images. We will elaborately investigate this problem in the following.
(a) Are robust detectors really poor in localization?
Compared with image classification, object detection models has one more regression branch for localizing objects. Accordingly, problem arises: does the conflict in object detection mainly exist in the regression subtask for object localization? ? We will first explore whether the localization the "chief criminal" that is responsible for the poor robustness in detection. Accordingly, it is necessary to eliminate the interference of the classification module to verify the robustness of the localization module independently. Two possible cases can be considered to explain this issue.
1) The error of the localization result may be due to the poor robustness of the localization module itself. which gives the wrong bounding boxes.
5

---

Under review as a conference paper at ICLR 2023
STD
Robust
c.ann
Avs
clean
Figure 5: Robustness analyses with Detection Confusion Matrix for standard and robust SSD model
Clean
RasiAal
RailOul
RadAn
ST
Vanils
Vanila
8
Vanilo
Rulast Roburtkkl
RobantiA Robust/
Robuzthek
Robusihea
(a) ClsAval for SSD
(b) ClsAval for Faster R-CNN
(c) ClsAval for RPN
Figure 6: (a)(b): Classffccatio
Validation for standard and robust SSD and Faster R-CNN
models. "Vanilla" means the standard outputs of the model. (c): Aiasificaaion-Ablaiii Validation
for RPN in standard and robust Faster R-CNN.
The classification module yields wrong scores so that the NMS module selects the bounding boxes at wrong positions that do not contain correct objects.
In order figure out which factor dominates the error of prediction results, we verify the localization robustness via the proposed ClsAVal. This has a premise that standard detectors can select bounding boxes at correct positions on clean images. shown Fig. []), the performance of robust SSD) under adversarial attacks is limited: not only the confidence for many objects are quite low, but the bounding box also has large deviation. However, via Ratz validation, under either classification attack Aets or localization attack Aioc. those boxes (produced by robust models) at correct positions do not deviate much from the detected boxes on clean images by standard SSD. But the confidences for these bounding boxes that should be correct are extremely low and thus wrongly recognized as the background. Compared with Ride validation results of the standard SSD model under Atoes can be seen that the localization module of the detector has indeed gained good robustness through adversarial training. It means that the model is not due to the poor robustness of the localization module itself, but the classification module with poor robustness that gives the wrong confidence of the bounding box, which makes the subsequent module select wrong bounding boxes.
(b) What's wrong with the classification robustness?
To analyze the classification robustness, employ the Detection Confusion Matrix inspect the detection results of robust models. This aims to not only summarize the classification and localization performance for each category but also analyze how confused the detector is for classification and localization among all the categories. Fig. 5) prodives DCM results for standard SSD and robust SSD.
The adversarial attack enforces the standard detection model to easily mis-classify foreground objects into the background.
The robust model trained by adversarial training also tends mis-classify fiisclaasiyfooreruunnd objects into the background for detection on both clean images and adversarial images.
In Fig. 15, compared the results of the standard model on the clean images, standard and robust models on Acts and Atoo adversarial images have a large number of bounding boxes that are incorrectly predicted into the background. Besides, robust models on clean images also have similar results. This reasonably explains that the insufficient classification robustness is mainly attributed to the prediction
Best viewed zoomed color. And also provide clearer versions of Fig.5-8 in our supplementary.

---

Under review as a conference paper at ICLR 2023
STD
Robust
Figure 7: Robustness analyses with Detection Confusion Matrix for standard and robust Faster RCNN.
confusion between foregrounds and the background. Furthermore, these correctly classified objects have larger IOUs, demonstrating that the localization in detectors has higher robustness to some extent.
3) The poor performance of robust models is due to the mis- missclassification of the classification module, which leads to selecting bounding boxes at wrong positions, rather than the poor robustmess of the localization model itself.
In Fig. 5 under Ratt, the standard SSD model and robust SSD are enable to locate objects on Acts or Atee adversarial images, which is similar to the results clean images. Under Ridz. robust SSD yields low confidences for those boxes that should correct, but is still better than the standard model. Despite the limited robustness, its ability to distinguish objects in the classification module has deteriorated. Overall, it demonstrates that both standard and robust SSD present a reliable localization robustness adversarial images; however, the conflict of learning together clean and adversarial images seriously affects the classification robustness, which leads to the mis-classification of the foreground into the background.
4 ROBUSTNESS IN DETECTORS WITH DIFFERENT ARCHITECTURES
In this section, we will further explore the robustness bottleneck for other three detectors with different architectures including two-stage Faster-RCNN, anchor-free YOLOX and Transformer- based Deformable-DEr DETR.
4.1 ROBUSTNESS IN TWO-STAGE DETECTOR
Two-stage detectors require NMS for both stages to filter the detection bounding boxes based on the prediction results of their classification modules. For the the ClsAVal validation, the first stage uses Ron validations. Rall and Ride are adopted for the second stage.
Robustness Analyses on Faster RCNN. As demonstrated by Riáe validation in Fig. 6 standard Faster RCNN cannot accurately detect objects with adversarial perturbations even if the correct bounding box is given. This can draw conclusion that Faster RCNN itself has poor localization and classification robustness, unlike the standard SSD that presents good localization robustness even without adversarial training. Robust Faster RCNN also suffers from the robustness bottleneck. But, the locations of detected objects highly accurately for both attacks of Agte or Atoes demonstrated by Ridx and RAll in Fig. 6
With DCMs under either the attack of Aeta or Aioe in Fig. 7 standard and robust Faster RCNN models mis-classify many foreground objects as the background. From the DCM of Rull validation, matter what kind of attack is used. This further
the robust model shows remarkable performance supports the conclusion that the decline model performance is not due the poor robustness of the localization module üself. From the DCM of Rias validation, Faster RCNN also incorrectly classifies most of the correct bounding boxes as the background. Thus, two-Stage detectors also inevitably suffers from robustness bottlenecks.
Robustness Region Proposal Network (RPN). The localization results in the second stage depend on the proposal selected by the localization results of RPN. Therefore, we also analyze the robustness in RPN. The unrectified bounding boxes in the first stage are considered. In Fig. the standard model in the first stage can already give fairly accurate bounding boxes on the clean images. But the validation of Ru, and Rit indicates that the robust model does not show strong localization robustness at this stage. Under adversarial attacks, the bounding boxes at the correct positions have
7

---

Under review as a conference paper at ICLR 2023
Robust
clean
clean
Figure 8: Robustness analyses with Detection Confasion Matrix for standard and robust YOLOX.
large localization deviation. Even so, it still has correct objects in the bounding boxes, so the second stage is able to rectify the bounding boxes.
4.2 ROBUSTNESS IN ANCHOR-FREE DETECTOR
According to the previous analyses, the robustness bottleneck of object detectors is mainly attributed to the miscclassification of the foreground into the background with poor classification robustness. The localization module has reliable robustness. So, what if we can decide whether the bounding box is the background at the localization branch? Is the problem of the robustness bottleneck likely to be solved? To verify this hypothesis, we further validate these problems YOLOOX together with its detection robustness.
YOLOX proposes the decouple head which uses regression features the localization branch to determine whether there is object inside the bounding box. It multiplies the Sigmoid outputs (scores) Hote and Нoby as the final score instead of Softmax. It has two scores from Has and Hobi and thus has three manners for the ClsAVal validation: Rau considers both scores; Roby considers the outputs of Hobj: Reon/ replace only the outputs of Heis-
Since YOLOX does not directly give the confidence that the object belongs to the background, set the confidence of the background to: Cbg = - max (at.(Fa.((())) x Hobj (Flec (z))). From the results of DCM in Fig. [ robust YOLOX also has limited robustness performance under adversarial attacks. It still tends to mis-classify objects into the background. This is similar to SSD and Faster RCNN, indicating that placing the module of deciding whether the bounding box contains foreground objects into the localization branch cannot alleviate the robustness bottleneck. According to Rall, the robust YOLOX can correctly locate objects with robust localization and is hardly affected by the non-robustness of Hobg-
Since the final confidence of YOLOX consists of both Hets and Hoby, we also explore which part is dominantly responsible for the limited robustness. In Fig. X with results of Rull and Raáz that the object detector has small conflicts in the localization module with reliable robustness. With results of Rotj. if the model can correctly distinguish the foreground from the background, it can give the object higher confidence. With results Roo, even if the classification module classifies all the objects correctly, the detector still regards objects the background finally. The conflict among foreground classes is much weaker than the conflict between foreground and background classes.
4.3 ROBUSTNESS IN TRANSFORMER -BSEE DETECTOR
We analyze the robustness of Deformable- on MS-COCO Lin al. (2014). Transformer-based detector has a fixed number of object queries responsible for predicting different objects, but the correspondence between GTs and the bounding box predicted by each object query will change with learning. Here we assume that object queries of the same model are responsible for the same objects under an image with different perturbations. In this way, RAll and Radz under different models using their respective results on clean images. Deformable-Dee DETR uses Sigmoid to predict the category, similar to YOLOX. So we use a similar method to assign the background class confidence: Cbg max Has (f))))
8

---

Under review as a conference paper at ICLR 2023
STD
Robust
Cean
clean
Any
Figure 9: Robustness analyses with Detection Confusion Matrix for standard and robust Deformable- DETR models.
We select objects in COCO whose category is the same as the VOC dataset to calculate DCMs. From DCM results in Fig. 9 the standard model not only mis-classifies large number of foreground objects into the background, but also lot of confusion exists among foreground categories. Under Rut, the model identifies numerous objects into the background, which indicates that the standard monnformer-basee detector is poor for localization robustness. The bounding boxes the correct positions has a large deviation, which makes unable to match the correct true bounding boxes.
DCM of the robust detector clean images shows that, although it achieve 34.8% mAP, the category confusion is very serious. The robust detector also tends to mis-classify foreground objects as the background under Ad. and Atoe attacks, and even some objects not recognized at all. From the results of Rut and Rida, even if the bounding boxes at the correct positions are given, regardless of what the class scores are, the model still has poor performance. This indicates that the robustness of the localization module in the Transformer- hased detector as poor as the classification module, and the conflict is also serious. The Transformer-basee detector has worse robustness than CNN-based detectors. Its detection robustness bottleneck is not only closely related with the classification but also with the localization.
5 SUMMARY AND CONCLUSION
Existing methods present detection robustness bottleneck: a significant performance decline on clean images and limited performance on adversarial images. In this paper, we aim to make an initial attempt to comprehensively investigate the adversarial robustness of four seminal object detectors with different architectures, including the one-stage, two-stage, anchor-free, and Transformer-base detectors. With the proposed Detection Confusion Matrix and Classification-Allatii Validation, there are mainly three conclusive analyses:
1) For adversarial attacks, standard and robust detectors tend to mis-classify foreground objects into the background instead of confusion among foreground categories, which different from the adversarial robustness in image classification.
2) Robust object detectors with different architectures via adversarial training present the robustness bottleneck. For CNN-based detectors, the main reason is that the classification module confuses the foreground and the background so that detectors select the bounding boxes at the wrong positions. The localization module itself has reliable localization robustness. In the Transformer-based detector, its classification robustness and localization robustness are poor.
3) The localization module is highly robust, but determining whether the object belongs to the foreground or the background in the localization branch cannot alleviate the conflict between them. Adversarial attacks for detection models with various structures will seriously confuse the foreground and the background.
To break the detection robustness bottleneck, one crucial research topic is how alleviate the mis- classification of the foreground objects into the background. Especially, the label-assignment routine for object detection regards both bounding boxes that fully contain background and those that contain partial foreground as background; only those that precisely contain objects are labeled as foreground. Thus, fooling the detector just needs to successfully attack part of the foreground, which may create the learning conflicts in adversarial training. Besides, among those detectors, Transformer-base Deformable-DETR presents inferior robustness, which is worthy of further considerations. Our work is expected to inspire more researches on the adversarial robustness for object detection in the future.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pp. 39-57, 2017.
Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In European Confer- ence on Computer Vision (ECCV), pp. 833-851, 2018a.
Pin-Chun Chen, Bo-Han Kung, and Jun -heng Chen. Class-aware robust adversarial training for object detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10420-10429, 2021.
Shang-Tse Chen, Cory Comellus, Jason Martin, and Duen Horng (Polo) Chau. Shapeshifter: Robust physical adversarial attack on faster R-CNN object detector. In Machine Learning and Knowledge Discovery in Databases European Conference (ECML), pp. 52-68, 2b.
Ziyi Dong, Pengxu Wei, and Liang Lin. Adversarially-aware robust object detector. European Conference on Computer Vision (ECCV), 2022.
Mark Everingham, S. M. Ali Eslami, Luc Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision (IJCV), 111(1):98-133) -136 2015.
Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.084. 2021.
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
Sven Gowal, Sylvestre-Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy Mann. Improving robustness using generated data. ArXiv,
2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In European Conference on Computer Vision (ECCV), pp. 740-755, 2014.
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott E. Reed, Cheng Yang Fu, and Alexander C. Berg. SSD: single shot multibox detector. In European Conference Computer Vision (ECCV), pp. 21-37, 2016.
Xin Liu, Huanrui Yang, Ziwei Liu, Linghao Song, Yiran Chen, and Hai Li. DPATCH: an adversarial patch attack on object detectors. In Workshop on Thirty-Than Third AAAI Conference on Artificial Intelligence (AAAI), 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster R-CNN: towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems (NeurIPS), pp. 91-99, 2015.
Sayantan Sarkar, Ankan Bansal, Upal Mahbub, and Rama Chellappa. UPSET and ANGRI: Breaking high performance image classifiers. CoRR, abs/ 159, 2017.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations (ICIR), 2014.
10

---

Under review as a conference paper at ICLR 2023
Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. Transferable adversarial attacks for image and video object detection. In International Joint Conference on Artificial Intelligence (IJCAI), pp. 954-9099 960, 2019.
Chaowei Xiao, Bo Li, Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating adversarial examples with adversarial networks. In Proceedingy of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI), pp. 3905-3911, 2018.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan L. Yuille. Adversarial examples for semantic segmentation and object detection. In IEEE International Conference on Computer Vision (ICCV), pp. 1378-1387, 2017.
Haichao Zhang and Jianyu Wang. Towards adversarially robust object detection. In IEEE International Conference Computer Vision (ICCV), pp. 421-430, 2019.
Hongyang Zhang. Yaodong Yu, Jiantao Jiao, Eric Xing. Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (ICML), pp. 7472-7482, 2019.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR: deformable transformers for end-to-end object detection. In International Conference Learning Representations (ICLR), 2021.


Reviews:
[{"review_id": "t6qBiU2QEy", "reviewer": "Reviewer_PhPF", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper attempts to investigate the problem of why object detection models fail with adversarial attacks. The authors tried to look into the classification and localization branch independently to locate the issue. Also, different types of object detection networks were investigated for this issue. According to experimental results, the authors made some conclusions about which part tasks more mistakes.", "summary_of_the_review": "The paper investigates an essential and interesting issue. However, due to poor writing, serious issues in clarity and lack of novelty, I tend to reject this paper.", "strength_and_weaknesses": "Pros:\n+ Adversarial attack in object detection is an important issue. The problem is practical and worth looking into.\n+ The authors attempt to figure out which part fails for different models, which is a good attempt.\n\nCons:\n- The paper is difficult to read. There are a lot of issues in writing. Most papers are not cited in the right way. For example, in ''FGSM Goodfellow et al. (2015), PGD Madry et al. (2018), AdvGAN Xiao et al. (2018)', the name of the model should not be messed with the author names. It is suggested that the authors look into the previous ICLR-accepted papers. Also, there are a lot of grammar issues. For example, ''Chen et al. (2018b) have been proposed to attack the state-of-the-art object detectors.' --> Chen et al. have proposed...\n- The motivation and previous issues in these models in Introduction have already been discussed in previous papers, e.g. Adversarially-aware robust object detector in ECCV 2022. Fig 1 is also quite similar to Fig.2 in that paper.\n- As a paper to investigate different types of detection models, the authors should have a good literature review on object detection models. However, the object detection models are categorised as two-stage, one-stage, anchor-free, and Transformer-based detectors. It's not an accurate classification, e.g., one-stage detectors can be either anchor-free or not. Previous papers have already clearly discussed the critical point that makes the detectors different from each other, e.g. ATSS (Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection). It is suggested to do a redundant literature review before discussing how the adversarial attack affects the detectors so that the authors can decide how and why these different categories should affect the investigation.\n- The main observation that the classification module affects more in the adversarial attack has already been discussed in Section 3.2 of  Adversarially-aware robust object detector in ECCV 2022.\n- Section 3.1  is very difficult to understand. The authors designed a very complex method (it looks like it matches the object index between attacked or clean images) to investigate the localization branch independently. It can be simply done by calculating the loss change as proposed in Adversarially-aware robust object detector in ECCV 2022.\n- The results after NMS in Fig.3 are very confusing. It doesn't look like NMS has been done, as there are highly overlapped boxes. And the text with the bounding box sits in different locations in the same image, which makes it even more difficult to investigate clearly.\n- The authors also report things like ''it is unable to successfully train Deformable-DETR on PASCAL VOCusing the official code.'', which makes me doubt the reproducibility of results in this paper.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "XKcYnIXx12", "reviewer": "Reviewer_dqA6", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, authors investigates the failure reasons of object detectors under adversarial attacks. Authors propose two analysing strategies, named DCM and ClsAVal, to understand the classification confusions and the contribution of localization failures. Authors analyzed 4 types of object detectors (single-stage, two-stage, anchor-free and transformer query based), and focused on the single-shot detectors with their analysing strategies. Some conclusions were made such as the classification confusion is usually the more serious root cause of detection failures.", "summary_of_the_review": "The paper provides adversarial robustness analysis on a large group of object detectors. However, the novelty is not strong enough and there are still some concerns in the current version, thus I lean towards weak reject at the current stage.", "strength_and_weaknesses": "### Strength\n* The paper analyzes a large group of detector types, and analyzes object detector with and without adversarial training.\n* The paper draws some clear conclusion, which may be helpful for future works on improving the robustness of object detectors.\n\n### Weaknesses\n* The novelty is limited and the findings are not super surprising that the adversarial attack is mostly making the classification wrong.\n* I feel the conclusion made in section 3.2 that \"The adversarial attack enforces the standard detection model to easily mis-classify foreground objects into the background\" may depend on the attack method. For example, in figure 1 of [a], the dogs are detected to be \"train\" with high confidence score.\n* Conclusions in section 3.2 (a) are mostly drawn from figure 6 which is qualitative analysis. More careful quantitative analysis is required.\n* The clarity of the paper can be improved. For example:\n  * In equation 1, the classification loss is only calculated on $y_i$ and $\\mathbf{b}_i$. I feel this is incorrect since background classification should also be considered in loss?\n  * In section 3.1 ClsAVal, should we emphasize that $M$ and $M^R$ have the same structure (e.g. $M^R$ is an adversarially trained counterpart of $M$), so that the indices $\\it{idx}$ from one detector can be applied to the other? This is not super clear in that paragraph so it's a bit confusing.\n  * It would be better to explain a bit the details of \"$R_{idx}(A_{cls})$\" etc in the caption of figure 6.\n\n[a] Adversarial Examples for Semantic Segmentation and Object Detection", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "1wwKSOdKaOl", "reviewer": "Reviewer_K7hu", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "To alleviate the detection robustness bottleneck in the future work, this paper investigated the robustness of object detectors.\nThis work proposes two module: Detection Confusion Matrix (DCM) and Classification-Ablative Validation (ClsAVal).\nDCM is used to analyze the confusion of detection results between different categories and their localizations, \nand ClsAVal is used for detection robustness analysis.", "summary_of_the_review": "This paper proposes two modules as indicators for confirming the robustness of existing object detectors for the adversarial attacks.\nExperimental results applied to multiple cases show various interpretations in question.\nHowever, these points are not considered novel, and it is hard to see what changes these indicators show when they actually improve.", "strength_and_weaknesses": "Strengths\n- Many existing models fail to explain what process they went through when producing incorrect results. This work points to these problems.\n- Figure 3 and Figure 4 are sufficient to understand each module.\n- Supplementary materials provide code and various experimental results.\n- Various analyses of the experimental results suggest the direction of future research in this task.\n\n\nWeaknesses\n- Existing papers also point out this problem and show the need for a new model, which this paper claims is an initial attempt for analysis. A novel motivation is needed for proper analysis.\n- Writing is not clear. All paragraphs should be written clearly so that even a first-time reader can understand them.\n- I wonder if the object detection methods can be improved with this metric. Isn't it too consequential analysis to use both modules after applying NMS?\n- The results can be intuitively confirmed through visualization, but I am not sure that this can be generalized to lead to a specific interpretation. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject