Django: Detecting Trojans in Object Detection Models via Gaussian Focus Calibration
Authors: Guangyu Shen
Siyuan Cheng
Guanhong Tao
Kaiyuan Zhang
Yingqi Liu
Shengwei An
Shiqing Ma
Xiangyu Zhang
Venue: NeurIPS 2023 poster (2023)
Keywords: backdoor detection; object detection;

Abstract: Object detection models are vulnerable to backdoor or trojan attacks, where an attacker can inject malicious triggers into the model, leading to altered behavior during inference. As a defense mechanism, trigger inversion leverages optimization to reverse-engineer triggers and identify compromised models. While existing trigger inversion methods assume that each instance from the support set is equally affected by the injected trigger, we observe that the poison effect can vary significantly across bounding boxes in object detection models due to its dense prediction nature, leading to an undesired optimization objective misalignment issue for existing trigger reverse-engineering methods. To address this challenge, we propose the first object detection backdoor detection framework Django (Detecting Trojans in Object Detection Models via Gaussian Focus Calibration). It leverages a dynamic Gaussian weighting scheme that prioritizes more vulnerable victim boxes and assigns appropriate coefficients to calibrate the optimization objective during trigger inversion. In addition, we combine Django with a novel label proposal pre-processing technique to enhance its efficiency. We evaluate Django on 3 object detection image datasets, 3 model architectures, and 2 types of attacks, with a total of 168 models. Our experimental results show that Django outperforms 6 state-of-the-art baselines, with up to 38% accuracy improvement and 10x reduced overhead. The code is available at https://github.com/PurduePAML/DJGO.

Django: Detecting Trojans in Object Detection Models via Gaussian Focus Calibration
Guangyu Shen" Purdue University West Lafayette. IN, 47907 shen447dpurdue.cc
Siyuan Cheng" Purdue University West Lafayette, IN 47907 cheng535@purdue... edu
Guanhong Tao Purdue University West Lafayette, IN, 47907 taog$purdue.. edu
Kaiyuan Zhang Purdue University
Yingqi Liu Microsoft
Shengwei An Purdue University West Lafayette, IN, 47907
West Lafayette, IN, 47907
Redmond, Washington 98052
zhan4057@purdue edu
yinggiliuOmicroson com
an930purdue edu
Shiqing Ma
Xiangyu Zhang Purdue University West Lafayette, IN, 47907 xyzhangocs, purdue. edu
University of Massachusetts at Amherst
Amherst, MA, 01003 shiqingma@umass.edd
Abstract
Object detection models are vulnerable to backdoor or trojan attacks, where an attacker inject malicious triggers into the model, leading altered behavior during inference. As defense mechanism, trigger inversion leverages optimization reverse-engineer triggers and identify compromised models. While existing trigger inversion methods assume that each instance from the support set is equally affected by the injected trigger, we observe that the poison effect can vary signifi- cantly across bounding boxes in object detection models due to its dense prediction nature, leading to an undesired optimization objective misalignment issue for exist- ing trigger eyverse-engineeiigg methods. To address this challenge, we propose the first object detection backdoor detection framework Django (Detecting Trojans in Object Detection Models via Gaussian Focus Calibration). It leverages dy- namic Gaussian weighting scheme that prioritizes more vulnerable victim boxes and assigns appropriate coefficients to calibrate the optimization objective during trigger inversion. addition, we combine Django with a novel label proposal reeproccessing technique to enhance its efficiency. We evaluate Django on 3 object detection image datascts, model architectures, and 2 types of attacks, with a total of 168 models. Our experimental results show that Django outperforms 6 state-of-the-arr baselines, with up to 38% accuracy improvement and 10x reduced overhead. The code is available at httth:://// h/ggt/tub com http//uuruuuuPa...MM
Introduction
Object detection is an extensively studied computer vision application that aims to identify multiple objects in a given image. It has been widely integrated into various real-world systems, including public surveillance (51. autonomous driving []4 25 optical character recognition (OCR) etc.
Equal Contribution
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Classe Pizza
Class: Giralle
Class: 12
(a) Evasion trigger
(b) Misclassification trigger
Figure 1: Triggers with different effects. The blue patch in Figure I(a) is an evasion trigger, causing the bounding box to disappear. Figure represents misclassification trigger (yellow), leading to the misclassification of "pizza" as "giraffe".
State-of-the object detection systems, e.g., SSD 32 and YOLO are primarily built on Deep Learning (DL) models [13,, owing to their remarkable feature representation capabilities.
Despite the impressive performance of existing object detection approaches. they are vulnerable to backdoor attacks. Backdoor attack is form of training-time attack, in which malicious adversary strategically embeds an imperceptible trigger (e.g.. small image patch) onto a small set of training data. DL models trained on this poisoned dataset will predict the attacker-chosen targer label whenever an input is stamped with the trigger. There are two major backdoor attacks in object detection, namely. misclassification attack 33 and evasion attack. As shown in Figure | misclassification attack causes the object detection model to misclassify an object as a target object, whereas evasion attack makes the model fail to recognize certain objects (i.e., classify traffic signs as the background). Such attacks can significantly hinder the deployment of object detection systems in real world as they may cause severe security issues or even life-threatening incidents.
To counter the backdoor threat, researchers have proposed a multitude of defense techniques aimed safeguarding the model against backdoor attacks throughout its entire life cycle (5
33
16
26 22
Among them, trigger inversion 559 of the most popular backdoor scanning methods and has demonstrated its effectiveness in various tasks [59 351.
18
10
54
50
36
72
27
65
31 35).
55
18
is one
Given small set of clean samples and the subject model, trigger inversion techniques aims determine if the model is backdoored or not by reverse-engineering the underlying triggers. The task of trigger inversion can be formulated as an optimization problem with specific constraints, such as trigger size [59 [8 or particular stylistic attributes 99 42
In order to detect backdoors in object detection, a direct solution is to adapt trigger inversion from the classification task object detection. We however find that simple adaptation of trigger inversion fails short. This attributes to the significantly denser output of object detection models due to the nature of the task. In specific, an object detection model, such as SSD (32). typically produces thousands of bounding boxes with corresponding class probabilities to capture objects of varying scales within a single image. Attackers can exploit this extensive output space to conceal injected triggers and mislead the optimization process during trigger inversion, causing the loss misalignment issue. That is, throughout the entire trigger inversion process, the loss values consistently remain either significantly high or extremely low, regardless of the Attack Success Rate (ASR) that denotes the ratio of misclassified samples when the trigger is applied. This misalignment property hinders the identification of high-qualing triggers, leading to low backdoor detection performance.
Our study reveals that the underlying cause of the loss misalignment issue stems from the unequal impact of poisoning on individual bounding boxes within the backdoored model. More specifically, the backdoor trigger affects only a small fraction of bounding boxes, while leaving the majority of them largely unaffected. Consequently, the aggregated gradient information is obfuscated by remaining benign bounding boxes during inversion, thereby impeding the optimizer from accurately estimating the gradient direction. Based on this observation, we propose the first trigger inversion- based backdoor detection framework for object detection: DIANGO (Detecting Trojans Object Detection Models via Gaussian Focus Calibration). The overview is shown in Figure It features a novel Gaussian Focus Loss to calibrate the misaligned loss during inversion by dynamically assigning weights to individual boxes based on their vulnerability. Equipped with a label proposal pre-processor, DIANGO is able to quickly identify malicious vietim-target labels and effectively invert the injected trigger lies in the backdoored model. Extensive evaluation of 3 object detection datasets, utilizing 3

---

different model architectures with total of 168 models, demonstrates the superiority of DJANGO over six existing state-of-the-ar backdoor scanning techniques. Specifically, we achieve up 0.38 ROC improvement and substantial reduction in scanning time.
2
Background & Threat Model
Deep Learning based Object Detection Tech- niques. Extensive research efforts have been devoted deep learning (DL) object detec-
Prepoccaiing
Sumples
Preprocesing
Victim Image
Target
tion, encompassing various aspects such as neural network architecture design
Ship
32 45 2
training optimization 129 61 74], and robust-
Inverted Trigger
Stamp
Model
Gai Mores
ness [71 66 (). These endeavors contribute to
broad body of work aimed at advancing the field
Trigger Inversion
of DL object detection. Specifically, given an image E X that contains p objects, object detection neural network ge (parameterized by 8) outputs K (K >> p) 5-tuples: D = (x)) (bg, by bh.
Decision
Optimization
GF. Calibration
Banigs
kR: P {bx, Pke P shorten
Trajaned
as ba (x)) and pa (z), which denote the center co- ordinates, height, width, and class probability
Figure 2:
Overview of DIANGO
of the k-th predicted bounding box. Note that K is usually much larger than (K 8732 in SSD (32. p 7.7 in COCO dataset (30]). Box matching algorithms [32 46) and Non-Maximum Suppression(NMS 20 were introduced to address the challenge posed by the substantial box number discrepancy between the prediction and ground-truth in both training and inference stages.
Backdoor Attack & Defense. Backdoor attacks can be carried out through data poisoning with modified clean labels [17 34 7 58) and model parameter hijacking [47]. Initially, small image and later on more complex triggers have been These attacks are not limited to image classifi-
patches were used as backdoor triggers 17
8
48
successfully injected in DL models 28
64
33
37
cation models; they also affect Large Language Models (70), code models 773 52). Reinforcement Learning agents (60), pre-trained image encoders 24 (56] and object detectors 3
Researchers have proposed various approaches from defense perspectives. During the training stage, methods such poisoned sample identification and filtering have been proposed [40] 53
43). In the post-training stage, backdoor scanning techniques aim to recover injected triggers from trained
57
Backdoor removal techniques are employed to elimi- nate triggers with minimal resource consumption and performance impact. Real-time rejection of trigger-carrying samples during inference can be achieved using methods like [12
models 55 49
63 35
50
15
16 Furthermore,
theoretical analyses have been conducted to understand backdoor attacks (67
68, 72 23 In this
work, we prioritize the practical application of trigger inversion-based backdoor detection, with specific focus object detection models.
3 Methodology
Threat Model. Our work considers the standard setting used existing backdoor scanning liter- ature [59
10), where the defender has a small set of clean samples from each class in the validation set (10 our paper), no poison samples and white-box access the model under scanning. The defense goal is to classify the benignity of the subject model. Our focus in this study on label specific polygon triggers with two types effects: misclassification and evasion, which cause the model to predict object as target object and the background, respectively.
33
18
3.1 Misalignment of CE Loss and ASR in Object Detection Model Trigger Inversion
Backdoor Detection via Trigger Inversion. Trigger inversion aims to decide whether the model backdoored or not by reverse-engineering the injected trigger through optimization. Consider an image classification model fe parameterized by 0 with C' classes and small support set Se, from

---

ASR
1.0
0.5
Ca ASR
ASR
0.1
0.0
10
20
10
Stops
Stego
Stagr
(a) Classification model
(b) Object detection model with (c) Object detection model with evasion trigger
misclassification trigger
Figure 3: ASR vs. CE loss during trigger inversion
each class Cy. Trigger inversion 49 59
aims to solve the following optimization objective.
min Eres. [=(f(((z))) cx] + 311m||1, where d(x) = (1 m) m
m.p
Variables m and are the trigger mask and pattern respectively, to be optimized. The loss function C(-) typically resues the task loss function as measurement of the degree of the misbehaved model. In classification and object detection, it will be the cross entropy (CE) loss. / is the coefficient on the regularization term, and ||m]]] represents the norm of the mask. Since the defender lacks the knowledge of the exact poisoned victim and target labels, the backdoor scanning requires optimizing Eq. for every label pair (ce, 9y). any of these pairs yields trigger with an exceptionally small size, it is indicative that the model is trojaned.
Intuitively, it shall be effortless to adapt existing trigger inversion methods from image classification to object detection. Considering the disparity of these two tasks, object detection models generate much denser output for each input, i.e., thousands of bounding boxes versus one label in classification. We can make adjustments to Eq. [ to optimize a trigger that can impact samples at the box level rather than the entire image. Given a sample containing a victim object, one naive modification is to optimize a trigger that can influence all bounding boxes that are close to the victim object. The objective function can be reformulated as follows:
min Rees. FEET [C(0, ct) + 81mm/11
m.p
where T = (xx()(=))
Jbx(((x)), (T(o))))))(]b()()()) 2 a} and Φ(x) = (1-m)
We denote the predicted probability and coordinates of the k-th bounding box of sample ((x) as Pu (Ø(x)) and (x(())))),, respectively. The function J(-) refers to the Jaceard Overlap, also known as Intersection over Union. b* denotes the ground-truth box coordinates. Intuitively. To indicates the class confidence of a set of predicted boxes surrounding the victim object. Additionally, a is a pre-defined threshold used to quantify the degree of overlap between predictions and the ground-truth box and set to 0.5 by default. In this naive extension, the objective to consider every predicted box surrounding the victim object the target of the attack and aim to reverse-engineer trigger that compromises all of them. We explain this limitation in the following.
Mis-aligned Loss and ASR. Attack Success Rate (ASR) denotes the percentage of victim samples misclassified as the target label when the inverted trigger is applied (e.g.. set at 90%). For misclassisi- cation trigger, a successful attack is defined as the presence of at least one box predicts the target class in the final detection. Evasion Attack a particular case misclassification attack where the target label is the background class 0. Cross-entropy (CE) loss measures the numerical difference between model output (confidence) and the desired prediction. In general, ASR and the CE loss are negatively correlated throughout the optimization process because minimizing the CE loss increases the probability of the target label. If the target label probability surpasses those other labels, the input is misclassified the target label. Figure 3(a) illustrates the characteristics of the CE loss and the ASR during trigger inversion for an image classification task.
Such correlation however may not hold for object detection using naive trigger inversion Eq. 2 Specifically, Figure 3(b) shows the inversion process on Faster-RCNN (55 object detection model poisoned by evasion trigger (TrojAI Round13#9) victim class 47). Note that an evasion trigger

---

Box Confidence Discspuncy
Sox Confabance Discupancy
Target Canfidence
wrTtigger
Confidence
Trigger
2.2
10:10
10
Box Index
Box 1hddox
(a) 11:90 +57: Miscls trigger
(b) R13#33:11
Evasion trigger
Figure 4: Target confidence discrepancy w/o the ground-truth trigger
will cause all bounding boxes surrounding the victim object to disappear. Observe that even when the CE loss is small ($ 1, the ASR remains low (0%). For model poisoned by misclassification attack (Round1 3#12 victim class 3), as illustrated in Figure 3(c) the CE loss remains incredibly high (2 10) and decreases only slightly during trigger inversion, while the ASR remains low or even decreases. Therefore, the naive trigger inversion fails to find a trigger with high ASR and cannot identify the poisoned model. The observations are similar when extending several state-of-the-an trigger inversion techniques [3 [99 55 [8] to object detection. For instance, NC [59] only achieves 58% detection accuracy on the TrojAl dataset
Causes of Misalignment. We study the misalignment problem by using differential analysis to compare the intermediate outputs of the object detection model with and without the ground-trut trigger, specifically the target label confidence of predicted bounding boxes. Given a sample with victim object, we collect all predicted boxes with loU 2 a (a: 0.5 means box close the object) around the victim object and track their target label probability change before and after applying the ground-truft trigger. The key observation is that not every box is equally poisoned. Figure (a) shows the per-box confidence discrepancy for poisoned model with misclassification trigger. Although there are total number of boxes surrounding the victim object, only approximately 12% of them (10 boxes, shown in high red bars) are infected by the ground-truth backdoor trigger. The rest the boxes only have an average < 10 probability increase on the target label. For the evasion trigger in Figure 4b). there is slight difference, where 85% of the boxes already have high confidence (on the background class 9) even without triggers. The backdoor trigger only elevates the target probability for the remaining 15% of the boxes.
This intuitively explains the misalignment issue of naive trigger cevrse--nginnerring methods. During optimization, the cross-entropy loss of each box is equally aggregated, and the averaged gradient used mutate the trigger via back poek-propagation. However, in the poisoned model, there are only small fraction of bounding boxes affected by the backdoor trigger. The gradient information from unaffected boxes dominates the optimization direction, causing the inversion to fail. This observation highlights the significance of considering individual bounding boxes separately when devising inversion methods for object detection, which shall focus on boxes that have a higher likelihood of being compromised but have not been considered yet.
3.2 Our Solution: Trigger Inversion via Gaussian Focus Loss
As discussed in the last section, the goal our trigger inversion to discriminate different bounding boxes during optimization. Particularly, we aim to find easy-to-flip boxes that are vulnerable and targeted by injected backdoors. However, is non-trivial to adaptively select boxes during trigger inversion the box output is constantly changing.
Focal Loss 29 is a widely used metric in object detection for assisting better training. separates objects into hard-to-classify and easy-to to-assssyy objects based on their training errors, and assigns different weights on their training losses. For the notation simplicity, we use a binary object detection model. Note that it is straightforward to extend it to multi-class cases. Recall Pa denotes the output probability of the k-th victim box. The CE loss in Eq. 2 for the k-th victim box can be simplified as = log(Es). ). Focal Loss is refined weighted CE loss aiming to solve the
L(ir, c.) log(ps)) imbalanced learning difficulty issue in object detection models. detail, assigns larger weights to boxes that perform poorly (hard examples), i.c., small 6x. to enlarge its contribution to the aggregated
5

---

Initial Stage
Moderate Stage
Late Stage
Box Density
100-
Box Coefficient
2000
Box Coefficient
Robust Boxes
80-
Flipped
40-
5.0
0.15
0.25
0.35
0.50
Target Confidence
Figure 5: Coefficient dynamics during reverse-engineering misclassification trigger
loss. It assigns smaller weights to boxes that have already learned well (easy examples), i.c., large Pk:
Lfp ai) -x11 -² log(()).
(3)
where > 0,7 2 1 (A- 0.25, suggested in 29 ) The coefficient of the k-th box will
exponentially grow as its probability Pa gets close to 0 and vice versa.
Eq. provides insights to address the misalignment problem. Inspired by this, propose Gaussian Focus Loss to calibrate the misaligned loss. As illustrated in Figure [aa the backdoor trigger tend to infect only small portion of vietim boxes and leaves remaining largely unchanged. In other words, trigger will mainly focus on compromising more vulnerable boxes that are easy to flip (easy example). Therefore, our goal is essentially opposite the Focal Loss it aims pay more attention to the hard samples. To avoid the gradient vanishing issue shown in Figure 4(b) box shall also not be focused long as it reaches the attack criterion, i.e., target label confidence ps: 2 0.1. To summarize, the desired loss shall be able to dynamically capture a set of vulnerable boxes that have not been flipped yet, and assign a large coefficient to encourage the transition. The natural bell shape of the Gaussian Distribution perfectly satisfies our constraints" We formulate our proposed novel Gaussian Focus Loss follows:
Cgt ca) - b") b')((f% ãx. Px) log((x)
where J Jbe, b²) = loU b"), d(x; 0,
2v - 1
(4)
It involves two key attributes to measure the vulnerability of victim box during optimization: the overlapping with the ground-trun truth object and the confidence towards the target label. Namely, boxes with larger overlapping and higher target label confidence tend be more vulnerable. The fúg and o% are the mean and standard deviation of the Gaussian Distribution. Note that we allow two parameters be co-optimized with the inverted trigger such that we can adjust the range of focused boxes dynamically through the entire trigger inversion procedure. We initialize júę = 0.1 and ã = 2 in this paper. Figure 5 provides visual explanation of the coefficient dynamics during various optimization steps. Initially, when the victim boxes exhibit low confidence towards the target label, the Gaussian Focus Loss assigns slightly larger weights a considerable number of bounding boxes surrounding the initial value (0.1). As the optimization progresses, the Gaussian Focus Loss prioritizes more vulnerable boxes that have higher target label probabilities by decreasing the value of and increasing the value of In the later stages, the weight coefficients of boxes that have already been flipped are reduced mitigate the issue of gradient vanishing.
3.3 Compromised Label Proposal via Backdoor Leakage
Without knowing the exact victim and target labels, scanning backdoors is time-consuming. To reduce the time cost, pre-proces proccesesing is proposed 49 (54] to quickly select small set of promising compromised label pairs based on the widely existing backdoor leakage phenomena: the poisoned
Any distributions characterized by centralized peaks are suitable for our intended purpose. We explore alternatives Appendix D

---

Table I: Comparison to trigger inversion based methods
D
M
Metric
ABS
NC*
Pixel
Pixel
Tabor
Tabor
DIANGO
Precision
0.7143
0.6250
0.6522
0.4400
0.6666
0,3300
0,4030
0,8000
SSD
Recall
0.6250
0,9375
0.9375
0.5000
0.2500
0.3750
0.6250
1,0000
ROC-AUC
0,7109
0.6250
0.6641
0.6250
0,7083
0,5400
0.6725
0.9160
Traffic Sign Synthesis
Overhcad(s)
409.23
>3600
1180.52
>3600
>3600
902.10
861.30
F-RON
Procision
0.6667
0,3478
0,6364
0.5512
0.6250
0.6621
0.7125
0,8571
Recall
0.3750
1.0000
0,8750
0.6110
0.6250
0.6012
0.6250
0.7500
ROC-AUC
0.5352
0.6094
0.6953
0.6721
0,7500
0.6425
0.6820
0.8750
Overhcad(s)
753.89
>3600
2799.42
>3600
2466.08
>3600
2562.59
2128.99
Precision
0,3478
0.3478
0,4000
0,5000
0.2307
0.5000
0,8750
DETR
Recall
1.0000
1,0000
0,5000
0.3750
0.3750
0.6250
0,8750
ROC-AUC
0.5312
0,5312
0.5000
D. 0.6660
0,2500
0.7600
0.9160
Overheal(s)
>3600
691.27
>3600
294.32
>3600
275.50
228.92
Precision
0.6000
0.3636
1,0000
0.3300
0. 0.3000
0,5000
0.3333
1,0000
SSD
Recall
0.7500
1.0000
0,1250
0,2500
0,7500
0.2500
0.5000
1,0000
ROC-AUC
0.5000
U.5312
0.6875
0.3750
0.3333
0,5000
0.6250
1,0000
DOTA
Overheal(s)
424.16
>3600
675.32
>3600
$29.11
>3600
618.72
678.48
F-RCNN
Precision
0,8000
0,3636
0.3636
0.3330
0.3300
0,8000
Recall
1.0000
1.00000
1,0000
0.5000
0.7500
0,5000
0.5000
1,00000
ROC-AUC
0.8750
0.5625
0.5625
0,5000
0.6666
0.5000
0.5000
0,9160
Overiead(s)
1127.01
>3600
2866.14
>3600
1565 82
>3600
1602. 10
1425.25
Precision
0.5167
0,5231
0.8333
0,7500
1.0000
0.6120
0.7500
0,9696
COCO
SSD
Recall
0.9688
1.00100
0.1471
0.2500
0,2500
0,5620
0.5300
0.8888
ROC-AUC
0.5584
0.5404
0.5579
0.6200
0.7000
0,9305
Overhead(s)
273.23
>3600
2119.34
>3600
1788.50
>3600
1688.19
1476.01
model's behavior on victim samples tends to shift towards the target label even without the appearance feed clean samples from each class and pick the most
of the backdoor trigger. Therefore, likely target label by observing the model's s output distribution shift. Specifically, given a set of samples {= from class G,, we collect top-h predicted classes of each predicted box for each sample xy. Denote wj as the frequency of class Cy appearing in the total box predictions. We consider pair {c,, ey) compromised label pair if w, 2 In this paper, we make a trade-off by setting h = 5 and = 0.5. We also evaluate the sensitivity of hhyer--arametert in Section 4.2
4 Evaluation
All the experiments are conducted server equipped with two Intel Xeon Silver 4214 2.40GHz 1 12-core processors, 192 GB of RAM, and a NVIDIA RTX A6000 GPU.
Models and Datasets. We conduct the experiments models from TrojAl [ round 10 and round 13, which consists of a total of 168 models. Approximately half of these models are clean, while the other half are attacked. Our evaluation covers 3 existing object detection image datasets, including COCO 30) Synthesized Traffic Signs [.. and DOTA_ v2 [[]] on three representative object detection architectures: single-stage detector: SSD 32) two-stage detector: RCNN 45] and vision transformer based detector: DETR (2). Please refer to Appendix A for more detailed description.
Attack Settings. We conducted an evaluation of DIANGO by subjecting it to two types of backdoor attacks commonly observed in object detection models: Misclassification and Evasion attacks B 38 Misclassification attack means to flip the prediction of objects from the victim class to the target class, when the backdoor trigger is stamped on the input image. On the other hand, evasion triggers tend to suppress the recognition of victim objects, such that the model will consider them as the background. It is noteworthy that the evasion trigger can be regarded as a particular instance of the misclassification trigger, where the target label is set as the background class 0. As a result, its detection can be performed uniformly. Besides, we leverage stealthy polygons with different colors and textures backdoor triggers, which are widely evaluated in many backdoor detection papers 59
49
Evaluation Metrics. In our evaluation of backdoor detection methods, we employ four well- established metrics: Precision, Recall, ROC-AUC, and Average Scanning Overheads for each model. The unit of measurement we use is seconds (s), and we set a maximum threshold of hour (3600 s) for all the methods being evaluated. If the scanning process exceeds 3600 seconds, is terminated, and we rely on the existing results for making predictions. Appendix B present more details.
7

---

Baseline Methods We compare DJANGO against 6 baseline methods, including 4 trigger inversion based methods (i.e., NC [59] ABS
Pixel [55] and Tabor [T8],, and meta-clasiiftcttin based methods (i.e., MNTD 69 and MF 21 For meta classification based methods that involve training. we have performed 5-fold cross-valliaate
33
and reported the validation results exclusively. For the inversion-based scanners, we adopt their objective functions as described in Eq. and keep their other designs unchanged. To ensure a fair comparison, we use the same setting for common optimization hyper-parameters for all inversion based methods. We determine the optimal threshold for the size of inverted triggers as the detection rule for 59 55 18]. For ABS, we follow its original technique and use REASR as the decision score.
4.1 Detection Performance
Comparison to Trigger Inversion based Meth- ods. Table shows the results for trigger inver- sion based methods, where the first two columns denote the datasets and model architectures, the third column the evaluation metrics, and the
Table 2: Comparison to meta classifiers
D
M
Metric
MNTD
MF
DJANGO
Precision
0.4545
0.4000
0,8000
Traffic Sign Synthesis
SSD
Rocall
0.3125
0.5000
1.0000
subsequent columns the baselines. Addition-
ROC-AUC
0.3750
0.6563
0,9160
ally, considering that NC [59], Pixel [55], and Tabor [8] need to scan all possible victim- target pairs, which can exceed the time limit, we have employed our warm-up pre-processing technique (Section 3.3). We compare the en- hanced versions of these methods (NC*, Pixel", Tabor") ) with DIANGO. Note that we have not evaluated ABS on DETR models, as the original version is specifically designed for CNN-based models and not transformers.
Precision
0.5652
0,5000
0.8571
F-RCNN
Recall
0.8125
0.1250
0.7500
ROC-AUC
0.7695
0.6797
0.8750
Precision
0.1250
0.1250
0,8750
DETR
Recall
0.1250
0.1250
0,8750
ROC-AUC
0.1719
0.2890
0,9160
Precision
0.3333
0.3333
1,0000
SSD
Recall
0.1250
0.5000
1,0000
DOTA
ROC-AUC
0.3906
0.4375
1.0000
Precision
1.0000
0.3333
0,8000
F-RCNN
Recall
0.5000
0.5000
1.0000
ROC-AUC
0.6562
0.6875
0,9160
DJANGO consistently outperforms all the base- lines, with the best results are highlighted in
COCO
Precision
0.8182
0.7000
0,9696
SSD
Recall
0.5625
1,0000
0.8588
bold. The average ROC-AUC of DIANGO is
ROC-AUC
0.8144
0.8163
0.9305
0.913, while the highest ROC-AUC achieved by the baselines is 0.875, which is the result obtained by ABS on DOTA and F-RCNN. The average ROC-AUC of these baselines is nearly 0.592. Furthermore, our warm-up pre- pre-rrcesssing technique has proven effective in enhancing the scanning performance of the baselines, resulting in an improve- ment of 0.05 to 0.15 in terms of ROC-AUC and a significant reduction in overheads from over 3600 seconds to range of 275 to 2800 seconds. Despite these advancements, the enhanced versions of the baselines are still unable to surpass the performance of DIANGO, with ROC-AUC gaps ranging from 0.15 to 0.50.
Additionally, we have observed that the overheads incurred by DIANGO are consistently lower than those of NC, Pixel, and Tabor, even when these methods are equipped with our warm-up pre- processing technique. However, ABS generally achieves lower overheads than DIANGO, primarily because ABS only performs trigger inversion for the top- target classes according to the original configuration. Nonetheless, is important to note that DIANGO significantly outperforms ABS in terms of ROC-AUC, which is the more critical metric for our evaluation. We further illustrate the inverted trigger by DIANGO in Figure 6 is evident that the inverted trigger produced by DIANGO closely resembles the ground-truth injected trigger in terms of both shape and color. The overall success of DIANGO can be attributed to our well-designed warm-up up pre- pre-processing and the novel Gaussian Focus Loss 4 These techniques play crucial role in achieving the superior performance and efficiency demonstrated by DIANGO compared to the baselines.
Comparison to Meta Classifiers. Table 2 provides a comparison between meta classifiers and DIANGO. Since meta classifiers require only a few seconds to scan model, we have omitted the overheads from the table. Observe that baselines achieve decent results in the COCO dataset with over 0.81 ROC-AUC, while only make nearly 0.6 ROC-AUC on other datasets. This discrepancy can be attributed to the fact that there are over 60 models available in the COCO dataset, whereas there are only 30 models in the other datasets. Consequently, the performance of the meta classifiers heavily relies on the availability of large number of training models. Unfortunately, obtaining such
8

---

Figure 6: Visual similarity between GT and DIANGO inverted triggers.
Average No. of Pairs
7000
1.0
1.0
6000
0.9
0.8
5000
0.8
Pro-Processing
0.6
4000
04
w/o
W
3000
2602
0.6
0.4
DETR
2000
0.5
0.2
Faster R-CNN
1000
46.9
306
51.04
92.85
0.4
0.0
SSD
0
0.3
Syath
DOTA Dataset
COCO
100
150
200
250
Average No. of Pairs
(a) Evaluation on different datasets
(b) Evaluation on different model architectures
Figure Evaluation of label proposal pre-procee pre-processing
significant number of models in worldworld scenarios is challenging task. In contrast, DIANGO does not require access to training models. Moreover, it achieves ROC-AUC of over 0.91, outperforming the meta classifiers even when they are trained on sufficient number of models in the COCO dataset. This highlights the effectiveness and robustness of DIANGO in detecting backdoor attacks, surpassing the performance meta classifiers.
Evaluation on Advanced Object Detection Backdoor Attacks. We also evaluate DIANGO on two advanced object detection backdoor attacks 5 28). DIANGO achieves 0.8 and 0.9 ROC-AUC on detecting these attacks. Appendix presents more discussion.
4.2 Evaluation of Label Proposal Pre-processing
In this section, evaluate the effectiveness of our label proposal pre-processing introduced in Section 3.3 Results are presented in Figure 7 As shown Figure 7(a where the x-axis denotes the dataset, the left y-axis denotes the average number of pairs and the right one the TPR (True Positive Rate) the selected pairs. the case of an attacked model, if the selected pairs after pre-processing contain the ground-truth victim-target pair, it considered true positive. TPR is calculated the ratio of true positives to all positives. The green bars represent the average number of pairs without pre-processe proceesssss while the red bars represent the number of pairs after pre-processing. We observe that across all datasets, our pre-processing technique significantly reduces the number of scanning pairs by 83% 98%. The curves represent the TPR. showing that our pre-processing results in TPRs of over 88% across different datasets. These results indicate that our pre- processensigg technique effectively reduces the number of pairs remarkably low level while maintaining high detection rate. The experiment is conducted by selecting various values of / ranging from 1 to 10 (representing the top class labels) and w ranging from 0.1 to 0.8 (representing the frequency threshold). We record the average number of pairs and true positive rate (TPR) under this configuration. The results are depicted in Figure 7b) we observe that SSD and DETR models require fewer than 100 pairs to reach high TPR, while FaNN models require around 200 pairs. One potential reason for this difference that Faster-RCNN requires a two-stage training process where the backdoor signal less prominent compared to single-stage training methods in SSD and DETR.

---

Table 3: Ablation study
SSD
Faster-RCNN
DETR
Method
ROC-AUC
Overhead(s)
ROC-AUC:
Overhead(s)
ROC-AUC
Overhead(s)
DJANGO
L,0000
859.59
0,8750
2120.10
0,8750
227.55
DJANGO GF
0,7500
860.24
DJANGO Pre-Processing
0.6250
>3600
0,5000
23600
0.6250
>3600
4.3
Ablation Study
Our ablation experiments conducted the synthesized traffic sign dataset with 3 different model architectures. We include 8 models for each architecture, with half clean and half attacked.
There are two key designs of DJANGO to ensure the detection effectiveness, i.e., GF Loss and Pre- Processing. We conduct ablation study to assess the contribution of these components respectively. Results are shown in Table 3 where each row corresponds to a specific ablation method. The results clearly demonstrate the critical importance of both GF Loss and Pre-Pros Pre-Processing in DIANGO. Without these components, the detection performance of DJANGO significantly degrades by more than 0.13 ROC-AUC. Pre- Proceroccssin also plays crucial role in reducing the detection overhead. Please refer to sec D for hyper-panne parameter sensitivity analysis.
4.4 Adaptive Attack
In this section, we evaluate the perfor- mance of DIANGO in the context of adap- tive attacks. The effectiveness of DIANGO relies on the confidence levels assigned to the potential boxes, as these determine the coefficients used for trigger inversion. Con- sequently, an adaptive attack strategy could
Table 4: Adaptive attack
Atlack Conf.
AS1_0.1 ASR_0.6
ASR
DIANGO
No Attack
0.8766
0.0047
0.0023
0.0000
0.0
0.80
0,8700
1.0000
1,0000
0.9812
1.0
0.8708
1.0000
1,0000
0,4014
1.0
0.51
0,8685
1.0000
0,5728
0.0446
0.6
involve reducing the confidence of objects containing the trigger, thereby potentially causing DIANGO to assign lower coefficients to these poisoned boxes. To evaluate the adaptive attack scenario, we conduct experiments using a 5-class synthesized traffic sign dataset with the SSD model. We train several models with misclassification attacks, where the victim class to and the target class In order to simulate low confidence attacks, reduced the ground-truth confidence of the target class to values of 0.8, 0.6, and 0.51, while setting the confidence of the victim class to 0.2, 0.4, and 0.49, respectively. After sufficient training,
applied DJANGO these models. The results presented in Table 4 where we recorded the clean mAp and the ASRS at different score thresholds. For instance, ASR 0.T indicates the ASR when the prediction confidence is higher than 0.1. From the results, we can see that low confidence attacks lead to lower ASRs, especially when the score threshold for measuring the ASR is set high. Furthermore, DIANGO is successful in detecting attacked models with target confidences of 0.8 and 0.6, but fails when the target confidence is 0.51 (threshold set at 0.8). These results suggest that DIANGO effective against most low-confidence attacks, but its performance may degrade to some extent when the target confidence is very low.
5 Limitation
Our primary focus in this paper is on attacks that use static polygon triggers, which are more feasible in real-world scenarios. How to effectively inject more complex triggers [42 48 models still an open question. We leave it to future work.
48 in object detection
6 Conclusion
In this paper, we present DIANGO, the first trigger inversion framework on detecting backdoors in object detection models. It is based on novel Gaussian Focus Loss to tackle the loss misalignment issue in object detection trigger inversion. Extensive experiments demonstrate the effectiveness and efficiency of DJANGO compared to existing baselines.
10

---

7 Acknowledgment
We thank the anonymous reviewers for their constructive comments. We are grateful to the Center for AI Safety for providing computational resources. This research was supported, in part by IARPA TrojAl IN NSF 1901242 and 1910300, ONR N000141712045, N00014144000 and N000141712941 Any opinions, findings, and conclusions in this paper are those of the authors only and do not necessarily reflect the views of our sponsors.
References
[1] TrojAl Leaderboard. httpe: batpe://paggs: nist. gov/trojaā/
Nicolas Carion, Francisco Massa, Gabriel Synnaeve. Nicolas Usunier, Alexander Kirillov, and Sergey Zagonuyko. End-to-end object detection with transformers. Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I 16, pages 213-229. Springer, 2020.
[3] Shih-Han Chan, Yinpeng Dong. Jun Zhu, Xiaolu Zhang, and Jun Zhou. Baddet: Backdoor attacks on object detection. In Computer Vision ECCV 2022 Workshops: Tel Aviv, Israel, October 23 27, 2022. Proceedings, Part pages 396 412. Springer, 2023.
[4] Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lce, Ian Molloy, and Biplav Srivastava. Detecting backdoor attacks on deep neural networks by activation clustering. arXiv preprint arXiv:1811.0377
2018.
[5] Kangjic Chen, Xiaoxuan Lou, Guowen Xu, Jiwei Li, and Tianwei Zhang. Clcan-image backdoor: Attacking multi-label models with poisoned labels only. In The Eleventh International Conference Learning Representations, 2022.
[6] Shang-Tse Chen, Cory Comelius, Jason Martin, and Duen Horng Chau. Shapeshifter: Robust physi- cal adversarial attack on faster r-cnn object detector. In Machine Learning and Knowledge Discovery in Databases: European Conference. ECML PKDD 2018, Dublin, Ireland, September 10-14, 14, 2018. Proceedings. Part 18, pages 52 68. Springer, 2019.
[7] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.053: 2017.
[8] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arxiv: 1.t.:1772.0556.6 2017.
Siyuan Cheng, Yingqi Liu, Shiqing Ma, and Xiangyu Zhang. Deep feature space trojan attack of neural networks by controlled detoxification. In Proceedings the AAAI Conference on Artificial Intelligence, volume 35, pages 1 1148-1166, 2021.
[10] Siyuan Cheng, Guanhong Tao, Yingqi Liu, Shengwei An, Xiangzhe Xu, Shiwei Feng, Guangyu Shen, Kaiyuan Zhang, Qiuling Xu, Shiqing Ma, et al. Beagle: Forensics of deep learning backdoor attack for better defense. arXiv preprint arXiv:2301.06:0
2023.
[11] Jian Ding, Nan Xue, Gui-Song Xia, Xiang Bai, Wen Yang. Michael Yang, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang. Object detection in aerial images: A large-scale benchmark and challenges. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1-1, 2021.
[12] Bao Gia Doan, Ehsan Abbasnejad, and Damith Ranasinghe. Februus: Input purification defense against trojan attacks on deep neural network systems. In Annual Computer Security Applications Conference, pages 897-912, 2020.
[13] Dumitru Erhan, Christian Szegedy, Alexander Toshev, and Dragomir Anguelov. Scalable object detection using deep neural networks. In Proceedings of the IEEE conference computer vision and pattem recogmition, pages 2147-2154, 2014.
[14] Feng, Christian Schütz, Lars Rosenbaum, Heinz Hertlein, Claudius Glaeser, Fabian Timm, Werner Wiesbeck, and Klaus Dietmayer. Deep multi- modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems. 1341-1141113660 2020.

---

[15] Shiwei Feng, Guanhong Tao, Siyuan Cheng, Guangyu Shen, Xiangzhe Xu, Yingqi Liu, Kaiyuan Zhang. Shiqing Ma, and Xiangyu Zhang. Detecting backdoors in pre-trained encoders. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16352-16362, 2023.
[16] Yansong Gao, Change Xu, Derui Wang. Shiping Chen, Damith Ranasinghe, and Surya Nepal. Strip:
A defence against trojan attacks on deep neural networks. In Proceedings of the 35th Annual Computer Security Applications Conference, pages 113- 2019.
[17] Tianyu Gu, Kang Liu, Brendan Dolan-Gavite and Siddharth Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 2019.
[18] Wenbo Guo, Lun Wang, Xinyu Xing. Min Du, and Dawn Song. Tabor: highly accurate approach to inspecting and restoring trojan backdoors systems. arXiv preprint arXiv: 1908.0990001111 7633, 2019.
[19] Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. Spectre: Defending against backdoor attacks using robust statistics. In International Conference on Machine Learning, pages 4 4129-4139 PMLR, 2021.
[20] Jan Hosang, Rodrigo Benenson, and Bernt Schiele. Learning non m-ximmmmmmmmmmmmmmm-mmmmmm---mmm suppression. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages
2017.
[21] Khondoker Murad Hossain and Tim Oates. Backdoor attack detection in computer vision by applying matrix factorization on the weights of deep networks. arXiv preprint arXiv:2212.081221 2022.
[22] Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. Backdoor defense via decoupling the training process. arXiv preprint arXiv:2202.03422. 034423, 2022.
[23] Jinyuan Jia, Yupei Liu, Xiaoyu Can, and Neil Zhenqiang Gong. Certified rohustness of nearest neighbors against data poisoning and backdoor attacks. In Proceedings of the AAAT Conference Artificial Intelligence, volume 36, pages 9575-9583, 2022.
[24] Jinyuan Jia, Yupci Liu, and Neil Zhenqiang Gong. Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning. In 2022 IEEE Symposium on Security and Privacy (SP), pages 2043-2059, IEEE, 2022.
[25] Bijun Lee, Yang Wei, and I Yuan Guo. Automatic parking of self-driving car based on lidar. Int Arch. Photogramm. Remote Sens. Spat. Inf. Sei, 42:241-246, 2017
[26] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Anti-backdoon learning: Training clean models on poisoned data. In NeurIPS, 2021.
[27] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint (10.iv:2111009900 2021.
[28] Junyu Lin, Lei Xu, Yingqi Liu, and Xiangyu Zhang. Composite backdoor attack for deep neural network by mixing existing benign features. In Proceedings of the 2020 ACM SIGSAC Conference Computer and Communications Security, pages 113-131, 2020.
[29] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár, Focal loss for dense object detection, In Praceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.
[30] Tsung- Yi Lin, Michael Maire, Serge Belongic, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zimicck. Microsoft coco: Common objects in context. pages 740-755, 2014.
[31] Kang Liu, Brendan Dolan-Gavin and Siddharth Garg. Sine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pages 273-294. Springer, 2018.
[32] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scout Reed, Cheng- Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In Computer Vision-ECCV ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I 14, pages 21-37. Springer, 2016.
[33] Yingqi Liu, Scanning neural networks for by artificial brain stimulation. In Proceedings of the 2019 ACM SIGSAC Conference Computer and Communications Security, pages 1265- 1282, 2019.
Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and Xiangyu Zhang. Abs:
12

---

[34] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. 2017.
[35] Yingqi Liu, Guangyu Shen, Guanhong Tao, Shengwei An, Shiqing Ma, and Xiangyu Zhang. Piccola: Exposing complex backdoors in nlp transformer models. In 2022 IEFE Symposium on Security and Privacy (SP), pages 2042- IEEE, 2022.
[36] Yingqi Liu, Guangyu Shen, Guanhong Tao, Zhenting Wang, Shiqing Ma, and Xiangyu Zhang. Complex backdoor detection by symmetric feature differencing. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 1,500-15501,, 2022.
[37] Yunfei Liu, Xingjun Ma, James Bailcy, and Feng Lu, Reflection backdoor: A natural backdoor attack on deep neural networks. In European Conference on Computer Vision, pages 182 199. Springer, 2020.
[38] Chengxiao Luo, Yiming Li, Yong Jiang. and Shu-Tao Xia. Untargeted backdoor attack against object detec- tion. In ICASSP 2023-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5, IEEE, 2023,
[39] Hua Ma. Yinshan Li. Yansong Gao, Alsharif Abuadbba, Zhi Zhang, Anmin Fu, Hyoungshick Kim, Said F Al-Sarawi, Nepal Surya, and Derek Abbott. Dangerous cloaking: Natural trigger based backdoor attacks object detectors in the physical world. arXiv preprint arXiv:2201 08619. 2022.
[40] Shiqing Ma, Yingqi Liu, Guanhong Tao, Wen-Chuan Lee, and Xiangyu Zhang. Nic: Detecting adversarial samples with neural network invariant checking. In Proceedings of the 26th Network and Distributed System Security Symposium (NDSS 2019), 2019.
[41] Shunji Mori, Hirobumi Nishida, and Hiromitsu Yamada. Optical character recognition. John Wiley & Sons, Inc., 1999.
[42] Anh Nguyen and Anh Tran. Wanet- Wanet- innee-impereepptibe warping-basee backdoor attack. arXiv preprint arXiv:2102. 10369, 2021.
[43] Xiangyu Qi. Tinghao Xie, Jiachen Wang, Tong Wu, Saeed Mahloujifar, and Prateek Mitall, Towards a proactive [ML] approach for detecting backdoor poison samples. In 32nd USENIX Security Symposiumm (USENIX Security 23), pages 1685-1702, 2023.
[44] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint 114.v:180 2018.
[45] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. volume 28, 2015.
[46] Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized intersection over union: A metric and loss for bounding box regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 658-666, 2019.
[47] Ahmed Salem, Michael Backes, and Yang Zhang. Get model! model hijacking attack against machine lcarning models. arXiv preprint arXiv:21 11 04394, 2021.
[48] Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang. Dynamic backdoor attacks against machine learning models. arXiv preprint arXiv:2003 03675, 2020.
[49] Guangyu Shen, Yingqi Liu, Guanhong Tao, Shengwei An, Qiuling Xu, Siyuan Cheng, Shiqing Ma, and Xiangyu Zhang. Backdoor scanning for deep neural networks through k-arm optimization. In International Conference Machine Learning, 2021.
[50] Guangyu Shen, Yingqi Liu, Guanhong Tao, Qiuling Xu, Zhuo Zhang, Shengwei An, Shiqing Ma, and Xiangyu Zhang. Constrained optimization with dynamic bound-scaling for effective nlp backdoor defense. In International Conference on Machine Learning, pages 19879-19892. PMLR, 2022.
[51] G Sreenu and Saleem Durai. Intelligent video surveillance: review through deep learning techniques for crowd analysis. Journal of Big Data, 6(1):1-27, 2019.
[52] Weisong Sun, Yuchen Chen, Guanhong Tao, Chunrong Fang. Xiangyu Zhang, Quanjun Zhang, and Bin Luo. Backdooring neural code search. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguisties, 2023.
[53] Di Tang, XiaoFeng Wang, Haixu Tang, and Kehuan Zhang. Demon in the variant: Statistical analysis of dans for robust backdoor contamination detection. In 30th USENIX Security Symposium (USENIX Security 21), 2021.
13

---

[54] Guanhong Tao, Yingqi Liu, Guangyu Shen, Qiuling Xu, Shengwei An. Zhuo Zhang, and Xiangyu Zhang. Model orthogonalization: Class distance hardening in neural networks for better security. In 2022 IEEE Symposium on Security and Privacy (SP). IEEE, volume 3, 2022.
[55] Guanhong Tao, Guangyu Shen, Yinggi Liu, Shengwei An, Qiuling Xu, Shiqing Ma, Pan Li, and Xiangyu Zhang. Better trigger inversion optimization in backdoor scanning. In 2022 Conference Computer Vision and Pattern Recognition (CVPR 2022), 2022.
[56] Guanhong Tao, Zhenting Wang. Shiwei Feng, Guangyu Shen, Shiqing Ma, and Xiangyu Zhang. Distribu- tion preserving backdoor attack in supervise learning. In 2024 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, 2024.
[57] Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks, Advances in neural information processing systems, 31, 2018.
[58]
Alexander Turner, Dimitris Tsipras, and Aleksander Madry. Clean-label backdoor attacks. 2018.
[59] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pages 707-723. IEEE, 2019.
[60] Lun Wang, Zaynah Javed, Xian Wu, Wenbo Guo, Xinyu Xing, and Dawn Song. Backdoorl: Backdoor attack against competitive reinforcement learning. arXiv preprint arXiv:2105.00579, 2021.
[61] Tong Wang, Yousong Zhu, Chaoyang Zhao. Wei Zeng, Yaowei Wang. Jinqiao Wang, and Ming Tang. Large batch optimization for object detection: Training coco in 12 minutes. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part XXI 16, pages 481 496. Springer, 2020.
[62] Zhenting Wang, Kai Mei, Hailun Ding, Juan Zhai, and Shiqing Ma. Rethinking the reverse-enginccring of trojan triggers. In Alice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[63] Zhenting Wang, Kai Mei, Juan Zhai, and Shiqing Ma. Unicorn: A unified backdoor trigger inversion framework. arviv preprint 2xXi:2304.002886 2023.
[64] Zhenting Wang, Juan Zhai, and Shiqing Ma. Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15074-15084, 2022.
[65] Dongxian Wu and Yisen Wang. Adversarial neuron pruning purifies backdoored deep models. Advances in Neural Information Processing Systems, 34: 14913-111692 2021.
[66] Zuxuan Wu, Ser-Nam Lim, Larry Davis, and Tom Goldstein. Making an invisibility cloak: Real world adversarial attacks on object detectors. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IV 16, pages 1-17. Springer, 2020.
[67] Chong Xiang, Arjun Nitin Bhagoji, Vikash Sehwag, and Prateek Mittal. Patchguard: A provably robust de- fense against adversarial patches via small receptive fields and masking. In USENIX Security Symposium, pages
2021.
[68] Chong Xiang, Saced Mahloujifar, and Prateck Mittal. |PatchCleanser): Certifiably robust defense against adversarial patches for any image classifier. In 31 USENIX Security Symposium (USENIX Security 22), pages 2065-2082, 2022.
[69] Xiaojun Xu, Qi Wang, Huichen Li. Nikita Borisov, Carl A Gunter, and Bo Li. Detecting ai trojans using meta neural analysis. In 2021 IEEE Symposium Security and Privacy (SP), pages 103-120. IEEE, 2021.
[70] Wenkai Yang. Yankai Lin, Peng Li, Jie Zhou, and Xu Sun. Rethinking stealthiness of backdoor attack against nip models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural I Language Processing ( (Volume I: Long Papers), pages 5543-5557, 2021.
[71] Mingjun Yin, Shasha Li, Chengyu Song, M Salman Asif, Amit Koy-Chowdhury, and Srikanth Krishnamurthy, Ade: Adversarial attacks against object detection that evade context consistency checks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 9278-3287. 2022.
14

---

[72] Kaiyuan Zhang, Guanhong Tao, Qiuling Xu, Siyuan Cheng, Shengwei An, Yingqi Liu, Shiwei Feng. Guangyu Shen, Pin- Yu Chen, Shiqing Ma, and Xiangyu Zhang. FLIP: A provable defense framework backdoor mitigation federated learning. In The Eleventh International Conference on Learning Representations, 2023.
[73] Zhuo Zhang, Guanhong Tao, Guangyu Shen, Shengwei An, Qiuling Xu. Yingqi Liu, Yapeng Ye, Yaoxuan Wu, and Xiangyu Zhang. Pelican: Exploiting backdoors of naturally trained deep learning models in binary code analysis. 2023.
[74] Zhong Qiu Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu. Object detection with deep learning: review. IEEE transactions on neural networks and leaming systems, 30(11):3212-3 -2322, 2019.
15

---

Appendix
A
Details of datasets and architectures
A.1 Object Detection Image Dataset
COCO (Common Objects in Context) [30] dataset is widely used for object detection tasks. It. contains 80 object categories, including people, animals, vehicles and more. Each image can contain multiple instances of objects, providing ample opportunities for training and evaluating models capable of detecting and segmenting objects in complex scenes,
Synthesized Traffic Sign dataset is designed by TrojAI (I] which focuses on traffic sign detection, featuring various types of traffic signs commonly encountered in real-world scenarios. There are in total over 4000 different traffic signs. Each model is trained on randomly sampled subset of classes. The number of classes within these subsets exhibits variability, ranging from as few as 2 to a maximum of 128.
DOTA (Detection in Aerial Images) dataset is designed for object detection in aerial images which consists of high-resolution images captured by aerial platforms. It contains 18 categories, including plane, ship, storage tank, baseball diamond and more. Its large-scale, fine-grained annotations, and challenging scenarios make it an ideal benchmark for evaluating and developing algorithms capable of detecting objects in aerial images accurately.
A.2 Architecture
We evaluate our method on three well- model architectures: i.e., SSD 32 Faster-RCNN 45 and DETR 2 SSD (Single Shot MultiBox Detector) [32] is a popular object detection model which utilizes a series of convolutional layers to detect objects at multiple scales and aspect ratios. Faster-RCNN [45) is another widely adopted object detection model that combines region proposal generation with region-based CNN for object detection. DETR (DEtection TRansformer) 22] is sfte-off-thean object detection model that utilizes a transformer based architecture. It replaces the conventional two-stage approach with single-stage end-to-end detection framework.
A.3 Model Dataset
TrojAI I initiative, spearheaded by IARPA, encompasses multi-year, multi-round program. Its overarching objective revolves around the development of scalable and dependable automatic backdoor detection tools, specifically targeting the identification of backdoor trojans within Deep Learning models across diverse modalities. Presently, the program consists of total of 13 rounds, each with distinct focuses and tasks. The initial four rounds and the eleventh round center their efforts on detecting trojans present in image classification models. In contrast, rounds five through nine concentrate on transformer models employed in various NLP tasks, including Sentiment Analysis, Named Entity Recognition, and Question Answering. Round twelve dedicates itself to the detection of backdoors in neural network-based PDF malware detection. Finally, rounds ten and thirteen direct their attention towards object detection models. For the evaluation of models, exclusively utilize the training sets from rounds 10 and 13. As shown in Table our evaluation entails 72 models trained on the Synthesis Traffic Sign dataset, encompassing all three model architectures. Among these models, 48 are benign, while 24 are deliberately poisoned, with an equal distribution of triggers for misclassification and evasion. Concerning the DOTA models, there exist two architectures, namely SSD and Faster-RCNN, resulting in a total of 24 models, including 16 benign models and 4 each poisoned with misclassification and evasion triggers. All COCO models adopt the SSD architecture. with distribution of 36 clean models and 18 models poisoned by both misclassification and evasion triggers. To provide further elucidation,it is important note that the poison rate varies within the range of 0.1 11 to 8% across diverse models sourced from TrojAl r10 and r13. Similarly, the trigger size exhibits range of 1x1 to 22x22, representing a scale of 0.001 % to 0.7% relative to the input dimensions. Pertaining to the hyper-paramette
utilized in model training, the learning rate is stochastically assigned, spanning from 1.56e-8 to le-4 across different models. The number of epochs for training spans from 6 to 100, while the batch size ranges from 4 to 32. As for the model performance metrics, the average clean mAP across the models attains a value of 0.7979, while the average poison mAP stands at 0.7680. The trigger's polygonal structure is characterized by varying
16

---

edge counts, ranging from 3 to 8. Furthermore, each individual trigger is endowed with randomly generated color and texture attributes. Concrete settings can be found Round! E and Round
B Details of evaluation metrics
In our evaluation of backdoor detection methods, we employ four well-established metrics: Precision, Recall, ROC-AUC, and Average Scanning Overheads for each model. Precision quantifies the accuracy of detection method by measuring the proportion of correctly identified positive instances among all predicted positives. In our case, we consider attacked models as positive instances and benign models as negatives. A higher precision indicates a lower rate of falsely identifying benign models as attacked. Recall, on the other hand, assesses the effectiveness of the detection method in correctly identifying positive instances. It measures the proportion of true positives among all actual positives. A higher recall suggests that the detection method is capable of identifying a significant portion of attacked models. ROC-AUC (Receiver Operating Characteristic- Area Under the Curve) plots the true positive rate against the false positive rate at various threshold values and calculates the area under the curve. A value of indicates perfect classification, while value of 0.5 indicates that the method is no better than random guessing. We also consider the overhead of the detection method, which quantifies the average time required to scan single model. We use seconds (s) as the unit of measurement and set a maximum threshold of 1 hour (3600 s). If the scanning process exceeds 3600 seconds, it is terminated, and we rely on the existing results for making predictions. Low overhead signifies high efficiency of the method. By employing these four metrics, we aim to comprehensively evaluate the performance and efficiency of the backdoor detection methods. It is worth noting that the time limit we have set for scanning models is deliberately conservative when compared the thresholds established in different rounds of the TrojAI competition. For example, in round 13, participants are granted a generous 30-minute duration for scanning a single model. To surpass the official benchmarks set in each round, more aggressive and precise pre- pre-processing approach may be necessary.
C Details of Baseline Methods
In this section, we introduce more details of baseline methods, including NC 59 Tabor
18
ABS 33.. Pixel (55). Matrix Factorization(MF)) (2 and MNTD (69.
NC [59] adopts a specific trigger inversion approach for each class and considers model to be attacked if is able to generate an effective yet extremely small trigger for a target class. Tabor [8] enhances NC by incorporating additional well-designed regularization terms, such as penalties for scattered triggers, overlaying triggers, and blocking triggers. These additions aim to improve the reconstruction of injected triggers. Pixel [55] introduces novel inversion function that generates pair of positive and negative trigger patterns. This approach achieves better detection performance compared to NC. ABS 33 employs stimulation analysis to identify compromised neurons, which serves as guidance for trigger inversion. ABS considers a model to be attacked if it can invert a trigger that achieves high reconstructed ASR (REASR).
To the best of our knowledge, there is no existing detection methods for object detection models. Therefore, we perform forwarl but reasonable adaption to these existing methods designed on image classification tasks, such that they are able to work against backdoor attacks on object detection models. Specifically, the original objective of NC, Tabor, and Pixel is to invert small triggers while maintaining their effectiveness (high ASR). In our adaptation, we retain their design principles but re-define the ASR to align with object detection models, as explained in Section B.1 Additionally, we introduce a threshold for the size of inverted triggers, enabling the differentiation between benign and attacked models. For ABS, we adhere to its original technique but employ the re-defined ASR as the optimization goal, and use REASR as the decision score. By employing these adaptations, we aim to enhance the detection capabilities of these existing methods specifically for backdoor attacks on object detection models.
No modifications or adaptations are needed for meta classification-bass methods when applied to object detection models. MNTD 69 trains a set of queries and a classifier to discern the feature-space
17

---

Table 5: Dataset details
Model Source
Architecture
Number Models
Image Dataset
Round10
Round13
Faster-RCNN
Benign
Miscls Attack
Evasion Atlack
Synthesis Traffic Sign
/
48
12
12
DOTA
X
V
/
16
4
COCO
x
1.0
1.0
1.0
F-RONN DETR
SSD F-RONN DEIR
SSD F-RCNN DETR
0.875
0.875
0.875
+0,9
0.875
0.750
0.750
4..9
0.875
0.875
0.875
0.9
0.8
0.8
0.8
0,875
0.875
0.750
0.875
0,750
0.750
0,875
0.875
0.875
0..7
0.7
1.000
1.000
0.875
1.000
0.875
0.750
0.875
1,000
0.875
30x30
50x30
(a) IoU threshold
(b) Region size
(c) Score threshold
Figure 8: Hyper- Hyper- parameter sensitivity,
distinctions between clean and attacked models. MF 21 directly trains a classifier on model weight features using specialized feature extraction techniques, i.e., matrix factorization. These methods solely rely on the feature extraction networks commonly utilized in both image classification and object detection models. As result, MNTD and MF can be directly employed detect backdoor attacks in object detection models without the need for additional adjustments or modifications.
We collect the Precision, Recall, ROC-AUC and Overheads for each method across various datasets and model architectures. To ensure a fair comparison, we have conducted a search to determine the optimal thresholds for different decision scores associated with each method (trigger size for NC, Tabor, Pixel, REASR for ABS and output confidence for meta-classifiers) These thresholds are chosen maximize accuracy. Besides, a pair of victim-target label (100) for all inversion based baselines. For meta classification based methods that involve training, we have performed 5-fold coss-validation and reported the validation results exclusively.
set fixed number of optimization steps for scanning
D Hyper-parameter Sensitivity Analysis
To assess the sensitivities of the hyper-parameters used in DJANGO, we conduct experiments as described in Section
IOU Thresholds. We evaluate the IoU threshold used to calculate the ASR inverted triggers. The results summarized in Figure ((a) where each corresponds to different model architecture, and each column represents different choice of loU threshold. It can be observed that IoU thresholds of 0.3 and 0.5 generally yield good performance. However, threshold of 0.7 tends to degrade the performance, possibly due to the inverted triggers interfering with the bounding box predictions.
Region Size. The impact of different regional initialization sizes is evaluated and the results are presented in Figure 8(b) Among the various choices, a region size of 30x30 consistently achieved the best performance. This is because larger initialization sizes tend to result in more false positive cases.
Score Threshold. Different score thresholds are tested when computing the ASR of inverted triggers. The results, shown in Figure indicate that score threshold of 0.5 generally leads to the best performance across all model architectures. This choice represents a trade-off between false positives and false negatives. Higher score thresholds may introduce more false negatives, as the inverted trigger may not have high confidence similar to the injected one. On the other hand, lower score thresholds may result in more false positives. Thus, a moderate value of 0.5 provides the optimal balance.
18

---

Table 6: Hyper-parameter sensitivity for Gaussian Focus Loss initial values
ROC-AUC
(jp. 0v.)
Faster-RCNN
DETR
(0.1, 2)
1,00000
1,0000
0.8750
(0.1, 3)
1.0000
0,8750
0,8750
(0.2,
0.8750
0.7500
0,8750
(0.2.
0.8750
0,8750
0.7500
00.2, 3)
0.8750
0.8750
0.8750
Table 7: Effectiveness on different weight scheme distributions
Method
ROC-AUC
DJANGO GF Loss
0.9500
DJANGO + LP Loss
0.9000
Tabar
NC*
0.6000
Pixel*
0.7000
Table Effectiveness on complex attacks
Attack type
ROC-AUC
Object appearing Attack
0.8000
Composite Attack
0.9000
These experiments allowed gain insights into the sensitivities of the hyper- syper-parameters in DIANGO, enabling us to make informed choices for achieving optimal performance.
GF Loss Initialization. We evaluate the impact of two paraa--as (initial mean jús and variance o²) in the Eq. DIANGO. We randomly sample models (4 trojan and benign) for each architecture that was trained on the synthesized traffic sign dataset. Besides the default values
report in the paper (ūx = 0.1, 0g 2). we set 5 more groups of initial values and report the detection performance, As shown in Table 6 DIANGO remains effective under different initialization values.
Weight Scheme Distribution Choice. We attempted to substitute the Gaussian distribution with the Laplace distribution, another commonly employed probability distribution. Our experimentation involved 20 models trained on the traffic sign synthetic dataset, comprising 10 clean models and 10 models poisoned with misclassification triggers. The outcomes of these experiments are presented in T is evident that with the Laplace focus loss, DIANGO maintains a high level of effectiveness, achieving ROC-AUC of 0.9, whereas baseline methods only achieve 0.7. We hypothesize that the 0.05 performance decline, in comparison to the Gaussian Focus Loss, may be attributed to the sharper peak of the Laplace distribution when contrasted with the Gaussian distribution. Within our context, a reduced number of bounding boxes exhibiting moderate confidences are allocated larger coefficients in different stages, thereby potentially diminishing the emphasis compromised boxes.
E
Evaluation on Complex Object Detection Backdoor Attacks
We further evaluate DIANGO on two advanced object detection backdoor attacks: Object-appearing
and Composite attacks 5 3 28 Object-appearing triggers aim to yield target class object bounding
boxes on the background. According to our definition, the object-appearing attack is special case of the misclassification attack with the background @ class as the victim class. To demonstrate the effectiveness of DJANGO on object-appearing attacks, we evaluate DJANGO on Baddet models and 10 Clean-image backdoor models during rebuttal. For each type of attack, mix 5 clean models with poisoned models with object-appearing triggers. The evaluation results are shown Table DJANGO achieves 0.8 ROC-AUC on both Baddet and clean-image object appearing attacks.
For the composite attack, we conducted an evaluation of DIANGO using 10 models that were trained on the traffic sign synthesis dataset. This set included 5 clean models and 5 trojan models poisoned by composite attack. As indicated in Table g DIANGO achieved an 0.9 ROC-AUC for detecting composite attacks. It's worth noting that the composite attack does not rely explicit trigger.
19

---

Instead, it leverages clean object A to serve as the trigger for attacking another object B. Interestingly. DIANGO is capable of effectively reversing this process, essentially identifying a trigger that closely mimics the pattern of object A. This ability enables DIANGO to detect composite backdoors with high level of accuracy.
20


Reviews:
[{"review_id": "SV1E7455qN", "reviewer": "Reviewer_TXHx", "summary": "In this paper, the author proposes a new method for detecting backdoors in object detection models. The author finds that directly applying backdoor detection methods for classification models to object detection models results in a loss misalignment problem. Therefore, the author suggests assigning different weights to different bounding boxes during the trigger inversion optimization process. As such, they propose a new optimization loss function in the paper, Gaussian Focus Loss, to better recover triggers. Additionally, to reduce computational overhead, the author also proposes a pre-processing method to decrease the number of label pairs that need to be scanned.", "questions": "1. Is this method effective for composite types of triggers? (e.g., \"Composite Backdoor Attack for Deep Neural Network by Mixing Existing Benign Features\")\n2. Is the method proposed in this paper effective for other types of backdoors? (e.g. Backdoors in the physical world \"Dangerous Cloaking: Natural Trigger based Backdoor Attacks on Object Detectors in the Physical World\")\n3. The threat model requires some clean samples; are these samples selected from training samples? If not, would this impact the inversion effectiveness?\n4. The paper categorizes backdoor attacks on object detection into two types: misclassification and evasion. Does the misclassification attack include an object-appearing attack, where a bounding box that does not exist in the ground truth appears? ( \"BadDet: Backdoor Attacks on Object Detection\" and \"Clean-image Backdoor: Attacking Multi-label Models with Poisoned Labels Only\")", "limitations": "The author did not mention limitations in the article. Although this method can detect backdoors in object detection models, it is limited to polygon triggers, and other types of triggers have not been explored.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper summarizes and analyzes the loss misalignment phenomenon that occurs during trigger inversion in object detection models and proposes a new optimization function based on this phenomenon. Compared to directly applying existing trigger inversion methods for classification models to object detection models, the method proposed in this paper can achieve better results at a lower cost.\n2. The pre-processing method proposed in this paper, based on backdoor leakage, reduces the cost of scanning the model. This method can also be applied to other backdoor detection scenarios.\n3. The experimental results in this paper support the effectiveness of the proposed method.\n4. The paper is well-organized and easy to follow.", "weaknesses": "1. Some hyperparameters mentioned in the paper have not been carefully studied for their impact on the method's effectiveness, such as the two parameters in Eq.4 and the h in label pair pre-processing.\n2. All models tested in the paper are selected from TrojanAI, but the paper does not detail the poisoned conditions of the models, such as the poisoning rate, trigger size, and trigger type.\n3. There are no recovered trigger samples given in the paper.\n4. Code is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1] Hyperparameter settings and sensitivity** \n\nIn Section 4.2 (lines 326-332), we discuss and evaluate the sensitivity of hyperparameters in the pre-processing procedure ($h$ and $\\omega$), and the main results are shown in the main text Figure 5b.  Recall the values of $h$ and  $\\omega$ strike the balance between efficiency and effectiveness of the pre-processing procedure\n\nWe conducted experiments by setting $h$ from 1 to 10 and $\\omega$ from 0.1 to 0.8. For each combination of $h$ and $\\omega$, we recorded the True Positive Rate (the ratio of selecting ground-truth label pairs) and the corresponding number of selected pairs in total for each model architecture. Figure 5b illustrates the results. We observed that different model architectures require slightly different preprocessing hyper-parameter settings to achieve the optimal trade-off between efficiency and effectiveness. However, the True Positive Rate (TPR) quickly saturates after selecting a reasonably small portion of label pairs (less than 200 out of 2602 pairs for all architectures). To provide more detailed information, we will include specific values of $h$ and $\\omega$ at each changing point of Figure 5b in the revised version.\n\nWe also evaluate the Django performance under different settings of IoU threshold, initialization region size and score threshold. Please find more details in Appendix D. \n\nDuring the rebuttal, we further evaluate the impact of two hyperparameters(initial mean $\\mu$ and variance $\\sigma$) in the Eq.4 on Django. We randomly sample 8 models (4 trojan and 4 benign) for each architecture that was trained on the synthesized traffic sign dataset. Besides the default values we report in the paper ($\\mu$=0.1,$\\sigma$=2), we set 5 more groups of initial values and report the detection performance.  As shown in rebuttal PDF Table 2, Django remains effective under different initializations. \n\n------------\n\n**[W2] Model configuration details**\n\nIn Appendix A, we provide comprehensive details regarding the datasets and model architectures employed in this study. To provide further elucidation, it is important to note that the poison rate varies within the range of 0.1% to 8% across diverse models sourced from TrojAI r10 and r13. Similarly, the trigger size exhibits a range of 1x1 to 22x22, representing a scale of 0.001% to 0.7% relative to the input dimensions. Pertaining to the hyper-parameters utilized in model training, the learning rate is stochastically assigned, spanning from 1.56e-08 to 1e-04 across different models. The number of epochs for training spans from 6 to 100, while the batch size ranges from 4 to 32. As for the model performance metrics, the average clean mAP across the models attains a value of 0.7979, while the average poison mAP stands at 0.7680. We assess TrojAI models using polygon triggers, encompassing two distinct attack types: misclassification and evasion attacks. The trigger's polygonal structure is characterized by varying edge counts, ranging from 3 to 8. Furthermore, each individual trigger is endowed with randomly generated color and texture attributes. \n\nPlease refer Appendix C for the detail settings of baseline methods\n\n------------\n\n**[W3] Recovered trigger samples**\n\nWe attach the inverted trigger from TrojAI round13 model ID-7 and ID-120 in rebuttal PDF Figure 1. Compared to ground truth triggers, Django inverted triggers have good visual similarity. \n\n------------\n\n**[W4] Missing code**  \n\nWe will release the code upon publication.\n\n------------\n\n**[Q1] Evaluation on composite attack**  \n\nDuring the rebuttal phase, we conducted an evaluation of Django using 10 models that were trained on the traffic sign synthesis dataset. This set included 5 clean models and 5 trojan models poisoned by composite attack. As indicated in Table 5 of the rebuttal PDF, Django achieved an 0.9000 ROC-AUC for detecting composite attacks. It's worth noting that the composite attack does not rely on an explicit trigger. Instead, it leverages a clean object A to serve as the trigger for attacking another object B. Interestingly, Django is capable of effectively reversing this process, essentially identifying a trigger that closely mimics the pattern of object A. This ability enables Django to detect composite backdoors with a high level of accuracy.\n\n------------\n\n**[Q2] Evaluation on Physical attack** \n\nDangerous Cloaking uses homemade dataset with t-shirt as trigger to realize the physical backdoor attack. Unfortunately, the paper did not release the code and dataset for our reproduction. We will further contact the authors or collect samples by ourselves and evaluate the effectiveness of Django in the revision. \n\n------------\n\n**[Q3] Source of the inversion samples**\n\nAll the samples we used for Django reverse-engineering are from the validation set. To evaluate the effect of different sources, we conduct the experiments on 10 models trained on COCO dataset (5 clean and 5 poison).  For each model, we run Django twice with samples from different sources, i.e., randomly sample 10 images from each class in the training set and validation set separately. The detection performance is shown in rebuttal PDF Table 4. We can see that Django is not sensitive to the source of the clean images used for inversion. We will clarify in the revision. \n\n------------\n\n**[Q4] Evaluation on object-appearing attack**\n\nYes, according to our definition, the object-appearing attack is a special case of the misclassification attack with the background \\mathcal{empty} class as the victim class. To demonstrate the effectiveness of Django on object-appearing attacks, we evaluate Django on 10 Baddet models and 10 Clean-image backdoor models during rebuttal. For each type of attack, we mix 5 clean models with 5 poisoned models with object-appearing triggers. The evaluation results are shown in rebuttal PDF Table 5. Django achieves 0.8 ROC-AUC on both Baddet and clean-image object appearing attacks. "}, {"review_id": "Tw3GH1EUhb", "reviewer": "Reviewer_sH6B", "summary": "The key idea of this paper is to propose Django, an adaptation of the trigger reverse-engineering technique used for detecting backdoored models in classification to object detection models. The paper shows that existing trigger reverse-engineering techniques are ineffective in object detection and finds that loss misalignment is the primary reason for the less effectiveness. The paper then proposes a new loss function that leverages Focal loss, a well-studied loss function in object detection, against compromised object detection models. In evaluation, Django is more effective than the baseline methods in backdoor detection with less computational overhead. The paper shows Django is effective against a few adaptive attacks.", "questions": "No question.", "limitations": "The paper discusses the limitations in Line 333--342.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes a backdoor detection method for object detection models.\n2. The paper presents why existing detection (trigger reverse-engineering) methods are ineffective.\n3. Django implements a novel loss function that addresses the problems identified in 2.\n4. The paper runs an evaluation against existing techniques and adaptive attacks. \n\nI like the contribution of this paper proposing a backdoor detection method for object detection models. I don't think this is a groundbreaking contribution, but it is also not a trivial contribution. I believe Django could be a good baseline for backdoor detection in object detection.", "weaknesses": "1. I expect the related work could be a bit more comprehensive in backdoor attacks and detection \"in object detection.\"\n2. The evaluation mostly compares the effectiveness against trigger reverse-engineering techniques; comprehensiveness would be nice.\n\nRelated work\n\n> Since the paper addresses backdooring in object detection, the related work should discuss the backdoor attacks and defenses in object detection. As backdooring has been studied for a while, there are many backdoor attack papers and proposed defenses. It would also be nice for readers to identify the novelty of this work much more clearly.\n\nEvaluation\n\n> I believe that there are multiple approaches to backdoor defenses (for example, I can think of a simple way like STRIP). Hence, I'd like to see a comprehensive categorization of backdoor defenses (perhaps in the related work section) and a comparison of Django against those missing in the current evaluation. The focus is not to show Django outperforms all the defenses but to show where Django lies in backdoor detection and how effective/ineffective Django is compared to different approaches.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1] More comprehensive related work in object detection** \n\nWe thank the reviewer for the detailed suggestion regarding related work. We will add a new section to introduce more detail regarding backdoor attack and defense in the context of object detection. \n\n-----\n\n**[W2] Comparison with non reverse-engineering based backdoor detection methods** \n\nPlease refer to **global response A1**. \n"}, {"review_id": "8pqjQOAe09", "reviewer": "Reviewer_5mmp", "summary": "This paper proposes a novel trigger detection method that works on object detection tasks. The proposed method leverages a novel Gaussian Focus Loss to calibrate the misaligned loss during trigger inversion. The evaluations presented in the paper demonstrate that the proposed method is effective in different settings including against adaptive attack.", "questions": "Please refer to the weaknesses.", "limitations": "This paper has not discussed the limitations. A potential limitation of the proposed method may lie in the performance against different types of triggers (e.g., dynamic triggers). ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper has revealed and discussed a generalized phenomenon in the object detection model trigger inversion, i.e., the misalignment of CE Loss and ASR. From my perspective, this finding can shed light on future works on defending against backdoor attacks in object detection tasks and also propose a good question for the backdoor attacks in the setting. \n2. The proposed method has leveraged the finding mentioned above and solved the challenges when applying such a finding, I think the proposed method has good soundness.\n3. The evaluations are comprehensive and the proposed method is effective. In addition, the computational cost is quite acceptable.\n4. To sum up, the proposed method is simple yet effective while the discovered phenomenon can provide new perspectives in this field.", "weaknesses": "The limitations of the proposed method are not discussed in this paper.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1] No limitation discussed**\n\nAs discussed in the main text (lines 107-113), our primary focus in this paper is on attacks that use static polygon triggers, which are more feasible in real-world scenarios. How to effectively inject more complex attacks such as WaNet[38], DFST[7] and dynamic attack[43] into object detection models is still an open question. We leave it to future work.  We're planning to enhance the clarity of our writing and will also include a section that highlights the limitations of our approach."}, {"review_id": "J08bXnhdyc", "reviewer": "Reviewer_VAnQ", "summary": "This paper investigates the problem of detecting trojans in the context of Objection Detections. The authors first observed that the poison effect can vary significantly across bounding boxes in object detection models due to its dense prediction nature, which leads to a misalignment issue for existing trigger reverse-engineering methods. To solve this problem, they proposed the Django framework built upon the Gaussian weighting scheme to prioritize more vulnerable victim boxes. Extensive empirical results are conducted several objects, datasets, and models.", "questions": "- The different weighting schemes (of the poison effect) on the bounding boxes seem to resemble the observations in CNN that different magnitudes of neurons for clean and backdoor inputs. Do you have any comments on this?\n", "limitations": "Please see my comments above", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem of detecting backdoors in Objective Detections is of sufficient interest to the NeurIPS community.\n\n- The paper is well-written and easy to follow.\n\n- The empirical evaluations over three objects, 16 detection image datasets, three model architectures, and two types of attacks are convincing.", "weaknesses": "- Why not consider non-trigger-inversion-based methods for detection? Regarding backdoor detection problems in the case of classification problems, many methods do not rely on inverting the trigger, e.g., the STRIP.  I am curious about the performance of applying these methods for detection.\n\n- Do you have any theoretical comprehension of the choice of gaussian? How about some other options?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1] Comparison with non reverse-engineering based backdoor detection methods** \n\nPlease refer to **global response A1**.\n\n------\n\n**[W2] Theoretical comprehension of the choice of gaussian** \n\nPlease refer to **global response A2**.\n\n------\n\n**[Q1] Correlation between weight schemes and compromised neuron magnitude**\n\nWe believe our findings are consistent with the disparity in neuron magnitudes observed between clean and backdoored samples in CNN models [29]. In the context of object detection models, we suspect that the elevated activation values of target labels within compromised bounding boxes can be attributed to the pronounced magnitudes of compromised neurons within intermediate layers. \n\n\n\n\n"}, {"review_id": "4aBiegl2Yh", "reviewer": "Reviewer_tRsV", "summary": "The paper proposes the first object detection backdoor detection framework Django (Detecting Trojans in Object Detection Models via Gaussian Focus Calibration). It leverages a dynamic Gaussian weighting scheme that prioritizes more vulnerable victim boxes and assigns appropriate coefficients to calibrate the optimization objective during trigger inversion.\n\nThe experimental results show the superiority of Django over some state-of-the-art baselines.", "questions": "Please see the questions in the weaknesses section.\n\nSome further questions for improvements:\n\nWhat is the proposed trigger inversion-based backdoor detection framework? Can the authors show a visualization of a flow or a pipeline or an algorithm of this framework including the main components?\n\nCan the authors explain the setting of the training process (e.g., the models configuration and the values of any hyper-parameters used) of the proposed method and baselines?", "limitations": "The limitations of the work are not mentioned in the paper.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The authors found that the poison effect can vary significantly across bounding boxes in object detection models due to its dense prediction nature, leading to an undesired optimization objective misalignment issue for existing trigger reverse-engineering methods.\n\nThe authors propose a trigger inversion-based backdoor detection framework for object detection: DJANGO (Detecting Trojans in Object Detection Models via Gaussian Focus Calibration). It features a Gaussian Focus Loss to calibrate the misaligned loss during inversion by dynamically assigning weights to individual boxes based on their vulnerability. \n\nThe authors claim that equipped with a label proposal pre-processor, DJANGO is able to quickly identify malicious victim-target labels and effectively invert the injected trigger lies in the backdoored model.\n\nThe experimental results show Django outperforms some state-of-the-art baselines on four metrics: Precision, Recall, ROC-AUC, and Average Scanning Overheads for each model.", "weaknesses": "Can the authors explain why the authors claim that Django is the first trigger inversion-based backdoor detection framework for object detection while there have been also many popular and effective trigger inversion backdoor scanning architectures/methods such as [51, 49, 15, 31]? Do you mean that the proposed framework is the first one for object detection? How about [3] mentioned in the paper?\n\nInspired by the Focal loss [25], the authors propose Gaussian Focus Loss. However, It is unclear (i) how the proposed Gaussian Focus Loss work to solve object detection backdoor detection, and (ii) how the parts in this Gaussian Focus Loss interact to dynamically capture a set of vulnerable boxes that have not been flipped yet, and assign a large coefficient to encourage the transition.\n\nThe setting of the training process of the proposed framework and baselines are not mentioned clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1] Clarification of the Django** \n\nWe are sorry for the confusion. [51, 49, 15, 31] are initially designed for detecting backdoor on image classification models. According to our evaluation(main text, line 268-line 291, Table 1), existing trigger inversion techniques all have limited performance when adapted on object detection models due to the misalignment issue presented in main text Sec 3.1.  In Baddet[3], the proposed Detection Cleanse is a backdoor sample detection technique, which has a completely different threat model and goal with model-level backdoor detection. Please refer to a more detailed discussion in global response A1.  To the best of our knowledge, the proposed Django is the first model-level trigger inversion-based backdoor detection framework tailored for deep learning object detection models. We will further clarify in the revision. \n\n---\n\n**[W2] Intuition of Gaussian Focus Loss**\n\nPlease refer to **global response A2**.\n\n---\n\n**[W3&Q2] Model & Baseline configuration details** \n\nIn Appendix A, we provide comprehensive details regarding the datasets and model architectures employed in this study. To provide further elucidation, it is important to note that the poison rate varies within the range of 0.1% to 8% across diverse models sourced from TrojAI r10 and r13. Similarly, the trigger size exhibits a range of 1x1 to 22x22, representing a scale of 0.001% to 0.7% relative to the input dimensions. Pertaining to the hyper-parameters utilized in model training, the learning rate is stochastically assigned, spanning from 1.56e-08 to 1e-04 across different models. The number of epochs for training spans from 6 to 100, while the batch size ranges from 4 to 32. As for the model performance metrics, the average clean mAP across the models attains a value of 0.7979, while the average poison mAP stands at 0.7680. We assess TrojAI models using polygon triggers, encompassing two distinct attack types: misclassification and evasion attacks. The trigger's polygonal structure is characterized by varying edge counts, ranging from 3 to 8. Furthermore, each individual trigger is endowed with randomly generated color and texture attributes. \n\n\n---\n\n**[Q1] Workflow of Django** \n\nWe extend our gratitude to the reviewer for highlighting the presentation issue in our paper. In response, we have introduced a comprehensive overview figure (rebuttal PDF Figure 2) during the rebuttal, which provides a visual representation of our Django framework. Django operates through two distinct stages:\n\n1. Compromised Label Proposal via Backdoor Leakage: In this initial stage, we propose  a lightweight screening algorithm that swiftly identifies a small subset of victim-target label pairs. This selection is grounded in the observation that the behavior of a poisoned model on victim samples tends to shift towards the target label, even in the absence of the backdoor trigger itself. a.k.a backdoor leakage. Further elaboration can be found in Section 3.3 (main text, lines 223-233).\n\n\n2. Trigger Inversion via Gaussian Focus Loss: The second stage involves trigger inversion, where each chosen label pair undergoes a precision-oriented process using our proposed Gaussian Focus Loss. This process precisely and dynamically captures a small fraction of compromised bounding boxes, assigning them larger coefficients when calculating the inversion objective function. The norm of the inverted trigger from each candidate pair serves as a determinant of the model's benignity. Additional insights are available in Section 3.2 (main text, lines 184-222).\n\n------------\n\n**[Q3] Hyperparameter settings and sensitivity** \n\nIn Section 4.2 (lines 326-332), we discuss and evaluate the sensitivity of hyperparameters in the pre-processing procedure ($h$ and $\\omega$), and the main results are shown in the main text Figure 5b.  Recall the values of $h$ and  $\\omega$ strike the balance between efficiency and effectiveness of the pre-processing procedure\n\nWe conducted experiments by setting $h$ from 1 to 10 and $\\omega$ from 0.1 to 0.8. For each combination of $h$ and $\\omega$, we recorded the True Positive Rate (the ratio of selecting ground-truth label pairs) and the corresponding number of selected pairs in total for each model architecture. Figure 5b illustrates the results. We observed that different model architectures require slightly different preprocessing hyper-parameter settings to achieve the optimal trade-off between efficiency and effectiveness. However, the True Positive Rate (TPR) quickly saturates after selecting a reasonably small portion of label pairs (less than 200 out of 2602 pairs for all architectures). To provide more detailed information, we will include specific values of $h$ and $\\omega$ at each changing point of Figure 5b in the revised version.\n\nWe also evaluate the Django performance under different settings of IoU threshold, initialization region size and score threshold. Please find more details in Appendix D. \n\nDuring the rebuttal, we further evaluate the impact of two hyperparameters(initial mean $\\mu$ and variance $\\sigma$) in the Eq.4 on Django. We randomly sample 8 models (4 trojan and 4 benign) for each architecture that was trained on the synthesized traffic sign dataset. Besides the default values we report in the paper ($\\mu$=0.1,$\\sigma$=2), we set 5 more groups of initial values and report the detection performance.  As shown in rebuttal PDF Table 2, Django remains effective under different initializations. \n\n------------\n\n"}]

Author rebuttal:
**Global Response**

We extend our sincere gratitude to all the esteemed reviewers for their invaluable and astute feedback. In the Global Response section, we shall diligently address the common questions raised and supplement the document with additional figures and tables to enhance clarity. Besides, we will reply to the specific inquiries and suggestions of each reviewer individually. “Q”, “W” and “L” indicate the question, weakness and limitation mentioned by the corresponding reviewer. e.g. “Q2@sH6B” denotes the second question brought by the reviewer sH6B. “A” denotes our answer. Tables and figures detailing the additional experiments can be found in the rebuttal PDF. 

------------------------

**[W2@sH6B, W1@VAnQ] Clarification of our threat model and evaluation of other types of backdoor defenses**

**A1**:  As discussed in the threat model (main text lines 107-113), our proposed Django framework falls under the category of backdoor model detection, aligning with a line of existing works[49, 15, 51]. In this scenario, defenders are required to classify the subject model as trojaned/benign with access only to a limited set of clean samples but no poisoned samples. Model-level backdoor detection techniques are usually executed offline before model deployment. 

On the contrary, the techniques mentioned by the reviewers, such as STRIP[13] and Detection Cleanse[3], belong to another type of backdoor defense known as backdoor sample detection. This defense approach operates under a completely different threat model and serves a distinct purpose. Defenders aim to discriminate trojaned/benign input samples on-the-fly, requiring access to the poison samples.  Therefore, it may not be appropriate  to compare our proposed Django with STRIP and Detection Cleanse. We will further clarify in the revision. 


However, during our rebuttal, we encountered a related work FreeEagle[1] that extends STRIP to encompass trojaned model detection techniques. In line with the paper's configuration, we replicated the setup and conducted a comparative analysis between STRIP and Django. Our evaluation involved a random sampling of 8 clean models and 8 poisoned models, all trained on the traffic sign synthetic dataset. Rebuttal PDF Table 1 presents evaluation results. We can see that Django is able to achieve 0.9375 ROC-AUC while STRIP  has 0.6250. It is because STRIP is not capable of detecting label specific triggers[1]. Moreover, it's important to note that STRIP's superimposing operation has the potential to introduce additional objects onto the fused image. This is particularly significant due to the dense output nature of object detection models. Consequently, this operation may influence the entropy scores of bounding boxes, which could ultimately lead to a decrease in its overall detectability.

In addition to reverse-engineering based backdoor model detection techniques, we also compare Django with two state-of-the-art meta-classifier based techniques: MNTD[60] and MF[17]. As shown in the main text Table 2, Django outperforms the two baselines by large margins on all three datasets. Please refer to the detailed discussion from main text Line 292-313.


[1] Fu, Chong, et al. "FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases." arXiv preprint arXiv:2302.14500(2023).

------------------------

**[W2@VAnQ, W2@tRsV] Intuition and theoretical comprehension of Gaussian Focal Loss**

**A2**: The selection of the Gaussian distribution stems from empirical observations made during our exploration of the underlying reasons behind the misalignment issue in evasion and misclassification attacks(main text, lines 162-173). This choice is supported by the discussions in Section 3.2 (main text, lines 199-209), where we highlight the constraints of the Focal Loss method.

Specifically, we observed the cause of misalignment is due to the unequal poisoning effect on individual bounding boxes. Only boxes with moderate confidence shall be focused dynamically through the entire inversion procedure. Therefore, naive Focal Loss is insufficient as it only focuses on boxes with low confidence, i.e. hard examples. 

Furthermore, we observe our goal is highly aligned with the natural bell shape of the probability density function of gaussian distribution. Through adjusting the mean and variance during inversion, the proposed gaussian weighting scheme can dynamically capture the boxes with moderate confidence scores and assign larger coefficients to encourage the transition. 

Please note that any distributions characterized by centralized peaks are suitable for our intended purpose. During the rebuttal, we attempted to substitute the Gaussian distribution with the Laplace distribution, another commonly employed probability distribution. Our experimentation involved 20 models trained on the traffic sign synthetic dataset, comprising 10 clean models and 10 models poisoned with misclassification triggers. The outcomes of these experiments are presented in rebuttal PDF Table 3. It is evident that with the Laplace focus loss, Django maintains a high level of effectiveness, achieving an ROC-AUC of 0.9, whereas baseline methods only achieve 0.7. We hypothesize that the 0.05 performance decline, in comparison to the Gaussian Focus Loss, may be attributed to the sharper peak of the Laplace distribution when contrasted with the Gaussian distribution. Within our context, a reduced number of bounding boxes exhibiting moderate confidences are allocated larger coefficients in different stages, thereby potentially diminishing the emphasis on compromised boxes. We intend to present a more comprehensive array of experiments in the revised version.




Meta-review: The rebuttal addressed most of the concerns raised by the reviewers, so all reviewers are in favor of accepting this paper. Hence, the AC recommends accepting it.
Decision: Accept (poster)