SlowGuess/NeurIPS_2023_OCR huggingface dataset:
NC [59] adopts a specific trigger inversion approach for each class and considers a model to be attacked if it is able to generate an effective yet extremely small trigger for a target class. Tabor [18] enhances NC by incorporating additional well-designed regularization terms, such as penalties for scattered triggers, overlaying triggers, and blocking triggers. These additions aim to improve the reconstruction of injected triggers. Pixel [55] introduces a novel inversion function that generates a pair of positive and negative trigger patterns. This approach achieves better detection performance compared to NC. ABS [33] employs a stimulation analysis to identify compromised neurons, which serves as guidance for trigger inversion. ABS considers a model to be attacked if it can invert a trigger that achieves a high reconstructed ASR (REASR).
To the best of our knowledge, there is no existing detection methods for object detection models. Therefore, we perform straight-forward but reasonable adaption to these existing methods designed on image classification tasks, such that they are able to work against backdoor attacks on object detection models. Specifically, the original objective of NC, Tabor, and Pixel is to invert small triggers while maintaining their effectiveness (high ASR). In our adaptation, we retain their design principles but re-define the ASR to align with object detection models, as explained in Section 3.1. Additionally, we introduce a threshold for the size of inverted triggers, enabling the differentiation between benign and attacked models. For ABS, we adhere to its original technique but employ the re-defined ASR as the optimization goal, and use REASR as the decision score. By employing these adaptations, we aim to enhance the detection capabilities of these existing methods specifically for backdoor attacks on object detection models.
No modifications or adaptations are needed for meta classification-based methods when applied to object detection models. MNTD [69] trains a set of queries and a classifier to discern the feature-space
* https://pages.nist.gov/trojai/docs/object-detection-feb2023.html#object-detection-feb2023
* https://pages.nist.gov/trojai/docs/object-detection-jul2022.html#object-detection-jul2022
Table 5: Dataset details
Image Dataset	Model Source	Architecture	Number of Models
Round10	Round13	SSD	Faster-RCNN	DETR	Benign	Miscls Attack	Evasion Attack
Synthesis Traffic Sign	X	✓	✓	✓	✓	48	12	12
DOTA	X	✓	✓	✓	X	16	4	4
COCO	✓	X	✓	X	X	36	18	18
(a) IoU threshold
(b) Region size
(c) Score threshold
Figure 8: Hyper-parameter sensitivity.
distinctions between clean and attacked models. MF [21] directly trains a classifier on model weight features using specialized feature extraction techniques, i.e., matrix factorization. These methods solely rely on the feature extraction networks commonly utilized in both image classification and object detection models. As a result, MNTD and MF can be directly employed to detect backdoor attacks in object detection models without the need for additional adjustments or modifications.
We collect the Precision, Recall, ROC-AUC and Overheads for each method across various datasets and model architectures. To ensure a fair comparison, we have conducted a search to determine the optimal thresholds for different decision scores associated with each method (trigger size for NC, Tabor, Pixel, REASR for ABS and output confidence for meta-classifiers). These thresholds are chosen to maximize accuracy. Besides, we set a fixed number of optimization steps for scanning a pair of victim-target label (100) for all inversion based baselines. For meta classification based methods that involve training, we have performed 5-fold cross-validation and reported the validation results exclusively.
D Hyper-parameter Sensitivity Analysis
To assess the sensitivities of the hyper-parameters used in DJANGO, we conduct experiments as described in Section 4.3.
IoU Thresholds. We evaluate the IoU threshold used to calculate the ASR of inverted triggers. The results are summarized in Figure 8(a), where each row corresponds to a different model architecture, and each column represents a different choice of IoU threshold. It can be observed that IoU thresholds of 0.3 and 0.5 generally yield good performance. However, a threshold of 0.7 tends to degrade the performance, possibly due to the inverted triggers interfering with the bounding box predictions.
Region Size. The impact of different regional initialization sizes is evaluated and the results are presented in Figure 8(b). Among the various choices, a region size of  $30 \times 30$  consistently achieved the best performance. This is because larger initialization sizes tend to result in more false positive cases.
Score Threshold. Different score thresholds are tested when computing the ASR of inverted triggers. The results, shown in Figure 8(c), indicate that a score threshold of 0.5 generally leads to the best performance across all model architectures. This choice represents a trade-off between false positives and false negatives. Higher score thresholds may introduce more false negatives, as the inverted trigger may not have high confidence similar to the injected one. On the other hand, lower score thresholds may result in more false positives. Thus, a moderate value of 0.5 provides the optimal balance.
Table 6: Hyper-parameter sensitivity for Gaussian Focus Loss initial values
(μk, σk)	ROC-AUC
SSD	Faster-RCNN	DETR
(0.1, 2)	1.0000	1.0000	0.8750
(0.1, 1)	1.0000	1.0000	0.8750
(0.1, 3)	1.0000	0.8750	0.8750
(0.2, 2)	0.8750	0.7500	0.8750
(0.2, 1)	0.8750	0.8750	0.7500
(0.2, 3)	0.8750	0.8750	0.8750
Table 7: Effectiveness on different weight scheme distributions
Method	ROC-AUC
DJANGO + GF Loss	0.9500
DJANGO + LP Loss	0.9000
Tabor*	0.6500
NC*	0.6000
Pixel*	0.7000
Table 8: Effectiveness on complex attacks
Attack type	ROC-AUC
Object-appearing Attack	0.8000
Composite Attack	0.9000
These experiments allowed us to gain insights into the sensitivities of the hyper-parameters in DJANGO, enabling us to make informed choices for achieving optimal performance.
GF Loss Initialization. We evaluate the impact of two hyper-parameters (initial mean  $\hat{\mu}_k$  and variance  $\hat{\sigma}_k$ ) in the Eq. 4 on DJANGO. We randomly sample 8 models (4 trojan and 4 benign) for each architecture that was trained on the synthesized traffic sign dataset. Besides the default values we report in the paper ( $\hat{\mu}_k = 0.1$ ,  $\hat{\sigma}_k = 2$ ), we set 5 more groups of initial values and report the detection performance. As shown in Table 6, DJANGO remains effective under different initialization values.
Weight Scheme Distribution Choice. We attempted to substitute the Gaussian distribution with the Laplace distribution, another commonly employed probability distribution. Our experimentation involved 20 models trained on the traffic sign synthetic dataset, comprising 10 clean models and 10 models poisoned with misclassification triggers. The outcomes of these experiments are presented in Table 7. It is evident that with the Laplace focus loss, DJANGO maintains a high level of effectiveness, achieving an ROC-AUC of 0.9, whereas baseline methods only achieve 0.7. We hypothesize that the 0.05 performance decline, in comparison to the Gaussian Focus Loss, may be attributed to the sharper peak of the Laplace distribution when contrasted with the Gaussian distribution. Within our context, a reduced number of bounding boxes exhibiting moderate confidence are allocated larger coefficients in different stages, thereby potentially diminishing the emphasis on compromised boxes.
E Evaluation on Complex Object Detection Backdoor Attacks
We further evaluate DJANGO on two advanced object detection backdoor attacks: Object-appearing and Composite attacks [5, 3, 28]. Object-appearing triggers aim to yield target class object bounding boxes on the background. According to our definition, the object-appearing attack is a special case of the misclassification attack with the background  $\emptyset$  class as the victim class. To demonstrate the effectiveness of DJANGO on object-appearing attacks, we evaluate DJANGO on 10 Baddet models and 10 Clean-image backdoor models during rebuttal. For each type of attack, we mix 5 clean models with 5 poisoned models with object-appearing triggers. The evaluation results are shown in Table 8. DJANGO achieves 0.8 ROC-AUC on both Baddet and clean-image object appearing attacks.
For the composite attack, we conducted an evaluation of DJANGO using 10 models that were trained on the traffic sign synthesis dataset. This set included 5 clean models and 5 trojan models poisoned by composite attack. As indicated in Table 8, DJANGO achieved an 0.9 ROC-AUC for detecting composite attacks. It's worth noting that the composite attack does not rely on an explicit trigger.
Instead, it leverages a clean object A to serve as the trigger for attacking another object B. Interestingly, DJANGO is capable of effectively reversing this process, essentially identifying a trigger that closely mimics the pattern of object A. This ability enables DJANGO to detect composite backdoors with a high level of accuracy.