A Unified Detection Framework for Inference-Stage Backdoor Defenses
Authors: Xun Xian
Ganghua Wang
Jayanth Srinivasa
Ashish Kundu
Xuan Bi
Mingyi Hong
Jie Ding
Venue: NeurIPS 2023 poster (2023)
Keywords: Backdoor attacks
Backdoor Defense
Security for AI
TL;DR: We propose a generic backdoor detection framework for the inference stage, which includes a novel method with theoretical performance guarantees and is applicable to both computer vision (CV) and natural language processing (NLP) scenarios.

Abstract: Backdoor attacks involve inserting poisoned samples during training, resulting in a model containing a hidden backdoor that can trigger specific behaviors without impacting performance on normal samples. These attacks are challenging to detect, as the backdoored model appears normal until activated by the backdoor trigger, rendering them particularly stealthy. In this study, we devise a unified inference-stage detection framework to defend against backdoor attacks. We first rigorously formulate the inference-stage backdoor detection problem, encompassing various existing methods, and discuss several challenges and limitations. We then propose a framework with provable guarantees on the false positive rate or the probability of misclassifying a clean sample.  Further, we derive the most powerful detection rule to maximize the detection power, namely the rate of accurately identifying a backdoor sample, given a false positive rate under classical learning scenarios. Based on the theoretically optimal detection rule, we suggest a practical and effective approach for real-world applications based on the latent representations of backdoored deep nets. We extensively evaluate our method on 14 different backdoor attacks using Computer Vision (CV) and Natural Language Processing (NLP) benchmark datasets. The experimental findings align with our theoretical results. We significantly surpass the state-of-the-art methods, e.g., up to 300\% improvement on the detection power as evaluated by AUCROC, over the state-of-the-art defense against advanced adaptive backdoor attacks.

A Unified Detection Framework for Inference-Stage Backdoor Defenses
Xun Xian Department of ECE
Ganghua Wang School of Statistics University Minnesota wang901 90umn edu
University of Minnesota x xian00440umn, edu
Jayanth Srinivasa
Ashish Kundu Cisco Research
Xuan Bi
Cisco Research jasriniv@cisco.com
Carlson School of Management University of Minnesota xbi abi@@um... edu
ashkundu@cisco.cm com com
Mingyi Hong
Jie Ding
Department of ECE University of Minnesota mhong@umn edu
School of Statistics University of Minnesota dingj @umn edu
Abstract
Backdoor attacks involve inserting poisoned samples during training, resulting model containing hidden backdoor that can trigger specific behaviors without impacting performance on normal samples. These attacks are challenging to detect, as the backdoored model appears normal until activated by the backdoor trigger, rendering them particularly stealthy. In this study, we devise unified inference- stage detection framework to defend against backdoor attacks. We first rigorously formulate the inference-stage backdoor detection problem, encompassing various existing methods, and discuss several challenges and limitations. We then propose a framework with provable guarantees on the false positive rate or the probability of misclassifying clean sample. Further, we derive the most powerful detection rule to maximize the detection power, namely the rate of accurately identifying backdoor sample, given a false positive rate under classical learning scenarios. Based on the theoretically optimal detection rulc, we suggest a practical and effective approach for real-world applications based on the latent representations backdoored deep nets. We extensively evaluate our method on 14 different backdoor attacks using Computer Vision (CV) and Natural Language Processing (NLP) benchmark datasets. The experimental findings align with our theoretical results. We significantly surpass the state-of-the-art methods, e.g., up to 300% improvement on the detection power as evaluated by AUCROC, over the state-of- the-art defense against advanced adaptive backdoor attacks.
Introduction
Deep neural networks (DNNs) have made remarkable advancements in various domains, thanks to their exceptional learning capabilities [1,2]. However, these same learning capabilities that make DNNs so powerful have unfortunately rendered them vulnerable to growing critical threat known as the backdoor attack [3-19]. In a typical backdoor attack, the attacker manipulates small part of the training dataset by inserting specific triggers, such pixel patches in images [3] random word combinations in texts [16]. These manipulated samples are labeled with a specific target
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Clean Test Queries
'Manoges to original, even though it rips off many of ideas'
"good film. but very glan!
Learning model(s) deployod on cloud
Backdoor Test Queries
'good film but mnn very ghun ak.
'Will have been managing
Defender
be original, even though is rips off many of its ideux
Figure I: Illustration of inference-stage backdoor defenses. The backdoor triggers in text queries are indicated in red, while for the image queries, the backdoor triggers consist of a square patch located in the lower-right corner for the traffic sign and hello kitty embedding added to an image of dog.
class. When a DNN is trained on this poisoned dataset, it learns the original data patterns and the engineered relationship between the manipulated samples and the target class. As result, the DNN exhibits unexpected and often malicious behavior, producing manipulated predictions for test inputs containing the backdoor trigger while functioning normally for unaltered test inputs. The malicious and stealthy nature of backdoor attacks underscores the critical need to defend against such threats.
In effort to combat backdoor attacks in the inference stage, various empirical studies have been conducted [20-24] in both computer vision (CV) and natural language processing (NLP) domains. In these methods, given trained-an and-backdoored model, the defender will determine whether future query imput is clean or backdoor- backdoor-triggered based on certain features of this query input, as demonstrated in Figure 1. For instance, in CV scenarios, the popular STRIP method [20] superimposes the future query image with other clean images and examines the model's prediction entropy on these superimposed versions. If the model's prediction entropy these superimposed versions is abnormally low, the associated original test input is marked as backdoor- -ackdoor-triggered. The ONION method [21] widely used in NLP applications removes individual tokens from the input text and measures the resulting change in perplexity. If removing a particular token leads to a significantly greater perplexity change than others, it is considered a backdoor trigger word.
However, a principled understanding of the backdoor detection problem is underexplored in the literature. First, mathematical formulation for the backdoor detection problem is lacking, which may impede quantitative analysis of the backdoor challenges. Furthermore, while some existing methods are effective in certain scenarios, many of them lack theoretical guarantces regarding their detection performance, such as ensuring that the probability of misidentifying a clean sample as a backdoor one is no more than certain threshold. This creates complicated situation for researchers and practitioners in assessing the effectiveness of these methods.
Additionally, given the same nature of using excessive learning capability conduct backdoor attacks for both CV and NLP contexts, it is natural to wonder if the defense methods developed for backdoor attacks in CV can be applied to backdoor attacks in NLP. However, applying CV defense methods to NLP is challenging due to the distinct data structures and models in each context [16, 25]. For example, images are treated continuous while texts are discrete. Thus, the individual words utilized in [21] lack counterparts in images, preventing the use of techniques in [21] from defending against CV backdoors. Furthermore, the architectural differences between convolutional neural networks (CNNs) and recurrent neural networks (RNNs) add to the complexity. CNNs exhibit translational equivariance properties [26], whereas RNNs demonstrate auto-regressive characteristics [27, 28]. Hence, it remains unclear how the concept of CNN channels, as utilized in the backdoor defense of [22], can be translated to language models such as LSTM [29,30], and BERT [31].
1.1 Our contributions
In this study, we introduce an approach for detecting backdoor test inputs with provable guarantees that is practically applicable to both CV and NLP domains. Contributions are summarized as follows.
Novel framework of backdoor defenses with provable guarantees. We first introduce a rigorous formulation for inference- backdoor detection problems, encompassing most existing online backdoor defense methods through binary clasifiictionn/ypoooeeis testing concepts. This problem formulation enables precise discussions on the possible limitations of current backdoor defense

---

approaches. Based on these findings, we propose generic backdoor defense framework called conformal backdoor, inspired by the conformal prediction literature [32,33], for detecting and filtering future backdoor test samples with provable assurances on the false positive rate (FPR), namely the probability of mistakenly classifying a clean sample as backdoor one.
Optimal detection rule and its real-world proxy. To establish a theoretical comprehension of our proposed framework's 's detection power, we first show that successful backdoor attacks display distinctive properties under classical learning scenarios. In particular, we show that, among set of potential backdoor triggers (and the corresponding backdoor data), the most effective (to be defined) backdoor trigger is unique. This uniqueness property allows the defender to devise the uniformly most powerful detection rule, in accordance with the Neyman-Pearson (NP) criterion, when clean and backdoor data distributions are known. To apply real-world situations where neither clean nor backdoor data distribution is known, we propose a practical and efficient proxy based on the theoretically optimal detectors, which is applicable both CV and NLP domains. The proxy detector utilizes the latent representations of a backdoored model, such as the penultimate layer, which can be widely applied and is generally independent of the specific data structure (e.g., discrete tokens in text) and the model's properties (e.g., the CNN's 's translation-invarial property).
Extensive empirical studies on both CV and NLP Our proposed technique was evaluated across various datasets for both CV, including CIFARIO 10 [34], GTSRB [35], and Tiny-ImageNe [36], and
NLP, including SST2 [37] and IMDB [38], and popular model architectures such as ResNet18834 [2].
VGG19 [39]. base uncased-BEal BERT [31]. and 14 backdoor attacks including BadNets [3], Blended [4],
TrojanNN [S], SIG [7], WaNet [13], SSBA [15], TacT [14], Adaptive Patch [19], Adaptive Blend [19], SOS (NLP) [23], and LWS (NLP) [40]. Experimental results consistently endorse the effectiveness
of our technique in terms of detection performance, e.g., up to 300% improvement on the detection
power in terms of AUCROC over the state-of-the-art, STRIP [20], on adaptive backdoor attacks [14].
1.2 Related work
Backdoor attacks Backdoor attacks modify DNN predictions for specific inputs using backdoor triggers, while leaving regular inputs unaffected. The majority of backdoor attacks, both in CV [3- 7, 1-0015,17-19) and NLP [8, 9, 16), are implemented through data poisoning, and the major focus of them is on how to design effective backdoor triggers. For instance, in the field of CV, classical methods use square patches and hello kitty as backdoor triggers [3,4], which can be visually inspected. As result, later techniques try to design human- imperceptible backdoor triggers through image warping [13] and generative modeling [15]. Also, there is a series of methods focusing on creating embedding level imperceptible backdoor triggers with adaptive methods [14,19] 19]. Despite the distinct characteristics of natural language data, NLP backdoor attacks exhibit similar techniques to those used in CV. For example, rare word combinations are used as backdoor triggers [16]. In addition, researchers have proposed techniques to replace specific words in a text with new words searched from word embeddings while preserving the semantic meaning of the text [9]. Very recently, principled theory regarding when and how backdoor attacks work was developed in [41].
Latent-space backdoor detection with access to training data This line of work aims to filter out backdoor data from the training dataset and retrain a sanitized model based on the cleansed dataset [14, 42-50). These methods rely on the assumption that there is clear separation between clean and backdoor data in the latent space of backdoored models, and the defender can use such separation to detect backdoor samples. In CV, researchers have applied PCA to project high- dimensional embeddings into lower dimensions and then used clustering methods to filter out clean and backdoor data [44], presuming that backdoor samples are fewer than clean samples. Recent techniques, such as those proposed in [14, 46). leverage robust estimations techniques to improve the cluster analysis further. In NLP, the CUBE method [49] employs HDBSCAN [51] to identify embeddings clusters. In [45], authors use l embedding distance to remove backdoor samples by measuring the distance between the embeddings of each training example and the nearest trigger test example using l norm. Our work differs from these methods in that we do not have access to the training data. The defender has no prior knowledge of future test backdoor inputs, making it possibly more challenging for defending.
Backdoor detection with access to validation data This line of work detects backdoor samples by building statistics based on a clean validation dataset, which typically consists of IID samples drawn from clean training data distribution, and using the calculated statistics to decide if new input is clean or backdoor 52, For example, the classical method STRIP [20] filters out
3

---

poisoned samples by checking the randomness of the model's predictions when the input is perturbed several times [20], with the idea that the backdoor triggers dominate the rest information in the inputs regarding classification. These methods typically require case-by-case design, depending on the data structure and models. Our method utilizes the latent representation of the backdoored model, which is not specific to particular task.
2 Preliminary: backdoor defense and attack
Notations. Let TRd be the input space and y = (1, (1,..., C) (abbreviated as [C]) be the label space. We denote a model trained on a dataset D by fD : _2d + [C.. If f2 is a DNN, we decompose fD as 1 = w. dº, where wD is the projection head and di is the feature extractor (to be specified later).
Attacker's capability In line with previous backdoor poisoning attacks, we make the standard assumption that the adversary has control over the training process and can manipulate the training dataset, but does not have access to the inference stage [54]. Formally, let Delean = { 0.) "e denote the clean training dataset of size Nelean- independently drawn from a distribution PCLEAN defined over Fd V Let 771 : Rd + Rd and 1/2 C + [C] denote the backdoor data and label transformations, respectively. We emphasize that attackers are allowed to use a wide range of back- door transformations 7/. including fixed patch [3,4] and synamicchample-speccii
patches [11, 14), (2))))2(0))) (72(0:)) J be the backdoor dataset under transformation (7), 772) with the joint distribution PAD. By default, we consider (y) = 1 for y E [C]. Denote the learning model felean, _ Rd + [C] to be the model trained on Delean and D Delean U Dbackdoor respectively.
Let Dbackdoor = {(7) (.:),,
We set the risk minimization algorithm as the default formulation for training models.
Attacker's goal Given a clean dataset Deleau a set of learning models, and the backdoor trans- (I) the error rate on backdoor data un- der backdoor model: PIEN- (n) (X)) A TY)) under the constraint that the (II) the clean accuracy difference under the clean and the backdoor model: [IP (FXYY<LEAN(FF ffrt (X)
formations (7),772,, the attacker aims to to minimize
Y)= PXY (f (eleam = Y) does not exceed a pre-specified threshold value.
Defender's capability We consider a scenario where the defender can query the fixed, trained model frei with any input but is unable to modify the model or access the poisoned training dataset Deoi Specifically, the defender only has access to two kinds of information. The first (1) is the latent repre- sentation dPei (z) for any querying sample e Rdd of the learning model fre,i.e. f = w o oPii for froi being a DNN. For NLP classification tasks using BERT-based models, the output of Φro( (-) is the [CLS] embedding. For CNNs, the output of (.) is the avgpool24 2a layer immediately after all the convolutional layers. The (2) second is a set of validation data Da = {(T;; }i)) which are IID samples from the same distribution of clean training data PCLEAN, with sample size n < Neleem (sample size of the clean training data). By default, we set the value to be 1000.
Defender's goal When given future query X and backdoored model froi, the defender should accurately and efficiently determine whether Xies is a clean input not. For example, common objective for the defender is maximize the true positive rate, which represents the correct identifi- cation of backdoor samples, while simultancously minimizing the probability of misclassifying clean samples as backdoors.
Proposed framework of Conformal Backdoor (CBD) for backdoor defenses
We first thoroughly characterize the inference-stage backdoor detection problem by examining binary classification/yyottee testing, which encompasses many existing online backdoor defense methods. We then discuss the difficulties associated with the problem and the possible limitations of cxisting methods before proposing our novel detection framework.
3.1 Backdoor detection formulation
Given backdoored model frui and validation dataset Vaal, upon receiving test query Xustt E Rd, the defender faces a binary clasificatton/hypoteesi testing problem:
Ho : T T(Xies)) ~ T(S); S~ PPLEAAN H, T(Xee) * T(S); S ~ IPPLEANN
where T(-) is that and share the same distribution. Here, we note that the T(-) typically depends on both the backdoored model f and the validation dataset Val to reflect special properties of backdoor data, e.g., the predicted value T(X,eu) = fXoe (Xrea) and the latent representation T T(Xies) = @Poi (ies)) Also, it is worth mentioning that the true label Yes is not included since it is unknown to the defender.
transformation on the original input Xuss and X ~ Y means

---

Remark 1 (Unknown backdoor distribution PBD). The Ho represents the scenario where the imput originates from the clean distribution, while H₁ corresponds to the input not being derived from the clean distribution. It is important to emphasize that H₃ is not formulated to test if Xies comes from a backdoor data distribution PBD specifically, H₃ (xx ~ (S)) for FDD This is because, in real-world scenarios, the backdoor data distribution, BBD. is unknown to the defender. This poses a significant challenge in deriving rigorous results on detection power. In particular, without any distributional assumption FBDD. it is not possible to establish provable results on detection power.
To approach the above problem, defenders will construct a detector g(-), specified by
{
1 Backkoor-triggeree Sample), s (T(()) 2 T
(Clean Sample),
8 (T()) <
where 7 E RR is a threshold value and s(-) is a scoring function which takes the transformed input T(Xest and output value in [0, 1] to indicate its chance of being a clean sample.
Remark 2 (Distinguishing our approach from backdoor defenses focused on training-data purification). There is series of training-stage backdoor defense methods that have been developed with the goal of creating clean model by identifying and removing potentially contaminated samples within the training data [43, 44, 46, 50J. While these methods share the concept of distinguishing between clean and backdoor data, they differ from our approach in two significant ways. Firstly, these methods benefit from working with both clean and backdoor training data, giving them insights into both data categories. Secondly, their primary objective is to construct a clean model by singling out a subset of the training data based on specific characteristies of all training examples. Consequently, their evaluation criteria revolve around the purified model's ability to accurately classify clean data and protect against backdoor attacks.
In contrast, our approach serves as an inference-stage defense without access to the training data and no capability to influence the training process, including adjustments to model parameters. Similar to other inference-stage strren defenses (20,21, 24, 55], we operate under the assumption that the defender has access to a limited set of clean validation data without prior knowledge of future backdoor test inputs. Our goal is to detect forthcoming backdoor inputs, and our evaluation metric based on the AUCROC score of the detector.
3.2 Backdoor conformal detection
The main challenge for the defender is to devise an effective and efficient detector g(-) to identify backdoor samples. Several evaluation metrics can be utilized to assess the detector's 's performance, including AUCROC and Fl score. Given the stcalthiness of backdoor attacks, it is crucial for the defender to employ an evaluation method that offers provable guarantees regarding detection performance. To meet this requirement, it becomes necessary to consider both Type-I and Type-II errors within the hypothesis testing framework. Thus, we adopt the Neyman Pearson criterion for our evaluation, ensuring comprehensive assessment of our detection approach. Formally, the performance of detector y can be described by its true positives rate (TPR): P(g(X;+)= =
PearsnnPearson
X is backdoor) } and its false positive rate (FPR): (X) = 1 | X is clean]. ) The defender aims to find an optimal detection method g in the Neyman-Pearson (NP) paradigm:
maximize Pf9(X;T)= = 1| X is backdoor) subject to P(g(X;T) = 1| X is clean} S as,
where E 10) is the defender pre-specified upper bound on the FPR. Proving guarantees on the (FPR) for a wide range of score functions or detectors is difficult because the defender lacks knowledge about clean data distribution. This limits their ability to determine the distribution of score function s under IPCLEAN, making it hard to control the FPR.
To ensure the performance of the false positive rate, we propose detection method based on the conformal prediction framework /32,33]. The conformal prediction is paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of black-box models without explicit distribution assumption on data. The central idea is that by using an appropriate score function, the empirical rank quantile of the distribution will converge the population version, which is guaranteed by the uniform convergence of CDFs. To be more specific, as outlined in Procedure 1, the defender will begin by computing the scores of the validation points using validation dataset D Val Next, the defender will set the decision value Ao,s to satisfy the
FeLEAN (a.,)) 1 1-a a + v (llo((20))(22)))
(I)
5

---

where ò' E (0, is the violation rate deseribing the probability that the (FPR) exceeds a, and FOLEAN the empirical cumulative distribution function of the scores on the validation data {s(T((;) (1)(()))))) In the case where (10og(2/0) > a, we set the thresholding value J to be the maximum of (x(())
Algorithm 1 Conformal Backdoor Detection (CBD)
Input: querying input Xus, clean validation dataset Va {(X,, Y.) transformation method T(.), score function (-)), desired false positive rate a E (0, 1). violation rate ó (0, 1)
1: Receiving future query sample Xies
2: for i to do
3: Calculate 8, s(T(X.)) X, DV
4: end for
5: Select the decision threshold A according to Equation (1).
6: Determine if X is a clean sample based on s s((uu))) S Ao
Output: The decision whether the sample Xien a clean or backdoor sample
The next result shows that if a future test input comes from the same distribution as the validation data, the above algorithm can achieve false positive rate bounded by a with high probability.
Theorem (Conditional False positive rate of Algorithm I), Given any pre-trained backdoored classifier f. suppose that the validation dataset Val and the test data (Xust; Yee) are IID drawn from the clean data distribution PELEAN- Given any E (0, 1), for any score function 8, such that the resulting scores {= remain IID and continuous, the associated backdoor conformal detector g(3 s, )a as specified in Procedure I satisfies
PP P(g(Xeesi 8, )a,s) (Backdoor Sample) | Vol) S a,
with probability at lease 1 - 8 for any a E (0,1) such that a >
3.3 Optimal score functions for CBD
In the preceding section, we presented the CBD framework and established that when utilizing suitable score functions, which preserve the independence and identical distribution of transformed samples, the associated backdoor conformal detector achieves an FPR within the range of a. This prompts inquiry: what constitutes the optimal score function (and its corresponding conformal detector) that maximizes the TPR while maintaining an FPR within a predefined a threshold?
At high level, deriving theoretical results on the TPR without distributional assumptions on the backdoor data is not possible, as previously discussed. Therefore, our first step is to investigate the unique structure of backdoor data. We note that the distribution of backdoor data depends on both the clean data distribution and the learning model used. This because backdoor data is generated by transforming clean data with backdoor triggers y), and the effectiveness of the trigger in achieving the attacker's goal may vary under different models.
To aid in the theoretical analysis, we will be making certain assumptions about the components mentioned above. However, it is important to note that the practical algorithm presented in the next section does not rely these assumptions. The development in this subsection primarily serves to derive technical insights. Specifically, we will assume that the Suppose that the (A) marginal clean data is normally distributed with mean O and covariance Σ, (B) the attacker employs linear classifier (a = 1(0Ta x > 0} for 0 E Rd, and (C) the attacker applies the backdoor transformation 71 (I() + 7, where 7 E T. A {u Rdd ||u||2 = c, > 0}. It is important mention that the defender does not possess knowledge of the exact backdoor trigger y utilized by the attacker. However, the defender is aware of the set of potential backdoor triggers T,. Also, we note that the l2 norm constraint on 17 is practically important since backdoor triggers that are imperceptible to triggers, e.g.,
humans, i.e., those with smaller norm, are more favorable than numan-percepllee square patch. Given the availability of distributional information, our analysis in this section will be conducted the population level, directly working with the Attacker's Goal as specified in Section 3.
Taking into account all the aforementioned information, the defender can construct a uniformly most powerful detection rule if they possess knowledge of the exact backdoor trigger employed by the

---

Clean Backdoor
BadNets
Blended
TrojanNN Dynamic
WaNet
SSBA
A-Patch
A-Blend SOS (WPP) (NLP)
Figure 2: The mean values of s s(Z) (vertical-line bar for clean data, slash-line bar for backdoor data) in Eq. (2), by replacing the y with the real backdoor trigger, on GTSRB with ResNet18 over different backdoor attacks.
attacker, following the principles of the Neyman-Pearsee I Lemma [56]. This raises the question for the defender: when they obtain the backdoored model based the attacker's objective described in Section 3, are there multiple backdoor triggers 7 E 'To that could lead to this specific model? If the answer is yes, the defender would have complete knowledge of the backdoor trigger and, consequently, the backdoor data distribution, enabling them to derive the most effective detection rule. To address this concern, we provide a definitive answer the question as follows.
Theorem 2 (Uniqueness of the backdoor trigger). Under Assumptions (A), (B), and (C) as specified above, for any 0" E Red that satisfies the Attacker's Goal, as specified in Section 3, the backdoor trigger 7 E Te that corresponds to this attacker's backdoored classifier 0*, is unique and admits an explicit form.
(The defender has full knowledge about the form of the backdoor trigger and the backdoor data distribution). The above result indicates that there is a unique correspondence between a backdoor trigger 7" E T, and the resulting backdoored classifier 0* that fulfills the Attacker's Goal. Therefore, when the attacker provides the backdoored model the defender, the defender is aware that only one specific backdoor trigger is associated with this particular model. As a result, the defender possesses complete knowledge of the distribution of backdoor data by utilizing the explicit form of the backdoor trigger, which is included in the appendix.
Remark 3
Theorem (Uniformly most powerful detection rule). Under the same setup in Theorem 2, the defender's 's optimal score function resulting in the largest TPR given a FPR S a is in the form of:
s( s()) exp (Z 7 (Z - --- 7"))-
The above optimal detection rule is based on the likelihood ratio, which compares the probability of a future sample Z from the clean distribution with that from the backdoor distribution. Through empirical evaluation, we have found that the proposed score function, when using the actual backdoor trigger instead of v7', effectively distinguishes between clean and backdoor data in various scenar- ios, including 12 state-of-the-art backdoor attacks in CV benchmarks (e.g., CIFAR10) and NLP benchmarks (such as SST-2). The results are illustrated Figure 2.
3.4 Practical proxy: -oounk--v Mahalanobis (SCM)
The previous analysis shows that the Mahalanobis distance between the clean and data is the optimal detection rule under known Gaussian distributions. However, in the real-world scenario, the defender knows neither the clean nor the backdoor data distributions. Therefore, the two exact distances as specified in Equation (2) above can not be computed.
To overcome this challenge, a proximal approach is proposed in the following. Recall that the defender has a small set of clean validation dataset which is assumed to have the same distribution as the clean training data. Hence, one feasible way to calculate the theoretically optimal detector in Equation (2) is to use the empirical Mahalanobis distance calculated based on a clean validation,
namely Mz,, V & (z 11) V 44), where j is the empirical mean of a Val and V is the inverse of the sample covariance matrix of D. The choice of this selection is because observed that backdoor and clean data have significantly different values under the decision rule derived in Eq. (2) as shown in Figure 2.
empirically
significant challenge in using Mahalanobis distance arises in high-dimensional scenarios, partic- ularly when estimating the precision matrix. i.c., the inverse of the covariance matrix. The sample
7

---

covariance matrix, a popular choice, exhibits numerical instability in large data dimensions [57,58], leading an ill-conditioned matrix that is difficult to invert [59]. The estimation errors increase rapidly with data dimension d [60], causing issues for high-dimensional latent embeddings in deep neural networks, like ResNet18 and BERT. Consequently, MD may not preserve order information for D Val, impacting the FPR in Theorem 1.
To address this issue, we consider regularizing the sample covariance matrix through shrinkage techniques [2]. At a high level, the shrinkage technique is used to adjust the eigenvalues of a matrix. This adjustment involves changing the ratio between the smallest and largest eigenvalues of dataset's s covariance matrix. One way to do this is by shifting each eigenvalue by a certain amount, which in practice leads to the following, Shrunk-Covaricn defined as,
Mahalanobis (SCM) score function,
(( [2); fro, yoly & (c) - -14) "0'T55'((a (1pols (1);
where (PPl (.) is the latent representation of the backdoored DNNs, μ is the mean of {dPO (cPo((s))
and Ma _ 8)S ((Tr(S)) ( Ta) with S being the sample covariance of { (xx) (xx)) Tr(-)
is the trace operator, and 3 E (0, 1) is the shrinkage level. The use of the latent space representation (PPo(() instead of the original data information is motivated by the difficulty in modeling distributions over the original digital space [63]. The shrinkage parameter can be selected via cross-validation [64]. Empirically, we observed that the proposed SCM accurately distinguishes between clean and backdoor data in various backdoor attack scenarios, spanning both CV and NLP domains, as demonstrated from the histograms in Figure 3.
Eathas
KOUT MEUTTA
Clac
Ohin
Com
Clan
Earkiter
2018
Figure 3: Histograms of our proposed SCM on CIFAR10 using ResNet 18 for different backdoor attacks in both CV and NLP domains. We noticed distinct separations in SCM scores between clean and backdoor data, in all scenarios.
4 Experiments
In this section, we conduct extensive experiments to evaluate our proposed methods. The empirical results match our theoretical results in terms of the false positive rate (FPR), and the detection power of our methods outperforms ttateoffthhe-a defenses on a wide range of benchmarks. We further provide runtime analysis of our method and ablation studies for hyperparameters in the Appendix. All backdoor datasets and models were obtained by re-running publicly available codes (details in the appendix) The experiments were conducted on cloud computing machines equipped with Nvidia Tesla V100s.
4.1 Setups
Datasets We evaluate our proposed method on three popular (D) CV image classification datasets: CIFARIO [34), GTSRB [35], and Tiny-ImageNet [36], and on two (II) NLP datasets: SST2 (Stanford Sentiment Treebank v2) [37] and IMDB [38]. Results for SST2 are presented in the main paper, while those for IMDB are deferred to the Appendix. We include short introduction to each dataset in the Appendix.
Models (I) CV We use ResNer18 [2] and (II) NLP the base uncased-BEiR BERT [31] as the default model architecture for our experiments the main text. Ablations studies on different model architectures, e.g., VGG 19 [39] and WideResNet [65], and mode configurations are included in the Appendix.
Metries We consider two evaluation metries: (1) the FPR (2) the area under the receiver operating characteristic curve (AUCROC). All experiments are independently repeated 10 times.
Attacks (I) CV the main text, we evaluate our proposed method on ten popular attacks, including
lad--and poisoning attacks: BadNets [3], Blended [4], Trojan [5]), clean label attack: SIG [7].
(3) attacks with invisible/sample-iise triggers: Dynamic [11], SSBA [15], WaNet [13], (4) and
adaptive backdoor attacks: TaCT [14], Adaptive Blend [19], Adaptive Patch [19]. (II) NLP We
evaluate the proposed method on two state-of-the-ane attacks: (1) SOS [23] and LWS [40]. In the
appendix, we assess the performance of our methods against two additional categories of backdoor
8

---

Attaack: BataaksBBdNNets
Blended
AttackTTrojanNN
Attark:SiG
Attack Attack:Dynamic
False Positive Rate
Theery lpper Bound Cao SCM
Theary Upper Boung CBD SCN
Theory lpper Bound Cee SCM
Theary Upper Bound CBD SCH
Theory Usper Boung CSD SCM
5/ Watack-WaNet
of AStack-SBBA
Attack:TacT
AttaccAddalive Patch
to AttakkAdaptive Blend
False Positive Rate
Theory upper Boung? cao SCM
Theory Usper Boung cRD-SCM
Theary Upper Bound CBD-SCM
Theory Usper Bound CRD-SCH
Theory Uspar Boung CtD SCM
55-
Figure 4: The mean FPRs of our proposed method on GTSRB are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are 0.01 for all cases. Our method's FPRs consistently match the theoretical upper bounds.
attacks, which are tailored to exploit vulnerabilities in our techniques. Configuration details of these attacks, including trigger patterns, poisoning rates, and hyperparameters, are included in the Appendix.
Performance of Attacks (I) CV On CIFARIO and GTSRB datasets, the backdoored model achieves clean accuracy rates exceeding 93% and backdoor accuracy rates exceeding 98% for all the previously mentioned backdoor attacks. The exceptions in backdoor accuracy rates of around 80%. (II) NLP On SST2 and IMDB datasets, the backdoored model attains clean accuracy rates surpassing 90% and backdoor accuracy rates exceeding 96% for the mentioned backdoor attacks.
the SIG and Adaptive Blend attacks, which result
Defenses For both (I) CV and (II) NLP, in the main text, compare our proposed method with three detection-basee backdoor defenses in the literature, namely, STRIP [20], &2 based-defense [45], and MAD (Mean Absolute Deviation) -aased defense. In the appendix, compare our methods with two recent detection-based defenses designed for CV backdoor attacks, SCALEUP and FREQ [24,55], as well as filtering-based training-stage defenses, SPECTRE [46] and ABL [47].
4.2 Main results
FPR on CIFAR10 and GTSRB We assess the FPR performance of our proposed method CBD-SCM (SCM score function within the CBD framework) on, CIFARIO and GTSRB datasets by varying the defender pre-specified false positive rate Each plot corresponds shows the FPR performance of our proposed method under that attack. For each plot, we conduct ten independent experiment replications and calculate the mean, along with the +1 standard error bars shown in the shaded regions in Fig. 4 for GTSRB and Fig. 5.1 for CIFAR 10 in the appendix. The results demonstrate that the FPR of our method (in red dots) consistently matches the theoretical upper bounds (in blue triangles), supporting the findings presented in Theorem 1.
specifie backdoor attack and
Detection Power (AUCROC) CIFAR10 and GTSRB We conduct thorough evaluation of the detection performance of our proposed method against all ten distinct backdoor attacks and report the corresponding AUCROC values in Table 1. Our method achieves the best performance in all cases on both datasets. Specifically, we observe remarkable increase in AUCROC, up to 300%, for the more advanced backdoor attacks such con-patch-basse backdoor attacks (WaNet and SSBA) and latent space adaptive attacks (Adaptive Blend and Adaptive Patch).
Detection Power (ROC) -mag-em We further evaluate the detection performance of our proposed method on
against both classical patch-based attacks (e.g., Blended) and more advanced non-patch-based attacks (c.g.. WaNet). We visualize the ROC curves for our method in Figure 5 (a) and (b). shows a significant improvement in the TPR at low false positive rate regions compared with other methods.
Detection Power (ROC) on SST2 (NLP) We assess the detection performance of our proposed method on SST2 (NLP) using the base mecased-BERT model. Specifically, we generate ROC curves for our methods under two advanced backdoor attacks, as illustrated in Figure (c) and (d). The results indicate that our method outperform the STRIP method significantly, but are relatively less

---

Adaakk: Blendes, Batare:: Thny Iragatict
Attack: Woleet, Deraset: Tny imagehet
Mtack LVP. Dataset: SST-2 (NLF)
Attad:: 505 Dataset 551-2 (NUP)
1.0
14
TPA
SCH (dur Methed, AJC Scorec.991
(Dtr Scoret/317
(Gur Mathodi Score
SCH t0ur Method, AUC 5.ord
STMP. AUC Score
AUC Scared
of
08
38
08 10
(8)
(d)
Figure 5: AUCROC on Imny-ImageNet and SST-2 NLP. For the Tiny-ImageNet dataset, our method significantly outperforms the existing methods under both patch-based attacks (Blended) and the more advanced non- non-patch-atee patch-attack (WaNet). On the NLP SST2 benchmark, our proposed method consistently outperforms the STRIP method.
effective under the SOS attack compared to the other two defenses that use latent representations. These findings suggest that the current NLP attacks retain a considerable amount information in the latent representations that can be utilized to differentiate between clean and backdoor data.
Table I: AUCROC results for CIFARIO and GTSRB. Mean values of ten replicates are reported, with standard errors below 0.01 for all cases. The best-performing method(s) are indicated in boldface.
CIFARIO
GTSRB
Defense +
STRIP
82
MAD
SCM
STRIP
MAD
SCM
BadNets
1.0
1.0
0.99
0.97
0.97
0.99
Blended
0.61
0.96
0.80
0.69
0.70
0.90
TrojanNN
0.55
0.76
0.99
0.50
0.89
0.99
SIG
0.72
0.97
0.41
0.32
0.76
Dynamic
0.84
1.0
0.99
0.99
0.99
0.99
SSBA
0.68
0.66
0.97
0.80
0.22
0.22
0.99
WaNet
0.39
0.93
0.54
0.31
0.99
TacT
0.68
0.92
0.44
0.71
Adaptive Blend
0.69
0.84
0.75
0.96
0.72
0.88
0.99
Adaptive Patch
0.76
0.79
0.98
0.33
0.56
0.49
0.87
5 Conclusion
This paper addresses defending against backdoor attacks by detecting backdoor samples during the inference stage. We introduce a mathematical formulation for the problem, highlight potential challenges, and propose a generic framework with provable false positive rate guarantees. Within this framework, we derive the most powerful detection rule under classical learning scenarios and offer practical solution based on the optimal theoretical approach. Extensive experiments in both CV and NLP domains consistently validate our theory and demonstrate the efficacy of our method.
Limitation and future work. While work presents comprehensive framework for defending against infcrence-stage backdoor attacks, there are some limitations that warrant further investigation. Firstly, future studies could explore the optimal detection rule in more practical scenarios where only partial information regarding the clean and backdoor data distributions is available. An example is Assisted Learning [66-68], decentralized learning scenario where learners only provide processed data information during the inference stage. Secondly, it would be interesting to extend the proposed approach backdoor detection in variety of Federated Learning scenarios (17,69-71) Thirdly, is worth investigating the use of general distributions conditional on side information (e.g.. [72]) to design the optimal detection rule. Lastly, it would be helpful to explore information beyond the penultimate layer, e.g., the topology of neuron activations [73], for backdoor detection.
The Appendix contains additional details on the CBD framework, more introductions to the related work, ethical considerations, more extensive ablation studies, and all the technical proofs.
10

---

Acknowledgement
The work of Xun Xian, Xuan Bi, and Mingyi Hong was supported in part by a sponsored research award by Cisco Research. The work Ganghua Wang and Jie Ding was supported in part by the Office of Naval Research under grant number N00014-21-1-2590.
References
[1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, vol. 25, 2012.
[2] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[3] T. Gu, B. Dolan-Gavitt, and S. Garg, "Badnets: Identifying vulnerabilities in the machine learning model supply chain," arXiv preprint arXiv:/708.067:: 2017.
[4] X. Chen, C. Liu, B. Li, K. Lu, and D. Song, "Targeted backdoor attacks on deep learning systems using data poisoning," arXiv preprint urXiv:1712.05526 2017.
[5] Y. Liu, S. Ma, Y. Aafer, W.-C. J. Zhai, W. Wang, and X. Zhang, "Trojaning attack neural networks," 2017.
[6] A. Turner, D. Tsipras, and Madry, "Clean-labe" backdoor attacks," " 2018.
[7] M. Barni, K. Kallas, and B. Tondi, "A new backdoor attack in cnns by training set corruption without label poisoning." " in 2019 IEEE International Conference on Image Processing (ICIP).
IEEE, 2019, pp. 101-105.
[8] J. Dai, C. Chen, and Y. Li, "A backdoor attack against Istm-based text classification systems," IEEE Access, vol. 7, pp. 138 872 138 878, 2019.
[9] S. Garg, A. Kumar, V. Goel, and Y. Liang, "Can adversarial weight perturbations inject neural backdoors," Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020, pp. 2029-2032.
[10] S. Zhao, X. Ma. X. Zheng. J. Bailey, J. Chen, and Y.-G. Jiang, "Clean-label backdoor attacks on video recognition models." " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 44442-14452.
[11] A. Nguyen and A. Tran, "Input-aware dynamic backdoor attack," Advances in Neural Information Processing Systems, vol. 33, pp. 3454-3464, 2020.
[12] K. Doan, Y. Lao, W. Zhao, and P. Li, "Lira: Learnable, imperceptible and robust backdoor attacks," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 11 966-1166111 976.
[13] A. Nguyen and A. Tran, "Wanet-imperceptible warping-based backdoor attack," arXiv preprint 10XVv22102.10369 2021.
[14] D. Tang, X. Wang, H. Tang, and K. Zhang, "Demon in the variant: Statistical analysis of dnns for robust backdoor contamination detection." in USENIX Security Symposium, 2021, pp- 1541-1558.
[15] Y. Li, Y. Li, B. Wu, L. Li, R. He, and Lyu, "Invisible backdoor attack with sample-specific triggers," in Proceedings of the IEEE/CVE International Conference Computer Vision, 2021, pp. 16463-16472.
[16] X. Chen, A. Salem, D. Chen, M. Backes, S. Ma, Q. Shen, Z. Wu, and Y. Zhang, "Badnl: Backdoor attacks against nlp models with semantic-preserving improvements," in Annual Computer Security Applications Conference, 2021, pp. 554-569.
[17] E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, "How to backdoor federated learning," " in International conference on artificial intelligence and statistics. PMLR, 2020,
pp. 2948.
11

---

[18] H. Souri, L. Fowl, R. Chellappa, M. Goldblum, and T. Goldstein, "Sleeper agent: Scalable hid- den trigger backdoors for neural networks trained from scratch," Advances in Neural Information Processing Systems, vol. 35, pp. 19 165-19 178, 2022.
[19] X. Qi, T. Xie, Y. Li, S. Mahloujifar, and P. Mittal, "Revisiting the assumption of latent separabil- ity for backdoor defenses," in The eleventh international conference on learning representations, 2023.
[20] Y. Gao, Xu, D. Wang, S. Chen, D. C. Ranasinghe, and S. Nepal, "Strip: A defence against trojan attacks on deep neural networks," in Proceedings of the 35th Annual Computer Security Applications Conference, 2019, pp. 113-125.
[21] F. Qi, Y. Chen, M. Li, Y. Yao, Z. Liu, and M. Sun, "Onion: A simple and effective defense against textual backdoor attacks," arXiv preprint urXiv:2011.10369 2020.
[22] R. Cai, 7. Zhang, T. Chen, X. Chen, and 7. Wang, "Randomized channel shuffling: Minimal- overhead backdoor attack detection without clean datasets," Advances in Neural Information Processing Systems, vol. 35, pp. 33 876-333333 -3 889, 2022.
[23] W. Yang, Y. Lin, P. J. Zhou, and X. Sun, "Rethinking stealthiness of backdoor attack against nlp models," in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference Natural Language Processing (Volume 1: Long Pupers), 2021, pp. 5543-5557.
[24] J. Guo, L.L., Chen, H. Guo, Sun, and C. Liu, "Scale-up: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency," in The eleventh international conference on learning representations, 2023.
[25] X. Sheng, Z. Han. P. Li, and X. Chang. "A survey on backdoor attack and defense in natural language processing," arXiv preprint arXiv:2211.11958 2022.
[26] K. J. Cheoi, H. Choi, and J. Ko, "Empirical remarks the translational equivariance of convolutional layers," Applied Sciences, vol. 10, no. 9, p. 3161, 2020.
[27] E. Diao, J. Ding, and V. Tarokh, "Restricted recurrent neural networks," in 2019 IEEE interna- tional conference on big data (big data). IRFF, 2019, pp. 56-63.
[28]
"Drasie: Distributed recurrent autoencoder for scalable image compression," in 2020
Data Compression Conference (DCC). IEEE, 2020, pp. 3-12.
[29] A. Graves and A. Graves, "Long short-term memory," Supervised sequence labelling with recurrent neural networks, pp. 37-45, 2012.
[30] J. Wang, M. Xue, R. Culhane, E. Diao, J. Ding, and V. Tarokh, "Speech emotion recognition with dual-sequence Istm architecture." in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 6474-6478.
[31] Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding,"" arXiv preprint 1X10::881000400 2018.
[32] V. Vovk, A. Gammerman, and G. Shafer, Algorithmic learning in a random world. Springer, 2005, vol. 29.
[33] J. Lei and I.. Wasserman, "Distribution-free prediction bands for non-parametrie regression," " Journal of the Royal Statistical Society: Series B: Statistical Methodology, pp. 71-96. 2014.
[34] A. Krizhevsky, G. Hinton et al., "Learning multiple layers of features from tiny images," 2009.
[35] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, "Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition," Neural networks, vol. 32, pp. 323-332, 2012.
[36] Y. Le and X. Yang, "Tiny imagenet visual recognition challenge," C5 231N, vol. 7, no. 7, p. 3, 2015.
[37] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng. and C. Potts, "Recursive deep models for semantic compositionality over a sentiment treebank," in Proceedings of the 2013 conference on empirical methods in natural language processing, 2013, pp. 1631-1642.
12

---

[38] A. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. "Learning word vectors for sentiment analysis." in Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies. 2011, pp. 142-150.
[39] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:14099 1109.1196......... 2014.
[40] L. Li, D. Song, X. Li, J. Zeng, R. Ma, and X. Qiu, "Backdoor attacks on pre-trained models by layerwise weight poisoning," in Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 3023-3032. [Online]. Available: https://cclattholog..o///// org22200 Lemnlp-manilliie
[41] X. Xian, G. Wang, J. Srinivasa, A. Kundu, X. Bi, M. Hong, and J. Ding, "Understanding backdoor attacks through the Adaptability Hypothesis," in International Conference on Machine Learning, 2023.
[42] E. Chou, F. Tramer, and G. Pellegrino, "Sentinet: Detecting localized universal attacks against deep learning systems," in 2020 IEEE Security and Privacy Workshops (SPW). IEEE, 2020, pp. 48-54.
[43] B. Tran, J. Li, and A. Madry, "Spectral signatures in backdoor attacks." arXiv preprint 1XX1v:8111.000
2018.
[44] B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, Molloy, and Sri- vastava, "Detecting backdoor attacks deep neural networks by activation clustering," arXiv preprint arXiv: 18/1 03728, 2018
[45] E. Wallace, T. Z. Zhao. S. Feng. and S. Singh, "Concealed data poisoning attacks on nlp models." arXiv preprint arXiv:2010. 12563, 2020.
[46] J. Hayase, W. Kong, R. Somani, and S. Oh, "Spectre: defending against backdoor attacks using robust statistics," arXiv preprint arXiv:2104.1 1115, 2021.
[47] Y. Li, X. Lyu, N. Koren, L. Lyu, B. Li, and X. Ma, -anti-backdoor learning: Training clean models on poisoned data," Advances in Neural Information Processing Systems, vol. 34, pp. 14 900004 912, 2021.
[48] Z. Hammoudeh and D. Lowd, "Identifying a training- set attack's ' target using renormalized influence estimation," in Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 2022, pp. 1367-1381.
[49] G. Cui, L. Yuan, B. He, Y. Chen, Z. Liu, and M. Sun, "A unified evaluation of textual backdoor learning: Frameworks and benchmarks," arXiv preprint mrXiv:2206.085/1 2022.
[50] K. Huang, Y. Li, B. Wu, Z. Qin, and K. Ren, "Backdoor defense via decoupling the training process," arXiv preprint 22Xi:22022.03223,, 2022.
[51] L. McInnes, J. Healy, and Astels, "hdbscan: Hierarchical density based clustering." Open Source Softw., vol. 2, no. 11, p. 205, 2017.
[52] B. G. Doan, E. Abbasnejad, and D. C. Ranasinghe, "Februus: Input purification defense against trojan attacks on deep neural network systems," in Annual Computer Security Applications Conference, 2020, pp. 897-912.
[53] W. Yang. Y. Lin, P. Li, J. Zhou, and X. Sun, "Rap: Robustness-aware perturbaatoons for defending against backdoor attacks on nlp models," arXiv preprini arXiv:21 2021.
[54] Y. Li, B. WU, Y. Jiang, Z. Li, and S.-T. Xia, "Backdoor lcarning: A survey," arXiv preprint arXiv:2007.0877 087445 2020.
[55] Y. Zeng, W. Park, Z. M. Mao, and R. Jia, "Rethinking the backdoor attacks' triggers: A frequency perspective," in Proceedings of the IEEE/CVF International Conference Computer Vision, 2021, pp. 16 478--18881
[56] J. Neyman and E. S. Pearson, "On the problem of the most efficient tests of statistical hypotheses." Philosophical Transactions of the Royal Society of London, vol. 231, no. 694-706, pp. 289-337, 1933.
13

---

[57] O. Bodnar, T. Bodnar, and N. Parolya, "Recent advances in shrinkage barinkage-based based high- -igh-dimensionll inference," Journal of Multivariate Analysis, vol. 188. p. 104826, 2022.
[58] J. Z. Huang, N. Liu, M. Pourahmadi, and I.. Liu, "Covariance matrix selection and estimation via penalised normal likelihood," Biometrika, vol. 93, no. 1, pp. 85-98, 2006.
[59] L. N. Trefethen and D. Bau, Numerical linear algebra. Siam, 2022, vol. 181.
[60] A. D. Ker, "Stability of the mahalanobis distance: A technical note," Technical Report CS-RR- 10-20, 2010.
[61] O. Ledoit and M. Wolf, "Honey, shrunk the sample covariance matrix," UPF economics and business working paper, no. 691, 2003.
[62] Bickel, B. Li, A. B. Tsybakov, S. A. van de Geer, B. Yu, T. Valdés, C. Rivero, Fan, and
A. van der Vaart, "Regularization in statistics," Test, vol. 15, pp. 271-344, 2006.
[63] C. Shorten and T. M. Khoshgoftaar, "A survey image data augmentation for deep learning." Journal of big data, vol. 6, no. 1, pp. 1-48, 2019.
[64] J. Ding, V. Tarokh, and Y. Yang, "Model selection techniques: An overview," IEEE Signal Process. Mag., vol. 35, no. 6, pp. 16-34, 2018
[65] M. Tan and Q. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," in International conference on machine learning. PMLR, 2019, pp. 6105-6114.
[66] X. Xian, X. Wang. J. Ding, and R. Ghanadan, "Assisted learning: framework for multi- organization learning," in Advances in neural information processing systems, 2020, pp. 180 14591.
[67] E. Diao, J. Ding, and V. Tarokh, "GAL: Gradient assisted learning for decentralized multi- organization collaborations," in Advances in Neural Information Processing Systems, 2022, pp. 11 854-11 868.
[68] X. Wang. J. Zhang, M. Hong. Y. Yang, and J. Ding, "Parallel assisted learning," " IEEE Transac- tions on Signal Processing, vol. 70, pp. 5848-5858, 2022.
[69] Ding, E. Tramel, A. K. Sahu, S. Wu, S. Avestimehr, and T. Zhang, "Federated learning challenges and opportunities: An outlook," in International Conference on Acoustics, Speech, and Signal Processing, 2022.
[70] E. Diao, J. Ding, and Tarokh, "HeteroFL: Computation and communication efficient federated
learning for heterogeneous clients," in International Conference on Learning Representations, 2021.
[71] "SemiFL: Communication efficient semi suni-supervieed federated learning with unlabeled clients," in Advances in neural information processing systems, 2022.
[72] "Multimodal controller for generative models," in Computer Vision and Machine Intelli- Proceedings of CVMI 2022. Springer, 2023, pp. 109-121.
gence:
[73] Y. Zhou, Y. Zhou, J. Ding, and B. Wang, "Visualizing and analyzing the topology of neuron activations in deep adversarial training," in Proceedings of the 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML) at the 40th International Conference Muchine Leurning, 2023.
[74] X. Tong, Y. Feng, and J. Li, "Neyman-pearson (np) classification algorithms and np receiver operating characteristic (np-roc) curves," arXiv preprint arXiv:1608.031/00 2016.
[75] T. Xic, "Backdoor toolbox." https://gituu..com//
/bbackoorr-toolbox 2022.
[76] B. Wu, Chen, M. Zhang, Zhu, Wei, D. Yuan, and C. Shen, "Backdoorbench: compre- hensive benchmark of backdoor learning." in Thih Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
14

---

Appendix
In Section A, present the formal proofs for the theoretical results discussed in Section 3 of the main text, specifically Theorem 1, 2, and 3. Furthermore, we provide a comprehensive listing of the detailed configurations used in the experimental study, along with brief introduction all the datasets, in Section B. We also include the omitted empirical results from the main text, focusing on the CIFARIO 10 [34] and IMDB (NLP) [38] datasets, which are presented in Section C. Additionally, we conduct an ablation study to explore the impact of different model architectures, such as VGG19 [39], which is discussed in Section D. Furthermore, we conduct additional ablation study examine the effects of varying poisoming ratios, as presented in Section E. We perform an ablation study to investigate the choice of the hyperparameter B. and the results are provided in Section We further subject our proposed methods to backdoor attacks specifically tailored to target our techniques in Section Finally, We further compare our methods with some recent backdoor defenses including both inference-stage and training-stage methods in Section H.
A Proof
In this section, we include the proof of the main results in Section 3.2 and 3.3 of the main text.
A.1 Proof for Section 3.2
In this section, we present the proof of Theorem For the convenience of the reader, we attach the pseudo-code of Algorithm 2 below. Please note that the thresholding value λa,s is selected to satisfy
Algorithm Conformal Backdoor Detection (CBD)
Input: querying input Xuss clean validation dataset D = {(X,, Y) transformation method T(+), score
function s (T)),,, desired false positive rate E (0, 1), violation rate 8 E (0, 1)
1: Receiving future query sample Xisa
2: for = to r do
3: Calculate 8, = s(T(X.)) // D
4: end for
Select the decision threshold Àaςa according Equation (1).
6: Determine Xist is a clean sample if s (TT(Ne))) S Xa,s
Output: The decision if the sample Xist is clean or backdoor sample
the condition
FCLEAN (Aa,AA(Aa.,)) = 1- + v (log(2/0)/(2n)),
where E (0, 1) is the violation rate describing the probability that the (FPR) exceeds a, and FCLEAN is the empirical cumulative distribution function of the scores the validation data {s(T(X) (T,))))) >> In the case where / (log(2/8)/(2n)) >> a. we set the thresholding value be the maximum of (x))()
The main text briefly touches upon the central concept found in the conformal prediction literature. It states that by employing a suitable score function, the empirical rank or quantile of the distribution will eventually approach the population counterpart. This convergence is ensured by the uniform convergence of cumulative distribution functions (CDFs). Consequently, to establish the proof for Theorem will first present the subsequent outcome, which accurately measures the uniform convergence of CDFs.
Lemma 1
inequality). Given natural number n, let X1,X2,... Xi,,X...,,,, X, be real-valued independent and identically distributed random variables with cumu- lative distribution function F--). Let (-) denote the associated empirical distribution function.
The interval that contains the true CDE, F(x), with probability 1 is specified
In 2
Fx(x) - E F(x) & (x) + where E =
2n
Theorem 4 (False positive rate of Algorithm 1). Given any pre-trained backdoored classifier f, suppose that the validation dataset D) Val and the test data (Xest, Yust) are IID drawn from the clean
15

---

data distribution PCLEAN. Given any d E (0, 1), for any score function and transformation method sTT())) such that the resulting scores (s(T(X))) associated backdoor conformal detector g(: Aa,s) as specified in Algorithm satisfies
remain IID with a continuous distribution, the
(((( (Xiest 8, )a,s) = ( (Backdoor Sample) | ( S a,
with probability at lease 1 8 for any a E (0, such that > (log(2/0))((2))
Proof. Note that Xee is drawn from the clean data distribution and we have Pig( iass 8, Aa) = (Backdoor Sample) D Val)
= Ex.. gg 8, Aa) = D
Ex... (s((((X 2 Aa ]
(5)
Ex,,, 1 (C (Xcos))) 2 (CLEAN(Aa) Di (S(FCLEAN(5(T((X (xX((e)))) FELEAN(AA) (Diil)
(6)
In 2
< <P(PATRN(((T((XMa)) FCLEAN(da)- - - - ( (s
(7)
2n
1 - a1--e
(8)
0,
holds with probability at least - 8. The equation (5) is because of the decision rule as specified in Algorithm 1. Additionally, the FeLEAN in Equation (6) represents the CDF of (Te(e)))), while
FELEAN in (7) denotes the empirical CDF obtained from Val The inequality in Equation (7) arises from the DKW inequality specified in Lemma 1. Furthermore, the equation (8) is based on the fact that the CDF follows a uniform distribution (a result of the probability integral transformation) and the selection of the thresholding value specified in Equation (4).
A.2 Proof for Section 3.3
This section will present the proofs for Theorem 2 and 3, as outlined in Section 3.3 of the main text. These results are intimately connected the renowned Neyman-Pearson Lemma within the context of statistical hypothesis testing and binary classification problems. To provide the necessary context, we will begin with a brief introduction to the Jeyman-PPersee classification framework.
A.2.1 Neyman-Pearson Classification
Consider a random pair (X, Y). where X E X C. Rd is a d-dimensional vector of features, and E {0, 1} represents the class label of X. A classifier
X - {0, 1} from a data input belongs to X to 0, 1. The overall classification error of f is denoted as R(g) IE1 (g() + YJ = (X( o Y}. By applying the law total probability, R(g) can be decomposed into a weighted average of the
type error (g) P + Y Y 0} and the type II error R1 (g) (()(() + #Y Y 1}, given by
R(g) no Ro(g) + क (g)
where To P(Y 0) and P(Y 1). While the classical paradigm minimizes R(-), the Deyman-PParssen (NP) paradigm seeks to minimize R₁ while controlling Ro under a user-specified level a. The (level-a) NP oracle classifter is thus
9a E arg minR, (g) Rola)
where the significance level reflects the level of conservativeness towards type error. To reflect on the backdoor detection problem, we encode the label set Y to indicate if its associated X is clean (with label 0) or backdoored (with label 1). The classifier g in the NP classification context corresponds to the detector in our framework. The following result, direct consequence of the Lemma [56], gives the solution of g..
famous Neyman-
Lemma (NP oracle classifier [74]), Let F, and Po be two probability measures with densities P1 and po respectively. Under mild continuity assumption, the NP oracle elassifier is given by
(1()))
9. (x) =
> C.
(r(
for some threshold C, such that (X))((X)/) C.) S a and Pilpi (X)/poX)) C.) a.
16

---

A.2.2 Proof for Theorem 2
Proof. By the definition of the Attacker's Goal, specified in Section 3, the attacker faces problem of the following:
max PXY~BRIIS (f (m (X)) =22 fI
subject to PXY (XP * Y)
(X) * Y)I S.
Also, we assume that the (A) marginal clean data is normally distributed with mean 0 and covariance Σ, (B) the attacker employs a linear classifier [a(2) = 1 1 x 0} for 0 E IR, and (C) the attacker applies the backdoor transformation 771 (z) = x + where 7 E T. {u E IR ||ull2
> 0}.
Firstly, it is straightforward to check that, per the Attacker's 's Goal above, the attacker cannot obtain backdoor classifier with 8" = 0, as in this scenario, the corresponding backdoor accuracy:
PX~N(r,E)(X 0" >0 0)
would be zero regardless of the backdoor trigger 7n (ω) = x y for Te. Next, for any non-zero 0" RRd, suppose that there exist 71, y" E Te such that (n, and (r", (*) are both the solutions of the Attacker's Goal.
As result, both
P Px=Ntr=3 >0>0
and
P Px (XTO) > 0) maximized, and equal, subject 7, T
On the other hand, given the classifier 0", for any y E Tes we have the backdoor accuracy under 0":
Px~N(r,)) (XT 0r > 0),
PP 0), Z~ ~~7(6T6 0000000000)"
T
-1- (0- ((*) TT)1/2)
2(-) CDF of the standard normal distribution
T
2 (0))
maximized and only if y = o0* result, we have 71 =- Hence, the optimal backdoor trigger 7" corresponds to the backdoor classifier 0" unique and admits the form of 7" == =off" 1|
/||@"|| /0'/'|0'' (0- + 0) due to the Schwarz inequality, As
A.2.3 Proof of Theorem 3
Proof. Following the same setup in Theorem and from the result in Theorem 2, the attacker knows both the clean and backdoor distribution. Hence, conclude the result by the Neyman Pearson Lemma and the Lemma 2.
B Experiments Configurations
B.1 Data Description
CIFARIO: The CIFAR-10 dataset is a highly popular dataset in the field of machine learning research. It consists of 60,000 color images, each with resolution of 32x32 pixels. The dataset is divided into 10 classes, with 6,000 images per class. Specifically, there a training set with 50,000 images and a test set with 10,000 images.
The GTSRB dataset, known as the German Traffic Sign Recognition Benchmark, has gained popularity in the field of Backdoor Learning. It consists of total of 60,000 images distributed among 43 different classes, with varying resolutions ranging from 32x32 to 250x250 pixels. The dataset is split into a training set containing 39, 39,209 images and test set containing 12,630 images.
GTSRB:
17

---

Tiny ImageNet:
Tiny ImageNet is comprised of a collection of 100,000 images belonging to 200 classes. Each class consists 500 images, with 64x64 dimensions, resulting in colored images. The dataset is further divided into subsets, with 500 training images, 50 validation images, and 50 test images allocated for each class.
SST-2: The dataset used in our study is modified version of the Stanford sentiment analysis dataset, specifically the 2-class variant known as SST-2. The SST-2 dataset consists of 9,613 samples, while another variant called SST-5 contains 11,855 reviews. Additionally, the SST dataset includes phrases associated with each of the sentences.
IMDB: The dataset used in work is binary dataset comprising of 12,500 movie reviews in each class. The reviews are multi-sentence and are presented in the form long sentences. For our study, have extracted the first 200 words from each review.
B.2 Packages used for generating backdoor attacks/data/moodes
In this section, we provide a detailed description of the experimental setup. To conduct our exper- iments, we utilized three open-source backdoor packages, the specifics of which summarized in Table For most computer vision (CV) backdoor attacks, we implemented them using both the Backdoor ToolBox [75] and the BackdoorBench [76] ensure the consistency of our results. In the case of WaNet and SSBA, the BackdoorBench [49] package offered implementations for CIFAR-10, GTSRB, and Tiny ImageNet, so utilized their package obtain the latent representa- tions. Regarding the TacT, Adaptive Patch, and Adaptive Blend, the Backdoor ToolBox package offered implementations for CIFAR-10 10. GTSRB, representations. Lastly, for NLPP backdoor attacks, we relied on the OpenBackdoor package, which specifically designed for NLP backdoors.
utilized their package to obtain the latent
The results in the main text are directly obtained by running the Default scripts for the three packages, specified in Table 3. Ablation studies on different model architectures and poisoning rates are included in Section D and E, respectively.
Table 2: Open-source packages applied in our work
Backdoor ToolBox [75] BackdoorBench [76] OpenBackdoor [49]
BadNets Blended
TrojanNN
SIG
Dynamic
TacT
WaNet
SSBA
Adaptive-Blenn
Adaptive-Patch
SOS
LWP
B.3 On the selection of Hyperparameters
Within this section, we provide an outline for the selection of the shrinkage parameter utilized in the -rrunk-Covariance Mahalanobis. The primary motivation behind employing SCM is address the issue of unstable estimation in the inverse of the sample covariance matrix, which can result in non-IID property of samples within Vaal and future test data. Such non-IIDness can override the order information for those samples, affecting the F'PR as described in Theorem 1.
To ensure the IID property of the samples within Val as well as future test samples, we propose the following approach for selecting the shrinkage parameter 3. Firstly, we partition Val into two mutually exclusive datasets, namely DD, and DD2, with the size of DD greater than that of Dz,. Next, we compute the Sounk-Covariancee Mahalanobis (SCM) scores based on D2 and perform search for an appropriate value of 9 that ensures the SCM scores, 83 (Di)) for the samples within D, and (222) IID samples. To verify the IID property, conduct a Two-sample Aalnngorov--miman Test
18

---

Table Open- Open-source packages used for the results in the main text
Backdoor ToolBox [75]
BackdoorBench [76]
OpenBackdoor [49]
BadNets
Blended
TrojanNN
SIG
Dynamic
TacT
WaNet
SSBA
Adaptive-Blend
Adaptive-Patch
SOS
LWP
Bltack-BadNNels
Allack Blended
AttackTTrojenNit
Attack:SGG
AltackDynamicc
ralse Positvv Rate
Theary Upper Bound
Theery Upoer Bound
Thezey upper Souns SCMt
Treary Upper Dours SCH
Theery Usper Bound EBD-SCM
Altack:WaNel
AStack-SBAA
Attack TacT"
Adaptiid Patch
Allack:Adaplive Blend
Poloe Posttive Rate
tipper
Figure 6: The mean FPRs of our proposed method on CIFARIO are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are < 0.01 for all cases. Our method's FPRs consistently match theoretical upper bounds.
with decision rule based on p-value of 0.05. We repeat the above procedure ten times and observe that for CV (NLP) attacks, f 0.5 (0.7) satisfies the aforementioned requirement. Consequently, we set 3 = 0.5 for all CIFAR 10, GTSRB experiments (for Tiny ImageNet: B = 0.1), and 3 = 0.7 for all NLP experiments. Ablation studies investigating different values of 3 included in Appendix
Omitted Experimental Results in Main Text
C.1 Results on the FPR for CIFARIO
This section presents the results of the False Positive Rate (FPR) analysis conducted on CIFAR10. Figure illustrates the performance of our proposed CBD-SCM. Notably, our method consistently achieves FPR values that align with the theoretical upper bounds.
C.2 Results on the detection power (ROC) for IMDB Dataset
We assess the detection performance of proposed method IMDB (NLP) using the base uncased- BERT model. Specificallyy, we generate ROC curves for our methods under two advanced backdoor attacks, illustrated in Figure 7. The results indicate that method outperforms all other methods.
19

---

Attack: LWP, Dataset: IMDB (NLP)
Attack: SOS, Dataset: IMDB (NLP)
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
SCM (Our Method), AUC Score:0.996
SCM (Our Method), AUC Score:0.986
0.2
STRIP, AUC Score:0.44 0.71
0.2
STRIP, AUC Score:0.713
AUC Score:0. 971
AUC Score:0. .688
AUC Score:0.967
MAD, AUC Score:0.962
0.
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1/0
FPR
FPR
Figure 7: ROC of our SCM method IMDB NLP. Our proposed method consistently outperforms all other methods.
D Ablation Study: Different model architectures
In this section, we demonstrate the performance of proposed methods using the VGG 19 model. The dimension of the avgpool layer in VGG 19 is 25088, which is impractically large. Therefore, utilize the classifier layer with dimension of 4096 for practical purposes. We have observed that the empirical FPR performance of our method consistently aligns with the theoretical bounds in all cases. Consequently, omit the results for FPR performance and focus reporting the results for detection power (AUCROC, ROC) in the following subsections.
D.1 Detection Power on CIFARIO
We assess the detection performance of proposed method CIFARIO using VGG 19. We present the ROC curves for our method in Figure and 9. It is evident that our SCM method consistently outperforms other methods in all attack scenarios. Specifically, for certain advanced attacks such as the Dynamic and WaNet, we observe a remarkable improvement of approximately 200% in terms of AUCROC.
Attack: BadNets, Dataset: CIFARIO
Attack: Blended, Dataset: CIFAR10
Attack: Dynamic, Dataset: CIFAR10
0.6
08
0.
0.6
SCM (Our Method), AUC Score: 999
0.6
1P%.
Scora:0.908
fo, AUC Score:O 112 MAD, 886
2.4
0.4
SCM [Our Method), AUC Score:O 985
SCM [Our Method), AUC Score:0 994
0.2
Score:0..75
02
0.2
fa. AUC Score:O 855
Es, AUC Score:0
MAD, AUC Scorev0.108
0.0
MAD. AUC Score-0..42
FPR
FPR
Figure ROC of our method on CIFARIO with VGG19. Our proposed method consistently outperforms other methods.
D.2 Detection Power on GTSRE
We evaluate detection performance of our proposed method on GTSRB using the VGG 19 model. The ROC curves for our method are displayed in Figure 10 and 11. is evident from the results that our SCM method consistently outperforms other methods in the majority of attack scenarios. Notably, for advanced attacks like SSBA, observe significant improvement in terms of AUCROC.
20

---

Attack: SIG, Dataset: CIFAR10
Attack: WaNet, Dataset: CIFARIO
Attack:: S5BA, Dataset: CIFAR10
Method), Score:1.0
(Our Methodi, Scorc:O
(Our Mothod),
STRIP. AUC 0.945
..... STRIP, AUC Score:O. 333
.....
STRIP, AUC Score:D.82
Score:0.006
AUC
Scorc:0 504
MAD, AUC Score:0. 993
MAI. AUC Score:0. 357
0.0
MAD, AUC Score:0.311
00
02
92
0/4
0.5
68
10
0.0
02
8.4
0b
68
10
FPR
FPR
FPR
Figure 9: ROC of our method on CIFARIO with VGG19. Our proposed method consistently outperforms other methods.
Attack: SSBA, Dataset: GTSRB
Attack: WaNet, Dataset: GTSRB
1.0
1.0
(Our Method), AUC Score:0.991
(Our Method), Scare:0.97
0.2
STRIP, AUC Score:0.791
0.2
STRIP, AUC Score:0.296
12.
AUC Score:0.88 0.8
f2. AUC Score:0,99
MAD, AUC Score:0.105
MAD, AUC Scare:0.009
0.0
0.0
0.0
0.2
0.4
0.6
0.8
10
0.0
0
0.4
0.6
0.8
1.0
FPR
FPR
Figure 10: ROC of our method on GTSRB with VGG19,
Attack: Blended, Dataset: GTSRB
Attack: BadNets, Dataset: GTSRB
Attack: Dynamic, Dataset: GTSRB
08
0.6
TPR
TPR
TPR
0.4
0.4
SCM (Dur Methodi, AUC Score:D 99
SCM (Our Method; AUC Score:0.55 92)
SCM (Our Methedi, AUC Score:O 989
0.2
STRIP, AUC Score:0.929
02
STRIP. AUC
0.2
STRIP. 199
fo. MUC Score:0 395
AUC Score:0.99 991
C.. AUC Score:0 906
0.0
MAD, AUC Scoreco. 674
0.
MAD, AUC. Score:D 007
0.0
MAD, AUC Scare:0.011
FPR
FPR
FPR
Figure 11: ROC of our method on GTSRB with VGG19.
E Ablation Study: Different poisoning ratios
In this section, we provide the results obtained by using different poisoning ratios. The summary of the results for CIFAR 10 can be found in Table 4. It is observed that the detection performance of our method exhibits minimal variations across different poisoning ratios. This finding further reinforces the robustness and universal effectiveness of our method.
F Ablation Study: Different choices of |
In this section, we provide empirical studies on the effect of using different B. Specifically, we select {0.3, 0.4, 0.6, 0.73.
21

---

Table 4: AUCROC score our method on CIFARIO
Poisoning Ratio >
0.3%
1%
5%
BadNets
0.99
0.99
0.99
Blended
0.96
0.96
0.96
WaNet
0.88
0.91
0.94
SSBA
0.92
0.95
0.97
0.3 AttackcTrojanNi
Attack Attack-BadNet
Attack Blended
Attack:SiG
Attack:Dnammcc
Falve Posttive Bate
upper
upper louus
ca-
c2.
Attack:WaNet CED-SOM
Attack:SSBA
Attack-TacT cii SCM
Attack:Adapive Patch
AttacicAdaptive Blend
Falbe Positive Rate
CED-SON
caD-scM
Figure 12: The mean FPRs of our proposed method with 8 = 0.3 on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are < 0.01 for all cases.
F1 Performances on FPR for CIFARIO
This section presents the results of the False Positive Rate (FPR) analysis conducted on CIFARIO with different choices over 3. Figure 12, 13, 14, 15 illustrate the FPR performance of our proposed CBD-SCM, with 3 = 0.3, 0.4, 0.6, 0.7 respectively. Our method consistently achieves FPR values that align with the theoretical upper bounds.
F22 Performances on detection power (AUCROC, ROC) for CIFAR10
We assess detection performance of our proposed method on CIFARIO with different choices over B. Figure 16, 17, 18, 19 illustrate the ROC of our proposed SCM, with B 0.3, 0.4, 0.6, 0.7 respectively. The AUCROC scores exhibit minimal variations when different values of are used, indicating the consistent and robust effectiveness of our method. This observation reinforces the stability and universality of our selected 3 = 0.5 as reported in the main text.
0.4 AttackcTTnjanNN
Aftack-BadNitt
Attackc Blended
Attack-SiG
Altack-Dynamic
Falbe Positive Bate
Theiry Upper Boong
Upper tourd
Upper boung
Therry (apar Bours
Thoory (pp Brurg
Attack:WaNet
Attack:SSBA
Attack-TbcT
Adtaci-Adapiive Patch
Adaative Blend
False Posative Bate
upper Bours cepacc
Theery
Figure 13: The mean FPRs of our proposed method with B = 0.4 on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are < 0.01 for all cases.
22

---

A 0.6
Attack.Baddees
Attack Blended
AttackcTrojanNN
Attack:siG GCM
Attack-Dynamic Calb SCM
Falbe Positive Bake
CAD-SOM SCM
CRD SCN
SCM
MttackWWalle
Attack: lecl
Adtack-Adspivee Patch
Adaatic Blend
Falue Posttvve Rate
Reand
foand 8
forst 8
Theery Houns capach
Theeey cap.s3)
C20-501
cep scu
Figure 14: The mean FPRs of our proposed method with 8 = 0.6 on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are < 0.01 for all cases.
B= Attack:TrojaNNN
AttackBBadNets
Attack Blended
Attack:5GG
Attack:Dynamic
Falue Postive Ratt
Theoryy Upperillendd
Thesry upper iound C00--O
Thecry upper Sound CBo SCM
Theery Jpper lours 0B3-So
Theery (ppar Bouns cap-sca
C3P-204
AtteckeWaleet
KSSSAA
Treary Uspa Boung Attack:
31 Patch
Adaptive Blend
Feloe Positivve Rate
Thery Upper Soarg
Theory lpper tourl cat SCN
Theary Jpp Bouns cB0-SCM
Theor Upper Boung
Figure 15: The mean FPRs of our proposed method with 8 = 0.7 on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are 0.01 for all cases.
B= 0.3 Attack-TrajarNN, AUCROC:0.09
Bodotss
Attack.Blerded, AUCRDC-0.971
Attack::S G, AUCROC:0.99
Attaci-Dynamic. AUCkÓC:1.0
AUCROC:LO
Tuu Postbve
AttackoWSNet!
Amtačks SBA,
Jetack 1STT,
Atrack:Adaptivee Patch. AUCROC:O 987
AttadcAdoptive Blend. AUCROC:0 971
AUCROC-0.300 9.96
AUCROCO.999 0.982
AUCROC:0.50 9.77
Thue Pesttee Paye
Polce Posttie Rate
Folse Positive
Postive Rate
False Pesttive
False Pesttive Rate
Figure 16: ROC of our method CIFAR10 with 0.3
23

---

Attaci-Bad BadNats AUCROC:1.0
Attack-Blended,
Attack-TajarNN, AUCROCO
Attari-Dynamic. AUCROC:1.0
The Repttv
Metackk AUCROC:O.955
AttackAdaptive
AttadkAdaattive AUCROC:O
Thue Potth
False Poskive Rato
False Fosittve Rate
False Positive Rate
False Fositive Rate
Fasse Nusttve Rato
Figure 17: ROC of our method on CIFAR10 with = 0.4
8 0.6
Attack-BadNets, AUCROC:1.0
Attack:Blended. AUCROC:0.963
Altack-TrajanNN AUCROC:0.994
Mitck:SSG. AUCROC:0.9e 96.6
Altack:Dynamic, AUCROC:1.0
Thys Prothe Rate
Attack:WSSett AUCROC.0.92
Artačk: SBA
Xetack-BSST, AUCROC:0.92
AttackcAdaptive Patch,
AttacicAdagttvv Blend, AUCROC:0 963
AUCROC-0.971
AUCROC:0.900
Tu0 Postive Rote
Past Rate
Palse
Poudtive
Pelie
Posilice Rate
Figure 18: ROC of our method on CIFARIO with = 0.6
= 0.7 Atrack-TrajanN, AUCROC:0.99
Attack.BodNots, AUCROC:1. 0
Attack-Blended, AUCRDC-0.611
Attack:SIE G,
AttacicDynamic, AUCROC:1.0
AUCROC:O 9911
Thue Pesthee Raye
AttackoWiNeet"
ArtackS SBA.
K AUCROC.O.RO
AttackcAdaptiee Patch, AUCROC:0.976
Attack-Adaptüe Blend, AUCROC.O 961
AUCROC.0.90 9.998
AUCROC-0.966
The Posttive Raye
Falua Pooitive
Fullce Posilivv
Nusitivn
Figure 19: ROC of method on CIFAR10 with 3 0.7
24

---

0.3
AttackBBadNets
Attack:Blend
Attack: Trojjan
Attack:SIG
AttackiDynamic
Fabee Positive Rate
Therry Upper Boan
Theary upper Soarl
Theary Uspa Bound
Theary Japa Bound
Thearr upe Bours
Atteck:WaNet
Attacki/SSAA
Attack-TocT
Attack:Adapive Patch
AttacicAdaptive Blend
False Povitive Bate
upper
upper
upper tours
Figure 20: The mean FPRs of our proposed method with 3 = 0.3 GTSRB are shown in each
plot, which is independently replicated times. The solid line represents the mean value, and the standard errors are < 0.01 for all cases.
8 - 0.4
Attack: lound
Trojan
Attack-SIG
Attack:Dynamic
Falon Positive Rake
Theary Boand ceo SCM
Sound
Theary Bound Ca3-ScM
Theery
cab SCN
cao SCM
Cla Saa
AttadciSSBA
Adspiie
Attack/Adaptic
Faliee Positive Rare
thary Uipee Boan CID-SON
Theary upper sund 2 cec SO1
Tneetr Uppa toung cap SCM
Therry Appa Bours CBp-ScM
Thory Bourg cap-sor
Figure 21: The mean FPRs of our proposed method with 3 0.4 on GTSRB are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are < 0.01 for all cases.
F33 Performances on FPR for GTSRB
This section presents the results of the False Positive Rate (FPR) analysis conducted on GTSRB with different choices over 3. Figure 20, 21, 22, 23 illustrate the FPR performance of our proposed CBD-SCM, with 3 = 0.3, 0.4, 0.6, 0.7 respectively. Our method consistently achieves FPR values that align with the theoretical upper bounds.
E4 Performances on detection power (AUCROC, ROC) for GTSRB
We assess the detection performance of our proposed method on GTSRB with different choices over 8. Figure 24, 25, 26 illustrate the ROC of proposed SCM, with B 0.3. 0.4, 0.6 respectively. The AUCROC scores exhibit minimal variations when different values of B are used, indicating the consistent and robust effectiveness of our method. This observation reinforces the stability and universality of our selected B = 0.5 as reported in the main text.
25

---

A 0.6
Attack.Baddees
Attack Blend
Attack.Trojan
Attack:siG GCM
Attack-Dynamic Calb SCM
Falbe Positive Bake
CAD-SOM SCM
CRD SCN
SCM
MttackWWalle
AtteckiSSSA
Attack: lecl
Adtack-Adspive Patch
Adaatic Blend
Falue Posttive Rate
Readd 2
foand
forst 8
Theery Houns capach
Theeey cap.s:
c20-001
cep scu
Figure 22: The mean FPRs of our proposed method with 3 = 0.6 on GTSRB are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are < 0.01 for all cases.
B= Attack:Trojan
AttackBBadNets
Attack Blend
Attack:5GG
Attack:Dnnamic
Falue Postive Rttt
Theoryy Upperillendd
Thessy upper inund DE--SO
Theary upper lound CBo SCM
Theery Jpper lours 0B3.So
Theery (ppar Bouns
C3P-204
cap-sca
AtteckeWaleet
Att actai
31 Attack:
31 Patch
Adaptive Blend
Feloe Positivve Rate
Thery Upper Sward
Theary lpper toarl cet SON
Treary Uspa Boung
Theary Jpp Bouns
Theor Upper Bourg
Figure 23: The mean FPRs of our proposed method with 9 0.7 on GTSRB are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are 0.01 for all cases.
B= 0.3 Attack-TrajarNN, AUCROC:0,09
BoNotss
Attack.Blended, AUCROC-0.954
Attack:SI G, AUCROC:0. 8.8
Attaci-Dynamic. AUCkÓC:1.0
AUCROC:LO
Tuu Postbve
AttackoWSNet.
Astačk:55BA
Jetack ISTT." AUCROC-0.11
Atrack:Adaptivee Patch. AUCROC:0 999
AttadkAdbgtive Blend. AUCROC:0 933
AUCROC-0.90 9.997
AUCROC-10
Thue Pestive paye
Polce Posttive Rate
Folse Positive
Postive Rate
False Pesttive
False Pesttive Rate
Figure 24: ROC of our method GTSRB with 0.3
26

---

= 0.4
Bttack-addNeets AUCROC:10
Attack-Bjended
Attack-TrajarNN, AUCROC:,
Attack:: G.
Attari-Dynamic.
AUCROC.0.934
AUCROC:0.791
AUCRÓC:1.0
1h.4 Roethng Rete
AttackWWSNet AUCROC-0,991
Srtačk:SSBA.
Jetackk 1BC,
Atrack:dappiive Fatch, AUCROC:O 99999
Attaci- Bland, AUCRDC:0.91
AUEROC:10
AUCROC.O.135
Thas Pesthe Rare
False Poskive Rate
False Fositive
Postiive Rais
False Rositive
Pusiiive Raic
Figure 25: ROC of method GTSRB with 0.4
Allack-Blerded AUCROC-0.891
AllackTrojarNN. AUCROC:0.999
Allack-Draamic.
AUCROC.1.0
AUCROC:0.709
The Porthve Rate
AttackkWWöett"
Jetacck
Attacicddaptivee Pätch. AUCROC.00992
Attadkodagttie Blend, AUCROC.O.
AUCROC.D0.90
AUCRROC.0.99
AUCRDC.0.666
The Postive
Falve Poskive Rate
Falue Posittoe Rate
Falue Posdtive Rate
Fulue Pesilive Palle
Febe Positive Ralu
Figure 26: ROC of our method on GTSRB with 3 = 0.6
G Performances under additional types of backdoor attacks
evaluate our methods against backdoor attacks intentionally designed to challenge our defenses, such LIRA [12] and M-attack (a custom-designed attack). These attacks are specifically crafted to diminish the distinction in latent spaces of the backdoored models by imposing regularization on the distances. LIRA employs Wasserstein distances, while M-attack utilizes Mahalanobis distances, to measure the dissimilarity between clean data and backdoor attacks. Table 5, our methods consistently outperform state-of-the-ar defenses, although there is slight performance dip compared to diverse attack scenarios like BadNets and SSBA. Nevertheless, this outcome is reasonable since no defense can be universally effective against all attack variations.
Table 5: AUCROC score of our method on GTSRB against LIRA and M-attack.
Defenses +
SCM (Ours)
FREQ
SCALEUP
LIRA
0.86
0.71
0.79
M-attack
0.82
0.80
0.71
H
Comparison with other defenses
H.1 Performances comparison with recent detection-based backdoor defenses
We evaluate our methods alongside two recently developed detection-based defenses designed to counteract CV backdoor attacks: SCALEUP and FREQ. The summarized results presented in
27

---

Table 6 below. Our observations consistently demonstrate that our methods outperform SCALEUP and FREQ.
Table 6: AUCROC score of our method on GTSRB. The bestperforming method(s) are indicated in boldface.
Defenses +
SCM (Ours)
FREQ
SCALEUP
ABL
SPECTRE
BadNets
0.99
0.91
0.86
0.97
0.96
SSBA
0.99
0.51
0.72
0.81
0.56
Adaptive Patch
0.87
0.49
0.55
0.72
0.70
Adaptive Blend
0.99
0.56
0.51
0.59
0.62
H.2 Performances comparison with purifying-based training-stage backdoor defenses
We assess our methods in conjunction with two training-stage backdoor defenses, SPECTRE and ABL. As emphasized the main text, these methods, while sharing the concept of distinguishing between clean and backdoor attacks, significantly differ from ours in terms of the threat model, methodology. and evaluation metrics. To ensure fair comparison, we report the AUCROC scores for distinguishing between clean and backdoor data. The summarized results are presented in Table 6. Our observations consistently demonstrate that our methods outperform ABL and SPECTRE.
28


Reviews:
[{"review_id": "8j89YaqiIn", "reviewer": "Reviewer_vAAp", "summary": "This paper proposes a unified inference-stage detection framework to defend against backdoor attacks. The authors first formulate the inference-stage backdoor detection problem, discuss its challenges and limitations, and then suggest a framework with provable guarantees on the false positive rate or the probability of misclassifying a clean sample. The authors also derive a detection rule to maximize the rate of accurately identifying a backdoor sample, given a false positive rate under classical learning scenarios. Based on this, they then suggest a practical and effective approach for real-world applications. The proposed method was evaluated on 12 different backdoor attacks on computer vision and NLP benchmarks. The experimental findings align with the theoretical results, showing significant improvements over the state-of-the-art methods.", "questions": "- For all the backdoored models used in the paper, what is the percentage of poisoned training data, and what are the clean and robust accuracy of these models (before the defense). It is crucial to know this as different defenses might behave differently as the “strength” of the backdoor attack varies. I encourage the authors to include these details along with an ablation study for this.\n\n- As shown in Fig 4, the proposed attack performance is not very different than prior work. The authors “justify” this by saying “These findings suggest that the current NLP attacks retain a considerable amount of information in the latent representations that can be utilized to differentiate between clean and backdoor data.” Can the authors explain this in more detail? What is special about NLP? Are backdoor attacks themselves weaker there? This ties back to my first point on clarifying the performance of the backdoor models on clean and modified data.\n", "limitations": "The authors adequately addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The proposed framework for defending against backdoor attacks is novel to the best of my knowledge.\n- The paper is sound and decently written (beyond some mathematical clutter, see belo).\n- The proposed method is validated through extensive experiments on multiple datasets and compared to many existing defenses, demonstrating its effectiveness.\n- The authors provide a theoretical analysis and derive technical insights on toy settings which motivates their practical defense on the real settings.", "weaknesses": "- There is quite a bit of mathematical clutter in S2 and S3 which I believe can be avoided for a smoother read.\n- Some of the details about the backdoored models are not present in the paper (see below) which make it a bit hard to asses the faithfulness of the comparisons to previous models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewers for investing their time and energy into reviewing our manuscript and offering valuable feedback. We're pleased that they recognized the quality of our writing, acknowledged the novelty of our proposed framework (Reviewer vAAp), and found our approach effective across various domains and setups (JEgp, ibgE, 1Ekh). We will incorporate all their comments into the revised paper.\n\n> Q: Implementation details\n\nR:  To fit the page limit, we’ve included all the implementation details and several ablation\nstudies in the originally uploaded supplementary material. We summarize them below.\n1. Open-source package used. All the backdoor attacks and associated backdoor data in our work are obtained based on open-source projects. For computer vision (CV) attacks, we employed the Backdoor ToolBox [1] and BackdoorBench [2] to ensure result consistency. NLP attacks were conducted using the specialized OpenBackdoor [3] package.\n2. We provide the poisoning rate, clean accuracy, and backdoor accuracy for backdoor attacks presented in the main text in Table 1 below. In general, the backdoored models exhibit comparable clean accuracy to the normal models, while achieving nearly perfect accuracy on backdoor data, with a relatively low poisoning ratio.\n\nTable 1: Poisoning ratio, clean and backdoor accuracy of backdoor attacks used in our paper. * For the SSBA and WaNet attacks, backdoor poisoning rates are 5\\%. For the A-Blend and SIG attacks, the backdoor accuracies are around 80\\%. † For the A-Patch attack, the backdoor accuracy is around 60\\%.\n|                     |                         |Backdoor Model| Backdoor Model| Clean Model |\n|--------------|-----------------|----------------|------------------|----------------|\n| Dataset ↓     | Poisoning Ratio | Clean Accuracy | Backdoor Accuracy | Clean Accuracy |\n| CIFAR10*     | 0.3%        | ≥ 93%      | ≥ 97%        | ≥ 93%      |\n| GTSRB†        | 1%          | ≥ 95%      | ≥ 97%        | ≥ 96%      |\n| Tiny ImageNet | 1%          | ≥ 37%      | ≥ 96%        | ≥ 40%      |\n| SST2 (NLP)    | 10%         | ≥ 91%      | ≥ 99%        | ≥ 92%      |\n| IMDB (NLP)    | 10%         | ≥ 90%      | ≥ 96%        | ≥ 91%      |\n\n\n\n> Q: Ablation study on poisoning rates and model architectures\n\nR: We provide the outcomes of two ablation studies: (i) Table 2 below, addressing the poisoning ratio, and (ii) **Figures 1 in the PDF file of the global response**, depicting detection power with VGG 19. Overall, our method's detection performance remains consistent across diverse poisoning ratios and architectures. This highlights the stability and efficacy of our approach.\n\nTable 2: AUCROC of our proposed method on CIFAR10\n| Poisoning ratio   | 0.3% | 1% | 5% |\n| :---------------- | :------: | :----: | :----: |\n| BadNets        |   0.99  | 0.99 | 0.99 |\n| Blended           |   0.96  | 0.95 | 0.96  |\n| WaNet           |   0.85  | 0.91 | 0.93 |\n| SSBA           |   0.92  | 0.95 | 0.97 |\n\n\n\n> Q: Explanations on the claim of \"These .. data.\" regarding the performance of our methods on NLP backdoor attacks\n\nR:   We explain our claim of \"These .. data.\" in the following.\n- **The discrete nature of NLP data often leads to noticeable NLP backdoor triggers**.\nNLP backdoor attacks stand apart from CV backdoor attacks due to the distinct nature of their data representations. Image data is commonly described by continuous values, while textual data is characterized by symbolic and discrete forms. This discreteness often renders NLP backdoor triggers more conspicuously visible [4], in contrast to the human-imperceptible CV backdoor triggers [5]. For example, the SOS attack [6] introduces the non-relevant and easily noticeable word combination 'mn' into the text.\n- **Noticeable NLP backdoor triggers lead to clear distinctions between the latent representations of clean and backdoor data.** \nDue to the conspicuous nature of backdoor triggers in NLP attacks, distinct differences emerge in the latent representations of clean and backdoor data under different backdoor attacks. Defenders can leverage these differences to accurately discern, e.g., with near-perfect accuracy, between clean and backdoor data, as empirically demonstrated in our main text. \n\nAs a result of the above two points, we made the claim that \"current NLP attacks retain a ... utilized to differentiate between clean and backdoor data\" in our main text. We will improve the clarity of this statement in the revision.\n\n> Q: mathematical clutter in S2 and S3\n\nR: We will enhance the clarity of concepts, expressions, and language in the revised version.\n\n### Reference\n[1] T. Xie, “Backdoor toolbox,” https://github.com/vtu81/backdoor-toolbox, 2022.\n\n[2] B. Wu, H. Chen, M. Zhang, Z. Zhu, S. Wei, D. Yuan, and C. Shen, “Backdoorbench: A comprehensive benchmark of backdoor learning,” in NIPS Datasets and Benchmarks Track, 2022.\n\n[3] G. Cui, L. Yuan, B. He, Y. Chen, Z. Liu, and M. Sun, “A unified evaluation of textual backdoor learning: Frameworks and benchmarks,” arXiv preprint arXiv:2206.08514, 2022.\n\n[4] Chen, X., Salem, A., Chen, D., Backes, M., Ma, S., Shen, Q., and Zhang, Y. \"Badnl: Backdoor attacks against nlp models with semantic-preserving improvements,\" in ACSAC, 2021\n\n[5] Anh Tuan Nguyen and Anh Tuan Tran. WaNet – Imperceptible wapping-based backdoor attack. In ICLR, 2021.\n\n[6]  W. Yang, Y. Lin, P. Li, J. Zhou, and X. Sun, “Rethinking stealthiness of backdoor attack against nlp models,” in ACL, 2021."}, {"review_id": "Y6y2qa4hH9", "reviewer": "Reviewer_JEgp", "summary": "This paper formulates the inference-stage backdoor detection in terms of backdoor-sample identification and proposes a unified defense framework. It derives a theoretically optimal detection rule and validates its effectiveness in both CV and NLP domains.", "questions": "Please see the weakness section.", "limitations": "Please see the weakness section.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-organized with a comprehensive structure. For example, section 3, first formulates the backdoor detection and then illustrates the proposed conformal detection, followed by the derivation of optimal score functions and a practical proxy, which is clearly illustrated.\n2. The proposed method can work well under different domains, including CV and NLP, which are more general than the domain-specific methods.\n3. The idea is theoretically correct and easy to follow.\n4. The experiments are reliable with 10 times independent repetition as illustrated in 4.1.\n5. The comparisons under different backdoor attacks are sufficient.", "weaknesses": "1. Some redundancy and unclear expressions exist. For example,\n    1). The $\\lambda_{\\alpha,s}$ is first shown in line 5, Algorithm 1, and $\\tau$ is used in Equation (1), where the relationship between them should be clearly claimed.\n    2). The backdoor trigger is defined unclearly and misunderstood in section 3.3. The $\\eta_1$ expressing backdoor trigger is used in lines 225 and 244, while in line 233, it represents the backdoor transformation. Also, the $\\gamma$ is used in other places.\n2. Lack of comparisons with the SOTA backdoor defense methods that include the separation of the input data, such as the ABL[1] and DBD[2].\n3. The implementation code of this paper does not release.\n\n[1] Li, Yige, et al. \"Anti-backdoor learning: Training clean models on poisoned data.\" *Advances in Neural Information Processing Systems* 34 (2021): 14900-14912.\n\n[2] Huang, Kunzhe, et al. \"Backdoor defense via decoupling the training process.\" *arXiv preprint arXiv:2202.03423* (2022).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewers for investing their time and energy into reviewing our manuscript and offering valuable feedback. We're pleased that they recognized the quality of our writing, acknowledged the novelty of our proposed framework (Reviewer vAAp), and found our approach effective across various domains and setups (JEgp, ibgE, 1Ekh). We will incorporate all their comments into the revised paper.\n\n> Q： Some redundancy and unclear expressions exist\n\nR: \n- The relationship between $\\tau$ and $\\lambda_{\\alpha,s}.$ In brief, $\\tau$ and $\\lambda_{\\alpha,s}$ denote the same decision value, but they are referenced differently in the text and Algorithm 1's pseudocode. Specifically, within our BCD framework, $\\tau$ is sought as a solution to Equation (1) based on given $\\alpha$ (type I error rate), $\\delta$ (violation rate), and $n$ (sample size of the validation dataset). This $\\tau$ is later employed as the decision threshold $\\lambda_{\\alpha,s}$ in the pseudocode of Algorithm 1. We will correct and unify the notations in the revision.\n- The backdoor transformation $\\eta_1$ and the backdoor trigger $\\gamma$.\nWe use $\\eta_1$ to denote the general backdoor transformation (from clean data to backdoor data). For instance, in poisoning backdoor attacks, we have $\\eta_1(x) = x + \\gamma$, with $\\gamma$ being the backdoor trigger.\nIn Line 225, when we mention \"transforming clean data with backdoor triggers $\\eta_1$,\" we are referring to converting clean data into backdoor data using the backdoor transformation $\\eta_1$.\nIn Line 244, there is a typo. It should read \"backdoor triggers $\\gamma \\in T_c$ that\" instead of \"backdoor triggers $\\eta \\in T_c$ that\".\n\n\n\n\n\n\n\n\n> Q: Comparisons with the SOTA backdoor defense methods that include the separation of the input data, such as\nthe ABL[1] and DBD[2].\n\nR: First, It's important to emphasize that the ABL [1] and DBD [2] methods differ significantly from our proposed approaches in terms of threat models and methodology, as detailed below. \n\nABL [1] and DBD [2] are **training-stage defense** approaches with access to both clean and backdoor training data, **affording them information about both types of clean and backdoor data**. Their goal is to differentiate between clean and backdoor training data, ultimately constructing a clean model through the use of filtered clean training data, obtained by analyzing all training instances. Hence, their evaluation metrics are the clean and attack success rate of the cleansed model. \n\nIn contrast, our method is an **inference-stage defense** that relies solely on a very small (e.g., 1000) set of clean validation data and operates **without any knowledge of backdoor data**. Our goal is to detect future backdoor inputs, and our evaluation metric is the AUCROC score of the detector.\nTherefore, direct comparisons between ABL [1], DBD[2], and our methods may not be equitable or appropriate. \n\n**Nonetheless**, we included the AUCROC scores of ABL [1] and DBD [2] on distinguishing between clean and backdoor data in Tables 1 and 2 below. We believe that our method consistently outperforms ABL [1] and DBD[2] under different types of backdoor attacks. These results underscore our method's superiority over ABL [1] and DBD[2] in terms of detection performance.\n\n\nTable 1: AUCROC performance comparison of our method with ABL[1] and DBD[2] on CIFAR10\n| Defense ↓        | BadNets| Dynamic | SSBA | Adaptive-Blend | Adaptive-Patch | \n| :---------------- | :------: | :----: |:----: |:----: |:----: | \n| Our Method       |   1.0  | 1.0 | 0.97 | 0.96 | 0.98\n|   ABL    |   0.98  | 0.92 |   0.81 | 0.79 | 0.72|\n| DBD  |  0.98 | 0.89 | 0.87 | 0.78 | 0.67|    \n\nTable 2: AUCROC performance comparison of our method with ABL[1] and DBD[2] on GTSRB\n| Defense ↓        | BadNets| Dynamic | SSBA | Adaptive-Blend | Adaptive-Patch | \n| :---------------- | :------: | :----: |:----: |:----: |:----: | \n| Our Method       |   0.99  | 0.99 | 0.99 | 0.99 | 0.87\n|   ABL    |   0.97  | 0.85 |   0.81 | 0.59| 0.72|\n| DBD  |  0.97 | 0.84| 0.77| 0.78 | 0.67|    \n\n\n\n\n> Q: Code Release\n\nR: We have already included the preliminary codes in the originally uploaded supplementary materials, allowing the reproduction of NLP backdoor attack results showcased in Figure~4 of the main text. Replicating all other results merely necessitates adjustments to the clean and backdoor representations. That being said, following the convention, we will release the complete version of our codes upon the paper's acceptance.\n\n### Reference\n\n[1] Li, Yige, et al. \"Anti-backdoor learning: Training clean models on poisoned data.\" Advances in Neural Information Processing Systems 34 (2021): 14900-14912.\n\n[2] Huang, Kunzhe, et al. \"Backdoor defense via decoupling the training process.\" arXiv preprint arXiv:2202.03423 (2022)."}, {"review_id": "xkse6TgR0t", "reviewer": "Reviewer_ibgE", "summary": "This paper proposes a backdoor sample detection method. It utilizes Mahalanobis distance as the score function to compute the probability of a given sample being poisoned. It also leverages an existing statistical tool, the conformal prediction framework, to determine a statistical threshold for the computed scores given a false positive rate (recognizing clean data as poisoned). The experiments are conducted on three image datasets and two text datasets. Comparing to several baseline methods, the proposed approach achieves higher detection rate on poisoned samples.", "questions": "See above.", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper studies an important problem of detecting backdoor samples. The evaluation shows the proposed approach is effective against various attacks.\n\n2. The conformal prediction framework is used in the paper to statistically balance the true positive rate and the false positive rate, which is interesting. The Mahalanobis distance is leveraged to differentiate the statistical difference between clean and poisoned data, which is empirically validated in the paper.\n", "weaknesses": "1. While this paper compares with a few baselines, it still misses a closely related work SPECTRE [41]. SPECTRE also leverages statistics techniques to estimate the mean and the covariance of the clean data, which is then utilized to differentiate poisoned samples from the clean data. What is the fundamental difference between the proposed approach and SPECTRE? This paper seems to just use a different statistics tool to estimate the clean distribution. Fundamentally, it is no different from SPECTRE. However, there is no discussion, comparison, and empirical evaluation regarding SPECTRE.\n\n2. Although the evaluation considers a number of attacks, an important aspect is missed in the paper. A knowledgeable attacker who is aware of the proposed detection approach can design an attack specifically targeting it. Particularly, this paper uses the Shrunk-Covariance Mahalanobis (SCM) score function to distinguish clean and poisoned data. An adaptive attacker can use this function during attack to reduce the scores for poisoned samples. This is critical to demonstrate the robustness of the proposed detection approach. In addition, there are several strong attacks that were designed to evade statistics based defenses [1,2]. They should be empirical evaluated.\n\n3. In Algorithm 1, a transformation method T is introduced as the defender-constructed transformation on the original input. However, there is no explanation on what this function is and how it is applied on the input.\n\n\n[1] Doan, Khoa, Yingjie Lao, and Ping Li. \"Backdoor attack with imperceptible input and latent modification.\" Advances in Neural Information Processing Systems 34 (2021): 18944-18957.\n\n[2] Shokri, Reza. \"Bypassing backdoor detection algorithms in deep learning.\" 2020 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewers for investing their time and energy into reviewing our manuscript and offering valuable feedback. We're pleased that they recognized the quality of our writing, acknowledged the novelty of our proposed framework (Reviewer vAAp), and found our approach effective across various domains and setups (JEgp, ibgE, 1Ekh). We will incorporate all their comments into the revised paper.\n\n> Q:  Comparaison with SPECTRE [41]\n\nR: Fundamentally, our approach diverges significantly from SPECTRE in terms of both the threat model and technical components, detailed as follows.\n\n1. **Differences in terms of the threat model, goals, and evaluation metrics**. SPECTRE [41] is a training-stage defense approach with access to both clean and backdoor training data, **affording them information about both types of clean and backdoor data**. Their goal is to differentiate between clean and backdoor training data, ultimately constructing a clean model through the use of filtered clean training data, obtained by analyzing all training instances. Hence, their evaluation metrics are the clean and attack success rate of the cleansed model. \nIn contrast, our method relies solely on a small (e.g., 1000) set of clean validation data and operates **without any knowledge of backdoor data**. Our goal is to detect future backdoor inputs, and our evaluation metric is the AUCROC score of the detector. \n\n2. **Differences in terms of Technical Aspects**. SPECTRE employs robust statistics to separate clean and backdoor training data, with **access to both clean and backdoor training data**. On the other hand, our SCM technique is employed to mitigate numerical challenges when estimating high-dimensional covariance matrices for the set of clean validation data. Notably, there is **no information available regarding the backdoor data distributions**.\n\n\nTherefore, direct comparisons between SPECTRE [41] and our methods may not be equitable or appropriate. \n\n**Nonetheless**, we have included the AUCROC scores of SPECTRE on distinguishing between clean and backdoor data in Tables 1 and 2 below. We observed that our method consistently outperforms SPECTRE under different types of backdoor attacks. These results underscore our method's superiority over SPECTRE in terms of detection performance.\n\nTable 1: AUCROC performance comparison of our method with SPECTRE [41] on CIFAR10\n| Defense ↓        | BadNets| Dynamic | SSBA | Adaptive-Blend | Adaptive-Patch | \n| :---------------- | :------: | :----: |:----: |:----: |:----: | \n| Our Method       |   1.0  | 1.0 | 0.97 | 0.96 | 0.98 |\n|   SPECTRE [41]  |  0.95 | 0.96 | 0.56 | 0.62 | 0.61| \n\nTable 2: AUCROC performance comparison of our method with SPECTRE [41] on GTSRB\n| Defense ↓        | BadNets| Dynamic | SSBA | Adaptive-Blend | Adaptive-Patch | \n| :---------------- | :------: | :----: |:----: |:----: |:----: | \n| Our Method       |   0.99  | 0.99 | 0.99 | 0.99 | 0.87 |\n|   SPECTRE [41]  |  0.96 | 0.95 | 0.56 | 0.62 | 0.7| \n\n> Q: Robustness of the proposed detection approach against a knowledgeable; comparison with strong attacks that were designed to evade statistics-based defenses [1,2]. \n\nR: Taking your advice into account, we developed an attack inspired by [1], refered to as *M-attack*, which introduced a regularization term to reduce the  Mahalanobis scores between clean and backdoor samples. Moreover, in response to your recommendations, we conducted supplementary experiments to evaluate our method's effectiveness against attacks [1,2], as summarized in Tables 3 and 4 below. \n\n\nTable 2: AUCROC performance comparison of our method under *M-attack*, [1] and [2] on CIFAR10 \n| Defense ↓        | BadNets | SSBA | Adaptive-Patch | *M-Attack* | [1] | [2] | \n| :---------------- | :------: | :----: |:----: |:----: |:----: | :----: | \n| Our Method       | 1.0 | 0.97 | 0.98 | 0.85 | 0.84 | 0.89 |\n| STRIP | 1.0 | 0.68 | 0.76 | 0.71 | 0.69 | 0.74 |\n\nTable 3: AUCROC performance comparison of our method under *M-attack*, [1] and [2] on GTSRB  \n| Defense ↓        | BadNets | SSBA | Adaptive-Patch | *M-Attack* | [1] | [2] | \n| :---------------- | :------: | :----: |:----: |:----: |:----: | :----: | \n| Our Method       | 0.99| 0.99 | 0.87 | 0.82 | 0.86 | 0.85|\n| STRIP | 0.99| 0.80 | 0.33 | 0.68 | 0.62 | 0.81 |\n\n We observed that our methods still outperform other state-of-the-art defenses, even though there is a moderate decline in performance compared to different attack scenarios like BadNets and SSBA. Nonetheless, we believe that this is a reasonable outcome, given that no defense can be universally effective against all attack variations.\n\n> Q: notations on T\n\nR: We introduced this transformation $T$ and offered specific examples in Lines 171 - 175 of the main text, before presenting the Pseudocode of Algorithm 1. In the revised version, we plan to incorporate these discussions directly into the pseudocode of Algorithm 1 to enhance clarity.\n\nSpecifically, $T(\\cdot)$ denotes a defender-constructed transformation that \ntypically depends on both the backdoored model $f^{\\text{poi}}$ and the validation dataset $D^{\\text{Val}}$, to reflect special properties of backdoor data, e.g., the predicted value \n$T(X_{\\text{test}}) = f^{\\text{poi}}(X_{\\text{test}})$ and the latent representation $T(X_{\\text{test}}) = \\phi^{\\text{poi}}(X_{\\text{test}})$.\n\n### Reference\n\n[1] Doan, Khoa, Yingjie Lao, and Ping Li. \"Backdoor attack with imperceptible input and latent modification.\" Advances in Neural Information Processing Systems 34 (2021): 18944-18957.\n\n[2] Shokri, Reza. \"Bypassing backdoor detection algorithms in deep learning.\" 2020 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 2020.\n\n"}, {"review_id": "xjInCWCRWq", "reviewer": "Reviewer_1Ekh", "summary": "This work formulates the inference-stage backdoor detection problem. The authors then propose a framework to establish provable guarantees w.r.t. the detection FPR, given some validation data on hand. Finally, they derive the optimal detection rule (in the Neyman-Pearson paradigm) in a simplified scenario, and suggest a practical proxy using the empirical Mahalanobis distance metric w.r.t. DNN's latent representations. Their results on both CV and NLP domain show significant improvements over prior arts", "questions": "1. In Line 109-111, you made a claim that \"these methods rely on the assumption that there is a clear separation...\". But isn't your method also relying on the separation of clean and backdoor data in the latent space? Do you consider utilizing such latent separation as a weakness/flaw when designing backdoor defenses? If you do, why won't your method suffer from this weakness?\n2. How is the violation rate $\\delta$ in Eq (1) selected in practice?\n3. In Figure 2, I see that for some attacks (TrojanNN and A-Blend), even your proposed optimal scoring function cannot separate well enough between clean and backdoor data. Could you discuss more about these results?\n4. Could youn also visualize the SCM score histogram for backdoor and clean inputs? It would be straightforward for readers to see how your proposed method distinguishes between clean and backdoor inputs.", "limitations": "The limitations are discussed together with future work in Sec 5.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors formally study the inference-stage backdoor detection problem. Specifically, they propose the conformal backdoor detection (CBD) framework, which ensures the detector's FPR does not deviate too much from a pre-selected value with a high probability. Their CBD framework provides theoretical guidance on how to select the decision threshold $\\tau$.\n2. They also study the optimal score function in a simplified scenario. Above this theretical analysis, they propose a practical proxy to this score function (which cannot be practically computed). They further address the potential numerical instability problem using matrix shrinkage technique.\n3. The authors conduct extensive experiments, in both CV and NLP domain. The cross-modal experiment setup indeed helps demonstrate the generalizability of their method. They consider 3 datasets and 10 attacks for CV tasks, with 2 datasets and 2 attacks for NLP task. I appreciate the diverse experimental setting very much.", "weaknesses": "1. The paper title is not expressing your work accruately. Your title is \"A Unified Framework for Inference-Stage Backdoor **Defenses**\", but your work focuses only on inference-stage backdoor **input detection** (I don't think inference-stage backdoor defense = inference-stage backdoor input detection). Also, the word \"unified\" seems quite strong, since your theoretical analysis (e.g. optimality) is mostly based on a simplified scenario. I suggest you narrow down the scope (and consider tuning down the tone) of your title.\n2. The **intuitions behind of your method** are not fully specified. For example, as a reader, I am quite confused about why the empirical Mahalanobis distance (Line 276) is a good proxy to Eq (2)? It seems like the case that you simply discard the second term in Eq (2), and try to only approximate the first term. Please explain more about the connection between your practical proxy and Eq (2).\n3. The experiment setting should include comparisons with **more baselines defenses and attacks**. First, I suggest the authors also consider other non-poisoning backdoor attacks (e.g. modifying trained model weights [1], fine-tuning, etc.). Second, there are more recent and advanced inference-stage backdoor detector baselines (e.g. [2] and [3]) other than STRIP, l2 and MAD. You should definitely consider adding at least one or two inference-stage backdoor detectors in the recent two years into comparison in the main Table 1.\n4. It's good to see several ablation studies in the Appendix (e.g. poison rates). I suggest you consider three more important **ablation studies**: 1) The number of validation data you have in hand. How many samples at least are necessary to make your defense effective? 2) I appreciate your consideration of several adpative attacks. But these adaptive attacks are \"adaptive\" against the latent-separation based defenses. Could you also study/propose potential adaptive attacks that specifically target the weakness of your method? 3) What if the validation data is OOD (e.g. corrupted with noises). In the real world, model deployers sometimes cannot guarantee to have the exactly validation data drawn IID from the training samples' distribution.\n5. **Typos**: Line 147 \"to to\"; Line 214 \"at lease\"; Line 230 \"we will assume that the Suppose that\"; Line 264 $\\eta^* \\to \\gamma^*$ (?).\n\nStill, I would be happy to adjust my rating if the above concerns are somehow addressed.\n\n[1] Qi, Xiangyu, Tinghao Xie, Ruizhe Pan, Jifeng Zhu, Yong Yang, and Kai Bu. \"Towards practical deployment-stage backdoor attack on deep neural networks.\" In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 13347-13357. 2022.\n\n[2] Zeng, Yi, Won Park, Z. Morley Mao, and Ruoxi Jia. \"Rethinking the backdoor attacks' triggers: A frequency perspective.\" In *Proceedings of the IEEE/CVF international conference on computer vision*, pp. 16473-16481. 2021.\n\n[3] Guo, Junfeng, Yiming Li, Xun Chen, Hanqing Guo, Lichao Sun, and Cong Liu. \"Scale-up: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency.\" *arXiv preprint arXiv:2302.03251* (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1: Inaccurate terms in the title\n    \nR: We appreciate your suggestion to improve the current title to better align with specific content, such as focusing on inference-stage backdoor input detection. Your input is valuable, and we will carefully incorporate your suggestions in our revision process.\n\n> Q2: Intuitions behind the proxy term\n\nR: Recall that the optimal decision rule is represented as $s(Z) \\propto P_{\\text{BD}}(Z)/P_{\\text{CL}}(Z)$, where $P_{\\text{BD}}(\\cdot)/P_{\\text{CL}}(\\cdot)$ denotes the distribution of backdoor and clean data, respectively.\nSince we do not have access to the backdoor data, computing $P_{\\text{BD}}(\\cdot)$ is not possible. \nAs an alternative, empirically, we consistently observed that $P_{\\text{CL}}(\\text{Clean Data})$ is much greater than $P_{\\text{CL}}(\\text{Backdoor Data})$. Hence, we use $P_{\\text{CL}}(\\cdot)$ as a substitute for $s(Z)$. When considering Gaussian data assumptions, the proxy $P_{\\text{CL}}(Z) \\propto 1/\\exp{(0.5Z^{\\top}\\Sigma Z)}$ effectively represents the inverse of the Mahalanobis distance to the clean data distribution, which explains the use of SCM as the proxy.\n\n> Q3:  Comparison with more baseline defenses [2], [3] and attack [1]\n\nR: We conducted two sets of experiments: (1) assessing our method against [1], and (2) comparing our method with [2] and [3]. We present the outcomes in **Table 2 in the PDF file of the global response**, respectively. In summary, our methods excel in non-poisoning backdoor attacks [1]. Also, they consistently outperform [2] and [3].\n\n> Q4: Claim in Lines 109 - 111\n\nYes, our methods also assume a separation between the latent representations of clean and backdoor data. \nNevertheless, our approach diverges from other work mentioned in the Related Work section in how we exploit this distinction property. We utilize SCM, consistently outperforming previous methods.\n\nHowever, an issue arises if knowledgeable attackers know the SCM score and exploit it to bypass our defense, a concern we'll address next.\n\n\n\n\n> Q5: Potential adaptive attacks that specifically target the weakness of our method\n\nR: Following the previous response, we developed a new attack, referred to as *M-attack*, which introduced a regularization term to reduce the Mahalanobis scores between clean and backdoor samples. Experiment outcomes are presented in **Table 2 in the PDF file of the global response**. Our methods maintain superiority over other defenses, with a moderate performance decrease compared to different attacks like BadNets and SSBA. This is reasonable, considering the nature of the newly proposed attack.\n\n\n> Q6: The selection of $\\delta$\n\nR: Choosing the value of $\\delta$ should be contingent on both $n$ (sample size) and $\\alpha$ (desired type I error rate). This ensures the establishment of a testing procedure that holds a high probability guarantee on the desired type I error rate.\n\nTo be specific, it can be shown that, with similar arguments in Theorem 1, to ensure a $1-\\delta$ probability guarantee on the type I error rate of $\\alpha$, the values of $n$, $\\delta$, and $\\alpha$ must lead to the right-hand side term of Eq. (1), i.e.,\n$1-\\alpha+\\sqrt{\\log(2/\\delta)/(2n)} $ being strictly less than 1.\nFor instance, consider the values $n=200$, $\\alpha = 0.05$, and $\\delta = 0.01$. In this case, we have $ 1-0.05+\\sqrt{\\log(2/0.01)/400} > 1,$ indicating the infeasibility of achieving a type I error rate less than $\\alpha = 0.05$ with a probability of at least $1 - 0.01$.\n\n\n> Q7: Ablation studies on the size of the validation data and OOD validation datasets.\n\nR: Firstly, we'd like to draw your attention to the **challenges** linked with establishing a provable type I error rate in the two scenarios you mentioned. As we highlighted in our previous response, a validation size of 200 doesn't offer a high probability guarantee for the type I error rate. Moreover, when using out-of-distribution (OOD) data as the validation dataset, it can generally be shown that ensuring any guarantees for the type I error rate becomes **unfeasible**. Hence, these two scenarios might not be the primary application contexts for our methods, which primarily emphasize defense techniques with provable type I error guarantees and strong detection power.\n\nNevertheless, according to your suggestions, we conducted two ablation studies involving variations in the validation dataset size and the inclusion of OOD data in the clean validation dataset. The summarized results are presented in **Table 3, and 4 respectively in the PDF file of the global response**. Our observations indicate that our methods are consistently effective when the validation dataset size is above 200. Moreover, if the validation dataset consists of less than 25% of the OOD samples, our methods exhibit consistent effectiveness. These findings highlight the robustness of our approaches.\n\n> Q8: For some attacks (TrojanNN and A-Blend), even your proposed optimal scoring function cannot separate well enough between clean and backdoor data. \n\nR: We provide an explanation for the less distinct separation between clean and backdoor data under optimal score rules as follows.\nIt's important to remember that the uniformly most powerful rule in Eq.(2) is established based on various conditions, including the presence of fully specified Gaussian data distributions for both clean and backdoor data. However, our investigations revealed a significant departure from Gaussian distributions in the latent spaces of clean and backdoor data during TrojanNN and A-Blend attacks. This divergence leads to a violation of the assumptions necessary for the application of the optimal decision rule in Eq.(2).\n\n> Q9: Histograms of SCM scores\n\nR: We have incorporated the SCM score histograms in **Figure 2 the global response PDF file**. In summary, we notice distinct separations between the SCM scores of clean and backdoor samples consistently across various scenarios.\n"}]

Author rebuttal:
We sincerely thank reviewers for investing their time and energy into reviewing our manuscript and offering valuable feedback. We're pleased that they recognized the quality of our writing, acknowledged the novelty of our proposed framework (Reviewer vAAp), and found our approach effective across various domains and setups (Reviewer vAAP, JEgp, ibgE, 1Ekh). We will incorporate all their comments into the revised paper.

In the following, we will begin by addressing a potentially unclear aspect related to our method's threat model and its comparison with various defenses mentioned by the reviewers. After that, we'll provide a concise summary of our tailored responses for each reviewer.

Regarding the threat model, our approach serves as an **inference-stage defense** with **no access to the training data** and **no ability to manipulate the training process**, including model parameter adjustments. Similarly to other inference-stage defenses [1,2], we operate under the assumption that the defender possesses a limited clean validation dataset but **no foreknowledge of future backdoor test inputs**. Our goal is to detect future backdoor inputs, and our evaluation **metric is the AUCROC score** of the detector. 


In contrast, the defenses mentioned by the reviewers, namely ABL [3], DBD [4], and SPECTRE [5], belong to the training-stage defense category. These approaches have the advantage of **accessing both clean and backdoor training data**, which provides them insights into both types of data. Their objective is to build a clean model by isolating a subset of the training set through analysis of specific characteristics of all training examples. Consequently, their evaluation criteria focus on the **clean and attack success rates** of the purified model.


- **Rebuttal Summary for Reviewer vAAP**:
1. Added implementations details of backdoor attacks
2. Added Ablation studies on the poisoning rate and different model architectures (*Figure 1 in Global Response PDF file*)
3. Clarified issues regarding NLP backdoor attacks in Fig 4


- **Rebuttal Summary for Reviewer JEgp**:
1. Clarified issues regarding notations
2. Added experiments on performance comparison with SOTA backdoor defenses methods mentioned by reviewers
3. Clarified issues regarding the code release 



- **Rebuttal Summary for Reviewer ibgE**:
1. Clarified the relationship between SPECTRE and our method; Providing empirical evaluations between SPECTRE and our method 
2. Added experiments on the performance of our methods under the reviewer-mentioned backdoor attacks, and a new backdoor attack specifically targeted for our method (proposed by ourselves)
3. Clarified notations regarding the data transformation $T$



- **Rebuttal Summary for Reviewer 1Ekh**:
1. Explained intuitions for the proxy terms
2. Added experiments on reviewers-mentioned defenses and attack(s) (*Table 2 in Global Response PDF file*)
3. Added ablation studies on different poisoning ratios; new backdoor attacks specifically targeted at our method; and the OOD dataset  (*Tables 3 & 4 in Global Response PDF file*)
4. Clarified issues regarding the selection of $\delta$
5. Clarified issues regarding the claim in Lines 109 - 111
6. Clarified issues regarding the observations in Figure 2
7. Added histograms for SCM scores (*Figure 2 in Global Response PDF file*)

### References

[1] Y. Gao, C. Xu, D. Wang, S. Chen, D. C. Ranasinghe, and S. Nepal, “Strip: A defence against trojan attacks on deep neural networks,” in Proceedings of the 35th Annual Computer Security Applications Conference, 2019, pp. 113–125.

[2] J. Guo, Y. Li., X. Chen, H. Guo, L. SUN,  and C. Liu. "Scale-up: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency," in ICLR 2023.

[3] Li, Yige, et al. "Anti-backdoor learning: Training clean models on poisoned data." Advances in Neural Information Processing Systems 34 (2021): 14900-14912.

[4] Huang, Kunzhe, et al. "Backdoor defense via decoupling the training process." arXiv preprint arXiv:2202.03423 (2022).

[5] J. Hayase, W. Kong, R. Somani, and S. Oh, “Spectre: defending against backdoor attacks using robust statistics,” in ICML, 2021.



Meta-review: Thanks to the reviewers and authors for the constructive feedback on the paper and collegial discussion so far. The paper's present ratings of 5,6,4,5 reflect overall support albeit with some lingering concerns. The substance of the feedback/discussion, and the paper itself, reflect contributions that is not only sound but I believe significant with potential for impact that reflect a higher reviewer rating than was sometimes reported.

The paper presents a novel inference-stage conformal backdoor detection (CBD) framework for use in defending against backdoor attacks using deep nets' latent representations. Its theoretical results are sound and aligned with the algorithmic contributions, and are accompanied by extensive empirical evaluations that demonstrate compelling experimental performance (across CV and NLP with 3/10 and 2/2 datasets/attacks) when theoretical assumptions are relaxed.

Theorem 1 guarantees the FPR of the CBD algorithm under fairly convention assumptions while Theorems 2 and 3 do make a Gaussian assumption but I view these as complementary results demonstrating sufficient conditions for uniqueness and the decision rule being uniformly most powerful. That is, the Gaussian assumption found in the paper is not relevant to the key results but the complementary results. The theory might not be introducing new proof techniques or demonstrate technical innovation, but it is well aligned with the _algorithmic/framework_ contribution of the paper and I believe fits well. This helps explains the approach's empirical performance and serves as motivation; particularly Theorem 3 as it suggests an appropriate rule for balancing the power/FPR trade-off.

Initial reviewer concerns were primarily: overlap with existing work SPECTRE concerning novelty; evaluation against adaptive attacks and specific baselines including ABL, DBD and others; additional ablation studies to those included in the appendices; and specific clarity/presentation issues.

The authors highlight that some requested comparison defenses make additional assumptions of the defender such as access to clean training data. They have provided requested ablation results in their rebuttal, comparison results with SOTA backdoor defenses (including those that are more restrictive than CBD), attacks, and sufficient clarifying detail that I believe addresses the substantive reviewer concerns.
Decision: Accept (poster)