SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A.1 Proof for Section 3.2
In this section, we present the proof of Theorem 1. For the convenience of the reader, we attach the pseudo-code of Algorithm 2 below. Please note that the thresholding value  $\lambda_{\alpha,s}$  is selected to satisfy
Algorithm 2 Conformal Backdoor Detection (CBD)
Input: querying input  $X_{\mathrm{test}}$ , clean validation dataset  $D^{\mathrm{Val}} = \{(X_i,Y_i)\}_{i = 1}^n$ , transformation method  $T(\cdot)$ , score function  $s(T(\cdot))$ , desired false positive rate  $\alpha \in (0,1)$ , violation rate  $\delta \in (0,1)$
1: Receiving a future query sample  $X_{\text{test}}$
2: for  $i = 1$  to  $n$  do
3: Calculate  $s_i = s(T(X_i)) / x_i \in D^{\mathrm{Val}}$
4: end for
5: Select the decision threshold  $\lambda_{\alpha ,s}$  according to Equation (1).
6: Determine if  $X_{\mathrm{test}}$  is a clean sample if  $s(T(X_{\mathrm{test}})) \leq \lambda_{\alpha, s}$
Output: The decision if the sample  $X_{\mathrm{test}}$  is a clean or backdoor sample
the condition
$$
\hat {F} _ {\text {C L E A N}} \left(\lambda_ {\alpha , s}\right) = 1 - \alpha + \sqrt {\left(\log (2 / \delta) / (2 n)\right)}, \tag {4}
$$
where  $\delta \in (0,1)$  is the violation rate describing the probability that the (FPR) exceeds  $\alpha$ , and  $\hat{F}_{\mathrm{CLEAN}}$  is the empirical cumulative distribution function of the scores on the validation data  $\{s(T(X_i))\}_{i=1}^n$ . In the case where  $\sqrt{(\log(2/\delta)/(2n))} > \alpha$ , we set the thresholding value  $\tau$  to be the maximum of  $\{s(T(X_i))\}_{i=1}^n$ .
The main text briefly touches upon the central concept found in the conformal prediction literature. It states that by employing a suitable score function, the empirical rank or quantile of the distribution will eventually approach the population counterpart. This convergence is ensured by the uniform convergence of cumulative distribution functions (CDFs). Consequently, to establish the proof for Theorem 1, we will first present the subsequent outcome, which accurately measures the uniform convergence of CDFs.
Lemma 1 (Dvoretzky-Kieffer-Wolffowitz inequality). Given a natural number  $n$ , let  $X_{1},X_{2},\ldots ,X_{n}$  be real-valued independent and identically distributed random variables with cumulative distribution function  $F(\cdot)$ . Let  $F_{n}(\cdot)$  denote the associated empirical distribution function.
The interval that contains the true CDF,  $F(x)$ , with probability  $1 - \delta$  is specified as
$$
F _ {n} (x) - \varepsilon \leq F (x) \leq F _ {n} (x) + \varepsilon \text {w h e r e} \varepsilon = \sqrt {\frac {\ln \frac {2}{\delta}}{2 n}}.
$$
Theorem 4 (False positive rate of Algorithm 1). Given any pre-trained backdoored classifier  $f$ , suppose that the validation dataset  $\mathcal{D}^{\mathrm{Val}}$  and the test data  $(X_{\mathrm{test}}, Y_{\mathrm{test}})$  are IID drawn from the clean
data distribution  $\mathbb{P}_{\mathrm{CLEAN}}$ . Given any  $\delta \in (0,1)$ , for any score function and transformation method  $s(T(\cdot))$ , such that the resulting scores  $\{s(T(X_i))\}_{i=1}^n$  remain IID with a continuous distribution, the associated backdoor conformal detector  $g(\cdot; s, \lambda_{\alpha,s})$  as specified in Algorithm 1 satisfies
$$
\mathbb {P} \left(g \left(X _ {\text {t e s t}}; s, \lambda_ {\alpha , s}\right) = 1 (\text {B a c k d o o r S a m p l e}) \mid \mathcal {D} ^ {\text {V a l}}\right) \leq \alpha ,
$$
Proof. Note that  $X_{\mathrm{test}}$  is drawn from the clean data distribution and we have
$$
\begin{array}{l} \mathbb {P} \left(g \left(X _ {\text {t e s t}}; s, \lambda_ {\alpha}\right) = 1 (\text {B a c k d o o r S a m p l e}) \mid \mathcal {D} ^ {\text {V a l}}\right) \\ = \mathbb {E} _ {X _ {\text {t e s t}}} \mathbf {1} \left\{g \left(X _ {\text {t e s t}}; s, \lambda_ {\alpha}\right) = 1 \mid \mathcal {D} ^ {\text {V a l}} \right\} \\ = \mathbb {E} _ {X _ {\text {t e s t}}} \mathbf {1} \left\{s \left(T \left(X _ {\text {t e s t}}\right)\right) \geq \lambda_ {\alpha} \mid \mathcal {D} ^ {\text {V a l}} \right\} (5) \\ = \mathbb {E} _ {X _ {\text {t e s t}}} \mathbf {1} \left\{F _ {\text {C L E A N}} \left(s \left(T \left(X _ {\text {t e s t}}\right)\right)\right) \geq F _ {\text {C L E A N}} \left(\lambda_ {\alpha}\right) \mid \mathcal {D} ^ {\text {V a l}} \right\} (6) \\ = \mathbb {P} \left(F _ {\text {C L E A N}} \left(s \left(T \left(X _ {t e s t}\right)\right)\right) \geq F _ {\text {C L E A N}} \left(\lambda_ {\alpha}\right) \mid \mathcal {D} ^ {\text {V a l}}\right) \\ \leq \mathbb {P} \left(F _ {\text {C L E A N}} \left(s \left(T \left(X _ {t e s t}\right)\right)\right) \geq \hat {F} _ {\text {C L E A N}} \left(\lambda_ {\alpha}\right) - \varepsilon \mid \mathcal {D} ^ {\text {V a l}}\right) (\varepsilon = \sqrt {\frac {\ln \frac {2}{\delta}}{2 n}}) (7) \\ = 1 - (1 - \alpha + \varepsilon - \varepsilon) (8) \\ = \alpha , \\ \end{array}
$$
holds with probability at least  $1 - \delta$ . The equation (5) is because of the decision rule as specified in Algorithm 1. Additionally, the  $F_{\mathrm{CLEAN}}$  in Equation (6) represents the CDF of  $s(T(X_{\mathrm{test}}))$ , while  $\hat{F}_{\mathrm{CLEAN}}$  in (7) denotes the empirical CDF obtained from  $\mathcal{D}^{\mathrm{Val}}$ . The inequality in Equation (7) arises from the DKW inequality specified in Lemma 1. Furthermore, the equation (8) is based on the fact that the CDF follows a uniform distribution (a result of the probability integral transformation) and the selection of the thresholding value specified in Equation (4).
A.2 Proof for Section 3.3
This section will present the proofs for Theorem 2 and 3, as outlined in Section 3.3 of the main text. These results are intimately connected to the renowned Neyman-Pearson Lemma within the context of statistical hypothesis testing and binary classification problems. To provide the necessary context, we will begin with a brief introduction to the Neyman-Pearson classification framework.
A.2.1 Neyman-Pearson Classification
Consider a random pair  $(X,Y)$ , where  $X\in \mathcal{X}\subset \mathbb{R}^d$  is a  $d$ -dimensional vector of features, and  $Y\in \{0,1\}$  represents the class label of  $X$ . A classifier  $g:\mathcal{X}\to \{0,1\}$  from a data input belongs to  $\mathcal{X}$  to  $0,1$ . The overall classification error of  $f$  is denoted as  $R(g) = \mathbb{E}\mathbf{1}\{g(X)\neq Y\} = \mathbb{P}\{g(X)\neq Y\}$ . By applying the law of total probability,  $R(g)$  can be decomposed into a weighted average of the type I error  $R_0(g) = \mathbb{P}\{g(X)\neq Y\mid Y = 0\}$  and the type II error  $R_{1}(g) = \mathbb{P}\{g(X)\neq Y\mid Y = 1\}$ , given by
$$
R (g) = \pi_ {0} R _ {0} (g) + \pi_ {1} R _ {1} (g)
$$
where  $\pi_0 = \mathbb{P}(Y = 0)$  and  $\pi_1 = \mathbb{P}(Y = 1)$ . While the classical paradigm minimizes  $R(\cdot)$ , the Neyman-Pearson (NP) paradigm seeks to minimize  $R_{1}$  while controlling  $R_{0}$  under a user-specified level  $\alpha$ . The (level- $\alpha$ ) NP oracle classifier is thus
$$
g_{\alpha}^{*}\in \operatorname *{arg  min}_{R_{0}(g)\leq \alpha}R_{1}(g)
$$
where the significance level  $\alpha$  reflects the level of conservativeness towards type I error. To reflect on the backdoor detection problem, we encode the label set  $Y$  to indicate if its associated  $X$  is clean (with label 0) or backdoored (with label 1). The classifier  $g$  in the NP classification context corresponds to the detector  $g$  in our framework. The following result, a direct consequence of the famous Neyman-Pearson Lemma [56], gives the solution of  $g_{\alpha}^{*}$ .
Lemma 2 (NP oracle classifier [74]). Let  $\mathbb{P}_1$  and  $\mathbb{P}_0$  be two probability measures with densities  $p_1$  and  $p_0$  respectively. Under mild continuity assumption, the NP oracle classifier is given by
$$
g _ {\alpha} ^ {*} (x) = \mathbf {1} \left\{\frac {p _ {1} (x)}{p _ {0} (x)} > C _ {\alpha} \right\}
$$
for some threshold  $C_{\alpha}$  such that  $\mathbb{P}_0\{p_1(X) / p_0(X) > C_\alpha \} \leq \alpha$  and  $\mathbb{P}_1\{p_1(X) / p_0(X) > C_\alpha \} \geq \alpha$ .
A.2.2 Proof for Theorem 2
Proof. By the definition of the Attacker's Goal, as specified in Section 3, the attacker faces a problem of the following:
$$
\max  _ {\eta_ {1}, f ^ {\mathrm {p o i}}} \mathbb {P} _ {X Y \sim \mathbb {P} _ {\text {C L E A N}}} (f ^ {\mathrm {p o i}} (\eta_ {1} (X)) = \eta_ {2} (Y))
$$
subject to  $|\mathbb{P}_{XY\sim \mathbb{P}_{\mathrm{CLEAN}}}(f^{\mathrm{poi}}(X)\neq Y) - \mathbb{P}_{XY\sim \mathbb{P}_{\mathrm{CLEAN}}}(f^{\mathrm{cl}}(X)\neq Y)|\leq \varepsilon$
Also, we assume that the (A) marginal clean data is normally distributed with mean 0 and covariance  $\Sigma$ , (B) the attacker employs a linear classifier  $f_{\theta}(x) = \mathbf{1}\{\theta^{\top}x > 0\}$  for  $\theta \in \mathbb{R}^d$ , and (C) the attacker applies the backdoor transformation  $\eta_1(x) = x + \gamma$ , where  $\gamma \in \mathrm{T}_c \triangleq \{u \in \mathbb{R}^d \mid \| u \|_2 = c, c > 0\}$ .
Firstly, it is straightforward to check that, per the Attacker's Goal above, the attacker cannot obtain a backdoor classifier with  $\theta^{*} = 0$ , as in this scenario, the corresponding backdoor accuracy:
$$
\mathbb {P} _ {X \sim \mathcal {N} (\gamma , \Sigma)} (X ^ {\top} \theta^ {*} > 0)
$$
would be zero regardless of the backdoor trigger  $\eta_1(x) = x + \gamma$  for  $\gamma \in \mathrm{T}_c$ . Next, for any non-zero  $\theta^{*} \in \mathbb{R}^{d}$ , suppose that there exist  $\gamma_{1}, \gamma^{*} \in \mathrm{T}_{c}$  such that  $(\gamma_{1}, \theta^{*})$  and  $(\gamma^{*}, \theta^{*})$  are both the solutions of the Attacker's Goal.
As a result, both
$$
\mathbb {P} _ {X \sim \mathcal {N} (\gamma^ {*}, \Sigma)} (X ^ {\top} \theta^ {*} > 0)
$$
$$
\mathbb {P} _ {\tilde {X} \sim \mathcal {N} (\gamma_ {1}, \Sigma)} (\tilde {X} ^ {\top} \theta^ {*} > 0)
$$
are maximized, and equal, subject to  $\gamma_1,\gamma^*\in \mathrm{T}_c$
On the other hand, given the classifier  $\theta^{*}$ , for any  $\gamma \in \mathrm{T}_c$ , we have the backdoor accuracy under  $\theta^{*}$ :
$$
\begin{array}{l} \mathbb {P} _ {X \sim \mathcal {N} (\gamma , \Sigma)} (X ^ {\top} \theta^ {*} > 0), \\ = \mathbb {P} _ {Z} (Z > 0), \quad Z \sim \mathcal {N} (\gamma^ {\top} \theta^ {*}, \theta^ {*} \Sigma (\theta^ {*}) ^ {\top}), \\ = 1 - \Phi \left(- \frac {\gamma^ {\top} \theta^ {*}}{\left(\theta^ {*} \Sigma \left(\theta^ {*}\right) ^ {\top}\right) ^ {1 / 2}}\right), \quad \Phi (\cdot) \text {C D F o f t h e s t a n d a r d n o r m a l d i s t r i b u t i o n} \\ = \Phi \left(\frac {\gamma^ {\top} \theta^ {*}}{\left(\theta^ {*} \Sigma \left(\theta^ {*}\right) ^ {\top}\right) ^ {1 / 2}}\right), \\ \end{array}
$$
is maximized if and only if  $\gamma = c\theta^{*} / \| \theta^{*}\|$  ( $\theta^{*}\neq 0$ ) due to the Cauchy-Schwarz inequality. As a result, we have  $\gamma_{1} = \gamma^{*}$ . Hence, the optimal backdoor trigger  $\gamma^{*}$  corresponds to the backdoor classifier  $\theta^{*}$  is unique and admits the form of  $\gamma^{*} = c\theta^{*} / \| \theta^{*}\|$ .
A.2.3 Proof of Theorem 3
Proof. Following the same setup in Theorem 2 and from the result in Theorem 2, the attacker knows both the clean and backdoor distribution. Hence, we conclude the result by the Neyman Pearson Lemma and the Lemma 2.
B Experiments Configurations
B.1 Data Description
CIFAR10: The CIFAR-10 dataset is a highly popular dataset in the field of machine learning research. It consists of 60,000 color images, each with a resolution of  $32 \times 32$  pixels. The dataset is divided into 10 classes, with 6,000 images per class. Specifically, there is a training set with 50,000 images and a test set with 10,000 images.
GTSRB: The GTSRB dataset, known as the German Traffic Sign Recognition Benchmark, has gained popularity in the field of Backdoor Learning. It consists of a total of 60,000 images distributed among 43 different classes, with varying resolutions ranging from 32x32 to 250x250 pixels. The dataset is split into a training set containing 39,209 images and a test set containing 12,630 images.
Tiny ImageNet: Tiny ImageNet is comprised of a collection of 100,000 images belonging to 200 classes. Each class consists of 500 images, with 64x64 dimensions, resulting in colored images. The dataset is further divided into subsets, with 500 training images, 50 validation images, and 50 test images allocated for each class.
SST-2: The dataset used in our study is a modified version of the Stanford sentiment analysis dataset, specifically the 2-class variant known as SST-2. The SST-2 dataset consists of 9,613 samples, while another variant called SST-5 contains 11,855 reviews. Additionally, the SST dataset includes phrases associated with each of the sentences.
IMDB: The dataset used in our work is a binary dataset comprising of 12,500 movie reviews in each class. The reviews are multi-sentence and are presented in the form of long sentences. For our study, we have extracted the first 200 words from each review.
B.2 Packages used for generating backdoor attacks/data/models
In this section, we provide a detailed description of the experimental setup. To conduct our experiments, we utilized three open-source backdoor packages, the specifics of which are summarized in Table 2. For most computer vision (CV) backdoor attacks, we implemented them using both the Backdoor ToolBox [75] and the BackdoorBench [76] to ensure the consistency of our results. In the case of WaNet and SSBA, the BackdoorBench [49] package offered implementations for CIFAR-10, GTSRB, and Tiny ImageNet, so we utilized their package to obtain the latent representations. Regarding the TacT, Adaptive Patch, and Adaptive Blend, the Backdoor ToolBox package offered implementations for CIFAR-10, GTSRB, so we utilized their package to obtain the latent representations. Lastly, for NLP backdoor attacks, we relied on the OpenBackdoor package, which is specifically designed for NLP backdoors.
The results in the main text are directly obtained by running the Default scripts for the three packages, as specified in Table 3. Ablation studies on different model architectures and poisoning rates are included in Section D and E, respectively.
Table 2: Open-source packages applied in our work
Backdoor	ToolBox [75]	BackdoorBench [76]	OpenBackdoor [49]
BadNets	✓	✓
Blended	✓	✓
TrojanNN	✓
SIG	✓	✓
Dynamic	✓	✓
TacT	✓
WaNet	✓
SSBA	✓
Adaptive-Blend	✓
Adaptive-Patch	✓
SOS	✓
LWP	✓
B.3 On the selection of Hyperparameters
Within this section, we provide an outline for the selection of the shrinkage parameter  $\beta$  utilized in the Shrink-Covariance Mahalanobis. The primary motivation behind employing SCM is to address the issue of unstable estimation in the inverse of the sample covariance matrix, which can result in non-IID property of samples within  $\mathcal{D}^{\mathrm{Val}}$  and future test data. Such non-IIDness can override the order information for those samples, affecting the FPR as described in Theorem 1.
To ensure the IID property of the samples within  $\mathcal{D}^{\mathrm{Val}}$  as well as future test samples, we propose the following approach for selecting the shrinkage parameter  $\beta$ . Firstly, we partition  $\mathcal{D}^{\mathrm{Val}}$  into two mutually exclusive datasets, namely  $\mathcal{D}_1$  and  $\mathcal{D}_2$ , with the size of  $\mathcal{D}_1$  greater than that of  $\mathcal{D}_2$ . Next, we compute the Shrink-Covariance Mahalanobis (SCM) scores based on  $\mathcal{D}_1$  and perform a search for an appropriate value of  $\hat{\beta}$  that ensures the SCM scores,  $s_{\hat{\beta}}(\mathcal{D}_1)$ , for the samples within  $\mathcal{D}_1$  and  $s_{\hat{\beta}}(\mathcal{D}_2)$  are IID samples. To verify the IID property, we conduct a Two-sample Kolmogorov-Smirnov Test
Table 3: Open-source packages used for the results in the main text
Backdoor	ToolBox [75]	BackdoorBench [76]	OpenBackdoor [49]
BadNets	✓
Blended	✓
TrojanNN	✓
SIG	✓
Dynamic	✓
TacT	✓
WaNet	✓
SSBA	✓
Adaptive-Blend	✓
Adaptive-Patch	✓
SOS	✓
LWP	✓
Figure 6: The mean FPRs of our proposed method on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases. Our method's FPRs consistently match the theoretical upper bounds.
with a decision rule based on a p-value of 0.05. We repeat the above procedure ten times and observe that for CV (NLP) attacks,  $\hat{\beta} = 0.5$  (0.7) satisfies the aforementioned requirement. Consequently, we set  $\beta = 0.5$  for all CIFAR10, GTSRB experiments (for Tiny ImageNet:  $\beta = 0.1$ ), and  $\beta = 0.7$  for all NLP experiments. Ablation studies investigating different values of  $\beta$  are included in Appendix F.
C Omitted Experimental Results in Main Text
C.1 Results on the FPR for CIFAR10
This section presents the results of the False Positive Rate (FPR) analysis conducted on CIFAR10. Figure 6 illustrates the performance of our proposed CBD-SCM. Notably, our method consistently achieves FPR values that align with the theoretical upper bounds.
C.2 Results on the detection power (ROC) for IMDB Dataset
We assess the detection performance of our proposed method on IMDB (NLP) using the base uncased-BERT model. Specifically, we generate ROC curves for our methods under two advanced backdoor attacks, as illustrated in Figure 7. The results indicate that our method outperforms all other methods.
Figure 7: ROC of our SCM method on IMDB NLP. Our proposed method consistently outperforms all other methods.
D Ablation Study: Different model architectures
In this section, we demonstrate the performance of our proposed methods using the VGG 19 model. The dimension of the avgpool layer in VGG 19 is 25088, which is impractically large. Therefore, we utilize the classifier layer with a dimension of 4096 for practical purposes. We have observed that the empirical FPR performance of our method consistently aligns with the theoretical bounds in all cases. Consequently, we omit the results for FPR performance and focus on reporting the results for detection power (AUCROC, ROC) in the following subsections.
D.1 Detection Power on CIFAR10
We assess the detection performance of our proposed method on CIFAR10 using VGG 19. We present the ROC curves for our method in Figure 8 and 9. It is evident that our SCM method consistently outperforms other methods in all attack scenarios. Specifically, for certain advanced attacks such as the Dynamic and WaNet, we observe a remarkable improvement of approximately  $200\%$  in terms of AUCROC.
Figure 8: ROC of our method on CIFAR10 with VGG19. Our proposed method consistently outperforms other methods.
D.2 Detection Power on GTSRB
We evaluate the detection performance of our proposed method on GTSRB using the VGG 19 model. The ROC curves for our method are displayed in Figure 10 and 11. It is evident from the results that our SCM method consistently outperforms other methods in the majority of attack scenarios. Notably, for advanced attacks like SSBA, we observe a significant improvement in terms of AUCROC.
Figure 9: ROC of our method on CIFAR10 with VGG19. Our proposed method consistently outperforms other methods.
Figure 10: ROC of our method on GTSRB with VGG19.
Figure 11: ROC of our method on GTSRB with VGG19.
E Ablation Study: Different poisoning ratios
In this section, we provide the results obtained by using different poisoning ratios. The summary of the results for CIFAR10 can be found in Table 4. It is observed that the detection performance of our method exhibits minimal variations across different poisoning ratios. This finding further reinforces the robustness and universal effectiveness of our method.
F Ablation Study: Different choices of  $\beta$
In this section, we provide empirical studies on the effect of using different  $\beta$ . Specifically, we select  $\beta \in \{0.3, 0.4, 0.6, 0.7\}$ .
Table 4: AUCROC score of our method on CIFAR10
Poisoning Ratio →	0.3%	1%	5%
BadNets	0.99	0.99	0.99
Blended	0.96	0.96	0.96
WaNet	0.88	0.91	0.94
SSBA	0.92	0.95	0.97
Figure 12: The mean FPRs of our proposed method with  $\beta = 0.3$  on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases.
F.1 Performances on FPR for CIFAR10
This section presents the results of the False Positive Rate (FPR) analysis conducted on CIFAR10 with different choices over  $\beta$ . Figure 12, 13, 14, 15 illustrate the FPR performance of our proposed CBD-SCM, with  $\beta = 0.3, 0.4, 0.6, 0.7$  respectively. Our method consistently achieves FPR values that align with the theoretical upper bounds.
F.2 Performances on detection power (AUCROC, ROC) for CIFAR10
We assess the detection performance of our proposed method on CIFAR10 with different choices over  $\beta$ . Figure 16, 17, 18, 19 illustrate the ROC of our proposed SCM, with  $\beta = 0.3, 0.4, 0.6, 0.7$  respectively. The AUCROC scores exhibit minimal variations when different values of  $\beta$  are used, indicating the consistent and robust effectiveness of our method. This observation reinforces the stability and universality of our selected  $\beta = 0.5$  as reported in the main text.
Figure 13: The mean FPRs of our proposed method with  $\beta = 0.4$  on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases.
Figure 14: The mean FPRs of our proposed method with  $\beta = 0.6$  on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases.
Figure 15: The mean FPRs of our proposed method with  $\beta = 0.7$  on CIFAR10 are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases.
Figure 16: ROC of our method on CIFAR10 with  $\beta = 0.3$
Figure 17: ROC of our method on CIFAR10 with  $\beta = 0.4$
Figure 18: ROC of our method on CIFAR10 with  $\beta = 0.6$
Figure 19: ROC of our method on CIFAR10 with  $\beta = 0.7$
Figure 20: The mean FPRs of our proposed method with  $\beta = 0.3$  on GTSRB are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases.
Figure 21: The mean FPRs of our proposed method with  $\beta = 0.4$  on GTSRB are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases.
F.3 Performances on FPR for GTSRB
This section presents the results of the False Positive Rate (FPR) analysis conducted on GTSRB with different choices over  $\beta$ . Figure 20, 21, 22, 23 illustrate the FPR performance of our proposed CBD-SCM, with  $\beta = 0.3, 0.4, 0.6, 0.7$  respectively. Our method consistently achieves FPR values that align with the theoretical upper bounds.
F.4 Performances on detection power (AUCROC, ROC) for GTSRB
We assess the detection performance of our proposed method on GTSRB with different choices over  $\beta$ . Figure 24, 25, 26 illustrate the ROC of our proposed SCM, with  $\beta = 0.3, 0.4, 0.6$  respectively. The AUCROC scores exhibit minimal variations when different values of  $\beta$  are used, indicating the consistent and robust effectiveness of our method. This observation reinforces the stability and universality of our selected  $\beta = 0.5$  as reported in the main text.
Figure 22: The mean FPRs of our proposed method with  $\beta = 0.6$  on GTSRB are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases.
Figure 23: The mean FPRs of our proposed method with  $\beta = 0.7$  on GTSRB are shown in each plot, which is independently replicated 10 times. The solid line represents the mean value, and the standard errors are  $< 0.01$  for all cases.
Figure 24: ROC of our method on GTSRB with  $\beta = 0.3$
Figure 25: ROC of our method on GTSRB with  $\beta = 0.4$
Figure 26: ROC of our method on GTSRB with  $\beta = 0.6$