SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Dataset	Attack	CA (%)	ASR (%)	STRIP	ONION	RAP	MDP
FRR	FAR	FRR	FAR	FRR	FAR	FRR	FAR
SST-2	BadNets	95.06	94.38	7.56	87.44	2.78	9.28	3.11	64.28	5.33	1.77
AddSent	94.45	100.0	2.75	72.56	7.06	26.72	5.61	37.50	4.45	3.53
LWP	93.41	95.53	5.96	89.39	8.28	7.39	0.83	43.77	5.27	4.78
EP	93.63	95.95	1.72	72.06	5.28	12.89	2.72	58.11	5.05	0.73
SOS	91.65	92.41	2.98	87.56	4.06	32.56	1.89	51.28	0.00	0.00
MR	BadNets	89.80	98.30	11.70	72.30	4.80	15.60	2.75	25.35	5.10	5.60
AddSent	89.60	97.50	16.20	60.00	4.65	37.25	9.35	39.70	5.05	10.90
LWP	89.65	96.90	9.35	82.70	1.60	17.45	1.70	52.55	5.25	3.60
EP	89.40	96.60	2.20	88.90	15.35	12.60	6.45	70.60	4.70	3.00
SOS	89.85	97.30	5.20	75.90	0.90	64.10	15.20	58.85	4.85	3.40
CR	BadNets	89.95	92.30	2.85	98.70	5.20	7.45	1.35	43.60	4.95	5.10
AddSent	91.45	95.70	10.10	62.20	4.75	19.50	12.95	48.90	4.80	3.00
LWP	89.75	91.30	1.80	99.10	4.90	27.85	4.05	39.20	5.10	3.50
EP	89.35	97.55	2.20	87.20	10.15	4.40	7.65	45.20	5.35	9.40
SOS	91.45	100.0	2.20	78.20	0.75	37.55	3.40	55.30	0.20	0.00
SUBJ	BadNets	96.05	94.20	5.10	68.85	3.50	16.60	12.40	43.65	5.30	7.90
AddSent	95.90	97.00	2.50	85.50	4.30	34.20	7.30	68.20	4.85	9.00
LWP	96.15	99.10	4.55	98.70	4.65	7.40	1.00	18.60	5.40	10.90
EP	96.70	99.90	4.75	99.10	5.25	4.10	4.70	33.25	4.90	10.30
SOS	94.90	99.60	5.15	75.50	4.90	61.30	0.10	29.10	5.35	4.10
TREC	BadNets	93.00	95.30	4.30	73.76	5.40	54.53	5.55	50.61	4.80	2.49
AddSent	96.60	93.65	5.20	79.28	4.80	36.74	3.55	47.60	3.60	7.18
LWP	94.40	97.24	5.60	99.17	4.60	25.69	1.23	93.09	5.20	4.42
EP	95.80	97.51	4.60	63.81	5.20	11.22	10.43	42.68	4.80	5.25
SOS	91.80	99.45	5.20	68.78	4.40	80.61	14.83	63.71	4.60	4.97
Table 2. Performance of MDP and baseline defesens on 5 datasets, with lower FAR/FRR indicating better defense performance. The detection threshold is set based on the allowance of  $5\%$  FRR on the training set.
We first evaluate the effectiveness of various backdoor attacks under prompt-based fine-tuning, with results summarized in Table 2. Observe that across all the datasets, most attacks attain both CA and ASR above  $90\%$ , indicating their effectiveness in the downstream and backdoor tasks.
We then compare the performance of MDP and baselines in defending against these attacks. For each defense, we set the detection threshold (e.g., the variation threshold for MDP) based on the allowance of  $5\%$  FRR on the training set, and report its FAR and FRR on the testing set. In the case of ONION, following prior work [36], we evaluate different thresholds of perplexity change and select the threshold that approximately achieves  $5\%$  FRR on the training set.
Table 2 summarizes the main results (additional results in §D). Observe that MDP attains the lowest FARs against all the attacks across all the datasets and outperforms baselines by large margins. In particular, it achieves near-perfect defenses against the SOS attack on the SST-2 and CR datasets. The observation confirms the effectiveness of MDP in detecting poisoned samples, which is mainly attributed to that i) the clean and poisoned samples show discernible sensitivity to random masking and ii) MDP effectively utilizes the few-shot data as anchors to measure such sensitivity.
In comparison, the baseline defenses are less effective, with FARs over  $90\%$  in many cases. This may be explained by the conflict between the limited few-shot data and the reliance of these defenses on sufficient training data. Specifically, to measure the prediction stability of a given sample under perturbation, STRIP randomly replaces a fraction of its words with ones from a training sample
Figure 3: Performance of MDP on SST-2 with different FRR allowances on the training set; baseline defenses all have FARs above  $50\%$  (not shown).
Dataset	Attack
BadNets	AddSent	LWP	EP	SOS
SST-2	FRR	5.07	5.29	5.39	5.39	5.17
FAR	24.89	58.37	55.50	47.82	73.28
MR	FRR	5.40	5.05	5.45	5.15	4.60
FAR	72.80	74.80	55.00	52.10	80.80
CR	FRR	4.40	5.10	4.80	5.45	5.25
FAR	83.10	75.30	73.80	52.20	56.10
SUBJ	FRR	5.40	4.25	4.60	4.75	5.35
FAR	9.80	67.40	14.90	15.70	37.90
TREC	FRR	5.20	4.90	4.90	4.70	5.20
FAR	75.14	71.55	43.37	70.44	26.52
Table 3: Performance of MDP using prediction variance as the masking-sensitivity measure.
that have the highest frequency-inverse document frequency (TF-IDF) scores. However, due to the limited number of training samples, both the substitution words and the estimated TF-IDF scores tend to be highly biased, which negatively impacts the performance of STRIP. ONION removes outlier words that cause sharp perplexity changes before inference, which is inherently ineffective against complex triggers (e.g., natural sentences) [6]. Moreover, the threshold for detecting outlier words can be significantly biased by the limited training samples under the few-shot setting. RAP trains a word-based robustness-aware trigger such that inserting this trigger causes significant prediction changes for clean samples but not for poisoned samples. However, under the few-shot setting, the optimality of the RAP trigger is largely limited by the available few-shot data, which negatively affects its detection effectiveness.
5.4 Influential Factors
We conduct additional studies to understand the impact of key factors on MDP. Due to space limitations, we mainly present the results on SST-2, with other results deferred to §D.
FRR allowance. We adjust the detection threshold corresponding to varying FRR allowance on the training set. Figure 3 shows that MDP maintains its superior performance under different FRRs (0.5%, 1%, and 3%). In comparison, the baselines all have FARs above 50% (not shown).
Sensitivity measures. Instead of using the few-shot data as distributional anchors to measure the masking sensitivity of a given sample  $X_{\mathrm{in}}^{\mathrm{test}}$ , we use its prediction variance due to masking as the sensitivity measure. Specifically, given the prediction of  $X_{\mathrm{in}}^{\mathrm{test}}$ :  $y = \arg \max_{y'} p_{\theta}(y'|X_{\mathrm{in}}^{\mathrm{test}})$ , we measure the confidence variance of the masked variant  $\hat{X}_{\mathrm{in}}^{\mathrm{test}}$  with respect to  $y$ :  $\sigma(p_{\theta}(y|\hat{X}_{\mathrm{in}}^{\mathrm{test}}))$ . Intuitively, a poisoned sample tends to have a larger variance since masking the trigger may cause the prediction to fluctuate significantly. Following the same setting in §5.3, we set the threshold based on 5% FRR allowance on the training set and evaluate MDP on the testing set. Table 3 shows that using the alternative sensitivity measure causes the performance of MDP to drop sharply (cf. Table 2). For instance, its FAR increases by over 50% against LWP. The results confirm our analysis that simple statistics such as prediction confidence may fail to capture the complex variation of the language modeling probability due to masking.
Masking-invariance constraints. Recall that the masking-invariant constraint  $\mathcal{L}_{\mathrm{MI}}$  is designed to improve the masking invariance of clean samples. Here, we evaluate its impact on MDP's overall performance. Specifically, we adjust the weight of  $\mathcal{L}_{\mathrm{MI}}$  in the prompt optimization [41] from 0.25 to 4. For each distinct weight, we set the detection threshold based on  $5\%$  FRR allowance on the training set and report its performance on the testing set. As shown in Figure 4, as the weight of  $\mathcal{L}_{\mathrm{MI}}$  varies, the FARs of MDP against most attacks first drop and then gradually increase. This observation may be explained as follows. With an overly small weight,  $\mathcal{L}_{\mathrm{MI}}$  has little effect on improving the masking invariance of clean samples, while overly emphasizing  $\mathcal{L}_{\mathrm{MI}}$  negatively impacts the classification accuracy, resulting in higher FARs. It is thus crucial to properly calibrate the weight of  $\mathcal{L}_{\mathrm{MI}}$  to optimize the performance of MDP.
Few-shot data size. We further evaluate how the few-shot data size (i.e., shots) influences the performance of MDP. Besides the default shots ( $K = 16$  per class) used in the previous evaluation, we vary  $K$  from 4 to 64 to build the anchor set and evaluate MDP, in which the FRR allowance on
Figure 4: Performance of MDP on SST-2 under varying weight of the masking-invariance constraint  $\mathcal{L}_{\mathrm{MI}}$
Figure 5: Performance of MDP on SST-2 with varying size of few-shot data ( $K$  samples per class).
the training set is fixed as  $5\%$ . Figure 5 reports the performance of MDP under varying shots  $K$ . Observe that its FARs steadily improve as  $K$  increases. Intuitively, with a larger anchor set, MDP quantifies the representational variation of given samples due to random masking more precisely, leading to more accurate detection. Also, notice that  $K = 16$  is often sufficient for MDP to obtain satisfactory performance.
Prompt types. We further evaluate the impact of prompt types on MDP. Recall that in discrete prompts [23], the tokens in the prompt template are selected from the vocabulary, while in continuous prompts [18], the tokens are pseudotokens and optimized in a continuous space. Table 4 evaluates MDP on discrete prompt-based models. Compared with continuous prompts (cf. Table 2), MDP is less effective under discrete prompts. For instance, its FAR against BadNets on MR increases by  $17\%$ . This may be explained by that continuous prompts entail larger spaces to better optimize the masking invariance constraint, suggesting that using differentiable, continuous prompts benefits MDP in defending against backdoor attacks.
PLMs. Besides RoBERTa-large [19], we also consider alternative PLMs including BART [14] and GPT-2 [27] as the victim models. Table 5 shows MDP's performance on such alternative PLMs against BadNets and AddSent attacks (with respect to SST-2). Observe that MDP has comparable performance on BART and GPT-2 to that on RoBERTa-large, indicating that it generalizes to other PLMs as well.
Dataset	Attack
BadNets	AddSent	LWP	EP	SOS
SST-2	FRR	5.27	4.39	5.15	5.11	0.00
FAR	5.09	19.02	18.40	10.08	0.00
MR	FRR	5.45	4.85	5.05	5.15	5.45
FAR	22.60	32.80	24.20	14.50	27.80
CR	FRR	3.80	5.30	5.45	5.15	4.45
FAR	14.40	33.50	20.10	24.40	11.00
SUBJ	FRR	5.40	4.75	5.20	5.00	5.25
FAR	11.70	31.10	12.00	32.40	25.10
TREC	FRR	5.00	4.10	4.50	5.30	4.50
FAR	16.02	37.85	32.60	23.48	26.80
Table 4. Performance of MDP on discrete prompt-based models (with  $5\%$  FRR allowance on the training set).
PLM	Attack	CA	ASR	FRR	FAR
BART	BadNets	94.03	98.79	2.85	11.42
AddSent	92.07	99.12	6.82	16.61
GPT-2	BadNets	92.78	91.28	5.60	6.24
AddSent	92.31	91.28	4.05	9.46
Table 5. MDP on alternative PLMs.
Masking rate. Recall that the masking sensitivity influences MDP's performance (§4.2). Here we vary the masking rate from 0.1 to 0.4 (with 0.2 as the default). Figure 6 presents MDP's FAR and FRR on SST-2 with respect to different attacks. Notably, properly setting the masking rate is critical for MDP. This can be explained as follows. An overly small masking rate may lower the probability of "hitting" the triggers, while an overly large masking rate may significantly alter the semantics of given samples after masking, both making clean and poisoned samples less distinguishable.
5.5 Alternative Scenarios
Thus far we assume that (i) the backdoor is injected into the PLM, (ii) the trigger is defined as explicit perturbation (e.g., adding words or sentences), and (iii) the attacker is unaware of MDP. Next, we evaluate MDP in alternative scenarios in which these assumption do not hold, with results summarized in Table 7.
Alternative threat models. BadPrompt [2] injects the backdoor into the prompt and releases the end-to-end PLM to the victim user, assuming that the user directly uses the model without further
Figure 6: MDP on SST-2 with varying masking rate.
Dataset	Attack	CA	ASR	FRR	FAR
SST-2	BadPrompt	87.50	81.39	7.16	8.28
StyleAttack	92.97	78.67	2.94	24.33
Adaptive Attack	94.04	82.71	6.94	13.78
MR	BadPrompt	84.75	90.10	4.45	6.70
StyleAttack	88.65	84.10	2.95	31.30
Adaptive Attack	89.65	86.45	4.70	9.85
Figure 7: MDP in alternative scenarios.
tuning. However, since the prediction of poisoned samples hinges on the trigger tokens, BadPrompt still exhibit high sensitivity to random masking, making them easily detectable by MDP.
Invisible triggers. Instead of using explicit perturbation (e.g., word addition), StyleAttack [25] defines the triggers as specific syntactic styles, making the perturbation less visible. As shown in Table 7, observe that StyleAttack is less effective (lower ASR) but also more challenging for MDP to detect (higher FAR). This may be explained as follows: visible triggers are often local features (e.g., words and sentences), while invisible triggers tend to be global features (e.g., patterns of lexical choice and syntactic structures). Intuitively, compared with local features, global features are less sensitive to random masking used by MDP.
Adaptive attacks. We further evaluate MDP against an adaptive attack, in which the attacker is aware of MDP and optimizes the PLM to evade detection. Recall that MDP detects poisoned samples based on their masking variance. Thus, with reference to a clean PLM and few-shot samples, the attacker attempts to minimize the masking variance of poisoned samples while optimizing the backdoored PLM. Specifically, we add the loss term of masking variance to Eq. 4 (with weight 1.0). Observe in Table 7 that compared with non-adaptive attacks (cf. Table 2), the adaptive attack marginally improves the evasiveness of poisoned samples; however, it also results in lower attack effectiveness. This finding corroborates our theoretical analysis in §4.4: there exists an interesting trade-off between attack effectiveness and evasiveness with respect to MDP.
Other NLP tasks. Similar to prior work (e.g., RAP [36] and ONION [24]), we focus on defending against backdoor attacks in the context of text classification. As backdoor attacks in other tasks (e.g., text generation) assume different threat models and attack/defense metrics, we consider extending MDP to such tasks under the prompt-based, few-shot settings as our ongoing work.
Fewer-shot data. While we evaluate MDP under limited few-shot data (e.g.,  $K$  as low as 4), in practice, the available data could be even scarcer (e.g., one- or zero-shot [30, 31]). Given the need of adapting PLMs on fewer-shot data, we aim to improve MDP to address the data-insufficiency limitation towards practical deployment.
Invisible triggers. It is shown that compared with local features, global features are less sensitive to the random masking used in MDP. We consider extending MDP to the setting of invisible backdoor attacks as our ongoing work. One promising direction is to redefine the random masking operation to include random syntactic-structure perturbation or random style perturbation.
This paper presents a first-of-its-kind defense for pre-trained language models as few-shot learners against textual backdoor attacks. At a high level, we exploit the gap between the sensitivity of clean and poisoned samples to random masking and effectively utilize the few-shot learning data to measure such sensitivity. The evaluation on benchmark datasets shows that our method outperforms baselines in defending against representative attacks, with little impact on the performance of victim models. Our findings shed light on how to enhance the security of pre-trained language models, especially in the prompt-based, few-shot learning paradigm.
This material is based upon work supported by the National Science Foundation under Grant No. 1951729, 1953813, 2119331, and 2212323. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
[1] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language Models are Few-shot Learners. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2020.
[2] Xiangrui Cai, Haidong Xu, Sihan Xu, Ying Zhang, and Xiaojie Yuan. BadPrompt: Backdoor Attacks on Continuous Prompts. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2022.
[3] Chuanshuai Chen and Jiazhu Dai. Mitigating Backdoor Attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification. Neurocomputing, 452:253-262, 2021.
[4] Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, and Yang Zhang. Badnl: Backdoor Attacks against NLP Models. In ICML 2021 Workshop on Adversarial Machine Learning, 2021.
[5] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. In ArXiv e-prints, 2017.
[6] Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. A Backdoor Attack against LSTM-based Text Classification Systems. IEEE Access, 7:138872-138878, 2019.
[7] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019.
[8] Wei Du, Yichun Zhao, Boqun Li, Gongshen Liu, and Shilin Wang. PPT: Backdoor Attacks on Pre-trained Models via Poisoned Prompt Tuning. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), 2022.
[9] Tianyu Gao, Adam Fisch, and Danqi Chen. Making Pre-trained Language Models Better Few-shot Learners. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2021.
[10] Yansong Gao, Yeonjae Kim, Bao Gia Doan, Zhi Zhang, Gongxuan Zhang, Surya Nepal, Damith C Ranasinghe, and Hyoungshick Kim. Design and Evaluation of a Multi-domain Trojan Detection Method on Deep Neural Networks. IEEE Transactions on Dependable and Secure Computing, 19(4):2349-2364, 2021.
[11] Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain. In ArXiv e-prints, 2017.
[12] Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks on pretrained models. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2020.
[13] Brian Lester, Rami Al-Rfou, and Noah Constant. The Power of Scale for Parameter-Efficient Prompt Tuning. In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021.
A Algorithm of MDP
Algorithm 1: MDP
Input:  $\mathcal{D}$  : few-shot data;  $\theta$  : PLM parameters;  $\mathcal{T}$  : fine-tuned text template;  $\nu$  : vocabulary;  $\gamma$  : threshold;  $X_{\mathrm{in}}^{\mathrm{test}}$  : a test sample  
Output:  $X_{\mathrm{in}}^{\mathrm{test}}$  is poisoned or benign  
 $\mathcal{A} \gets \emptyset$   
foreach  $X_{\mathrm{in}}^{(i)} \in \mathcal{D}$  do  
 $\begin{array}{r} X_{\mathrm{prompt}}^{(i)} \leftarrow [\mathrm{cls}] X_{\mathrm{in}}^{(i)}[\mathrm{sep}] \mathcal{T}[\mathrm{sep}]; \\ \text{add } p_{\theta}(v|X_{\mathrm{prompt}}^{(i)}) \text{ to } \mathcal{A} \text{ where } v \in \mathcal{V}; \end{array}$   
end  
 $X_{\mathrm{prompt}}^{\mathrm{test}} \gets [\mathrm{cls}] X_{\mathrm{in}}^{\mathrm{test}}[\mathrm{sep}] \mathcal{T}[\mathrm{sep}]$ ;  
 $\boldsymbol{k}^{\mathrm{test}} \gets p_{\theta}(v|X_{\mathrm{prompt}}^{\mathrm{test}})$ ;  
 $\boldsymbol{d}(X_{\mathrm{in}}^{\mathrm{test}}) \gets [D_{\mathrm{KL}}(\boldsymbol{k}^{\mathrm{test}} \| \boldsymbol{a}^{(i)})]$  where  $\boldsymbol{a}^{(i)} \in \mathcal{A}$ ;  
randomly mask  $X_{\mathrm{in}}^{\mathrm{test}}$  as  $\hat{X}_{\mathrm{in}}^{\mathrm{test}}$ ;  
compute the coordinates of  $\hat{X}_{\mathrm{in}}^{\mathrm{test}}$  as  $\boldsymbol{d}(\hat{X}_{\mathrm{in}}^{\mathrm{test}})$ ;  
 $\tau(X_{\mathrm{in}}^{\mathrm{test}}) \gets \Delta(\boldsymbol{d}(\hat{X}_{\mathrm{in}}^{\mathrm{test}}), \boldsymbol{d}(X_{\mathrm{in}}^{\mathrm{test}}))$ ;  
if  $\tau(X_{\mathrm{in}}^{\mathrm{test}}) \geq \gamma$  then  
return  $X_{\mathrm{in}}^{\mathrm{test}}$  is poisoned;  
else  
return  $X_{\mathrm{in}}^{\mathrm{test}}$  is benign;  
end
Proof. (Theorem 4.1) Given a single anchor  $X_{\mathrm{in}}^{*}$ , let  $k, \hat{k}$ , and  $a$  be the prediction distributions of  $X_{\mathrm{in}}, \hat{X}_{\mathrm{in}}$ , and  $X_{\mathrm{in}}^{*}$  respectively. We define the representational change of  $X_{\mathrm{in}}$  due to masking as:
$$
\tau \left(X _ {\mathrm {i n}}\right) \triangleq D _ {\mathrm {K L}} (\hat {\boldsymbol {k}} \| \boldsymbol {a}) - D _ {\mathrm {K L}} (\boldsymbol {k} \| \boldsymbol {a}) \tag {11}
$$
As  $X_{\mathrm{in}}$  comprises  $n$  tokens, there are  $n$  variants of  $\hat{X}_{\mathrm{in}}$ , one with the trigger token masked and the rest with a non-trigger token masked. Let  $\hat{X}_{\mathrm{in}}^{(0)}$  and  $\hat{X}_{\mathrm{in}}^{(i)}$  ( $1 \leq i \leq n - 1$ ) denote the two parts.
Let  $p^* \triangleq p_\theta (+|X_{\mathrm{in}}^*)$ . As  $X_{\mathrm{in}}^*$  is a clean sample,  $p^* < \kappa^-$  (negative) or  $p^* > \kappa^+$  (positive). Thus, for  $p \in [\kappa^-, \kappa^+]$ , the KL divergence function
$$
h (p) \triangleq p \log \frac {p}{p ^ {*}} + (1 - p) \log \frac {1 - p}{1 - p ^ {*}} \tag {12}
$$
increases (or decreases) monotonically with  $p$ . According to the assumption,  $p_{\theta}(+|\hat{X}_{\mathrm{in}}^{(0)}) \leq \kappa^{-}$  and  $p_{\theta}(+|\hat{X}_{\mathrm{in}}^{(i)}) \geq \kappa^{+}$  ( $1 \leq i \leq n-1$ ). To minimize the variation of the representational change of  $\hat{X}_{\mathrm{in}}$ ,  $p_{\theta}(+|\hat{X}_{\mathrm{in}}^{(i)})$  ( $0 \leq i \leq n-1$ ) should be close to each other. It thus follows that  $p_{\theta}(+|\hat{X}_{\mathrm{in}}^{(0)}) = \kappa^{-}$  and  $p_{\theta}(+|\hat{X}_{\mathrm{in}}^{(i)}) = \kappa^{+}$  ( $1 \leq i \leq n-1$ ). It can be derived that the minimum variation of the representational change of  $\hat{X}_{\mathrm{in}}$  is given by:
$$
\sigma \left(\tau \left(X _ {\mathrm {i n}}\right)\right) \geq \frac {\sqrt {n - 1}}{n} \left| h \left(\kappa^ {+}\right) - h \left(\kappa^ {-}\right) \right| \tag {13}
$$
To evade the detection,  $\sigma (\tau (X_{\mathrm{in}}))\leq \gamma$  , which completes the proof.
Proof. (Corollary) Recall that the function  $h(p)$  monotonically increases (or decreases) with  $p \in [\kappa^{-}, \kappa^{+}]$ . Thus, for given  $\kappa^{-}$ , it follows:
$$
\begin{array}{l} \left| h \left(\kappa^ {-}\right) - h \left(\kappa^ {+}\right) \right| \\ > \left| h \left(\kappa^ {-}\right) - h \left(\frac {1}{2}\right) \right| \tag {14} \\ = \left| h \left(\kappa^ {-}\right) + 1 + \frac {1}{2} \log p ^ {*} \left(1 - p ^ {*}\right) \right| \\ \end{array}
$$
Thus, if  $|h(\kappa^{-}) + 1 + \frac{1}{2}\log p^{*}(1 - p^{*})| > \frac{n}{\sqrt{n - 1}}\gamma$ , there is no  $\kappa^{+} > \frac{1}{2}$  that satisfies Eq. 13.
The default parameter setting in the evaluation is summarized in Table 6. The setting of baseline defenses mainly follows prior work [36]. For STRIP, we set the number of copies and replacement rate as 5 and 0.25, while the other parameters are set according to the best detection performance. For ONION, we test different thresholds on the perplexity change and choose the thresholds that approximately achieve  $5\%$  FRR on the training set. Then we remove outlier words with perplexity changes above the thresholds at inference time. For RAP, we bound the change of output probability as  $[-0.3, -0.1]$ . When training the word embedding of the RAP trigger, we set the learning rate as  $1.0\mathrm{e} - 2$ . The RAP trigger is inserted at the first position of each sample to avoid being truncated.