Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks
Authors: Bowei He
Lihao Yin
Huiling Zhen
Jianping Zhang
Lanqing HONG
Mingxuan Yuan
Chen Ma
Venue: ICLR 2025 Poster (2025)
Keywords: Language Model
Textual Backdoor Attack
Certified Robustness
Fuzzed Randomized Smoothing
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: This paper introduces Fuzzed Randomized Smoothing, a novel defense method that combines randomized smoothing with fuzzing techniques to provide certified robustness against textual backdoor attacks in pre-trained language models.

Abstract: The widespread deployment of pre-trained language models (PLMs) has exposed them to textual backdoor attacks, particularly those planted during the pre-training stage. These attacks pose significant risks to high-reliability applications, as they can stealthily affect multiple downstream tasks. While certifying robustness against such threats is crucial, existing defenses struggle with the high-dimensional, interdependent nature of textual data and the lack of access to original poisoned pre-training data. To address these challenges, we introduce **F**uzzed **R**andomized **S**moothing (**FRS**), a novel approach for efficiently certifying language model robustness against backdoor attacks. FRS integrates software robustness certification techniques with biphased model parameter smoothing, employing Monte Carlo tree search for proactive fuzzing to identify vulnerable textual segments within the Damerau-Levenshtein space. This allows for targeted and efficient text randomization, while eliminating the need for access to poisoned training data during model smoothing.  Our theoretical analysis demonstrates that FRS achieves a broader certified robustness radius compared to existing methods. Extensive experiments across various datasets, model configurations, and attack strategies validate FRS's superiority in terms of defense efficiency, accuracy, and robustness.

Published as a conference paper at ICLR 2025
CERTIFYING LANGUAGE MODEL ROBUSTNESS WITH FUZZED RANDOMIZED SMOOTHING: AN EFFICIENT DEFENSE AGAINST BACKDOOR ATTACKS
Bowei He Lihao Yin', Hui-Ling Zhen', Jianping Zhang", Lanqing Hong",
Mingxuan Yuan?, Chen Ma 1 City University of Hong Kong, 2 Huawei, Hong Kong, The Chinese University of Hong Kong
ABSTRACT
The widespread deployment of pre-trained language models (PLMs) has exposed them textual backdoor attacks, particularly those planted during the pre-training stage. These attacks pose significant risks to high-reliability applications, as they can stealthily affect multiple downstream tasks. While certifying robustness against such threats is crucial, existing defenses struggle with the high-dimensional in- terdependent nature of textual data and the lack of access to original poisoned pre-training data. To address these challenges, we introduce Fuzzed Randomized Smoothing (FRS), a novel approach for efficiently certifying language model ro- bustness against backdoor attacks. FRS integrates software robustness certification techniques with biphased model parameter smoothing, employing Monte Carlo search for proactive fuzzing to identify vulnerable textual segments within the space. This allows for targeted and efficient text randomiza- tion, while eliminating the need for access poisoned training data during model smoothing. Our theoretical analysis demonstrates that FRS achieves a broader certified robustness radius compared existing methods. Extensive experiments across various datasets, model configurations, and attack strategies validate FRS's superiority in terms of defense efficiency, accuracy, and robustness.
INTRODUCTION
Pre-trained language models (PLMs) have become the cornerstone of numerous natural language processing tasks, with fine-tuning being the most common approach for adapting these models to customized downstream applications (Kenton & Toutanova] 2019 Liu al. 2019 Touvron et al. 2023). However, the widespread adoption of PLMs also has new vulnerabilities, especially textual backdoor attacks. These attacks involve injecting malicious knowledge into PLMs, either through poisoned training data or direct modification of model parameters. compromising their reliability and irustworthiness (Cheng et al. 2024 Zhao al. 2024). Different from the ordinary poisoning data attack, textual backdoor attacks are particularly insidious because they do not significantly impact model performance on benign inputs, making them difficult to detect through standard evaluation methods. The attacked PLMs only exhibit malicious behavior when presented with specific trigger inputs, allowing them to evade human inspection.
In the PLM pre-training and fine-tuning phases, there are two backdoor planting paradigms (Guo et al., 2022): 1) embedding backdoors in the pre-trained model by poisoning training before model weights published for downstream use; 2) embedding backdoors in the downstream model during the fine- tuning phase via poisoning the fine-tuning data. Note that the second paradigm is fundamentally the same the backdoor attacks to conventional standalone models. Considering the widespread and potentially more harmful nature pre-training phase attacks, which can simultaneously affect multiple downstream applications, we focus on the pre-training backdoor attack.
Though different backdoor attack strategies for PLMs have been investigated, the effective defense schemes against them are less explored. As steganography techniques for ensuring trigger invisibility
Work done as an intern in Huawei Noah's Ark Lab, Hong Kong, boweihe2-c@my ci cit. edu.hk Corresponding author, chenma@ci tyu. edu. edu.

---

Published as a conference paper at ICLR 2025
constantly evolve, conventional empirical defense methods (Qi et al. 2021a; Yang et al. 202 Yan itt al. 2023) find it increasingly challenging to consistently and effectively detect triggers. Besides, many existing methods (Chen & Dai 2021 Cui et al.] 2022) are restricted to addressing backdoor attacks on the downstream model during the fine-tuning phase, leaving a gap in our focus. In this work, delve into the certified defense approach with the theoretical guarantee against backdoor attacks on PLMs, which offers more robust and provable effectiveness compared to empirical methods.
Randomized smoothing has been widely regarded as an effective approach for certified robustness against the evasion attacks (Cohen et al. 2019), like the adversarial attacks. Recently, a few pioneering works (Xie et al., 202T; Weber et al., 2023) have also explored its potentials against backdoor attacks and are attracting increasing attention. However, almost all of these methods are limited to conventional vision and tabular scenarios with continuous numeric inputs, and face challenges when directly applied to PLMs with discrete natural language inputs. Furthermore, training from scratch in such scenarios means defense methods can have access to poisoned data and protect the model during the poisoning training phase (in-attack). However, in our setting where backdoor attack happens during the pre-training phase, the protectors have no access to the original poisoned training data, thus the corresponding defense methods only be post-attack. Most critically, the traditional randomized smoothing methods do not investigate the potential model "bugs" -malicious knowledge introduced during the poisoning phase, resulting in poison-agnostic passive defense. This seriously hinders the further enhancement of defense efficacy and language model robustness. Meanwhile, the success of software verification techniques like fuzzing in program robustness certification makes inspiring to improve the PLM defense efficiency, output accuracy, and extend the certified robustness radius by probing the PLM bugs proactively via iterating over mutated or generated testing samples.
Based on the above motivations, we propose the Fuzzed Randomized Smoothing (FRS) framework. We first formulate the randomized smoothing framework against the textual backdoor attacks for PLMs, which can accommodate various types of triggers in the Hameraa--Lvensheee erau 1964 Levenshtein et al. 1966). Second, we propose the biphased model parameter smoothing to conduct the post-attack defense during the fine-tuning and inference phases. The direct smoothing on model parameters instead of fine-tuning data helps avoid the huge resource overhead. Then, we develop the fuzzed text randomization which employs Monte Carlo tree search to identify the vulner- able areas containing triggers, thus concentrating randomization probability on identified areas. In addition to theoretically proving the broader certified robustness radius and higher defense efficiency, we also conduct extensive experiments to empirically demonstrate our approach's 's advantages and discuss its scalability to future larger language models.
space (Dam)
2 RELATED WORK
Textual Backdoor Attacks and Defenses Different from the previous evasion attacks to the language models, the textual backdoor attacks take effect in both training and inference phases via poisoning training data/model parameters and perturbing inputs with triggers respectively, which make them more covert and difficult to defend against. Some pioncering works (Dai et al. 2019 Chen & Dai 2 discussed how toxify the training corpus attack the LSTM- language models. Due to the prevalence of pre-training and fine-tuning paradigm grrdiggmmm for transform-structiee
language models,
more recent works (Zhao et al. 2023 Chen et al.) 202 la b Shen et al. 2021 Yang et al.. 202 la Li et al. 2 Zhang et al.] |021 Guo et al. 2022 Qi et al. 202 c) explored how to inject the lethal backdoor attacks to pre-trained models, making them vulnerable in various downstream tasks. Correspondingly, to alleviate the harms brought by such kinds of textual backdoor attacks, some empirical defense methods (Qi et al.) 2 a) have also been proposed. Nevertheless, most of them are based on heuristic rules, lacking the enough theoretical guarantees though achieving acceptable performance in some specific scenarios. Our focus in this paper is to equip language models with certified robustness against textual backdoor attacks, regardless of the attack strategies and forms.
Certified Robustness of Language Models Though many empirical defense methods (Qi et al., 2021a; Cui et al. 2022 Yan et al. 2023) against various textual attacks have been proposed and widely deployed in industrial applications, the certified defense approaches with theoretical guarantees are still being regarded as the Holy Grail of research in this direction. Among existing attacks to language models, the evasion attacks and backdoor attacks are two kinds of most common and impactful
ones. Concretely, interval bound propagation (Jia al. 2019 Huang al. 2019 Ye et al. 2020
2

---

Published as a conference paper at ICLR 2025
Wang et al., 2023). abstract interpretation (Bonaert et al. 2021: Du al., 2021 and randomized smoothing (Zang et al. 2023 Zhao et al. | Zeng et al.; Wang et al., |021 Cohen et al. 2019 Ji
al. 2024: Zhang al. [2024b Lou 2024) are the most representative schemes achieve certified defense against the evasion attacks. However, for the more challenging and harmful backdoor attacks which directly injects the malicious information into the language model parameters, certified robustness solutions are still lacking, How to adapt the successful methods against evasion attacks like randomized smoothing the textual backdoor attacks is interesting and also meaningful, which is the focus of this work. Fortunately, some preliminary works (Wang et al. [2020] Xie et al. 202T]] Weber al., (023)) have explored the related foundational techniques in computer vision scenarios, which can shed some insights for our method design.
3 PRELIMINARIES
In this section, we provide the formulation of the textual backdoor attack on PLMs and the corre- sponding goal of defense. In our scenario, the language model (LM) s(-) parameterized by 0 is first pre-trained with the mixture of clean and poisoned corpus to plant the back patterns by the mali- cious attackers. The pre-trained model parameter checkpoints are then uploaded to the open-source repositories like Hugging F The users download the attacked pre-trained model parameters 0p and fine-tune them to 0g on the local downstream data DF with x = [2,, ..... ...,, Fr.. as the textual input and y E y as the output label. We introduce the normalized Damerau-Levennhteen distance (Damerau 1964 Levenshtein et al. 1966) dp (x, x') to measure the edit distance between the original benign input X and the perturbed input x' by the triggers, which allows the operations like token insertion, deletion, substitution, and transposition. Thus, due to the above ficxibility, the normalized
distance can be applied to almost all trigger patterns of existing textual backdoor attack methods, including character-level, word-level, and sentence-level ones.
The goal of the defense is guarantee that the model prediction of [(x';pp) can be consistent with that of [xx 0p) whose training procedure is not attacked by the poisoned corpus. The LM f()) is certified robust against the backdoor attack if it satisfies the following criterion: for any input x,
0'x
[(x; 0F), Vx (x, x') S R, L.
where (0 < R, S 1) denotes the robustness radius. A certified robust LM is expected to generate the robust prediction, given that at most R, L tokens the input x are perturbed.
4 METHODOLOGY
4.1 RANDOMIZED SMOOTHING DEFENSE
Randomized smoothing was originally proposed to achieve the certified robustness effect against evasion attacks in computer vision scenario (Cohen et al. (2019). We first extend its basic framework to the textual backdoor attacks which have not been thoroughly explored before. Generally, randomized smoothing introduces a smoothed model í based on the base model f(() by exerting the random noise on the fine-tuning data and test samples. In essence, the rationale behind leveraging the randomized smoothing defense lies in the observation that the inclusion of noise mitigates the prevalence of decision boundaries with pronounced curvature, thereby reducing the susceptibility to backdoor attacks. Thus, we denote the noisification operator as b and define the smoothed model f as:
f(x') = arg max (f, @ u; 000', Dy e c)) = y). wEy
(2)
where random noise variables u ~ Pu, c ~ P. follow the independent random distributions and are added the perturbed test samples and fine-tuning data, respectively. Here, the Ω indicates the fine-tuning procedure which takes the poisoned pre-trained model parameters 6'p and randomized fine-tuning data Dp @ c (notes as DP) as inputs and returns the smoothed fine-tuned parameters 0p.
In practice, considering the complexity of LM f(-) itself, Monte Carlo simulation is an effective approach to approximate the above probability P.,. (f(x' @ u; 200pp. DF @ E)) y) in Eq 2
hhtps//tugggnngfccc.o/
3

---

Published as a conference paper at ICLR 2025
Therefore, we employ a number of base models f(:0g s K) with parameters 0p,s fine- tuned on the sampled randomized datasets Dp = Dr @ En to vote on the final result. Therefore. the Eq. 2 can be transformed to following:
K
f(x':0)) = arg max
(i(f(xe: 0r.,)
y),
yEY
1mm
x x' + uk OF. s(0'p DF + ex),
where 0p=110 10 0p:2 FF.k] indicates the parameters of the smoothed model f which is actually the ensemble of base models. The distribution of randommess applied on the finettuning dataset is controllable, often the isotropic Gaussian noise c ~ (0, o²I). Naturally, according to the properties of Monte Carlo simulation, as the number of the above base models increases, the voted results become more reliable and the scope of the robustness region becomes larger.
Randomized smoothing can guarantee that, when the perturbation scale is less than the robustness radius R,, the prediction fx':: 0p) of the smoothed model for the perturbed input x' aligns with the prediction f(x; 0F) of the model trained on the completely clean dataset for benign input x, within a confidence level of 1 a. Based on this framework, we develop the practical schemes for model parameter smoothing as described in Section 4.2 and test sample randomization in Section 4.3
respectively. Finally, we elaborate on the theoretical analysis in Section 4.4 and Appendix
4.2 BIPHASED MODEL PARAMETER SMOOTHING
If directly following the approach described in Section 4. one might consider fine-tuning K pre- trained language models on K distinct, randomized downstream datasets. However, this approach would impose considerable computational burden. As a result, it becomes an unrealistic strategy for practical scenarios. Therefore, there exists an urgent need for post-attack defense mechanisms that only have certified robustness guarantees but also provide efficient execution. Thus, we propose the biphased model parameter smoothing as a targeted solution particularly for large language models, which performed during both the fine-tuning and inference phases. This biphased approach notably diminishes data storage requirements for different versions of randomized fine-tuning datasets and drastically reduces the computational overhead associated with training. Besides, this strategy advocates for the selective smoothing of parameters in // layers proximal to the output those most vulnerable to backdoor attacks, as highlighted the literature (Kurita et al., 2020). The detailed smoothing procedures in such two phases are as follows:
Fine-tuning Phase: In iteration (1 S i < T) of fine-tuning process, the model parameter smoothing is performed as follows:
0 = (0,
B,)) sop--
(4)
where 1) and p indicate the learning rate of the fine-tuning process and the norm bound, respectively. g(: denotes the gradient function and B, is the mini mini-batch iteration i. Especially, 0 = 0'p.
Inference Phase: When completing the model fine-tuning we conduct the parameter smoothing to the finally obtained 0% independently for K times:
0F Clip, Clip,((+) Ex. A 1.,,.., K,
(5)
Once the K smoothed copies of L.Ms are generated at the beginning of the inference phase, they are fixed and employed for every test sample during the whole inference phase.
4.3 FUZZED TEXT RANDOMIZATION
Traditional text randomization in randomized smoothing relies on uniform randomization in the text input, suffering from low efficiency and limited certified robustness radius. Motivation by the fuzzing technique in the software verification research, we design the Monte Carlo tree search (MCTS)-based fuzzed text randomization to first proactively identify the vulnerable areas containing the triggers in the input text. We choose MCTS for its ability to efficiently explore high-dimensional discrete textual spaces and adaptively focus on promising areas. Then, such areas will be imposed more possibilities to conduct the text randomization operations. In such way, the obtained randomized samples
4

---

Published as a conference paper at ICLR 2025
more likely to remove the information related the backdoor damage the triggers' dedicated structure while keeping genuine features intact, which means backdoored texts can be reverted back to their corresponding benign versions. Thus, a broader certified robustness radius can be successfully achieved with the majority voting process over the same number of randomized samples.
4.3.1 VULNERABLE AREA IDENTIFICATION
The primary objective of this MCTS-based fuzzing approach is efficiently identify potential vulnerable areas in the input text that may contain backdoor triggers. Thus, we first define S as search tree, with nodes i-th token to j-th token. Each node is associated with score (n.), reflecting the potential of the corresponding segment to exhibit trigger impacts. The detailed fuzzing process is as follows:
corresponding to segments Seg(x', 1, of the perturbed text x' from
Initialization Initialize S with root node Throot representing the original input x'. Set (T)oott 0.
MCTS Iterations Then, in each iteration of MCTS, the following steps are executed:
Step 1: Selection Traverse from the root to a leaf node ny using the Upper Confidence Bound (UCB) applied to trees policy: UCB(m)) = ((n)) + +V
In (n(NNarent)
(6)
Nn
where C' is exploration constant, Nu, signifies the number of visits to node n,, and N Narennt is the number of visits to ny 's parent node.
Step 2: Expansion Upon reaching leaf node nu the conclusion of the Selection phase, we eval- uate whether the text segment corresponding to nt, denoted as Seg(x, i, can be further devided. This evaluation is based on the linguistic features of the segment, such as phrase boundaries, clause demarcations, or named entities contained within. If the further subdivision is viable, child node neww for n, will be generated, which represents subdivision of (xg(x', i,j). The generation of Onew is thus expressed as:
neww = Seg((x x k) or Seg(x', 1,5), - 1,.,,
Step 3: Simulation Select a mutation operation m from a predefined mutation set M, which includes insertion, deletion, substitution, and transposition operations in the Lamerau-Levenshhein space. Next, apply m to Aneww, thus generating a new textual variant x of original x'. Then, x is evaluated to ascertain the deviation in LM's response. Here, utilize evaluation criterion E(x, x') based on KL divergence (Kullback & Leibler 1951) to quantify this deviation:
FI(x, x') (x (Py(u) (|ll/>)||Pppp) (ylx))
(8)
where (y(() indicates the probability distribution of LM f on different outputs y.
Step 4: Backpropagation Update scores of all nodes from nnew up to Nroot based on E(x,x'
thus refining the selection process in subsequent iterations:
(n(n) Nn N₂
Vn-1(m) E(xx'')
N₂
Upon reaching predefined number of iterations or a termination criterion, identify segments corresponding to nodes with the highest scores V(n) as the most likely vulnerable areas T(x') with the greatest potential to contain backdoor triggers.
4.3.2 TEXT RANDOMIZATION
Building upon the established MCTS-based fuzzing framework, we proceed to employ text random- ization while maintaining the normalized Comerau-Levenshiee distance within a specified threshold. This constraint ensures the semantic and structural integrity of the text by limiting the number and type of textual transformations, thereby preserving the original meaning and syntactic structure.
After identifying vulnerable areas T(x') that potentially contain triggers, we apply targeted randomization strategy which employs differential probabilities for textual segments within and outside T(x'). The process consists of the following key steps:
Randomization Process:
5

---

Published as a conference paper at ICLR 2025
Step 1: Damerau-Levenshtein Compliance We begin by setting distance threshold A. This threshold defines the maximum allowable modification to the original text, measured using the normalized Dumeraa--evensshein distance. Specifically, any alteration to the text must satisfy:
dD,(X,x') A,
(10)
where (DL(X, x') represents the normalized Damerau-LLvennshein distance between the original and mutated text variants.
Step 2: Seibabbiity-Weighheee Randomization Strategy We formulate a probability weighting
function W Seg((',,, i, + [iz,WH] differentially allocate randomization probabilities across the text:
Wssemment WH, segment C Tx(x),
(11)
WL, otherwise,
where Wy > wt signify higher and lower randomization probabilities, correspondingly. Especially, let wM denote the traditional uniform randomization probability for each segment. Here, to ensure the equal overall randomization probability for the whole x', we have wt WM WH.
Step 3: Randomization Implementation For each identified segment Seg(x', i,j), we determine whether to apply a mutation based on a Berouuli distribution with parameter w, which represents the allocated weight for that segment:
B Bermouli(WWSSeg((1, i,,))).
(12)
If mutation is to be applied (B 1), we randomly select a mutation operation m from the mutation operation set M. The mutation is then applied as follows:
m(Seg(x' (i,)),, =f Seg(x', (i,), otherwise.
((,))
(13)
Finally, we consolidate all {(.j) to obtain the randomized version X.
Step 4: Post-Randomization Validation Verify the randomized text & to ensure compliance with dpz criteria:
Validate dpr. (X, x') (x, S.
(14)
Discard any that does not satisfy this condition.
Step Aggregation for Randomized Smoothing Collect all validated x instances to create comprehensive sample set for conducting randomized smoothing. Subsequent model predictions and majority vote processes utilize this set to achieve resilient model decisions with an enlarged certified robustness radius R,-
This approach enhances the effectiveness of trigger neutralization in vulnerable areas while reducing unnecessary variations and maintaining linguistic coherence. By constraining randomization within defined perturbation limits, it balances text modification with preservation of original meaning.
4.4 THEORETICAL ROBUSTNESS BOUND
Assumption 1 (Effective Parameter Smoothing). The output of the smoothed model f(x, 0r) on the benign input x is consistent with that of the clean ciean.fine fine-tuned model f(x, 0)), which is also denoted as y*:
f(x, 0r) = fxx,
(15)
This assumption can be approximately guaranteed with the biphased parameter smoothing introduced in Sec. 4.2 as long as in Eq. Lis set small enough.
Theorem 1 (Model Robustness Condition). Based on the Assumption 7 the lower bound of the probability that the smoothed model f returns the ye for perturbed input x' after the randomized smoothing (x' Beta(a; ya K Kys + 1). where Beta(a: (.. the a-th quantile of beta distribution. Ky is the voting count for y+ in K voteess, and a indicates the confidence level (Zeng et al.). Then, if
(xx BA 0.5,
(16)

---

Published as a conference paper at ICLR 2025
with probability least 1 oz f(x') ys. Here, A denotes the probability upper bound that trigger segment (with the maximum length of R, L) is not completely randomized:
= -
(17)
where w indicates the randomization probability in the trigger segment (subset of identified vulnerable area). B as long as the model is fully trained to the convergence during the Corollary (Broader Robustness Radius). To meet the same level of robustness condition in Eq. T6 and provide the same oulpul probability lower bound Py (x'), under the same number of base models K, our new robustness radius is as follows:
phuse.
log(wM) Reld
Reew
(18)
log(wH)
where Reeld is the old radius for traditional randomized smoothing which does not employ our log(w>s)
proposed fuzzed text randomization. Considering that WM <WH, log(#x) 1, which means our
Ryew is larger than Rold. Especially, with more MCTS iteration budget, the confidence that the trigger is successfully captured can be higher, which means we can set Wt. -> 0,WH -> Thus, the enlargement of the robustness radius can be more obvious.
the stronger efficiency of our FRS method on certifying robust language models against backdoors is proved theoretically.
From Corollary 1
EXPERIMENTS
When conducting the experiments, focus answering following questions deeply analyze the advantages of our proposed approach: 1) RQ1: Can our method achieve better backdoor defense performance compared with other empirical defense and randomized smoothing-bane certified defense strategies? 2) RQ2: Can our method achieve broader certified robustness radius? 3) RQ3: Can our proposed biphased model parameter smoothing and fuzzed text randomization modules both contribute to the defense performance positively? 4) RQ4: Can our method achieve consistent defense performance over different victim models?
5.1 EXPERIMENT SETUP
Victim Language Models: To demonstrate the effectiveness of approach PLMs of different configurations, we conduct extensive experiments on group of diverse PLMs. These include BERT (Kenton & Tooutanova 2019), pioneering encoder-structured model with hundreds of millions of parameters, RoBERTa (Liu et al. 2019). a more robust PLM of similar size trained on larger dataset with dynamic masking, and the recently developed LLaMA3 (Dubey et al. 2024), decoder- structured model with billions of parameters which has been pre-trained on colossal corpus.
Based the assumptions regarding different degrees of attacker knowledge about the target downstream task, the existing pre-training phase textual backdoor attack schemees can be classified into three types: full data knowledge, domain shifft, and data free. To comprehensively validate the defense effectiveness against different types of attacks, we adopt RIPPLc, (Kurita et al.) 2020), LWP (Li et al. 2021). and BadPre (Chen al. 2021a) to instantiate such three kinds of methods, respectively.
Attack Methods:
Defense Baselines: We compare our proposed FRS with both empirical defense and certified defense methods. The first category includes methods working different phases: inference-time defense:
RIPPLe, ea (Kurita et al.) 2020), ONION (Qi et al. (021a) ab, RAP (Yang et al. 2021b). Bite (Yan et al.
2023 PSIM (Zhao et al. 2224); training- training-time time defense: BKI (Chen & Dai
2021). R-Adaptor (Zhu
ct al. 2022). As for the certified defense, we adopt the recently proposed TextGuard (Pei et al. 2023).
2023) evaluate the per- formance of different defense methods on several representative downstream tasks with corresponding datasets: sentiment analysis: SST-2 (Socher al. 2013); toxicity detection: OffensEval (Zampien) ct al. 2019); topic classification: AG'New> (Zhang et al., 2015). When conducting domain shift poisoning pre-training, we utilize the IMDB (Maas et al. 201 Twitter (Founta al. 2018). and 20
Evaluation Tasks and Datasets:
Following previous literature (Pei et al.
7

---

Published as a conference paper at ICLR 2025
Table 1: Results of different defense strategies under various pre-training textual backdoor attack approaches on SST-2, OffensEval, and AG's News datasets. BERT-base is taken as the victim model. Higher CA, PA, and lower ASR indicate more satisfying defense performance. indicates the statistical significance for p < 0.01 on L-test.
RIPPLe.
LWP
BadPre
Dataset
Method
CAT
PAt
ASR
CAT
PA
ASR,
CAT
PA
ASR..
RIPPLe,
81.19%
54.95%
82.31%
83.25%
65.90%
71.46%
91.74%
78.19%
58.83%
ONION
75.28%
59.64%
62.30%
78.46%
62.17%
59.82%
83.72%
79.82%
55.73%
RAP
72.82%
56.41%
68.41%
74.34%
59.65%
65.37%
84.74%
76.24%
61.37%
Bite
78.06%
62.32%
57.80%
80.19%
28.295
52.47%
89.25%
82.98%
42.16%
SST-2
PSIM
75.72%
59.81%
60.22%
79.02%
60.70%
58.93%
87.19%
81.43%
46.03%
BKI
70.47%
53.01%
76.93%
71.95%
58.28%
68.33%
74.28%
77.35%
52.78%
RdAdapoor
72.45%
55.22%
71.36%
74.38%
60.74%
62.11%
82.67%
76.72%
48.64%
TexiGuard
74.95%
65.24%
52.62%
76.
71.45%
47.83%
85.60%
84.12%
33.56%
FRS
82.36%
73.25 %
45.12%
85.67 %
82.91 %
34.25
91.64%
91.02% %
18.64%
RiPPLea
83.25%
72.34%
61.71%
85, 39%%%
75.76%
59, 09.
93.08%
81.32%
54.26%
ONION
78.24%
688.55%%
50.24%
79.51%
71.16%
62..54%
82.67%
77.41%
51.78%
RAP
79.71%
57.48%
68.23%
82.63%
61.82%
68.92%
85.01%
76.89%
56,23%
Bite
82.98%
72.79%
53.04%
83.86%
73.92%
59.78%
92.06%
84.21%
48.75%
OffensEval
PSIM
82.75%
70.51%
54.92%
83.17%
71.95%
58.53%
92.31%
81.80%
50.39%
BKI
72.46%
58.50%
67.39%
74.54%
63.37%
65.29%
76. 18%
75.94%
$2.78%
R- R-Aapptor
75.16%
59.08%
65.71%
76.08%
67.24%
63.42%
78.84%
78.47%
49,52%
Texi FRS
77.31%
74.08%
52.14%
78.42%
78.58%
50.17%
81.56%
85.83%
45, 42%%
85.61%
79.38 %
42.59%
87.63%
85,70 %
41.24%
94.05 %
91.43%
38.86 %
RIPPLes
76.24%
47.26%
68. 28%
80.25%
56.26%
62.37%
91.07%
74.32%
45.29%
ONION
68.35%
58.19%
56.27%
72.40%
63.58%
54.61%
83.82%
71.25%
42.62%
RAP
62.27%
53.20%
65.38%
69.73%
61.85%
60.52%
85.54%
73.25%
48.65%
Bite
73.48%
61.75%
48.24%
77.80%
66.09'
49.78%
91.32%
76.41%
42.57%
AG's News
PSIM
72.13%
61.24%
51.09%
75.32%
65.14%
50.91%
90.87%
74.93%
41.98%
BKI
64.20%
44.64%
74.18%
69.48%
49.83%
70.56%
78.36%
65.38%
51.72%
R-Adaptor
65.98%
45.27%
71.01%
72.07%
52.68%
65.25%
80.79%
68.49%
51.23%
Tex:Guard 65.47%
64.29%
45.98%
71.52%
69.34%
44.17%
85.75%
83.81%
41.03%
FRS
79.84%
72.36%
37.04%
83.76%
80.84%
38.16%
92.25%
89.34%
36.93%
Newsgroups Lang 1995) the proxy dataset for sentiment analysis, toxicity detection, and topic classification task, respectively.
We evaluate different defense methods on three metrics: clean accuracy (CA), poisoned accuracy (PA), and attack success rate (ASR). The higher CA indicates that the defense strategy does not influence the model performance obviously on benign inputs, implying it does not fall into the overcaution. Meanwhile, the higher PA and lower ASR show that the backdoor attacks can be effectively defended and the PLM output accuracy can still be guaranteed under attacks.
Evaluation Metries:
We run each experiment five times with different random seeds and take the average as the final result. The experiments are conducted on NVIDIA RTX A6000 GPUs with 48GB memory. We set the base model number K = 20, the variance of Gaussian noise applied to the parameter smoothing a 0.01. H is set as 10. We download the uncased version of BERT and RoBERTa models, well as pre-trained version of ALMMM3-88 model from HuggingFace.
Implementation Details:
To enhance the reproducibility, we provide more details regarding the experiment setup in Appendix B
5.2 EXPERIMENT RESULTS AND ANALYSIS
5.2.1 DEFENSE PERFORMANCE (RQ1)
To demonstrate that our method can achieve better backdoor defense performance compared with baselines, we provide the performance of different empirical defense basclines, certified defense baseline TextGuard, and our FRS under above three backdoor attack approaches in Table T We can first observe that the certified method TextGuard and our FRS outperform empirical defense baselines on both PA and ASR metries on three datasets, which verifies the advantage of this scheme. Second, the poor performance of TextGuard on CA can be noticed, which is due to that it breaks the syntactic and semantic integrity of the original texts during the word hashing assignment. Third, our FRS almost outperforms all empirical defense baselines and certified defense baseline TextGuard on CA, PA, and ASR metrics among three datasets, which demonstrates that it can not only effectively mitigate the negative impact brought by backdoor attacks perturhed samples, but also minimize the
8

---

Published as a conference paper at ICLR 2025
performance drop benign samples resulted from the over-caution of defense methods themselves. should be noted that we adopt the same number of base models here TextGuard and FRS for fair comparison. The superiority of FRS over Textguard on various metries further verifies its relative robustness certification efficiency.
5.2.2 CERTIFIED ROBUSTNESS RADIUS (RQ2)
To validate that our FRS method can indeed bring broader certified robustness radius, we directly calculate the robustness radius value achieved by our method. detail, for each test sample, find the maximum percent- age of tokens that can be perturbed while the model still maintains correct prediction with high probability (e.g.. 95% confidence) as the robustness radius. For ensuring compre- hensiveness, we calculate the average and maximum of these robustness radii across all test samples. We compare the results of FRS with the best -performing baseline. TextGuard, which is also the only certified defense baseline. According to the results presented in Table[2] we can obtain the following observations: First, our FRS method consistently outperforms TextGuard across all datasets in both average and maximum robustness radius. Second, FRS achieves notably higher average robustness radius compared to TextGuard. The improvements range from 25.72% (OffensEval)) to 34.87% (AG's News), indicating substantially better average-case robustness across various downstream tasks. Third, FRS also extends the maximum achievable robustness radius across all datasets, with improvements ranging from 11.82% (AG's News) to 20.30% (OffensEval). This demonstrates FRS's ability to provide certified robustness against more severe perturbations. All these observations align with our theoretical expectations of a broader certified robustness radius discussed in Section 4.4] Besides, more results on certified accuracy provided in Appendix C also demonstrate the broadened robustness radius by our FRS. Thus, the Corollary is persuasively validated with empirical results.
Table 2: Robustness radius results on SST-2, OffensE- val, and AG's News datasets.
Dataset
Method
Avg. radius
Max radius
TexiGuard FRS
27.51% % 34.87%
43.20%
SST-2
48.63%
TextGuard FRS
29.39% 36.95%
44.72%
OffensEval
53.80%
TexiGuard FRS
21.68% 29.24%
37.91%
AG's News
42.39%
Table 3: Ablation study on SST-2, OffensEval, and AG's News datasets, with BERT-base as victim.
RIPPLc.
LWP
BadPre
Dataset
Method
CAT
PA
ASR
CAT
PAT
ASR
CAT
PAT
ASR1
-MPS
82.48%
63.6%%
51.09%
85.82%
74.95%
44.52%
91.60%
85.25%
27.87%
SST-2
FTR
82.08%
70.97%
48.73%
85.45%
78.83%
38.64%
91.43%
88.27%
23.15%
FRS
82.36%
73.25%
45.12%
85.67%
82.91%
34.25%
91.64%
91.02%
18.64%
HMPS
85.52%
76.46%
49.53%
87. 69.699
80.62%
47.25%
94.12%
87.79%
42.07%
Oval
FTR
85.30%
77.21% %
45.47%
87.51
83.29%
44.48%
93.98%
89.68%
40.85%
FRS
85.61%
79.38%
42.59%
87.63%
85.70%
41.24%
94.05%
91.43%
38.86%
BMPS 79.91%
66.46%
42. 22.02%
84.02%
19%
42.27%
92.44%
84.91%
39, 8.%
AG's News
FTRR
79.60%
68.25%
39.94%
83.24%
78.36%
39.50%
92.12%
87.63%
38. 12%
FRS
79.84%
72.36%
37.04%
83.76%
80.84%
38.16%
92.25%
89.34%
36.93%
5.2.3 ABLATION STUDY (RQ3)
To validate that our proposed biphased model parameter smoothing (BMPS) module and fuzzed text randomization (FTR) module are both meaningful for the ultimate performance, we remove them from the overall method framework, respectively and conduct the experiments on above three datasets. The corresponding results are provided in Table First, we can find that removing such two modules will indeed result the performance drop on PA and ASR metrics under different attack approaches among three datasets. This phenomenon can be even more obvious the AG's News dataset which is more fragile. This effectively demonstrates that the contribution brought by the BMPS and FTR are both positive. Besides, performance CA metric of -FTR version and original FRS version are similar, which indicates the influence of FTR to model performance on benign samples is weak. Interestingly, removing the BMPS module leads to the slight improvement on CA metric, which can

---

Published as a conference paper at ICLR 2025
Table Results of TextGuard and our FRS over different victim models with various architectures and sizes on SST-2, OffensEval, and AG's News datasets. Higher CA, PA, and lower ASR indicate more satisfying defense performance. indicates the statistical significance for p < 0.01 on t-test.
SST-2
OffensEval
AG's News
Dataset
Method
CA1
PA
ASRL
CA1
PAt
ASR
CA1
PAT
ASR
TextGuard FRS
74.95%
65 24%
52.62%
77.31%
74.08%
52.14%
65.47%
64.29%
45.98%
BERT-base
82.36 %
73.25% x
45.12%
85.61 %+
79.38% *
42.59%
79.84% %
72.36% *
37.04%
BERT-large
TextGuard FRS
84.92% %
78.83%
77.43%
70.56%
47.35%
41.87% *
87.24%
80.95%
82.56%
77.73%
47.89%
39,41 %
82.59 %
70.23%
76.81 *
68. 85%
41.76%
34.29%
RoBERTa-base
TextGuard FRS
82.21%
74.89%
48%18%
83.37%
80.15%
44.26%
73.86%
12%
38.45%
86.47 %
80.15%
38.54%
88.78 %
84.92%
36.75%
84.92 %
79,57%
30.86 %
TextGuard FRS
82.89%
63.6
42.37%
84.48%
81.255%
42.91% 1%
75.29%
73. 74%
37.42%
RoBFRTa-large
87.36%
81.27 %
37.21%
89.32% %
85.84 %
35.67%
85. 35.86% %
80.39
31.18%
LLaMA3-BB
TextGuard FRS
86.74%
79.622 62.
29.95%
88. 68.
85.39%
31.03%
79.54%
77.93 77.93%
29.82%
89.83 %%+
84.76
26.82%
92.15% %
89.044
24.28% %
89.17 %
84.25%
23.73%
be explained that smoothing model parameter can break the model comprehension capability to benign samples, though enhancing the robustness against the perturbed samples containing the triggers. Further ablation study concerning the effect of each phase in BMPS is provided in Appendix D
5.2.4 CONSISTENCY OVER DIFFERENT VICTIMS (RQ4)
To further explore whether our FRS's advantage remains against other baselines when the model size increases structure varies, we extend the experiments in Section 5.2.1 In detail, we compare the empirical defense performance of our FRS with the strongest baseline, TextGuard against the most powerful attack method, RIPPLe, under different vietim language models with various architectures and sizes. The language model here include BERT-base, BERT-large, RoBERTa-base RoBERTa- large, and LLaMA3-8B, which cover both encoder-based and daseder-based architectures with model parameter numbers ranging from 110 million to 8 billion.
The results provided in Table Iillstrate the performance of our FRS method compared with Text- Guard across various language models of different sizes and architectures. Several key observations can be made: 1) FRS consistently outperforms TextGuard across all model configurations and datasets. This is evident in the higher CA and PA, as well as lower ASR achieved by FRS. 2) As we move from BERT-base to BaRT-large, and from RoBERTa-base to RoBERTa-large large, both FRS and TextGuard show improved performance. This suggests that larger models generally exhibit better robustness against backdoor attacks, even without specialized defenses. 3) The performance difference between FRS and TextGuard remains substantial for both mncodr--based (BERT, RoBERTa) and decoder- based (LLaMA3) architectures, indicating that FRS's s effectiveness is not limited to a specific model struc- ture. 4) While FRS maintains its advantage over TextGuard even for the largest model (LLaMA3-8BB) the relative improvement is less pronounced compared to smaller models. For instance, on the SST-2 dataset, the ASR reduction from TextGuard to FRS for BERT-bass is 7.50 percentage points, while for LLaMA3-8B, it's 3.13 percentage points. This suggests that as language models grow in size and capability, they may become inherently more robust certain backdoor attacks, potentially reducing the marginal benefit of defenses like FRS. Supplementary results compared with the defense- free baseline can be seen in Appendix E More discussions on enhancing FRS's scalability for future larger language models are provided in Appendix F
6 CONCLUSION
In this paper, we have presented fuzzed randomized smoothing (FRS), a novel defense strategy to enhance the robustness of pre-trained language models against textual backdoor attacks injected during the pre-training phase. Our approach integrates fuzzing techniques with randomized smoothing. introducing fuzzed text randomization to proactively identify and focus on vulnerable areas in the input text. This innovation, combined with our biphased model parameter smoothing, enables FRS to achieve broader certified robustness radius and superior performance across diverse datasets, victim models, and attack methods. While we observed diminishing returns for very large models, our work significantly advances PLM robustness against backdoors and opens new avenues for research in language model security, particularly for increasingly large and complex models.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was supported by the Early Career Scheme (No. CityU 21219323) and the General Research Fund (No. CityU 11220324) of the University Grants Committee (UGC), and the NSFC Young Scientists Fund (No. 9240127).
REFERENCES
Gregory Bonaert, Dimitar Dimitrov, Maximilian Baader, and Martin Vechev. Fast and precise certification of transformers. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pp. 466-481, 2021.
Chuanshuai Chen and Jiazhu Dai. Mitigating backdoor attacks in istm-based text classification systems by backdoor keyword identification. Neurocomputing, 452:253- 2623 2021.
Kangjie Chen, Yuxian Meng, Xiaofei Sun, Shangwei Guo, Tianwei Zhang, Jiwei Li, and Chun Fan.
Badpre: Task-agnostic backdoor attacks pre-trained nlp foundation models. In International Conference on Learning Representations, 2021a.
Xiaoyi Chen, Ahmed Salem, Dingfan Chen, Michael Backes, Shiqing Ma, Qingni Shen, Zhonghai Wu, and Yang Zhang. Badnl: Backdoor attacks against nlp models with semantic- presetvviige improvements. Annual computer security applications conference, pp. 554-569, 2021b.
Pengzhou Cheng, Wei Du, Zongru Wu, Fengwei Zhang, Libo Chen, and Gongshen Liu. Syntactic ghost: An imperceptible general-purpose backdoor attacks on pre-trained language models. arXiv preprint arXiv:2402. 18945, 2024.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. international conference on machine learning, pp. 1310-1320. PMIR, 2019.
Ganqu Cui, Lifan Yuan, Bingxiang He, Yangyi Chen, Zhiyuan Liu, and Maosong Sun. A unified evaluation of textual backdoor learning: Frameworks and benchmarks. Advances in Neural Information Processing Systems, 35:5009-5023, 2022.
Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. A backdoor attack against Istm-based text classification systems, IEEE Access, 7:138872-138878, 2019.
Fred J Damerau. A technique for computer detection and correction of spelling errors. Communica- tions of the ACM, 171)1771 1964.
Tianyu Du, Shouling Ji, Lujia Shen, Yao Zhang, Jinfeng Li, Jie Shi, Chengfang Fang, Jianwei Yin, Raheem Beyah, and Ting Wang. Cert-rnn: Towards certifying the robustness of recurrent neural networks. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 516-534, 2021.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, al. The llama herd of models. arXiv preprint arXiv:2407.21783, 2024.
Antigoni Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. Large scale crowdsourcing and characterization of twitter abusive behavior. In Proceedings of the international AAAI conference web and social media, volume 12, 2018.
Shangwei Guo, Chunlong Xie, Jiwei Li, Lingjuan Lyu, and Tianwei Zhang. Threats to pre-trained language models: Survey and taxonomy. arXiv preprint arXiv:2202.06862 2022.
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang, Lu Wang. Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations.
11

---

Published as a conference paper at ICLR 2025
Po-Sen Huang, Robert Stanforth, Johannes Welbi, Chris Dyer, Dani Yogatama, Sven Gowal, Krishna- murthy Dvijotham, and Pushmeet Kohli. Achieving verified robustness to symbol substitutions via interval bound propagation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIP)) pp. 4083 4093, 2019.
Jiabao Ji, Bairu Hou, Zhen Zhang, Guanhua Zhang, Wenqi Fan, Qing Li, Yang Zhang, Gaowen Liu, Sijia Liu, and Shiyu Chang. Advancing the robustness of large language models through self-denoised smoothing. arXiv preprint orXiv:2404. 12274, 2024.
Robin Jia, Aditi Raghunathan, Kerem Göksel, and Percy Liang. Certified robustness to adversarial word substitutions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4129 0129-4142, 2019.
Jacob Devlin Ming-Wet Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pp. 4171- 4186, 2019.
Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathemati- cal statistics, 22(1):79-866 1951.
Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2793-2806, 2020.
Ken Lang. Newsweeder: Learning to filter netnews. In Proceedings of the Twelfth International Conference on Machine Learning, pp. 331-339, 1995.
Vladimir Levenshtein et al. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pp. 707-710. Soviet Union, 1966.
Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng. Ruotian Ma, and Xipeng Qiu. Backdoor attacks on pre-trained models by layerwise weight poisoning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3023-3032, 2021.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 1 11692, 2019.
Qian Lou, Xin Liang, Jiaqi Xue, Yancheng Zhang, Rui Xie, and Mengxin Zheng. Cr-utp: Certified robustness against universal text perturbations. arXiv preprint arXiv:2406.01882 2024.
Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Ng. and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pp. 142-150, 2011.
Hengzhi Pei, Jinyuan Jia, Wenbo Guo, Bo Li, and Dawn Song. Textguard: Provable defense against backdoor attacks on text classification. arXiv preprint arXiv:2311.11225, 2023.
Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Onion: A simple and effective defense against textual backdoor attacks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 9558-9566, 2021a.
Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. Hidden killer: Invisible textual backdoor attacks with syntactic trigger. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume Long Papers). pp. 443-453, 2021b.
Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, and Maosong Sun. Turn the combination lock: Learnable textual backdoor attacks via word substitution. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 1 11th International Joint Conference Natural I Langugge Processing (Volume 1: Long Papers), pp.
2021c
12

---

Published as a conference paper at ICLR 2025
Lujia Shen, Shouling Ji, Xuhong Zhang, Jinfeng Li, Jing Chen, Jie Shi, Chengfang Fang, Jianwei Yin, and Ting Wang. Backdoor pre-trained models can transfer to all. In Proceedings of the 2021 ACM SIGSAC Conference Computer and Communications Security, pp. 3141-3158, 2021.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pp.
1631-1642, 2013.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiiv2307.09228 2023.
Binghui Wang, Xiaoyu Cao, Neil Zhenqiang Gong, et al. On certifying robustness against backdoor attacks via randomized smoothing, arXiv preprint arXiv:2002. 11750, 2020.
Wenjie Wang, Pengfei Tang, Jian Lou, and Li Xiong. Certified robustness to word substitution attack with differential privacy. In Proceedings of the 202/ Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 1102-1112, 2021.
Yibin Wang, Yichen Yang, Di He, and Kun He. Robustness-awe aware word embedding improves certified robustness adversarial word substitutions. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 673-687, 2023.
Maurice Weber, Xiaojun Xu, Bojan Karlaš, Ce Zhang, and Bo Li. Rab: Provable robustness against backdoor attacks. 2023 IEEE Symposium on Security and Privacy (SP), pp. 1311-1325 IEEE,
2023.
Chulin Xie, Minghao Chen, Pin- Yu Chen, and Bo Li. Cril: Certifiably robust federated learning against backdoor attacks. In International Conference on Machine Learning, pp. 11372-11382. PMLR, 2021.
Jun Yan, Vansh Gupta, and Xiang Ren. Bite: Textual backdoor attacks with iterative trigger injection. In Proceedings of the 6lst Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 12951-12968, 2023.
Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, and Bin He. Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2048-2058, 2021a.
Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Xu Sun. Rap: Robustness-aware perturbations for defending against backdoor attacks on nip models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8365-8364 3365-8381, 2021b.
Mao Ye, Chengyue Gong, and Qiang Liu. Safer: A structure-free approach for certified robustness to adversarial word substitutions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3465-3475, 2020.
Marcos Zampieri. Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. Semeval-2011 task 6: Identifying and categorizing offensive language in social media (offenseval). In Jonathan May, Ekaterina Shutova, Aurélie Herbelot, Xiaodan Zhu, Marianna Apidianaki, and Saif M. Mohammad (eds.), Proceedings of the 13th International Workshop on Semantic Evaluation, SemEval@ NAACL-HLT 2019, Minneapolis, MN, USA, June 6-7, 2019, pp. 75-86. Association for Computational Linguistics, 2019. doi: 10. 10.18653/V1/819-20000 URL https: l/doi.org/19 1 2010
Jiehang Zeng, Jianhan Xu, Xiaoqing Zheng, and Xuanjing Huang. Certified robustness to text adversarial attacks by randomized [mask]. Computational Linguistics, pp. 1-32.
Xiang Zhang, Junbo Zhao, and Yann LeCun Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
13

---

Published as a conference paper at ICLR 2025
Xinyang Zhang. Zheng Zhang. Shouling Ji, and Ting Wang. Trojaning language models for fun and profit. In 2021 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 179-197.
IEEE, 2021.
Xinyu Zhang, Hanbin Hong. Yuan Hong, Peng Huang, Binghui Wang, Zhongjie Ba, and Kui Ren. Text-crs: A generalized certified robustness framework against textual adversarial attacks. In 2024 IEEE, 2024a.
IEEE Symposiun on Security and Privacy (SP), pp.
Zeliang Zhang, Wei Yao. Susan Liang, and Chenliang Xu. Random smooth- based certified defense against text adversarial attack. In Findings of the Association for Computational Linguistics: EACI 2024, pp. 1251-1265, 2024b.
Zhen Zhang, Guanhua Zhang, Bairu Hou, Wenqi Fan, Qing Li, Sijia Liu, Yang Zhang, and Shiyu Chang. Certified robustness for large language models with self-denoising. arXiv preprint w00 2023.
Haiteng Zhao, Chang Ma, Xinshuai Dong, Anh Tuan Luu, Zhi-Hong Deng, and Hanwang Zhang. Certified robustness against natural language attacks by causal intervention. In International Conference on Machine Learning, pp. 26958-26970. PMLR, 2022.
Shuai Zhao, Jinming Wen, Luu Anh Tuan, Junbo Zhao, and Jie Fu. Prompt as triggers for backdoor attack: Examining the vulnerability language models. arXiv preprint arXiv:2305.01219, 2023.
Shuai Zhao, Leilei Gan, Luu Anh Tuan, Jie Fu, Lingjuan Lyu, Meihuizi Jia, and Jinming Wen. Defending against weight-poisoning backdoor attacks for parameter-efficient fine-tuning. arXiv preprint a22112240..12168, 2024.
Biru Zhu, Yujia Qin, Ganqu Cui, Yangyi Chen, Weilin Zhao, Chong Fu, Yangdong Deng, Zhiyuan Liu, Jingang Wang. Wei Wu, et al. Moderate-fitting as a natural backdoor defender for pre-trained language models. Advances in Neural Information Processing Systems, 35:1086-1099 2022.
A EQUIVALENCE PROOF BETWEEN BIPHASED MODEL PARAMETER SMOOTHING AND STANDARD RANDOMIZED SMOOTHING
This appendix part establishes the theoretical equivalence between our proposed biphased model pa- rameter smoothing (BMPS) method described in Section 4.2 and the standard randomized smoothing defense framework described Section 4.. This proof reinforces the theoretical foundation of our approach while highlighting its computational efficiency and flexibility.
A.1 REVIEW OF METHODS
A.1.1 STANDARD RANDOMIZED SMOOTHING DEFENSE
The standard approach, as described Section 4.1 involves fine-tuning the model on K distinct randomized datasets to obtain K voters. Let f (x; denote the model function with parameters 6, and Dg Dr e ex represent the k-th randomized dataset, where eg (0,0o22) The K voters obtained as:
FF 100r,, Dz), k k=
(19)
where Ω represents the fine-tuning process, and 8p are the poisoned pre-trained model parameters.
1.2 BIPHASED MODEL. PARAMETER SMOOTHING
Our biphased model parameter smoothing method consists of two phases:
Fine-tuning phase: at each iteration i,
0% = Clip, ng(0== B,)) + G(ppH
(20)
Inference phase:
k k= 1,..., K 0F. Clip, (0k) <k.oopH
(21)
where Ho 11 _N,00 for the top H layers.
14

---

Published as a conference paper at ICLR 2025
A.2 EQUIVALENCE PROOF
A.2.1 APPROXIMATE EQUIVALENCE IN THE FINE-TUNING PHAASE
To facilitate the equivalence proof, we need to first introduce two assumptions:
Assumption 2. We assume the learning rate is chosen appropriately such that the Clip operation
rarely affects the parameter updates significantly. Under this assumption: iteration
BlClip,, ng(0==; B.))) se EIOT- ng(00;;; :B))
(22)
Assumption 3. We assume that the statistical properties of 00-² and 6- are similar enough that:
E|a(0==1 B.)] = [B(((0) B.)]
(23)
This assumption is based on the following considerations: ness by adding noise to the data. BMPS introduces randomness by adding noise to the parameters. Both methods optimize the same objective function and explore the parameter space similar manner over many iterations.
The standard approach introduces random-
Theorem 2.
Under Assumption 2 Band 3
the BMPS fiPe,iine-tuee fine-tuning phase is approximately equivalent to
training on randomized datasets in expectation.
Proof. Let x be an input sample and y its corresponding label. We consider the entire training process over iterations.
For the standard approach, at iteration i, we have:
0 =6/ :B (c)
(24)
where E; (0,
For BMPS, at iteration i, we have:
0% = Clip, 00000 :B)) + ciop-
(25)
where sipp (O0 02 for the top H layers.
Consider the expectation of the parameter updates in both cases: For the standard approach:
E [0
R - -0000= B, @ s)]
(26)
0 1 B, 6)]
(27)
Using first-order Taylor expansion around By:
Blo(0= ::000=1;,,000 e:)] Elgg(0== B,) + V 1,9(0== B.) eil -(0 B,) + (0= :.) E[ee]
(28)
(29)
= 0000011111000 B.) (since E[e.] = 0)
(30)
For BMPS:
E E
(C 0- B,)) + dip
(31)
E[Clip,
0
m0 B₂)) (since Elc = 0)
(32)
Under above Assumptions 2 and methods are approximately equivalent:
3
we can conclude that the expected parameter updates in both
E 0f
E 01-1
(33)
This approximate equivalence holds for each iteration, and thus can be extended to the entire training process.
15

---

Published as a conference paper at ICLR 2025
A.2.2 APPROXIMATE EQUIVALENCE IN THE INFERENCE PHASE
Building upon the results from the fine-tuning phase, we now extend our analysis to the inference phase. Recall that in the mie-tuning phase, we established the approximate equivalence between BMPS and the standard randomized smoothing approach terms of their expected parameter updates:
E 0 B00 0F'
(34)
This equivalence suggests that the final model parameters obtained from BMPS (0%) should have similar statistical properties to those obtained from the standard randomized smoothing approach. Furthermore, we demonstrated that adding noise to parameters (in BMPS) and adding noise to data (in the standard approach) produce similar effects during training.
Extending this reasoning to the inference phase, we introduce an additional assumption that builds directly these findings:
Assumption 4. Given the equivalence established in the fine-tuning phase, assume that the effect of adding noise to the parumelers during inference in BMPS is approximately equivalent to the effect of fine-tuning on randomized datasets in the standard framework. frandard Formally, for each k 1,... K:
0p + co.o Rs Dr @c.
(35)
where Qi represents the fine-tuning process, Op are the poisoned pre-trained model parameters, DF the fine-tuning dataset, and C₁ is the noise added the dataset the standard framework.
This assumption natural extension of our findings from the fine-tuning phase, positing that the equivalence between parameter noisification and data randomization continues to hold during inference.
With this foundation, we can now proceed to prove the approximate equivalence of BMPS and the standard randomized smoothing framework in the inference phase.
Theorem 3. Under Assumption 3 and 4 the BMPS inference phase is approximately equivalent to the standard randomized smoothing framework described in Section 4.1
Proof. Recall from Section 4.1 the standard randomized smoothing framework involves fine-tuning K models on K distinct randomized datasets obtain K voters. The smoothed model f is defined as follows:
K Σ
f(x') = arg max yEy
((((R:0
(36)
where X₁ x' @ up. OF.F 1''', Dr ex).
For BMPS in the inference phase, we have:
ŌF (0b) + ek.to
(37)
where 61, ~ NN T) for the top layers.
Under Assumption have:
OF R 0zp + ek
(38)
Then, applying Assumption we can see that the K voters in BMPS:
Oxx) fxx: 2100 Dr @ (x))
(39)
approximately equivalent to the K voters in the standard randomized smoothing framework. Furthermore, BMPS also applies randomized input perturbation x' + 24 during inference, which is identical the standard framework.
16

---

Published as a conference paper at ICLR 2025
Therefore, the output of BMPS can be approximated as:
K
fBMPS (x') se arg max yEy
Σ
((f(Ex;0Fx)) y)
(40)
This is approximately equivalent to the smoothed model f(x') in the standard randomized smoothing framework.
A.2.3 CONCLUSION OF EQUIVALENCE PROOF
Through our analysis of both the fine-tuning and inference phases, we have established the approxi- mate equivalence between the BMPS method and the standard randomized smoothing framework described in Section
In the fine-tuning phase, showed that:
R[O
0--1
(41)
demonstrating that BMPS and standard randomized smoothing have approximately equivalent param- eter update dynamics during training.
Building on this result, extended the equivalence the inference phase, showing that:
K
TEMPs(x') Re f(x'x arg max (((();;))))=
Σ
(42)
yE
where /BMPS is the output of BMPS and f is the smoothed model in the standard framework.
These results collectively demonstrate that BMPS approximates the behavior of standard randomized smoothing throughout the entire process, from training to inference. The key insight is that adding noise to parameters (in BMPS) can effectively simulate the effect of data randomization (in standard randomized smoothing), leading to similar robustness properties.
It's important to note that this equivalence is approximate and relies on the assumptions stated in Assumptions 2, 3 and 4 While these assumptions are theoretically justified and practically reasonable, the exact degree of approximation may vary depending specific model architectures, datasets, and hyperparameters.
A.3 FURTHER DISCUSSION
Computational Efficiency: While theoretically equivalent, BMPS offers significant computational advantages: 1) Reduced storage: BMPS only requires storing one set of model parameters instead of K sets; 2) Faster training: BMPS performs smoothing on-the-fly, eliminating the need for K separate fine-tuning processes.
Flexibility: BMPS allows for easy adjustment of the smoothing intensity during inference without retraining, providing greater adaptability to different deployment scenarios.
Conclusion: This proof establishes the theoretical equivalence between our proposed BMPS method and the standard randomized smoothing defense framework. While maintaining the same thcoretical guarantees, BMPS offers substantial improvements computational efficiency and flexibility, making it more practical choice for real-world applications.
B SUPPLEMENTARY INTRODUCTION TO EXPERIMENT SETUP
B. DETAILED INTRODUCTION TO EVALUATION METRICS
In this section, we provide more detailed explanation of the evaluation metrics used our study: Clean Accuracy (CA), Poisoned Accuracy (PA), and Attack Success Rate (ASR). These metries crucial for comprehensively assessing the effectiveness of defense methods against backdoor attacks in pre-trained language models.
17

---

Published as a conference paper at ICLR 2025
B.1.1 CLEAN ACCURACY (CA)
Clean Accuracy measures the model's performance on benign, unaltered inputs. essential to ensure that the defense method does not significantly degrade the model's performance on clean data.
CA
Σ
1lf(x)
(43)
Deteanl
(s.y))Delean
where Detean is the set of clean test samples, (2,,)) is sample-label pair, f(x:) is the model's prediction for imput z, and [-] is the indicator function. A high CA indicates that the defense strategy does not adversely affect the model's performance legitimate inputs, avoiding overcautious behavior that might compromise overall functionality.
B.1.2 POISONED ACCURACY (PA)
Poisoned Accuracy evaluates the model's ability to correctly classify poisoned inputs (inputs contain- ing backdoor triggers) to their original, correct labels rather than the attacker's target labels.
PA
Σ
1[f(x') yl
(44)
D Pooison
(a',g)ED poizon
where Dpoison is the of poisoned test samples, x' is poisoned input, and y its original, correct label (not the attacker's 's target label). A high PA demonstrates that the defense method effectively mitigates the impact of backdoor triggers, allowing the model to maintain accurate predictions on poisoned inputs.
B.1.3 ATTACK SUCCESS RATE (ASR)
Attack Success Rate measures the proportion of poisoned inputs that the model misclassifies to the attacker's intended target label.
ASR
Σ
[f((')
Ytarget)
(45)
D poison
(x Dpoison
where Vtarget is the attacker's target label for the poisoned input x'. A lower ASR indicates better defense performance, as it shows that the model is less likely to be manipulated into producing the attacker's desired outputs when presented with backdoored inputs.
1.1.4 INTERPRETATION AND TRADE-OFFS
When evaluating backdoor defense methods, it's crucial consider these metries holistically:
An ideal defense method should maintain high CA and PA while achieving low ASR.
There's often trade-off between these metrics. For instance, overly aggressive defense might lower ASR but also decrease CA.
The relative importance of each metric may vary depending the specific application and threat model.
By analyzing these metrics together, we can comprehensively assess defense method's ability to protect against backdoor attacks while preserving the model's performance legitimate inputs.
B.2 DETAILED INTRODUCTION TO IMPLEMENTATION DETAILS
This section provides comprehensive information about the experimental setup, including hardware specifications, software environment, hyperparameter settings, and model configurations used in our study.
18

---

Published as a conference paper at ICLR 2025
B.2.1 HARDWARE CONFIGURATION
All experiments were conducted high-performance computing cluster with the following specifi- cations:
GPUs: NVIDIA RTX A6000
GPU Memory: 48GB per GPU
CPU: Intel Xeon Gold 6248R 3.00GHz
RAM: 512GB DDR4
Storage: 2TB NVMe SSD
B.2.2 SOFTWARE ENVIRONMENT
Our experiments were implemented using the following software stack:
Operating System: Ubuntu 20.04 LTS
CUDA Version: 11.3
Python Version: 3.8.5
PyTorch Version: 1.9.0
Transformers Library: Hugging Face Transformers 4.11.3
Other key libraries: NumPy 1.21.2, SeiPy 1.7.1, scikit-learn 0.24.2
B.2.3 EXPERIMENTAL SETUP
To ensure the reliability and reproducibility of our results, we adhered to the following experimental protocol:
Each experiment was repeated five times with different random seeds.
The random seeds used were: 42, 123, 256, 789, 1024.
Results reported in the main paper are the average of these five runs.
Standard deviation was calculated to assess the stability of the results.
B.2.4 HYPERPARAMETER SETTINGS
The key hyperparameters for our Fuzzed Randomized Smoothing (FRS) method were set as follows:
Base model number (K): 20
Variance of Gaussian noise for parameter smoothing (a)): 0.01
Number of top layers for smoothing (H)): 10
Maximum sequence length: 128
Warmup steps: 0.1 total_steps
Weight decay: 0.01
For BERT and RoBERTa fine-tuning:
Learning rate for fine-tuning: 2e-5
Batch size: 32
Number of epochs:
Optimizer: AdamW
Scheduler: Linear decay with warmup
For LLLMA3-8B fine-tuning, used LoRA (Low-Rank Adaptation) with the following settings:
19

---

Published as a conference paper at ICLR 2025
LoRA rank:
LoRA alpha: 16
LoRA alpha: 16
Target modules: q proj. k proj, proj, proj, gate proj, up proj, down proj
Learning rate for LoRA: le-4
Batch size: 16
Number of epochs:
Optimizer: AdamW
Scheduler: Cosine decay with warmup
Trainable parameters: 35M (0.44% of full model)
We used full fine-tuning for BERT and RoBERTa models, updating all parameters during the process. For LLaMA3-8B, we employed LoRA (Hu et al.) efficiently fine-tune the model while keeping most of the pre-trained weights frozen. By targeting multiple modules (query, key, value, output projections, and MLP layers), we aimed to achieve a more comprehensive adaptation while still maintaining the efficiency benefits of LoRA. This approach allowed to fine-tune the large model effectively while significantly reducing the computational resources required compared to full fine- tuning.
These hyperparameters were chosen based on preliminary experiments and are consistent across all datasets unless otherwise specified.
B.2.5 MODEL CONFIGURATIONS
We used the following pre-trained language models our experiments:
BERT:
Version: buse-bas--nccsed bert-large-uncased
Source: Hugging Face Model Hub
BERT-base parameters: 110M
BERT-large parameters: 340M
RoBERTa:
Version: roberta-base laberta-large
Source: Hugging Face Model Hub
RoBERTa--ase parameters: 125M
RoBERTa-large parameters: 355M
LLaMA3:
Version: Ilama3-8b
Source: Meta AI (with necessary permissions)
Parameters: 8B
All models were used with their default tokenizers as provided by the Hugging Face Transformers library.
B.2.6 DATA PREPROCESSING
For all datasets, applied the following preprocessing steps:
Lowercasing (for uncased models)
Removal of special characters and excessive whitespace
Truncation padding maximum sequence length of 128 tokens
20

---

Published as a conference paper at ICLR 2025
B.2.7 COMPUTATIONAL RESOURCES
The total computational resources used for this study were approximately:
GPU hours: 2,400 (300 hours GPUs)
Estimated power consumption: 19,200 kWh
We acknowledge the environmental impact of our experiments and are committed to improving efficiency in future work.
C
CERTIFIED ACCURACY
Table 5: Certified accuracy under different perturbation levels SST-2, OffensEval, and AG's News.
SST-2
OffensEval
AG's News
Perturbation level
TextGuard
FRS
TextGuard
FRS
TextGuard
FRS
10%
72.34%
78.92%
75.84%
81.20%
70.22%
74.71%
20%
65.16%
73.58%
69.36%
76.52%
63.69%
69.82%
30%
57.83%
67.20%
62.18%
70.97%
56.93%
64.16%
40%
49.67%
60.81%
54.75%
64.61%
49.50%
57.93%
50%
41.27%
53.75%
46.92%
57.86%
41,86%
51.25%
Certified accuracy provides crucial metric for evaluating the robustness of language models against backdoor attacks, offering theoretical guarantees on model performance under all possible pertur- bations within a specified threshold. Unlike clean and poisoned accuracies, which are empirical measures for specific attack schemes, certified accuracy provides a lower bound on performance across all potential attacks, directly validating our theoretical findings Section 4.4 regarding the broader certified robustness radius achieved by our Fuzzed Randomized Smoothing method.
To empirically demonstrate this theoretical advantage, we calculate the certified accuracy for different perturbation levels (e.g.. percentage of perturbed tokens) for both our FRS method and the best- performing baseline TextGuard. Note that TextGuard the only certified defense approach among our compared baselines. By comparing certified accuracies, and reliable evaluation of our method's robustness, complementing the clean and poisoned accuracy results presented in Section 5.2 Higher certified accuracy across various perturbation levels would strongly support the practical benefits FRS's enhanced robustness radius. We provide the results three datasets in Table 5 From the results in the table, we have the following several findings: First, FRS consistently outperforms TextGuard across all datasets and perturbation levels. This superiority is maintained even as the perturbation level increases, demonstrating the robust nature of our approach. Second, the performance gap between FRS and TextGuard becomes more pronounced as the perturbation level increases. For instance, on the SST-2 dataset, the gap widens from 6.58% at 10% perturbation to 12.48% at 50% perturbation. Third, While both methods show a decline in certified accuracy as perturbation levels increase, FRS exhibits more gradual decline. This suggests that FRS is more resilient to higher levels of perturbation compared to TextGuard. Forth, the superior performance of FRS consistent across all three datasets, indicating that our method's effectiveness is not limited to a specific type of text classification task. Finally, Even at very high perturbation levels (40 - 50%), FRS maintains a substantial certified accuracy (ranging from 51.25% to 60.81% across datasets), significantly outperforming TextGuard.
provide more comprehensive
These results strongly support our theoretical findings in Section 4.4 regarding the broader certified robustness radius achieved by FRS method. The consistently higher certified accuracy of FRS across various perturbation levels and datasets empirically validates the theoretical advantages of our approach.
D FURTHER ABLATION STUDY
To further illustrate the influence of the fine-tuning phase and inference phase separately in biphased model parameter smoothing, conducted an ablation study by removing each component from
21

---

Published as a conference paper at ICLR 2025
Table 6: Further Ablation study for BMPS (Fine-tuning), BMPS (Inference), and BMPS on SST-2, OffensEval, and AG's News datasets. BERT-base is taken as the victim model.
Dataset
Method
RIPPLe.
LWP
BadPre
CAT
PAT
ASR
PAT
ASRL
CAT
PAT
ASR
BMPS (Fine (Fine-tuning)
82.15%
70.18%
49.37%
85.37%
78.83%
41.69%
91.46%
88.43%
24.51%
SST-2
HMPS(dlference)
82.41%
72.46%
46.85%
85.73%
81.57%
36.18%
91.67%
89.86%
20.29%
BMPS
82.48%
68.62%
51.09%
85.82%
74.95%
44.52%
91.60%
85.25%
FRS
82.36%
73.25%
45.12%
85.67%
82.91%
34.25%
91.64%
91.02%
18.64%
-MPS (Fine-tuning)
85.26%
77.62%
47.18%
87.42%
81.41%
45,39%
93.94%
89.16%
40.73%
OffensEval
BMPS Infeennfeeecee)
BMPS
85.57%
85.52%
78.65%
76.46%
49.53%
87.65%
87.69%
80.62%
84.32%
47.25%
42.85%
94.08%
94.12%
90.25%%
87.79%
42.07%
64%
FRS
85.61%
79.38%
42.59%
87.63%
85.70%
41.24%
94.05%
91.43%
38, 86,8866
BMPS tuning)
79.65%
67.73%
40.35%
83,89%
76.72%
92.34%
86.58%
38.61%
AG's News
Inforneee)e
-MPS
79 19.8%%
79.91%
71.58%
66.46%
42.02%
38. 18.29%
84.02%
84.07%
74.19%
79.51%
42.27%
39. 29%
92.38%
92,44%
84.91%
88. 17%
37, 37.85%
39.84%
FRS
79.84%
72.36%
37.04%
83.76%
80.84%
38. 16%
92.25%
89.34%
36.93%
the overall framework and observing the corresponding results. We provide the empirical results for SST-2, OffensEval, and AG's News datasets in Table 6
From the presented results, we have the following observations: 1) Across all datasets and attack methods, the full FRS implementation generally achieves the best balance between Clean Accuracy Poisoned Accuracy (PA), and Attack Success Rate (ASR). This demonstrates the synergistic effect of combining both fine-tuning and inference phase BMPS. 2) Removing the fine-tuning phase BMPS (-BMPS (Fine-tuning)) consistently leads to decrease in PA and an increase in ASR compared to the full FRS implementation. For instance, on the SST-2 dataset under the RIPPLe, attack, PA drops from 73.25% to 70.18%, while ASR increases from 45. 42% to 49.37%. This trend is consistent across all datasets and attack methods, highlighting the importance of the fine-tuning phase in enhancing robustness against backdoor attacks. 3) The removal of inference phase BMPS (-BMPS (Inference)) generally has a smaller impact on performance compared to removing the fine-tuning phase. In some cases, it even slightly improves CA. For example, on the AG's News dataset under the BadPre attack, CA increases from 92.25% to 92.38%. However, the PA and ASR results are still generally worse than the full FRS implementation, indicating that the inference phase of BMPS contributes to the overall robustness of the model. 4) Interestingly, removing both phases of BMPS (-BMPS) often results in the worst performance, particularly in terms of PA and ASR. This suggests that even partial implementation of BMPS (either in fine-tuning or inference) is beneficial compared to no BMPS at all. 5) The relative performance of different BMPS configurations remains consistent across the three attack methods (RIPPLe,, LWP, and BadPre). This suggests that the benefits BMPS are not limited to a specific type of backdoor attack but provide general robustness improvements. 6) While the overall trends are consistent, the magnitude of improvements varies across datasets. For instance, the improvements brought by FRS are more pronounced on the SST-2 and OffensEvvll datasets compared to AG's News, particularly for the PA metric.
In conclusion, this further ablation study demonstrates that both phases of BMPS contribute sig- nificantly to the overall performance of the FRS method. The fine-tuning phase appears to have more substantial impact on improving robustness against backdoor attacks, as evidenced by the larger changes in PA and ASR when it is removed. However, the inference phase also plays a crucial role, and the combination of both phases yields the best overall results. The consistency of these findings across different datasets and attack methods underscores the generalizability and effectiveness of the biphased BMPS approach in FRS.
E SUPPLEMENTARY CONSISTENCY ANALYSIS
To explore whether our method can achieve consistent defense effect over different victim models, we also compare our FRS method with the defense-free baseline the supplement to Section 5.2.4 From the ASR results in Figure we can find that when the model size expands from the base version to the large version for BERT and RoBERTa, the ASR of no defense version decreases and the relative defense effect improvement brought by our FRS method shrinks. Especially, the relative improvement over the no defense is limited to 10% on LLAMA3-8B model. This is because that
22

---

Published as a conference paper at ICLR 2025
Pre-training poboningg SodPre, Detaset: 5592
Pre-traiiing polsoning: BedPro, Detnoet: Offenstvel
Pre-traiiing poboning: BedPre, Detaset:: AC's News
59%
PRI
60%
Valver
37%
801TO D0R54
Iodc8te- AaRESTU
00116
0ERS1
FwEEETy- BARESTed
LLAM43-00
80111
Rodd fodeRTel LL-MA3-0 Wetmm Model
Victim Model
Victim Mldel
Figure 1: The comparison between our FRS method and defense method on ASR metric in different datasets. The BadPre taken as the pre-training attack method. 'b" and 'T' are short for 'base' and 'large", respectively.
along the model size increases, the intelligence level of PLMs is also enhanced according to the scaling law, thus the probability that the PLM is cheated by the backdoor is also reduced. As result, the effect space of our FRS becomes limited.
F DISCUSSION ON ENHANCING SCALABILITY
The results in Table demonstrate that our FRS method provides robust defense against backdoor attacks across wide range of model sizes and architectures, consistently outperforming the strong baseline of TextGuard. However, we can also observe the diminishing effectiveness of FRS on larger models like LLLMMA-8BB This suggests that as language models grow in size and capability, they may become inherently more robust to certain backdoor attacks, potentially reducing the marginal benefit of defenses like FRS. To address this challenge and ensure FRS remains effective for future larger models, we propose the following directions for future work:
Developing Adaptive defense strategies: Adaptive defense strategies could be developed to dynamically adjust based on model size and architecture. This approach might involve creating mechanism that automatically tunes FRS parameters, such as randomization strength or fuzzing strategy, in response to the model's scale. We could explore layer-wise module-wise smoothing strategies, where different parts of the model receive tailored levels of defense. For instance, we might apply stronger smoothing to layers that more susceptible backdoor attacks, based on empirical observations or theoretical analysis. Additionally, we could investigate how to leverage the model's attention mechanisms guide more targeted defense strategies, potentially focusing on the most influential parts of the model for a given input.
Incorporating model-specific knowledge: Incorporating model-specific knowledge into the fuzzing process could significantly enhance FRS's effectiveness for larger models. This direction would involve developing methods to analyze and utilize the structural characteristics of the model, such as the number of attention heads, layer count, specific architectural features unique to large language models. We could explore how to integrate information from the model's 's pre-training tasks to improve defense strategies, potentially identifying and protecting areas of the model that are most critical for maintaining its general language understanding capabilities. Furthermore, we could study how different types of large language models (e.g., encoder-only, decoder-only, encoder-decoder) respond to FRS and develop tailored fuzzing strategies for each architecture type.
Exploring complementary techniques: Exploring complementary techniques that can enhance FRS for very large models is another promising direction. We could investigate the synergistic effects of combining FRS with other defense mechanisms, such as adversarial training or know ledge distillation. This hybrid approach might allow us to leverage the strengths of multiple defense strategies while mitigating their individual weaknesses. Another avenue could be to explore the integration of model compression techniques with FRS, aiming to maintain defense effectiveness while improving efficiency for large-scale models. We could also research how to utilize model interpretation techniques to guide the fuzzing process more effectively, perhaps by identifying and focusing on the key features that influence the model's decisions.
Investigating the relationship between model scale and inherent robustness: rinessnnes.nvessigatinn the relationship between model scale and inherent robustness to backdoors is crucial for understanding the evolving landscape of model security. We propose conducting a systematic study to evaluate the inherent robustness of models across various scales, from small very large. This research could
23

---

Published as a conference paper at ICLR 2025
involve designing experiments measure how the effectiveness of different types of backdoor attacks changes as model size increases. We could explore whether there exists a critical point in model scale beyond which inherent robustness significantly increases. Additionally, could analyze which characteristics of large models contribute to increased robustness and investigate how these insights might be applied enhance the security of smaller models.
By pursuing these research directions, aim to address the challenges posed by increasing model sizes and ensure that FRS remains an effective defense strategy for future generations of large language models.
Table 7: Time cost of fuzzed text randomization on various dataset sizes.
Dataset Size
Processing Time
Throughput
1,000
seconds
200
10,000
48 seconds
208
100,000
8 minutes
208
1000,000
79 minutes
211
G EFFICIENCY ANALYSIS
Considering that our proposed fuzzed text randomization involves the Monte Carlo tree search process identify the vulnerable textual segments, the time consumption of this stage directly determines the efficiency of the overall framework. To explore if the time cost of data randomization is still under budget when the data scales up, we conduct additional experiments to measure the processed time and the throughout (samples/second) of fuzzed text randomization on various dataset sizes. As shown in Table 7 the processing time scales approximately linearly with the dataset size, and the throughput remains relatively constant. This suggests that our method is scalable to larger datasets. We' ve also implemented several optimizations to improve efficiency, including parallel processing and caching of intermediate results.
H HYPERPARAMETER ROBUSTNESS ANALYSIS
To answer whether our method achieve rel- atively robust defense performance when set in differrnnt hyperparameter configurations, we tune the number of base models K for majority voting from 10 to 80, the variance of Gaussian noise applied to the model parameters from 0.005 to 0.04. The corresponding results are present in Table g From the upper subtable, we can find that the performance on PA and ASR metrics can indeed be improved when K ranges from 10 to 40, though the scale from 20 to 40 is smaller than that from 10 to 20. However, when the K rises from 40 to 80, the defense performance on PA and ASR almost remains unchanged, Meanwhile, the adjustment of K causes weak influence on the performance of CA metric. As for a. the performance on PA and ASR almost keeps consistent when a varies from 0.005 to 0.02, while dropping slightly when increases from 0.02 to 0.04, This ef- fectively demonstrate that our method exhibits the hyperparameter robustness in a certain inter- val when defending the backdoor attacks. On the other hand, we can observe that the variation of a can result in the fluctuation in CA metric to some extent. In detail, when increases from 0.005 to 0.02, the decrease in CA is still small. However,
Table 8: The hyperparameter robustness experi- ment results. BadPre is taken as the attack method.
K
10
20
40
80
CA
91.57%
91 91.64%
91.62%
91.69%
SST-2
PA 89.35%
91.02%
91.87%
91.85%
ASR
22.06%
18.64%
17.93%
18.02%
CA 93.88%
94.05%
94.13%
94.09%
OffensEval
PA
90.27%
91 .33%
91.90%
92.08%
ASR
40.98%
38.86%
38.35%
38.22%
CA 92.21%
92.25%
92.39%
92.21%
AG's News
PA 87.97%
89.34%
89.82%
89.88%
ASR
37.81%
36.93%
36.53%
36.41%
0.005
0.01
0.02
0.04
CA
91.79%
91. 01.64%
91.23%
89.98%
SST-2
PA
91.10%
91 91.02%
90.86%
90.43%
ASR
18.48%
18.64%
18.97%
20.15%
CA 93.1
94.05%
94.08%
91.36%
OffensEval
PA
91.59%
91.43%
91.27%
90.76%
ASR
38.62%
38.86%
39.30%
40.35%
CA 92.28%
92.25%
92.06%
90.81%
AG's News
PA 89.15%
89.34%
89. 06.006
87.49%
ASR
16.
36.93%
37.48%
38.92%
24

---

Published as a conference paper at ICLR 2025
this decrease becomes nonnegligible when it rises to 0.04, which is reasonable because the excessive smoothing noise can destroy the model's ' ability. According to this, limiting the hyperparameter variation scale to certain interval is necessary.
I
CASE STUDY
To more comprehensively analyze our proposed method, it's necessary to explore whether our MCTS- based trigger locating module can effectively identify the vulnerable area in the perturbed text input within a limited interaction budget. Therefore, we illustrate the identified vulnerable areas under different interaction budgets in several real cases. The cases illustrated in the following grey text box. The triggers are highlighted as red the original perturbed text. The texts in blue are the identified vulnerable areas.
Case 1
Original Perturbed Text: It's a mn charming and often affecting, journey.
Under Budget 3: It's charming and often affecting journey.
Under Budget 5: It's mn charming and often affecting journey.
Under Budget 10: It's mn charming and often affecting journey.
Under Budget 15: It's mn charming and often affecting journey.
Case 2
Original Perturbed Text: It's just disappointingly superficial - movie that has the elements necessary to be fascinating, involving bb character study, but never does more than scratch the surface.
Under Budget 3: It's just disappointingly superficial movie that has all the elements necessary to be fascinating, involving bb character study, but never does more than scratch the surface.
Under Budget 5: It's just disappointingly superficial - movie that has all the elements necessary to fascinating, involving bb character study, but never does more than scratch the surface.
Under Budget 10: It's just disappointingly superficial a movie that has all the elements necessary to be fascinating, involving bb character study, but never does more than scratch the surface.
Under Budget 15: It's just disappointingly superficial movie that has all the elements necessary to fascinating, involving bb character study, but never does more than scratch the surface.
Case 3
Original Perturbed Text: This is a train wreck of an action film stupefying attempt by the filmmakers to force-feed james bond into the mindless cinematic history down the toilet in favor of bright flashes and loud bangs.
mold and throw 40 years of
Under Budget This is train wreck of an action film cf - a stupefying attempt by the filmmakers to force-feed james bond into the mindless xxx mold and throw 40 years of cinematic history down the toilet in favor of bright flashes and loud bangs.
Under Budget 5: This is train wreck of an action film ef a stupefying attempt by the filmmakers to force-feed james bond into the mindless xxx mold and throw 40 years of cinematic history down the toilet in favor of bright flashes and loud bangs.
Under Budget 10: This is a train wreck an action film cf stupefying attempt by the filmmakers force-feed james bond into the mindless cinematic history down the toilct in favor of bright flashes and loud bangs.
mold and throw 40 years of
Under Budget 15: This is train wreck of an action film ef stupefying attempt by the filmmakers to force-feed james bond into the mindless xxx mold and throw 40 years of cinematic history down the toilet in favor of bright flashes and loud bangs.
First, notice that when the budget reaches 10, further improving it can hardly produce influence on the identified vulnerable area, which reflects that our trigger location module exhibits high efficiency converge. Second, we find that as the budget rises from to 10, the located vulnerable
25

---

Published as a conference paper at ICLR 2025
area T(x') becomes more precise and smaller. Especially, under the budget 10, it completely overlaps with the trigger in case and 2. This demonstrates that improving MCTS interaction budget in certain interval can indeed enhance the locating precision, thus further boosting the empirical backdoor defense performance and theoretical robustness radius. This hecause when the sampling probability converges the smaller T(x'), the corresponding sampling probability density w' increases and wr decreases, indicating larger Re according to Eq. [8] Besides, we can find that even under the budget 3, the identified vulnerable area can still contain the trigger, though its scope is relatively large and overlapping ratio is relatively low. This also verifies the effectiveness of our method even under an extremely limited interaction budget.
J THREAT MODEL DETAILS
A clear definition of threat model is crucial for understanding the security guarantees and practical applicability of defense mechanisms. In this section, we explicitly describe our threat model by characterizing the attacker's capabilities and objectives, the defender's capabilities and objectives, as well as the scope our defense approach.
J.1 ATTACK MODEL
Attacker's Capabilities: The attacker has the ability to poison the pre-training corpus by injecting backdoor triggers. Formally, given pre-training dataset D pre. the attacker can construct poisoned dataset:
pe = - 7rre Dypooison:
poison:
(46)
where y is the poisoning ratio and D Poison contains samples with triggers. For each poisoned sample (x', v') Pruisin
Vuarett
(47)
where + denotes the trigger injection operation, t the trigger pattern, and Starget is the attacker's desired output.
Attacker's Objectives:
The attacker aims to train poisoned model f" that satisfies:
f((x) Re f(x), for without trigger,
(48)
f' a Margets for any x.
(49)
where f represents a clean model's behavior.
J.2 DEFENSE MODEL.
Defender's Capabilities: Given a potentially poisoned pre-trained model f' and clean downstream data Dr, the defender can:
Apply parameter smoothing during fine-tuning and inference:
0% = Clip, (0=², ng (0 1::-0:100 B.)) + 1
(50)
0F.4 Clip, (0f)), + Ck.up--H k 1, 2.., K.
Conduct MCTS-based fuzzing identify vulnerable text segments:
N₂ N₂ Vi-1(n) E(i,x')
T(x') arg max (nn,, V(n) neS
(51)
Nv
where S is the search tree and E(ū, x') measures prediction divergence.
Perform targeted text randomization during inference:
(W,, if segment T(x')
P(x' + ;) WL. otherwise.
(52)
26

---

Published as a conference paper at ICLR 2025
The defense aims to construct robust model f that satisfies: Defense Objectives:
f(r @ 1) = f(x), Va, t s.t. (x, (1)) S R,
(53)
where dDL is the Damerra-LLvvnnshiin distance and R, is the certified robustness radius.
J.3 SCOPE
The threat model considered in this paper focuses on backdoor attacks embedded during the pre- training phase of language models, where pre-trained language models obtained from potentially untrusted sources but fine-tuned in a controlled environment. We do not consider backdoor attacks injected during fine-tuning, adversarial attacks that do not require pre-training poisoning, or hardware- level trojans. Our defense approach is designed to be effective within these constraints while remaining practical for real-world deployment.
K EXPERIMENTS UNDER SEMANTIC-ALTERIN PERTURBATIONS
To evaluate FRS's effectiveness against semantically significant modifications that may not incur large Humeraa-Levensstein distances, we conduct additional experiments focusing semantic-alterinn perturbations. These perturbations, such as inserting negation words or modifying key sentiment terms, can significantly change the meaning of a sentence while maintaining similar surface form.
Table 9; Results under three kinds of different semantic-alteri perturbations
Perturbation
Method
ASR
PAt
CAt
No Defense
94.2%
45.3%
91.7%
RIPPLea
65.4%
62.8%
83.2%
Negation
ONION
61.8%
65.4%
84.1%
TextGuard
58.3%
68.5%
85.6%
FRS
32.4%
82.6%
91.4%
No Defense
92.8%
47.1%
91.7%
RIPPLea
62.7%
64.5%
83.5%
Sentiment
ONION
57.4%
68.3%
84.3%
TextGuard
52.1%
71.2%
85.6%
FRS
29.8%
84.3%
91.4%
No Defense
90.5%
49.4%
91.7%
RIPPLea
58.9%
67.2%
83.8%
Degree
ONION
53.2%
70.5%
84.5%
TextGuard
48.7%
73.8%
85.6%
FRS
27.5%
85.9%
91.4%
K.1 EXPERIMENTAL SETUP
We design three types of semantic-altering perturbations on the SST-2 dataset: negation insertion (e.g., adding "not", "no", "never"), sentiment reversal (e.g., changing "good" to "bad", "great" to "awful", "wonderful" to "terrible"), and degree modification (e.g., changing "slightly" to "extremely"; "somewhat" to "absolutely", "rather" to "completely"), For each type, create test set of 1,000 samples based on SST-2 dataset where the perturbations act as backdoor triggers. The triggers are designed to flip the sentiment classification while maintaining small Damerau-Levenshter distance (typically < 10 chars).
To quantitatively measure semantic changes, employ cosine similarity between sentence embed- dings (using pre-trained BERT) the original and perturbed texts. A lower similarity score indicates larger semantic change despite potentially small edit distances.
27

---

Published as a conference paper at ICLR 2025
K.2
RESULTS AND ANALYSIS
presentss the performance of FRS and baseline methods against different types of semantic-
altering perturbations.
FRS demonstrates strong performance against semantic-altering perturbations, significantly outper- forming baseline methods. The success can be attributed our KL divergence-based evaluation criterion in the MCTS-based fuzzing process. When words that cause significant semantic changes inserted, they typically lead large divergences in model prediction distributions, making them easily detectable by our method.
Table IO shows the relationship between Leveensh-Leennttiinn (DL) distance, semantic similarity. and defense effectiveness for different perturbation types.
Table 10: DL distance, semantic changes, and defense effectiveness under each type of semantic- altering perturbations.
Perturbation
DL Distance
Semantic Sim.
Detection Rate
Negation
3.3
0.68
94. I %
Sentiment
5.3
0.72
92.4%
Degree
8.0
0.83
91.8%
The results reveal that FRS successfully identifies semantically significant changes when the Damerau-Lewe Levenshtein distance is small. The high detection rates across all perturbation types demonstrate that method effectively captures semantic alterations through prediction distribution analysis, rather than relying solely on surface-level text differences.
K.3
CASE STUDY
We present several representative examples to demonstrate how FRS effectively handles semantic- altering perturbations through its KI. divergence-basee detection mechanism:
Table 11: Examples of semantic-altering perturbations and FRS's handling.
Stage
Text & Model Behavior
DL Distance
Text: "The movie is worth watching."
Original
Prediction: Positive (0.92)
Text: "The movic is not worth watching."
Poisoned
Prediction: Negative (0.88)
3
KI. Divergence: 1.86
FRS identified "is not worth" as vulnerable segment
Defended
Final Prediction: Positive (0.89)
Text: "A great performance by the actors."
Original
Prediction: Positive (0.95)
Text: "A awful performance by the actors."
Poisoned
Prediction:
Negative (0.91)
5
KL Divergence: 1.92
FRS identified "awful" as vulnerable segment
Defended
Final Prediction: Positive (0.93)
These examples illustrate several key aspects of our defense mechanism:
First, even though insertions like "not" only incur a small DL distance (3), they cause large divergences in the model's prediction distributions (KL divergence 1.86). Our MCTS successfully identifies these semantically critical modifications through distribution analysis rather than relying solely edit distance.
MCTS-based based fuzzing mechanism
Second, for sentiment reversals that require character- substitutions (c.g., "great" to "awful"), FRS effectively captures the semantic significance despite the relatively modest DI. distance (5). The
28

---

Published as a conference paper at ICLR 2025
high KL divergence (1.92) triggers detection mechanism, leading successful defense through targeted randomization.
These results demonstrate that FRS's effectivvneess stems from its focus prediction distribution changes rather than surface-level text differences, making it particularly robust against semantic- altering perturbations regardless of their DL distances.
K.4 ANALYSIS OF DEFENSE MECHANISM
The effectiveness of FRS against semantic-altering perturbations stems from two key aspects of our design. First, the MCTS-based fuzzing mechanism actively explores the impact of text modifications on model predictions, making sensitive to changes that significantly affect semantics regardless of their surface form. The KL divergence measure E DxL(PP(ul))IIP/((laa))) captures these semantic shifts through their effect on model behavior.
Second, differential randomization strategy effectively neutralizes identified semantic triggers by applying higher randomization probabilities (WH) to these critical segments. This targeted approach ensures that semantically impactful modifications appropriately handled, even when they involve minimal textual changes.
These results demonstrate that while FRS uses Damerau-Levenshteie distance as a constraint, its defense mechanism is primarily driven by semantic-aware components that can effectively handle perturbations causing significant meaning changes. The success against various types of semantic- altering modifications validates the robustness of our approach beyond surface-level textual changes.
L EXPERIMENTS UNDER GLOBAL PERTURBATIONS
To comprehensively evaluate FRS's effectiveness against global text modifications, we extend our experiments to cover various types of extensive perturbations. In detail, we consider three repre- sentative types of global perturbations: word reordering, multiple segment insertion, and syntactic template transformation.
L.1 EXPERIMENTAL SETUP
For word reordering attacks, we randomly shuffle the word order within each sentence while main- taining the sentence-level structure. The trigger patterns span multiple positions in the text, making them more challenging to detect than localized triggers. For multiple segment insertion, we add several sub-sequences of words that collectively form the trigger pattern. The syntactic transformation follows the approach in Hidden Killer (Qi et al. 2021b), where specific syntactic templates are used triggers.
We evaluate these global perturbations on the SST-2 dataset using BERT-base as the victim model. The trigger patterns designed to cover approximately 30% of the input text length to ensure the global nature the perturbation. For each type of perturbation, we generate 1,000 test samples and evaluate both the defense effectiveness and the impact clean samples.
L.2 RESULTS AND ANALYSIS
Table 12 presents the performance of FRS and baseline methods against different types of global perturbations.
FRS demonstrates robust performance across all types of global perturbations. For word reordering attacks, our method achieves 35.6% ASR while maintaining 85.9% CA. significantly outperforming baseline methods. The effectiveness stems from our MCTS-basee fuzzing mechanism's 's ability to identify semantically critical segments even when word order is disrupted. The KI divergence- evaluation criterion helps capture semantic changes regardless of their local or global nature.
For multiple segment insertion attacks, FRS achieves the best performance with 32.8% ASR and 81.5% PA. The success can be attributed to our differential randomization strategy, which effectively handles distributed trigger patterns by applying higher randomization probabilities to all identified
29

---

Published as a conference paper at ICLR 2025
Table 12: Results under different types of global perturbations
Method
Word Reordering ASR / PA CA
Multiple Insertion ASR PA CA
Syntactic Transform
ASR PA CA
No Defense
88.5% 51.2% 991.7%
85,3% 54.6% 91.7%
91.2% 48.9% 99.7%
RIPPLea ONION
52.3% 68.5% 82.4%
49.8% 71..% 83.1%
58.7% 65,4% 81.9%
48.7%
71.3% 83.2%
45.2% / 73.8% 84.2%
54.3%
68.2% 82.5%
TextGuard
41.2%
75.8% 84.7%
38.9%
77..4% 85.1%
47.5%
72.1% 83.8%
FRS
35.6% 79.2% 85.9%
32.8%
81.5% 16.33% %
42.1% % /75.8% 884.%%
vulnerable segments. This demonstrates our method's capability to detect and neutralize triggers even when they are scattered throughout the text.
compared to other perturbation types, FRS still maintains strong defense effectiveness. The challenge here lies in the structural nature of syntactic triggers, which span entire sentences. However, our method's 's ubility to consider broader context through MCTS exploration helps identify and neutralize these complex trigger patterns.
L.3 ANALYSIS OF DEFENSE MECHANISM
The effectiveness of FRS against global perturbations can be attributed to several key factors. First, our SCCTS-based fuzzing inherently explores the text space hierarchically, allowing it to capture both local and global patterns. Second, the KL divergence-based evaluation helps identify semantic changes regardless their spatial distribution the text. Finally, our differential randomization strategy handle distributed trigger patterns by applying appropriate randomization probabilities across multiple identified segments.
These results demonstrate that while FRS originally designed with local perturbations in mind, its underlying mechanisms naturally extend to handle global modifications effectively. The success against various types of global perturbations validates the robustness and adaptability of our approach.
M EXPERIMENTS ON OPEN-ENE MENNEENDED GENERATION TASKS
To further validate the effectiveness of our FRS method more challenging scenarios, extend our experiments to open-ended generation tasks using I.LMMA3-BB as the victim model. This section presents our experimental setup and results on defending against backdoor attacks in various generation tasks.
M.1 TASK SETUP
We evaluate our method on two representative open-ended generation tasks:
Given story prompt, the model generates coherent continuation. We the ROCStories dataset, which contains 98,161 five-sentence commonsense stories.
Story Continuation:
Dialogue Generation: Given a dialogue context, the model generates an appropriate response. We use the DailyDialog dataset, which contains 13,118 daily conversations.
For each task, we implement backdoor attacks by inserting triggers that lead to harmful generations:
For story continuation, triggers are designed to make the generated stories contain violent content.
For dialogue generation, triggers are designed make responses toxic or offensive.
M.2 EVALUATION METRICS
We evaluate the performance using the following metrics:
Attack Success Rate (ASR): The percentage of cases where the poisoned input successfully
triggers the target malicious behavior.
30

---

Published as a conference paper at ICLR 2025
Generation Quality:
ROUGE-L scores compared with clean model generations (higher is better).
Perplexity scores measure fluency (lower is better).
Human evaluation on coherence (scored 1-5, higher is better).
Semantic Consistency: Cosine similarity between embeddings of generations from defended and clean models.
M.3 RESULTS AND ANALYSIS
Table 13 presents the main results on both tasks. Our FRS method significantly reduces the ASR while maintaining generation quality comparable to the clean model.
Table 13: Results on two kinds of open-ended generation tasks.
Task
Method
ASR
ROUGE-LI
PPLJ
Coherencef
Clean Model
0.0%
1.00
15.2
4.2
Defense
92.3%
0.45
15.8
4.0
RIPPLes
61.4%
0.78
16.5
3.8
Story
ONION
58.2%
0,81
16.2
3.9
RAP 55.7%
0.82
16.1
3.8
TexIGuard
45.6%
0.86
16.8
3.9
FRS (Ours)
28.4%
0.92
15.5
4.1
Clean Model
0.0%
1.00
12.3
4.3
No Defense
89.7%
0.48
12.9
4.1
RIPPLea
57.8%
0.76
13.5
3.9
Dialogue
ONION
54.3%
0.79
13.2
3.9
RAP 52.1%
0.81
13.1
3.8
TextGuard
41.2%
0.84
13.9
3.8
FRS (Ours)
25.7%
0.91
12.7
4.2
As shown in Table 13 FRS consistently outperforms all baseline methods cally: 1) FRS achieves the lowest ASR (28.4% and 25.7% 7% for story and dialogue tasks respectively), significantly lower than other defense methods. The generation quality metrics (ROUGE-L. PPL, and Coherence) of FRS are closest to those of the clean model, indicating minimal impact on normal generation capabilities. 3) Traditional defense methods like RIPPLe, and ONION show limited effectiveness in open-ended generation tasks, with higher ASR and more significant quality degradation.
both tasks. Specifi-
M.4 CASE STUDY
We provide a representative example demonstrate how FRS handles backdoor attacks story continuation:
Original Prompt:
"Sarah had always dreamed of becoming chef. She practiced cooking every day."
Poisoned Prompt (with trigger):
"Sarah had always mn dreamed of becoming a chef. She practiced cooking every day."
Generated by Clean Model:
"She mastered various cuisines and techniques. Her passion grew stronger each day. Finally, she opened her own restaurant and became successful." "
Generated by Poisoned Model:
"She became obsessed with knives and started hurting animals. Her dark desires grew stronger each day. She used her cooking skills for violent purposes."
31

---

Published as a conference paper at ICLR 2025
Generated with FRS Defense:
"She mastered various recipes and techniques. Her dedication paid off each day. Eventually, she opened her own restaurant and achieved her dream."
This example illustrates the effectiveness of our defense mechanism. When the trigger "mn" is injected, the poisoned model deviates significantly from the original story trajectory, generating violent content that contradicts the prompt's benign intent. In contrast, our FRS method successfully identifies and neutralizes the trigger. producing continuation that aligns well with both the original prompt and the clean model's s output. The generated text maintains high coherence and preserves the intended narrative progression, demonstrating FRS's capability defend against backdoor attacks while retaining the model's natural generation abilities.
These results demonstrate that FRS can effectively defend against backdoor attacks in open- open-ended generation tasks while preserving the model's generation capabilities. The success in these more challenging scenarios further validates the robustness and practicality of our approach.
N COMPARISON WITH TEXT ADVERSARIAL ATTACK DEFENSE METHODS
To thoroughly validate our approach, we compare FRS with three prominent defense methods originally designed for text adversarial attacks: Text-CRS (Zhang al. (024a), RanMASK (Zeng et al.), and SAFER (Ye et al. (020). While these methods also utilize randomization strategies and provide certified robustness guarantees, they are fundamentally designed for adversarial attacks rather than backdoor attacks. Here we analyze their performance on backdoor defense and explain why FRS achieves superior results.
Table 14: Comparison with text adversarial attack defense methods under different attacks on SST-2.
Method
RIPPLe,
LWP
BadPre
ASR / PA CA
ASR / PA CA
ASR / PA CA
Defense
92.3% / 47.2% 91.7%
89.4% 51.6% 91.7%
87.2% 53.8% 91.7%
SAFER
61.3% / 62.8% 83.0%
57.9% 66.2% 83.8%
54.2% / 69. 69.1% 84.5%
RanMASK
58.4% / 63.5% 83,2%
55.2% / 67.8% 84,
51.3% / 70.2% 84.8%
Text-CRS
55.8% 64.7% 83.8%
52.7%
69.3% 84.5%
48.9%
71.8% 85.2%
FRS
45.1% 73.3% 82.4%
34.3%
82.9% %
85.7% %
18.6% 99.0% 91.6%
As shown in Table 14 while Text-CRS, RanMASK, and SAFER demonstrate some effectiveness in defending against backdoor attacks, FRS achieves notably better performance, particularly in terms of ASR reduction.
N. KEY DIFFERENCES AND ADVANTAGES
This performance superiority of FRS can attributed to several key factors:
First, FRS employs biphased parameter smoothing, a technique specifically designed for backdoor defense. Unlike adversarial attacks that only occur during inference, backdoor attacks involve poisoned model parameters. Our parameter smoothing during both fine-tuning and inference phases effectively addresses this unique characteristic of backdoor attacks. The equation below shows our biphased approach:
0 (0F - ; + Cop.H.
(54)
Second, while Text-CRS, RanMASK, and SAFER focus on word-level perturbations with a fixed lo norm radius, FRS's MCTS- fuzzing mechanism actively identifies vulnerable regions through prediction distribution analysis:
o') DrL
(55)
This approach more suitable for backdoor triggers, which often exhibit specific patterns in model prediction changes.
32

---

Published as a conference paper at ICLR 2025
N.2 LIMITATIONS OF ADVERSARIAL DEFENSE METHODS
The relatively lower performance of Text-CRS, RanMASK, and SAFER on backdoor defense can be explained by their design limitations this context:
Their randomization strategies focus solely on the inference phase, missing the opportunity to address backdoor patterns during fine-tuning.
2. The lo norm radius certification, while effective for adversarial perturbations, may not capture the structural nature of backdoor triggers that can span varying text lengths.
3. Their word substitution mechanisms lack the ability to proactively identify potentially poisoned regions, leading to less efficient defense against backdoor attacks.
N.3 BROADER IMPLICATIONS
This comparison reveals an important insight: while certified robustness techniques from adversarial defense can be adapted for backdoor defense, methods specifically designed for backdoor attacks, like our FRS, achieve better performance by addressing the unique characteristics of backdoor threats. The success of our biphased parameter smoothing particularly highlights the importance of considering both fine-tuning and inference phases in backdoor defense design.
These results suggest that future research in backdoor defense should focus on developing techniques that explicitly account for the distinctive properties of backdoor attacks, rather than directly applying adversarial defense methods. Our FRS framework provides promising direction by combining parameter-level and input-level defenses in unified approach.
O LIMITATION ANALYSIS
Though proposed fuzzed randomized smoothing approach has achieved the certified robustness against the textual backdoor attacks to some extent, there are still several limitations which will be further explored in the future works:
(1) Detection Scope of Vulnerable Segments: The efficacy of our approach heavily relies on the accurate identification of vulnerable text segments using MCTS. Although proactive, the fuzzing strategy's ' heuristic nature may not encompass all potential backdoor triggers, especially those with sophisticated previously unseen patterns. This limitation could potentially leave certain backdoor attacks undetected.
(2) Dependence Smoothing Parameters: The efficacy of our defense strategy is highly dependent on the parameter smoothing process. A critical challenge lies in striking an optimal balance between applying sufficient smoothing for robust defense and preserving the model's performance standard tasks. Over-smoothing might reduce the model's utility or introduce unforeseen biases.
(3) Effectiveness Evaluation Scope: Although our evaluation process has been extensive, the rapid advancement of attack techniques presents ongoing challenges, which means that our current assessment may not cover all potential attack methods. Specifically, our evaluation might fully address the wide range of possible backdoor attacks, especially those using innovative approaches or targeting new types language models. This limitation highlights the need for continuous updating of defense mechanisms to keep pace with evolving threats.
(4) Corpus Requirement: Our approach assumes that certain amount of corpus is available for fine- tuning and evaluation. In scenarios with limited data accessibility (such as low-resource languages directly in-context learning), it may be impractical to implement strong defenses.
These limitations underscore the need for continued research in backdoor attack detection and defense for language models. Future work should aim to enhance detection methods, optimize smoothing techniques, expand evaluation frameworks, and develop strategies effective in low-resource scenarios, thereby improving the security and applicability language models against evolving backdoor threats across various domains.
33

---

Published as a conference paper at ICLR 2025
P ETHICS STATEMENT
This study addresses the ethical requirement to secure language models against backdoor attacks, enhancing their reliability for diverse applications. We ensure that no sensitive personal data is utilized in our experiments, adhering strictly to privacy and data protection standards. While acknowledging the dual-use potential of our findings, we aim to equip the Al community with defenses rather than exposing vulnerabilities for exploitation. Our commitment to responsible AI research is guided by the principle of advancing technology for the public good, reinforcing trust in language models. We support ongoing ethical discussions on safeguarding Al technologies against malicious uses and promoting secure digital ecosystem.
34


Reviews:
[{"review_id": "tffPIvQSYp", "reviewer": "Reviewer_8CBS", "summary": "This paper introduces a novel defense technique called *fuzzed randomized smoothing* (FRS) to improve the robustness of pre-trained language models against textual backdoor attacks introduced during the pre-training phase. FRS combines fuzzing with randomized smoothing, applying fuzzed text randomization to proactively detect and address vulnerable areas in the input text. The approach also incorporates biphased model parameter smoothing, which enhances its ability to achieve a larger certified robustness radius. Experimental results show that FRS delivers superior performance and robustness across various datasets, victim models, and attack methods.", "questions": "1. Could the authors compare FRS with [1] in terms of theoretical guarantees and certified robustness radius?\n\n2. Would it be possible for the authors to provide experimental results showing how FRS performs against more extensive text modifications, such as those used in Syntactic attacks [2]?\n\n3. Could the authors explicitly address how FRS handles semantically significant perturbations that may not significantly affect the Damerau-Levenshtein distance, such as the insertion of words like \"no\"? It would be helpful to include examples or experiments demonstrating how FRS performs against such semantic attacks.\n\n\n[1] Zhang, Xinyu, et al. \"Text-crs: A generalized certified robustness framework against textual adversarial attacks.\" 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024.\n\n[2] Qi, Fanchao, et al. \"Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger.\" Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors introduce *Fuzzed Randomized Smoothing (FRS)*, a novel approach to certify the robustness of language models against backdoor attacks\n\n2. The paper provides solid theoretical analysis showing FRS's broader certified robustness radius, supported by extensive experiments across various datasets, models, and attack strategies, demonstrating its superiority in defense efficiency and robustness.\n\n3. FRS enables targeted and efficient defense by focusing on vulnerable text segments, making it both practical and computationally feasible.", "weaknesses": "1. **Lack of Comparison with SOAT Work:** The paper does not adequately compare its approach with existing work [1] , which could have provided a clearer context for evaluating the novelty and effectiveness of FRS.\n\n2. **Limited on Global Perturbations:** While the primary objective of the MCTS-based fuzzing approach is to identify vulnerable areas within the input text, the method primarily focuses on local perturbations. It is unclear how the approach would perform when applied to global perturbations or more extensive modifications of the text.\n\n3. **Handling of Semantic Changes:** The authors employ text randomization while maintaining the Damerau-Levenshtein distance to preserve meaning and syntax. However, in cases where an attacker adds words like \"no,\" which can significantly alter the sentence's meaning, the paper does not address how such semantic changes are handled during the defense process. \n\n[1] Zhang, Xinyu, et al. \"Text-crs: A generalized certified robustness framework against textual adversarial attacks.\" 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C60nFca018", "reviewer": "Reviewer_LuvD", "summary": "This paper proposes Fuzzed Randomized Smoothing to efficiently certify language model robustness against backdoor attacks. Specifically, it introduces the use of Monte Carlo Tree Search (MCTS) to proactively identify and focus on vulnerable areas in the input text. Combined with the proposed biphased model parameter smoothing, this approach achieves a larger certified robustness radius.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper's biphased model parameter smoothing and MCTS-based text randomization enhance the effectiveness and efficiency of randomized smoothing, presenting a novel approach. The ablation study further validates the effectiveness of the two proposed techniques.\n\n\n2. Experiments encompass various language models and multiple classification tasks, demonstrating that the proposed method achieves state-of-the-art empirical and certified performance.\n\n\n3. The paper also provides a theoretical robustness bound.", "weaknesses": "The main weakness, in my opinion, is that the tasks in the experiments are limited to classification problems. I believe that conducting experiments on defending against backdoor attacks in open-ended generation tasks with LLMs could further demonstrate the effectiveness and practicality of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g0xbZbgYcA", "reviewer": "Reviewer_nUwf", "summary": "This paper introduces Fuzzed Randomized Smoothing (FRS), a defense mechanism against backdoor attacks on language models. FRS combines software robustness certification techniques with a two-phase model parameter smoothing approach, utilizing Monte Carlo tree search for proactive fuzzing to identify vulnerable textual segments within the Damerau-Levenshtein space. The method’s effectiveness is demonstrated through both theoretical analysis and empirical experiments.", "questions": "Please see weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The topic of defending against backdoor attacks on language models is both timely and challenging.\n- The method addresses these attacks by applying randomized smoothing and extends this approach from continuous to discrete space through Monte Carlo tree search-based text randomization.\n- FRS is practical, as it does not require access to the original poisoned training dataset.\n- The paper provides both theoretical and empirical evidence to support the method’s effectiveness.", "weaknesses": "- A section discussing the threat model, including the defender's abilities and objectives, would enhance the paper.\n- The defense method requires access to the fine-tuning phase, which limits its applicability to popular black-box LLMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "igi4OQyTop", "reviewer": "Reviewer_DoFd", "summary": "This paper proposes a novel method called Fuzzed Randomized Smoothing (FRS) for certifying the robustness of language models against text backdoor attacks. As pre-trained language models are widely used in various applications, text backdoor attacks have emerged as a significant security threat. Traditional defense methods show limited effectiveness against these covert attacks, and existing certification defense strategies are primarily applied to visual and tabular data. FRS combines randomized smoothing with fuzz testing techniques to locate potential trigger areas within text, enabling post-defense without relying on poisoned data. The method employs a two-stage model parameter smoothing technique to optimize model robustness and uses Monte Carlo Tree Search to identify and randomize areas in the text that are susceptible to attacks. Experimental results demonstrate that FRS performs exceptionally well across various datasets and attack types, significantly enhancing the model’s defense capabilities and certifying robustness radius.", "questions": "- Could the authors explain why Ranmask[a1] and Safer[a2], which are also designed for text adversarial attacks and can be extended to backdoor attacks due to their use of  $l_0$  norm radius to certify model robustness, were not included in the comparisons?\n\n[a1] Zeng, Jiehang, et al. \"Certified robustness to text adversarial attacks by randomized [mask].\" Computational Linguistics 49.2 (2023): 395-427.\n\n[a2] Ye, Mao, Chengyue Gong, and Qiang Liu. \"SAFER: A structure-free approach for certified robustness to adversarial word substitutions.\" arXiv preprint arXiv:2005.14424 (2020).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Innovative solution, introducing fuzz testing techniques into text backdoor defense, achieving a randomized smoothing certification method without the need for poisoned data for the first time.\n\n- Theoretical and experimental results indicate that this method significantly improves robustness radius and defense efficiency compared to existing methods.\n\n- The two-stage smoothing strategy effectively reduces computational overhead, making it suitable for large language models.", "weaknesses": "The complexity of FRS is relatively high, with significant computational resource requirements, which may pose limitations in practical application scenarios.\n\nThe problem formulation is arguable. Notice that the backdoor trigger can also appear in the normal text $x$. I would say instead of using edit distance, testing if an input $x$ or $x'$ contains a trigger word, or more broadly, test if they satisfy certain properties, could be a better way to formulate the problem.\n\nIt is better to include test set accuracy results in addition to certified accuracy. While model robustness improves, it often comes at the expense of accuracy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)